Content mappers: pass an import's query or attributes to transform #64912

🔍 Search Terms

content mapper, contentMappers, query, ?raw, import attributes, module specifier query

✅ Viability Checklist

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This isn't a request to add a new utility type: https://github.com/microsoft/TypeScript/wiki/No-New-Utility-Types
• This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion

Let a content mapper opt in, in its manifest, to seeing which variant an import asks for, through its query, its import attributes, or both. Each distinct variant would be its own module, and the transform request would carry it:

import config from "./app.yaml";                                // as today
import widened from "./app.yaml?widened";                       // query: "?widened"
import alsoWidened from "./app.yaml" with { types: "widened" }; // attributes: { types: "widened" }

// package.json: "typescript": { "contentMapper": { "exec": [...], "variants": true } }
interface TransformParams {
    fileName: string;
    content: string;
    projectHandle: string;
    /** The import's query as written, starting with "?". */
    query?: string;
    /** The import's attributes. */
    attributes?: Record<string, string>;
}

Either key would serve. Attributes are standard syntax and leave resolution as it is: TypeScript already accepts with { types: "widened" } on a mapped file and resolves the path as usual. The query needs the resolver to look past the ?, and in return it also covers variants whose value differs. Node keys modules by URL, so it makes each query its own module, while two imports of one file with different attributes share one module there. That's harmless when only the type differs, as with ?widened, and not when the value differs, as with Vite's ?raw. esbuild separates both.

An import with no query or attributes, and any file no opted-in mapper claims, would resolve as today. Open questions: where to report a variant the mapper doesn't support, and what declaration emit writes for one.

📃 Motivating Example

Vite's ?raw is the prior art: import shader from "./shader.glsl?raw" gives the file's text. vite/client types it with declare module "*?raw" { const src: string; export default src }, which works because a string doesn't depend on the file.

A type that does depend on the file can't be written that way, and the content mapper, which can see the file, never sees the query or the attributes. Motherload types data files as as const would:

import config from "./app.yaml";          // { readonly port: 8080; readonly tags: readonly ["a", "b"] }
import widened from "./app.yaml?widened"; // TS2307; wanted: { port: number; tags: string[] }

// main.ts; tsconfig: "contentMappers": [{ "package": "identity-foo-mapper", "extensions": [".foo"] }]
import config from "./config.foo";
import widened from "./config.foo?widened";
import attributed from "./config.foo" with { types: "widened" };

main.ts(2,21): error TS2307: Cannot find module './config.foo?widened' or its corresponding type declarations.

{"method":"transform","params":{"fileName":"/repro/config.foo","projectHandle":"identity-foo-mapper@0.0.0:0","content":"…"}}

import { appendFileSync } from "node:fs";

const log = (message) => process.env.MAPPER_LOG && appendFileSync(process.env.MAPPER_LOG, JSON.stringify(message) + "\n");

const handlers = {
  initialize: () => ({ positionEncoding: "utf-16", diagnosticSource: "foo" }),
  openProject: () => ({}),
  transform: ({ content }) => ({ text: content, extension: ".ts", mappings: [[0, content.length, 0, content.length, 0]] }),
  closeProject: () => ({}),
};

let buffer = Buffer.alloc(0);
process.stdin.on("data", (chunk) => {
  buffer = Buffer.concat([buffer, chunk]);
  for (;;) {
    const end = buffer.indexOf("\r\n\r\n");
    if (end === -1) return;
    const length = Number(/Content-Length:\s*(\d+)/i.exec(buffer.subarray(0, end).toString())[1]);
    if (buffer.length < end + 4 + length) return;
    const request = JSON.parse(buffer.subarray(end + 4, end + 4 + length).toString());
    buffer = buffer.subarray(end + 4 + length);
    const { content, ...params } = request.params ?? {};
    log({ method: request.method, params: request.method === "transform" ? { ...params, content: "…" } : params });
    if (request.id === undefined) continue;
    const payload = Buffer.from(JSON.stringify({ jsonrpc: "2.0", id: request.id, result: handlers[request.method]?.(request.params) ?? {} }));
    process.stdout.write(`Content-Length: ${payload.length}\r\n\r\n`);
    process.stdout.write(payload);
  }
});

💻 Use Cases

1. Typing one file differently per import when the type depends on its content, as ?widened does above.
2. Ambient wildcards can't see the file, and the mapper is never told the query or the attributes. A contentMappers entry's options choose per extension, not per import.
3. No workaround: Motherload types every data file as const, and users widen by hand.