[scalvert]

Problem

Markdown decodes backslash escapes and character references in a fence's info string. We read directives from remark's decoded meta, but setDirective searched and wrote the raw fence line. Where the two differ, reading and writing disagree:

Fence line	Read as	Before: setDirective
snippet=a.ts title=x	a.ts	replaced a.ts title=x, dropping title=x
snippet="my docs/a.ts"	my docs/a.ts	removed snippet="my, leaving docs/a.ts"
write a&b.ts		wrote it raw, which reads back as a&b.ts
write a`b.ts		wrote a raw backtick, so the line is no longer a fence

The last two show up through the CLI. extract in backtick.mdwrites a directive that breaks the fence, andejectthen can't remove it.extractinodd&name.mdwrites a directive whose snippetcheck` reports as missing.

The directive grammar also had its own rule that \" doesn't end a quoted value. That rule applied to text CommonMark had already decoded, so a quoted path ending in a backslash (snippet="C:\my docs\") couldn't be written and read back.

Fix

Escaping is now handled in one place: CommonMark's info-string rules. Reading and writing a directive both go through the same decoded text.

• src/info-string.ts (new, about 70 lines):
  • decodeInfoString(source) returns the decoded value and sourceOffset(index), which maps a decoded offset back to the source. It splits the source into literal text and escape/reference parts with one matchAll pattern, and decodes each part with micromark-util-decode-string. That's the decoder remark uses, and it's already in the dependency tree; it is now a direct dependency.
  • encodeInfoString(value) escapes only what would otherwise decode, plus a trailing backslash so it can't combine with the text after the directive. Backticks are written as `.
• src/fence-info.ts:
  • readDirective(info) decodes the metadata and parses the directive. MarkdownDocument now uses it instead of node.meta. As in CommonMark, the language is split from the metadata only at a space or tab.
  • withDirective(info, directive) finds the token in the decoded text and maps it back to the source. Replacing writes the encoded directive over that range. Removing also removes the separator after the directive, or the one before it if the directive is last, so the surrounding words can't join even when the separator is written as  .
• src/snippet-directive.ts: the character-by-character tokenizer (about 50 lines of index arithmetic) is replaced by a token regex plus quoted/unquoted value patterns. A quoted value now ends at the next matching quote. A differential test against the old tokenizer found the same result on all 37,449 strings of up to 5 fragments, except inputs with a backslash before a quote.

Directives without escapes or references read and write exactly as before. No existing test or snapshot changed.

parseFencedBlockAt (from #110) now accepts U+2028 and U+2029 in the info string. Before this fix, a fence containing one made MarkdownDocument.parse throw.

Tests

• info-string.test.ts: exhaustive over all 11,111 strings of up to 4 characters from \ & # 9 ; _ x ␠ "`, plus named, numeric, astral and invalid references. It checks that:
  • decoding matches micromark's decodeString;
  • decoding matches remark's meta;
  • every decoded offset maps to source text that decodes to exactly the preceding value;
  • decode(encode(v)) === v and the encoded form has no backticks.
• fence-info.test.ts: readDirective with escapes, references and " quotes; source-range replacement and removal; encoding on write.
• markdown-document.test.ts:
  • 10 file paths (docs\_draft.ts, a&b.ts, a`b.ts, C:\my docs\, and others) each round-trip through setDirective, read back unchanged, and removing the directive restores the source;
  • the table cases above;
  • MDX.
• snippet-directive.test.ts: a quoted value ending in a backslash; quoted values that don't end their token.
• End to end: extract → check → eject on backtick.mdandodd&name.md. Both resolve, and eject restores both byte for byte. On main, the backtick file kept its directive and &` was reported missing.

README: the Snippet Syntax section now covers quoting and escapes.

• I have run npm run lint and npm run format:check.
• I have run npm run build to verify the project builds.
• I have run npm test to ensure all tests pass. (550 passed, plus npm run test:package)

[scalvert]

Review

There were two reviews, plus two adversarial fuzz rounds. The fuzzing compared our directive reads with remark's meta (backtick and tilde fences, MDX on and off) and checked:

• set-then-reparse;
• removal;
• byte-identical add-then-remove;
• old-vs-new tokenizer behaviour.

Standards: approved. I applied its changes:

• an offset test that compares decoded prefixes, so it can actually fail;
• one matchAll segment pattern with an explicit decoded flag;
• sourceOffset(index) instead of sourceRange;
• separate removeDirective and replaceDirective;
• clearer sample names.

The reviewer judged copying micromark's private pattern acceptable, since the exhaustive "decodes like micromark" test catches drift. micromark-util-decode-string is the right direct dependency. No importable library would replace the encoder.

Adversarial round 1 (about 130k cases) found three bugs:

• NBSP after the language was read as starting the metadata. That was a regression, because CommonMark splits only at a space or tab.
• Removing a directive whose neighbouring separator is a character reference could join the words around it.
• A path ending in \ could escape the reference that follows it.

It also found that parseFencedBlockAt (#110) threw on U+2028 in an info string.

Fixed in f8f880e:

• The language is split at space and tab only.
• Removal takes the separator after the directive, or the one before it if the directive is last.
• A trailing backslash is escaped.
• The opening-line pattern accepts U+2028 and U+2029.

Round 2 covered 154,100 sources, 924,600 sets and 426,096 add-then-remove pairs, with 0 failures in reads, sets and restores. Removal works for single directives. A fence with two snippet= tokens still shows the second after the first is removed. main behaves the same way, and only the first token is ever read, so I'm treating it as intended.

Pre-existing, not changed here:

• an unclosed quote earlier in the metadata hides an appended directive;
• a path ending in #N reads back as a line range;
• a path containing both quote characters throws, and extract leaves an orphan snippet file when that happens;
• a file name containing \ isn't found by the glob on POSIX.

CI is green on Ubuntu, macOS and Windows × Node 22 and 24. 550 tests pass locally.