Repository navigation
Deepen Markdown document module; deprecate offset-based parser helpers - #98
Merged
Merged
Conversation
Introduce MarkdownDocument: parse a Markdown/MDX string once, expose its code blocks (language, content, directive, line, column, lineEnding), queue setBody/setDirective edits, and serialise with toString(). Fence handling (marker, indentation, meta, line endings) now lives in one place instead of three offset splicers in sync, extract, and eject — each of which recent fixes (#82, #86) had to patch separately. Move the snippet directive grammar into snippet-directive.ts: parseSnippetDirective (moved, quote-aware scanner from #85), formatSnippetDirective (canonical output, quotes paths with whitespace), and findSnippetDirectiveToken, which the document uses so fence edits touch exactly the token the parser read. Line-ending helpers move to line-endings.ts and remain re-exported from parser.ts. Behaviour fixes that fall out of the shared grammar: - eject removes quoted directives (snippet="a b.ts") whole instead of leaving a dangling fragment - extract quotes generated references whose directory contains spaces, so they parse back correctly - setBody no longer treats the last content line of an unclosed fence as its closing fence - check's out-of-sync message uses the directive form (snippet=a.ts#L10) instead of snippet://a.ts#L10-L10 parseMarkdownFile, parseMarkdownForExtraction, replaceCodeBlock, and the CodeBlock/MarkdownFile types are deprecated wrappers over the new module and will be removed in 2.0. Adds CONTEXT.md with the project's domain glossary.
- Only keep a closing fence in setBody when it matches the opening fence's character and is at least as long, so an unclosed ~~~ block ending in ``` no longer loses content. - setDirective is a no-op when the directive is unchanged (including after an intermediate edit), so fence lines are rewritten only when their directive actually changes. - formatSnippetDirective falls back to single quotes when the reference contains a double quote, and rejects references containing both, so formatted directives always parse back. - Move readMarkdownDocument into parser.ts; the deprecated wrappers and eject reuse it instead of a duplicate helper and a sync.js import. - Rename the resolved directive local in sync.ts to match CONTEXT.md. - Pin check's out-of-sync message in an integration test. - Update CONTRIBUTING.md project structure for the new modules.
Owner
Author
Second-pass reviewTwo independent passes: Standards (AGENTS.md / CONTRIBUTING.md + smell baseline) and Spec (the agreed design for this refactor). Fixed in 398df33
Accepted as-is
Deferred (pre-existing or edge-case)
CI green on Node 22 and 24; 254 tests pass locally. |
3 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a
MarkdownDocumentmodule that owns every read and edit of a Markdown/MDX file's code blocks.sync,check,extract, discovery, andejectnow go through it instead of each re-deriving fence lines from raw offsets.Recent fixes show why: #82 (eject metadata) and #86 (line endings) each had to patch three separate splicers —
replaceCodeBlock, extract's annotation insert, and eject's directive removal. Those now live in one place.src/markdown-document.ts—parse→codeBlocks(language,content,directive?,line,column,lineEnding) →setBody/setDirective→toString(). Edits are queued and applied bottom-up; offsets never leave the module. Strings only — callers do file I/O.src/snippet-directive.ts—parseSnippetDirective(moved; fix: validate snippet directives and ranges #85's quote-aware scanner),formatSnippetDirective(canonical#L10,#L10-,#L10-L20; quotes paths containing whitespace), andfindSnippetDirectiveToken, which the document uses so fence edits touch exactly the token the parser read.src/line-endings.ts—getLineEnding/normalizeLineEndingsmoved out ofparser.ts(still re-exported there) to avoid an import cycle.CONTEXT.md— domain glossary: Markdown document, Code block, Snippet directive, Snippet.Behaviour changes
ejectremoves quoted directives (snippet="a b.ts") whole; previously the regex leftb.ts"behind.extractquotes generated references when the markdown filename contains spaces, so they parse back.setBodyonly keeps a closing fence that matches the opening fence (same character, at least as long); an unclosed block no longer treats its last content line as the closing fence.check's out-of-sync message now readssnippet=a.ts#L10(open-ended:#L10-) instead ofsnippet://a.ts#L10-L10. Pinned by a new integration test.Deprecations (no breaking change)
parseMarkdownFile,parseMarkdownForExtraction,replaceCodeBlock, and theCodeBlock/MarkdownFiletypes are@deprecatedthin wrappers over the new module, to be removed in 2.0. New public exports:MarkdownDocument,isMdxPath,formatSnippetDirective,DocumentCodeBlock,ParseOptions.Tests
test/markdown-document.test.ts: string-in/string-out cases — md and MDX, list/JSX indentation,~~~and```` fences, meta preservation, LF/CRLF bodies, quoted directives, empty and unclosed blocks, multi-edit, add→remove round-trip.parser.test.ts: parse/replace suites collapsed to one smoke test per deprecated wrapper; fix: preserve markdown line endings #86's CRLF case moved to the document tests.snippet-directive.test.ts: format + parse round-trip cases.cli.test.ts: eject end-to-end (piped confirmation) andinit --extract→ejectbyte-for-byte round trip.integration.test.ts: pins the out-of-sync message.npm run lint,npm run typecheck,npm run build,npm test(254 passed) all green locally.npm run lintand fixed any issues.npm run buildto verify the project builds.npm testto ensure all tests pass.Notes
MarkdownFilestays a separate@deprecatedinterface rather than an alias ofMarkdownDocument: the wrappers still return the old shape, and an alias would be breaking.DocumentCodeBlock.lineEndingis exposed so sync/extract can normalise snippet content to the block (fix: preserve markdown line endings #86).createMarkdownProcessor(never exported fromindex.ts) is gone fromparser.ts.Follow-ups (not in this PR)
meta; they can disagree only for backslash escapes / entities inside the directive.column - 1spaces, so fences inside blockquotes or tab-indented lists are not re-prefixed correctly (pre-existing onmain).