Skip to content

Copy verbatim runs in the plaintext renderer, 2x faster parse + plaintext - #397

Open
czj wants to merge 2 commits into
github:masterfrom
czj:perf/plaintext-verbatim-runs-upstream
Open

czj wants to merge 2 commits into
github:masterfrom
czj:perf/plaintext-verbatim-runs-upstream

Conversation

@czj

@czj czj commented Oct 26, 2026 •

Copy link
Copy Markdown

Hi, thanks for maintaining cmark-gfm. We use it through the Ruby gem markly, and to_plaintext showed up in our profiles.

S_out in render.c decodes every code point, hands it to the renderer's outc, which re-encodes it, and appends it to the buffer one at a time. The plaintext renderer's outc is the identity, and every renderer writes LITERAL text back unchanged too, so most of that work rebuilds the bytes it started from. This PR lets S_out validate a run of such characters and copy it with a single cmark_strbuf_put, updating column, begin_line and begin_content the way the per-character loop would. Spaces when wrapping, newlines in LITERAL, invalid UTF-8 and extensions with a commonmark_escape_func still take the existing path, and a run never starts once a break opportunity is pending, so wrapping is unchanged. plaintext.c opts in by passing a NULL outc to cmark_render; render.h is internal and not installed, so the public API and ABI stay the same.

Parsing then rendering to plain text with CMARK_OPT_DEFAULT goes from 36.9 µs to 18.7 µs per text (1.97x) on 916 real texts from our app (French product descriptions and posts, median 927 bytes), and from 384k to 163k instructions per text (−58%). On 1,084 Markdown files taken from 265 Ruby gems (READMEs, changelogs, docs) it goes from 1.85M to 1.10M instructions per text (−41%). Instructions retired come from /usr/bin/time -l, median of 5; times are the median of 5 interleaved rounds; Apple M5, macOS, Apple clang 21.0.0, CMAKE_BUILD_TYPE=Release, master at 27d942c (0.29.0.gfm.13).

The output is byte-identical to master: 66,832 inputs (the 916 real texts, 60,000 random soups of Markdown tokens, 5,000 random byte strings, whitespace-mangled copies of the real texts) through 5 option and extension sets, rendered to plaintext, commonmark, latex and man at widths 0, 1, 5, 8, 30 and 80 plus HTML, along with API-built text, code and code block nodes holding invalid UTF-8: 8,354,225 outputs with the same SHA-256 on both sides. A mutant that miscounts the column of long runs changes that hash, so the harness does reach the new path. ctest passes. The new api_test case covers wrapping inside a list item, multibyte characters and a code block's prefix; it passes on master too, since the change is meant to be invisible.

The tradeoff is that S_out now keeps its bookkeeping in two places, so a later change to the per-character loop has to be mirrored in the run path. I kept that path to about 30 lines right next to the loop to make it hard to miss.

The same commit is proposed to markly's fork in socketry#7 and socketry/markly#35. Would you be open to taking it here as well?


🤖 Generated with Claude Code, but prompted, read, and verified by @czj, a caring human :)

S_out decoded every code point, re-encoded it and appended it to the
buffer one at a time, even when the renderer writes it back unchanged
(LITERAL escaping, or the plaintext renderer whose outc is the identity).
Runs of such characters are now validated and copied with a single put.
Spaces when wrapping, newlines in LITERAL, invalid UTF-8 and extensions
with a commonmark_escape_func still take the per-character path, and a
run never starts once a break opportunity is pending, so wrapping is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmark_utf8proc_encode_char writes these two code points as a single byte, so copying their original three bytes changed the output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@czj

czj commented Oct 27, 2026

Copy link
Copy Markdown
Author

A correction to the description: the output was not byte-identical for U+FFFE and U+FFFF. cmark_utf8proc_encode_char writes those two code points as a single byte, while a run copied their original three bytes, and my fuzzing never generated them. fb2c8e6 stops a run on either one; with it, outputs match master again on the same 8.35M renders, plus 61M random lines that now include both code points.

One more thing I should have said upfront: the 2x is at width 0, which is markly's default for to_plaintext. With wrapping at 80 columns the gain drops to about 10%, because a run stops at every break opportunity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant