Skip to content

perf: share one span render context per transaction - #321

Closed
vinceblock99 wants to merge 3 commits into
checkpoint/span-native-integrationfrom
perf/shared-span-render-context
Closed

vinceblock99 wants to merge 3 commits into
checkpoint/span-native-integrationfrom
perf/shared-span-render-context

Conversation

@vinceblock99

Copy link
Copy Markdown
Contributor

What

Every caller of retrieve_span_if_present built a fresh SpanRenderContext, which verifies and decodes the whole visible closure, for every file it rendered. On a span import of hyperfine (1,097 commits, 75 files) one atomic view switch built it 312 times, atomic record -a 466 times and atomic restore -f 157 times, at roughly 55 ms each, while the materialization itself took milliseconds.

  • SharedSpanRender: a lazily built context shared by every file read through one transaction and visibility closure. Per-file root, capability and visibility checks are unchanged; only the closure decode is shared. Used in project_tree, project_change_closure_with_txn and plan_file_moves.
  • Repository::get_file_contents_on_view and recorded_presence_on_view: one transaction, one projection and one context for a batch of paths, with the absent-liveness check scoped to exactly those paths. Used by the shim checkpoint observation, working-copy verification and the CLI and daemon restore paths, which rendered (and discarded) every file one at a time.

Why

The span backend only pays off if the decode happens once per operation. Before this change a span-backed repository was barely faster than the graph backend on the CLI and used about twice the CPU.

Measured (hyperfine span import, 75 files)

Command Before After
atomic view switch 25–34 s, 312–387 context builds 4–7 s, 12 builds
atomic restore -f (all 75 files) 15 s, 157 builds 4.8 s, 8 builds

Working-copy tree digests match before and after on every switch and restore; graph-only repositories build no span context and are unaffected.

Verification (local)

  • New batch_content_tests compare the batched reads byte for byte with the single-path reads for graph-origin, span-rooted, empty, directory and absent paths on two views.
  • cargo test -p atomic-repository --lib (1561 passed), cargo test -p libatomic, cargo test -p atomic-cli --bin atomic -- restore checkpoint (43 passed), cargo clippy --workspace -- -D warnings and cargo fmt --check all pass.

Every caller of retrieve_span_if_present built a fresh SpanRenderContext,
which verifies and decodes the whole visible closure, for every file it
rendered. On a span import of hyperfine (1,097 commits, 75 files) one
`atomic view switch` built the context 312 times, `atomic record -a` 466
times and `atomic restore -f` 157 times, at roughly 55 ms each, while the
materialization itself took milliseconds.

- Add SharedSpanRender: a lazily built context shared by every file read
  through one transaction and visibility closure. The per-file root,
  capability and visibility checks are unchanged; only the closure decode
  is shared.
- Use it in project_tree, project_change_closure_with_txn and
  plan_file_moves, which render every projected file in one transaction.
- Add Repository::get_file_contents_on_view and recorded_presence_on_view
  so callers that read many paths get one transaction, one projection and
  one context, with the absent-liveness check scoped to exactly those
  paths. Use them in the shim checkpoint observation, working-copy
  verification and the CLI and daemon restore paths, which previously
  rendered (and discarded) every file one at a time.

Batched reads are tested byte for byte against the single-path reads for
graph-origin, span-rooted, empty, directory and absent paths on two views.
On the hyperfine span import, `atomic view switch` drops from 25-34 s to
4-7 s (12 context builds) and `atomic restore -f` of all 75 files from
15 s to 4.8 s (8 builds); graph-only repositories are unaffected.
Checkpoint observation no longer reads graph content for paths that carry
conflict-marker bytes. Name-conflict errors carry the same path and
claimant counts as the single-path reader. The daemon and CLI restore
paths classify every path up front but still walk and report them in
status order, so untracking, failures and output lines happen exactly
where they did before. The batch-read test now asserts that the fixture
really records span-rooted files.
@vinceblock99

Copy link
Copy Markdown
Contributor Author

These changes were already merged through Aaron's PR #323, so closing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant