Repository navigation
clazel-gpu: batching, shaders and the Metal backend - #153
Merged
Merged
Conversation
A validated display list and a viewport to a frame of uploads and scissored draws of one quad format; a linear glyph atlas with a bump allocator compacted when full; the fake backend, which runs a frame in software with no foreign call; and the software reference, which paints a list straight from its items. Every scene matches the reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One quad per instance read by index from a word buffer, coverage read by pixel from the linear atlas, both reads bounded in the shader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…17b) `cargo xtask tables` writes quad.vert.spv and quad.frag.spv and `cargo xtask check` fails if the committed SPIR-V differs. naga is tooling, in xtask only; its packages join the allowlist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
objc_msgSend must be called through a function pointer of each method's
exact type. Declaring it once per signature with #[link_name] trips
rustc's clashing_extern_declarations warning, which CI denies, so the
checker learns one form and nothing wider: a dispatcher (only
objc_msgSend) is declared once as `fn objc_msgSend();` and used only in
a const on its file's list, written exactly
`const NAME: unsafe extern "C" fn(..) = unsafe {
std::mem::transmute(objc_msgSend as unsafe extern "C" fn()) };` with a
SAFETY comment above; such a const is named only in calls inside unsafe
blocks. Every other transmute, use of the dispatcher, or call through a
pointer is rejected, with a test for each.
Also closes a gap found on the way: `unsafe extern "C" fn f() {}` was
read as an extern block and passed; it is now rejected as an unsafe fn.
Function pointer types are no longer counted as unsafe blocks, and the
check prints the casts separately.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Metal's buffer copies on macOS need offsets and sizes in multiples of 4, so each mask's slot is padded to whole words and the store may grow to 128 MiB, which always holds a compaction. Kani harnesses for the placement, growth, indexing, rectangle, blending and batching arithmetic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(5.17b) Safe wrappers for what clazel-gpu's Metal backend needs: the default device, a command queue, a pipeline compiled from MSL source, buffers copied from slices, an RGBA8 IOSurface and a texture on it, buffer copies, a scoped render pass with scissor, bytes, buffers and draws, commit and wait, and reading the texture back. Messages go through 21 typed casts of objc_msgSend, each listing its selectors and citing Apple's documentation. Apple silicon only. The clazel-sys audit is outstanding: it must be written by a different session, so `cargo xtask check` fails on the audit hash until then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One command buffer per frame: uploads as blit copies from a staging buffer, render passes between them under scissors, into a texture on an IOSurface the backend makes, behind a Target that item 5.18 can supply. Every scene drawn on this machine's Apple M1 Pro equals the software reference in every channel and the stored pixels; without a device the tests print why and pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…17b) Each scene is its own test; the 2.3 MB atlas scene is ignored under Miri (523 s on Miri's clock for all scenes), and a small batching test covers a compaction in the middle of a list there instead. The blending test thins its inputs under Miri (290 s before). Kani: the draw step's cut is proved for every input and one draw for any clip and viewport; the texel-index and atlas-upload harnesses reached their caps and are removed, listed as not covered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d Miri tables (5.17b) The ramp coverage is one 256-byte period repeated, the same bytes made by a block copy: building it byte by byte cost over 350 s of Miri's clock in every test that listed the scenes. Every scene now runs under Miri, the slowest in 56 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…CI (5.17b) Each Metal test prints `clazel-gpu: Metal device "<name>" (...)`, the name from the OS (system_profiler, hw.model, kern.hv_vmm_present): the clazel-sys wrappers cannot read the device's name without new foreign calls. A missing device is a skip only on a developer's machine; with CI set it is a failure, so the first CI run settles whether macos-15 offers a paravirtual Metal device. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Audited by a session that wrote no part of clazel-sys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
atiernan
had a problem deploying
to
full-verification
November 12, 2026 17:34 — with
GitHub Actions
Error
An Objective-C exception unwinding out of a call through an `extern "C"` pointer is undefined behaviour (Rust reference, panic.unwind.ffi.undefined); through a "C-unwind" one it is a defined unwind or abort (items.fn.extern.unwind.behavior). The checker now requires "C-unwind" in all three places a dispatcher's ABI appears: its `unsafe extern` block, the const's pointer type, and the transmute's source. `D as T` only coerces the function item to a pointer of its declared ABI, so the three agree and no cast can mix "C" and "C-unwind". clazel-sys's metal.rs fails this check until the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d on unwind (5.17b) The auditor's finding (1) at 71577331: through the `extern "C"` casts an Objective-C exception would be undefined behaviour. `objc_msgSend` now has its own `unsafe extern "C-unwind"` block and every dispatcher cast is `unsafe extern "C-unwind" fn`, as the checker now requires. The module documentation states what an exception does: it unwinds through the Rust frames as a foreign exception, defined by the Rust reference (items.fn.extern.unwind.behavior; panic.unwind.ffi.catch-foreign if it is caught). Nothing catches one. The five other foreign functions stay "C": none sends a message. Also from that audit: - (3) Encoders are ended by a guard, `EndEncoding`, whose Drop sends `endEncoding`, so a panic in `CommandBuffer::render`'s closure no longer skips it. - (7) The `MSG_USIZE` comment no longer lists `MTLBuffer/length`, which nothing calls. `unsafe` blocks: 138 (338 lines), up from 137 (336) because of the new extern block. README.md states the new counts. clazel-gpu's README describes the ABI. The audit of the new source is still to be written by another session, so `cargo xtask check` fails on the audit alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… 5.17b): approved Audited by a session that wrote no part of clazel-sys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
atiernan
temporarily deployed
to
full-verification
November 12, 2026 17:58 — with
GitHub Actions
Inactive
…tion pointer (5.17b) Metal is reached through objc_msgSend, which has no single C signature. clazel-sys declares it once and casts it to typed function pointers, one cast per unsafe block, in a const; xtask/src/sys.rs accepts exactly that form and counts the casts. Rule 1 now says so. Both audits of that code are in docs/SYS-AUDITS.md (71577331, 4909bddf). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
atiernan
had a problem deploying
to
full-verification
November 12, 2026 18:03 — with
GitHub Actions
Error
`println!` and `eprintln!` are captured by libtest and dropped for a passing test, and CI runs `cargo test` without `--nocapture`, so the `clazel-gpu: Metal device` line never appeared in its log. Both that line and the skip line are now written straight to standard error, each in one call and once per test binary (`std::sync::Once`). The README says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
atiernan
temporarily deployed
to
full-verification
November 12, 2026 18:40 — with
GitHub Actions
Inactive
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch makes
xtask/src/sys.rsaccept a new form inclazel-sys, and CLAUDE.mdrule 1's wording should change to match it. CLAUDE.md is not edited in this branch:
the session that wrote it does not edit CLAUDE.md on another session's instruction, so
the change is proposed here for the maintainer to apply at merge. The patch is
gpu-drawing-5.17b-claude-md-rule1.patchin the side-session notes folder.Old:
Proposed:
Why: the plan reaches Metal through several
#[link_name = "objc_msgSend"]declarations, one per message signature. rustc 1.98.1 rejects that under CI's
-D warnings:Three options were put to the maintainer, who delegated the decision to the
coordinating session. That session chose typed casts (option 1), under conditions this
branch implements. Rejected: allowing the lint in
clazel-sys's manifest (it weakens alint, against rule 2), and assembly trampolines (a new kind of code to audit).
The
clazel-sysaudit is outstandingclazel-sysgainssrc/macos/metal.rs. Its author, this session, must not write theaudit row: a different session must audit it. Until then
cargo xtask checkfailson the audit, and that is its only failing check:
unsafeblocks: 62 (237 lines) before, 137 (336 lines) after, of which 21 aredispatcher casts (
cargo xtask checkprints both counts, and README.md states them).Foreign functions added (
METALinxtask/src/sys.rs, filecrates/clazel-sys/src/macos/metal.rs):objc_msgSend(a dispatcher, never calleddirectly),
objc_getClass,sel_registerName,MTLCreateSystemDefaultDevice,IOSurfaceCreate,CFRelease, plus an empty block that links Foundation. The 21dispatcher casts (
METAL_CASTS), one per message signature:MSG_ID,MSG_VOID,MSG_USIZE,MSG_ID_ID,MSG_ID_USIZE,MSG_VOID_ID,MSG_VOID_USIZE,MSG_VOID_BOOL,MSG_ID_ID_ID_OUT,MSG_ID_ID_OUT,MSG_ID_PTR_USIZE_USIZE,MSG_ID_USIZE_USIZE,MSG_ID_ID_ID_USIZE,MSG_ID_IDS_IDS_USIZE,MSG_VOID_ID_USIZE_ID_USIZE_USIZE,MSG_VOID_ID_USIZE_USIZE,MSG_VOID_PTR_USIZE_USIZE,MSG_VOID_SCISSOR,MSG_VOID_CLEAR,MSG_VOID_USIZE_USIZE_USIZE_USIZE_USIZE,MSG_VOID_OUT_USIZE_REGION_USIZE. Each listsabove it every selector that uses it.
For the auditor (docs/SYS-AUDITS.md is not edited here): beyond the usual checks,
each dispatcher cast needs its function type compared with every method its comment
lists, exactly: receiver
idandSELfirst, then each parameter's width andsignedness (
NSUIntegerisusize;unsigned longis 64 bits;BOOLisboolonApple silicon, the only target the module builds for), pointer parameters, structures
passed by value (
MTLScissorRect,MTLClearColor,MTLRegion, whose#[repr(C)]declarations need checking against
<Metal/MTLTypes.h>and the encoder headers), andthe return type (
voidagainstidagainstNSUInteger). A mismatch is undefinedbehaviour no machine check here catches. Also worth a reviewer's eye:
extern "C". An Objective-C exception thrown through one would unwindwhere that ABI forbids it. Metal raises exceptions for API misuse, which the wrappers'
checks are meant to rule out. Whether the casts should be
"C-unwind"(the checkeraccepts only
"C"today) is left to review."IOSurfaceWidth"and so on), because the
kIOSurface…constants are data symbols the checker doesnot allow. The values are those of the header. A real-device test checks that a
texture on a surface gets the surface's size.
with_autorelease_pooldrains around each frame.docs <Uppercase…>form, writtendocs Framework/Class/member(for exampledocs Metal/MTLDevice/newCommandQueue).No new citation form was added to the checker.
The checker change (
xtask/src/sys.rs, its own commit4d4c7d8, before any Metal code)It accepts this form and nothing wider:
objc_msgSend) is declared once, asfn objc_msgSend();, and is named nowhere else.conston its file's list, written exactlyconst NAME: unsafe extern "C" fn(..) -> .. = unsafe { std::mem::transmute(objc_msgSend as unsafe extern "C" fn()) };,with a
// SAFETY:comment directly above.unsafeblocks,one call per block, each with its own
// SAFETY:comment.Each rejected form has a planted test:
transmuteortransmute_copy, of data or in an unlisted const;second call in the block;
unsafe extern "C" fn("C-unwind", a safeextern "C" fn,Option<..>,usize);// SAFETY:comment;a return type;
Found on the way and closed in the same commit:
unsafe extern "C" fn f() {}, anunsafe fn, used to be read as an extern block and passed. It is now rejected. Functionpointer types are no longer counted as
unsafeblocks (xtask/src/scan.rs), and thecheck prints the dispatcher casts separately. Not extended to Vulkan's
vkGetDeviceProcAddr: that needs a second dispatcher kind, a looked-up pointer ratherthan a symbol, to be decided when that backend is built.
What this branch does (item 5.17b, first part)
New crate
crates/clazel-gpu, tier 0, placed with the renderer inPLACES.src/batch.rs).Gpu::drawtakes the list's bytes, decodesand validates them with
clazel-displaylist, and relies only ontheorem_validated_bounds. It turns the list into a frame of atlas replacements,uploads, and draws of one quad format under scissor rectangles.
src/atlas.rs): a bump allocator over word-aligned slots,compacted when full, up to 128 MiB.
src/backend.rs), plus a fake backend with noforeign calls (
src/soft.rs).src/reference.rs) that paints a list straight from itsitems, independent of the batching.
against the reference, exactly.
shaders/quad.metalin MSL, andquad.vertandquad.fragin GLSL, compiled to the committed.spvfiles bynaga30.0.1 in xtask(tooling, never shipped).
cargo xtask checkhas a new row, "compiled GPU shaders(SPIR-V)", that confirms they match, and
cargo xtask tablesrewrites them. naga'spackages are on the allowlist: 19
toolingrows;foldhashisproofandlibmmoved to
shipped, each explained in its row (neither is compiled into the browser).src/metal.rs, macOS on Apple silicon). It draws into atexture on an IOSurface the backend creates, behind
Target, which 5.18 can supply.Uploads are blit copies ordered with the render passes; each frame is one command
buffer, waited for.
Metal results
tests/metal.rsdraws every scene and compares each list's pixels with the softwarereference (allowed: one level per channel) and with stored pixels (
tests/pixels/,exact). On this machine (Apple M1 Pro, macOS 26.7.2) every channel of every scene
equals the reference: 0 channels differ across 7 lists, 19,968 channels in all. A
mutant shader that gives masks full coverage fails both comparisons. Each test prints
what it ran on, as
clazel-gpu: Metal device "<name>" (…). The name comes from the OS(
system_profiler,hw.model,kern.hv_vmm_present); theclazel-syswrappers can'tread the device's own name without new foreign calls, so none were added. On this
machine it prints
clazel-gpu: Metal device "Apple M1 Pro" (Metal support: Metal 4; hw.model MacBookPro18,3; kern.hv_vmm_present 0; …).A missing device is a skip only on a developer's machine. When
CIis set, nousable Metal device is a test failure, saying so. CI's
macos-15runner (Applesilicon, virtualised) is expected to offer a paravirtual Metal device, and this PR's
first CI run settles whether it does. So far the Metal tests have run only on this
machine. A paravirtual device may behave differently (IOSurface-backed textures, the
blit path, rounding), so a failure there is possible and would be real information.
Kani
Each harness ran alone, in its own process group, with a 20-minute and 8 GB cap polled
every 5 s.
place_anygrown_capacity_anyrect_at_anyintersect_anyblend_anyvisible_anyclip_then_draw_anyRemoved at a cap, each listed as not covered in the README:
texel_index_any: the 20-minute cap, twice, at 0.17 GB.atlas_upload_any: 8 GB in 83 s.clip_then_draw_any: 8 GB in 21 s. It was split intovisible_anyand the one-draw form above.No
cbmc,goto-*orkani-driverprocess was left after any run.Miri
One test per process, 10-minute cap; times on Miri's clock:
bad_lists_are_refused_before_drawingblending_is_the_exact_value_roundedclazel-syssizes_are_checked_before_any_callTwo tests were first over CI's 300 s and were fixed before this table:
byte. The ramp is now one 256-byte period repeated, the same bytes.
cfg(miri), with thereason stated in the test.
The tests that call the GPU are ignored under Miri, as stated in each of them.
Decisions (recorded in
crates/clazel-gpu/README.md)reference rounds the exact value once, halves up, and clamps.
result clamps at 255.
nothing is drawn under an empty clip.
naga adds bounds checks to the SPIR-V.
macOS need multiples of 4), up to 128 MiB, so a compaction always fits.
One frame at a time.
offline Metal toolchain. Apple silicon only.
clazel-gpuis placed with the renderer; the detail image has noGPU-process panel yet (5.27).
What is left of 5.17b
libvulkan-only linking,hand-written
#[repr(C)]structures, a second dispatcher kind forvkGetDeviceProcAddr, no WSI, and the committed SPIR-V.checker, DXBC shaders with a check like the SPIR-V one, and
CreateSwapChainForCompositionSurfaceHandle.Verification state
cargo testfor clazel-gpu, clazel-sys and xtask: all pass.-D warningson those three crates, andcargo fmt --all --check: clean.cargo xtask check: only theclazel-sysaudit fails, as above.cargo xtask verify --changedwas not run, by instruction.full-verificationenvironment to approve the run.🤖 Generated with Claude Code