[brkosebay]

A discussion PR for merging the Apple A9 (iPad 5, G5P) GPU work into OrchardLinux, as Henrik asked: the questions are below and the full 2026-11-03 update is the first comment. Our A9 code is not public yet (see A7 in the update), so the branch holds one empty commit.

Position

Position. We still agree with your §4 direction: one Mesa driver on the upstream PowerVR Vulkan driver with per-generation Apple backends, and one Apple GPU kernel driver with typed, kernel-validated submits and per-context slots, and a firmware backend per chip. Nothing since 11-02 argues against it. A2 is a further reason for per-context PM records in the shared design.

Questions (kernel, firmware and clocks)

1. Step 2 (the code comparison). Where is it on your side? Our half is the 11-02 ISA table plus A1–A6 here. What is left that only you can compare: the firmware records, your UAPI, and the parts of pvr_arch_* you replaced?
2. The kernel UAPI. Ours validates typed register images and builds every firmware record. Yours takes Rogue-style commands through the powervr UAPI. Is a new Apple UAPI acceptable to you, shaped like Asahi's asahi_drm.h, with VM_BIND, syncobjs and per-context slots? And drm_sched for jobs? You already use it. Ours is a serial scheduler with round-robin between contexts.
3. The firmware-backend boundary. Our proposal: the common driver owns VMs, contexts, jobs, fences, the scheduler, the register licence and the power state machine. A backend owns boot, the work records, the bind or unbind of a context root, the kick and completion detection. Does that cut fit your RTKit backend?
4. Where the shared code lives, and the norms. An OrchardLinux organisation repository with both of us as maintainers? Branch and review rules (every change reviewed by the other side?), and testing: one Vulkan CTS subset plus per-chip hardware rungs that must stay exact before a merge?
5. AI authorship and upstream. Both stacks are mostly AI-written. Mesa's src/imagination and Asahi do not take that upstream. We propose that the shared trees label it (a Generated-by trailer) and do not aim upstream unless human authors take over. Is that your view too?
6. Order and owners. Your published code is the base on both sides. Mesa: you own the skeleton and the shared compiler, and we port G5P onto it. Kernel: we start from your published GPU kernel driver and add the A9 backend, the per-context PM model and the register licence onto it. The submit interface follows your answer to 3; until then our A9 backend keeps its current submit path behind the chip backend, so nothing waits on it. Swap anything you would rather own.
7. Clock and DVFS. You run firmware DVFS off and the host writes Fender through devfreq. On A9 that bypass (meta+0x6c8 = 0) also drops the firmware path our typed submits run on. So we run firmware policy with host power-on at q. Should the shared driver carry both modes, host-owned Fender (yours) and firmware policy plus power-on (ours), behind one devfreq interface?
   • One data request: could you send your A10's GPU OPP table as your m1n1 builds it (dt_set_gpu_pstates): opp-hz and opp-microvolt per state, up to the ~1 GHz state? IPSW device trees only carry placeholders. The A9 is close to the A10 in process and design, so your voltage at each frequency is the best guide we have for an A9 GPU overclock above 723 MHz. The raw PSINFO record words (PMGR GPU perf-state records) would be even better.

---

Written by our AI agents (Claude) from our notes and hardware logs; labels: [V] read from a hardware log, [I] inferred, [O] built and checked offline. Draft for discussion only: the commit is empty and nothing here is meant to be merged.

🤖 Generated with Claude Code

[brkosebay]

Apple A9 GPU on Linux: update after the 11-02 reply (2026-11-03)

A follow-up to our reply of 2026-11-02, not a re-send. Part A is what changed on A9 since then, proven results first. Part B is our position on the merge and the questions that decide the plan. Headline: Hyprland with GPU clients, with the GPU never parked, runs at the 60 Hz cap on A9 with 0 faults, and the GPU runs above 340 MHz for the first time.

Labels: [V] read from a hardware log, [I] inferred, [O] built and checked offline, not yet run on hardware. Written by our AI agents from our notes and hardware logs.

---

Part A. What changed on A9

A1. Correction to our 11-02 §1.2: register reads after jobs

Our 11-02 reply was wrong about R1. It said two kernels hung with one readl(R1+0x1000) as the first or only register access after a job. That account was second-hand, and we have found no primary log for it. Everything since contradicts it. The rule we now run, taken from iOS:

• The licence. Every runtime R0 read in Apple's A9 kext sits behind one predicate, isGPURegisterAccessible (firmware vtable +0x290). It passes only when all three hold:

  • the PMGR gfx domain is on (a software flag set at platform power-on);
  • 1 ≤ (R1+0x1000 & 0xf) ≤ the maximum pstate;
  • R1+0 bit 29 is clear. The firmware sets bit 29 in its power-down path after its cache drain and before it drops the GPU to pstate 0. That is the handshake we had never implemented.

  When the check fails, iOS skips the read. It never wakes the GPU to read a register, and a read never writes the kick register R1+0x14. Perf counters come from a firmware device-control command, not from MMIO.

• What hung A9 [I, high]: R0 reads made on a pstate check that had gone stale, at the moment the firmware had just gone idle and its governor was parking the GPU. The R0 bus does not respond while the GPU is at pstate 0 or on its way there. It is bus-level, not a read side effect: the same offsets read fine at other moments.

• R1 is safe while PMGR gfx is on [V]. Apple reads R1 in every state, including mid-transition. Two of our logs read R1+0x1000 mid-transition (0x10 and 0x11) and both returned. A 1 kHz census on the production kernel made 455,000 R1 reads with 0 faults, and the doorbell path reads R1 after every retirement.

• Hardware proof of the licence [V]: with parks forced on purpose, 758 licensed R0 reads (0 stale, 0 at pstate 0) and 254 reads refused at bit 29, with no hang. Our read paths no longer write R1+0x14.

So your Fender+0x1000 governor writes being harmless on A10 fit this picture: your firmware never parks the GPU, so R0 is never behind a stale check. Bit 29 is a firmware-side handshake, so it may exist on A10 too. Does your firmware ever set R1+0 bit 29?

A2. Never-park, several clients, and four faults on the way [V]

• Never-park. On A9 the firmware parked the GPU (pstate 0) around jobs and the host woke it again. With the GPU kept powered at pstate 1 and a host doorbell for every phase (iOS's submit model), together with the fast memory clock of A4, full-panel 3D passes on our fill fixture went from the park-mode times of clear 12 ms, solid 18 ms and textured blit 36 ms to 2, 2 and 4 ms. The memory clock alone accounts for part of that (A4). In this mode the GPU stays powered at pstate 1 through ≥ 30 s of full idle.
• Result: Hyprland plus quickshell, 8 gears runs and 16 foot open/close cycles in 16 minutes: 0 faults, every page flip completed at a 60 Hz vblank, and gears 31–35 FPS against 8.4 before (before the performance work of A4).
• The four faults, in the order we found them. Parking hid at least the shared-state fault and the PM write-back: a park drains the parameter manager (PM) and completes its pending write-back, and the faults never appeared in park mode [I for faults 2 and 3, which had no park-mode comparison]. If you move A10 to per-context slots with a GPU that never parks, expect the same class of fault.
  1. PM state shared across contexts. One PM record (ID 0, control, free list, pool) served every context. The PM's control words are global and advance with every job, so a context's saved PM state no longer matched the pool when it ran again. Symptoms: PM MMU faults and 3D stalls on whichever context ran next.
  2. The per-job free-list rebuild recycled pages the PM still held. The host rebuilt the PM's pool and free list before every job, and that handed out catalogue pages the PM was still using. Fix: build the PM pool once per context and let the firmware rebuild its free list, as iOS does.
  3. PM IDs aliased the device PM. Fix: a PM record, pool and PM ID per context.
  4. PM write-back into a closed slot. When the next TA switches PM contexts, the PM writes the previous context's state back through that context's slot. If that context had just closed and been unbound, the write walked an unbound slot (PM write fault, level 2). Fix: defer a close whose job was the last to run until another context's job has run.
• iOS's model, which this now matches [V decode]: iOS takes a ParamBuffer ID per prepare from a 127-entry pool and builds control, state and aux pages per PM. So the PM belongs to the context, not the device.

A3. The GPU clock

Proven [V]: the GPU runs at 474 MHz (pstate 2) on Linux, and in today's ladder up to 723 MHz (below), the first boot above 340 MHz. The host powers the GPU on directly at pstate q when it releases the firmware, and writes the firmware's target words to the same q first. ALIVE came at state 2, every job ran at pstate 2 (1812 doorbells read state 2), and there were 0 faults. In today's ladder run, gears stayed at the 60 Hz cap and the median job time fell from about 4.3 ms (340 MHz) to 3.9 ms. That is a hardware log of today's run, not yet written up.

What we learned on the way:

• The A9 firmware sets the pstate only once, in its boot sampler. In the policy mode we run, it never calls its pstate setter at runtime while the GPU is powered. So a host "pin" to a higher pstate does nothing after boot, and a mismatch between the firmware's target and the actual state at its park handshake stalled the firmware.

• meta+0x784 is an abstime offset. Our seeded-policy attempts (pre-filling the firmware's policy record so that its boot sampler sets q) all failed. The firmware's clock is the AIC timer plus the 64-bit word meta+0x784/0x788. iOS writes ml_get_abstime_offset() there. We wrote the absolute counter, so the firmware's "now" ran about 111 s ahead of our timestamps. Its sampler then clamped the elapsed time, ran its load governor on empty accumulators and parked the GPU. A seed with the offset fixed is built and has not run. Host power-on made it unnecessary. Note that host power-on at q is not iOS's route: iOS powers on at pstate 1 and lets the firmware move. Ours is consistent with the firmware's words, not a copy of iOS (see B8).
  If your firmware's init data has a time-offset word, check what you write there.

• The full stock ladder passed in today's hardware run (logs not yet written up). Each step had ALIVE at the target state, about 15k doorbells reading that state, 0 faults and no freeze. All three runs kept gears at the 60 Hz cap. SoC temperature after 60 s of gears stayed under 60 °C at every step.

  pstate	MHz	TA / 3D / whole job, medians
  1	340	1.26 / 2.78 / 4.28 ms
  2	474	1.09 / 2.55 / 3.92 ms
  3	550	1.05 / 2.36 / 3.74 ms
  4	723	0.86 / 2.26 / 3.39 ms

  Going from 340 to 723 MHz (+113 % clock) cut the whole-job time by only about 21 %. For our desktop jobs, memory, firmware round trips and cache sync dominate over shader clock.

• In progress:

  • Runtime switching is not built. Our first plan is to re-release the firmware at the new pstate through the in-boot recovery path (about 21 ms, every context kept), at idle.
  • Overclocking above 723 MHz is a goal. The first candidate is an 804 MHz descriptor that Apple's boot loader already carries.

• Today's hardware log, defaults and overclock [V, not yet written up]:

  • 723 MHz is now our release default: host power-on at pstate 4 every boot, 0 faults.
  • Overclock: we added a fifth pstate record (Apple's 804 MHz descriptor, then our own PLL settings) at the pstate 4 voltage (881.25 mV core, 987.5 mV buck8). 804, 852, 900 and 948 MHz each ran a desktop session plus a GPU benchmark set with 0 faults and below 72 °C. 996 MHz at the same voltage faulted the 3D on its first jobs and hung the SoC. No voltage step above stock was tried; we have paused the overclock to work on the drivers.
  • At 948 vs 723 MHz, shader-heavy glmark2 scenes at 1536x2048 gained 11–21 %. The full-screen blur scene stayed at 31–33 FPS: bandwidth-bound. Whether each step's frequency really reaches the core is being measured next, with the Rogue cycle counters against CR_TIMER.

A4. Performance: Hyprland at the 60 Hz cap at 340 MHz [V]

Gears windowed under the Omarchy Hyprland session, the same session every step, all at 340 MHz:

Step	Gears FPS
never-park + doorbells (A2)	33–35
+ memory clock fast (SoC state 12)	39–40
+ completion wake on the firmware event	55.0
+ cache maintenance by range	58.5
+ GPU clear rects, empty-job removal, upload trim	59.4–59.6 (the 60 Hz cap)

Per-job host time fell from 7–8 ms to about 5 ms.

• Memory clock (your tip, thank you). On A9 it is the same register and protocol as T8010: PMGR+0x68000, low nibble, only states 12 and 13 accepted, poll bit 31. iOS writes no voltage for it: PMGR's sequencer moves the SoC rail from the state record. Our driver checks the record codes and the sequencer configuration first, then makes the change and confirms that the live rail code followed it. CPU memset 4.6 → 7.0 GB/s; textured blit 36 → 16 ms and solid fill 18 → 12 ms on the fixture (before never-park). It is an S8000 variant of the pmgr-misc driver.
• Completion wake: −1.33 ms per job. The waiter used to sleep 1–2 ms per RAM sample. It now sleeps until the next firmware event, decoded in the IRQ thread, with a 2 ms safety timeout. Median job 5.73 → 4.40 ms, TA phase −0.44 ms, 3D −1.17 ms, with 0 lost wakeups. Nothing on the GPU side changed, so the whole gain was detection latency.
• Cache maintenance by range. A job's cache maintenance now covers each resource's range (page-widened) instead of whole 64 MiB arena chunks and dedicated buffers. +6 % FPS. The software-rendered shell's buffer uploads were the main cost.
• GPU clear rects. Our driver used to split a pass at vkCmdClearAttachments into a clear-only GPU job plus CPU memsets. It now draws the rectangles as quads inside the pass. Draw-less passes no longer submit a GPU job.
• Upload trim. A job uploads only the arena bytes it can read: about 150 kB per job instead of 1.4 MB.
• Qt Quick: on the GPU it caused a redraw storm (about 23 GPU jobs/s at idle). quickshell uses Qt's software renderer, at 0 jobs and 0.2 % CPU at idle.

Direct-to-swapchain Hyprland (one pass per frame). Hyprland 0.56.2 always renders into a work buffer and copies it to the swapchain, so every full-damage frame costs two full-panel passes. Our patch renders straight into the output buffer when nothing needs the offload (no blur, no colour management, no mirror, no screen shader, zoom 1), and otherwise takes the stock path in the same binary.

• The ABI lesson. The first build added data members to CHyprOpenGLImpl and CRenderPass. Omarchy's titlebars plugin is built against the stock headers, so it read shifted fields and crashed in boundingBox() as soon as a window floated, and start-hyprland restarted in safe mode. The fixed patch adds no data members to any header (state in file-local statics, only non-virtual member functions declared). Offline with the plugin loaded: 1.00 passes per frame against stock's 2.00, no crash.
• On hardware: one Hyprland instance for the whole run, no safe-mode restart, one full-panel job per frame, gears
  59.6–60.0 FPS. That is a hardware log of today's run, not yet written up.
• Buffer-age guard [O, not yet run]. Aquamarine's swapchain reports age = its length on every next(), and Hyprland's damage ring trusts it. That is fine with a work buffer. With direct rendering it is only correct if that buffer was last drawn exactly age rotations ago. The patch tracks this per buffer and repaints the whole output on any doubt (a new buffer, a reallocated swapchain, a skipped or repeated next()). Relevant to you if you ever render straight into the swapchain.

A5. Fixes you may want to reuse

• Mesa loader, kmsro and a dma-buf leak [V]. Our kernel driver's name, apple_g5p, has no Gallium descriptor, so the loader gave its render node the kmsro fallback. Zink then imported every client dma-buf a second time through renderonly and released it with DESTROY_DUMB, which a render node refuses. Every buffer Hyprland ever imported stayed pinned, and Shmem grew all session. Fix: map the name straight to Zink in pipe_loader_drm.c, as upstream does for powervr. Also, kmsro duplicated its GPU fd with plain dup(), which drops close-on-exec, so every child of the compositor inherited the render node. It now uses os_dupfd_cloexec(). Check: if your DRM driver still registers as powervr, you already take the Zink path. If you renamed it, check for the kmsro fallback.
• The meta+0x784 time offset (A3), if your firmware has a similar time word.
• Discard with depth/stencil (punch-through) [O]. Built and checked against Apple's A9 compiler and driver decode (FAST_PUNCH_THROUGH, or PUNCH_THROUGH when a written channel reads the destination; tag write kept for a discarding program that writes no colour). Offline: 9 scenes and the blend ladder pass, and Hyprland with blur or with titlebars runs 40 s (about 2,180 frames) where stock died after 3 frames on the punch-through refusal. The hardware ladder has not run, so treat the encoding as unproven.
• The R0 licence (A1) and the PM-per-context rules (A2), for when A10 goes to per-context slots.

A6. ES 3.0, and an offer for 3.1

Status [V]: Zink offers OpenGL ES 3.0 / GLSL ES 3.00 with 0 failing terms. Exact on hardware: 2 and 4 colour targets, transform feedback (two buffers, 3000 vertices captured once), depth/stencil formats and sampling, the cube/NPOT/array texture types and 4× MSAA fixtures. Not done: 8 of 16 vertex inputs, storage images, narrow and integer colour targets, robustBufferAccess, and compute (the firmware takes a compute job and never retires it).

When we read your fork on 11-01, its eglinfo showed OpenGL ES 3.1, and your notes listed MRT and transform feedback as not done. That is a normal state with upstream's tables (Zink does not check that an advertised feature works). If it helps, our MRT and XFB implementations and their hardware rungs are yours to use. The ISA match makes the compiler parts likely to carry over. The rest of 3.1 is our next feature target (B8).

A7. Publishing, unchanged in substance

The notes repository is public. The kernel and Mesa are still not. The kernel's desktop boot still runs through our bring-up ladder, including a 4-byte patch of Apple's firmware in RAM (the test without the patch has not run yet). The Mesa driver still embeds Apple's helper programs (153 instructions and 14 PDS programs we cannot yet re-encode). Neither is a matter of style. Both are tied to our own skeletons, and the merge below is how we plan to retire them: on your Mesa skeleton those programs come from upstream pdsc and our compiler, not from Apple's bytes, and the firmware patch has to go before our A9 kernel backend is published on your base.

Our open questions to you from 11-02 §6 still stand. For the merge, the ones that matter most are 2 (post-job power), 3 (cache visibility) and 9 (the power envelope).

---

Written by our AI agents (Claude) from our notes and hardware logs; labels: [V] read from a hardware log, [I] inferred, [O] built and checked offline. Draft for discussion only: the commit is empty and nothing here is meant to be merged.

🤖 Generated with Claude Code

[henrikklee]

Same request as on OrchardLinux/mesa#1: please fork OrchardLinux/linux and OrchardLinux/mesa and push your A9 code there first. We'll review it and answer the kernel questions here once it's up.