Skip to content

fuel: the SGI Fuel (IP35) machine - #368

Draft
jamesbraid wants to merge 31 commits into
techomancer:mainfrom
jamesbraid:fuel-ip35
Draft

jamesbraid wants to merge 31 commits into
techomancer:mainfrom
jamesbraid:fuel-ip35

Conversation

@jamesbraid

@jamesbraid jamesbraid commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

This adds the SGI Fuel: a 500 MHz R14000 on a Bedrock node with 512 MiB, the dual-head XBridge, the IOC3 (Ethernet, serial, parallel, PS/2), the two-channel ISP12160, the L1 system controller, AMD boot flash and OHCI USB. PROM 6.210 passes its diagnostics and boots IRIX 6.5.30 to the 4Dwm desktop, and with a V12 board OpenGL works: SGI Quake, SGI Doom and Performer all draw.

I ported most of the devices from my sgi-origami QEMU branch, and their tests follow its qtests. HACKING.md and docs/fuel-ip34.md cover the machine.

This sits on #362 (the VPro boards) and the three R10000-family CPU PRs, and lands after them. Those bring the R14000 model, the 44-bit addresses IRIX's n64 stack needs, the cache diagnostic arrays and tag mask the PROM's cache test walks, and refused-store handling. What's left here for the CPU is Fuel-only:

  • R14000 uncached SB/SH go out as write64_masked. The PROM's ECC memory test reads the other byte lanes and fails without it.
  • R10000-family CPUs keep the uncached attribute from EntryLo 63:62 on mapped pages. Bedrock decodes it as the address space. The other R10000 machines drop those bits before the bus, so it doesn't change them.

The PROM's startup stack uses the primary data cache tag-store hook from #366: the Fuel PROM keeps its stack in the cache before any memory is set up, so Bedrock answers that 32 KiB window from a private page while the tags are valid.

PS/2 through the IOC3 is the default keyboard and mouse, as the Fuel manual shows. USB HID is there too with [fuel] usb_input. There's no audio yet: the Fuel had none built in, and SGI's option was a PCI card.

fuel-final-desktop fuel-hinv-quake fuel-final-doom

Testing

  • PROM 6.210 runs its diagnostics clean ("ALL DIAGS PASSED") and boots IRIX 6.5.30 to login; hinv shows the R14000 rev 2.3, both QL12160s, the V12 and OHCI.
  • 4Dwm desktop, SGI Quake in a window, SGI Doom's timedemo and Performer's truck demo all draw correctly on the V12.
  • cargo test --release --lib --features jitv2: 3147 passed, 0 failed, and the 91 VPro golden frames match.

EntryHi, the TLB match and the xuseg bounds check all stopped at VA bit 39,
but the R10000 family implements 44 bits: VPN2 is 43:13 and the user segment
is 16 TiB (R10000 UM pp. 219, 288). A 64-bit kernel can put an n64 user
stack above 40 bits (0x0fff_ffff_aee4, for one). EntryHi dropped VA[43:40]
and such an address took an Address Error.

The width now comes from the part's VA_BITS in each place: the EntryHi write
mask, the TLB's 64-bit compare (Tlb::set_va_bits, installed by the executor
beside set_cache_attrs), the xuseg bounds check, the micro-TLB's cacheable
segment test and the EntryHi value a TLB exception loads. The mask was
spelled three times and is now one const fn, mips_tlb::vpn2_mask. All of it
is a const of the CPU model or set once at construction; the TLB lookup is
unchanged.

This reverses a comment in the executor (b7d2a18) that said widening the EntryHi mask
breaks the IP28 and Octane kernels. Those two machines and an O2 set to the
R10000 now see 44 bits. The R4400 and R5000 (Indy, Indigo2, O2 with an
R5000) stay at 40.

One test maps a page at 0x123_4000_0000 on the R10000 and the R4400: the
R10000 keeps EntryHi and reads the page, the R4400 masks it and takes
Address Error.
A PROM's cache walk reads and writes the primary caches with Index Load
Data and Index Store Data. Only the secondary cache had a data array, so
an Index Load Data on a primary cache always returned zero and an Index
Store Data was dropped.

Give the primary caches data arrays and mask each store to its
diagnostic field (R10000 UM pp. 262, 271-272): 36 data bits and 1 parity
bit for the instruction cache, 32 and 4 for the data cache, 64 and 10 for
the secondary. An Index Store Data to the secondary cache writes a
quadword, so the other doubleword becomes zero and takes the same check
bits (p. 91). A mapped primary Index Load Data or Index Store Data
still takes TLB Refill and Invalid, so CACHE now translates the operand
for those two; the other index operations still do not.

The secondary data slot is now the address >> 3 where it was >> 4, so
the walk gets one slot per doubleword instead of two doublewords
sharing one. This affects the Octane, the IP28 and an O2 with an R10000.

Tests cover a primary round trip and the quadword store.
Fuel uses two independent SCSI buses behind one ISP12160 function. Reuse
the ISP1040 command engine and let the part trait own the firmware,
register and channel differences.

A bus reset reaches every attached target as unit attention rather than
a media change. Keep the reset attention on the shared target interface
so a controller can notify each device through the same bus. The
ISP12160 is its only caller: the WD33C93A, AIC-7880 and ISP1040 models do
not call it, so their targets behave as before. A bus reset restores each
disk's write cache bit per channel: a reset of one channel leaves the other's
disks alone, and a chip or soft reset restores both.

The engine changes reach the Octane's ISP1040: mailbox commands now run
with dispatch held, and the target interface gains the bus reset. The
Octane's IRIX 6.5.30 SCSI disk (Octane2, R10000, 256 MiB, V12, jitv2,
release build, headless, NAT) boots to the login prompt and runs commands
on this branch's tip and on main c6dea8c: login after 126 s on the
branch and 109 s on main in one run each (host load makes the times
indicative only), the same `hinv` listing of both integral controllers
and the disk, and the same command output.
Fuel fits a DP83843 at MII management address 1, where the Octane has an
ICS1890. The IOC3 Ethernet controller needs the same packet and link
machinery with either PHY, so keep that machinery shared: move the
Octane's ICS1890 register model behind a PHY interface, and supply the
DP83843's register and negotiation behavior through it. The MAC keeps
its existing packet path.

The interface carries the snapshot hooks too: the ICS1890 keeps the
register save and load it had in the IOC3 model, and the DP83843 saves
its registers and negotiation timers the same way, so the shared Ethernet
block saves whichever PHY it holds.
Fuel stores its clock and nonvolatile configuration in an M48T35 on the
IOC3 byte bus. Keep the register overlay and battery-backed bytes
together so latching and persistence share one owner.
Fuel attaches an M48T35 and has different PCI header policy from the
Octane. Carry those choices in the population configuration and wire the
timekeeper once so byte-bus accesses need no mutable attachment slot.

The Fuel's onboard IOC3 also carries the PS/2 keyboard and mouse
controller: OpenBSD configures a keyboard controller through the
onboard IOC3 (pci/ioc.c, dev/iockbc.c), and the controller core is the
one the Octane already has. Turn the controller on in the Fuel's
population, as the Octane's is, and test it by sending a host key through the IOC3
registers: it latches the keyboard interrupt and arrives in the K_RX
register, and a mouse command answers from its own port, both before and
after a chip reset.
Fuel boot flash uses AMD unlock, program and erase commands. Keep that
state machine separate from array storage and bus decode so its command
handling can serve each fitted flash array.

The Intel and AMD command sets need the same rules for seeding and
writing flash images, so share the file storage while each command engine
keeps its own command state.
Fuel PROM 6.210's memory test fails when uncached SB and SH reach the bus
carrying only their low bits. Its L1 console shows:

  mtest_bank_main failed: Walking Address
  RSLT mtest_bank_mai FAIL                diag_rc = 43  Walking Address

In both memory banks the expected halfwords 0xa55a, 0x5aa5, 0x9999 and
0x6666 read back from the ECC check-bit view as 0x0000. The PROM stores
each check word with SH from a register whose low 16 bits are zero; the
word sits higher in the register, at a position that depends on the
tested address. With the whole register visible to the ECC view and the
same PROM configuration it reports ALL DIAGS PASSED and enables all
512 MiB.

Add BusDevice::write_narrow_from, whose default stores the architectural
byte or halfword exactly as write8/write16 do, and call it for every
uncached SB and SH. Ordinary devices therefore still see the low bits of
the register, and only a device that models a path observing the other
lanes overrides it; Bedrock's ECC view does, in its own commit. Which
lanes real hardware drives for a narrow store is undefined. Debug writes
keep the ordinary-width path, and the JIT callouts pass the same source
register. The default adds one virtual call to a path that already makes
one.
I/O (bedrock_io):
Fuel needs Bedrock I/O to route programmed PIO windows to PCI widgets,
check inbound DMA permissions and run its two block-transfer engines.
Fault reporting and interrupt delivery stay with the block that owns the
state, and unsupported accesses are reported.

Register map and behaviour are ported from the QEMU model.

Node routing (bedrock_ni):
Fuel runs as one node with no NUMAlink peer, but the guest still reads
and programs these local control words during machine setup. Retain the
writes and keep the network fabric out of the model.

Crossbar (bedrock_xb):
Record the fabric error queues and let software service them, with the
outputs available to the Bedrock processor interface. Unsupported queue
reads return zero and writes are logged and ignored.

Local bus (bedrock_lb):
Add the node and region registers, atomic scratch aliases, error and
vector status, and the MicroLAN master that board identity parts sit on.
The tests drive it together with the I/O, node-interface and crossbar
blocks it reports through, which land in this same commit.
The Fuel CPU receives interrupts and timing events through Bedrock PI.
Model the fitted processor slot and its node callbacks while keeping
unconnected graphics control explicit.
Fuel reaches its AMD flash through executable PROM aliases and
programming windows. Share the array bytes and persistent store while
preserving the command and erase state of each flash part.
Fuel routes PCI configuration, DMA and interrupt packets through two
XBridge heads. Keep small sparse PCI decode in window lists so the
bridge can share the machine memory map without allocating more Physical
tables.

A board attached to one of the crossbar's external ports raises its
interrupts as crosstalk packets, the same way the PCI heads do. Route
them through the same sender, so the Fuel's graphics board reaches the
Bedrock's processor interface with no path of its own, and say whether
the destination took the packet.

The crossbar answers with the Crossbow's widget 0 register map, so it
takes its offsets from the Octane's Crossbow model instead of restating
them. What stays separate is behavior that differs between the two: the
XBridge crossbar's status register carries the requesting port, wide
and malformed accesses set the access-error status and answer zero, its
interrupts go out as crosstalk packets to any destination, and it
resets the heads and ports its fences select.
The Fuel console first passes through the L1 system controller. Reuse
the shared 16550 core and keep controller framing, power commands and
identity records in its protocol model.
Fuel enumerates a Lucent OHCI controller on PCI. Provide its registers
and descriptor scheduling through the USB function interface while
leaving devices unattached in the machine configuration.

The frame counter follows the machine clock the board hands the
controller, and every register read and write first delivers the frame
boundaries that clock has passed. Advancing it only from the board's
poll, which runs between CPU batches, left HcFmNumber at zero after the
guest's own 1 ms wait had expired, and the Fuel PROM reported that SOF
did not start within 2 ms. The sync holds one lock from reading the
clock to applying the frames, so a timer thread cannot leave a
concurrent reader behind it.

Entering UsbOperational from Reset or Suspend re-anchors the frame:
it loads HcFmRemaining from HcFmInterval, toggle included, and
increments HcFmNumber, with the first SOF one frame later (OpenHCI 1.0a
6.2.1, 7.3.2, 7.3.3). The tests cover that entry with frame numbers at
and around the overflow flag.

Add two low-speed USB functions for the OHCI root ports: a HID boot
keyboard and a HID boot mouse, each with one configuration, one boot
interface and one interrupt-IN endpoint. They answer the standard
requests (descriptors, SET_ADDRESS taking effect after its status
stage, SET_CONFIGURATION, status and interface queries) and the HID
class requests (GET/SET_REPORT, GET/SET_IDLE, GET/SET_PROTOCOL), and
report host keys and pointer moves as boot reports. Reports queue one
per change so a key tapped between two polls still shows both edges.

Written from the USB 1.1 and HID 1.11 specifications (boot descriptors
from appendix B) and the HID Usage Tables, with no code from another
emulator. The wheel, remote wakeup and the host layout translation that
the PS/2 path has are left out. Ohci::attach loses its test-only gate so
a machine can plug the devices in.
Build the Fuel IP34 board around its R14000, Bedrock, XBridge, IOC3 and
dual-channel ISP12160. Use one Physical memory table with sparse
processor and peripheral views, and retain the CPU memory fast paths
after the PROM startup store retires.

Uncached byte and halfword stores reach a device through
BusDevice::write_narrow_from, which each window forwards at the
device-local address; only the memory controller's ECC view reads more
than the low bits of the register.

With the jitv2 feature, start the compile pool after the memory windows
are published, as the Octane and O2 do. Each worker copies the cache
geometry once as it starts, and without the windows that geometry has no
inline load or store path. Workers read code through the cached plane, which
serves plain physical addresses; the uncached views would decode them as
node registers.

The machine describes where its memory can answer to the CPU: the four
1 GiB slots Bedrock's controller places its two 256 MiB banks in, and the
reset PROM. Its disks on both ISP12160 channels are flushed at power-off,
quit and exit like the other machines', so a copy-on-write overlay keeps
its record of held sectors after a guest `halt -p`.

The processor is R14000FuelShadowCache, the Octane's R14000 at revision 2.3
(PRId 0x0f23), the revision a public software inventory of a Fuel prints,
where the Octane's part reports 2.4. The raw CP0 word, the FIR and the
128-byte lines are taken from the Octane part, not read from a Fuel. Like
the Octane's R14000 it reports "R12000" as its name. The machine loads
its PROM through prom_image.

Feed the IOC3 PS/2 controller from the window, VNC and CI input, and let
a USB keyboard on root port 0 and a USB mouse on port 1 share it. The
Fuel's manual shows PS/2 as the standard keyboard and mouse, so
ps2_input under [fuel] defaults on and usb_input defaults off; setting
usb_input fits the USB pair, and ps2_input = false leaves the PS/2 pair
out. The monitor's ps2 command reaches the keyboard and mouse too.

PROM loader (prom_image):
Add a device-independent loader for PROM images. It accepts a raw
payload or a version-six stored or gzip container, decodes the
recognized containers explicitly and reports malformed inputs with their
path before any bytes reach guest memory.

Config profile (config):
The Fuel is a new machine profile with two memory banks, wide SCSI buses,
no Ethernet adapter choice beyond its built-in one, and no graphics board
by default. The [fuel] section selects the serial console (the L1
controller or either IOC3 port), the L1 TCP port and log, and the
persistent boot flash store. l1_revision and brick_serial replace
the L1's reported firmware revision (default 1.48.1) and brick serial
(default EMUN00). ps2_input defaults on and usb_input off, as
the Fuel's manual shows PS/2 as the standard keyboard and mouse; setting
usb_input fits a USB keyboard and mouse, and ps2_input = false leaves the
PS/2 pair out.

The key names the part: machine.cpu = "r14000" (r10000, the family, is
still accepted), and a Fuel config cannot claim another processor. The
Octane's R14000 is revision 2.4; the Fuel's is 2.3.
PCI functions need different writable and write-one-to-clear header
fields. Describe those fields in one construction policy while keeping
the current header defaults for existing functions.

Forward narrow Bridge configuration writes with their byte enables so
write-one-to-clear bits are handled correctly when a function supplies
them. Existing Octane functions do not use that status policy, so their
behavior stays unchanged.
Index Store Tag stored all 64 bits of a secondary-cache tag, so a
walking-1s test over the tag read back bits the R10000 defines as
reserved. The manual says they return zero (R10000 UM p. 270,
figure 11-31).

Keep MRU, the physical tag, state, virtual index and ECC, which
assembled is 0x8000_000f_ffff_cdff, and drop the rest. The mask is
CpuModel::SECONDARY_TAG_MASK, all ones except on the R10000 family. MRU stays
because the PROM writes it as TagHi bit 31 and reads it back.

The later parts share the tag layout (the R12000 manual agrees; the R14000
and R16000 are taken to match), and a PROM's tag test on the R14000 expects
the reserved bits to read as zero. Until the R12000 and later models exist
here, the family is just the R10000.

Tests: an all-ones store on the R10000 and R10000 O2 reads back the
mask, an R4400 control keeps TagLo bits 12 and 13, and the walking-1s
round trip expects the reserved bits as zero.
The Octane shipped with an R12000 or R14000 and the O2 with an R12000,
but `cpu` took only r4400, r5000 and r10000. An Octane hinv could not
show the later parts, and CP0 Config reported 1 MB of secondary cache
for every R10000-family part, so a 2 MB model printed "1 Mbyte".

Add r12000, r14000 and r16000. They share the R10000's programming
model (44-bit virtual, 40-bit physical, the R10000 cache operations)
through model::r10k_family. Config SS is now log2(secondary size /
512 KB). The Octane's R14000 is the R12000's 2 MB type with its PRId
set at run time, as the Octane R10000 already does, so only three new
executors are compiled, not four.

  Octane  r10000, r12000, r14000 (195, 360, 600 MHz by default)
  O2      r5000, r10000, r12000 (400 MHz; 64-byte secondary lines,
          because with 128 the O2 PROM panics before its banner)
  IP28    r10000 only

There are two R16000 parts, r16000 (revision 2.2, 800 MHz, PRId 0x0f42)
and r16000_700 (revision 2.1, 700 MHz, PRId 0x0f41). Both have a 4 MB
secondary cache and present their own Config reset word; the 700 MHz part
is the 800 MHz type with PRId and that word set at run time, as the Octane
R14000 sets its PRId, so it adds no executor. The 800 MHz, 8 MB part
would need a third executor, as the cache size is a type parameter, and is
left out. The R16000 has no machine yet. The Indy and Indigo2 profiles
build any part, and iris-bench uses that for r16000 and r16000-700
cells. The bench harness names an implementation 0x0f CPU an R16000 above
major revision 2, and the prebuilt guest image is refreshed for it.

On the R12000 and later, Index Writeback Invalidate (D) also clears the
line's primary data tag, as Index Load Tag then shows; the R10000 keeps it.

The new parts also keep the upper word on a partial LWR, as the R10000 does:
it is implementation-defined, and keeping it matches what the IRIX CPU
instruction test expects and the qemu-sgi-origami model, so for the R12000
and later it is a model choice.

R10000 hot path: +0.09% host instructions per guest instruction on
iris-bench, inside the ~1% noise (an R4400 control moved +0.86%).
Every Fuel has a VPro graphics slot: a Fuel IRIX inventory lists its
board under xtalk/13 and OpenBSD finds Odyssey at crossbar widget 13.
Build the board there when the machine is given a V12 or a V10, through
the widget-offset constructor, and keep none, the serial machine, as the
default.

The XBridge now reports link 13 alive, the board's DMA enters Bedrock's
inbound path as widget 13, and a link reset of port 13 resets it. Its
interrupt packets cross the crossbar to Bedrock like the PCI heads'. The
status bar's activity (SCSI, IOC3 and Ethernet) and the CPU's Compare
ticks reach its LEDs as on the Octane. IRIX's VPro driver writes each
processor interface's graphics flow-control registers at attach and tears
the board down when the credit counter reads zero, so the interfaces keep
those registers; they count no stores and return no credits, so no
processor interface is linked to the board's credit accounting.

With PROM 6.210 the board passes the graphics diagnostic, and IRIX
6.5.30 with the VPro driver lists Graphics board: V12 and draws the X
login screen on it. Input comes from the PS/2 pair the Fuel already has.
Configuration accepts the board on the Fuel too, see the config note below.

Tests build a Fuel board with each board and check that the processor
reaches the board at widget 13 and finds its link alive, that its
interrupt packets set and clear a processor interface's pending bit, that
a link reset of its port resets it, that a headless Fuel has nothing
there, and that the processor interfaces keep their graphics registers.

Config (config):
A Fuel is reported with both the V12 and the V10, so take either with
[graphics] board; the other boards stay refused, with a message that
names the Fuel's choices, and none, the serial machine, stays the
default. The doc comments for the two boards now say they fit the Fuel as
well as the Octane.

The Fuel's validation case now accepts both boards and refuses the V6.
Document the supported serial configuration, image inputs, the host
input options and the explicit limits for graphics, USB devices and
snapshots, in the feature list, hacking notes, debug notes, the sample
config and a Fuel page. Startup storage remains a PROM-specific
compatibility approximation.
FrameMask (CP0 register 21) is new with the R10000. Bits 15:0 clear
EntryLo bits 33:18 as a TLB write passes them on, and the rest of the
register reads as zero (R10000 UM v2.0 p. 260). iris had no register 21: mfc0
read zero and mtc0 was dropped, so a kernel that sets FrameMask to keep
software flags out of the physical address got no effect.

The core holds the register; the executor sets which bits are writable,
all 16 on the R10000 and none on the R4400 and R5000. TLBWI and TLBWR pass
the masked EntryLo values, and TLBR returns the masked entry. The register
is saved with the rest of CP0 and loads as zero from an older save.
CpuModel::R10K_FAMILY says a part is in the R10000 family; for now that is
the R10000.

Nothing changes while FrameMask is zero, its reset value, and the write
path runs only on a TLB write.

Tests write FrameMask on the R10000 and the R4400, map a page and check the
physical address and the readback; a second writes with TLBWR as well as
TLBWI and reads the entry back with TLBR.
In 64-bit mode a partial LWR (offset 0 to 2) sign-extended the merged
word on every CPU. The manual calls the upper word of a partial LWR
implementation specific (NEC U12754E pp. 125-126), and the reference
results observed for the R10000 keep the old bits 63:32 at all
three partial offsets. With sign extension a test that compares those
results miscompares.

The interpreter and the JIT now merge the old upper word on the R10000; a
full word (offset 3) still sign-extends. The upper word is kept only when
64-bit operations are enabled for the current mode, matching QEMU;
otherwise it sign-extends. The later parts are not covered, since nothing
shows what they do. The choice is a JitConsts field set from
CpuModel::PARTIAL_LWR_KEEPS_UPPER, so compile workers see it. The code-cache
fingerprint gains lwr_upper=1 only when it is set, so the R4400 and
R5000 machines keep their warm caches.

Behaviour changes on the Octane, IP28 and an O2 with the R10000. The
R4400 and R5000 are unchanged.

Tests run LWR at every offset in the interpreter on the R4400, R5000
and R10000 in kernel, supervisor and user mode with the mode's own
64-bit bit set and clear, run the same offsets as compiled code on the
R10000 and R5000 and compare with the interpreter, and check the
fingerprint.
On the R10000 EntryLo holds a 28-bit PFN in bits 33:6, a 40-bit physical
address, and the two-bit uncached attribute in bits 63:62; 61:34 are
reserved (R10000 UM v2.0 pp. 239-240). iris kept the R4400's 30 bits, so the
attribute and PFN bits 33:30 read back as zero and never reached the TLB.

The core now has an EntryLo mask that the executor sets per part: the R4400
and R5000 keep 0x3FFF_FFFF, the R10000 gets 0xC000_0003_FFFF_FFFF. The TLB
carries the PFN up to physical 39:12, but a TLB write clears the PFN bits
below the page size, so TLBR does not return what was written to them
(R10000 UM v2.0 pp. 299-300, TLBWI and TLBWR; NEC VR10000 instruction set
manual U12754EJ1V0UM pp. 199-200). That is documented for the R10000;
for the R12000, R14000 and R16000 no manual describes a TLB write, so they
do the same as a model choice. The R4400 and R5000 keep the whole PFN. The
uncached attribute is stored and read back only; it is not carried to the
bus.

Bits 33:32 are physical 39:38, and the Octane's guest keeps software flags
there. FrameMask (previous commit) clears them on a TLB write, so the page
lands at its real address and the Octane's 40-bit physical mask needs no
narrowing.

On the Octane, the IP28 and an O2 set to the R10000, mfc0 of EntryLo now
returns bits 63:62 and PFN bits 33:30 if the guest wrote them; before they
read as zero. The R4400 and R5000 are unchanged.

A test checks the readback on the R10000 and the R4400, and that a page
with PFN bits 33:32 set maps to physical 39:38 and is brought back by
FrameMask.

The Octane PROM's tlb diagnostic writes 0xa5a5a5a4 and 0x5a5a5a58 to
EntryLo with 16 KB pages and expects 0xa5a5a524 and 0x5a5a5a18 back:
EntryLo bits 7:6 are physical bits 13:12, inside the page. It reported
320 compare errors. A write now clears EntryLo bits PageMask >> 7;
translation already ignored them, so only TLBR changes. A second test
checks the diagnostic's patterns through TLBWI and TLBWR on the R10000 and
the R4400, and that a 16 KB page translates the same on both.
…sses

The R10000 drives a two-bit uncached attribute on the system bus. For an
unmapped xkphys access it is address bits 58:57; for a mapped reference
it comes from the TLB's uncached-attribute field, which EntryLo0/1 bits
63:62 load (R10000 User's Manual 007-2490-001). EntryLo already keeps
those bits on the family, and a TLB entry holds them, but the entry's
physical base dropped them, so a mapped uncached page always reached
attribute zero.

Carry them to physical bits 58:57 only for pages the model's C table
decodes as uncached, so cached mappings are unchanged. The TLB already
holds that table for its shadow entries; reading it there, and
recomputing each entry's base when a table is installed, avoids a second
decode of C.

The attribute reaches the bus only on a board that decodes physical
bits 58:57. The Octane, Indigo2 and O2 mask the physical address to 40,
32 and 32 bits before the bus, which folds it away, and a test shows
that at the executor. The R4400 and R5000 never store the bits.

Tests cover the attribute and the board masks, and the C table (a value
the model decodes as cached carries nothing; a table installed after the
entry was written decides).

The uncached-attribute comments cite NEC U10278EJ4V0UM, the fourth edition
of the VR10000/VR12000 user's manual, section 11.3, by document number;
the manual itself is not added to the repository.
Bedrock memory configuration changes the RAM decode. Keep ordinary CPU
and DMA memory on Physical directly, divert diagnostic pages through MD,
and update their generation state when memory configuration or ECC
changes.

The ECC view overrides write_narrow_from to take an uncached SH's check
word from the whole source register, at the position Fuel PROM 6.210's
memory test uses. Every other view stores the architectural low bits.
@jamesbraid
jamesbraid marked this pull request as draft October 31, 2026 05:28
@jamesbraid

Copy link
Copy Markdown
Collaborator Author

wip, doing some rework and cleanup

@jamesbraid
jamesbraid force-pushed the fuel-ip35 branch 2 times, most recently from 1b77e64 to 687bc0e Compare October 31, 2026 09:49
The IP35 PROM fetches its own code uncached at plain physical 0x1fcxxxxx.
Bedrock serves that from the reset PROM, but the compile pool read through
the cached view, and once a memory bank covers the window the cached view
answers RAM there. The memory test zeroes it, so the JIT compiled pages of
NOPs in place of PROM routines and the PROM faulted (an instruction fetch
bus error, or endless refills). Code pages were keyed by frame alone, so the
two views could not be told apart.

Where the machine says its uncached view differs (the Fuel; other machines
do not opt in and keep one page per frame), an uncached fetch keys
frame | UNCACHED_VIEW_PFN. The page is read and its generation counter taken
through UNCACHED_VIEW_ADDR on a compile bus that sends such addresses to the
uncached attribute-0 view, the device the CPU's own fetch reached. Writes
invalidate by the device that holds the changed bytes: a cached store to RAM
over the reset window leaves the PROM's page alone, a flash command
invalidates it, and where both views reach the same RAM they share its
counter.

The view bit is a constant, bit 22 of the key (the 16 GiB window's frame
count), so the fetch and SMC paths strip it with an immediate mask and
the per-instruction path is untouched. No machine has a frame at or past
it, so the strip is a no-op where the bit is never set: an 8 GB Octane's
frames above 4 GiB, which use bit 20, keep their true addresses. Any
machine could opt in. Opting in grows its frame map from its memory's
frames to the view bit plus those frames (16 MiB plus four bytes a
frame), so both key spaces index it directly; the other machines keep
the map sized to their memory. The Fuel's CHANGELOG entry sits under an
"SGI Fuel" heading whose first bullet arrives with the documentation
commit.

The compile pool restarted after a page-pool flush used the CPU's uncached
bus instead of the cached view it was started with; the executor now keeps
the compile bus and passes it to every restart and to the inline compile.
The monitor uses it too: `j2 inline off`, `dumppcp`, `corpus` and `j2 html`
read pages through it, and `j2 clear` and `j2 deny` take a trailing
`uncached` to name the uncached view's page.

Tests: a pool restart after a flush keeps the compile bus; an executor that
has not opted in keeps the plain key and the frame map sized to its memory;
uncached attributes 1 to 3 and addresses over 4 GiB, even where a device
answers them, get no code page; frames above 4 GiB keep their true
address and bus address without the view, and the uncached view still
names its frame and view for frames above 4 GiB. Both new tests fail
with the view bit at 20.
…unting

The board assumed the Octane: its interrupt packets went only to HEART
at widget 8, and it took the processor's addresses in the IP30 widget
windows. A Fuel's processor reaches widget 13 through Bedrock, which hands
the board widget offsets, and the board's packets cross the XBridge's
crossbar to Bedrock like any widget's.

Add a constructor for such a machine: it takes a packet sender for the
interrupts and decodes every address as an aperture offset. The Octane's
constructor and behaviour are unchanged. A test routes a packet to a
widget other than HEART and requires the sender to get it, not HEART's
line.

Graphics credits (odyssey):
A Bedrock or Hub processor interface counts graphics credits: each
counted store to a graphics head takes one, and the head returns it as
the store leaves its command FIFO. The Octane's HEART has no such block,
so the board left the accounting out. Bring it back for a machine that
links a processor interface: the PiGfx register block and the PiGfxLink
trait (return, owe and repay credits), a counted and an owed mark on each
queued store, the credit loop in retire and clear, the one-store runs
while a counted store waits, and the processor's one-shot store mark.
With no link, as on the Octane, every credit call is a no-op.

How long a V12 takes to return a credit is not published. The host's
credit window is 100 doublewords (IP35) or 256 (IP30), a credit is
inferred to come back when the board takes the doubleword into its FIFO,
and the FIFO must hold more than the window. The model credits a store
as it enters while fewer than 16,384 stores wait (cfifo::DEPTH). The
initialise call stores its start-up program, about 12,291 words, with no
level check before the low-water interrupt is routed; with a depth of 512
two boots in five timed out with "CFIFO timeout" and no screens, and
16,384 is the smallest power of two that covers the program even at one
word per store. The depth is a model choice derived from that program. A
deeper store owes its credit and hurries the board thread; a reset
refunds what the abandoned stores owed. Inside the depth the board
thread takes full runs, so a credited backlog does not drain one store at
a time.

The board thread reads the count of counted stores once per run taken and
once per run retired, on every machine. Both reads are Relaxed: a run's
length is only a hint, a stale count costs a longer run and never a lost
credit, and the retire reads it under the lock a counted push takes to
change it. Nothing else on the Octane's store path changed.

The tests drive the credit rules with a stand-in link: a counted store is
credited on entry, a reset cancels owed credits, the mark counts exactly
one store, a widget that runs out of credits starves, the FIFO set-up
words do not widen the credit depth, and an owed credit hurries the
board thread. Others fill the largest host window plus one with the board
taking nothing and require every credit back with none owed, and post
16,384 counted stores and require each credited at entry and the next one
owed.

The refused-push test used to sleep 150 ms and require a push thread to
be unfinished, which a loaded host could break once the board thread's
timed hold ran out first. The hold now lasts until the test releases it,
and the test waits for the push to be refused before it asserts.
A mapped page whose physical address has a bit set above bit 31 could
reach the PCI window instead of RAM on the O2. The O2 aliases RAM at
0x8000_0000, and fold_phys tested that window on the full translated
address before cutting it to the 32 bits the board decodes. IRIX keeps
software bits in its page-table entries just above the PFN, so such an
address missed the window, then lost the high bit in the mask and landed
on the PCI range at 0x8000_0000.

Cut to the decoded bits first, then apply the alias, then cut its target.
The window lies within the decoded bits, so a page already inside it
lands where it did. The extra AND runs only on an alias hit.

Only the O2 has a physical alias; the Indy, Indigo2 and Octane are
unchanged. A test maps a page at 0x1_8000_0000 with the O2's mask and
window and expects RAM at 0. It reads 0x8000_0000 without the fix.
@jamesbraid
jamesbraid force-pushed the fuel-ip35 branch 11 times, most recently from b642a58 to 24e0013 Compare November 2, 2026 01:45
The R10000 picks the TLB refill vector from the address space the
missing address lies in and that space's Status bit: user addresses
follow UX, xsseg SX, and kseg3, sseg/ksseg and xkseg KX. The current
mode only decides which space an address is in (R10000 UM v2.0 pp.
333-334, 342). The manual contrasts this with the R4400, which picks by
the current mode's bit. iris did the R4400 thing on every model.

So a 64-bit kernel (KX=1, UX=0) missing on a kuseg address went to the
XTLB vector at 0x080 on the R10000, where the part uses 0x000. The
Octane PROM's utlb diagnostic loads VA 0 in exactly that state and
waits at the 32-bit vector. It got "Read TLB miss exception" from the
XTLB vector instead and failed.

On the R10000 family a refill miss now reads the region of the address,
sign-extended from 32 bits in 32-bit mode, and tests UX, SX or KX. Only
the miss path changes. The TLB compare width still follows the mode.

The test misses in each space with each bit on the R10000, and checks
the R4400 keeps choosing by mode.
@jamesbraid
jamesbraid force-pushed the fuel-ip35 branch 3 times, most recently from 2604eef to 9225aaa Compare November 2, 2026 05:22
The R10000 and R12000 do not shut down on a multiple match. A TLB write
that would let one lookup match two entries first invalidates every
entry it overlaps, the wired ones included, then writes the new entry
and sets Status.TS (NEC U10278EJ4V0UM, Status register TS bit p.223 and
"Avoiding TLB Conflict" p.300). iris stored the entry and left the
duplicates matchable, so which one answered depended on how the lookup
map happened to be rebuilt. On the two-processor Tezro the firmware
flushes the TLB with 64 invalid entries on the kernel's page number,
and the kernel then writes the valid global 16 MiB entry over them;
a later write elsewhere left an invalid duplicate answering, and the
fetch took TLB Invalid.

The R14000 and R16000 get the same rule as a model choice, matching
QEMU's r10k_invalidate_conflicts(); no manual for them was read. The
R4400 and R5000 keep their behaviour.

An overlap is two entries a lookup could match together: same region
and page number under the wider of the two page masks, and equal ASIDs
or either entry global. Invalid entries count. The removed entry reads
back as an unwritten slot except for its PageMask, as in QEMU: the
kernel TLBRs its wired sentinel entries and reuses PageMask for the next
write, and a zeroed mask made it write a per-CPU mapping as 4 KiB.

The cost is confined to TLB writes on these models: a compare against
the (at most 63) entries written since power-on. The hit path, the vmap
fast path and the miss path are unchanged, and the rule is off on every
other model. TLBWI and TLBWR are never compiled by jitv2, so both reach
the same write and set TS in the interpreter.

Tests cover the Tezro sequence, wired entries, PageMask kept on a
read-back, entries that cannot match together, and TS from TLBWI and
TLBWR. They fail with the rule off, and the R4400 and R5000 test fails
if the rule is enabled for them.
A store that nothing answers raised a synchronous Data Bus Error with a
precise PC on every CPU. The R10000 cannot do that. It sends an uncached
store as a write request, and a write request gets no response; Bus Error
comes only from an error response to a read or upgrade request, or from
uncorrectable read data (R10000 UM pp. 104, 316). The processor never
learns that a store failed.

The R12000 manual documents the same no-response write path, and the later
parts are taken to match, so the test is the R10000 family and not the R10000
alone.

Add CpuModel::POSTS_UNCACHED_STORES, true for the R10000 family's caches. On such a
model a refused store logs and completes in write_data_impl and in the
masked-store path (SWL, SWR, SDL, SDR), which the interpreter and the
compiled code both go through. An uncached SC or SCD that is refused
completes and succeeds (rt = 1); that is a model choice, since the manual
says nothing about uncached store-conditional. Loads, instruction fetches,
cached accesses and the R4400 and R5000 keep the exception, and device
return values are untouched because BUS_ERR also tells non-CPU callers
that a request was declined.

The error goes no further. Nothing in iris raises an interrupt or sets an
error register for a store the Octane's bus refuses. The IP22/IP28 bus
already completes a write to an unanswered address, latching MC
CPU_ERROR/GIO_ERROR and raising no interrupt, so the stores that change
here are the ones a device refuses itself.

The log line stays visible ("Unsupported uncached write ... posted,
completes") but a probe loop must not flood stderr: it prints the first
four refused stores and then each power of two, with the running count.

The Octane's ide diagnostics store 0x591204 and then 0x4 to physical
0x1e130400, where nothing answers. Before, the first store took a Data Bus
Error; now both print one line and the run carries on.

Tests: the interpreter on the R10000 (full and shadow caches) and the
R5000 for SW, SWL and SC plus the load that still faults; the JIT against
the interpreter for SW and SWL on the R10000 and for the load and the
R5000 store that fault; the report schedule. cargo test --release
--features jitv2 --lib: 2422 passed, 0 failed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant