Skip to content

tezro: add the SGI Tezro, with the V12 and up to four processors - #409

Draft
jamesbraid wants to merge 117 commits into
techomancer:mainfrom
jamesbraid:tezro
Draft

jamesbraid wants to merge 117 commits into
techomancer:mainfrom
jamesbraid:tezro

Conversation

@jamesbraid

@jamesbraid jamesbraid commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Iris runs the Octane and now the Fuel, but not the Fuel's big brother, the Tezro. I wanted IRIX running on one, so this adds it.

It comes from my QEMU work. jamesbraid/qemu-sgi-origami already runs the Tezro, including four processors, and the hardware notes and tools behind it are in jamesbraid/origami. This is a port of that model into iris, not a new design, and each ported file says which QEMU file it comes from. The Tezro shares the Fuel's Bedrock node, so most of it reuses the Fuel's code. It has one to four R16000s, 4 GiB of memory by default, the IO9 board's devices (serial, clock, PS/2, SCSI, gigabit Ethernet), and a V12 on the VPro code the Octane and Fuel already use. IRIX 6.5.30 boots to the desktop and runs OpenGL.

This sits on the Fuel (#368) and on the R10000 and R16000 CPU changes. Until those land the branch carries their commits too. The Tezro's own commits are the ones prefixed tezro:, plus the ip35: and Bedrock ones for the processors.

With cpus = 2 or 4 the real PROM and kernel start the other processors, the same way the Octane's two-CPU port does. Nothing is faked. With four, sar shows every CPU busy under a parallel load. One CPU behaves as before.

Two changes sit outside the Tezro's own files. Each is its own commit.

  • Store-conditional is now an atomic compare-exchange for RAM above 4 GiB. The Octane's version only covered the direct window.
  • The Bedrock PI can say which processor slots are running.

01-hinv 02-gltest 04-quake-over-hinv

Things to look at:

  • The PROM assumes every processor leaves reset on the same clock edge. Our host threads don't. Four waits cover for that. They're ported from QEMU, each is labelled an approximation, and each has a test that fails if you take the wait out. QEMU ended a wait after 2 seconds of its virtual clock, which keeps running while a host thread is stalled. Freezing one QEMU vCPU thread for 2.5 seconds made the PROM find three processors or restart, so QEMU had the same flaw. Here a wait ends when the processors it waits for have run 2 billion cycles, so a starved thread can't run it out. QEMU got the matching fix, counting the awaited threads' own CPU time. A 120 second host-time limit stays as a deadlock guard. The Octane's two-CPU port needs none of this: its PROM and IRIX start the second processor with explicit HEART handshakes, so no step depends on the two running in lockstep. The IP35 PROM's joins do.
  • A four-processor Tezro runs one host thread per processor plus the device threads. With the default features an idle guest doesn't park them: each processor thread uses a full host core at the desktop, 403% in total at four processors. The Fuel and the Octane do the same at the login screen. Parking is behind the idle-pause feature, which only dist builds turn on. With it the Tezro's idle loop is recognised and an idle four-processor desktop uses about 16% of one core in total. I changed nothing there.
  • The two-processor board needs two flash records, or the PROM power-cycles on the empty slots. It's seeded the way QEMU's own boot tests seed it.
  • The CPU is your choice. r16000_700 is the default: 700 MHz, revision 2.1. r16000 is 800 MHz, revision 2.2, and four processors only, as in QEMU. The 800 MHz 8 MB part is refused, because the cache size is fixed at compile time. FIR 0x900 is a model choice; nobody knows the real value for any R16000.
  • The JIT refuses memory above 4 GiB with a message, because its uncached code views can't address it. It used to panic. The default Tezro layout reaches 7.5 GiB, so with the JIT set banks = [1024, 1024, 1024, 1024].
  • It carries the R10000 cache-index commits, because the PROM stops without them. They go when they land separately.

Tested at the final head:

  • IRIX boots to the desktop at 1, 2 and 4 processors on the interpreter and at 2 and 4 with the JIT, and hinv shows the right count. X answers after login, and SYSLOG and stderr have no TLB or panic lines.

  • The OpenGL tests match the Octane exactly, re-run at the final head. pixtest is 323 passed, 57 failed, glstate is 410 and 5, and the failing tests are the same ones. blendtest, seltest, toktest, the X tests, ivview, perfly and Quake run. Quake's colour shift is its own damage flash.

  • olvst, olperi, oldisk and olmem pass. olmem takes about two hours on the interpreter.

  • cargo test --release --lib -- --test-threads=1: 2973 pass, and 3483 with --features jitv2. cargo check --bins --tests passes at every commit, with and without jitv2.

  • Benchmarks, the way the Octane port measured them, on one lightly loaded host (load 21 to 57):

    Build CPUs Dhrystone DMIPS Whetstone loops/s compute 40M, s
    interpreter 1 40.3 741 6.40
    interpreter 2 42.9 803 6.06
    interpreter 4 44.0 837 5.59
    JIT 1 285 4626 0.19
    JIT 2 307 4993 0.17
    JIT 4 306 5046 0.17

    Running 2 copies on 2 processors takes 1.04 times as long as one, and 4 on 4 takes 1.02 times, on the interpreter. With the JIT, 4 copies take 1.5 times as long, which is start-up skew on 0.17 second jobs; each copy still takes 0.17 s. The JIT's first Dhrystone pass is slower, 8 to 9 s against 2, which I think is translation warm-up but didn't measure. One processor on this branch against the uniprocessor branch showed no difference within the noise, measured on an earlier head.

  • Screenshots, above: hinv with four processors, gltest, and Quake beside the four-processor hinv.

One four-processor boot in about 19 found only three processors, under a very loaded host. I couldn't reproduce it again or find its cause. A processor starved for 2 seconds drops out of the PROM's count, which the wait change above fixes, and on the final head 4 clean boots and 6 injected stalls of 2.5 and 10 seconds at three slots all find four. I can't show that this was the original failure.

Not run: macOS and Windows.

EntryHi, the TLB match and the xuseg bounds check all stopped at VA bit 39,
but the R10000 family implements 44 bits: VPN2 is 43:13 and the user segment
is 16 TiB (R10000 UM pp. 219, 288). A 64-bit kernel can put an n64 user
stack above 40 bits (0x0fff_ffff_aee4, for one). EntryHi dropped VA[43:40]
and such an address took an Address Error.

The width now comes from the part's VA_BITS in each place: the EntryHi write
mask, the TLB's 64-bit compare (Tlb::set_va_bits, installed by the executor
beside set_cache_attrs), the xuseg bounds check, the micro-TLB's cacheable
segment test and the EntryHi value a TLB exception loads. The mask was
spelled three times and is now one const fn, mips_tlb::vpn2_mask. All of it
is a const of the CPU model or set once at construction; the TLB lookup is
unchanged.

This reverses a comment in the executor (b7d2a18) that said widening the EntryHi mask
breaks the IP28 and Octane kernels. Those two machines and an O2 set to the
R10000 now see 44 bits. The R4400 and R5000 (Indy, Indigo2, O2 with an
R5000) stay at 40.

One test maps a page at 0x123_4000_0000 on the R10000 and the R4400: the
R10000 keeps EntryHi and reads the page, the R4400 masks it and takes
Address Error.
A PROM's cache walk reads and writes the primary caches with Index Load
Data and Index Store Data. Only the secondary cache had a data array, so
an Index Load Data on a primary cache always returned zero and an Index
Store Data was dropped.

Give the primary caches data arrays and mask each store to its
diagnostic field (R10000 UM pp. 262, 271-272): 36 data bits and 1 parity
bit for the instruction cache, 32 and 4 for the data cache, 64 and 10 for
the secondary. An Index Store Data to the secondary cache writes a
quadword, so the other doubleword becomes zero and takes the same check
bits (p. 91). A mapped primary Index Load Data or Index Store Data
still takes TLB Refill and Invalid, so CACHE now translates the operand
for those two; the other index operations still do not.

The secondary data slot is now the address >> 3 where it was >> 4, so
the walk gets one slot per doubleword instead of two doublewords
sharing one. This affects the Octane, the IP28 and an O2 with an R10000.

Tests cover a primary round trip and the quadword store.
Fuel uses two independent SCSI buses behind one ISP12160 function. Reuse
the ISP1040 command engine and let the part trait own the firmware,
register and channel differences.

A bus reset reaches every attached target as unit attention rather than
a media change. Keep the reset attention on the shared target interface
so a controller can notify each device through the same bus. The
ISP12160 is its only caller: the WD33C93A, AIC-7880 and ISP1040 models do
not call it, so their targets behave as before. A bus reset restores each
disk's write cache bit per channel: a reset of one channel leaves the other's
disks alone, and a chip or soft reset restores both.

The engine changes reach the Octane's ISP1040: mailbox commands now run
with dispatch held, and the target interface gains the bus reset. The
Octane's IRIX 6.5.30 SCSI disk (Octane2, R10000, 256 MiB, V12, jitv2,
release build, headless, NAT) boots to the login prompt and runs commands
on this branch's tip and on main c6dea8c: login after 126 s on the
branch and 109 s on main in one run each (host load makes the times
indicative only), the same `hinv` listing of both integral controllers
and the disk, and the same command output.
Fuel fits a DP83843 at MII management address 1, where the Octane has an
ICS1890. The IOC3 Ethernet controller needs the same packet and link
machinery with either PHY, so keep that machinery shared: move the
Octane's ICS1890 register model behind a PHY interface, and supply the
DP83843's register and negotiation behavior through it. The MAC keeps
its existing packet path.

The interface carries the snapshot hooks too: the ICS1890 keeps the
register save and load it had in the IOC3 model, and the DP83843 saves
its registers and negotiation timers the same way, so the shared Ethernet
block saves whichever PHY it holds.
Fuel stores its clock and nonvolatile configuration in an M48T35 on the
IOC3 byte bus. Keep the register overlay and battery-backed bytes
together so latching and persistence share one owner.
Fuel attaches an M48T35 and has different PCI header policy from the
Octane. Carry those choices in the population configuration and wire the
timekeeper once so byte-bus accesses need no mutable attachment slot.

The Fuel's onboard IOC3 also carries the PS/2 keyboard and mouse
controller: OpenBSD configures a keyboard controller through the
onboard IOC3 (pci/ioc.c, dev/iockbc.c), and the controller core is the
one the Octane already has. Turn the controller on in the Fuel's
population, as the Octane's is, and test it by sending a host key through the IOC3
registers: it latches the keyboard interrupt and arrives in the K_RX
register, and a mouse command answers from its own port, both before and
after a chip reset.
Fuel boot flash uses AMD unlock, program and erase commands. Keep that
state machine separate from array storage and bus decode so its command
handling can serve each fitted flash array.

The Intel and AMD command sets need the same rules for seeding and
writing flash images, so share the file storage while each command engine
keeps its own command state.
Fuel PROM 6.210's memory test fails when uncached SB and SH reach the bus
carrying only their low bits. Its L1 console shows:

  mtest_bank_main failed: Walking Address
  RSLT mtest_bank_mai FAIL                diag_rc = 43  Walking Address

In both memory banks the expected halfwords 0xa55a, 0x5aa5, 0x9999 and
0x6666 read back from the ECC check-bit view as 0x0000. The PROM stores
each check word with SH from a register whose low 16 bits are zero; the
word sits higher in the register, at a position that depends on the
tested address. With the whole register visible to the ECC view and the
same PROM configuration it reports ALL DIAGS PASSED and enables all
512 MiB.

Add BusDevice::write_narrow_from, whose default stores the architectural
byte or halfword exactly as write8/write16 do, and call it for every
uncached SB and SH. Ordinary devices therefore still see the low bits of
the register, and only a device that models a path observing the other
lanes overrides it; Bedrock's ECC view does, in its own commit. Which
lanes real hardware drives for a narrow store is undefined. Debug writes
keep the ordinary-width path, and the JIT callouts pass the same source
register. The default adds one virtual call to a path that already makes
one.
I/O (bedrock_io):
Fuel needs Bedrock I/O to route programmed PIO windows to PCI widgets,
check inbound DMA permissions and run its two block-transfer engines.
Fault reporting and interrupt delivery stay with the block that owns the
state, and unsupported accesses are reported.

Register map and behaviour are ported from the QEMU model.

Node routing (bedrock_ni):
Fuel runs as one node with no NUMAlink peer, but the guest still reads
and programs these local control words during machine setup. Retain the
writes and keep the network fabric out of the model.

Crossbar (bedrock_xb):
Record the fabric error queues and let software service them, with the
outputs available to the Bedrock processor interface. Unsupported queue
reads return zero and writes are logged and ignored.

Local bus (bedrock_lb):
Add the node and region registers, atomic scratch aliases, error and
vector status, and the MicroLAN master that board identity parts sit on.
The tests drive it together with the I/O, node-interface and crossbar
blocks it reports through, which land in this same commit.
The Fuel CPU receives interrupts and timing events through Bedrock PI.
Model the fitted processor slot and its node callbacks while keeping
unconnected graphics control explicit.
Fuel reaches its AMD flash through executable PROM aliases and
programming windows. Share the array bytes and persistent store while
preserving the command and erase state of each flash part.
Fuel routes PCI configuration, DMA and interrupt packets through two
XBridge heads. Keep small sparse PCI decode in window lists so the
bridge can share the machine memory map without allocating more Physical
tables.

A board attached to one of the crossbar's external ports raises its
interrupts as crosstalk packets, the same way the PCI heads do. Route
them through the same sender, so the Fuel's graphics board reaches the
Bedrock's processor interface with no path of its own, and say whether
the destination took the packet.

The crossbar answers with the Crossbow's widget 0 register map, so it
takes its offsets from the Octane's Crossbow model instead of restating
them. What stays separate is behavior that differs between the two: the
XBridge crossbar's status register carries the requesting port, wide
and malformed accesses set the access-error status and answer zero, its
interrupts go out as crosstalk packets to any destination, and it
resets the heads and ports its fences select.
The Fuel console first passes through the L1 system controller. Reuse
the shared 16550 core and keep controller framing, power commands and
identity records in its protocol model.
Fuel enumerates a Lucent OHCI controller on PCI. Provide its registers
and descriptor scheduling through the USB function interface while
leaving devices unattached in the machine configuration.

The frame counter follows the machine clock the board hands the
controller, and every register read and write first delivers the frame
boundaries that clock has passed. Advancing it only from the board's
poll, which runs between CPU batches, left HcFmNumber at zero after the
guest's own 1 ms wait had expired, and the Fuel PROM reported that SOF
did not start within 2 ms. The sync holds one lock from reading the
clock to applying the frames, so a timer thread cannot leave a
concurrent reader behind it.

Entering UsbOperational from Reset or Suspend re-anchors the frame:
it loads HcFmRemaining from HcFmInterval, toggle included, and
increments HcFmNumber, with the first SOF one frame later (OpenHCI 1.0a
6.2.1, 7.3.2, 7.3.3). The tests cover that entry with frame numbers at
and around the overflow flag.

Add two low-speed USB functions for the OHCI root ports: a HID boot
keyboard and a HID boot mouse, each with one configuration, one boot
interface and one interrupt-IN endpoint. They answer the standard
requests (descriptors, SET_ADDRESS taking effect after its status
stage, SET_CONFIGURATION, status and interface queries) and the HID
class requests (GET/SET_REPORT, GET/SET_IDLE, GET/SET_PROTOCOL), and
report host keys and pointer moves as boot reports. Reports queue one
per change so a key tapped between two polls still shows both edges.

Written from the USB 1.1 and HID 1.11 specifications (boot descriptors
from appendix B) and the HID Usage Tables, with no code from another
emulator. The wheel, remote wakeup and the host layout translation that
the PS/2 path has are left out. Ohci::attach loses its test-only gate so
a machine can plug the devices in.
Build the Fuel IP34 board around its R14000, Bedrock, XBridge, IOC3 and
dual-channel ISP12160. Use one Physical memory table with sparse
processor and peripheral views, and retain the CPU memory fast paths
after the PROM startup store retires.

Uncached byte and halfword stores reach a device through
BusDevice::write_narrow_from, which each window forwards at the
device-local address; only the memory controller's ECC view reads more
than the low bits of the register.

With the jitv2 feature, start the compile pool after the memory windows
are published, as the Octane and O2 do. Each worker copies the cache
geometry once as it starts, and without the windows that geometry has no
inline load or store path. Workers read code through the cached plane, which
serves plain physical addresses; the uncached views would decode them as
node registers.

The machine describes where its memory can answer to the CPU: the four
1 GiB slots Bedrock's controller places its two 256 MiB banks in, and the
reset PROM. Its disks on both ISP12160 channels are flushed at power-off,
quit and exit like the other machines', so a copy-on-write overlay keeps
its record of held sectors after a guest `halt -p`.

The processor is R14000FuelShadowCache, the Octane's R14000 at revision 2.3
(PRId 0x0f23), the revision a public software inventory of a Fuel prints,
where the Octane's part reports 2.4. The raw CP0 word, the FIR and the
128-byte lines are taken from the Octane part, not read from a Fuel. Like
the Octane's R14000 it reports "R12000" as its name. The machine loads
its PROM through prom_image.

Feed the IOC3 PS/2 controller from the window, VNC and CI input, and let
a USB keyboard on root port 0 and a USB mouse on port 1 share it. The
Fuel's manual shows PS/2 as the standard keyboard and mouse, so
ps2_input under [fuel] defaults on and usb_input defaults off; setting
usb_input fits the USB pair, and ps2_input = false leaves the PS/2 pair
out. The monitor's ps2 command reaches the keyboard and mouse too.

PROM loader (prom_image):
Add a device-independent loader for PROM images. It accepts a raw
payload or a version-six stored or gzip container, decodes the
recognized containers explicitly and reports malformed inputs with their
path before any bytes reach guest memory.

Config profile (config):
The Fuel is a new machine profile with two memory banks, wide SCSI buses,
no Ethernet adapter choice beyond its built-in one, and no graphics board
by default. The [fuel] section selects the serial console (the L1
controller or either IOC3 port), the L1 TCP port and log, and the
persistent boot flash store. l1_revision and brick_serial replace
the L1's reported firmware revision (default 1.48.1) and brick serial
(default EMUN00). ps2_input defaults on and usb_input off, as
the Fuel's manual shows PS/2 as the standard keyboard and mouse; setting
usb_input fits a USB keyboard and mouse, and ps2_input = false leaves the
PS/2 pair out.

The key names the part: machine.cpu = "r14000" (r10000, the family, is
still accepted), and a Fuel config cannot claim another processor. The
Octane's R14000 is revision 2.4; the Fuel's is 2.3.
PCI functions need different writable and write-one-to-clear header
fields. Describe those fields in one construction policy while keeping
the current header defaults for existing functions.

Forward narrow Bridge configuration writes with their byte enables so
write-one-to-clear bits are handled correctly when a function supplies
them. Existing Octane functions do not use that status policy, so their
behavior stays unchanged.
Index Store Tag stored all 64 bits of a secondary-cache tag, so a
walking-1s test over the tag read back bits the R10000 defines as
reserved. The manual says they return zero (R10000 UM p. 270,
figure 11-31).

Keep MRU, the physical tag, state, virtual index and ECC, which
assembled is 0x8000_000f_ffff_cdff, and drop the rest. The mask is
CpuModel::SECONDARY_TAG_MASK, all ones except on the R10000 family. MRU stays
because the PROM writes it as TagHi bit 31 and reads it back.

The later parts share the tag layout (the R12000 manual agrees; the R14000
and R16000 are taken to match), and a PROM's tag test on the R14000 expects
the reserved bits to read as zero. Until the R12000 and later models exist
here, the family is just the R10000.

Tests: an all-ones store on the R10000 and R10000 O2 reads back the
mask, an R4400 control keeps TagLo bits 12 and 13, and the walking-1s
round trip expects the reserved bits as zero.
The Octane shipped with an R12000 or R14000 and the O2 with an R12000,
but `cpu` took only r4400, r5000 and r10000. An Octane hinv could not
show the later parts, and CP0 Config reported 1 MB of secondary cache
for every R10000-family part, so a 2 MB model printed "1 Mbyte".

Add r12000, r14000 and r16000. They share the R10000's programming
model (44-bit virtual, 40-bit physical, the R10000 cache operations)
through model::r10k_family. Config SS is now log2(secondary size /
512 KB). The Octane's R14000 is the R12000's 2 MB type with its PRId
set at run time, as the Octane R10000 already does, so only three new
executors are compiled, not four.

  Octane  r10000, r12000, r14000 (195, 360, 600 MHz by default)
  O2      r5000, r10000, r12000 (400 MHz; 64-byte secondary lines,
          because with 128 the O2 PROM panics before its banner)
  IP28    r10000 only

There are two R16000 parts, r16000 (revision 2.2, 800 MHz, PRId 0x0f42)
and r16000_700 (revision 2.1, 700 MHz, PRId 0x0f41). Both have a 4 MB
secondary cache and present their own Config reset word; the 700 MHz part
is the 800 MHz type with PRId and that word set at run time, as the Octane
R14000 sets its PRId, so it adds no executor. The 800 MHz, 8 MB part
would need a third executor, as the cache size is a type parameter, and is
left out. The R16000 has no machine yet. The Indy and Indigo2 profiles
build any part, and iris-bench uses that for r16000 and r16000-700
cells. The bench harness names an implementation 0x0f CPU an R16000 above
major revision 2, and the prebuilt guest image is refreshed for it.

On the R12000 and later, Index Writeback Invalidate (D) also clears the
line's primary data tag, as Index Load Tag then shows; the R10000 keeps it.

The new parts also keep the upper word on a partial LWR, as the R10000 does:
it is implementation-defined, and keeping it matches what the IRIX CPU
instruction test expects and the qemu-sgi-origami model, so for the R12000
and later it is a model choice.

R10000 hot path: +0.09% host instructions per guest instruction on
iris-bench, inside the ~1% noise (an R4400 control moved +0.86%).
Every Fuel has a VPro graphics slot: a Fuel IRIX inventory lists its
board under xtalk/13 and OpenBSD finds Odyssey at crossbar widget 13.
Build the board there when the machine is given a V12 or a V10, through
the widget-offset constructor, and keep none, the serial machine, as the
default.

The XBridge now reports link 13 alive, the board's DMA enters Bedrock's
inbound path as widget 13, and a link reset of port 13 resets it. Its
interrupt packets cross the crossbar to Bedrock like the PCI heads'. The
status bar's activity (SCSI, IOC3 and Ethernet) and the CPU's Compare
ticks reach its LEDs as on the Octane. IRIX's VPro driver writes each
processor interface's graphics flow-control registers at attach and tears
the board down when the credit counter reads zero, so the interfaces keep
those registers; they count no stores and return no credits, so no
processor interface is linked to the board's credit accounting.

With PROM 6.210 the board passes the graphics diagnostic, and IRIX
6.5.30 with the VPro driver lists Graphics board: V12 and draws the X
login screen on it. Input comes from the PS/2 pair the Fuel already has.
Configuration accepts the board on the Fuel too, see the config note below.

Tests build a Fuel board with each board and check that the processor
reaches the board at widget 13 and finds its link alive, that its
interrupt packets set and clear a processor interface's pending bit, that
a link reset of its port resets it, that a headless Fuel has nothing
there, and that the processor interfaces keep their graphics registers.

Config (config):
A Fuel is reported with both the V12 and the V10, so take either with
[graphics] board; the other boards stay refused, with a message that
names the Fuel's choices, and none, the serial machine, stays the
default. The doc comments for the two boards now say they fit the Fuel as
well as the Octane.

The Fuel's validation case now accepts both boards and refuses the V6.
Document the supported serial configuration, image inputs, the host
input options and the explicit limits for graphics, USB devices and
snapshots, in the feature list, hacking notes, debug notes, the sample
config and a Fuel page. Startup storage remains a PROM-specific
compatibility approximation.
FrameMask (CP0 register 21) is new with the R10000. Bits 15:0 clear
EntryLo bits 33:18 as a TLB write passes them on, and the rest of the
register reads as zero (R10000 UM v2.0 p. 260). iris had no register 21: mfc0
read zero and mtc0 was dropped, so a kernel that sets FrameMask to keep
software flags out of the physical address got no effect.

The core holds the register; the executor sets which bits are writable,
all 16 on the R10000 and none on the R4400 and R5000. TLBWI and TLBWR pass
the masked EntryLo values, and TLBR returns the masked entry. The register
is saved with the rest of CP0 and loads as zero from an older save.
CpuModel::R10K_FAMILY says a part is in the R10000 family; for now that is
the R10000.

Nothing changes while FrameMask is zero, its reset value, and the write
path runs only on a TLB write.

Tests write FrameMask on the R10000 and the R4400, map a page and check the
physical address and the readback; a second writes with TLBWR as well as
TLBWI and reads the entry back with TLBR.
In 64-bit mode a partial LWR (offset 0 to 2) sign-extended the merged
word on every CPU. The manual calls the upper word of a partial LWR
implementation specific (NEC U12754E pp. 125-126), and the reference
results observed for the R10000 keep the old bits 63:32 at all
three partial offsets. With sign extension a test that compares those
results miscompares.

The interpreter and the JIT now merge the old upper word on the R10000; a
full word (offset 3) still sign-extends. The upper word is kept only when
64-bit operations are enabled for the current mode, matching QEMU;
otherwise it sign-extends. The later parts are not covered, since nothing
shows what they do. The choice is a JitConsts field set from
CpuModel::PARTIAL_LWR_KEEPS_UPPER, so compile workers see it. The code-cache
fingerprint gains lwr_upper=1 only when it is set, so the R4400 and
R5000 machines keep their warm caches.

Behaviour changes on the Octane, IP28 and an O2 with the R10000. The
R4400 and R5000 are unchanged.

Tests run LWR at every offset in the interpreter on the R4400, R5000
and R10000 in kernel, supervisor and user mode with the mode's own
64-bit bit set and clear, run the same offsets as compiled code on the
R10000 and R5000 and compare with the interpreter, and check the
fingerprint.
On the R10000 EntryLo holds a 28-bit PFN in bits 33:6, a 40-bit physical
address, and the two-bit uncached attribute in bits 63:62; 61:34 are
reserved (R10000 UM v2.0 pp. 239-240). iris kept the R4400's 30 bits, so the
attribute and PFN bits 33:30 read back as zero and never reached the TLB.

The core now has an EntryLo mask that the executor sets per part: the R4400
and R5000 keep 0x3FFF_FFFF, the R10000 gets 0xC000_0003_FFFF_FFFF. The TLB
carries the PFN up to physical 39:12, but a TLB write clears the PFN bits
below the page size, so TLBR does not return what was written to them
(R10000 UM v2.0 pp. 299-300, TLBWI and TLBWR; NEC VR10000 instruction set
manual U12754EJ1V0UM pp. 199-200). That is documented for the R10000;
for the R12000, R14000 and R16000 no manual describes a TLB write, so they
do the same as a model choice. The R4400 and R5000 keep the whole PFN. The
uncached attribute is stored and read back only; it is not carried to the
bus.

Bits 33:32 are physical 39:38, and the Octane's guest keeps software flags
there. FrameMask (previous commit) clears them on a TLB write, so the page
lands at its real address and the Octane's 40-bit physical mask needs no
narrowing.

On the Octane, the IP28 and an O2 set to the R10000, mfc0 of EntryLo now
returns bits 63:62 and PFN bits 33:30 if the guest wrote them; before they
read as zero. The R4400 and R5000 are unchanged.

A test checks the readback on the R10000 and the R4400, and that a page
with PFN bits 33:32 set maps to physical 39:38 and is brought back by
FrameMask.

The Octane PROM's tlb diagnostic writes 0xa5a5a5a4 and 0x5a5a5a58 to
EntryLo with 16 KB pages and expects 0xa5a5a524 and 0x5a5a5a18 back:
EntryLo bits 7:6 are physical bits 13:12, inside the page. It reported
320 compare errors. A write now clears EntryLo bits PageMask >> 7;
translation already ignored them, so only TLBR changes. A second test
checks the diagnostic's patterns through TLBWI and TLBWR on the R10000 and
the R4400, and that a 16 KB page translates the same on both.
…sses

The R10000 drives a two-bit uncached attribute on the system bus. For an
unmapped xkphys access it is address bits 58:57; for a mapped reference
it comes from the TLB's uncached-attribute field, which EntryLo0/1 bits
63:62 load (R10000 User's Manual 007-2490-001). EntryLo already keeps
those bits on the family, and a TLB entry holds them, but the entry's
physical base dropped them, so a mapped uncached page always reached
attribute zero.

Carry them to physical bits 58:57 only for pages the model's C table
decodes as uncached, so cached mappings are unchanged. The TLB already
holds that table for its shadow entries; reading it there, and
recomputing each entry's base when a table is installed, avoids a second
decode of C.

The attribute reaches the bus only on a board that decodes physical
bits 58:57. The Octane, Indigo2 and O2 mask the physical address to 40,
32 and 32 bits before the bus, which folds it away, and a test shows
that at the executor. The R4400 and R5000 never store the bits.

Tests cover the attribute and the board masks, and the C table (a value
the model decodes as cached carries nothing; a table installed after the
entry was written decides).

The uncached-attribute comments cite NEC U10278EJ4V0UM, the fourth edition
of the VR10000/VR12000 user's manual, section 11.3, by document number;
the manual itself is not added to the repository.
Bedrock memory configuration changes the RAM decode. Keep ordinary CPU
and DMA memory on Physical directly, divert diagnostic pages through MD,
and update their generation state when memory configuration or ECC
changes.

The ECC view overrides write_narrow_from to take an uncached SH's check
word from the whole source register, at the position Fuel PROM 6.210's
memory test uses. Every other view stores the architectural low bits.
The Bedrock memory model hard-wired the Fuel's two 256 MiB banks: the
bank size was a constant in the decode, in the ECC, directory and
protection keys, in the self-test and in the processor bus's test for
where RAM can be, and only four software slots had generation counters.
A Tezro fits six banks of 1024, 1024 and four of 512 MiB, the last two
in slots 4 and 5, at 4 to 6 GiB.

Let the board pass the fitted bank sizes, locational bank 0 upward, and
check them: at most eight banks, each a whole number of direct-window
regions up to the 1 GiB the largest size code names. The decode, the
self-test and the sparse ECC, directory and protection state read the
size of the bank they touch, and key it by where the bank starts when
all banks are laid end to end, so banks of different sizes never share a
key. The processor bus asks a RamExtent where RAM can answer instead of
assuming four 256 MiB prefixes; the extent allows for the bank-select
swizzle, which permutes slots within groups of four, and the memory map
the processor is given lists the same slots. Generation counters
cover the slots that can hold RAM, which for a Tezro includes 4 to 6 GiB.

ppmem's direct window is 4 GiB, and its map_range panics on a range past
it. The remap now maps only the part of a bank inside the window; banks 4
and 5 answer through Physical's decode table, whose 40-bit range covers
them, with no direct fast path. A test shows the window bitmap stops at
4 GiB while those banks read, write and take DMA.

The Fuel passes [256, 256] and behaves as before; its tests still pass
unchanged. New tests port the qtest memory-bank, size-boundary,
bank-select, directory and self-test cases to the Tezro population.
The flash seeded its configuration block with the Fuel's values: boot
mode word 0x549aa985, 500 MHz and machine type 2, and the machine kept
the matching CP0 Config as a separate constant. A Tezro needs word
0x549bad85, 700 MHz and machine type 0, with a Config whose mode bits
agree, or the PROM stops in its error routine before it sets up GP.

Describe the record as a FirmwareRecord the board passes to the flash:
the PROM's boot-mode word, the reset mode bits the CPU latches into
Config, the clock and the machine type. The record yields the Config the
CPU resets to, so the flash and the CPU read one source. Fuel and Tezro
are the two records. The Fuel's flash bytes and Config are unchanged.

A reopened image still gets the opening board's record, and the byte sum
of the block is still preserved. The tests cover both records, port the
byte-sum rule from the initial-flash unit test, and pin the Tezro's check
adjustment, which that test does not cover.
The Tezro's IO9 board carries an IOC4 (PCI revision 0x53) where the Fuel
has an IOC3. Its serial console is one of four byte-wide 16550 ports
behind it, so the PROM and IRIX cannot reach a prompt without it.

The model has the four ports on the shared 16550 core, each with the
serial ring engine the IOC3 uses, a 66.67 or 33.33 MHz UART clock
selected by the board, and the local interrupt words: eight serial
bits per port, a second word for DMA errors, and one PCI pin for both.
A failed ring transfer is captured with its address and requester and
halts the port until a channel reset. The ATA DMA readback word the
PROM diagnostic probes and the PCI status the IP53 policy asks for are
latched only for a board that requests them.

IOC4 orders its own bytes, so register offsets reach it unswizzled.
Control registers take 32-bit accesses and the UART windows bytes;
anything else is refused and logged. The keyboard and mouse controller,
GPIO and the ATA channel are not modelled.

The tests build the chip alone against a block of bus memory and port
the register-contract cases of the QEMU model's qtest that apply to a
synchronous engine.
The PI answered as a node with one processor: slot A of PI0 present,
every other slot absent, and the Fuel machine set CPU 0's enable bit
itself. A Tezro tower can hold one, two or four processors, and the
PROM reads the present, enable and CPU-number registers to find them
before it decides how many to use.

Let the board pass a Population: which of the four slots hold a
processor. The board's order is PI0/A, PI1/A, PI0/B, PI1/B, so two
processors sit one per interface. Every fitted slot reads present. Only
CPU 0 starts, so only its enable bit is set at power-on; the others are
held disabled, which should lead the PROM to report one enabled
processor. A guest write that enables another slot is stored and still
reaches the node hooks, because the bit is register state, but no
processor ever starts.

The model cannot carry out what a missing processor would do, so it
says so. An enable bit for another slot, an NMI that no node takes, an
interrupt output for a slot with no running processor (an
interprocessor interrupt is one) and a timer enable for such a slot are
recorded with a count, logged on their first occurrence under the
bedrock module, and readable through BedrockPi::refused. Before, PI1 and
slot B lines were dropped without a trace. The CPU-number register
still answers slot A of its interface, for the lack of a requester
identity on the bus.

The Fuel keeps its one-processor population and its register values; the
machine no longer writes CPU 0's enable bit itself. New tests port the
presence, mask, CPU-number, NMI and interrupt-slot cases from the PI
qtest to the four-slot node.
The Tezro's IO9 carries a 32 KiB timekeeping SRAM with a BCD calendar
in its last eight bytes. The Fuel's M48T35 model has the same geometry
and clock offsets, but not the same registers. Run against the DS1744
contract, it reads zero from the control register where the part shows
the century, 0x40 where the part shows 0x60 after a read request, has
no weekday, and takes a month of 0x13 as a wrapped date where the part
refuses the commit. Those are a different calendar, so the DS1744 is
its own model rather than a flag on the Timekeeper.

The control register carries the century beside the write and read
bits, the weekday advances with the date, the oscillator-stop bit rides
in the seconds, a commit that is not a date is refused, and test mode
flips the seconds' low bit at 1024 Hz. A fresh-year base of 1968 shifts
the host date into 2000 to 2099 for the guest's year digits and keeps
the weekday of the real date. A backing file and a clock reference
beside it restore RAM and the calendar across a restart.

The tests step the clock and port the QEMU model's calendar, invalid
date, test mode, year boundary and persistence cases.
The processor interface raised the graphics interrupt once when a store
left the counter below the bias and let every later store through. That
is QEMU's Hub behaviour. On the Bedrock QEMU follows the SN1 header's
credit-counter comment: the store that crosses below the bias raises the
interrupt and stops the slot, and the slot's next counted store waits
until the processor has taken the interrupt, software has written the
counter, the bias or the interrupt count, or returned credits have
lifted the counter above the bias. Without the hold a writer the board
cannot keep up with drives the counter as far below the bias as it
likes, and IRIX's periodic check reports the shortfall as a graphics
error.

Model the stop. A held store is answered busy without being forwarded or
counted, so the processor retries it and pays once when it goes. The
pending-modify register's clear of a graphics bit tells the slot the
interrupt was taken. As in QEMU the hold ends after a second, logged,
since the chip's is not known to be unbounded, and every counted store
below the bias raises the interrupt, not only the first. A retrying
processor here can take an interrupt between attempts, where the chip
holds the store itself.
The Bedrock PI records what reaches a slot with no running processor,
but it hands CPU 0's NMI and enable writes to the node's hooks, and the
Tezro's hooks took the trait's default for both, which does nothing. An
NMI for CPU 0 and a write that clears its enable bit were dropped
without a trace.

The Tezro cannot carry either out: the CPU has no NMI entry, and CPU 0
runs from power-on with nothing to stop it. Its hooks now count both and
log each kind the first time, as the PI does for the other slots.
Setting CPU 0's enable bit, and anything for the other slots, is left
to the PI. The Fuel's hooks are unchanged.
The Bedrock network-interface port status word read 0 on a node with no
cable. The IRIX 6.5.30 bus-error handler starts with a network-interface
check that prints "Detected a link reset due to other partition reset"
when reset enable is 0 and port status bits 1:0 are 0, so every
faulting widget probe printed it.

Read bits 1:0 = 1 instead: link up (bit 1) clear and bit 0 set. The
value is inferred, not measured: bit 1 follows a real Tezro PROM
reporting its NUMAlink down, and bit 0 follows a real one-node Fuel
taking a PIO read bus error with no link message. Nothing is recorded in
the processor-interface error records. Rename the offset-0 register from
NODE_ID to PORT_STATUS, which is what it reports.

This block is shared with the Fuel board, so QEMU needs the matching
change.
Widget zero reported revision 5 because the reused Crossbow model takes
one revision for both behaviour and the identification word. Real Tezro,
Origin 350 and Onyx 4 PROM inventories print the crossbow as revision 3,
and the IP35 PROM takes that from bits 31:28 of the widget-zero
identification word.

Add a reported-revision override to the Crossbow and use it for the
PXBOW, so the word reads 0x3d100001: revision 3, part 0xd100,
manufacturer 0, bit 0 set. Behaviour stays selected by revision 5; the
IRIX 6.5.30 crossbow driver does not tell revisions 3 and 5 of this part
apart. Part and manufacturer remain inferred, and a 2009 OpenBSD dmesg
that names the part Bedrock is an open conflict.
Only the PXBOW completes the PROM's writes to the two read-only error
address registers. The Octane and any other standalone Crossbow must keep
refusing them. The existing test covered the lower register of one
Octane; cover the upper register and both revisions that select this
model, so a change to the PXBOW option cannot loosen them.
The model already answers these as the IP35 PROM expects, but no test ran
them on the Tezro board:

- the hub widget control keeps destination credits of 4 in bits 15:13
- a one-node boot reads port status 1, reset enable 0, port errors 0 and
  the error-clear register 0 on every read, through both register
  windows, and the network error bits (39, 40) of the processor
  interface's second pending register stay clear
- the PROM's crossbar check: both error address writes complete, the
  access error shows with the hub's widget in the status source,
  vector 0x10 reaches PI 0x90 while the error stands, and link 15's
  arbitration bit 1 is kept
The PROM fetches its own code uncached at plain physical 0x1fcxxxxx, and
Bedrock serves that from the reset PROM. The Tezro started its jitv2
compile pool on the cached view, which answers RAM there once a memory
bank covers the window, so the JIT could compile memory's bytes in place
of the PROM's.

The Fuel already compiles each page through the view the CPU fetched it
by. Give the Tezro board the same code bus over its cached view, start
the compile pool on it and opt the CPU into the uncached view keys.

The uncached view keys sit at frame 1 << 20 and up, so the JIT takes
them only on a machine whose memory ends at or below 4 GiB, and asserts
otherwise. The Tezro's default 4 GiB layout puts banks 4 and 5 above
4 GiB, so a jitv2 build panicked building the CPU. Refuse such memory
in a jitv2 build with a message that names a layout it takes, four
1024 MiB banks, instead.
A guest power-off ends the process without dropping the board, so the
flush the machine makes then is what carries a copy-on-write overlay's
sector list to the next run. The Fuel has a test for that path and the
Tezro had none.

The test attaches an overlay disk on each ISP12160 channel, writes,
flushes, forgets the board as a process exit would, and reopens the
overlay on a new board. It fails without the flush.
The guide still placed the second PIC directly on Bedrock IO and left
the V12 slot empty, and said nothing about how far the machine gets or
where it stops working. A reader trying the Tezro would hit the PROM
stopping while it copies itself to memory, the disabled processors and
the PROM and IRIX warnings with no word on whether they are expected.

Describe the second PIC on link 11 of the IO9 PIC's PXBOW, the PROM
disabling every processor but CPU 0, the PROM's dependence on the R10000
cache index operations this base lacks, the R16000 policies left out,
the empty-slot master timeout, the configuration-mismatch warning with
no EEPROM images, the Origin 300 device numbering of the tested disks,
IRIX's link-reset notices, and how far the V12 gets: Xsgi starts and a
login at the greeter reaches the desktop. The changelog and the [tezro]
reference say the same in short.
The decode cache reads a RAM page's words straight out of ppmem's
window. A page the shadow cache serves partly from lines the guest
staged has words the window does not hold, so it has to be read word by
word through the cache instead.

Add set_bus_only to mark such a page. It drops the page's decoded slots
and the page then has no window pointer, so it misses the fetch fast
path and takes the slow path, which re-reads and compares every word.
Pages not marked pay nothing.
A boot PROM can stage a secondary-cache line with Index Store Tag (S)
and Index Store Data (S) and then load or fetch from it before memory
works. The R10000-family shadow cache kept those arrays for diagnostics
only, so such an access read the bus. Per the NEC VR10000 manual, a valid
tag whose PA[39:18] matches the access is a secondary hit that supplies the
staged data, with no bus read.

The shadow arrays already hold the staged tags and data, so the hit is
a lookup in them: both ways at the access's index, valid state
(TagLo[11:10] != 0), matching tag. A count of valid tags makes the
normal path one load and a not-taken branch. Stores to a staged line
stay in it. Index Writeback Invalidate, Hit Invalidate and Hit
Writeback Invalidate (S) end the line, writing a DirtyExclusive one
back; so does Index Store Tag.

The interpreter's and the JIT's inline loads and stores reach RAM
through the micro-TLB and ppmem's window without calling the cache, and
the JIT compiles code from memory. The cache now reports which pages
hold staged lines. The micro-TLB declines to fill them,
jitv2_track_pcp leaves them to the interpreter, and a CACHE instruction
that changes the set flushes translations made before it, the way a
self-modifying store has to.

The word order of a refill is not settled by the manuals. The TagLo word
lands at the lower address, which is inferred from software that stages
lines this way and is marked as such. Predecode, the primary caches,
replacement, ECC and the virtual-index purge are not modelled.
Three gaps in the staged-line model could lose a guest's data or make a
restore read memory instead of the cache.

A store to a CleanExclusive or Shared secondary block asks for
DirtyExclusive (NEC U10278EJ4V0UM p.246, performance counter events 14
and 15). The model kept the stored bytes in the line but left the state,
so Index Writeback Invalidate dropped them. The store now sets
DirtyExclusive. The manual does not state the transition, so it is
marked inferred.

Snapshots now carry the staged lines, a restore ends any staged before
it, and writeback_all sends dirty ones to memory. Without that, a
snapshot taken with a line staged restored a guest whose cached accesses
read memory.

op[1:0] = 10 is undefined on the R10000 and now does nothing, instead of
staging or ending lines. The l2 probe monitor command reports the staged
count, and the comments say the R12000 and later use the R10000 layout by
inference and that the model has no replacement.

Tests cover the store transition and writeback, the undefined encoding,
a load with a different VA[13:12], five cache geometries (1 MiB with 128-
and 64-byte lines, 2 MiB, 4 MiB) and the snapshot round trip.
…er lightning

The compiled-load and compiled-fetch tests were excluded from lightning
builds, which are the builds the guests boot with, and nothing covered a
compiled store. Nothing in them depends on the feature: they pass under
jitv2 and under lightning,jitv2, so drop the exclusion.

Add the store case: a warm write micro-TLB entry sends the compiled store
into ppmem's window. Once the page holds a staged line the store stays in
the line, the window and memory do not change, and invalidating the line
writes the value back. Disabling the micro-TLB decline in nutlb_fill makes
it fail.
Staged lines hit until an explicit operation ends them, because ordinary
misses never fill or evict. Write down that limit, the staged-line counts
measured at the login prompt on the Octane, Fuel and O2 (all zero), the
guards the page fast paths need, and what is inferred or absent.
…al address

Every CACHE operand on the R10000 is translated like a data address
before the operation, index operations included. The secondary set,
way and quadword come from the physical address, the primary caches'
from the virtual one (NEC U12754EJ1V0UMJ1 pp.14-15, 70-72).

iris indexed the secondary cache by the operand's virtual address and
translated only hit operations. A mapped Index Store Tag or Index Store
Data landed at the virtual address's set, so a cached access through the
frame missed it.

exec_cache now translates every operand but Cache Barrier, as a read, so
it can take TLB Refill, TLB Invalid or a privilege Address Error and
never TLB Modified. The translated address reaches the cache through
set_cache_op_pa. An operand on an uncached page is undefined on the
R10000: it is translated and the operation goes ahead as if cached.

Tests cover a mapped stage landing on the physical set and hit by a
cached fetch and load, aliases sharing a line while the primary index
stays virtual, TLB Refill, a clean page taking no TLB Modified, and the
uncached page. A further test stores a quadword once and checks that it
survives many rewrites of the line's other quadwords, on 64- and 128-byte
lines; Index Store Data ignoring PA[6:4] makes it fail.
The Tezro PROM ends its staged secondary line and then runs from primary
instruction-cache lines it staged with Index Store Tag (I) and Index
Store Data (I). iris served fetches only from the secondary cache, so
those fetches read memory and execution ran off.

A line is valid when TagLo[6] is set. The tag is PA[35:12] in TagLo[31:8]
and PA[39:36] in TagHi[3:0], the index VA[13:6], the way VA[0], the word
VA[13:2]. A cached fetch compares both ways at its VA[13:6] against its
translated address and takes TagLo from a hit, before the secondary cache.
Index Invalidate (I), Hit Invalidate (I), an Index Store Tag with the
valid bit clear and power-on end the line. Such pages get the same guards
as secondary-staged ones: micro-TLB entries, JIT page tracking and the
decode cache. Snapshots carry the lines.

Serving is off unless IRIS_STAGED_L1I is set. The predecoded form is
unknown, and the Fuel PROM stages primary lines of its own code with
words taken from Index Load Data of lines the hardware had predecoded:
36-bit values that are not an instruction in TagLo. With serving on, the
Fuel PROM executes them and stops at "Copying PROM code to memory".
iris refills and predecodes nothing and cannot make those words. Tezro
can be tried with the variable set.

The primary data cache is not served. Tests stage whole blocks on way 1
of several sets, fetch with no memory read, check that the primary line
wins over a secondary one and outlives it, the four ways a line ends,
the snapshot round trip, and a compiled page dropped for a staged line.
Disabling the primary hit fails them.
…r parity

A primary instruction-cache entry is the instruction predecoded into 36
bits with an even parity bit. Fetches from staged lines took TagLo as
the instruction, which is wrong for the words the Fuel and Tezro PROMs
stage, so serving had to be off and those PROMs ran from flash instead.

A fetch hit now decodes the stored word with a layout inferred from the
words a shipped PROM stages: addiu $4,$0,2 is 0x3_4420_0004. Only the
sampled class is known, so a word of any other class is refused. It is reported once on
stderr and the fetch is treated as a miss that reads the secondary cache
or memory. A Cache Error would stop a guest the hardware runs, and
memory holds the instruction a word was built from in all but the few
stale entries. The Fuel PROM stages seven unsampled classes and boots on
that fallback.

Parity is checked on a fetch hit and a bad word posts a Cache Error
unless Status.DE or ERL is set, as the manual says. Index Load Data (I)
returns word and parity as stored, so a readback stored back verbatim
round-trips. A word with p[35:32] = 0xf is the model's own raw
encoding, a model choice with unknown hardware contents, so words
outside the sample can round-trip too. A refill is not modelled.

IRIS_STAGED_L1I is gone and serving is on. Tests cover the decode and
each field, refusal, the parity error and its Status.DE exception, the
round trip for sampled and raw words, and a staged word differing from
memory. Breaking a decode field or the parity check fails them.
…ache from staged lines

The predecoded primary instruction word has 11 known classes (all
inferred): nop, dsll/dsll32, or, addi/addiu/ori/lui, daddiu,
cache, ld, mtc0, jal, jr/j and REGIMM. A word decodes only if its
class, p[28:26] and, for the SPECIAL classes, the function field name an
instruction in the table. Anything else is still refused, reported and
read from memory. The Fuel PROM now boots with no refusals.

A cached fetch that misses the primary I cache and hits a staged
secondary line refills the 64-byte primary block from it (NEC
U10278EJ4V0UM pp.87-88), as the Tezro PROM expects: it invalidates a
way, fetches through a staged line, reads the refilled way back with
Index Load Data (I) and stores the words into another line. The victim is
an Invalid way before a Valid one and way 0 if both are valid. The
manual states no victim rule, so that is a model choice. Words are
stored in the raw encoding, the architectural word standing in for the
predecoded form hardware stores, a model choice that lets a verbatim
readback execute. Ending a secondary block now also
invalidates the primary I lines inside it, as the manual says.

Tests decode one example per class, refuse a known class with a foreign
instruction, and cover the refill into the invalid way, Index Load
Tag/Data, the readback round trip, and that no refill happens without a
staged hit. Dropping the refill or always using way 0 fails them.
A processor running with others makes SC atomic through its cache's
compare-and-swap. The shadow cache does that itself for RAM in its
direct window and hands anything else to the bus below it, and on the
IP35 boards that bus had no compare-and-swap, so an SC to a bank above
4 GiB, or to a checked page, became an ordinary store. The cached view
now passes the swap to MD's DMA view, which swaps in the bank and
advances the page's logical counter as a write does.

Fuel is unchanged: a single processor never takes this path. The
Tezro's processor count does not change here either.
A Bedrock node can hold four processors, and the PI only knew how to
run the one in PI0 slot A. The board now names its running slots
beside the fitted ones. A running slot starts with its CPU-enable bit
set, its interrupts, timers and enable writes are no longer refused,
and the CPU-number register answers the reading processor's own slice,
slot in bit 0 and interface in bit 1, as QEMU's PI does from the
issuing processor. A reader that is no processor reads slot A, QEMU's
model behavior for an access with no requester.

CPU enable stays stored only. QEMU grants every fitted slot at reset
and wires the register to nothing; that is an approximation, since no
cold-reset value is known and the bit is not known to stop a processor.
Clearing a running slot's bit is recorded as a refusal. An NMI is now
refused unless the node reports it taken, so an NMI to a running
processor, which has no NMI entry, is logged instead of dropped.

Fuel and the Tezro still run PI0/A alone, so their behaviour is
unchanged apart from that NMI log line. Tests follow
sn1-cpu-identity-test four-processors-know-their-slots and
sn1-bedrock-pi-test cpu-number-fallback, cpu-enable-lines and nmi.

Ported from hw/misc/sgi-bedrock-pi.c in jamesbraid/qemu-sgi-origami.
Builds on the tezro commit that models the four-slot population.

The PI now records a stop or an NMI it cannot carry out for a running
slot, so the Tezro's node hooks drop their own CPU 0 refusal counters
and their test.
A processor on PI1 reads its CPU number and programs its own
interrupts through the XIO local register form at offset 0, as QEMU's
IP35 node overlays PI1 there for every processor sitting on it
(OpenBSD expects the local form to select the requester's interface).
The node form still has PI0 at offset 0 for every processor. That the
chip answers PI1 there is an approximation pending a hardware read.

The selection is only built when the board runs a processor on PI1,
so the Fuel's register map is the one it had. The L1's soft reset
now goes through a helper that takes the list of processors to reset;
the Fuel passes its one.

Test follows sn1-cpu-identity-test four-processors-know-their-slots,
which reads through the local form.

Ported from hw/mips/sgi/sn1-node.c in jamesbraid/qemu-sgi-origami.
Builds on the previous commit's running slots.
The PROM runs its early code with its stack and data in primary cache
lines it tags for a 32 KiB window at 0x100000, before memory is set up.
Each processor's cache holds its own lines, so with several processors
in that phase at once each has a private window. MD had one page for
the whole node, so a second processor would have written over the
first one's stack.

Each processor now has its own page, started and ended by its own tag
stores. While any page stands, Physical's table holds a small selector
over the window; a processor whose window has ended reaches memory
through MD's checked path. A thread that runs no processor reaches CPU
0's page, which is what every path reached before.

Private per-processor storage is a model policy, as in QEMU's
per-processor window; when a window starts and ends is approximated
from the tag stores, as before. The test runs two processors through
the window at once and fails with a shared page or a shared end.

Fuel and the one-processor Tezro run CPU 0 alone and see the page they
saw before. Builds on the tezro commits that model the PROM's staged
cache lines.
A board that holds a processor until others catch up, as the IP35
node's boot rendezvous does, needs its own reason so that its release
does not end a world stop or HEART's disable, and the monitor can tell
the holds apart. Nothing uses it yet; the Octane and the Fuel are
unchanged.
The IP35 PROM brings a node's processors together through LB scratch
1, LB scratch 3 and II scratch 0, and a board that runs several
processors needs to see those stores to hold one that arrives early.
The LB takes optional hooks: with them connected a store to scratch 1
is the hooks' to make, a read of it is reported first, and a store that
leaves scratch 3 clear reports the hub lock released. Bedrock IO reports
each store to scratch 0 once it has landed. Every hook runs with the
block's lock dropped.

Nothing connects them on the Fuel, whose registers behave as before.

Hook points ported from hw/misc/sgi-bedrock-lb.c and
hw/misc/sgi-bedrock-io.c in jamesbraid/qemu-sgi-origami.
The IP35 PROM brings a node's processors up assuming they left reset on
one clock edge and have run in step since: four of its early handshakes
order one processor's stores before another's reads only by both
running at hardware speed. Each processor here is a host thread that
can fall far behind, as under QEMU's multi-threaded TCG, so this ports
QEMU's four holds: the reset barrier on LB scratch 1, the pair liveness
post in the processor's own real-time compare register, the presence
publisher at its release of the hub lock, and the partner at its
real-time counter read. The processor that arrived early is held until
the ones it must follow catch up. A hold gives up when the processors it
waits for have retired 2e9 cycles without arriving, not after host
time: a host thread the scheduler leaves off a core for seconds retires
nothing, so it cannot run a hold out and drop out of the PROM's count
(a 2.5 s stall of one processor made the PROM find two). Host time
backs the bound only as a 120 s deadlock safety net. QEMU's 2 s bound
is on its virtual clock, which follows the host under multi-threaded
TCG, so it has the same exposure.

Each hold is an approximation with no hardware counterpart: it changes
when a processor next runs, never a value it reads. Limits: a held
processor stops at its next batch boundary rather than at the end of
its current block; the publisher's hold is hooked on LB scratch 3, as
QEMU does, where the node's own description names the publication in
II scratch 0 and scratch 3 as the console lock, and the hub lock being
scratch 3's read-set twin is inferred. Nothing holds with fewer than two
processors running. Each hold's test fails with that hold removed; the
store-from-no-processor case follows sn1-bedrock-lb-test
scratch1-second-write-no-hold.

Once every populated interface has released its hub lock after
publishing, or the join was abandoned, the joins are over until the next
reset: a flag checked before any lock makes the PROM's and the kernel's
later scratch and real-time register traffic cost one load.

No board connects it yet.

Ported from hw/misc/sgi-bedrock.c in jamesbraid/qemu-sgi-origami.
Builds on the running slots and the rendezvous register hooks.
The two- and four-processor boards fitted their processors but ran
only CPU 0. Each fitted slot now gets its own R16000, numbered in the
board's slot order (CPU 1 in PI1 slot A, CPUs 2 and 3 in the B slots),
all starting together from reset as in QEMU's IP35 node. Each has its
CPU number, its share of one compile pool and page set, its direct
window bitmaps, its slot's interrupt lines from its own PI, its own
status bar counters and a place in the monitor. The board runs the
PROM's boot rendezvous across them, fed from the LB scratch registers,
II scratch 0 and each PI's real-time registers, and resets it with the
node. A processor's stores into its slot's graphics page count against
its own slot and raise its own interrupt.

cpus = 1 builds one processor, as before. Fuel is untouched.

Board composition ported from hw/mips/sgi/sn1-node.c in
jamesbraid/qemu-sgi-origami. Builds on the tezro commits for the
two- and four-processor board and the processor count, and on this
branch's running slots, per-processor startup windows and rendezvous.
On the two-processor IP35 board the stock PROM treats the empty B and
D processor positions as processors that failed to start: on each of
three boots it counts a retry and asks the L1 for a power cycle, and on
the fourth it writes a DisableB or DisableD record and goes on. QEMU's
boot tests seed the fresh flash with those two records so the first
boot runs without the power cycles; that the board ships so is a model
policy, not a recovered factory image.

A fresh store can now be seeded with environment records at given
offsets, and the two-processor records are provided byte for byte as
QEMU's tests write them. A store that already exists keeps what it
holds. Flash::new and with_file seed nothing, so the Fuel's flash is
unchanged.

Ported from tests/functional/mips64/lib_origin300.py in
jamesbraid/qemu-sgi-origami.
A fresh flash on the two-processor board now carries the DisableB and
DisableD environment records, so the PROM's first boot goes straight
through instead of power-cycling the module six times to give up on
the empty positions. The one- and four-processor boards leave no empty
position the PROM recovers and seed nothing, so cpus = 1 is unchanged.
A flash file that already exists keeps what it holds. The node passes
the board's records to its flash; the Fuel passes none.

Builds on the previous commit's seeded flash and on the tezro commit
that fits the two- and four-processor board.
A node restart from the LB soft reset, the NI local reset or an L1
event was silent, and so was each processor's cache-as-memory window
starting and ending. With several processors those are what tell a
restart loop apart from a hang; each is now logged under the bedrock
or l1 module, with the processor that caused it. Nothing else changes,
on the Fuel or the Tezro.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant