Repository navigation
riscv: implement multikernel lifecycle and safe shutdown - #39
Draft
pro-utkarshM wants to merge 98 commits into
Draft
pro-utkarshM wants to merge 98 commits into
pro-utkarshM wants to merge 98 commits into
Conversation
pro-utkarshM
force-pushed
the
riscv/multikernel-23-25-v1
branch
from
September 7, 2026 18:08
7fd9938 to
eef3dcb
Compare
Author
|
@congwang-mk, before updating this draft, could you confirm the dependency ordering? We have newer local #23–#25 lifecycle commits, with #26 kept as a separate follow-up. Standalone builds passed; QEMU PLIC/AIA validation passed on an integration stack with explicit runtime dependencies, including 50-cycle same-Image and alternating-Image re-spawns. The tested stack still has the shared receive-ring reset limitation. Nickolaev’s #7 overlaps that work but is not integrated or validated in our stack. Should we depend on #7 first, or submit the bounded RISC-V series with that limitation disclosed? Which public #27/#28 commits should we use instead of our temporary attributed runtime extracts? |
|
There are no public #27/#28 commits: both are unassigned with no PR or branch, and Please don't carry extracts from Nikolay's July Don't depend on #7; disclose the ring-reset limitation. For this PR:
|
congwang-mk
force-pushed
the
multikernel-riscv
branch
2 times, most recently
from
September 25, 2026 20:32
77417e1 to
88f5f29
Compare
|
@pro-utkarshM Please do a rebase. And also let me know when it is ready for review. I intentionally don't review it since you mark it as draft. BTW: the ARM port is already working (verified on a bare metal ARM server): https://github.com/multikernel/linux/tree/multikernel-arm |
With a transport assigned, __vsock_release() leaves removing the socket from the bound and connected tables to transport->release(), as virtio does in virtio_transport_close(). mk_transport_release() only sends a SHUTDOWN, so every closed socket stays hashed: it keeps its port, the socket itself and a module reference. On a host, each kerf exec binds a reserved port, and after 512 execs every port from 512 to 1023 is held by a dead socket and exec fails until reboot. Call vsock_remove_sock() on release in every state. vsock_remove_sock() keeps the binding when the socket is only moving to another transport. Fixes: 7b3d04b ("vsock: Introduce multikernel transport") Signed-off-by: Cong Wang <cwang@multikernel.io>
kexec_vmlinux_ops loads the ELF vmlinux that kerf extracts from a bzImage, so the PE signature on the bzImage never reaches the kernel and the loader has no verify_sig. With KEXEC_SIG enabled, every spawn kernel fails verification; with KEXEC_SIG_FORCE or lockdown, every spawn is refused, signed or not. Let the ELF vmlinux carry its own signature instead: a PKCS#7 signature appended in the module signature format, as written by scripts/sign-file. Add kexec_kernel_verify_appended_sig(), which checks it against the secondary keyring and then the platform keyring, so a key enrolled as a MOK works. An unsigned image returns -ENODATA, as an unsigned PE image does. s390 verifies its kernel images the same way and could share the helper. The loader copies PT_LOAD segments by program header offsets, so the appended signature is never loaded. Signed-off-by: Cong Wang <cwang@multikernel.io>
kimage_file_alloc_init() set instance->kimage before preparing the image, and its error path did not agree with it: - A failure in kimage_file_prepare_segments(), such as an unknown image format or a rejected signature, jumped past the multikernel cleanup. The freed image stayed in instance->kimage, so every later load failed with -EBUSY, exec booted the freed image, and unload, which the -EBUSY message suggests, freed it again. The manifest and IPI pages leaked. - The other failures dropped the instance reference, which the caller drops again when kimage_file_alloc_init() fails. - No failure returned segments already taken from the instance pool. Set instance->kimage only after the load completes, next to MK_STATE_LOADED, so exec, unload and IPI never see an unfinished image and a failed load leaves nothing to detach. Let kimage_file_alloc_init() consume the instance reference on every path, and unwind it with the multikernel part of kimage_free(), now kimage_free_multikernel(), so the two cannot drift apart again. Reject KEXEC_MULTIKERNEL together with KEXEC_FILE_ON_CRASH, which built a crash image that never took the instance reference. Fixes: b6e539f ("kexec: Integrate multikernel support with kexec_file_load()") Fixes: 3b1ab98 ("kernel: Introduce generic multikernel IPI communication framework") Signed-off-by: Cong Wang <cwang@multikernel.io>
Each load allocated a manifest page and an IPI ring into struct kimage, yet both belong to the instance: the ring is its inbox and the manifest its boot tree, rewritten at every exec. The instance already records the ring, so the address lived in two places, copied from the image at exec and derived again through instance->kimage on the first send. Every reader had to go through a kimage that might be unpublished or freed, and kimage_free() had to clear the instance's copy before freeing the pages. Allocate both once per instance, on its first load, and free them with the instance. Loaders and the manifest code reach them through image->mk_instance, and struct kimage drops mk_manifest and mk_ipi. ipi_data keeps its meaning of "the instance can take messages": it is published with instance->kimage when a load completes and cleared on unload. The exec-time copy, the lazy derivation and mk_instance_ipi_area() go away; the send path reads ipi_data once, which stays valid while it holds the instance reference because the ring now lives as long as the instance. Signed-off-by: Cong Wang <cwang@multikernel.io>
kexec_load() accepted KEXEC_MULTIKERNEL, but only with KEXEC_JUMP enabled, and did nothing multikernel with it: loading listed an ordinary image with no instance and no ID, which nothing could exec, and unloading freed up to ten multikernel images of every instance, bypassing per-instance unload. kerf and the instance lifecycle use kexec_file_load() only. Drop the flag from KEXEC_FLAGS and both branches, along with kimage_get_all_by_type(), their only user. With that, image->mk_instance is set for an image's whole lifetime exactly when its type is KEXEC_TYPE_MULTIKERNEL. Say so on the field and drop the NULL checks that assumed otherwise. Signed-off-by: Cong Wang <cwang@multikernel.io>
An instance's memory list and its instance pool, the gen_pool its kernel
image and control block come from, disagreed. The pool held only the
chunks the instance was created with, while regions added later came
straight from the multikernel pool. Removing a region returned its
memory without telling the pool, and releasing the instance returned
the pool's chunks, so:
- Removing a region the instance was created with left its chunk in the
pool, which kept allocating from memory the instance no longer owned
and freed it again on release:
kernel BUG at lib/genalloc.c:555!
gen_pool_free_owner
multikernel_free
multikernel_destroy_instance_pool
mk_instance_free_memory
mk_instance_release
or, had the range been handed out in between, freed another owner's
memory without a word.
- Regions added later were never returned: growing an instance by
128 MB and deleting it left the multikernel pool 128 MB short.
Make every region exactly one chunk of the instance pool, adding it to
the pool when it is added to the instance. A gen_pool cannot give a
chunk back, so removing a region reserves its whole chunk first, which
fails, and refuses the removal, if the kernel image or control block
lives there, and then returns the memory. Releasing the instance
returns every chunk still owned and drops the reservations of those
that left. This replaces the check that the loaded image does not live
in the region with a precise one that also covers the control block,
and leaves no window where a load allocates from memory being removed.
For a running instance, reserve before asking its kernel to give the
memory up, and drop the reservation again if it does not.
Signed-off-by: Cong Wang <cwang@multikernel.io>
Two things stand in the way of a caller that has to exchange a message with another kernel from a context that cannot sleep, such as an interrupt chip callback under the descriptor lock. Sending looks the instance up by ID, and mk_instance_find() takes a mutex. Everything else on that path is fit for atomic context, down to the GFP_ATOMIC allocation, and the console already gets there with interrupts off. Add mk_send_message_to() and multikernel_send_ipi_data_to() for a caller that holds a reference to the instance, and build the ID variants on top of them. Receiving depends on the doorbell interrupt, which such a caller may be unable to take: it may be the doorbell CPU itself, or stop_machine() may hold every CPU, as it does while a dying CPU migrates its interrupts. Add mk_ipi_poll() to drain the ring from the caller's context. The ring has a single consumer, which used to be implied by the doorbell going to one CPU, so the drain now runs under a lock. The interrupt waits for that lock instead of giving up: a poller that is just leaving may not look at the ring again. No change for existing callers. Signed-off-by: Cong Wang <cwang@multikernel.io>
A kernel that has to ask another one something and cannot go on before the answer is in, with interrupts off, has had to build the wait itself: a list of outstanding requests, a match of the answer against them, and a loop that drains the ring because the doorbell may never be taken. PCI config space access over the ring is the next such caller, and it runs under the PCI core's raw lock. Add mk_request_sync() and mk_request_answer(). The caller carries a sequence number from mk_request_seq() in its payload, and the handler of the answer hands it back with the answer, which is copied into the waiter's buffer. The wait polls the ring with mk_ipi_poll() until the answer is in or the timeout passes. An answer that arrives after the waiter gave up finds nothing to match and is dropped. A message handler cannot wait this way: it already holds the drain, and polling would take the lock again. mk_ipi_in_drain() tells, and such a request fails with -EDEADLK. Signed-off-by: Cong Wang <cwang@multikernel.io>
A spawn kernel generates config cycles to its PCI functions itself today, through the ECAM window its tree describes or, without one, through the 0xCF8/0xCFC port pair. That pair is an address register and a data register shared by every CPU of the machine: two kernels using it at once corrupt each other's accesses, and no lock spans kernels. Through either mechanism a spawn can also reach any function on the fabric, the host's bridges above its own devices included. Have the kernel that lent the functions answer for their config space instead, the way Xen's pciback answers pcifront. A new MK_MSG_PCI message type carries reads and writes of one function, and the answer carries the value read and the PCIBIOS result. The lending kernel makes the access with its own accessors, quirks included. What an instance may reach is a table of the functions lent to it, kept while it runs: filled from its device list right before its CPUs are started, emptied when it leaves the active state, however it stopped, and kept up to date by device hot add and remove. Each entry holds the function and keeps it out of runtime suspend, whose config cycles in D3cold would come back as all ones. Lending fails, and so does the start of the instance, when a function it was given does not exist here. Requests are answered from the doorbell interrupt, so that the instance can wait for the answer with interrupts off, under the PCI core's lock. Two cases go to a work item: a requester that was lent nothing, whose instance can only be looked up with a mutex, and a lending kernel that is itself an instance, whose own config space access goes over the ring and cannot be made from a message handler. Nothing asks yet: the front end of the instance follows. Signed-off-by: Cong Wang <cwang@multikernel.io>
The lending kernel passes every config write of an instance through to its function. That lets the instance move a BAR on top of another kernel's, or turn the function into SR-IOV VFs that nobody lent it. Filter writes the way xen-pciback does. The instance owns what the function does: the Command register's decode, bus master, parity, SERR and INTx disable bits, the write-one-to-clear Status bits, the cache line size, the latency timer, the interrupt line and every capability. The lending kernel keeps what places the function on the fabric: a BAR is never written. A write of anything but its value starts what looks like a sizing, and until its value is written back the BAR reads as the size of the resource the lending kernel found behind it, with its type bits, the upper half of a 64-bit BAR included. The rest of the header is read only, and so is the SR-IOV capability. A dropped write succeeds, as on a read-only register. Require MSI-X of a function to be lent. Without it the mask bits of an MSI vector are in config space, and the irq core masks around every interrupt of a threaded oneshot handler: each interrupt would cost two round trips, and would depend on the lending kernel being alive. multikernel.pci_allow_plain_msi=1 lends such a function anyway, with a warning. An entry is now counted, so that the work item that answers for a nested lender keeps the sizing state alive across the access. Signed-off-by: Cong Wang <cwang@multikernel.io>
The instance tree describes a device under every bridge on the way down from its root bus, so that the spawn kernel can reach the bus the device sits on. The spawn then enumerates those bridges as its own, though they belong to the kernel that spawned it, which has its port driver bound to them and may have other kernels' devices below them. The spawn's PCI core enables them, rewrites their windows and command registers, and sets up AER on them. A spawn has no use for the bridges. It needs to reach its devices, and to know what the bus they sit on decodes. Describe each bus that holds a device of the instance as a root bus with a bus range of that one bus, the windows of the bridge above it, or of the host bridge for a root bus, and the instance's devices on it as leaves. Bus, device and function numbers stay the real ones, so a device keeps its address, its node path and its alias in every kernel, and the parsers of the tree, which recurse into bridges, read it as before. The ECAM window given is the bus's own slice. A spawn kernel still generates the config cycles itself. Its devices are now the only functions it can enumerate, which is what lets the kernel that lent them serve their config space instead. Signed-off-by: Cong Wang <cwang@multikernel.io>
…its host An x86 spawn kernel generates config cycles itself: through the ECAM window of each bus in its tree, installed as MMCONFIG, and through the 0xCF8/0xCFC ports for whatever ECAM does not cover, its early quirks included. The port pair is shared by every kernel on the machine, which cannot lock against each other, and both mechanisms reach every function on the fabric, not just the spawn's. Add a generic front end for the buses of a spawn. It creates a root bus per bus node of the tree, as the x86 code did, with config accessors that ask the host over the message ring and poll for the answer, which works with interrupts off under the PCI core's lock. The host, which lent the devices, checks and filters each access. A request that gets no answer within 10ms fails like an access to an absent function. The ring handler is registered at arch_initcall, so that answers reach the first bus scan. The architecture supplies what its root buses need beyond that, through mk_arch_pcifront_prepare(): on x86 the pci_sysdata with the domain. x86 now sets up no config mechanism of its own in a spawn. Its probe bits are cleared before the early quirks run, and its PCI arch init installs raw accessors that go through the front end too, in place of the PCI BIOS and the direct mechanisms, so that x86 code reaching config space by address gets the same answers as the PCI core. The x86 PCI init still creates the buses, so that they exist before pcibios_init() and the IRQ setup, as before. Config space is only touched to probe, reset, power manage and recover a device; its BARs, DMA and MSI-X vectors are direct, so a running device costs no requests. Signed-off-by: Cong Wang <cwang@multikernel.io>
A spawn kernel outlives a host that crashes after boot. Its devices keep running, as their BARs, DMA and MSI-X vectors are direct, but config space goes through the host, and a dead host answers nothing. A request that goes unanswered fails only that access, as if the function were absent, and changes no state: whether the host is dead is for user space to decide, not for a timeout. User space acts on that decision by fencing the host, and a fence that confirms every host CPU parked is what switches config space to the ECAM window of each bus, the reg of its node as in the generic ECAM host bridge binding. The front end maps the window at boot and leaves it unused until then, so nothing has to be mapped after the host is gone. A bus without a window keeps failing its accesses. Reaching config space directly is safe only because the fenced host no longer generates config cycles itself; the 0xCF8 ports, which other surviving kernels may use, are never touched. A spawn that lends devices to an instance of its own passes on the window it was given, so the fallback reaches down a chain of spawns. Signed-off-by: Cong Wang <cwang@multikernel.io>
Fencing a host that panicked left the CPU that ran the panic unparked. panic() masks the PMI on that CPU but leaves the NMI watchdog counter running, and once the counter overflows, perf claims any NMI that arrives as its own. The force-halt NMI was claimed on NMI_LOCAL and never reached the force-halt handler on the unknown chain. Test the force-halt marker first on NMI_LOCAL, ahead of every local source: the marker is armed only for the duration of a force halt, so the handler claims nothing else. A spawn without a ring has no marker and parks on any unclaimed NMI, so that case stays behind perf on the unknown chain. Signed-off-by: Cong Wang <cwang@multikernel.io>
multikernel_kexec_by_id() did two jobs: reload the image, which is kexec's, and start the instance, which is not: empty its rings, lend it its PCI functions, start its CPUs, mark it active and undo the lending when the start fails. Every subsystem an instance reaches from its first instruction had to be wired into kexec_core.c. Move the second job into mk_instance_start() in the instance code, next to mk_instance_set_state(), which already takes back what an instance was given when it stops running. kexec reloads the image and hands it over. mk_instance_set_kexec_active() had no other caller and is gone. Signed-off-by: Cong Wang <cwang@multikernel.io>
…with A spawn probes only the PCI functions its tree describes, on the buses its tree describes, so a function handed to it while it runs had no node and often no bus: the device add found nothing and failed with -ENODEV. Only a function the spawn booted with could be taken away and given back. The host now sends, with the add, what it would have written into the boot tree: the function's IDs and its bus, as the bus's windows in the cells of a "ranges" property and its ECAM window. The boot tree is built from the same description, mk_pci_bus_describe(). The spawn adds the nodes it lacks with an OF changeset, which is never reverted, and then either makes the bus a root bus of its own through the path it boots with, or scans the one function on the bus it already has and claims its BARs. Buses are now added after boot, while config accessors look them up with interrupts off, so the list of buses is published with RCU and only ever added to under the PCI rescan lock. Signed-off-by: Cong Wang <cwang@multikernel.io>
A backup that outlives its host can take over the host's PCI functions only if it knows their buses: where each bus's ECAM window is and which windows it decodes. The boot tree describes only the buses of the backup's own devices, and once the host is fenced nobody is left to describe the others, as the host does when it lends a function to a running spawn. So when the host boots an instance armed with a host tree, it writes every bus it has into the instance's /chosen as multikernel,pci-buses, one node per bus with the reg and ranges a bus node of a boot tree carries, from the same mk_pci_bus_describe(). The nodes have no compatible, so the spawn makes none of them a bus until it takes a function on one. Signed-off-by: Cong Wang <cwang@multikernel.io>
Before the fence the host filters every config access of a spawn and answers only for the functions it lent, so a scan of a root bus finds the spawn's functions and nothing else. After the fence the spawn reaches each bus's ECAM window itself, with no filter left: a rescan, or a root bus made for a function taken later, finds every function on the bus, the root ports with them, and descends through those into buses that are not the spawn's, its own included. Keep the filter's effect: each bus remembers the functions of the spawn's tree, set from the bus node's children and from each function added later, and a direct access to any other reads as absent. Signed-off-by: Cong Wang <cwang@multikernel.io>
A function lent to a running spawn on a bus the spawn did not have became a root bus of its own, scanned slot by slot. The PCI core scans the other functions of a slot only when function 0 is present, which it is not when only function 2 was lent, an AHCI controller at 00:1f.2 for one: the bus came up and the function was not on it. Scan the function itself when the bus scan did not find it, as on a bus the spawn already has. Signed-off-by: Cong Wang <cwang@multikernel.io>
A spawn that outlives a crashed host could take its CPUs and memory but none of its devices: a device-add the spawn applies to itself binds a driver only to a function it already has, and the functions it lacks were only ever handed to it by the host, now dead. Once the fence has confirmed the host parked, a device-add of a function the spawn lacks takes it from what the host described when it booted the spawn: the function's IDs from the host tree user space armed it with, its bus from multikernel,pci-buses. The function then joins the spawn the way a lent one does, through mk_pcifront_add_function(), and its config space is reached through ECAM. Before the fence the function is still the host's, and the add is refused. Signed-off-by: Cong Wang <cwang@multikernel.io>
The PCI backend registers its message handler at arch_initcall, before multikernel messaging reaches subsys_initcall. Starting handler admission closed therefore rejects the PCI registration with -ESHUTDOWN and leaves the backend unable to receive forwarded configuration requests. Open admission from static initialization and let messaging init only mark the registry initialized. Cleanup still closes admission permanently, and a second init still fails. Extend the registry fixture to cover the early registration order and its cleanup boundary. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Pool control teardown now unmaps both control allocations before freeing their physical storage. The mailbox lifetime fixture still extracted only the outer teardown helper and counted frees, so it no longer modeled this retirement boundary. Extract the control-free helper, record map removals separately, and assert that quarantined slots trigger neither unmap nor free. A successful pool teardown must remove both mappings before releasing the three allocations. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Manifest generation restores the boot Device Tree and uses the early flat tree interfaces unconditionally. MULTIKERNEL could still be selected with those interfaces disabled, producing an unsupported configuration. Require OF and OF_EARLY_FLATTREE at the shared Kconfig boundary. Extend the configuration test to reject multikernel without boot Device Tree support and to verify that enabling OF restores the supported configuration. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Deferred PCI configuration work looked up its grant by instance ID and BDF when the worker ran. An instance could be retired and a replacement could reuse those identifiers first, redirecting old work and its reply into a later execution generation. Device reset could also race an admitted slow configuration access. Pin the exact grant while admitting deferred work. Revoke admission under the registry lock, then serialize the admitted access and its reply against reset with a per-grant mutex. Publish reusable instance state only after PCI retirement, and keep negative replies synchronous without deferred lookup. Add controlled fixtures for replacement-grant isolation, reply suppression, config/reset serialization, direct and deferred writes, and pin release. They model PCI and workqueue boundaries and do not exercise physical PCI or DMA quiescence. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Reserve PCI ownership before lending or publishing hotplug requests. Retain grants after unsuccessful published requests, keep pending memory removals reserved, and block instance reuse until resource uncertainty is resolved by controller reboot. Reject pool insertion of devices still assigned to another instance. Create inactive overlay records before applying effects and retain failed apply or rollback records without retrying a partial inverse. Remove the unused unsafe overlay exit path. Cover controller senders, pool admission and transaction lifetime with extracted-function host fixtures; these do not qualify remote execution or PCI hardware. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
After confirming every owned and tracked CPU parked, drain a captured inbox prefix before permitting grant or instance reuse. Dispatch PCI requests while their original grants still exist so deferred workers retain the existing pin and revocation guarantees. Return -EBUSY for unpublished slots or recursive polling. Preserve blocked slots and unrelated peer traffic, and count malformed slots within the captured bound. Cover retry, wraparound, continuous arrivals, PCI grant replacement and confirmation failure with extracted-function fixtures; cross-kernel ordering and runtime qualification remain separate. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Make pool unmap and instance memory retirement return errors. Unmap control memory before detaching region metadata, and stop destruction, re-reservation and baseline replacement on failure. Keep the record on an unexpected final-reference failure; x86 keeps its existing successful mapping removal and guards the physical free wrapper. Exercise failed unmap through the extracted production memory, baseline, reservation, destruction and final-release paths. Return CPU and device resources before memory as before; failed retirement is safe retention, not an atomic rollback of earlier resource returns. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
The first baseline clears an empty mk_self memory record while host CPUs are running. RISC-V architecture release confirms every assigned hart STOPPED, so the empty record incorrectly fails with -EBUSY. Skip architecture release only for mk_self with no pool, control mapping, regions or tracked park users. Keep the active or failed, Image and uncertain ownership refusals, and retain confirmation for every child or allocation. The extracted cleanup fixture covers initial admission and nine refusals. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
MK_SYS_SHUTDOWN is received while the raw IPI drain lock is held. Allocating and initializing work there can fail under memory pressure and adds allocator and workqueue initialization paths to hard IPI context, even though a kernel can accept only one terminal shutdown request. Use one static work item and atomically admit the first valid request. The worker drains resource and native CPU hotplug, sends the correlated reply and parks forever, so the slot never needs to reopen. Reject short requests before claiming it and coalesce all later requests without replacing the original ID. Extend the ACK fixture to cover allocator exhaustion, duplicate admission, short payloads and the permanent one-shot boundary. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Voluntary HALTED handling allocated work in IPI context and looked up the instance by ID only when that worker ran. A delayed notice could therefore settle a replacement record after ID reuse, while allocation failure could silently discard the only settlement attempt. Embed one work item and pending flag in each instance. Pin the exact instance before queueing, validate the halt result, coalesce duplicates and hold the pin until stop confirmation finishes. Keep start blocked while settlement is pending, and release the pin if queueing fails. Extend the ACK, halt settlement and start-lifetime fixtures for identity, duplicate coalescing, invalid notices, reference release and start exclusion. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Message sends allocated a temporary wire buffer with GFP_ATOMIC before taking the destination ring lock. Callers include raw-locked IPI callbacks, where allocator entry is unsafe and allocation failure can discard control replies. Reserve the ring entry first and assemble the fixed header and payload directly in that slot under the existing producer lock. Publish data_size only after both copies, preserve the wire layout and zero padding, and retain the existing reference rules for ID and held-instance routes. Extend the ACK and ring-writer fixtures for allocator exhaustion, full-size messages, null payloads, overflow and queue errors, byte layout and release publication ordering. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
CPU, memory and device resource messages allocated and initialized work items while the raw IPI drain lock was held. The shared ring already bounds accepted requests, so dynamic allocation is unnecessary and can fail in the most constrained receive context. Preinitialize separate ring-sized CPU and non-CPU work arrays. Under the hotplug raw lock, validate the payload, reserve a free slot, copy the request and account it as inflight before queueing. Release the slot only after the worker finishes mutation and reply transmission; shutdown still closes admission and waits for all accepted work. Extend the resource fixtures for saturation, slot reuse, copied payloads, allocator exclusion, queue identity and shutdown quiescence. Full capacity fails closed without overwriting an admitted request. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Nested PCI configuration requests allocated and initialized deferred work from the raw IPI receive path. Replace that allocation with a ring-sized array of preinitialized slots protected by the PCI registry lock. A slot pins the exact grant until its worker finishes access and any reply, then becomes reusable. Keep direct accesses and negative replies synchronous. Copy wire payloads into aligned local storage before reading their sequence number, reject short messages, and fail closed on slot exhaustion without taking an extra pin or overwriting older work. Extend the lifetime fixture for saturation, reuse, copied values, unaligned decoding, revocation and replacement-grant isolation. The fixture models PCI, IRQ and workqueue boundaries rather than physical device behavior. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
The network message callback allocated an skb and entered the ordinary skb queue while the raw IPI drain lock was held. Move packet ownership across that boundary with a fixed record FIFO protected by a raw lock. The callback now validates lengths and copies complete records without allocating or entering the socket stack. Let the existing worker allocate skbs and consume at most one ring's worth of records per invocation, requeueing if data remains. Drop a whole new packet on FIFO exhaustion and drop one record on allocation failure; neither case claims end-to-end delivery. Order init and exit so callbacks cannot race an absent transport or a destroyed workqueue. Add an extracted fixture for malformed input, copied lifetime, saturation and reuse, allocation failure, worker races and init or unregister errors. This does not qualify full VSOCK streams or physical hardware. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
MULTIKERNEL does not require PCI, but its local device add and remove helpers unconditionally reference pci_find_bus and pci_bus_type. Both declarations are unavailable with CONFIG_PCI=n, breaking the build. Compile the PCI-only bodies when PCI is enabled and return -EOPNOTSUPP otherwise. Keep the existing caller error propagation and resource ACK handling while preserving the PCI-enabled implementations. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
p4d_offset and pud_offset return the already-selected parent entry when their level is folded. The removal walkers add a nonzero index again, visit the wrong entry, and leave the removed range mapped in Sv39 or Sv48. Apply the index only when that level is enabled. An extracted-function fixture exercises nonzero indexes in Sv39, Sv48 and Sv57. Private QEMU pool tests on the multikernel integration candidate additionally confirm both Sv39 and Sv48 remove, map, use, unmap, return and reuse round trips; this is not a standalone non-multikernel hotplug runtime result. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
…fest Call architecture preparation and chosen-property hooks before children in the current persistent 256KB manifest tree. Keep other architectures on weak defaults and propagate preparation/emission errors without starting execution. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
The host-ipi-cpu emitter is already upstream. An absent optional target must not discard a valid host ring; preserve the legacy physical CPU zero behavior while honoring explicit sparse 64-bit targets. The extracted production restore test first failed its two legacy-host assertions and now passes all 11 checks, including allocation, mapping and publication errors. No broader mixed-peer ACK compatibility is claimed. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Combine the RISC-V architecture foundation with the reviewed generic lifecycle safety series. Keep the pool mapping implementation as separate RISC-V commits so its architecture-specific behavior remains reviewable. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Multikernel takes memory out of the host in subsection-aligned ranges. RISC-V hot-remove clears leaf mappings without splitting a larger PUD or P4D leaf, so automatic direct-map leaf selection must not exceed PMD_SIZE when MULTIKERNEL is enabled. Preserve ordinary-kernel and debug-pagealloc selection. Exercise the actual selector with Sv39, Sv48 and Sv57 flags, size and alignment boundaries, and multikernel enabled or disabled. This host fixture does not establish page-table execution or a pool-memory round trip. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Reject invalid ranges, online RAM and overlapping leaves. Allocate all page tables before publishing control mappings; on failure remove empty PTE tables without exposing partial mappings. Preflight unmap before clearing entries, preserve neighbors and flush non-leaf removals before table reuse. Retire empty PTE tables so PMD-sized RAM re-add can work; keep empty upper tables reusable. Use the existing common mapping and online-memory locks. Host fixtures execute these hooks with modeled Sv39, Sv48 and Sv57 tables, allocation failures and TLB boundaries. They do not establish a live hot-remove, map, use, unmap and hot-add round trip. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Early sparse sections retain their memmap storage when individual subsections are hot-removed. pfn_valid therefore remains true and rejects an otherwise removed pool range. Use pfn_to_online_page to reject RAM still online, under the existing online-memory lock and pool ownership contract. Keep range and existing leaf validation unchanged. Exercise removed early subsections separately from online RAM in the mapping fixture, including map and unmap refusal for online pages. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Bring the separate architecture manifest preparation hook into the local RISC-V validation stack without folding it into the lifecycle patches. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Bring the separate physical-CPU-zero fallback into the local RISC-V validation stack without making it part of the lifecycle patch series. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Bring the separate ordinary RISC-V hot-remove repair into the local pool and lifecycle validation stack without folding it into either series. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Use physical hart IDs for direct SBI HSM start/status/stop operations. Track all possible instance harts before starting any and retain their context when stop confirmation fails or STOP_PENDING times out. Prime each assigned hart through immutable host text before entering the copied fence.i stub. Publish the entry record before HART_START and preserve a0/a1 for primary and secondary boot. Document why a copied fence.i cannot protect its own first instruction fetch and why host text must stay reserved. Include the production-function HSM, entry and ownership fixtures with their formatting/extraction corrections. These are controlled host checks, not cache-coherence or native lifecycle proof. This patch must land with spawn reset containment and supported-configuration exclusions. Generic serialization/retirement and manifest hooks are separate prerequisites; multikernel#26 loading and multikernel#27/multikernel#28 boot-tree/notification work are not provided here. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Detect spawn identity after early mapping setup and before SBI reset registration. Contain halt, poweroff, restart, remote stop and crash shutdown within the spawn; never invoke host-wide SRST or legacy shutdown. For panic_timeout=0 stop the panic hart from the early notifier. For a nonzero timeout retain the generic panic restart policy, with the spawn restart override stopping locally rather than resetting the machine. Unexpected HART_STOP returns cannot authorize host-side resource reuse. Include boot-order and shutdown fixtures and the integration limitations. Leave host reset behavior unchanged. No parent notification, SBI doorbell transport, or multikernel#28 panic notification is introduced. The HSM/entry and configuration patches must land with this patch. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Require the firmware-supplied boot tree and ordered SBI HSM boot for multikernel. A built-in DTB hides manifest-based spawn detection; spinwait boot assumes bounded hart IDs and can leave a hart waiting in Image memory. Exclude BUILTIN_DTB and RISCV_BOOT_SPINWAIT in Kconfig and exercise exclusion and restoration through actual Kconfig evaluation. Document that both host and spawn need compatible builds: these gates cannot validate arbitrary foreign Image contents. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
pro-utkarshM
force-pushed
the
riscv/multikernel-23-25-v1
branch
from
November 3, 2026 20:06
eef3dcb to
d87954b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
physical hart IDs, bounded
HART_STATUSpolling, and stopped-state checksfence.istub on every start while preserving theRISC-V
a0/a1boot ABIHART_STOP; never route a spawn through SRSTAddresses #23, #24, and #25. These three changes remain together because HSM
handoff, stale-I-cache prevention, and safe shutdown share one lifecycle
contract. #26 remains a separate Image-loader follow-up.
Refreshed dependency topology
Current head:
d87954bebb4e34d5d053df5112968655abb514edThis draft is based on
multikernel-riscvand records its prerequisites asseparate signed merges so their ownership stays reviewable:
The separate #26 branch is
riscv/image-loader-submit-v44at
ec098e55e2a3799329918236033a0882cd561779.This does not depend on #7. The shared receive-ring/session limitation remains
disclosed. The private #27/#28 adapters are not folded into production history;
the public test-only integration stack remains available for reproducibility.
Validation
vmlinux+Imagelink atd87954bebb4evmlinux+Imagelink atec098e55e2a37prepare, entry, HSM, and shutdown fixtures passed
Sealed predecessor QEMU evidence covers:
run (50 boots, 49 replacements)
The v44 source differs from those reviewed/tested predecessors only by the
upstream
localversion-mkbump and a selftest-only formatting cleanup. Freshruntime was not rerun on the v44 commit IDs themselves.
Evidence boundary and draft status
The runtime results are QEMU/OpenSBI evidence, not physical-hardware proof of
stale-I-cache behavior. They do not claim full VSOCK streaming, whole-port
PREEMPT_RT, or completion of #7/#27/#28. The official SPDX parser is currently
blocked by a missing local Python
plymodule; manual SPDX-header review passedbut is not represented as an official parser result.
This remains a draft while prerequisite ordering and the separate #26 review
shape are settled. No review request is being made by this refresh.