Skip to content

kmorphd's log lines are silently dropped by the kernel's /dev/kmsg rate limit #1

Open

Description

@7z0423arsenova

What happens

On a default command line, kmorphd logs 10 lines and then nothing, including the TAKEN_OVER
transition. The takeover itself completes (CPUs and memory adopted, all vCPUs idle), only the log
is missing. The README says every transition is logged.

Cause

lib/log.c opens /dev/kmsg once and keeps the fd. The kernel rate-limits each open /dev/kmsg to
10 messages per 5 s unless printk.devkmsg=on, and drops the rest while write() still succeeds
(kernel/printk/printk.c: devkmsg_write(), per-open state from ratelimit_default_init()).

Evidence (multikernel/linux 57fd249, kmorph 9ec046e, QEMU/KVM x86_64)

Default successor cmdline = earlyprintk=serial,ttyS0,115200 keep_bootcon: exactly these 10 lines
reach the console, the 11th (adopted 0 devices ...) and everything after it do not.

kmorphd: instance successor (id 1): 1 own CPUs; host tree lists 4 CPUs, 4 RAM ranges, 4 PCI functions; ...
kmorphd: armed: probing the predecessor every 100 ms, fencing after 5 silences
kmorphd: predecessor alive
kmorphd: predecessor silent
kmorphd: state SUSPECT
kmorphd: state FENCING
kmorphd: predecessor fenced
kmorphd: state FENCED
kmorphd: error: device 0000:00:1f.2 not adopted (tx 2): Input/output error
kmorphd: error: device 0000:00:1f.3 not adopted (tx 3): Input/output error

The kernel's own log shows the adopt overlay applied, and info registers -a from the QEMU monitor
shows all four vCPUs in HLT. With printk.devkmsg=on added to the successor's cmdline, the same
run logs takeable: ..., adopted 3 CPUs and 3816 MB (tx 4) and state TAKEN_OVER.

Suggestions (none tested)

  • Have kmorph arm append printk.devkmsg=on to the successor's command line, as it already
    appends iomem=relaxed when dump is set.
  • Or open /dev/kmsg per message, which gets a fresh rate-limit state each time.
  • Or document the requirement.

Not checked: whether distros that set printk.devkmsg themselves hide this.

Activity

  1. 7z0423arsenova commented on Nov 8, 2026

    @7z0423arsenova
    Author

    It looks like this issue has already been resolved in commit 9a81047.

  2. 7z0423arsenova commented on Nov 8, 2026

    @7z0423arsenova
    Author

    I’m planning to test this takeover on a real bare-metal machine, but it still feels a bit too risky btw. :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions