Skip to content

kafka role: systemd unit 'failed', port 9092 not listening #7

Open

Description

@yakari007

Summary

Deployed a VM with the kafka service role attached (default/empty config

  • no version/node_id/heap_size override submitted), on the Ubuntu
    24.x "with PlusClouds Service" image. After SSH login succeeded, systemctl is-active kafka reported failed, and port 9092 (the default client port
    for the single combined broker+controller node this role describes)
    refused TCP connections outright. This role was given an extended 450s
    verification budget (vs the usual 300s, since it's a heavier JVM
    download+start) and still never came up.

Repro

POST /leo/iaas/virtual-machines/create-wizard with
service_roles: {"kafka": {"config": {}}} against the Ubuntu 24.x "with
PlusClouds Service" image, then after boot (waited up to 450s):

systemctl is-active kafka   # -> failed
exec 3<>/dev/tcp/127.0.0.1/9092   # -> Connection refused

The VM was deployed and torn down automatically as part of the test run,
so the live VM/journalctl -u kafka output is gone - filed from the
test's captured output, not a live inspection.

Found via leo4.tester's iaas-service-roles automated test scenario.

Activity

  1. yakari007 commented on Aug 3, 2026

    @yakari007
    ContributorAuthor

    Investigated but did not find a reproducible code-level bug, so no change made yet.

    Downloaded the actual Kafka 4.3.1 release tarball from downloads.apache.org (confirmed still hosted, matches the role's URL exactly), installed a real JDK 21 (matching the role's openjdk-21-jre-headless prerequisite), and ran the actual kafka-storage.sh format + kafka-server-start.sh sequence with the exact server.properties content the role renders. It formatted and started cleanly — KRaft broker came up, bound :9092, no crash. Kafka's JVM heap (heap_size default 1G) plus its own overhead is fairly heavy for a minimal test VM, so if this keeps reproducing, I'd suspect resource pressure (OOM/slow start) on the test image over a config bug — worth checking actual available RAM on the VM and/or journalctl -u kafka from a live instance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions