Skip to content

deployment: deploy streaming stack #75

Open

Description

@Tom-Willemsen

We need to make sure that our streaming software is deployed to the streaming server.

This includes:

  • kdaectrl
  • kdaediag
  • event_aggregator
  • event_udp_to_kafka
  • filewriter (once it is implemented)

Each of these pieces of software may have instrument-specific config files; we need a mechanism to define those config files and make the relevant software use them.

We want the installation steps of this software to be managed via ansible at the top-level. This ticket should output a playbook which is used for commissioning a new streaming machine and updating it.

This should be documented as an update to this ADR: https://docs.redpanda.com/streaming/current/deploy/redpanda/manual/production/production-deployment/

Note that the OS choice has now changed to Debian, please update that too.

Activity

  1. rerpha commented on Jul 14, 2026

    @rerpha
  2. moved this to research/needs discussion in Data streaming for HRPD-X & SANDALS-IIon Jul 14, 2026
  3. moved this from research/needs discussion to Ready in Data streaming for HRPD-X & SANDALS-IIon Jul 14, 2026
  4. Tom-Willemsen commented on Jul 14, 2026

    @Tom-Willemsen
    MemberAuthor

    Something to consider (though could be spun out into a separate issue if complicated): where do the config.toml files live? In a settings git repository somewhere? Or alongside the deployment scripts as static files?

  5. added this to the September 2026 milestone on Jul 20, 2026
  6. self-assigned this
    on Oct 13, 2026
  7. rerpha commented on Oct 15, 2026

    @rerpha
    Contributor

    7. Something to consider (though could be spun out into a separate issue if complicated): where do the config.toml files live? In a settings git repository somewhere? Or alongside the deployment scripts as static files?

    This can probably be spun off as a new ticket if needed but probably best done by whoever does this ticket too - it should start as a discussion which brings people involved in the data streaming stack in I think!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions