Repository navigation
deployment: Initial infrastructure/documentation for streaming servers #72
Description
Write a first-pass set of documentation and ansible playbooks to make the necessary software run on the necessary hardware.
This should follow previously-agreed ADR for deployment/infrastructure on HRPDX/SANDALS2/similar machines: https://isiscomputinggroup.github.io/ibex_developers_manual/specific_iocs/datastreaming/ADRs/005_os_deployment.html#decision
Acceptance criteria
- Physical installation steps documented (e.g. physical network connections)
- Steps necessary in iDRAC documented
- OS installation steps documented
- First-pass deployment infrastructure written using ansible - which should take over once base OS installation is present and accessible over SSH.
High-level goal is that if we have a brand-new-in-box server, anyone on the team could follow these instructions/playbooks step-by-step and end up with a fully-functional data streaming machine in the style planned for HRPDX/SANDALS2.
Metadata
Metadata
Assignees
Labels
No labels
Activity
[-]Initial deployment infrastructure/documentation for streaming servers[/-][+]deployment: Initial infrastructure/documentation for streaming servers[/+]Tom-Willemsen commentedon Jun 23, 2026
One fun thing is that the redpanda ansible collection really doesn't expect you to be deploying 3 separate redpanda clusters (it expects all hosts for a single cluster in a top-level
redpandagroup in ansible inventory, among other things). I initially got very confused why the server I was testing with was trying to join the old ESS cluster...Might need to either hack/overwrite the config the redpanda collection generates, or call some of the redpanda deploy steps more manually (still from ansible, but perhaps not using their top-level playbooks).
The redpanda collection also assumes RAID0 (striping) across SSDs, we bought multiple SSDs for reliability so that we can RAID1 (mirror) them. Again this is possible to work around, or we may call some of the low-level steps ourselves from ansible rather than using the redpanda playbooks directly.
Other option is to run redpanda in a container, I know various teams previously had issues running it virtualised, and ended up with it on bare-metal, but on native linux a container isn't a virtualisation layer so may work better.
Tom-Willemsen commentedon Jun 26, 2026
I think I will limit this issue to just the basic OS configuration + SSH configuration + RAID array creation - we can have separate issues to discuss how exactly we're deploying redpanda and other software.
PRs:
Follow-on issues:
rerpha commentedon Jun 30, 2026
I think i prefer this and it fits in well with how we're going to deploy our own software. They say that it's not officially supported as a production redpanda instance but then they'd probably also say a single node is a bad idea...