Repository navigation
feat(observability): define gateway-to-exporter extension contract #3224
Description
User Story
As an OpenShell operator, I want a simple, supported way to send observability data to my existing tools without assembling separate collectors.
Problem Statement
The observability work under #1055 needs a shared boundary between OpenShell producing authoritative records and an exporter processing them for customer destinations.
Impact / Why This Matters
Destination-specific formats, credentials, storage, queues, and retries add work and dependencies to the gateway. An external exporter can handle that work and scale independently. The gateway still carries collection and relay costs.
Proposed Design
Supervisor → Gateway → External exporter → Customer tools
- Supervisor: emit structured signals and source-health information through its existing authenticated gateway connection. No additional exporter connection or heavy export processing.
- Gateway: expose an authenticated, read-only, scoped source that preserves native records, authoritative identity and context, and reports known gaps and recovery limits. Keep telemetry resource use bounded.
- Exporter: handle destination-specific transformations, additional redaction, buffering, retries, and delivery outside the gateway.
Make this a first-class integration with one documented installation, configuration, and health-check workflow. Operators should not need to assemble multiple collectors.
Operational delivery and optional evidence processing can be functions within the same exporter deployment. Ordinary observability must not require evidence infrastructure; detailed evidence behavior remains in #2745.
This proposal defines the shared integration boundary for work such as #2762. Transport and storage choices remain open.
Acceptance Criteria
- Structured records reach a configured destination without parsing human-readable shorthand.
- Recovery limits and known gaps are visible.
- Slow exporters or destinations do not block gateway control operations or sandbox enforcement.
- Installation, configuration, and health checks follow one supported workflow.
Alternatives Considered
Gateway-owned destination integrations reduce deployment components but expand gateway responsibilities. Independent collectors leave operators assembling the integration themselves.
Checklist
- I've reviewed existing issues and the architecture docs
- This is a design proposal, not a "please build this" request
Activity
mattj-monad commentedon Sep 30, 2026
Agree with the scope @krishicks set on #2762: OpenShell produces the events, something else handles destinations. I'd propose the contract between them be OTLP push from the gateway rather than a source an exporter pulls from.
OTEL_*headers for auth, a bounded queue with gap markers on drops. Once the collector accepts a record, it's the collector's problem.I've got a working implementation of this ~0.4 µs per event through the gateway, ~290k events/s export (dev hardware). Happy to write the RFC.
krishicks commentedon Oct 14, 2026
Closing now that #4500 and #4545 have landed. The gateway and supervisors export OCSF events over standard OTLP, so any OpenTelemetry collector can receive them and route them to an exporter or SIEM without a dedicated gateway-to-exporter contract.