Repository navigation
feat(observability): OCSF event export from gateway to external SIEM destinations #2762
Description
User Story
As a security or platform operator running OpenShell in production, I want the gateway to export OCSF security events to an external SIEM or log aggregator so that sandbox activity appears in my existing security operations workflows without a custom intermediary.
Problem Statement
OpenShell already relays OCSF events from the sandbox supervisor to the gateway via the PushSandboxLogs gRPC RPC. The gateway collects them in an in-memory TracingLogBus (a 2000-line ring buffer per sandbox) and makes them available through openshell logs and WatchSandbox. The relay chain ends there. No mechanism exists to export events from the gateway to an external destination such as a SIEM, log aggregator, or persistent store.
As a result, operators must build and maintain an external bridge that reads from openshell logs output and forwards it. A PoC confirmed this path works (parsing the shorthand text stream and forwarding to Splunk HEC), but it is a workaround: the bridge must stay running alongside the gateway, re-parses already-structured events from their text representation, and provides no backpressure or delivery guarantees.
Impact / Why This Matters
Without a native export surface, OCSF events are ephemeral. They exist only in the gateway's in-memory buffer for the lifetime of the sandbox and are not retained after bus.remove() is called. For security compliance workflows — including incident investigation, SAFE evidence preservation, and audit reporting — events that exist only in memory are not evidence.
The openshell logs workaround requires a persistent sidecar process with its own failure mode, adds an unnecessary parse-and-reserialize step on already-structured data, and cannot provide delivery guarantees or backpressure signaling. It also ties export behavior to the CLI interface rather than the gateway's operational configuration.
The gateway already holds all OCSF events from all sandboxes. It is the correct place to export them.
Proposed Design
Add a configurable OCSF export destination to the gateway. The minimal viable form is a structured log file or a configurable HTTP endpoint (Splunk HEC, OpenSearch, or a generic webhook):
[openshell.gateway.ocsf_export]
destination = "http://splunk-hec:8088/services/collector"
token_file = "/etc/openshell/hec-token"
format = "jsonl" # jsonl | splunk_hec
batch_size = 100
flush_ms = 500The exporter should consume from TracingLogBus, not from openshell logs, to avoid re-parsing structured events. It should operate with backpressure (drop with counter, not block) consistent with the existing LogPushLayer semantics. On gateway restart, events already flushed need not be re-sent; the exporter need not guarantee exactly-once delivery.
For operators using OpenShift Logging or a Vector/Fluent Bit daemonset, a structured JSONL file destination (written by the gateway, picked up by the log collector) is an equally valid alternative and avoids the gateway needing outbound HTTP credentials.
The export surface is orthogonal to schema version selection (#2662) and trace correlation (#2640), but composes with both: exported events should carry trace_id/span_id when available, and should respect the configured ocsf_schema_version.
Acceptance Criteria
- OCSF events emitted by sandboxes on a configured gateway reach the configured external destination without a custom intermediary process.
- The exporter operates with backpressure: when the destination is unavailable, events are dropped with a counter rather than blocking the gateway.
- Exported events are structured OCSF JSON, not re-parsed shorthand text.
- The feature composes with feat(ocsf): add trace_id/span_id correlation fields to OCSF event builders #2640 (trace correlation fields) and feat: Configurable OCSF schema version for SIEM compatibility (v1.1/v1.3) #2662 (schema version selection).
- When no export destination is configured, behavior is unchanged.
Agent Investigation
crates/openshell-supervisor-process/src/log_push.rs—LogPushLayerpushes OCSF shorthand events from sandbox to gateway viaPushSandboxLogsgRPC RPC.crates/openshell-server/src/tracing_bus.rs—TracingLogBusis the gateway-side in-memory broadcast bus.bus.remove()drops all events on sandbox teardown.- Related: Enterprise Observability #1055 (Enterprise Observability umbrella), Centralized audit/event log #1933 (audit/event log placeholder), feat(observability): OpenTelemetry export surface for gateway operators #2507 (OTEL gateway export surface — explicitly excludes OCSF), feat(ocsf): add trace_id/span_id correlation fields to OCSF event builders #2640 (trace correlation fields), feat: Configurable OCSF schema version for SIEM compatibility (v1.1/v1.3) #2662 (OCSF schema version selection), feat(observability): export an atomic sandbox governance evidence bundle #2745 (sandbox evidence bundle).
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
krishicks commentedon Sep 21, 2026
This is not planned as it goes beyond the scope of what we want to put in OpenShell. OpenShell should produce a durable source of OCSF events that something else can slurp and export as appropriate, but that's it (for now).
feat(server): write gateway OCSF events to JSONL
feat(server): write gateway OCSF events to JSONL
feat(server): write gateway OCSF events to JSONL
feat(server): write gateway OCSF events to JSONL
feat(server): write gateway OCSF events to JSONL (NVIDIA#3264)
feat(helm): configure gateway OCSF JSONL output
feat(helm): configure gateway OCSF JSONL output
feat(helm): configure gateway OCSF JSONL output
feat(helm): configure gateway OCSF JSONL output (NVIDIA#3876)