diff --git a/.changeset/statsd-receiver.md b/.changeset/statsd-receiver.md new file mode 100644 index 0000000000..af8cb78bf9 --- /dev/null +++ b/.changeset/statsd-receiver.md @@ -0,0 +1,10 @@ +--- +"@hyperdx/otel-collector": minor +--- + +feat(otel-collector): compile in statsdreceiver + +Available for a user's own pipeline config (e.g. via +`CUSTOM_OTELCOL_CONFIG_FILE`) to ingest StatsD/DogStatsD metrics +directly, without a separate StatsD-to-OTLP bridge. Purely additive: +being compiled in changes no default pipeline or behavior on its own. diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index 54200ca08a..9c3c0491ec 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -55,6 +55,12 @@ services: - '${HDX_DEV_OTEL_GRPC_PORT:-4317}:4317' # OTLP gRPC receiver - '${HDX_DEV_OTEL_HTTP_PORT:-4318}:4318' # OTLP http receiver - '${HDX_DEV_OTEL_METRICS_PORT:-8888}:8888' # metrics extension + # Uncomment to enable statsd ingestion (also requires adding the + # `statsd` receiver to a pipeline in custom.config.yaml - see + # packages/otel-collector/README.md). Left commented out by default: + # a fixed host port mapping fails the whole service's startup if + # something else on the host already owns 8125/udp. + # - '${HDX_DEV_OTEL_STATSD_PORT:-8125}:8125/udp' # statsd receiver restart: always networks: - internal diff --git a/docker-compose.yml b/docker-compose.yml index 5a02572925..ada6f5c0d9 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -42,6 +42,14 @@ services: - '4317:4317' # OTLP gRPC receiver - '4318:4318' # OTLP http receiver - '8888:8888' # metrics extension + # Uncomment to enable statsd ingestion (also requires adding the + # `statsd` receiver to a pipeline via CUSTOM_OTELCOL_CONFIG_FILE - see + # packages/otel-collector/README.md). Left commented out by default: + # a fixed host port mapping fails the whole service's startup if + # something else on the host (e.g. a local StatsD daemon or Datadog + # Agent) already owns 8125/udp, which would otherwise break the + # default stack for anyone not using this feature. + # - '8125:8125/udp' # statsd receiver restart: always networks: - internal diff --git a/docker/hyperdx/Dockerfile b/docker/hyperdx/Dockerfile index 74f3cc7c77..6a3321822a 100644 --- a/docker/hyperdx/Dockerfile +++ b/docker/hyperdx/Dockerfile @@ -224,7 +224,7 @@ WORKDIR /app # Add hosts entry in entrypoint script instead of here # Expose ports -EXPOSE 8080 4317 4318 13133 8123 9000 +EXPOSE 8080 4317 4318 13133 8123 9000 8125/udp # all-in-one no auth ############################################################################################ FROM all-in-one-base AS all-in-one-noauth diff --git a/docker/otel-collector/Dockerfile b/docker/otel-collector/Dockerfile index a334d9150e..18355cd66e 100644 --- a/docker/otel-collector/Dockerfile +++ b/docker/otel-collector/Dockerfile @@ -79,7 +79,7 @@ COPY --chown=10001:10001 docker/otel-collector/config.standalone.promql.yaml /et COPY --chown=10001:10001 docker/otel-collector/supervisor_docker.yaml.tmpl /etc/otel/supervisor.yaml.tmpl COPY --chown=10001:10001 docker/otel-collector/schema /etc/otel/schema -EXPOSE 4317 4318 13133 +EXPOSE 4317 4318 13133 8125/udp HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \ CMD wget -q --spider http://localhost:13133/ || exit 1 @@ -103,7 +103,7 @@ COPY --chown=10001:10001 docker/otel-collector/config.standalone.promql.yaml /et COPY --chown=10001:10001 docker/otel-collector/supervisor_docker.yaml.tmpl /etc/otel/supervisor.yaml.tmpl COPY --chown=10001:10001 docker/otel-collector/schema /etc/otel/schema -EXPOSE 4317 4318 13133 +EXPOSE 4317 4318 13133 8125/udp HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \ CMD wget -q --spider http://localhost:13133/ || exit 1 diff --git a/packages/otel-collector/README.md b/packages/otel-collector/README.md index 3834b1be76..d200712ccf 100644 --- a/packages/otel-collector/README.md +++ b/packages/otel-collector/README.md @@ -68,6 +68,7 @@ custom OTel configurations without rebuilding the collector. | `k8scluster` | contrib | user configs | | `kubeletstats` | contrib | user configs | | `prometheus` | contrib | OpAMP controller, smoke tests | +| `statsd` | contrib | user configs | ### Processors @@ -322,6 +323,86 @@ run (`otelcontribcol --config ...`, not just `validate`): it reaches `span_metrics` connector, confirming `validate`'s static schema/graph checks aren't hiding a startup-only failure here. +## Ingesting StatsD metrics + +`statsdreceiver` is compiled in so StatsD-formatted metrics can be ingested +directly, without a separate StatsD-to-OTLP bridge — this covers any StatsD +client, not just a particular vendor's dialect. It also understands the +DogStatsD tag extension: valued tags (`#tag:value`) work with no extra +config, but bare tags (`#mytag`, no value) need `enable_simple_tags: true` +(set below) or they're silently dropped. + +**This component does not support horizontally-scaled deployments** — not +a HyperDX-specific limitation, it's stated directly in the upstream +component's own docs. Aggregation state lives independently in each +collector instance's memory with no cross-replica coordination, so the +same metric landing on two different replicas (e.g. behind a +load-balanced Service) gets aggregated separately by each one instead of +together, silently splitting/undercounting it. HyperDX's default +deployments (`docker-compose.yml`, the all-in-one image) run the +OpAMP-managed collector as a single instance, so this doesn't bite out of +the box — it only matters once *some* collector instance ingesting +`statsd` is scaled to multiple replicas, at which point that traffic +needs to go to one dedicated single-replica instance instead. + +`docker-compose.yml`/`docker-compose.dev.yml` have an `8125/udp` port +mapping for the `otel-collector` service, commented out by default — +uncomment it to enable statsd ingestion. It's opt-in rather than always +on because a fixed host port mapping fails that service's entire startup +if something else on the host (e.g. a local StatsD daemon or Datadog +Agent) already owns 8125/udp, which would otherwise break the default +stack for anyone not using this feature. The standalone/all-in-one images +also `EXPOSE 8125/udp` (metadata only, doesn't reserve the host port on +its own). + +**In standalone mode**, `metrics` is a plain pipeline in your own config +file, so add `statsd` to it directly: + +```yaml +receivers: + statsd: + # Defaults to localhost:8125, which only accepts local traffic - + # 0.0.0.0 is required here since traffic arrives from outside the + # container (the published port above forwards to the container's + # own loopback otherwise). + endpoint: 0.0.0.0:8125 + enable_simple_tags: true +service: + pipelines: + metrics: + receivers: [otlp/hyperdx, statsd] +``` + +**In OpAMP supervisor mode** (HyperDX's default), editing the `metrics` +pipeline this way will not work: `receivers:`/`exporters:` on it are set +dynamically by `opampController.ts`, and the remote config overwrites +(not merges) those keys — see the [span metrics section +above](#computing-span-metrics-from-ingested-traces) for the same +behavior. Adding `statsd` to `metrics.receivers` from +`CUSTOM_OTELCOL_CONFIG_FILE` gets silently dropped: no error, the receiver +just starts and is wired to nothing. Use a separate pipeline name instead, +which the remote config never touches: + +```yaml +receivers: + statsd: + endpoint: 0.0.0.0:8125 + enable_simple_tags: true +service: + pipelines: + metrics/statsd: + receivers: [statsd] + exporters: [clickhouse] + processors: [memory_limiter, batch] +``` + +Both shapes are verified against the real compiled binary via +`otelcontribcol validate --config`, and the standalone shape was also +actually run (`otelcontribcol --config ...`, not just `validate`), +reaching `"Everything is ready. Begin running and processing data."` — +the same rigor as the span metrics section above, using the same +multi-`--config` merge the container actually uses. + ## Overriding base components via `CUSTOM_OTELCOL_CONFIG_FILE` The collector ships with a default `memory_limiter` processor sized for a diff --git a/packages/otel-collector/builder-config.yaml b/packages/otel-collector/builder-config.yaml index dbe2f94f91..b0b7b3834c 100644 --- a/packages/otel-collector/builder-config.yaml +++ b/packages/otel-collector/builder-config.yaml @@ -51,6 +51,9 @@ receivers: - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v__OTEL_COLLECTOR_VERSION__ + - gomod: + github.com/open-telemetry/opentelemetry-collector-contrib/receiver/statsdreceiver + v__OTEL_COLLECTOR_VERSION__ processors: # Core