feat(otel-collector): add statsdreceiver for StatsD/DogStatsD ingestion

.changeset/statsd-receiver.md

@@ -0,0 +1,10 @@
+---
+"@hyperdx/otel-collector": minor
+---
+
+feat(otel-collector): compile in statsdreceiver
+
+Available for a user's own pipeline config (e.g. via
+`CUSTOM_OTELCOL_CONFIG_FILE`) to ingest StatsD/DogStatsD metrics
+directly, without a separate StatsD-to-OTLP bridge. Purely additive:
+being compiled in changes no default pipeline or behavior on its own.

docker-compose.dev.yml

@@ -55,6 +55,12 @@ services:
       - '${HDX_DEV_OTEL_GRPC_PORT:-4317}:4317' # OTLP gRPC receiver
       - '${HDX_DEV_OTEL_HTTP_PORT:-4318}:4318' # OTLP http receiver
       - '${HDX_DEV_OTEL_METRICS_PORT:-8888}:8888' # metrics extension
+      # Uncomment to enable statsd ingestion (also requires adding the
+      # `statsd` receiver to a pipeline in custom.config.yaml - see
+      # packages/otel-collector/README.md). Left commented out by default:
+      # a fixed host port mapping fails the whole service's startup if
+      # something else on the host already owns 8125/udp.
+      # - '${HDX_DEV_OTEL_STATSD_PORT:-8125}:8125/udp' # statsd receiver
     restart: always
     networks:
       - internal

docker-compose.yml

@@ -42,6 +42,14 @@ services:
       - '4317:4317' # OTLP gRPC receiver
       - '4318:4318' # OTLP http receiver
       - '8888:8888' # metrics extension
+      # Uncomment to enable statsd ingestion (also requires adding the
+      # `statsd` receiver to a pipeline via CUSTOM_OTELCOL_CONFIG_FILE - see
+      # packages/otel-collector/README.md). Left commented out by default:
+      # a fixed host port mapping fails the whole service's startup if
+      # something else on the host (e.g. a local StatsD daemon or Datadog
+      # Agent) already owns 8125/udp, which would otherwise break the
+      # default stack for anyone not using this feature.
+      # - '8125:8125/udp' # statsd receiver
     restart: always
     networks:
       - internal

docker/hyperdx/Dockerfile

@@ -224,7 +224,7 @@ WORKDIR /app
 # Add hosts entry in entrypoint script instead of here
 
 # Expose ports
-EXPOSE 8080 4317 4318 13133 8123 9000
+EXPOSE 8080 4317 4318 13133 8123 9000 8125/udp
 
 # all-in-one no auth ############################################################################################
 FROM all-in-one-base AS all-in-one-noauth

docker/otel-collector/Dockerfile

@@ -79,7 +79,7 @@ COPY --chown=10001:10001 docker/otel-collector/config.standalone.promql.yaml /et
 COPY --chown=10001:10001 docker/otel-collector/supervisor_docker.yaml.tmpl /etc/otel/supervisor.yaml.tmpl
 COPY --chown=10001:10001 docker/otel-collector/schema /etc/otel/schema
 
-EXPOSE 4317 4318 13133
+EXPOSE 4317 4318 13133 8125/udp
 
 HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
   CMD wget -q --spider http://localhost:13133/ || exit 1
@@ -103,7 +103,7 @@ COPY --chown=10001:10001 docker/otel-collector/config.standalone.promql.yaml /et
 COPY --chown=10001:10001 docker/otel-collector/supervisor_docker.yaml.tmpl /etc/otel/supervisor.yaml.tmpl
 COPY --chown=10001:10001 docker/otel-collector/schema /etc/otel/schema
 
-EXPOSE 4317 4318 13133
+EXPOSE 4317 4318 13133 8125/udp
 
 HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
   CMD wget -q --spider http://localhost:13133/ || exit 1

packages/otel-collector/README.md

@@ -68,6 +68,7 @@ custom OTel configurations without rebuilding the collector.
 | `k8scluster`    | contrib | user configs                                            |
 | `kubeletstats`  | contrib | user configs                                            |
 | `prometheus`    | contrib | OpAMP controller, smoke tests                           |
+| `statsd`        | contrib | user configs                                            |
 
 ### Processors
 
@@ -322,6 +323,86 @@ run (`otelcontribcol --config ...`, not just `validate`): it reaches
 `span_metrics` connector, confirming `validate`'s static schema/graph
 checks aren't hiding a startup-only failure here.
 
+## Ingesting StatsD metrics
+
+`statsdreceiver` is compiled in so StatsD-formatted metrics can be ingested
+directly, without a separate StatsD-to-OTLP bridge — this covers any StatsD
+client, not just a particular vendor's dialect. It also understands the
+DogStatsD tag extension: valued tags (`#tag:value`) work with no extra
+config, but bare tags (`#mytag`, no value) need `enable_simple_tags: true`
+(set below) or they're silently dropped.
+
+**This component does not support horizontally-scaled deployments** — not
+a HyperDX-specific limitation, it's stated directly in the upstream
+component's own docs. Aggregation state lives independently in each
+collector instance's memory with no cross-replica coordination, so the
+same metric landing on two different replicas (e.g. behind a
+load-balanced Service) gets aggregated separately by each one instead of
+together, silently splitting/undercounting it. HyperDX's default
+deployments (`docker-compose.yml`, the all-in-one image) run the
+OpAMP-managed collector as a single instance, so this doesn't bite out of
+the box — it only matters once *some* collector instance ingesting
+`statsd` is scaled to multiple replicas, at which point that traffic
+needs to go to one dedicated single-replica instance instead.
+
+`docker-compose.yml`/`docker-compose.dev.yml` have an `8125/udp` port
+mapping for the `otel-collector` service, commented out by default —
+uncomment it to enable statsd ingestion. It's opt-in rather than always
+on because a fixed host port mapping fails that service's entire startup
+if something else on the host (e.g. a local StatsD daemon or Datadog
+Agent) already owns 8125/udp, which would otherwise break the default
+stack for anyone not using this feature. The standalone/all-in-one images
+also `EXPOSE 8125/udp` (metadata only, doesn't reserve the host port on
+its own).
+
+**In standalone mode**, `metrics` is a plain pipeline in your own config
+file, so add `statsd` to it directly:
+
+```yaml
+receivers:
+  statsd:
+    # Defaults to localhost:8125, which only accepts local traffic -
+    # 0.0.0.0 is required here since traffic arrives from outside the
+    # container (the published port above forwards to the container's
+    # own loopback otherwise).
+    endpoint: 0.0.0.0:8125
+    enable_simple_tags: true
+service:
+  pipelines:
+    metrics:
+      receivers: [otlp/hyperdx, statsd]
+```
+
+**In OpAMP supervisor mode** (HyperDX's default), editing the `metrics`
+pipeline this way will not work: `receivers:`/`exporters:` on it are set
+dynamically by `opampController.ts`, and the remote config overwrites
+(not merges) those keys — see the [span metrics section
+above](#computing-span-metrics-from-ingested-traces) for the same
+behavior. Adding `statsd` to `metrics.receivers` from
+`CUSTOM_OTELCOL_CONFIG_FILE` gets silently dropped: no error, the receiver
+just starts and is wired to nothing. Use a separate pipeline name instead,
+which the remote config never touches:
+
+```yaml
+receivers:
+  statsd:
+    endpoint: 0.0.0.0:8125
+    enable_simple_tags: true
+service:
+  pipelines:
+    metrics/statsd:
+      receivers: [statsd]
+      exporters: [clickhouse]
+      processors: [memory_limiter, batch]
+```
+
+Both shapes are verified against the real compiled binary via
+`otelcontribcol validate --config`, and the standalone shape was also
+actually run (`otelcontribcol --config ...`, not just `validate`),
+reaching `"Everything is ready. Begin running and processing data."` —
+the same rigor as the span metrics section above, using the same
+multi-`--config` merge the container actually uses.
+
 ## Overriding base components via `CUSTOM_OTELCOL_CONFIG_FILE`
 
 The collector ships with a default `memory_limiter` processor sized for a

packages/otel-collector/builder-config.yaml

@@ -51,6 +51,9 @@ receivers:
   - gomod:
       github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver
       v__OTEL_COLLECTOR_VERSION__
+  - gomod:
+      github.com/open-telemetry/opentelemetry-collector-contrib/receiver/statsdreceiver
+      v__OTEL_COLLECTOR_VERSION__
 
 processors:
   # Core