Repository navigation
Record HTTP metrics once when both OpenTelemetry metrics and Prometheus are configured #312
Description
The HTTP instrumentation always gets the global meter provider: FastAPIInstrumentor.instrument_app(meter_provider=get_meter_provider()) (lite_bootstrap/bootstrappers/fastapi_bootstrapper.py:197-203), Litestar's OpenTelemetryConfig(meter_provider=...) (litestar_bootstrapper.py:239-248) and FastMCP's OpenTelemetryMiddleware (fastmcp_bootstrapper.py:235-244). Once opentelemetry_metrics_endpoint installs a real MeterProvider (lite_bootstrap/instruments/opentelemetry_instrument.py:300-315), the ASGI middleware records the server duration histogram, active requests, and request and response sizes on every request. The Prometheus instrument, on by default when its extra is installed (prometheus_metrics_path="/metrics", lite_bootstrap/instruments/prometheus_instrument.py:12), measures the same requests again through prometheus-fastapi-instrumentator or Litestar's PrometheusMiddleware. That is four to five extra metric updates per request. Without opentelemetry_metrics_endpoint the middleware records into the global proxy meter provider, which is a no-op, so only the real MeterProvider case costs anything.
Measured on 2026-10-24 at 705d8a3 on Linux with 2 CPUs (CPython 3.14.9, opentelemetry 1.44.0 / contrib 0.65b0, prometheus-client 0.26.0, prometheus-fastapi-instrumentator 8.1.0), in-process, each row over OTel tracing at 0% sampling (FastStream: per message, over OTel at 0% or 100%):
| OTel HTTP metrics | Prometheus middleware | both | |
|---|---|---|---|
| Litestar | +12.9 µs | +15.8 µs | +29.5 µs |
| FastAPI | +12.3 µs | +24.9 µs | +38.3 µs |
| FastStream | +5.2 to +6.4 µs | +19.0 to +20.4 µs | +26.7 µs |
Keeping OpenTelemetry's set saves 16-25 µs per request or message; keeping Prometheus's saves 5-13 µs. Prometheus is the dearer side on FastAPI and FastStream. The tables and scripts are to be published with #335.
Work
- Decide which side records HTTP metrics when both are configured, and settle it in this issue before a PR. The two candidates:
- keep OpenTelemetry's, because
opentelemetry_metrics_endpointis an explicit opt-in and it is the cheaper set, and stop adding the Prometheus HTTP middleware while still serving/metricsfor the user's own collectors and FastMCP's tool-call metrics; - keep Prometheus's, and pass
opentelemetry.metrics.NoOpMeterProvider()to the HTTP instrumentation, leaving the OTel meter provider to the user's own instrumentors.
- keep OpenTelemetry's, because
- Add one field to restore both, so a user mid-migration between the two keeps them.
- Test first: with both configured, a request moves exactly one of the two metric sets (read
prometheus_client.REGISTRYand an in-memory metric reader). It fails today, since both move. - FastStream has the same overlap per message:
opentelemetry_middleware_clsgetsmeter_provider=get_meter_provider()(faststream_bootstrapper.py:183-185) next toprometheus_middleware_cls. FastStream's Prometheus side is opt-in (prometheus_middleware_clsis required, see FastStream Prometheus: use the default registry and make the metrics path opt-in #321), so the overlap exists only when the user asked for both broker middlewares. Apply the same rule there or say why not. - Rerun
benchmarks/stack_scenarios.pybefore and after, and record it inbenchmarks/README.md. - Docs: a paragraph in
docs/introduction/configuration.mdunder both Prometheus and OpenTelemetry metrics.
Verify: the new test passes, and each signal still records when it is configured alone.
Breaking only for services that set opentelemetry_metrics_endpoint and also serve Prometheus metrics: one of the two HTTP metric sets disappears unless they set the new field.
Activity