Skip to content

Record HTTP metrics once when both OpenTelemetry metrics and Prometheus are configured #312

Closed

Description

@lesnik512

The HTTP instrumentation always gets the global meter provider: FastAPIInstrumentor.instrument_app(meter_provider=get_meter_provider()) (lite_bootstrap/bootstrappers/fastapi_bootstrapper.py:197-203), Litestar's OpenTelemetryConfig(meter_provider=...) (litestar_bootstrapper.py:239-248) and FastMCP's OpenTelemetryMiddleware (fastmcp_bootstrapper.py:235-244). Once opentelemetry_metrics_endpoint installs a real MeterProvider (lite_bootstrap/instruments/opentelemetry_instrument.py:300-315), the ASGI middleware records the server duration histogram, active requests, and request and response sizes on every request. The Prometheus instrument, on by default when its extra is installed (prometheus_metrics_path="/metrics", lite_bootstrap/instruments/prometheus_instrument.py:12), measures the same requests again through prometheus-fastapi-instrumentator or Litestar's PrometheusMiddleware. That is four to five extra metric updates per request. Without opentelemetry_metrics_endpoint the middleware records into the global proxy meter provider, which is a no-op, so only the real MeterProvider case costs anything.

Measured on 2026-10-24 at 705d8a3 on Linux with 2 CPUs (CPython 3.14.9, opentelemetry 1.44.0 / contrib 0.65b0, prometheus-client 0.26.0, prometheus-fastapi-instrumentator 8.1.0), in-process, each row over OTel tracing at 0% sampling (FastStream: per message, over OTel at 0% or 100%):

OTel HTTP metrics Prometheus middleware both
Litestar +12.9 µs +15.8 µs +29.5 µs
FastAPI +12.3 µs +24.9 µs +38.3 µs
FastStream +5.2 to +6.4 µs +19.0 to +20.4 µs +26.7 µs

Keeping OpenTelemetry's set saves 16-25 µs per request or message; keeping Prometheus's saves 5-13 µs. Prometheus is the dearer side on FastAPI and FastStream. The tables and scripts are to be published with #335.

Work

  • Decide which side records HTTP metrics when both are configured, and settle it in this issue before a PR. The two candidates:
    • keep OpenTelemetry's, because opentelemetry_metrics_endpoint is an explicit opt-in and it is the cheaper set, and stop adding the Prometheus HTTP middleware while still serving /metrics for the user's own collectors and FastMCP's tool-call metrics;
    • keep Prometheus's, and pass opentelemetry.metrics.NoOpMeterProvider() to the HTTP instrumentation, leaving the OTel meter provider to the user's own instrumentors.
  • Add one field to restore both, so a user mid-migration between the two keeps them.
  • Test first: with both configured, a request moves exactly one of the two metric sets (read prometheus_client.REGISTRY and an in-memory metric reader). It fails today, since both move.
  • FastStream has the same overlap per message: opentelemetry_middleware_cls gets meter_provider=get_meter_provider() (faststream_bootstrapper.py:183-185) next to prometheus_middleware_cls. FastStream's Prometheus side is opt-in (prometheus_middleware_cls is required, see FastStream Prometheus: use the default registry and make the metrics path opt-in #321), so the overlap exists only when the user asked for both broker middlewares. Apply the same rule there or say why not.
  • Rerun benchmarks/stack_scenarios.py before and after, and record it in benchmarks/README.md.
  • Docs: a paragraph in docs/introduction/configuration.md under both Prometheus and OpenTelemetry metrics.

Verify: the new test passes, and each signal still records when it is configured alone.

Breaking only for services that set opentelemetry_metrics_endpoint and also serve Prometheus metrics: one of the two HTTP metric sets disappears unless they set the new field.

Activity

  1. added this to the 2.0.0 milestone on Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-triageMaintainer needs to evaluate this issue

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions