Skip to content

Support OpenAI models (API) directly #27

Closed
Closed

Description

@codefromthecrypt

Please support OpenAI models (API) directly, as that opens up many options including ollama in a way compatible with opentelemetry. LiteLLM's telemetry support is callback based, so it requires manual setup, etc. If you used the OpenAI SDK or wrote direct HTTP calls, we could get better traces than we do today.

Right now, you can carefully re-route config to litellm, but it requires more dependencies and setup.

# LiteLLM uses different ENV variables for OpenAI and OpenTelemetry fields.
os.environ["OPENAI_API_BASE"] = os.getenv("OPENAI_BASE_URL")
otlp_endpoint = os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT") + "/v1/traces"
otel_config = OpenTelemetryConfig(exporter="otlp_http", endpoint=otlp_endpoint)
litellm.callbacks = [OpenTelemetry(otel_config)]

...

agent = Agent(name=app_name, model=LiteLlm(model="openai/"+model), ...

If the openai model used normal openai, it could use the normal openai instrumentation from opentelemetry with no programmatic setup

Activity

  1. codefromthecrypt commented on Apr 10, 2025

    @codefromthecrypt
    ContributorAuthor

    cc @aabmass in case interested

  2. aabmass commented on Apr 30, 2025

    @aabmass
    Member

    Thanks for opening the discussion @codefromthecrypt. I'm wondering if this topic has been brought up with LiteLLM as well. If they had cleaner support for OTel and followed the semantic conventions, is there any benefit of using OpenAI SDK directly?

  3. codefromthecrypt commented on Apr 30, 2025

    @codefromthecrypt
    ContributorAuthor

    @aabmass made a comment here about the status quo as I understand it! BerriAI/litellm#9972 (comment)

  4. added
    models[Component] This issue is related to model support
    on May 3, 2025
  5. added this to the Feature Roadmap milestone on Jun 1, 2025
  6. self-assigned this
    on Oct 7, 2025
  7. klateefa commented on Oct 7, 2025

    @klateefa

    Hi @codefromthecrypt and thanks to everyone who participated in this thread.

    We're currently doing some housekeeping on our issue tracker. Given the rapid pace of development and the number of updates since this was last active, there's a good chance this issue has been resolved in a newer version. To help us keep our backlog focused on current issues, we are closing this as stale.

    If you're still facing this problem, could you please update to the very latest version of the library and confirm the issue still exists? If it does, please comment here, and we will be happy to reopen this issue and investigate further.

    Thanks for your contribution and understanding!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

models[Component] This issue is related to model support

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions