Skip to content

Implement UTF8 Support #916

Closed
Closed
Implement UTF8 Support#916
Assignees
ywwgfedetorres93

Description

@ywwg
ywwg
opened on Jan 22, 2024

Part of prometheus/prometheus#13095, all client libraries will need to support the new scraping, query, and content negotiation formats.

Activity

  1. ywwg commented on Jan 22, 2024

    @ywwg
    ywwg
    on Jan 22, 2024
    MemberAuthor

    @fedetorres93

  2. fedetorres93 commented on Jan 23, 2024

    @fedetorres93
    fedetorres93
    on Jan 23, 2024
    Contributor

    I'll start by implementing UTF-8 support in the Java client library

  3. jesusvazquez
    assigned
    fedetorres93
    on Jan 23, 2024

  4. fstab commented on Jan 23, 2024

    @fstab
    fstab
    on Jan 23, 2024
    Member

    @fedetorres93 thanks for volunteering, I really appreciate that!

    Is there any general guidance yet on how to implement it, for example how to convert UTF-8 names to Prometheus names for older Prometheus servers, and how to deal with potential name collisions when registering metrics?

    It would be good to define the behavior first before implementing it. Ideally the behavior would be consistent across client libraries in all programming languages.

  5. fedetorres93 commented on Jan 29, 2024

    @fedetorres93
    fedetorres93
    on Jan 29, 2024
    Contributor

    @fstab You can find the proposals @ywwg worked on here and here.

    I'm working on adding UTF-8 metric and label name validations and support for parsing and formatting the UTF-8 text format, but there's still some discussion going on about the content negotiation implementation on writes and also regarding how the reads will be handled

  6. fstab commented on Jan 30, 2024

    @fstab
    fstab
    on Jan 30, 2024
    Member

    Thanks @fedetorres93!

    There is already support for dots in metric and label names in client_java. It will be easy to extend this to other characters. The motivation for allowing dots was to support metric/label names defined in the OpenTelemetry semantic conventions.

    Currently dots are only exposed in OpenTelemetry format. In Prometheus text format, OpenMetrics text format, and OpenMetrics protobuf format dots are replaced with underscores.

    I assume for UTF-8 characters in Prometheus format we will define a new OpenMetrics version, right?

    I think the following two considerations make sense:

    • When converting OpenTelemetry names to Prometheus names follow the rules defined here: https://opentelemetry.io/docs/specs/otel/compatibility/prometheus_and_openmetrics/. These are the rules that are also implemented in the OpenTelemetry collector. For a user it should not matter whether they scrape Prometheus format, or whether they push OpenTelemetry format and have a collector convert to Prometheus remote write. The resulting metric and attribute names should be the same, therefore Prometheus client libraries should implement the OpenTelemetry standard for converting arbitrary names to Prometheus names.
    • Prometheus client libraries have a "fail fast" approach: When you register metrics with conflicting names, registration fails. We don't defer these errors to scrape time. I think we should look at the classic Prometheus names when checking for conflicts, i.e. we should fail if a user registers a metric named requests.total and then registers a metric named requests_total. While this might theoretically work when exposing new names only, it will fail at scrape time for older Prometheus servers. We should consider this bad practice and prevent this in our client libraries.

    What do you think? If you feel we should have a small "client library support for UTF-8" proposal with the points above I'm happy to write one.

  7. fedetorres93 commented on Jan 31, 2024

    @fedetorres93
    fedetorres93
    on Jan 31, 2024
    Contributor

    Thanks for the info @fstab!

    I don't think another proposal is necessary, but I appreciate the points you mentioned and will take them into account for the implementation.

  8. ywwg
    self-assigned this
    on Feb 28, 2025

  9. punya commented on May 30, 2025

    @punya
    punya
    on May 30, 2025

    @fedetorres93 are you still planning to work on this?

  10. fedetorres93 commented on Jun 3, 2025

    @fedetorres93
    fedetorres93
    on Jun 3, 2025
    Contributor

    @fedetorres93 are you still planning to work on this?

    @punya Yes, here's the PR I'm working on #1255

  11. ywwg commented on Sep 2, 2025

    @ywwg
    ywwg
    on Sep 2, 2025
    MemberAuthor

    @fedetorres93 are we done here?

  12. fedetorres93 commented on Sep 2, 2025

    @fedetorres93
    fedetorres93
    on Sep 2, 2025
    Contributor

    @ywwg Yes, we can close this

  13. zeitlinger
    closed this as completedon Sep 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions