Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Release
on:
push:
tags:
- v*

permissions: {}

jobs:
release:
name: Build and publish GitHub release
runs-on: ubuntu-latest
permissions:
contents: write
id-token: write
attestations: write
env:
UV_LOCKED: '1'
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
with:
python-version: '3.13'
enable-cache: false
- name: Install Task
uses: go-task/setup-task@a00fbb05ce67b35648be3c78cbc9fd85354c757e # v2.2.0
with:
repo-token: ${{ github.token }}
- name: Check the tag matches the package version
env:
TAG: ${{ github.ref_name }}
run: |
version=$(uv version --short)
if [ "v${version}" != "${TAG}" ]; then
echo "::error::Tag ${TAG} does not match pyproject version ${version}"
exit 1
fi
- name: Install dependencies
run: task install
- name: Format, lint, and type checks
run: task check
- name: Tests
run: task test
- name: Build
run: uv build
- uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: dist/*
- name: Extract release notes from the changelog
env:
TAG: ${{ github.ref_name }}
run: |
awk -v version="${TAG#v}" '
$0 ~ "^## \\[" version "\\]" { found = 1; next }
found && /^## / { exit }
found && !/^\[[^]]+\]: / { print }
' CHANGELOG.md > release-notes.md
if ! grep -q '[^[:space:]]' release-notes.md; then
echo "::error::CHANGELOG.md has no section for ${TAG}"
exit 1
fi
- name: Publish the GitHub release
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ github.ref_name }}
run: gh release create "${TAG}" dist/* --verify-tag --title "${TAG}" --notes-file release-notes.md
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,17 @@

All notable changes to this package are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project uses [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Added

- Shorter README with a module overview and a limitations section.
- Tag-triggered release workflow that checks the tag against the package version, runs the checks, builds and attests the wheel and sdist, and publishes the GitHub release with notes from this changelog.

### Changed

- Relaxed the Langfuse requirement from an exact pin to the patch releases of the tested minor version (`langfuse>=5.2.0,<5.3`).

## [0.4.0] - 2026-11-15

### Added
Expand Down Expand Up @@ -39,4 +50,5 @@ All notable changes to this package are documented here. The format follows [Kee
- Replaced type-specific score convenience methods with the typed `Score` and `ScoreTarget` API.
- Limited support to Python 3.13–3.14 and Langfuse 4.14.2.

[Unreleased]: https://github.com/DigintLab/digintlab-langfuse-python/compare/v0.4.0...HEAD
[0.4.0]: https://github.com/DigintLab/digintlab-langfuse-python/releases/tag/v0.4.0
114 changes: 39 additions & 75 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,108 +1,79 @@
# digintlab-langfuse-python

Typed Python helpers for Langfuse tracing, scoring, callbacks, and LiteLLM metadata in DigIntLab agent runtimes.
Typed helpers for tracing, scoring, and redacting agent runs with Langfuse.

## Install

Requires Python 3.13, 3.14, or 3.15. Langfuse is pinned to the version tested by this package.

The package is not published on PyPI. Install it from a tagged GitHub release (replace `v0.4.0` with the release you want):
Python 3.13 to 3.15. Not on PyPI; install a tagged release from GitHub:

```bash
uv add "digintlab-langfuse-python @ git+https://github.com/DigintLab/digintlab-langfuse-python@v0.4.0"
```

With pip:

```bash
pip install "digintlab-langfuse-python @ git+https://github.com/DigintLab/digintlab-langfuse-python@v0.4.0"
```

Each [GitHub release](https://github.com/DigintLab/digintlab-langfuse-python/releases) also attaches a wheel and a source distribution, which can be installed directly:

```bash
uv add "digintlab-langfuse-python @ https://github.com/DigintLab/digintlab-langfuse-python/releases/download/v0.4.0/digintlab_langfuse_python-0.4.0-py3-none-any.whl"
```
Add the `[redaction]` extra for PII detection. Each [release](https://github.com/DigintLab/digintlab-langfuse-python/releases) also ships a wheel and an sdist.

## Usage

```python
from digintlab_langfuse import LangfuseClientConfig, init_langfuse_telemetry
from digintlab_langfuse.scores import NumericScore, ScoreTarget

telemetry = init_langfuse_telemetry(
LangfuseClientConfig(
public_key=public_key,
secret_key=secret_key,
environment="production",
release=release,
redact_http_auth_headers=True,
),
instrument_httpx=True,
LangfuseClientConfig(public_key=public_key, secret_key=secret_key, environment="production"),
)

async with telemetry:
with telemetry.trace_call(
name="agent-run",
session_id=session_id,
user_id=user_id,
root_observation=True,
):
with telemetry.trace_call(name="agent-run", session_id=session_id, user_id=user_id, root_observation=True):
await run_agent()

telemetry.emit_score(score=NumericScore(name="quality", value=0.95), target=ScoreTarget(trace_id=trace_id))
```

Scores use one typed API rather than type-specific convenience methods:
Create one telemetry owner per process, after workers fork, and don't construct `Langfuse(...)` yourself. Leaving the `async with` block flushes and shuts down; `await telemetry.flush()` exports without closing.

```python
from digintlab_langfuse.scores import NumericScore, ScoreTarget
To continue a trace in another process, capture `telemetry.trace_runtime_context()` and pass it to `with_litellm_proxy_trace_metadata()` on the other side.

telemetry.emit_score(
score=NumericScore(name="quality", value=0.95),
target=ScoreTarget(trace_id=trace_id),
)
```
## Modules

Important behavior:
- `client`: `LangfuseClientConfig`, `init_langfuse_telemetry`, and `LangfuseTelemetry` (`trace_call`, `flush`, `shutdown`, and wrappers for the helpers below).
- `spans`: node, generation, and guardrail observations; the HTTPX auth-header scrubbing hooks.
- `callbacks`: `ToolSpanCallback`, a LangChain handler that records tool calls.
- `litellm`: adds the active trace ids to LiteLLM proxy request metadata.
- `scores`: one typed score API (`NumericScore`, `BooleanScore`, `CategoricalScore`, `TextScore`, `CorrectionScore`).
- `prompts`, `experiments`: thin typed wrappers for Langfuse prompts, datasets, and local experiments.
- `budgets`, `timing`: numeric ceiling checks and span timing aggregation.
- `redaction`, `json_values`: the redaction policy and the JSON value type.

- Create one telemetry owner per process, after workers fork. Do not also construct `Langfuse(...)` directly.
- HTTPX and botocore instrumentation are opt-in. Instrumentation owned by other libraries is left untouched; initialization fails when HTTPX is externally instrumented and authorization-header redaction is requested because the redaction hooks cannot be guaranteed.
- Observation helpers fail open so telemetry errors do not replace application results or exceptions. Configuration, scoring, flush, and shutdown fail fast.
- Use `await telemetry.flush()` to export pending data without closing the owner. The async context manager shuts it down.
- Capture `telemetry.trace_runtime_context()` before sending work across a process or queue boundary. Pass it to `with_litellm_proxy_trace_metadata()` in the worker.
- Plain HTTP endpoints require `allow_insecure_http=True`.
For all configuration fields and their defaults, see `LangfuseClientConfig` in [`client.py`](src/digintlab_langfuse/client.py).

## Sensitive data

PII redaction is optional:

```bash
uv add "digintlab-langfuse-python[redaction] @ git+https://github.com/DigintLab/digintlab-langfuse-python@v0.4.0"
```

With pip, pass the same requirement string to `pip install`.
Nothing is redacted by default. To mask credential fields, bearer tokens, and PII before export:

```python
from digintlab_langfuse import LangfuseClientConfig, SensitiveDataRedactionConfig

config = LangfuseClientConfig(
public_key=public_key,
secret_key=secret_key,
sensitive_data_redaction=SensitiveDataRedactionConfig(
pii_entities=frozenset({"EMAIL_ADDRESS", "IP_ADDRESS", "PHONE_NUMBER"}),
pii_score_threshold=0.4,
redact_bearer_tokens=True,
),
sensitive_data_redaction=SensitiveDataRedactionConfig(),
)
```

The policy masks common credential fields and bearer tokens, then uses Presidio's local rule-based recognizers for PII. It uses Presidio's bundled domain-suffix snapshot and does not load an NLP model, call an LLM, or make network requests. Configure `pii_entities` and `pii_score_threshold` for the data you expect; use `mask` and `mask_otel_spans` for a fully custom policy.
PII detection uses Presidio's local rule-based recognizers. It loads no NLP model and makes no network calls, so it won't catch names, addresses, or non-English text. For a custom policy, pass `mask` and `mask_otel_spans` instead.

To strip `Authorization` headers from HTTPX spans, set `redact_http_auth_headers=True` and initialize with `instrument_httpx=True`.

> [!WARNING]
> Do not put sensitive data in span names, identifiers, links, events, or resource attributes. OpenTelemetry export hooks cannot rewrite them.
>
> `SensitiveDataRedactionConfig` also does not sanitize metadata forwarded to LiteLLM. Keep runtime-context, run-manager, and request metadata non-sensitive,
> or sanitize it before calling `with_litellm_proxy_trace_metadata()`.
> Span names, identifiers, links, events, resource attributes, and metadata forwarded to LiteLLM are never rewritten. Keep secrets out of them.

## Limitations

HTTP authorization-header redaction is separate. Enable both `instrument_httpx=True` and `redact_http_auth_headers=True`. Tool callbacks capture inputs and outputs unless you provide `summarize_output` or enable masking.
- One Langfuse project per process. A second `init_langfuse_telemetry()` raises, and `shutdown()` does not release the claim.
- The package uses private Langfuse SDK internals, so Langfuse is limited to one minor version (`>=5.2.0,<5.3`). `tests/package_contract/test_langfuse_internals.py` catches breaking changes.
- Observation helpers fail open: errors are logged and never replace your result. Configuration, scoring, flush, and shutdown raise.
- HTTPX and botocore instrumentation is opt-in and process-wide. Instrumentation owned by other code is left alone.
- Plain `http://` endpoints need `allow_insecure_http=True`.

## Development

Expand All @@ -113,24 +84,17 @@ task check
task test
```

Run the authenticated E2E suite only against a disposable Langfuse project:
The E2E suite needs a disposable Langfuse project:

```bash
RUN_LANGFUSE_E2E=1 \
LANGFUSE_E2E_PUBLIC_KEY=... \
LANGFUSE_E2E_SECRET_KEY=... \
LANGFUSE_E2E_BASE_URL=https://cloud.langfuse.com \
task test-e2e
RUN_LANGFUSE_E2E=1 LANGFUSE_E2E_PUBLIC_KEY=... LANGFUSE_E2E_SECRET_KEY=... \
LANGFUSE_E2E_BASE_URL=https://cloud.langfuse.com task test-e2e
```

## API stability

The public API is the names listed in each public module's `__all__` plus the top-level `digintlab_langfuse` exports. Modules and names that start with an underscore are private.

Versioning follows [SemVer](https://semver.org/). Before 1.0, a minor version (0.x to 0.(x+1)) may contain breaking changes, which are listed in `CHANGELOG.md` under "Changed" or "Removed". Patch versions are backward compatible.
## Versioning

Langfuse is pinned to an exact tested version, and a pin change is released as at least a minor version.
The public API is each public module's `__all__` plus the top-level exports; anything starting with an underscore is private. Releases follow [SemVer](https://semver.org/). Before 1.0, minor releases may break things, and those changes are listed in `CHANGELOG.md`. Moving to a new Langfuse minor version is always at least a minor release.

## License

This package is licensed under the GNU Lesser General Public License v3.0 or later (`LGPL-3.0-or-later`). You may use it in any application, including proprietary applications that import the library unmodified. If you distribute a modified version of the library itself, those modifications must be made available under the LGPL. See [`COPYING.LESSER`](COPYING.LESSER) for the LGPL terms and [`COPYING`](COPYING) for the GNU GPL v3 text it builds on.
[LGPL-3.0-or-later](COPYING.LESSER). You can use the library in any application, proprietary ones included. If you distribute a modified copy of the library itself, you must share those changes under the LGPL.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ dependencies = [
"botocore>=1.43.82,<2",
"httpx>=0.28.1,<1",
"langchain-core>=1.6.1,<2",
"langfuse==5.2.0",
"langfuse>=5.2.0,<5.3",
"opentelemetry-api>=1.44.0,<2",
"opentelemetry-instrumentation-botocore>=0.65b0,<1",
"opentelemetry-instrumentation-httpx>=0.65b0,<1",
Expand Down
103 changes: 103 additions & 0 deletions tests/package_contract/test_langfuse_internals.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
"""Contract tests for the private Langfuse SDK internals this package relies on.

Each failure names the attribute, the dependent module, and the required action so a
Langfuse version bump surfaces breakage here instead of at runtime.
"""

import importlib
import threading
from uuid import uuid4

from langfuse import Langfuse, get_client
from langfuse._client.resource_manager import LangfuseResourceManager
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter

from digintlab_langfuse import LangfuseClientConfig
from digintlab_langfuse.client import _init_langfuse_client

_UPGRADE_HINT = "update digintlab_langfuse before bumping langfuse"


def _message(attribute: str, module: str, expectation: str) -> str:
return f"{attribute} {expectation}; digintlab_langfuse.{module} depends on it: {_UPGRADE_HINT}."


def _build_client() -> tuple[str, Langfuse]:
public_key = f"pk-internals-{uuid4()}"
client = _init_langfuse_client(
LangfuseClientConfig(
public_key=public_key,
secret_key="internals-secret",
span_exporter=InMemorySpanExporter(),
)
)
return public_key, client


def test_resource_manager_lock_is_a_reentrant_context_manager() -> None:
lock = LangfuseResourceManager._lock
expectation = "must be a reentrant lock usable as a context manager (initialize_client holds it while Langfuse() re-acquires it)"
message = _message("LangfuseResourceManager._lock", "_lifecycle:58", expectation)

assert isinstance(lock, type(threading.RLock())), message
with lock, lock:
pass


def test_resource_manager_instances_is_a_dict_keyed_by_public_key() -> None:
public_key, _client = _build_client()
instances = LangfuseResourceManager._instances

assert isinstance(instances, dict), _message("LangfuseResourceManager._instances", "_lifecycle:74", "must be a dict")
assert public_key in instances, _message("LangfuseResourceManager._instances", "_lifecycle:83", "must be keyed by the client's public key")
assert isinstance(instances[public_key], LangfuseResourceManager), _message(
"LangfuseResourceManager._instances", "_lifecycle:83", "must map public keys to LangfuseResourceManager"
)


def test_resource_manager_instances_supports_key_deletion() -> None:
public_key, _client = _build_client()
instances = LangfuseResourceManager._instances

with LangfuseResourceManager._lock:
del instances[public_key]

assert not instances, _message("LangfuseResourceManager._instances", "_lifecycle:84", "must support `del` by public key")


def test_langfuse_resources_attribute_is_the_shared_resource_manager() -> None:
public_key, client = _build_client()
alias = get_client(public_key=public_key)

assert hasattr(client, "_resources"), _message("Langfuse._resources", "_lifecycle:38 and spans:131", "must exist on Langfuse")
assert isinstance(client._resources, LangfuseResourceManager), _message(
"Langfuse._resources", "_lifecycle:38 and spans:131", "must be a LangfuseResourceManager"
)
assert alias is not client, _message("get_client()", "_lifecycle:38", "must return a distinct facade for an existing key")
assert alias._resources is client._resources, _message(
"Langfuse._resources", "_lifecycle:38 and spans:131", "must be shared by every facade of one public key"
)


def test_resource_manager_shutdown_flag_and_method() -> None:
_public_key, client = _build_client()
resources = client._resources

assert resources is not None
flag = resources._shutdown
assert flag is False, _message("LangfuseResourceManager._shutdown", "tests/conftest.py cleanup", "must be a bool, False for a live client")
assert callable(resources.shutdown), _message("LangfuseResourceManager.shutdown", "tests/conftest.py cleanup", "must be callable")

resources.shutdown()

assert resources._shutdown is True, _message(
"LangfuseResourceManager._shutdown", "tests/conftest.py cleanup", "must become True after shutdown()"
)


def test_langfuse_observation_wrapper_is_importable_from_private_module() -> None:
module = importlib.import_module("langfuse._client.span")

assert hasattr(module, "LangfuseObservationWrapper"), _message(
"langfuse._client.span.LangfuseObservationWrapper", "callbacks:31", "must be importable"
)
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading