Skip to content

Stop allocating propagation tags for local child spans - #13000

Draft
dougqh wants to merge 3 commits into
dougqh/collector-trace-segmentfrom
dougqh/lazy-child-ptags
Draft

dougqh wants to merge 3 commits into
dougqh/collector-trace-segmentfrom
dougqh/lazy-child-ptags

Conversation

@dougqh

@dougqh dougqh commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Stacked: master ← #13008 ← this PR ← #13009

What Does This Do

Local child spans no longer allocate their own PropagationTags. They create one lazily, only when a write targets the child itself: _dd.p.ts or _dd.p.debug set on that span.

Motivation

Every local child span built an empty PropagationTags through propagationTagsFactory.empty(), but nearly all trace-level state is read from the local root's instance: sampling, origin, force-keep, and inject. In a low-tags-per-span stress test, PropagationTags is about 10% of allocation.

Additional Notes

Output is unchanged by design. #11701 shared the root's instance with children, which also changed what children emit and mutate, and it failed system-tests (_dd.p.tid) in ways that were never explained. This PR doesn't share anything:

  • A child holds no instance. Nothing on a child mutates the root's.
  • OWN_PROPAGATION_TAGS serialization (OTLP, CI Visibility, LLMObs, file and list writers) builds a transient empty instance, the same way the constructor used to. _dd.p.tid and the propagation-disabled error tag come out identical.
  • The agent mappers (v0.4, v0.5, v1) read ROOT_PROPAGATION_TAGS only on the first span of each chunk, which is unaffected.

ChildSpanPropagationTagsTest checks each child's serialized output against master's construction of an empty instance.

Known pre-existing behavior, kept as is: _dd.p.ts and _dd.p.debug set on a non-root span go to that span's own tags, so they are never injected and never reach the agent payload. That will be fixed in a follow-up PR with its own system-test run.

Allocation (TraceAssemblyBenchmark, JDK 17, @Threads(8), 2 forks × 5 iterations, -prof gc):

childCount master B/op branch B/op Δ
1 2280.001 ± 0.001 2080.001 ± 0.001 −200 (−8.8%)
5 5176.222 ± 420.370 3888.002 ± 0.003 −1288 (−24.9%)
20 14684.005 ± 19.124 10672.006 ± 0.010 −4012 (−27.3%)

That's about 200 B per child span.

Throughput is pending. These runs were on a busy host, so throughput numbers are not meaningful yet. All pairs overlap within their error bars. A quiet run on JDK 8, 17 and 21 (TraceAssemblyBenchmark plus PropagationScenariosBenchmark, 5 forks) will be added before this is marked ready. The change adds a volatile read on each PropagationTags access and a CAS when a child first writes its own tags.

Contributes to APMLP-1515 / APMLP-2049 (TraceSegment).

🤖 Generated with Claude Code

@dougqh dougqh added comp: core Tracer core tag: performance Performance related changes type: refactoring tag: ai generated Largely based on code generated by an AI or LLM labels Oct 23, 2026
@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
• Patch Coverage: 97.06%
• Overall Coverage: 59.86% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 35909f3 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.76 s 14.69 s [-0.5%; +1.5%] (no difference)
startup:insecure-bank:tracing:Agent 13.61 s 13.65 s [-1.0%; +0.5%] (no difference)
startup:petclinic:appsec:Agent 17.09 s 16.98 s [-0.1%; +1.3%] (no difference)
startup:petclinic:iast:Agent 17.00 s 17.04 s [-1.0%; +0.6%] (no difference)
startup:petclinic:profiling:Agent 16.72 s 16.73 s [-1.1%; +1.0%] (no difference)
startup:petclinic:sca:Agent 17.06 s 16.95 s [-0.4%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 16.23 s 16.00 s [+0.4%; +2.4%] (maybe worse)

Commit: 35909f39 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

dougqh and others added 2 commits October 23, 2026 00:28
Local child spans each built an empty PropagationTags they rarely used:
trace-level sampling, origin, and inject state is read from the local
root. Children now create their own instance lazily, only when a write
targets the child itself (_dd.p.ts / _dd.p.debug set on that span).

OWN_PROPAGATION_TAGS serialization builds a transient empty instance for
children without one, so serialized output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: performance Performance related changes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant