Repository navigation
Skip DSM tag/context building in Kafka producers when Data Streams is off (dsm - perf quick fix) - #13082
Conversation
… off The send advice built DataStreamsTags for every record, and the producer callback built DataStreamsTags for every acknowledged record to track the produce backlog, even with DSM disabled, where DSM injection and trackBacklog are no-ops. Guard both with the dynamic isDataStreamsEnabled() check so remote-config enablement still takes effect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
More details
The DSM guards preserve trace propagation, span completion, and user callbacks in both Kafka versions. The completed static review identified no reportable regression.
🤖 Bits Code Review · Commit d901d4f
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
Build pipeline has failing jobs for bc0d1d7: What to do next?
DetailsSince those jobs are not marked as being allowed to fail, the pipeline will most likely fail. |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
4b6dcdf
into
master
What Does This Do
Skips building Data Streams Monitoring (DSM) tags in the Kafka producer paths (
kafka-clients-0.11andkafka-clients-3.8) when DSM is disabled:KafkaProducerInstrumentation/ProducerContextPropagationAdvice):DataStreamsTagswere built for every record sent. With DSM off they were used only by the DSM injection, which does nothing then.KafkaProducerCallback, both versions):DataStreamsTags.createWithPartition(...)was built for every acknowledged record and passed totrackBacklog.DefaultDataStreamsMonitoringis used even with DSM off, and itstrackBackloghas no enabled check, so each acknowledgement also allocated aBacklogand queued it for the DSM thread to throw away.All guards use the dynamic
traceConfig().isDataStreamsEnabled()check, so turning DSM on through remote config still works.Not included: the offset-commit paths (
ConsumerCoordinator*,DDOffsetCommitCallback) have the same pattern, but run per commit and partition, not per message.Motivation
In the Kafka consumer, the same guard saved 548 B per record (−13.7% allocation) with DSM off (
KafkaConsumeScenariosBenchmark, #13075), and +6.6% throughput inkafka-consumer-perf-test. Most of that isDataStreamsTags.create: prefixed tag strings, hashing, and abyte[]per hashed tag. The producer side does it twice per record: once on send and once on acknowledgement.Additional Notes
Part of a sweep of the same pattern across messaging integrations, one PR per integration:
KafkaConsumeScenariosBenchmark)With DSM off at startup, the DSM propagator isn't registered, and
DataStreamsPropagator.injectalso returns early whenisDataStreamsEnabled()is false. New spans getNoopPathwayContext, sosetCheckpointdoes nothing. The skipped work had no observable effect.Follow-ups: APMLP-2116 (audit), APMLP-2117 (gated contributions, so a disabled product costs one branch by construction), APMLP-2147 (making
DataStreamsTagscheap when DSM is on).Tests:
kafka-clients-0.11+kafka-clients-3.8test+forkedTest: 228 tests, 0 failures, including the DSM-enabled suites.Jira ticket: APMLP-2116
🤖 Generated with Claude Code