Skip to content

Use the testing tool’s ingestion tuning as defaults - #6022

Open
rbev wants to merge 2 commits into
masterfrom
default-tuning
Open

rbev wants to merge 2 commits into
masterfrom
default-tuning

Conversation

@rbev

@rbev rbev commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Error and audit instances currently need explicit configuration to use the ingestion profile pinned by the testing tool. Apply that profile when settings are omitted, while preserving explicit overrides.

Setting Previous default New default
Receive concurrency Error: 10; audit: 32, or 10 for SQL Server and PostgreSQL transports 100
Batch size Receive concurrency 25, capped at receive concurrency
Batch timeout Zero 100 ms

Apply the same concurrency default to error-ingestion-only hosts. Keep monitoring instance defaults unchanged. Preserve explicitly configured batch sizes and timeouts, including a zero timeout.

Storage behavior

Keep the existing parallel-writer defaults and safety checks:

  • RavenDB error ingestion uses one writer because concurrent batches can merge updates into the same failed-message document without ordering guarantees.
  • SQL Server and PostgreSQL error ingestion use four writers.
  • Audit ingestion uses four writers on all three providers. RavenDB audit batches write independent documents through separate bulk-insert operations.

No separate SQL tuning profile was found in the repository. These defaults match the testing tool; this change does not establish that they are optimal for every storage provider or deployment.

Operational impact

The 100 ms timeout can add up to 100 ms of batching latency at low message volumes. Higher receive concurrency can increase resource usage. In the error instance, the shared concurrency setting also controls the hosted endpoint, retry forwarding, and throughput collection receivers.

Update the ingestion design, configuration docstrings, and affected test expectations.

@rbev rbev self-assigned this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant