Repository navigation
Conversation
Drain sibling handlers before returning a failed Kafka batch and atomically enqueue consumer events with expiring Redis deduplication markers. Preserve legacy envelope identity and derived session-start identity across redelivery. Self-hosted workers retain one additional Redis key per consumer-produced event for one hour. No schema migration or environment change is required; mixed-version deployments gain full protection after all workers upgrade.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the
Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
|
@coderabbitai run pre-merge checks |
✅ Actions performedRunning Pre-merge checks. Results will be posted in the summary comment once complete. |
|
@coderabbitai The latest review covers |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -uo pipefail
git rev-parse HEAD
gh pr view 603 --repo Openpanel-dev/openpanel --json headRefOid,baseRefOid,url
gh api --paginate repos/Openpanel-dev/openpanel/issues/603/comments --jq '.[] | select(.user.login | test("coderabbit")) | {id,updated_at,body}'
gh api --paginate repos/Openpanel-dev/openpanel/pulls/603/reviews --jq '.[] | {id,user:.user.login,commit_id,submitted_at,body}'
head_sha=$(gh pr view 603 --repo Openpanel-dev/openpanel --json headRefOid --jq .headRefOid)
gh api "repos/Openpanel-dev/openpanel/commits/$head_sha/check-runs" --jq '.check_runs[] | {id,name,status,conclusion,output}'
for file in AGENTS.md packages/core/AGENTS.md; do
if [ -f "$file" ]; then printf '\n--- %s ---\n' "$file"; cat "$file"; fi
done
ast-grep outline packages/core/src/buffers/event-buffer.ts --items all
ast-grep outline packages/core/src/modules/ingest/src/consumer.ts --items all
ast-grep outline packages/core/src/modules/ingest/src/incoming-event-handler.ts --items all
fd -H -i 'coderabbit|docstring' . --exclude .gitLength of output: 62911 🏁 Script executed: #!/bin/bash
set -uo pipefail
if [ -f WORKTREE.md ]; then cat WORKTREE.md; fi
git diff 3a5fbd05f5202a14ce3e224685a50f9cc03d18cb HEAD --stat
git diff --unified=5 3a5fbd05f5202a14ce3e224685a50f9cc03d18cb HEAD -- packages/core/src/buffers/event-buffer.ts packages/core/src/modules/ingest/src/consumer.ts packages/core/src/modules/ingest/src/incoming-event-handler.ts
rg -n -B14 -A14 '(^|[[:space:]])(private async writeToRedis|private async enqueueEvents|async function ingestIncomingEvent|async function processMessage|const eachBatch|async function eachBatch|function createEventsBatchHandler)' packages/core/src/buffers/event-buffer.ts packages/core/src/modules/ingest/src/consumer.ts packages/core/src/modules/ingest/src/incoming-event-handler.ts
ast-grep outline packages/core/src/modules/ingest/src/consumer.rebalance.integration.test.ts --items all
ast-grep outline packages/core/test/kafka-rebalance-worker.ts --items all
ast-grep outline packages/core/src/buffers/event-buffer-redelivery.test.ts --items all
git diff --unified=2 3a5fbd05f5202a14ce3e224685a50f9cc03d18cb HEAD -- packages/core/src/modules/ingest/src/consumer.rebalance.integration.test.ts packages/core/test/kafka-rebalance-worker.ts packages/core/src/buffers/event-buffer-redelivery.test.ts | head -360
gh api repos/Openpanel-dev/openpanel/commits/0b09797c426b0b0b4c7da20d7721786fe83d818f --jq '{sha,commit:.commit.message,files:[.files[]|{filename,patch}]}'Length of output: 47666
You are interacting with an AI system. |
|
Thanks for digging into this. The redelivery analysis is useful, and the dedup key is right: it only matches true redeliveries, so separate client requests are kept. We'd like to split this PR, though. In aaf02b1 we removed a per-event Redis marker because of its cost, one command per event and events/s × 120 s keys held in memory. This one brings a marker back with a 1 hour TTL, so about 30× the keys, and the micro-batch enqueue goes from one RPUSH to a Lua script doing several commands per event. The compose template runs Redis with Could you do this instead?
One thing to keep in mind if dedup does come back: |
A rebalance can reject a heartbeat after events have reached Redis but before their Kafka offsets are committed. The old
Promise.allalso returns while sibling handlers are still running. A second consumer then appends the same events again, even when their producer ids are stable.This waits for sibling handlers and their durability gate before propagating a batch error. Consumer-produced rows use an atomic Redis enqueue/dedup script. Old envelopes get stable ids from Kafka coordinates; derived
session_startrows get separate stable ids. Offsets still resolve only after durability.Fixes #563.
Reproduction
The regression starts separate Bun worker processes with the real batch handler and EventBuffer, actual Kafka/Redis/ClickHouse, and a controlled synthetic business-event handler. It forces a join during an in-flight batch and records partition/offset independently of the queue. Each case must observe a rebalance and repeat deliveries, then check committed offsets, physical rows, distinct ids and distinct event sequences.
The new integration assertion fails on the unchanged base with 3,536 rows. The heartbeat regression also fails on the base. Redis tests cover concurrent writers, retry after a refused enqueue, drained queues, marker expiry, corrupt marker types and project isolation.
Validation
tsc --noEmit, Biome on all changed TypeScript andgit diff --checkpass.packages/core/test/kafka-rebalance.mddocuments the opt-in test and a dedicated Compose stack using prebuilt images. Compose configuration was validated; the image-based stack was not run. No application builds were needed.Delivery limits / upgrade
Dedup markers expire after one hour: Redis retains one extra key per distinct consumer-produced row in that window. Replays after expiry, lost/evicted Redis state, and the existing ClickHouse-insert → Redis-trim crash window remain at-least-once. During rolling upgrades, old workers still append unconditionally; full protection starts after all consumers upgrade. No migration or environment change is required.
Summary by CodeRabbit