fix: use block_number/block_offset to uniquely identify log rows by karl-power · Pull Request #2071 · ClickHouse/ClickStack

karl-power · 2026-04-08T16:50:42Z

Summary

Fixes an issue where clicking a log row to expand it could show the wrong row's details when multiple rows share identical visible column values (Timestamp, ServiceName, Body, etc.).

Root cause: The detail panel query reconstructs a WHERE clause from only the visible columns, without the original search filters. When rows collide on those columns, LIMIT 1 returns whichever row ClickHouse finds first — not necessarily the one clicked.

Fix: Detect and inject ClickHouse's built-in _block_number and _block_offset virtual columns as hidden tiebreakers in the row SELECT. These are available when the table engine was created with enable_block_number_column = 1 and enable_block_offset_column = 1. Since the row WHERE clause builder already iterates all columns in the row data, the block columns are automatically included in detail queries — disambiguating rows without surfacing them in the UI.

A fallback to __hdx_id (a custom materialized column from the prior approach) is retained for any tables that already have that column.

No schema migration required for the primary fix — the approach relies entirely on engine-level settings that already exist on the otel_logs table.

Key changes

DBRowTable.tsx: Rename appendSelectWithPrimaryAndPartitionKey → appendSelectWithAdditionalKeys adding an extraKeys: string[] param. Rename useConfigWithPrimaryAndPartitionKey → useConfigWithAdditionalSelect; when a sourceId is present (row-level query), inspect engine_full metadata for block column flags and inject _block_number/_block_offset as extra keys, falling back to __hdx_id if present.
types.ts / source.ts / SourceForm.tsx: Remove uniqueRowIdExpression — the tiebreaker is now auto-detected from the table engine, not manually configured.
Schema seed: Remove the __hdx_id materialized column addition from otel_logs; the block column engine flags are sufficient.
Tests: New appendSelectWithAdditionalKeys cases covering extraKeys append, deduplication against primary/partition keys, and array-style selects. Scaffold for inferTableSourceConfig tests.

How to test locally

Reproduce the bug on `main`

Pick any log row from otel_logs. In the ClickHouse UI, run the query that fires when you click a row to open its sidebar.
Insert the same row into otel_logs but change one nested value — e.g. change ResourceAttributes.host.arch from arm64 to arm65.
On the search page, filter for ResourceAttributes.host.arch = arm65. One result appears. Click it to open the sidebar.
The sidebar shows arm64 — it selected the wrong row.

Verify the fix on this branch

Confirm otel_logs was created with enable_block_number_column = 1 and enable_block_offset_column = 1 (check via SHOW CREATE TABLE default.otel_logs).
Repeat steps 1–3 above.
The sidebar now shows arm65 — correct row selected.

Verify fallback path

On a table that has __hdx_id but no block columns, the app still injects __hdx_id as the tiebreaker (same behavior as the previous approach).

References

Linear: Closes HDX-2372, Closes HDX-3489
GitHub: Closes ‘expand log line’ not use all filter. #1155

changeset-bot · 2026-04-08T16:50:47Z

🦋 Changeset detected

Latest commit: cef75d2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@hyperdx/api	Patch
@hyperdx/app	Patch
@hyperdx/otel-collector	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2026-04-08T16:50:49Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview, Comment	May 4, 2026 1:00pm

github-actions · 2026-04-08T16:58:11Z

E2E Test Results

✅ All tests passed • 159 passed • 3 skipped • 1180s

Status	Count
✅ Passed	159
❌ Failed	0
⚠️ Flaky	4
⏭️ Skipped	3

Tests ran across 4 shards in parallel.

View full report →

github-actions · 2026-04-09T13:53:57Z

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

Critical-path files (1):
- docker/otel-collector/schema/seed/00002_otel_logs.sql
Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats

Production files changed: 6
Production lines changed: 81 (+ 73 in test files, excluded from tier calculation)
Branch: karl/id-field-in-log-filter
Author: karl-power

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

github-actions · 2026-04-09T13:55:45Z

PR Review

PR #2071 — fix: use block_number/block_offset to uniquely identify log rows

Solid fix for a real bug. Logic is correct, tests cover the new cases, and the fallback chain is sensible. A few things to check:

⚠️ Fragile engine string matching → engineFull.includes('enable_block_number_column = 1') breaks silently if ClickHouse ever serializes settings without spaces (e.g. enable_block_number_column=1). Consider a regex like /enable_block_number_column\s*=\s*1/ to be spacing-tolerant.
⚠️ Existing deployments without the engine flags → The seed SQL change only affects new otel_logs tables. Any instance that created otel_logs before this commit won't have the block columns and will silently fall back to __hdx_id (or no tiebreaker at all). The PR description says these settings "already exist" on existing tables — if that's true for all supported upgrade paths, fine; otherwise a migration note or runtime warning would help.
ℹ️ additionalKeysLength when extraKeys overlap primaryKeys → The deduplication via Set is correct, and the new test "should deduplicate extraKeys that overlap with primary/partition keys" confirms it. No action needed, just confirming it's covered.
✅ Removal of uniqueRowIdExpression from Mongoose model and Zod schema is clean — field was already UI-disabled and the auto-detection approach supersedes it.
✅ useColumns conditional fetch (enabled: !!sourceId) follows standard TanStack Query patterns and correctly skips aggregate (patterns) queries.
✅ New tests cover extraKeys append, deduplication, and array-style selects.

karl-power · 2026-04-09T13:55:53Z

@wrn14897 Do you know if the migrations are needed here? I tried to run them locally which invokes migrate, but I don't see that installed anywhere.

wrn14897

I think we should use the UUID type for the id column (https://clickhouse.com/docs/sql-reference/data-types/uuid). Using toUInt16(rand()) only provides 65,536 possible values, which could lead to collisions.

I’d also suggest raising this with the team, since it involves a schema change. It would be good to make sure we all align on the approach for the ID field before moving forward. cc @knudtty

karl-power · 2026-04-13T14:14:04Z

I think we should use the UUID type for the id column

@MikeShi42 do you have thoughts on this as you originally suggested UInt16?

I'm guessing the reasoning was that UInt16 combined with the other fields from the search query would be enough to reduce the chance of a collision, with the benefit of less storage than UUID

wrn14897 · 2026-04-13T16:00:43Z

I think we should use the UUID type for the id column

@MikeShi42 do you have thoughts on this as you originally suggested UInt16?

I'm guessing the reasoning was that UInt16 combined with the other fields from the search query would be enough to reduce the chance of a collision, with the benefit of less storage than UUID

I understand the reason for reducing storage (to achieve a better compression ratio), and that this ID is only used internally. Also, as you mentioned, the chance of collisions would be lower if the search is always combined with other fields. I just want to confirm that this is intentional

karl-power · 2026-04-16T13:50:53Z

@wrn14897 I have shared with the team but didn't receive any feedback yet. Are you happy to proceed like this or prefer to use uuid?

MikeShi42 · 2026-04-20T14:42:55Z

@karl-power instead of creating a new column, any thoughts on if we should instead just use _block_number and _block_offset and enabling enable_block_number_column and enable_block_offset_column by default in new tables? This will technically allow other features as well within ClickHouse - so more useful than a random id.

We can tell existing users to run an update to add this new random id column (as suggested in this PR) if they haven't set up their table to use block number/offsets.

karl-power · 2026-04-28T02:03:10Z

@karl-power instead of creating a new column, any thoughts on if we should instead just use _block_number and _block_offset and enabling enable_block_number_column and enable_block_offset_column by default in new tables? This will technically allow other features as well within ClickHouse - so more useful than a random id.

We can tell existing users to run an update to add this new random id column (as suggested in this PR) if they haven't set up their table to use block number/offsets.

@MikeShi42 Yeah I think the block_number/offset combo would work. So you're suggesting to remove the UI control for the unique identifier and try to fallback to __hdx_id if the block settings aren't enabled?

karl-power · 2026-05-01T01:49:23Z

@MikeShi42 updated to use block number/offset

wrn14897 · 2026-05-04T12:40:57Z

 ORDER BY (ServiceName, TimestampTime, Timestamp)
 TTL TimestampTime + ${TABLES_TTL}
-SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1;
+SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1, enable_block_number_column = 1, enable_block_offset_column = 1;


This is great! Although, I wonder if the multiple replica would return different block numbers, though. Maybe something to think about later.

Resolved

wrn14897

awesome fix!

…ble persists (#3364) ## Summary Opening log details on large text-log tables is much slower than the search itself (HDX-5696). The side panel already has a fast lookup that filters only on primary key + partition + block columns, but it was gated on the table persisting **both** `_block_number` and `_block_offset`. The affected table only sets `enable_block_offset_column = 1`, so every click fell back to the full-row WHERE, which filters on every selected column (including `Body`) and triggers expensive index loading. This PR relaxes the gate: the fast path is taken when the table persists **either** block column, and the WHERE uses whichever of `_block_number` / `_block_offset` is available. Combined with the primary key, either one is unique enough in practice: two rows would need identical key values (including a nanosecond timestamp) *and* the same insert block or the same offset within their blocks. Tables that persist both columns keep the current behavior; tables with neither still fall back to `__hdx_id` / the full row. Changes: - `getPersistedBlockColumns(engineFull)` in `DBRowTable.tsx` decides which block columns to inject; `useConfigWithAdditionalSelect` uses it for both the extra SELECT keys and `rowKeyColumns`. - Unit tests for the helper and `renderHook` coverage of `useConfigWithAdditionalSelect` (offset only, number only, both, `__hdx_id` fallback, no `sourceId`). - Changeset for `@hyperdx/app`. Known trade-off: rows with identical PK that also share the block number (or the offset) still collide and `LIMIT 1` picks one; the thread on the Linear issue accepted that given nanosecond timestamps. Verified against ClickHouse 26.8 with tables that set only one of the two settings: `engine_full` exposes the setting as matched here, the persisted column survives `OPTIMIZE FINAL` unchanged, and the WHERE shape HyperDX now generates (PK expressions + the block column, no `Body`) returns the clicked row. ### How to test on Vercel preview N/A — the demo tables persist both block columns, so the fast path was already taken there; this change only affects tables that persist one of them. To verify locally, create a table with `SETTINGS enable_block_offset_column = 1` (or `enable_block_number_column = 1`) alone, add it as a log source, click a row and check the side-panel query filters on PK columns + that block column only. ### References - Linear Issue: [HDX-5696](https://linear.app/clickhouse/issue/HDX-5696/speed-up-log-details-lookup-using-primary-key-and-block-offset) - Related PRs: #2071 (introduced block-column detection), #2320 (restricted the row WHERE to PK + block ids) Linear Issue: [HDX-5696](https://linear.app/clickhouse/issue/HDX-5696/speed-up-log-details-lookup-using-primary-key-and-block-offset) <div><a href="https://cursor.com/agents/bc-5fdae508-3c5f-4b0b-a60e-03f5e5eed062?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-5fdae508-3c5f-4b0b-a60e-03f5e5eed062&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div> Co-authored-by: Cursor Agent <199161495+cursoragent@users.noreply.github.com>

…kup (#3566) ## Summary Clicking a search result could open the wrong row's details when the table persists only one of `_block_number` / `_block_offset`. - Since #3364 the side panel used a key-only lookup (primary key + partition + whichever block column exists) when at least one block column was persisted. One block column is not unique: `_block_offset` repeats across insert blocks, `_block_number` across the rows of one block, and the primary key often doesn't break the tie (the OTel exporter default keys on the second-precision `TimestampTime`). - Since #3530 the block columns are also detected from server-level `merge_tree_settings`, so on servers that enable only one of them, every table without explicit settings takes this path. - The key-only lookup now requires both block columns. A lone block column is still appended to the full-row lookup as a tiebreaker. - Tables that persist only one block column return to the full-row lookup and lose the #3364 speed-up. A `_part` + `_part_offset` lookup is the follow-up to restore it. ### How to reproduce **Starting point:** `yarn dev`. Set the server default on the dev ClickHouse and restart it: ```sh docker exec <ch-server> sh -c 'echo "<clickhouse><merge_tree><enable_block_offset_column>1</enable_block_offset_column></merge_tree></clickhouse>" > /etc/clickhouse-server/config.d/block_offset.xml' docker restart <ch-server> ``` Create an exporter-shaped table and insert three rows as separate inserts, two in the same second: ```sql CREATE TABLE default.repro_logs ( Timestamp DateTime64(9), TimestampTime DateTime DEFAULT toDateTime(Timestamp), ServiceName LowCardinality(String), SeverityText LowCardinality(String), Body String, LogAttributes Map(LowCardinality(String), String) ) ENGINE = MergeTree PARTITION BY toDate(TimestampTime) PRIMARY KEY (ServiceName, TimestampTime) ORDER BY (ServiceName, TimestampTime, Timestamp); INSERT INTO default.repro_logs (Timestamp, ServiceName, SeverityText, Body) VALUES ('2026-11-05 09:00:39.845', 'saruman', 'INFO', 'Response x_span_id=ESLMARS7L7'); INSERT INTO default.repro_logs (Timestamp, ServiceName, SeverityText, Body) VALUES ('2026-11-05 09:00:39.100', 'saruman', 'INFO', 'Request x_span_id=ESOPOZ7FCV'); INSERT INTO default.repro_logs (Timestamp, ServiceName, SeverityText, Body) VALUES ('2026-11-05 09:00:36.909', 'saruman', 'INFO', 'Request x_span_id=ESLMARS7L7'); ``` Add a log source for `default.repro_logs` (timestamp column `TimestampTime`), then search it for `"ESLMARS7L7"` over a range covering 2026-11-05 09:00 UTC. **On `main`:** 1. Click the `Response x_span_id=ESLMARS7L7` row a few times. 2. The side panel shows `Request x_span_id=ESOPOZ7FCV` about half the time. The query in `system.query_log` is `WHERE ServiceName='saruman' AND TimestampTime=... AND toDate(TimestampTime)=... AND _block_offset='0' LIMIT 1`. **On this branch:** 1. Click the same row. The panel always shows `Response x_span_id=ESLMARS7L7`; the query also filters on `Timestamp`, `SeverityText` and `Body`. ### How to test on Vercel preview N/A — needs a ClickHouse server default; the demo server's user cannot read `system.merge_tree_settings`. ### References - Related PRs: #3530, #3364, #2320, #2071

vercel Bot deployed to Preview April 8, 2026 16:53

View deployment

karl-power force-pushed the karl/id-field-in-log-filter branch from a7ee44f to 3151b4d Compare April 9, 2026 09:00

vercel Bot deployed to Preview April 9, 2026 09:03

View deployment

karl-power marked this pull request as ready for review April 9, 2026 13:53

github-actions Bot added the review/tier-4 Critical — deep review + domain expert sign-off label Apr 9, 2026

karl-power requested a review from wrn14897 April 9, 2026 13:54

karl-power force-pushed the karl/id-field-in-log-filter branch from 3151b4d to 8772bd3 Compare April 9, 2026 13:59

vercel Bot deployed to Preview April 9, 2026 14:03

View deployment

wrn14897 previously requested changes Apr 10, 2026

View reviewed changes

karl-power force-pushed the karl/id-field-in-log-filter branch from 8772bd3 to 5d75767 Compare April 15, 2026 16:09

vercel Bot deployed to Preview April 15, 2026 16:12

View deployment

vercel Bot deployed to Preview April 28, 2026 02:06

View deployment

karl-power force-pushed the karl/id-field-in-log-filter branch from d790f62 to 327e41e Compare May 1, 2026 00:50

vercel Bot deployed to Preview May 1, 2026 00:53

View deployment

karl-power changed the title ~~fix: filter logs with ID field~~ fix: use block_number/block_offset to uniquely identify log rows May 1, 2026

karl-power force-pushed the karl/id-field-in-log-filter branch from 327e41e to e44d45a Compare May 1, 2026 01:36

vercel Bot deployed to Preview May 1, 2026 01:39

View deployment

fix: use block_number/block_offset to uniquely identify log rows

0f75ae4

karl-power force-pushed the karl/id-field-in-log-filter branch from e44d45a to 0f75ae4 Compare May 1, 2026 01:41

vercel Bot deployed to Preview May 1, 2026 01:44

View deployment

karl-power requested a review from MikeShi42 May 1, 2026 01:49

wrn14897 reviewed May 4, 2026

View reviewed changes

wrn14897 approved these changes May 4, 2026

View reviewed changes

Merge branch 'main' into karl/id-field-in-log-filter

cef75d2

karl-power added the automerge label May 4, 2026

vercel Bot deployed to Preview May 4, 2026 13:00

View deployment

kodiakhq Bot merged commit 88b2b64 into main May 4, 2026
18 checks passed

kodiakhq Bot deleted the karl/id-field-in-log-filter branch May 4, 2026 13:06

cflee mentioned this pull request May 18, 2026

Migrations for schema changes #2297

Closed

knudtty mentioned this pull request Oct 13, 2026

fix(app): resolve row details with PK + whichever block column the table persists #3364

Merged

pulpdrew mentioned this pull request Nov 2, 2026

Timestamp-only SELECT could open the wrong event #3518

Open

karl-power mentioned this pull request Nov 5, 2026

fix(app): require both block columns for the key-only row details lookup #3566

Merged

Conversation

karl-power commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key changes

How to test locally

Reproduce the bug on main

Verify the fix on this branch

Verify fallback path

References

Uh oh!

changeset-bot Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

vercel Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

github-actions Bot commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Tier 4 — Critical

Uh oh!

github-actions Bot commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review

Uh oh!

karl-power commented Apr 9, 2026

Uh oh!

wrn14897 left a comment

Choose a reason for hiding this comment

Uh oh!

karl-power commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wrn14897 commented Apr 13, 2026

Uh oh!

karl-power commented Apr 16, 2026

Uh oh!

MikeShi42 commented Apr 20, 2026

Uh oh!

karl-power commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

karl-power commented May 1, 2026

Uh oh!

wrn14897 May 4, 2026

Choose a reason for hiding this comment

Uh oh!

wrn14897 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

karl-power commented Apr 8, 2026 •

edited

Loading

Reproduce the bug on `main`

changeset-bot Bot commented Apr 8, 2026 •

edited

Loading

vercel Bot commented Apr 8, 2026 •

edited

Loading

github-actions Bot commented Apr 8, 2026 •

edited

Loading

github-actions Bot commented Apr 9, 2026 •

edited

Loading

github-actions Bot commented Apr 9, 2026 •

edited

Loading

karl-power commented Apr 13, 2026 •

edited

Loading

karl-power commented Apr 28, 2026 •

edited

Loading