Repository navigation
‘expand log line’ not use all filter. #1155
Description
search with '(ServiceName:"auth-service-auth-api-service-prod" OR ServiceName:"compliance-hugebase") LogAttributes.cache:"TCP_MISS_ABORTED"'
click exact log line, it would search with:
SELECT *,Timestamp AS "__hdx_timestamp",Body AS "__hdx_body",TraceId AS "__hdx_trace_id",SpanId AS "__hdx_span_id",SeverityText AS "__hdx_severity_text",ServiceName AS "__hdx_service_name",ResourceAttributes AS "__hdx_resource_attributes",LogAttributes AS "__hdx_event_attributes" FROM default.otel_logs WHERE (Timestamp = parseDateTime64BestEffort('2025-09-11T03:35:15.038000000Z', 9) AND ServiceName = 'compliance-hugebase' AND SeverityText = '' AND Body = '' AND TimestampTime = parseDateTime64BestEffort('2025-09-11T03:35:15Z', 9)) LIMIT 1
because the search not use '(ServiceName:"auth-service-auth-api-service-prod" OR ServiceName:"compliance-hugebase") LogAttributes.cache:"TCP_MISS_ABORTED"', the log result is not correct.
field LogAttributes.cache is not TCP_MISS_ABORTED !!!
Activity
MikeShi42 commentedon Sep 11, 2025
This is a consequence of not being able to definitively query for a given row since there is no concept of a row ID in ClickHouse (at least not a stable row ID by default). Instead we combine the columns in the
SELECTstatement combined with the primary key of the table to query for the row to show when clicking in. This for the vast majority of queries will yield the correctIn this case, a short-term workaround would be to include LogAttributes.cache to the
SELECTstatement (or any other unique identifier)A few long term fixes may include adding a unique log ID to each row from the ClickStack collector and using that ID if it's available to determine the specific row to query, or allowing users to customize how a unique row is determined in the table with custom additional columns.
zdyj3170101136 commentedon Sep 11, 2025
i think 'include LogAttributes.cache to the SELECT statement (or any other unique identifier)' is enough.
uuid of log is difficult to handle
MikeShi42 commentedon Sep 11, 2025
The random log id can be done relatively simply and automatically inferred by HyperDX (much like how we automatically already include the primary key in the where condition on behalf of the user).
The id can be somewhat small, a UInt16 would likely do the job well enough in this case (though likely at the cost of ~2% storage increase).
fix: use block_number/block_offset to uniquely identify log rows (#2071)