Filters sidebar is timing out

In #2128 some rollups tables were introduced to be able to use autocomplete.

The filters sidebar are timing out and not loading at all with this warning:

[2026-07-06T17:26:00.757Z][WARN][@clickhouse/client][Config] request_timeout is set to 3600000ms, but send_progress_in_http_headers is not enabled. Long-running queries may fail with socket hang-up errors if they exceed the load balancer idle timeout. Consider enabling progress headers with clickhouse_settings: { send_progress_in_http_headers: 1, http_headers_progress_interval_ms: '<interval>' }. See https://github.com/ClickHouse/clickhouse-js/blob/main/docs/howto/long_running_queries.md for more details.

The filters sidebar is based on these rollup tables:

CREATE TABLE IF NOT EXISTS ${DATABASE}.otel_logs_kv_rollup_15m
(
    `Timestamp` DateTime,
    `ColumnIdentifier` LowCardinality(String),
    `Key` LowCardinality(String),
    `Value` String,
    `count` UInt64,
    INDEX idx_count_minmax count TYPE minmax GRANULARITY 1,
    INDEX idx_timestamp_minmax Timestamp TYPE minmax GRANULARITY 1
)
ENGINE = SummingMergeTree
PARTITION BY toDate(Timestamp)
ORDER BY (ColumnIdentifier, Key, Timestamp, Value)
TTL Timestamp + ${TABLES_TTL}
SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1;

The query ran by HyperDx is not using any of the keys of the table, prompting a full scan of the whole partition.
Even then, slicing on Timestamp would not help a lot, given that its usually ran using the time window of the selection, which can be of multiple days.

SELECT ColumnIdentifier, Key, groupUniqArray(20)(Value) AS Values
FROM logs.otel_logs_kv_rollup_15m
WHERE Value != ''
  AND Timestamp >= toStartOfFifteenMinutes(fromUnixTimestamp64Milli(1783085400000)) AND Timestamp <= toStartOfFifteenMinutes(fromUnixTimestamp64Milli(1783087200000))
GROUP BY ColumnIdentifier, Key
ORDER BY ColumnIdentifier = 'NativeColumn' DESC, ColumnIdentifier = 'ResourceAttributes' DESC, ColumnIdentifier, Key
LIMIT 100

Can this be improved in any way?
Maybe a setting can be added to fix the SELECT params to a subset just for this filters query?

I am very surprised this would be timing out. We've tested this on some pretty large scale, and even then the MV queries finish in sub-second to single digit seconds.

Have you configured your source to use the MVs? "Metadata Materialized Views" is the relevant section.
Do you have "Show All Values" toggled on or off? It must be on to take advantage of the MVs.

If you have verified that it indeed is using the MVs, do you mind disclosing your ingest size/mo? Or even better, querying the size of this MV?

I'm using the MVs correctly.
I noticed investigating further that I'm ingesting a couple of high variance fields like a timestamp and a JSON in my LogAttributes and this balooned out the amount of values in the kv rollup table to a very big number which killed the performance of query.

I excluded these fields from the ingestion and cleaned up my tables and now the performance is much better.

EDIT: Nevermind, still seeing timeouts, re-opening the issue

In console

[2026-07-14T13:38:02.087Z][WARN][@clickhouse/client][Config] request_timeout is set to 3600000ms, but send_progress_in_http_headers is not enabled. Long-running queries may fail with socket hang-up errors if they exceed the load balancer idle timeout. Consider enabling progress headers with clickhouse_settings: { send_progress_in_http_headers: 1, http_headers_progress_interval_ms: '<interval>' }. See https://github.com/ClickHouse/clickhouse-js/blob/main/docs/howto/long_running_queries.md for more details.

SELECT
    ColumnIdentifier,
    Key,
    groupUniqArray(20)(Value) AS Values
FROM global_logs_kv_rollup_15m
WHERE (Value != '') AND (Timestamp >= toStartOfFifteenMinutes(fromUnixTimestamp64Milli(1783296000000))) AND (Timestamp <= toStartOfFifteenMinutes(fromUnixTimestamp64Milli(1783987200000)))
GROUP BY
    ColumnIdentifier,
    Key
ORDER BY
    ColumnIdentifier = 'NativeColumn' DESC,
    ColumnIdentifier = 'ResourceAttributes' DESC,
    ColumnIdentifier ASC,
    Key ASC
LIMIT 100
[...]
100 rows in set. Elapsed: 95.060 sec. Processed 4.44 billion rows, 316.08 GB (46.71 million rows/s., 3.33 GB/s.)
Peak memory usage: 505.17 MiB.

TTL for the table is 20 days

SELECT COUNT(*)
FROM global_logs_kv_rollup_15m

Query id: e9aca6a8-f4da-476a-aeeb-b907c477b2ea

   ┌────COUNT()─┐
1. │ 4484231159 │ -- 4.48 billion
   └────────────┘

I suppose over a 2 week timespan this is pretty inefficient, especially since probably ~95% of the keys and values are probably fetched in the first couple seconds. I'm working on a tangential filters overhaul and can fix this along with it.

I'm again noticing the filters sidebar timing out.
Updated from 2.29.0 to 2.43.1 and now in console I see:

[2026-11-05T12:29:48.592Z][WARN][@clickhouse/client][Config] request_timeout is set to 3600000ms, but send_progress_in_http_headers is not enabled. Long-running queries may fail with socket hang-up errors if they exceed the load balancer idle timeout. Consider enabling progress headers with clickhouse_settings: { send_progress_in_http_headers: 1, http_headers_progress_interval_ms: '<interval>' }. See https://github.com/ClickHouse/clickhouse-js/blob/main/docs/howto/long_running_queries.md for more details.

getKeyValues (raw table) failed; skipping this strategy for the current batch SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data
    NextJS 2

Absolutely guessing, but might this be that the key rollup has field names that the kv rollup doesn't have?

Navigation Menu

Repository navigation

Filters sidebar is timing out #2589

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Filters sidebar is timing out #2589

Description

Activity

knudtty commented on Jul 10, 2026

czephyr commented on Jul 14, 2026

czephyr commented on Jul 14, 2026

knudtty commented on Jul 16, 2026

czephyr commented on Nov 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions