Preserve short IN lists for dictionary strings and binary values by Rich-T-kid · Pull Request #26946 · apache/datafusion

Rich-T-kid · 2026-10-30T18:52:37Z

Which issue does this PR close?

Rationale for this change

For dictionary string and binary columns, rewriting a short IN list into equality chains repeats dictionary evaluation and key expansion for each value. Keeping the membership expression evaluates the dictionary values once and expands one result through the keys.

For example, given:

values = ["MAIL", "SHIP", "RAIL"]
keys   = [0, 2, 0, 1]

dict_col IN ('MAIL', 'SHIP') previously compared the dictionary values with MAIL, expanded that result through the keys, then repeated the work for SHIP before combining the two row-level results. This change checks the dictionary values against both literals together and expands through the keys once.

TPC-H Q12 has two of these expressions when Parquet RLE-to-dictionary conversion is enabled.

What changes are included in this PR?

The short IN simplifier now retains literal IN and NOT IN expressions for dictionaries with Utf8, Utf8View, Binary, or LargeBinary values. Other string and binary columns keep the existing rewrite behavior.

The existing type-based simplifier test covers the supported dictionary value types.

Are there any user-facing changes?

No.

Rich-T-kid · 2026-10-30T18:52:56Z

run benchmark tpch tpcds clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "841faafb6971aa0d180e93098a62974a71095aea"

adriangbot · 2026-10-30T18:55:51Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6530995846-4160-jzlx8 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 841faaf (841faaf) to 926901f diff

Run configuration

run benchmark tpch
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "841faafb6971aa0d180e93098a62974a71095aea"

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-10-30T18:57:35Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6530995846-4161-cwqnj 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 841faaf (841faaf) to 926901f diff

Run configuration

run benchmark tpcds
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "841faafb6971aa0d180e93098a62974a71095aea"

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-10-30T18:57:37Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6530995846-4162-mc6fs 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 841faaf (841faaf) to 926901f diff

Run configuration

run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "841faafb6971aa0d180e93098a62974a71095aea"

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:02:33Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 841faaf (841faaf) to 926901f diff

Run configuration

run benchmark tpch
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "841faafb6971aa0d180e93098a62974a71095aea"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃    Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 1  │  41.53 ms │                               41.05 ms │ no change │
│ QQuery 2  │  19.25 ms │                               19.08 ms │ no change │
│ QQuery 3  │  29.48 ms │                               29.05 ms │ no change │
│ QQuery 4  │  17.79 ms │                               17.89 ms │ no change │
│ QQuery 5  │  36.66 ms │                               36.83 ms │ no change │
│ QQuery 6  │  14.04 ms │                               14.14 ms │ no change │
│ QQuery 7  │  43.59 ms │                               44.18 ms │ no change │
│ QQuery 8  │  42.15 ms │                               42.20 ms │ no change │
│ QQuery 9  │  51.49 ms │                               52.37 ms │ no change │
│ QQuery 10 │  61.09 ms │                               61.20 ms │ no change │
│ QQuery 11 │  14.32 ms │                               14.06 ms │ no change │
│ QQuery 12 │  21.78 ms │                               21.22 ms │ no change │
│ QQuery 13 │  49.37 ms │                               49.81 ms │ no change │
│ QQuery 14 │  24.40 ms │                               24.87 ms │ no change │
│ QQuery 15 │  31.79 ms │                               32.21 ms │ no change │
│ QQuery 16 │  13.90 ms │                               13.93 ms │ no change │
│ QQuery 17 │  80.61 ms │                               79.54 ms │ no change │
│ QQuery 18 │ 121.46 ms │                              117.61 ms │ no change │
│ QQuery 19 │  30.86 ms │                               30.95 ms │ no change │
│ QQuery 20 │  35.40 ms │                               35.84 ms │ no change │
│ QQuery 21 │  57.20 ms │                               57.40 ms │ no change │
│ QQuery 22 │  16.46 ms │                               16.59 ms │ no change │
└───────────┴───────────┴────────────────────────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 854.63ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 852.02ms │
│ Average Time (HEAD)                                   │  38.85ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  38.73ms │
│ Queries Faster                                        │        0 │
│ Queries Slower                                        │        0 │
│ Queries with No Change                                │       22 │
│ Queries with Failure                                  │        0 │
└───────────────────────────────────────────────────────┴──────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query     ┃                              HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃    Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 1  │    41.53 / 42.02 ±0.90 / 43.82 ms │         41.05 / 41.88 ±1.11 / 44.08 ms │ no change │
│ QQuery 2  │    19.25 / 19.67 ±0.28 / 20.00 ms │         19.08 / 19.38 ±0.24 / 19.77 ms │ no change │
│ QQuery 3  │    29.48 / 30.21 ±1.06 / 32.32 ms │         29.05 / 29.24 ±0.12 / 29.39 ms │ no change │
│ QQuery 4  │    17.79 / 18.62 ±0.75 / 19.52 ms │         17.89 / 18.33 ±0.66 / 19.64 ms │ no change │
│ QQuery 5  │    36.66 / 37.20 ±0.39 / 37.85 ms │         36.83 / 36.97 ±0.11 / 37.14 ms │ no change │
│ QQuery 6  │    14.04 / 14.16 ±0.10 / 14.28 ms │         14.14 / 14.43 ±0.27 / 14.91 ms │ no change │
│ QQuery 7  │    43.59 / 45.90 ±2.30 / 50.08 ms │         44.18 / 46.57 ±2.50 / 50.97 ms │ no change │
│ QQuery 8  │    42.15 / 44.09 ±1.37 / 45.74 ms │         42.20 / 43.03 ±0.88 / 44.65 ms │ no change │
│ QQuery 9  │    51.49 / 52.64 ±0.75 / 53.61 ms │         52.37 / 53.08 ±0.62 / 54.06 ms │ no change │
│ QQuery 10 │    61.09 / 61.51 ±0.31 / 62.05 ms │         61.20 / 62.18 ±1.21 / 64.48 ms │ no change │
│ QQuery 11 │    14.32 / 14.47 ±0.14 / 14.65 ms │         14.06 / 14.60 ±0.79 / 16.17 ms │ no change │
│ QQuery 12 │    21.78 / 22.84 ±1.93 / 26.70 ms │         21.22 / 22.77 ±1.68 / 25.92 ms │ no change │
│ QQuery 13 │    49.37 / 49.93 ±0.36 / 50.34 ms │         49.81 / 50.50 ±0.36 / 50.79 ms │ no change │
│ QQuery 14 │    24.40 / 24.68 ±0.20 / 24.99 ms │         24.87 / 24.96 ±0.08 / 25.10 ms │ no change │
│ QQuery 15 │    31.79 / 31.96 ±0.13 / 32.18 ms │         32.21 / 32.43 ±0.29 / 32.98 ms │ no change │
│ QQuery 16 │    13.90 / 14.04 ±0.13 / 14.22 ms │         13.93 / 14.11 ±0.14 / 14.31 ms │ no change │
│ QQuery 17 │    80.61 / 81.47 ±0.63 / 82.42 ms │         79.54 / 82.03 ±1.85 / 84.63 ms │ no change │
│ QQuery 18 │ 121.46 / 122.24 ±0.53 / 122.78 ms │      117.61 / 121.53 ±3.65 / 126.95 ms │ no change │
│ QQuery 19 │    30.86 / 33.38 ±2.30 / 37.21 ms │         30.95 / 32.34 ±2.24 / 36.81 ms │ no change │
│ QQuery 20 │    35.40 / 36.09 ±0.83 / 37.70 ms │         35.84 / 36.12 ±0.33 / 36.53 ms │ no change │
│ QQuery 21 │    57.20 / 59.56 ±4.45 / 68.46 ms │         57.40 / 59.31 ±1.34 / 61.27 ms │ no change │
│ QQuery 22 │    16.46 / 16.79 ±0.30 / 17.31 ms │         16.59 / 16.75 ±0.12 / 16.89 ms │ no change │
└───────────┴───────────────────────────────────┴────────────────────────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 873.48ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 872.52ms │
│ Average Time (HEAD)                                   │  39.70ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  39.66ms │
│ Queries Faster                                        │        0 │
│ Queries Slower                                        │        0 │
│ Queries with No Change                                │       22 │
│ Queries with Failure                                  │        0 │
└───────────────────────────────────────────────────────┴──────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 926901f9001977e85df617b96328801b42f6af4a | Changed: 841faafb6971aa0d180e93098a62974a71095aea

tpch — tpch_sf1

Query	Base	Changed	Change
Query 1	30.1 MiB	31.2 MiB	+3.8%
Query 2	23.1 MiB	23.1 MiB	+0.0%
Query 3	11.5 MiB	11.5 MiB	+0.0%
Query 4	32.2 MiB	32.2 MiB	+0.0%
Query 5	42.7 MiB	42.8 MiB	+0.1%
Query 6	832 B	832 B	+0.0%
Query 7	107.6 MiB	107.6 MiB	+0.0%
Query 8	34.0 MiB	34.0 MiB	-0.0%
Query 9	155.6 MiB	155.6 MiB	+0.0%
Query 10	65.2 MiB	61.2 MiB	-6.1%
Query 11	61.7 MiB	62.1 MiB	+0.6%
Query 12	10.8 MiB	20.1 MiB	+85.5%
Query 13	72.3 MiB	53.0 MiB	-26.8%
Query 14	7.7 MiB	7.7 MiB	+0.0%
Query 15	12.7 MiB	13.0 MiB	+2.4%
Query 16	33.3 MiB	42.8 MiB	+28.6%
Query 17	150.3 MiB	150.3 MiB	+0.0%
Query 18	2.7 GiB	2.7 GiB	-0.0%
Query 19	228.6 KiB	228.6 KiB	+0.0%
Query 20	60.2 MiB	58.7 MiB	-2.6%
Query 21	37.7 MiB	37.7 MiB	+0.0%
Query 22	31.8 MiB	22.0 MiB	-30.9%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark	Side	Max pool peak	Peak RSS	Gap	RSS / pool
tpch	base (`926901f9001977e85df617b96328801b42f6af4a`)	2.7 GiB	1.3 GiB	−1.4 GiB	0.5×
tpch	changed (`841faafb6971aa0d180e93098a62974a71095aea`)	2.7 GiB	1.3 GiB	−1.4 GiB	0.5×

Resource Usage

tpch — base (merge-base)

Metric	Value
Wall time	5.0s
Peak memory	1.3 GiB
Avg memory	683.5 MiB
CPU user	21.8s
CPU sys	1.7s
Peak spill	0 B

tpch — branch

Metric	Value
Wall time	5.0s
Peak memory	1.3 GiB
Avg memory	692.2 MiB
CPU user	21.8s
CPU sys	1.7s
Peak spill	0 B

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:04:33Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 841faaf (841faaf) to 926901f diff

Run configuration

run benchmark tpcds
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "841faafb6971aa0d180e93098a62974a71095aea"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │    5.63 ms │                                5.23 ms │ +1.08x faster │
│ QQuery 2  │   76.68 ms │                               76.63 ms │     no change │
│ QQuery 3  │   28.66 ms │                               28.89 ms │     no change │
│ QQuery 4  │  857.78 ms │                              876.03 ms │     no change │
│ QQuery 5  │   51.72 ms │                               49.58 ms │     no change │
│ QQuery 6  │   34.21 ms │                               33.49 ms │     no change │
│ QQuery 7  │   74.08 ms │                               73.28 ms │     no change │
│ QQuery 8  │   38.94 ms │                               37.68 ms │     no change │
│ QQuery 9  │   51.64 ms │                               50.50 ms │     no change │
│ QQuery 10 │   61.62 ms │                               61.43 ms │     no change │
│ QQuery 11 │  503.55 ms │                              500.32 ms │     no change │
│ QQuery 12 │   26.51 ms │                               26.04 ms │     no change │
│ QQuery 13 │  111.94 ms │                              113.11 ms │     no change │
│ QQuery 14 │  386.68 ms │                              388.38 ms │     no change │
│ QQuery 15 │   55.06 ms │                               47.98 ms │ +1.15x faster │
│ QQuery 16 │    6.27 ms │                                6.71 ms │  1.07x slower │
│ QQuery 17 │   79.95 ms │                               80.31 ms │     no change │
│ QQuery 18 │  107.63 ms │                              110.53 ms │     no change │
│ QQuery 19 │   41.29 ms │                               41.64 ms │     no change │
│ QQuery 20 │   31.04 ms │                               30.95 ms │     no change │
│ QQuery 21 │   16.91 ms │                               17.44 ms │     no change │
│ QQuery 22 │   72.22 ms │                               72.56 ms │     no change │
│ QQuery 23 │  302.74 ms │                              305.46 ms │     no change │
│ QQuery 24 │  192.84 ms │                              191.44 ms │     no change │
│ QQuery 25 │  108.08 ms │                              109.89 ms │     no change │
│ QQuery 26 │   47.11 ms │                               46.82 ms │     no change │
│ QQuery 27 │    6.40 ms │                                5.93 ms │ +1.08x faster │
│ QQuery 28 │   60.25 ms │                               60.71 ms │     no change │
│ QQuery 29 │   99.30 ms │                               96.18 ms │     no change │
│ QQuery 30 │   37.52 ms │                               36.42 ms │     no change │
│ QQuery 31 │  114.09 ms │                              111.01 ms │     no change │
│ QQuery 32 │   21.72 ms │                               19.89 ms │ +1.09x faster │
│ QQuery 33 │   39.07 ms │                               38.59 ms │     no change │
│ QQuery 34 │    9.90 ms │                                9.90 ms │     no change │
│ QQuery 35 │   74.06 ms │                               70.88 ms │     no change │
│ QQuery 36 │    6.45 ms │                                6.42 ms │     no change │
│ QQuery 37 │    7.86 ms │                                8.00 ms │     no change │
│ QQuery 38 │   65.91 ms │                               69.22 ms │  1.05x slower │
│ QQuery 39 │   74.68 ms │                               72.02 ms │     no change │
│ QQuery 40 │   24.29 ms │                               22.34 ms │ +1.09x faster │
│ QQuery 41 │   14.52 ms │                               10.58 ms │ +1.37x faster │
│ QQuery 42 │   23.97 ms │                               23.71 ms │     no change │
│ QQuery 43 │    4.97 ms │                                4.80 ms │     no change │
│ QQuery 44 │    8.97 ms │                                8.33 ms │ +1.08x faster │
│ QQuery 45 │   35.52 ms │                               34.88 ms │     no change │
│ QQuery 46 │   12.15 ms │                               11.24 ms │ +1.08x faster │
│ QQuery 47 │  210.82 ms │                              203.42 ms │     no change │
│ QQuery 48 │   91.21 ms │                               90.79 ms │     no change │
│ QQuery 49 │   69.84 ms │                               70.77 ms │     no change │
│ QQuery 50 │   58.06 ms │                               58.30 ms │     no change │
│ QQuery 51 │   93.16 ms │                               92.40 ms │     no change │
│ QQuery 52 │   24.48 ms │                               24.89 ms │     no change │
│ QQuery 53 │   29.60 ms │                               29.10 ms │     no change │
│ QQuery 54 │   53.50 ms │                               53.75 ms │     no change │
│ QQuery 55 │   24.21 ms │                               24.38 ms │     no change │
│ QQuery 56 │   39.80 ms │                               38.52 ms │     no change │
│ QQuery 57 │  152.81 ms │                              153.34 ms │     no change │
│ QQuery 58 │  103.05 ms │                              102.96 ms │     no change │
│ QQuery 59 │  106.99 ms │                              107.47 ms │     no change │
│ QQuery 60 │   43.09 ms │                               43.49 ms │     no change │
│ QQuery 61 │   10.30 ms │                                9.98 ms │     no change │
│ QQuery 62 │   38.85 ms │                               38.49 ms │     no change │
│ QQuery 63 │   29.79 ms │                               28.90 ms │     no change │
│ QQuery 64 │  376.39 ms │                              370.93 ms │     no change │
│ QQuery 65 │  133.87 ms │                              134.34 ms │     no change │
│ QQuery 66 │   73.52 ms │                               71.63 ms │     no change │
│ QQuery 67 │  238.95 ms │                              242.04 ms │     no change │
│ QQuery 68 │   11.22 ms │                               11.83 ms │  1.05x slower │
│ QQuery 69 │   56.15 ms │                               56.47 ms │     no change │
│ QQuery 70 │   98.34 ms │                               98.00 ms │     no change │
│ QQuery 71 │   35.83 ms │                               35.37 ms │     no change │
│ QQuery 72 │ 1590.91 ms │                             1621.25 ms │     no change │
│ QQuery 73 │    9.84 ms │                                9.63 ms │     no change │
│ QQuery 74 │  248.93 ms │                              255.70 ms │     no change │
│ QQuery 75 │  139.31 ms │                              137.86 ms │     no change │
│ QQuery 76 │   35.49 ms │                               34.61 ms │     no change │
│ QQuery 77 │   60.53 ms │                               58.88 ms │     no change │
│ QQuery 78 │  163.86 ms │                              158.93 ms │     no change │
│ QQuery 79 │   70.64 ms │                               67.73 ms │     no change │
│ QQuery 80 │   98.39 ms │                               95.24 ms │     no change │
│ QQuery 81 │   30.02 ms │                               28.35 ms │ +1.06x faster │
│ QQuery 82 │   18.05 ms │                               17.78 ms │     no change │
│ QQuery 83 │   33.62 ms │                               32.86 ms │     no change │
│ QQuery 84 │   32.18 ms │                               30.83 ms │     no change │
│ QQuery 85 │  100.24 ms │                               99.96 ms │     no change │
│ QQuery 86 │   25.99 ms │                               24.28 ms │ +1.07x faster │
│ QQuery 87 │   66.69 ms │                               65.02 ms │     no change │
│ QQuery 88 │   60.73 ms │                               58.36 ms │     no change │
│ QQuery 89 │   36.40 ms │                               35.91 ms │     no change │
│ QQuery 90 │   15.85 ms │                               16.47 ms │     no change │
│ QQuery 91 │   42.39 ms │                               41.54 ms │     no change │
│ QQuery 92 │   29.91 ms │                               28.78 ms │     no change │
│ QQuery 93 │   49.47 ms │                               48.34 ms │     no change │
│ QQuery 94 │   38.98 ms │                               40.12 ms │     no change │
│ QQuery 95 │   81.28 ms │                               81.33 ms │     no change │
│ QQuery 96 │   23.81 ms │                               23.66 ms │     no change │
│ QQuery 97 │   50.79 ms │                               50.21 ms │     no change │
│ QQuery 98 │   38.70 ms │                               39.33 ms │     no change │
│ QQuery 99 │   56.70 ms │                               57.74 ms │     no change │
└───────────┴────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 9565.93ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 9555.62ms │
│ Average Time (HEAD)                                   │   96.63ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │   96.52ms │
│ Queries Faster                                        │        10 │
│ Queries Slower                                        │         3 │
│ Queries with No Change                                │        86 │
│ Queries with Failure                                  │         0 │
└───────────────────────────────────────────────────────┴───────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │           5.63 / 6.28 ±1.12 / 8.51 ms │            5.23 / 5.88 ±1.05 / 7.97 ms │ +1.07x faster │
│ QQuery 2  │        76.68 / 77.24 ±0.53 / 78.18 ms │         76.63 / 77.09 ±0.54 / 78.12 ms │     no change │
│ QQuery 3  │        28.66 / 29.27 ±0.39 / 29.86 ms │         28.89 / 29.59 ±0.46 / 30.23 ms │     no change │
│ QQuery 4  │    857.78 / 897.96 ±25.67 / 931.92 ms │     876.03 / 894.00 ±10.97 / 906.49 ms │     no change │
│ QQuery 5  │        51.72 / 52.83 ±0.86 / 54.36 ms │         49.58 / 51.01 ±1.44 / 53.74 ms │     no change │
│ QQuery 6  │        34.21 / 35.04 ±0.50 / 35.67 ms │         33.49 / 33.95 ±0.44 / 34.53 ms │     no change │
│ QQuery 7  │        74.08 / 74.52 ±0.38 / 75.23 ms │         73.28 / 74.64 ±0.92 / 75.79 ms │     no change │
│ QQuery 8  │        38.94 / 41.26 ±3.43 / 48.06 ms │         37.68 / 39.94 ±2.95 / 45.76 ms │     no change │
│ QQuery 9  │        51.64 / 55.28 ±2.06 / 57.33 ms │         50.50 / 54.29 ±3.11 / 59.75 ms │     no change │
│ QQuery 10 │        61.62 / 61.98 ±0.25 / 62.33 ms │         61.43 / 62.34 ±0.94 / 63.87 ms │     no change │
│ QQuery 11 │    503.55 / 516.63 ±10.66 / 531.85 ms │      500.32 / 511.32 ±9.52 / 524.93 ms │     no change │
│ QQuery 12 │        26.51 / 27.22 ±0.46 / 27.95 ms │         26.04 / 26.56 ±0.84 / 28.22 ms │     no change │
│ QQuery 13 │     111.94 / 113.21 ±1.12 / 115.11 ms │      113.11 / 116.11 ±2.04 / 118.76 ms │     no change │
│ QQuery 14 │     386.68 / 392.66 ±5.43 / 401.12 ms │      388.38 / 393.76 ±4.10 / 400.34 ms │     no change │
│ QQuery 15 │        55.06 / 59.25 ±4.73 / 68.21 ms │         47.98 / 50.70 ±2.63 / 55.69 ms │ +1.17x faster │
│ QQuery 16 │           6.27 / 6.62 ±0.37 / 7.27 ms │            6.71 / 6.86 ±0.12 / 7.01 ms │     no change │
│ QQuery 17 │        79.95 / 82.26 ±2.88 / 87.64 ms │         80.31 / 82.55 ±2.28 / 86.19 ms │     no change │
│ QQuery 18 │     107.63 / 109.84 ±1.57 / 112.42 ms │      110.53 / 111.91 ±1.24 / 113.70 ms │     no change │
│ QQuery 19 │        41.29 / 42.35 ±0.81 / 43.66 ms │         41.64 / 41.98 ±0.38 / 42.65 ms │     no change │
│ QQuery 20 │        31.04 / 31.48 ±0.39 / 32.05 ms │         30.95 / 31.66 ±0.57 / 32.66 ms │     no change │
│ QQuery 21 │        16.91 / 17.22 ±0.27 / 17.61 ms │         17.44 / 18.11 ±0.52 / 18.90 ms │  1.05x slower │
│ QQuery 22 │        72.22 / 73.69 ±1.48 / 75.60 ms │         72.56 / 74.57 ±1.60 / 77.47 ms │     no change │
│ QQuery 23 │     302.74 / 315.45 ±7.61 / 323.99 ms │      305.46 / 312.37 ±6.28 / 322.24 ms │     no change │
│ QQuery 24 │     192.84 / 201.74 ±5.92 / 207.80 ms │      191.44 / 195.92 ±3.02 / 200.68 ms │     no change │
│ QQuery 25 │     108.08 / 111.76 ±3.52 / 117.69 ms │      109.89 / 112.32 ±1.81 / 114.21 ms │     no change │
│ QQuery 26 │        47.11 / 47.77 ±0.55 / 48.41 ms │         46.82 / 47.56 ±0.73 / 48.86 ms │     no change │
│ QQuery 27 │           6.40 / 6.55 ±0.14 / 6.79 ms │            5.93 / 6.11 ±0.19 / 6.42 ms │ +1.07x faster │
│ QQuery 28 │        60.25 / 61.24 ±0.83 / 62.62 ms │         60.71 / 62.48 ±2.31 / 67.03 ms │     no change │
│ QQuery 29 │      99.30 / 100.76 ±2.10 / 104.90 ms │      96.18 / 110.54 ±23.67 / 157.69 ms │  1.10x slower │
│ QQuery 30 │        37.52 / 38.45 ±0.70 / 39.62 ms │         36.42 / 37.54 ±0.72 / 38.54 ms │     no change │
│ QQuery 31 │     114.09 / 117.42 ±2.42 / 120.73 ms │      111.01 / 113.55 ±3.56 / 120.47 ms │     no change │
│ QQuery 32 │        21.72 / 22.49 ±0.92 / 24.24 ms │         19.89 / 20.60 ±0.69 / 21.75 ms │ +1.09x faster │
│ QQuery 33 │        39.07 / 39.83 ±0.51 / 40.59 ms │         38.59 / 40.66 ±1.94 / 44.23 ms │     no change │
│ QQuery 34 │         9.90 / 10.23 ±0.24 / 10.59 ms │          9.90 / 10.11 ±0.23 / 10.56 ms │     no change │
│ QQuery 35 │        74.06 / 77.05 ±2.72 / 82.09 ms │         70.88 / 73.52 ±2.64 / 77.89 ms │     no change │
│ QQuery 36 │           6.45 / 6.64 ±0.12 / 6.77 ms │            6.42 / 6.53 ±0.08 / 6.68 ms │     no change │
│ QQuery 37 │           7.86 / 8.38 ±0.46 / 9.14 ms │           8.00 / 8.92 ±1.58 / 12.08 ms │  1.07x slower │
│ QQuery 38 │        65.91 / 67.64 ±1.54 / 70.06 ms │         69.22 / 73.54 ±2.51 / 75.87 ms │  1.09x slower │
│ QQuery 39 │        74.68 / 76.23 ±1.53 / 78.32 ms │         72.02 / 73.13 ±1.05 / 74.63 ms │     no change │
│ QQuery 40 │        24.29 / 25.09 ±0.82 / 26.57 ms │         22.34 / 22.68 ±0.25 / 22.94 ms │ +1.11x faster │
│ QQuery 41 │        14.52 / 14.67 ±0.09 / 14.78 ms │         10.58 / 10.78 ±0.21 / 11.19 ms │ +1.36x faster │
│ QQuery 42 │        23.97 / 25.59 ±1.89 / 29.30 ms │         23.71 / 24.15 ±0.40 / 24.73 ms │ +1.06x faster │
│ QQuery 43 │           4.97 / 5.13 ±0.14 / 5.35 ms │            4.80 / 5.01 ±0.22 / 5.36 ms │     no change │
│ QQuery 44 │           8.97 / 9.13 ±0.14 / 9.35 ms │            8.33 / 8.51 ±0.14 / 8.74 ms │ +1.07x faster │
│ QQuery 45 │        35.52 / 37.96 ±2.00 / 41.17 ms │         34.88 / 35.48 ±0.52 / 36.36 ms │ +1.07x faster │
│ QQuery 46 │        12.15 / 12.41 ±0.16 / 12.59 ms │         11.24 / 11.51 ±0.15 / 11.65 ms │ +1.08x faster │
│ QQuery 47 │     210.82 / 217.12 ±7.85 / 232.19 ms │      203.42 / 209.83 ±4.34 / 216.01 ms │     no change │
│ QQuery 48 │        91.21 / 92.54 ±1.13 / 94.06 ms │         90.79 / 91.81 ±0.88 / 93.30 ms │     no change │
│ QQuery 49 │        69.84 / 71.86 ±2.91 / 77.59 ms │         70.77 / 72.73 ±2.87 / 78.39 ms │     no change │
│ QQuery 50 │        58.06 / 58.66 ±0.30 / 58.85 ms │         58.30 / 60.08 ±2.40 / 64.60 ms │     no change │
│ QQuery 51 │       93.16 / 97.23 ±3.87 / 104.57 ms │         92.40 / 95.60 ±1.87 / 97.91 ms │     no change │
│ QQuery 52 │        24.48 / 25.00 ±0.32 / 25.35 ms │         24.89 / 25.29 ±0.32 / 25.72 ms │     no change │
│ QQuery 53 │        29.60 / 30.42 ±0.95 / 32.27 ms │         29.10 / 29.81 ±0.56 / 30.57 ms │     no change │
│ QQuery 54 │        53.50 / 54.42 ±0.74 / 55.58 ms │         53.75 / 54.68 ±1.34 / 57.25 ms │     no change │
│ QQuery 55 │        24.21 / 24.81 ±0.32 / 25.11 ms │         24.38 / 24.78 ±0.30 / 25.26 ms │     no change │
│ QQuery 56 │        39.80 / 40.17 ±0.27 / 40.51 ms │         38.52 / 38.81 ±0.23 / 39.08 ms │     no change │
│ QQuery 57 │     152.81 / 155.80 ±2.51 / 160.36 ms │      153.34 / 156.20 ±2.45 / 160.27 ms │     no change │
│ QQuery 58 │     103.05 / 106.11 ±2.24 / 108.83 ms │      102.96 / 106.03 ±2.55 / 109.71 ms │     no change │
│ QQuery 59 │     106.99 / 108.30 ±1.28 / 110.11 ms │      107.47 / 110.08 ±2.83 / 115.55 ms │     no change │
│ QQuery 60 │        43.09 / 44.55 ±1.36 / 47.00 ms │         43.49 / 44.14 ±0.46 / 44.83 ms │     no change │
│ QQuery 61 │        10.30 / 10.42 ±0.10 / 10.56 ms │          9.98 / 10.29 ±0.18 / 10.48 ms │     no change │
│ QQuery 62 │        38.85 / 39.94 ±1.59 / 43.07 ms │         38.49 / 40.14 ±1.72 / 43.31 ms │     no change │
│ QQuery 63 │        29.79 / 30.48 ±0.41 / 30.99 ms │         28.90 / 29.46 ±0.81 / 31.06 ms │     no change │
│ QQuery 64 │     376.39 / 379.89 ±3.54 / 384.94 ms │      370.93 / 378.98 ±5.13 / 385.75 ms │     no change │
│ QQuery 65 │     133.87 / 135.41 ±1.46 / 137.87 ms │      134.34 / 136.43 ±2.05 / 139.05 ms │     no change │
│ QQuery 66 │        73.52 / 76.70 ±3.28 / 82.63 ms │         71.63 / 75.00 ±2.84 / 79.97 ms │     no change │
│ QQuery 67 │     238.95 / 246.45 ±5.86 / 253.11 ms │      242.04 / 250.30 ±7.44 / 261.21 ms │     no change │
│ QQuery 68 │        11.22 / 11.41 ±0.22 / 11.84 ms │         11.83 / 12.16 ±0.30 / 12.64 ms │  1.07x slower │
│ QQuery 69 │        56.15 / 56.99 ±0.69 / 58.05 ms │         56.47 / 57.49 ±0.64 / 58.46 ms │     no change │
│ QQuery 70 │      98.34 / 102.03 ±2.20 / 105.06 ms │       98.00 / 102.46 ±4.56 / 111.08 ms │     no change │
│ QQuery 71 │        35.83 / 36.33 ±0.38 / 36.95 ms │         35.37 / 36.58 ±1.41 / 39.22 ms │     no change │
│ QQuery 72 │ 1590.91 / 1679.55 ±68.13 / 1797.13 ms │  1621.25 / 1725.11 ±67.97 / 1796.58 ms │     no change │
│ QQuery 73 │         9.84 / 10.12 ±0.27 / 10.64 ms │           9.63 / 9.88 ±0.19 / 10.18 ms │     no change │
│ QQuery 74 │    248.93 / 265.44 ±13.46 / 287.35 ms │      255.70 / 269.66 ±8.95 / 283.52 ms │     no change │
│ QQuery 75 │     139.31 / 145.25 ±4.18 / 151.72 ms │      137.86 / 141.18 ±1.88 / 143.11 ms │     no change │
│ QQuery 76 │        35.49 / 35.94 ±0.33 / 36.45 ms │         34.61 / 35.66 ±1.22 / 38.04 ms │     no change │
│ QQuery 77 │        60.53 / 61.11 ±0.52 / 62.00 ms │         58.88 / 59.93 ±0.94 / 61.52 ms │     no change │
│ QQuery 78 │     163.86 / 174.02 ±8.54 / 187.39 ms │      158.93 / 165.38 ±5.33 / 173.06 ms │     no change │
│ QQuery 79 │        70.64 / 71.43 ±0.74 / 72.41 ms │         67.73 / 68.83 ±0.84 / 69.84 ms │     no change │
│ QQuery 80 │      98.39 / 102.10 ±3.45 / 108.26 ms │         95.24 / 97.08 ±1.58 / 99.84 ms │     no change │
│ QQuery 81 │        30.02 / 30.30 ±0.35 / 30.95 ms │         28.35 / 30.17 ±2.72 / 35.57 ms │     no change │
│ QQuery 82 │        18.05 / 18.17 ±0.11 / 18.32 ms │         17.78 / 17.85 ±0.06 / 17.93 ms │     no change │
│ QQuery 83 │        33.62 / 33.82 ±0.18 / 34.06 ms │         32.86 / 33.43 ±0.33 / 33.76 ms │     no change │
│ QQuery 84 │        32.18 / 32.47 ±0.28 / 32.84 ms │         30.83 / 31.09 ±0.23 / 31.42 ms │     no change │
│ QQuery 85 │     100.24 / 104.39 ±5.32 / 114.68 ms │       99.96 / 103.15 ±4.13 / 111.17 ms │     no change │
│ QQuery 86 │        25.99 / 26.26 ±0.24 / 26.56 ms │         24.28 / 25.09 ±0.47 / 25.55 ms │     no change │
│ QQuery 87 │        66.69 / 67.81 ±1.44 / 70.60 ms │         65.02 / 66.49 ±1.07 / 67.94 ms │     no change │
│ QQuery 88 │        60.73 / 63.59 ±4.68 / 72.88 ms │         58.36 / 61.26 ±3.39 / 67.83 ms │     no change │
│ QQuery 89 │        36.40 / 37.04 ±0.56 / 38.06 ms │         35.91 / 36.74 ±0.60 / 37.76 ms │     no change │
│ QQuery 90 │        15.85 / 16.22 ±0.27 / 16.60 ms │         16.47 / 16.71 ±0.35 / 17.40 ms │     no change │
│ QQuery 91 │        42.39 / 43.02 ±0.38 / 43.47 ms │         41.54 / 41.85 ±0.17 / 42.03 ms │     no change │
│ QQuery 92 │        29.91 / 30.30 ±0.31 / 30.77 ms │         28.78 / 29.78 ±0.81 / 30.74 ms │     no change │
│ QQuery 93 │        49.47 / 51.46 ±1.16 / 52.88 ms │         48.34 / 49.46 ±0.91 / 50.93 ms │     no change │
│ QQuery 94 │        38.98 / 40.03 ±1.17 / 42.31 ms │         40.12 / 41.43 ±1.70 / 44.50 ms │     no change │
│ QQuery 95 │        81.28 / 82.55 ±1.08 / 84.32 ms │         81.33 / 84.18 ±2.97 / 89.66 ms │     no change │
│ QQuery 96 │        23.81 / 24.08 ±0.23 / 24.48 ms │         23.66 / 23.91 ±0.25 / 24.37 ms │     no change │
│ QQuery 97 │        50.79 / 53.44 ±1.72 / 56.10 ms │         50.21 / 51.14 ±0.61 / 51.86 ms │     no change │
│ QQuery 98 │        38.70 / 39.57 ±1.12 / 41.74 ms │         39.33 / 39.60 ±0.17 / 39.87 ms │     no change │
│ QQuery 99 │        56.70 / 57.74 ±0.54 / 58.21 ms │         57.74 / 58.87 ±1.36 / 61.46 ms │     no change │
└───────────┴───────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 9893.91ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 9876.44ms │
│ Average Time (HEAD)                                   │   99.94ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │   99.76ms │
│ Queries Faster                                        │        10 │
│ Queries Slower                                        │         5 │
│ Queries with No Change                                │        84 │
│ Queries with Failure                                  │         0 │
└───────────────────────────────────────────────────────┴───────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 926901f9001977e85df617b96328801b42f6af4a | Changed: 841faafb6971aa0d180e93098a62974a71095aea

tpcds — tpcds_sf1

Query	Base	Changed	Change
Query 1	200.1 KiB	135.4 KiB	-32.3%
Query 2	69.2 MiB	72.8 MiB	+5.2%
Query 3	2.1 MiB	1.9 MiB	-10.4%
Query 4	79.7 MiB	79.7 MiB	+0.0%
Query 5	12.5 MiB	13.0 MiB	+4.2%
Query 6	4.7 MiB	4.7 MiB	-0.2%
Query 7	3.7 MiB	3.7 MiB	-0.4%
Query 8	1.7 MiB	1.7 MiB	-1.0%
Query 9	32.1 KiB	32.1 KiB	+0.0%
Query 10	2.0 MiB	2.0 MiB	+0.0%
Query 11	62.0 MiB	62.0 MiB	-0.1%
Query 12	114.4 MiB	114.4 MiB	+0.0%
Query 13	13.7 MiB	13.7 MiB	+0.0%
Query 14	171.6 MiB	170.9 MiB	-0.4%
Query 15	4.2 MiB	4.1 MiB	-2.3%
Query 16	118.6 KiB	118.6 KiB	+0.0%
Query 17	41.0 MiB	40.6 MiB	-1.1%
Query 18	21.3 MiB	21.2 MiB	-0.3%
Query 19	1.9 MiB	2.1 MiB	+9.7%
Query 20	114.6 MiB	114.6 MiB	-0.0%
Query 21	3.0 MiB	4.1 MiB	+37.0%
Query 22	61.8 MiB	61.8 MiB	+0.0%
Query 23	212.0 MiB	212.6 MiB	+0.2%
Query 24	38.7 MiB	38.7 MiB	+0.0%
Query 25	99.9 MiB	99.9 MiB	+0.0%
Query 26	3.5 MiB	3.6 MiB	+2.6%
Query 27	1.6 MiB	1.6 MiB	+0.0%
Query 28	1.5 MiB	1.3 MiB	-9.0%
Query 29	104.7 MiB	104.7 MiB	+0.0%
Query 30	3.8 MiB	4.0 MiB	+4.7%
Query 31	13.4 MiB	13.4 MiB	+0.0%
Query 32	2.7 MiB	2.7 MiB	+0.0%
Query 33	5.6 MiB	5.5 MiB	-2.7%
Query 34	6.0 MiB	6.0 MiB	+0.0%
Query 35	16.6 MiB	16.6 MiB	+0.0%
Query 36	10.3 MiB	10.3 MiB	+0.0%
Query 37	3.3 MiB	3.3 MiB	+0.0%
Query 38	10.8 MiB	10.4 MiB	-3.2%
Query 39	52.0 MiB	62.9 MiB	+20.9%
Query 40	12.2 MiB	12.2 MiB	+0.0%
Query 41	2.1 MiB	2.1 MiB	+0.0%
Query 42	1.7 MiB	1.3 MiB	-25.2%
Query 43	275.0 KiB	275.0 KiB	+0.0%
Query 44	233.2 KiB	233.2 KiB	+0.0%
Query 45	4.5 MiB	4.5 MiB	+0.0%
Query 46	6.2 MiB	6.2 MiB	+0.0%
Query 47	138.2 MiB	128.5 MiB	-7.0%
Query 48	11.8 MiB	11.8 MiB	+0.0%
Query 49	52.8 MiB	52.4 MiB	-0.8%
Query 50	18.0 MiB	17.3 MiB	-4.0%
Query 51	177.7 MiB	177.7 MiB	+0.0%
Query 52	1.5 MiB	1.9 MiB	+24.1%
Query 53	30.6 MiB	30.4 MiB	-0.5%
Query 54	2.1 MiB	2.1 MiB	+0.0%
Query 55	1.2 MiB	1.5 MiB	+24.1%
Query 56	3.5 MiB	3.3 MiB	-6.1%
Query 57	120.8 MiB	120.8 MiB	+0.0%
Query 58	4.2 MiB	4.4 MiB	+5.1%
Query 59	11.7 MiB	11.7 MiB	+0.0%
Query 60	7.5 MiB	7.2 MiB	-4.4%
Query 61	2.4 KiB	2.4 KiB	+0.7%
Query 62	3.6 MiB	3.3 MiB	-9.0%
Query 63	30.6 MiB	30.6 MiB	+0.0%
Query 64	93.7 MiB	93.7 MiB	-0.0%
Query 65	29.7 MiB	29.7 MiB	+0.0%
Query 66	14.9 MiB	14.9 MiB	+0.0%
Query 67	140.7 MiB	141.3 MiB	+0.4%
Query 68	6.2 MiB	6.2 MiB	+0.0%
Query 69	4.0 MiB	3.7 MiB	-8.5%
Query 70	44.0 MiB	44.0 MiB	+0.0%
Query 71	50.4 MiB	50.4 MiB	-0.0%
Query 72	97.2 MiB	97.2 MiB	+0.0%
Query 73	6.0 MiB	6.0 MiB	+0.0%
Query 74	34.6 MiB	34.6 MiB	-0.0%
Query 75	21.3 MiB	19.6 MiB	-8.2%
Query 76	7.9 MiB	7.9 MiB	+0.0%
Query 77	4.2 MiB	4.1 MiB	-3.2%
Query 78	109.8 MiB	111.1 MiB	+1.2%
Query 79	9.3 MiB	8.0 MiB	-14.0%
Query 80	41.7 MiB	42.0 MiB	+0.8%
Query 81	4.1 MiB	4.0 MiB	-0.6%
Query 82	4.9 MiB	4.9 MiB	+0.0%
Query 83	8.6 MiB	8.6 MiB	+0.0%
Query 84	11.9 MiB	11.9 MiB	+0.0%
Query 85	17.0 MiB	17.0 MiB	+0.0%
Query 86	82.1 MiB	82.1 MiB	+0.0%
Query 87	12.9 MiB	12.9 MiB	+0.0%
Query 88	1.8 MiB	1.6 MiB	-12.7%
Query 89	112.2 MiB	122.0 MiB	+8.8%
Query 90	363.7 KiB	348.6 KiB	-4.2%
Query 91	30.9 MiB	31.0 MiB	+0.3%
Query 92	2.7 MiB	2.7 MiB	+0.0%
Query 93	15.2 MiB	15.4 MiB	+1.4%
Query 94	3.7 MiB	3.7 MiB	-0.0%
Query 95	18.3 MiB	18.4 MiB	+0.3%
Query 96	224.7 KiB	224.7 KiB	+0.0%
Query 97	33.1 MiB	33.1 MiB	+0.0%
Query 98	114.7 MiB	124.8 MiB	+8.7%
Query 99	1.7 MiB	1.9 MiB	+15.3%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark	Side	Max pool peak	Peak RSS	Gap	RSS / pool
tpcds	base (`926901f9001977e85df617b96328801b42f6af4a`)	212.0 MiB	1.8 GiB	1.6 GiB	8.7×
tpcds	changed (`841faafb6971aa0d180e93098a62974a71095aea`)	212.6 MiB	1.8 GiB	1.6 GiB	8.8×

Resource Usage

tpcds — base (merge-base)

Metric	Value
Wall time	50.0s
Peak memory	1.8 GiB
Avg memory	1.3 GiB
CPU user	200.8s
CPU sys	5.7s
Peak spill	0 B

tpcds — branch

Metric	Value
Wall time	50.0s
Peak memory	1.8 GiB
Avg memory	1.3 GiB
CPU user	197.6s
CPU sys	5.7s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-10-30T19:13:24Z

run benchmark tpch tpcds clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "a7aa6d133e1b9ab56c2a81dec061ed6e847178ee"

adriangbot · 2026-10-30T19:16:31Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6531446836-4172-m95kj 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing a7aa6d1 (a7aa6d1) to 926901f diff

Run configuration

run benchmark tpcds
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "a7aa6d133e1b9ab56c2a81dec061ed6e847178ee"

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:17:07Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6531446836-4171-s8w7b 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing a7aa6d1 (a7aa6d1) to 926901f diff

Run configuration

run benchmark tpch
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "a7aa6d133e1b9ab56c2a81dec061ed6e847178ee"

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:17:13Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6531446836-4173-995x7 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing a7aa6d1 (a7aa6d1) to 926901f diff

Run configuration

run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "a7aa6d133e1b9ab56c2a81dec061ed6e847178ee"

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:17:45Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 841faaf (841faaf) to 926901f diff

Run configuration

run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "841faafb6971aa0d180e93098a62974a71095aea"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     1.16 ms │                                1.15 ms │     no change │
│ QQuery 1  │    11.16 ms │                               11.13 ms │     no change │
│ QQuery 2  │    35.38 ms │                               35.90 ms │     no change │
│ QQuery 3  │    31.28 ms │                               31.28 ms │     no change │
│ QQuery 4  │   237.96 ms │                              245.19 ms │     no change │
│ QQuery 5  │  1472.05 ms │                             1555.94 ms │  1.06x slower │
│ QQuery 6  │     1.19 ms │                                1.22 ms │     no change │
│ QQuery 7  │    12.39 ms │                               12.63 ms │     no change │
│ QQuery 8  │   342.53 ms │                              348.86 ms │     no change │
│ QQuery 9  │   486.91 ms │                              500.62 ms │     no change │
│ QQuery 10 │   199.96 ms │                              200.91 ms │     no change │
│ QQuery 11 │   210.50 ms │                              212.77 ms │     no change │
│ QQuery 12 │  1542.78 ms │                             1531.28 ms │     no change │
│ QQuery 13 │  2102.60 ms │                             2123.68 ms │     no change │
│ QQuery 14 │  1660.80 ms │                             1671.91 ms │     no change │
│ QQuery 15 │   299.10 ms │                              299.00 ms │     no change │
│ QQuery 16 │  3631.63 ms │                             3572.88 ms │     no change │
│ QQuery 17 │  3745.50 ms │                             3789.22 ms │     no change │
│ QQuery 18 │  7060.72 ms │                             6843.49 ms │     no change │
│ QQuery 19 │    26.74 ms │                               26.74 ms │     no change │
│ QQuery 20 │  1057.52 ms │                             1056.88 ms │     no change │
│ QQuery 21 │  1367.23 ms │                             1376.71 ms │     no change │
│ QQuery 22 │  3215.96 ms │                             3227.31 ms │     no change │
│ QQuery 23 │  9211.07 ms │                             9340.84 ms │     no change │
│ QQuery 24 │    97.02 ms │                               96.75 ms │     no change │
│ QQuery 25 │   352.15 ms │                              353.53 ms │     no change │
│ QQuery 26 │    97.97 ms │                               98.37 ms │     no change │
│ QQuery 27 │  1147.38 ms │                             1144.12 ms │     no change │
│ QQuery 28 │  2811.13 ms │                             2801.86 ms │     no change │
│ QQuery 29 │    38.75 ms │                               38.59 ms │     no change │
│ QQuery 30 │   550.51 ms │                              554.25 ms │     no change │
│ QQuery 31 │   533.62 ms │                              542.53 ms │     no change │
│ QQuery 32 │  1097.07 ms │                             1039.52 ms │ +1.06x faster │
│ QQuery 33 │ 14081.37 ms │                            13949.06 ms │     no change │
│ QQuery 34 │ 14148.61 ms │                            14374.34 ms │     no change │
│ QQuery 35 │   298.50 ms │                              314.20 ms │  1.05x slower │
│ QQuery 36 │   181.73 ms │                              173.90 ms │     no change │
│ QQuery 37 │    96.87 ms │                               97.84 ms │     no change │
│ QQuery 38 │    78.48 ms │                               77.28 ms │     no change │
│ QQuery 39 │   416.23 ms │                              415.52 ms │     no change │
│ QQuery 40 │    14.38 ms │                               14.21 ms │     no change │
│ QQuery 41 │    13.62 ms │                               13.75 ms │     no change │
│ QQuery 42 │    13.21 ms │                               13.30 ms │     no change │
└───────────┴─────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 74032.72ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 74130.45ms │
│ Average Time (HEAD)                                   │  1721.69ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  1723.96ms │
│ Queries Faster                                        │          1 │
│ Queries Slower                                        │          2 │
│ Queries with No Change                                │         40 │
│ Queries with Failure                                  │          0 │
└───────────────────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                      HEAD ┃    rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │              1.16 / 4.28 ±6.09 / 16.45 ms │              1.15 / 4.31 ±6.14 / 16.58 ms │     no change │
│ QQuery 1  │            11.16 / 11.26 ±0.08 / 11.36 ms │            11.13 / 11.28 ±0.15 / 11.48 ms │     no change │
│ QQuery 2  │            35.38 / 35.82 ±0.41 / 36.38 ms │            35.90 / 36.32 ±0.26 / 36.64 ms │     no change │
│ QQuery 3  │            31.28 / 31.97 ±1.08 / 34.13 ms │            31.28 / 31.74 ±0.38 / 32.37 ms │     no change │
│ QQuery 4  │         237.96 / 242.80 ±3.08 / 246.17 ms │         245.19 / 248.23 ±3.34 / 253.93 ms │     no change │
│ QQuery 5  │     1472.05 / 1546.70 ±80.78 / 1658.33 ms │     1555.94 / 1618.77 ±61.76 / 1722.61 ms │     no change │
│ QQuery 6  │               1.19 / 1.34 ±0.24 / 1.81 ms │               1.22 / 1.38 ±0.24 / 1.84 ms │     no change │
│ QQuery 7  │            12.39 / 12.73 ±0.18 / 12.90 ms │            12.63 / 12.82 ±0.16 / 13.06 ms │     no change │
│ QQuery 8  │         342.53 / 353.84 ±6.05 / 358.93 ms │         348.86 / 353.96 ±4.61 / 359.81 ms │     no change │
│ QQuery 9  │        486.91 / 503.98 ±10.19 / 518.96 ms │        500.62 / 524.14 ±19.48 / 549.54 ms │     no change │
│ QQuery 10 │         199.96 / 202.21 ±3.12 / 208.38 ms │         200.91 / 202.94 ±1.47 / 205.10 ms │     no change │
│ QQuery 11 │         210.50 / 211.71 ±1.12 / 213.50 ms │         212.77 / 215.16 ±2.81 / 220.68 ms │     no change │
│ QQuery 12 │     1542.78 / 1584.87 ±28.83 / 1615.64 ms │     1531.28 / 1607.27 ±43.84 / 1654.71 ms │     no change │
│ QQuery 13 │    2102.60 / 2537.05 ±344.09 / 3086.47 ms │    2123.68 / 2539.61 ±421.67 / 3206.21 ms │     no change │
│ QQuery 14 │     1660.80 / 1708.60 ±42.92 / 1777.36 ms │     1671.91 / 1743.79 ±51.47 / 1813.71 ms │     no change │
│ QQuery 15 │         299.10 / 311.80 ±9.83 / 329.15 ms │        299.00 / 314.98 ±13.62 / 338.65 ms │     no change │
│ QQuery 16 │     3631.63 / 3759.30 ±93.69 / 3863.51 ms │    3572.88 / 3788.68 ±137.14 / 3947.42 ms │     no change │
│ QQuery 17 │    3745.50 / 3848.90 ±112.11 / 4048.89 ms │     3789.22 / 3868.53 ±64.46 / 3957.99 ms │     no change │
│ QQuery 18 │     7060.72 / 7167.09 ±74.60 / 7250.14 ms │    6843.49 / 7093.62 ±210.79 / 7483.44 ms │     no change │
│ QQuery 19 │           26.74 / 36.81 ±11.64 / 57.59 ms │            26.74 / 29.66 ±4.78 / 39.15 ms │ +1.24x faster │
│ QQuery 20 │      1057.52 / 1064.31 ±5.00 / 1069.42 ms │      1056.88 / 1065.06 ±4.83 / 1071.64 ms │     no change │
│ QQuery 21 │     1367.23 / 1385.00 ±16.60 / 1414.73 ms │      1376.71 / 1386.79 ±7.51 / 1396.01 ms │     no change │
│ QQuery 22 │     3215.96 / 3277.02 ±66.46 / 3402.58 ms │     3227.31 / 3288.37 ±49.47 / 3369.79 ms │     no change │
│ QQuery 23 │    9211.07 / 9495.77 ±168.94 / 9701.05 ms │    9340.84 / 9612.54 ±203.93 / 9947.47 ms │     no change │
│ QQuery 24 │         97.02 / 126.25 ±44.75 / 213.28 ms │            96.75 / 97.61 ±0.89 / 99.05 ms │ +1.29x faster │
│ QQuery 25 │        352.15 / 363.09 ±19.54 / 402.14 ms │        353.53 / 362.09 ±13.05 / 388.05 ms │     no change │
│ QQuery 26 │          97.97 / 100.24 ±2.58 / 105.06 ms │          98.37 / 100.09 ±1.57 / 102.94 ms │     no change │
│ QQuery 27 │     1147.38 / 1156.95 ±11.75 / 1179.24 ms │      1144.12 / 1158.16 ±9.48 / 1168.49 ms │     no change │
│ QQuery 28 │     2811.13 / 2870.43 ±38.74 / 2921.69 ms │     2801.86 / 2858.56 ±31.25 / 2888.73 ms │     no change │
│ QQuery 29 │           38.75 / 49.03 ±16.45 / 81.31 ms │           38.59 / 61.67 ±24.36 / 92.36 ms │  1.26x slower │
│ QQuery 30 │         550.51 / 559.69 ±7.60 / 570.99 ms │         554.25 / 564.16 ±8.83 / 580.40 ms │     no change │
│ QQuery 31 │        533.62 / 552.12 ±14.45 / 574.11 ms │         542.53 / 551.20 ±8.12 / 566.48 ms │     no change │
│ QQuery 32 │     1097.07 / 1222.07 ±69.94 / 1303.75 ms │    1039.52 / 1136.47 ±107.66 / 1331.18 ms │ +1.08x faster │
│ QQuery 33 │ 14081.37 / 14830.46 ±776.41 / 16249.37 ms │ 13949.06 / 14454.54 ±730.41 / 15904.28 ms │     no change │
│ QQuery 34 │ 14148.61 / 14736.04 ±545.99 / 15418.05 ms │ 14374.34 / 15055.22 ±641.21 / 16224.50 ms │     no change │
│ QQuery 35 │        298.50 / 343.92 ±47.38 / 430.21 ms │        314.20 / 367.68 ±97.28 / 562.09 ms │  1.07x slower │
│ QQuery 36 │         181.73 / 194.30 ±6.81 / 201.11 ms │        173.90 / 195.95 ±22.15 / 235.32 ms │     no change │
│ QQuery 37 │          96.87 / 101.77 ±7.48 / 116.68 ms │          97.84 / 101.44 ±4.01 / 108.51 ms │     no change │
│ QQuery 38 │            78.48 / 84.26 ±4.68 / 91.83 ms │            77.28 / 81.82 ±2.40 / 84.32 ms │     no change │
│ QQuery 39 │         416.23 / 424.88 ±6.11 / 435.34 ms │         415.52 / 421.97 ±5.65 / 431.69 ms │     no change │
│ QQuery 40 │            14.38 / 15.07 ±0.49 / 15.89 ms │            14.21 / 14.69 ±0.55 / 15.74 ms │     no change │
│ QQuery 41 │            13.62 / 15.55 ±2.88 / 21.29 ms │            13.75 / 15.26 ±2.34 / 19.93 ms │     no change │
│ QQuery 42 │            13.21 / 13.37 ±0.16 / 13.67 ms │            13.30 / 13.56 ±0.41 / 14.38 ms │     no change │
└───────────┴───────────────────────────────────────────┴───────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 77094.69ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 77212.08ms │
│ Average Time (HEAD)                                   │  1792.90ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  1795.63ms │
│ Queries Faster                                        │          3 │
│ Queries Slower                                        │          2 │
│ Queries with No Change                                │         38 │
│ Queries with Failure                                  │          0 │
└───────────────────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 926901f9001977e85df617b96328801b42f6af4a | Changed: 841faafb6971aa0d180e93098a62974a71095aea

clickbench_partitioned

Query	Base	Changed	Change
Query 0	0 B	0 B	0.0%
Query 1	104 B	104 B	+0.0%
Query 2	936 B	936 B	+0.0%
Query 3	312 B	312 B	+0.0%
Query 4	833.3 MiB	831.0 MiB	-0.3%
Query 5	1.4 GiB	1.4 GiB	-0.1%
Query 6	0 B	0 B	0.0%
Query 7	50.0 MiB	60.0 MiB	+20.0%
Query 8	973.2 MiB	967.4 MiB	-0.6%
Query 9	593.9 MiB	594.0 MiB	+0.0%
Query 10	73.8 MiB	71.9 MiB	-2.7%
Query 11	76.2 MiB	78.7 MiB	+3.2%
Query 12	1.6 GiB	1.6 GiB	+0.5%
Query 13	1.8 GiB	1.9 GiB	+3.6%
Query 14	1.9 GiB	1.9 GiB	+0.1%
Query 15	1.2 GiB	1.2 GiB	-4.6%
Query 16	3.4 GiB	3.6 GiB	+7.0%
Query 17	3.5 GiB	3.4 GiB	-2.4%
Query 18	7.4 GiB	7.4 GiB	-0.2%
Query 19	0 B	0 B	0.0%
Query 20	104 B	104 B	+0.0%
Query 21	1.3 MiB	1.4 MiB	+6.3%
Query 22	2.4 MiB	2.5 MiB	+6.6%
Query 23	12.3 MiB	11.7 MiB	-4.8%
Query 24	6.9 MiB	6.8 MiB	-2.3%
Query 25	8.1 MiB	8.1 MiB	+0.0%
Query 26	7.7 MiB	7.8 MiB	+1.5%
Query 27	2.5 MiB	2.5 MiB	+0.0%
Query 28	1.9 GiB	1.9 GiB	-0.2%
Query 29	624 B	624 B	+0.0%
Query 30	682.2 MiB	633.2 MiB	-7.2%
Query 31	1.5 GiB	1.6 GiB	+7.9%
Query 32	7.2 GiB	7.1 GiB	-0.2%
Query 33	12.7 GiB	12.7 GiB	+0.1%
Query 34	12.7 GiB	12.7 GiB	+0.0%
Query 35	657.4 MiB	674.6 MiB	+2.6%
Query 36	474.1 MiB	474.1 MiB	+0.0%
Query 37	8.1 MiB	8.4 MiB	+3.1%
Query 38	5.3 MiB	5.6 MiB	+7.2%
Query 39	450.8 MiB	450.8 MiB	+0.0%
Query 40	2.2 MiB	2.4 MiB	+10.0%
Query 41	3.3 MiB	3.3 MiB	+0.0%
Query 42	1.8 MiB	1.7 MiB	-5.2%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark	Side	Max pool peak	Peak RSS	Gap	RSS / pool
clickbench_partitioned	base (`926901f9001977e85df617b96328801b42f6af4a`)	12.7 GiB	13.8 GiB	1.1 GiB	1.1×
clickbench_partitioned	changed (`841faafb6971aa0d180e93098a62974a71095aea`)	12.7 GiB	13.1 GiB	395.9 MiB	1.0×

Resource Usage

clickbench_partitioned — base (merge-base)

Metric	Value
Wall time	390.1s
Peak memory	13.8 GiB
Avg memory	6.0 GiB
CPU user	3345.4s
CPU sys	212.0s
Peak spill	0 B

clickbench_partitioned — branch

Metric	Value
Wall time	390.1s
Peak memory	13.1 GiB
Avg memory	6.1 GiB
CPU user	3359.7s
CPU sys	211.7s
Peak spill	0 B

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:22:36Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing a7aa6d1 (a7aa6d1) to 926901f diff

Run configuration

run benchmark tpch
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "a7aa6d133e1b9ab56c2a81dec061ed6e847178ee"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃    Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 1  │  40.98 ms │                               40.71 ms │ no change │
│ QQuery 2  │  18.96 ms │                               19.47 ms │ no change │
│ QQuery 3  │  28.92 ms │                               28.71 ms │ no change │
│ QQuery 4  │  17.52 ms │                               17.52 ms │ no change │
│ QQuery 5  │  36.00 ms │                               36.66 ms │ no change │
│ QQuery 6  │  13.93 ms │                               14.05 ms │ no change │
│ QQuery 7  │  43.05 ms │                               44.04 ms │ no change │
│ QQuery 8  │  41.97 ms │                               41.85 ms │ no change │
│ QQuery 9  │  52.38 ms │                               51.87 ms │ no change │
│ QQuery 10 │  60.57 ms │                               61.44 ms │ no change │
│ QQuery 11 │  14.23 ms │                               14.29 ms │ no change │
│ QQuery 12 │  21.29 ms │                               20.97 ms │ no change │
│ QQuery 13 │  49.10 ms │                               49.61 ms │ no change │
│ QQuery 14 │  24.31 ms │                               24.56 ms │ no change │
│ QQuery 15 │  31.39 ms │                               31.99 ms │ no change │
│ QQuery 16 │  13.73 ms │                               13.66 ms │ no change │
│ QQuery 17 │  78.91 ms │                               79.85 ms │ no change │
│ QQuery 18 │ 117.89 ms │                              119.39 ms │ no change │
│ QQuery 19 │  30.62 ms │                               30.49 ms │ no change │
│ QQuery 20 │  34.48 ms │                               35.32 ms │ no change │
│ QQuery 21 │  55.53 ms │                               56.57 ms │ no change │
│ QQuery 22 │  16.03 ms │                               16.22 ms │ no change │
└───────────┴───────────┴────────────────────────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 841.79ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 849.23ms │
│ Average Time (HEAD)                                   │  38.26ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  38.60ms │
│ Queries Faster                                        │        0 │
│ Queries Slower                                        │        0 │
│ Queries with No Change                                │       22 │
│ Queries with Failure                                  │        0 │
└───────────────────────────────────────────────────────┴──────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query     ┃                              HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃    Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 1  │    40.98 / 41.90 ±0.99 / 43.77 ms │         40.71 / 41.87 ±1.01 / 43.75 ms │ no change │
│ QQuery 2  │    18.96 / 19.24 ±0.28 / 19.72 ms │         19.47 / 20.17 ±0.57 / 21.18 ms │ no change │
│ QQuery 3  │    28.92 / 28.98 ±0.06 / 29.07 ms │         28.71 / 29.44 ±0.88 / 31.17 ms │ no change │
│ QQuery 4  │    17.52 / 17.66 ±0.14 / 17.92 ms │         17.52 / 17.75 ±0.22 / 18.16 ms │ no change │
│ QQuery 5  │    36.00 / 36.83 ±0.60 / 37.60 ms │         36.66 / 36.95 ±0.25 / 37.38 ms │ no change │
│ QQuery 6  │    13.93 / 14.38 ±0.82 / 16.03 ms │         14.05 / 14.54 ±0.53 / 15.19 ms │ no change │
│ QQuery 7  │    43.05 / 44.63 ±1.28 / 46.64 ms │         44.04 / 46.46 ±2.66 / 51.60 ms │ no change │
│ QQuery 8  │    41.97 / 42.27 ±0.22 / 42.53 ms │         41.85 / 42.08 ±0.19 / 42.26 ms │ no change │
│ QQuery 9  │    52.38 / 53.46 ±0.89 / 54.65 ms │         51.87 / 52.27 ±0.28 / 52.62 ms │ no change │
│ QQuery 10 │    60.57 / 61.26 ±0.64 / 62.39 ms │         61.44 / 62.34 ±1.03 / 64.30 ms │ no change │
│ QQuery 11 │    14.23 / 14.51 ±0.31 / 15.10 ms │         14.29 / 14.37 ±0.06 / 14.46 ms │ no change │
│ QQuery 12 │    21.29 / 21.80 ±0.33 / 22.26 ms │         20.97 / 22.29 ±1.50 / 25.11 ms │ no change │
│ QQuery 13 │    49.10 / 49.66 ±0.72 / 51.06 ms │         49.61 / 50.19 ±0.45 / 50.95 ms │ no change │
│ QQuery 14 │    24.31 / 24.76 ±0.61 / 25.96 ms │         24.56 / 24.96 ±0.34 / 25.47 ms │ no change │
│ QQuery 15 │    31.39 / 32.20 ±0.70 / 33.21 ms │         31.99 / 32.36 ±0.30 / 32.91 ms │ no change │
│ QQuery 16 │    13.73 / 13.91 ±0.11 / 14.05 ms │         13.66 / 13.83 ±0.11 / 13.97 ms │ no change │
│ QQuery 17 │    78.91 / 80.51 ±1.56 / 83.16 ms │         79.85 / 80.18 ±0.35 / 80.73 ms │ no change │
│ QQuery 18 │ 117.89 / 120.65 ±2.74 / 125.66 ms │      119.39 / 124.95 ±5.72 / 135.40 ms │ no change │
│ QQuery 19 │    30.62 / 31.06 ±0.59 / 32.17 ms │         30.49 / 30.63 ±0.09 / 30.72 ms │ no change │
│ QQuery 20 │    34.48 / 34.99 ±0.43 / 35.71 ms │         35.32 / 35.53 ±0.16 / 35.74 ms │ no change │
│ QQuery 21 │    55.53 / 59.52 ±3.85 / 66.14 ms │         56.57 / 59.45 ±3.73 / 66.66 ms │ no change │
│ QQuery 22 │    16.03 / 16.60 ±0.51 / 17.25 ms │         16.22 / 16.54 ±0.34 / 17.17 ms │ no change │
└───────────┴───────────────────────────────────┴────────────────────────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 860.76ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 869.15ms │
│ Average Time (HEAD)                                   │  39.13ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  39.51ms │
│ Queries Faster                                        │        0 │
│ Queries Slower                                        │        0 │
│ Queries with No Change                                │       22 │
│ Queries with Failure                                  │        0 │
└───────────────────────────────────────────────────────┴──────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 926901f9001977e85df617b96328801b42f6af4a | Changed: a7aa6d133e1b9ab56c2a81dec061ed6e847178ee

tpch — tpch_sf1

Query	Base	Changed	Change
Query 1	30.1 MiB	30.2 MiB	+0.2%
Query 2	23.1 MiB	23.1 MiB	+0.0%
Query 3	11.5 MiB	11.5 MiB	+0.0%
Query 4	22.3 MiB	32.2 MiB	+44.5%
Query 5	32.9 MiB	32.9 MiB	+0.0%
Query 6	832 B	832 B	+0.0%
Query 7	107.6 MiB	107.6 MiB	+0.0%
Query 8	35.4 MiB	34.0 MiB	-4.1%
Query 9	155.6 MiB	155.6 MiB	+0.0%
Query 10	61.2 MiB	64.4 MiB	+5.1%
Query 11	41.2 MiB	41.7 MiB	+1.3%
Query 12	20.1 MiB	20.1 MiB	+0.0%
Query 13	62.3 MiB	52.8 MiB	-15.2%
Query 14	7.7 MiB	7.7 MiB	+0.0%
Query 15	12.7 MiB	12.9 MiB	+1.7%
Query 16	33.3 MiB	43.1 MiB	+29.3%
Query 17	150.3 MiB	150.3 MiB	+0.0%
Query 18	2.7 GiB	2.7 GiB	-0.1%
Query 19	228.6 KiB	228.6 KiB	+0.0%
Query 20	60.6 MiB	60.3 MiB	-0.5%
Query 21	37.7 MiB	37.7 MiB	+0.0%
Query 22	32.0 MiB	31.8 MiB	-0.6%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark	Side	Max pool peak	Peak RSS	Gap	RSS / pool
tpch	base (`926901f9001977e85df617b96328801b42f6af4a`)	2.7 GiB	2.0 GiB	−687.3 MiB	0.8×
tpch	changed (`a7aa6d133e1b9ab56c2a81dec061ed6e847178ee`)	2.7 GiB	2.0 GiB	−695.9 MiB	0.7×

Resource Usage

tpch — base (merge-base)

Metric	Value
Wall time	5.0s
Peak memory	2.0 GiB
Avg memory	817.8 MiB
CPU user	21.9s
CPU sys	1.7s
Peak spill	0 B

tpch — branch

Metric	Value
Wall time	5.0s
Peak memory	2.0 GiB
Avg memory	810.8 MiB
CPU user	21.8s
CPU sys	1.8s
Peak spill	0 B

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:23:20Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing a7aa6d1 (a7aa6d1) to 926901f diff

Run configuration

run benchmark tpcds
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "a7aa6d133e1b9ab56c2a81dec061ed6e847178ee"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │    5.38 ms │                                5.40 ms │     no change │
│ QQuery 2  │   76.44 ms │                               76.56 ms │     no change │
│ QQuery 3  │   28.82 ms │                               28.61 ms │     no change │
│ QQuery 4  │  834.33 ms │                              830.07 ms │     no change │
│ QQuery 5  │   49.82 ms │                               49.86 ms │     no change │
│ QQuery 6  │   34.17 ms │                               34.47 ms │     no change │
│ QQuery 7  │   73.27 ms │                               73.22 ms │     no change │
│ QQuery 8  │   38.31 ms │                               38.20 ms │     no change │
│ QQuery 9  │   52.41 ms │                               51.04 ms │     no change │
│ QQuery 10 │   62.05 ms │                               61.98 ms │     no change │
│ QQuery 11 │  470.53 ms │                              471.84 ms │     no change │
│ QQuery 12 │   26.16 ms │                               26.01 ms │     no change │
│ QQuery 13 │  112.78 ms │                              112.47 ms │     no change │
│ QQuery 14 │  382.95 ms │                              381.57 ms │     no change │
│ QQuery 15 │   52.47 ms │                               48.77 ms │ +1.08x faster │
│ QQuery 16 │    6.11 ms │                                6.09 ms │     no change │
│ QQuery 17 │   79.45 ms │                               79.45 ms │     no change │
│ QQuery 18 │  105.22 ms │                              106.94 ms │     no change │
│ QQuery 19 │   41.20 ms │                               41.35 ms │     no change │
│ QQuery 20 │   30.97 ms │                               31.01 ms │     no change │
│ QQuery 21 │   16.88 ms │                               17.09 ms │     no change │
│ QQuery 22 │   72.10 ms │                               71.88 ms │     no change │
│ QQuery 23 │  305.92 ms │                              306.31 ms │     no change │
│ QQuery 24 │  192.39 ms │                              192.70 ms │     no change │
│ QQuery 25 │  107.30 ms │                              107.04 ms │     no change │
│ QQuery 26 │   47.06 ms │                               47.10 ms │     no change │
│ QQuery 27 │    6.36 ms │                                6.01 ms │ +1.06x faster │
│ QQuery 28 │   61.05 ms │                               57.36 ms │ +1.06x faster │
│ QQuery 29 │   95.46 ms │                               94.85 ms │     no change │
│ QQuery 30 │   37.05 ms │                               36.67 ms │     no change │
│ QQuery 31 │  111.79 ms │                              111.07 ms │     no change │
│ QQuery 32 │   20.18 ms │                               20.15 ms │     no change │
│ QQuery 33 │   38.35 ms │                               37.31 ms │     no change │
│ QQuery 34 │    9.84 ms │                                9.65 ms │     no change │
│ QQuery 35 │   71.32 ms │                               71.57 ms │     no change │
│ QQuery 36 │    5.85 ms │                                5.80 ms │     no change │
│ QQuery 37 │    7.28 ms │                                7.34 ms │     no change │
│ QQuery 38 │   65.09 ms │                               65.02 ms │     no change │
│ QQuery 39 │   73.28 ms │                               73.38 ms │     no change │
│ QQuery 40 │   23.11 ms │                               23.14 ms │     no change │
│ QQuery 41 │   13.78 ms │                               10.98 ms │ +1.25x faster │
│ QQuery 42 │   23.49 ms │                               23.42 ms │     no change │
│ QQuery 43 │    4.77 ms │                                4.76 ms │     no change │
│ QQuery 44 │    8.50 ms │                                8.58 ms │     no change │
│ QQuery 45 │   35.29 ms │                               35.44 ms │     no change │
│ QQuery 46 │   11.35 ms │                               11.25 ms │     no change │
│ QQuery 47 │  203.56 ms │                              204.23 ms │     no change │
│ QQuery 48 │   90.57 ms │                               91.07 ms │     no change │
│ QQuery 49 │   69.26 ms │                               69.32 ms │     no change │
│ QQuery 50 │   58.59 ms │                               57.85 ms │     no change │
│ QQuery 51 │   93.34 ms │                               93.39 ms │     no change │
│ QQuery 52 │   24.10 ms │                               23.97 ms │     no change │
│ QQuery 53 │   29.31 ms │                               28.86 ms │     no change │
│ QQuery 54 │   52.93 ms │                               53.19 ms │     no change │
│ QQuery 55 │   23.36 ms │                               23.68 ms │     no change │
│ QQuery 56 │   39.10 ms │                               38.39 ms │     no change │
│ QQuery 57 │  151.79 ms │                              152.21 ms │     no change │
│ QQuery 58 │  103.30 ms │                              102.50 ms │     no change │
│ QQuery 59 │  106.69 ms │                              107.41 ms │     no change │
│ QQuery 60 │   43.13 ms │                               42.93 ms │     no change │
│ QQuery 61 │    9.50 ms │                                9.74 ms │     no change │
│ QQuery 62 │   37.64 ms │                               38.46 ms │     no change │
│ QQuery 63 │   29.32 ms │                               29.02 ms │     no change │
│ QQuery 64 │  365.10 ms │                              365.12 ms │     no change │
│ QQuery 65 │  132.61 ms │                              134.31 ms │     no change │
│ QQuery 66 │   70.53 ms │                               70.95 ms │     no change │
│ QQuery 67 │  228.36 ms │                              231.04 ms │     no change │
│ QQuery 68 │   11.42 ms │                               11.46 ms │     no change │
│ QQuery 69 │   56.93 ms │                               56.82 ms │     no change │
│ QQuery 70 │   98.90 ms │                               98.60 ms │     no change │
│ QQuery 71 │   34.55 ms │                               34.51 ms │     no change │
│ QQuery 72 │ 1638.63 ms │                             1563.81 ms │     no change │
│ QQuery 73 │    9.57 ms │                                9.43 ms │     no change │
│ QQuery 74 │  248.94 ms │                              244.83 ms │     no change │
│ QQuery 75 │  138.07 ms │                              137.54 ms │     no change │
│ QQuery 76 │   34.18 ms │                               34.03 ms │     no change │
│ QQuery 77 │   59.45 ms │                               59.35 ms │     no change │
│ QQuery 78 │  160.96 ms │                              161.50 ms │     no change │
│ QQuery 79 │   66.43 ms │                               66.80 ms │     no change │
│ QQuery 80 │   94.39 ms │                               93.33 ms │     no change │
│ QQuery 81 │   28.54 ms │                               28.62 ms │     no change │
│ QQuery 82 │   17.12 ms │                               17.30 ms │     no change │
│ QQuery 83 │   32.40 ms │                               32.18 ms │     no change │
│ QQuery 84 │   31.14 ms │                               31.24 ms │     no change │
│ QQuery 85 │   99.82 ms │                              100.16 ms │     no change │
│ QQuery 86 │   25.08 ms │                               25.04 ms │     no change │
│ QQuery 87 │   65.05 ms │                               65.47 ms │     no change │
│ QQuery 88 │   58.75 ms │                               59.17 ms │     no change │
│ QQuery 89 │   35.21 ms │                               35.44 ms │     no change │
│ QQuery 90 │   16.16 ms │                               15.89 ms │     no change │
│ QQuery 91 │   40.54 ms │                               41.27 ms │     no change │
│ QQuery 92 │   28.76 ms │                               29.19 ms │     no change │
│ QQuery 93 │   48.50 ms │                               48.60 ms │     no change │
│ QQuery 94 │   38.21 ms │                               37.81 ms │     no change │
│ QQuery 95 │   79.48 ms │                               80.19 ms │     no change │
│ QQuery 96 │   23.66 ms │                               23.83 ms │     no change │
│ QQuery 97 │   50.06 ms │                               50.19 ms │     no change │
│ QQuery 98 │   38.21 ms │                               37.64 ms │     no change │
│ QQuery 99 │   56.32 ms │                               56.68 ms │     no change │
└───────────┴────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 9455.19ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 9366.33ms │
│ Average Time (HEAD)                                   │   95.51ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │   94.61ms │
│ Queries Faster                                        │         4 │
│ Queries Slower                                        │         0 │
│ Queries with No Change                                │        95 │
│ Queries with Failure                                  │         0 │
└───────────────────────────────────────────────────────┴───────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │           5.38 / 6.05 ±1.12 / 8.29 ms │            5.40 / 6.17 ±1.06 / 8.23 ms │     no change │
│ QQuery 2  │        76.44 / 76.89 ±0.39 / 77.42 ms │         76.56 / 76.86 ±0.24 / 77.24 ms │     no change │
│ QQuery 3  │        28.82 / 28.98 ±0.17 / 29.25 ms │         28.61 / 28.73 ±0.10 / 28.91 ms │     no change │
│ QQuery 4  │     834.33 / 839.75 ±2.96 / 842.78 ms │      830.07 / 833.85 ±2.89 / 837.97 ms │     no change │
│ QQuery 5  │        49.82 / 51.44 ±2.74 / 56.90 ms │         49.86 / 50.57 ±0.46 / 51.28 ms │     no change │
│ QQuery 6  │        34.17 / 34.70 ±0.47 / 35.46 ms │         34.47 / 36.23 ±2.64 / 41.48 ms │     no change │
│ QQuery 7  │        73.27 / 74.22 ±0.78 / 75.12 ms │         73.22 / 73.72 ±0.40 / 74.21 ms │     no change │
│ QQuery 8  │        38.31 / 38.69 ±0.28 / 39.12 ms │         38.20 / 38.49 ±0.25 / 38.81 ms │     no change │
│ QQuery 9  │        52.41 / 55.02 ±3.36 / 61.58 ms │         51.04 / 52.60 ±1.01 / 53.97 ms │     no change │
│ QQuery 10 │        62.05 / 62.68 ±0.41 / 63.30 ms │         61.98 / 63.98 ±2.52 / 68.95 ms │     no change │
│ QQuery 11 │     470.53 / 481.15 ±5.50 / 485.48 ms │      471.84 / 477.10 ±3.67 / 480.48 ms │     no change │
│ QQuery 12 │        26.16 / 26.41 ±0.14 / 26.55 ms │         26.01 / 26.28 ±0.23 / 26.62 ms │     no change │
│ QQuery 13 │     112.78 / 114.81 ±3.10 / 120.91 ms │      112.47 / 113.43 ±1.10 / 115.48 ms │     no change │
│ QQuery 14 │     382.95 / 388.08 ±3.33 / 392.46 ms │      381.57 / 386.41 ±2.57 / 389.22 ms │     no change │
│ QQuery 15 │        52.47 / 52.77 ±0.28 / 53.13 ms │         48.77 / 51.24 ±3.69 / 58.54 ms │     no change │
│ QQuery 16 │           6.11 / 6.27 ±0.25 / 6.76 ms │            6.09 / 6.34 ±0.26 / 6.78 ms │     no change │
│ QQuery 17 │        79.45 / 80.84 ±1.16 / 82.89 ms │         79.45 / 80.12 ±0.57 / 80.99 ms │     no change │
│ QQuery 18 │     105.22 / 106.30 ±0.83 / 107.50 ms │      106.94 / 109.53 ±3.35 / 115.64 ms │     no change │
│ QQuery 19 │        41.20 / 41.93 ±0.63 / 43.08 ms │         41.35 / 42.09 ±0.51 / 42.87 ms │     no change │
│ QQuery 20 │        30.97 / 31.29 ±0.32 / 31.86 ms │         31.01 / 31.35 ±0.24 / 31.74 ms │     no change │
│ QQuery 21 │        16.88 / 17.65 ±0.96 / 19.54 ms │         17.09 / 17.24 ±0.15 / 17.53 ms │     no change │
│ QQuery 22 │        72.10 / 73.39 ±0.75 / 74.06 ms │         71.88 / 72.69 ±0.71 / 73.76 ms │     no change │
│ QQuery 23 │     305.92 / 308.16 ±2.05 / 311.76 ms │      306.31 / 309.13 ±1.82 / 311.06 ms │     no change │
│ QQuery 24 │     192.39 / 196.93 ±5.31 / 207.29 ms │      192.70 / 195.55 ±1.51 / 197.14 ms │     no change │
│ QQuery 25 │     107.30 / 108.90 ±2.17 / 113.21 ms │      107.04 / 108.70 ±1.51 / 111.00 ms │     no change │
│ QQuery 26 │        47.06 / 47.39 ±0.23 / 47.77 ms │         47.10 / 48.52 ±2.52 / 53.55 ms │     no change │
│ QQuery 27 │           6.36 / 6.47 ±0.10 / 6.66 ms │            6.01 / 6.15 ±0.16 / 6.45 ms │ +1.05x faster │
│ QQuery 28 │        61.05 / 61.48 ±0.50 / 62.44 ms │         57.36 / 60.79 ±1.74 / 62.13 ms │     no change │
│ QQuery 29 │       95.46 / 97.46 ±1.77 / 100.36 ms │        94.85 / 96.90 ±2.57 / 101.65 ms │     no change │
│ QQuery 30 │        37.05 / 37.38 ±0.27 / 37.72 ms │         36.67 / 37.61 ±0.68 / 38.71 ms │     no change │
│ QQuery 31 │     111.79 / 112.06 ±0.16 / 112.22 ms │      111.07 / 112.22 ±1.15 / 114.14 ms │     no change │
│ QQuery 32 │        20.18 / 20.27 ±0.07 / 20.35 ms │         20.15 / 20.41 ±0.15 / 20.59 ms │     no change │
│ QQuery 33 │        38.35 / 39.75 ±1.91 / 43.45 ms │         37.31 / 38.25 ±1.28 / 40.78 ms │     no change │
│ QQuery 34 │         9.84 / 10.09 ±0.37 / 10.82 ms │           9.65 / 9.84 ±0.14 / 10.00 ms │     no change │
│ QQuery 35 │        71.32 / 72.07 ±0.47 / 72.58 ms │         71.57 / 72.12 ±0.35 / 72.60 ms │     no change │
│ QQuery 36 │           5.85 / 6.46 ±0.85 / 8.10 ms │            5.80 / 5.99 ±0.21 / 6.39 ms │ +1.08x faster │
│ QQuery 37 │           7.28 / 7.60 ±0.44 / 8.46 ms │            7.34 / 7.46 ±0.10 / 7.62 ms │     no change │
│ QQuery 38 │        65.09 / 65.54 ±0.44 / 66.29 ms │         65.02 / 65.48 ±0.44 / 66.16 ms │     no change │
│ QQuery 39 │        73.28 / 75.16 ±2.80 / 80.73 ms │         73.38 / 75.12 ±3.07 / 81.24 ms │     no change │
│ QQuery 40 │        23.11 / 23.40 ±0.21 / 23.64 ms │         23.14 / 23.64 ±0.33 / 24.06 ms │     no change │
│ QQuery 41 │        13.78 / 13.97 ±0.15 / 14.23 ms │         10.98 / 11.22 ±0.18 / 11.45 ms │ +1.25x faster │
│ QQuery 42 │        23.49 / 23.80 ±0.26 / 24.23 ms │         23.42 / 23.84 ±0.47 / 24.76 ms │     no change │
│ QQuery 43 │           4.77 / 4.96 ±0.24 / 5.42 ms │            4.76 / 4.91 ±0.22 / 5.36 ms │     no change │
│ QQuery 44 │           8.50 / 8.65 ±0.09 / 8.77 ms │           8.58 / 9.06 ±0.67 / 10.38 ms │     no change │
│ QQuery 45 │        35.29 / 35.82 ±0.31 / 36.22 ms │         35.44 / 35.91 ±0.49 / 36.77 ms │     no change │
│ QQuery 46 │        11.35 / 11.62 ±0.17 / 11.81 ms │         11.25 / 11.53 ±0.16 / 11.70 ms │     no change │
│ QQuery 47 │     203.56 / 205.27 ±1.64 / 207.68 ms │      204.23 / 206.44 ±2.48 / 210.79 ms │     no change │
│ QQuery 48 │        90.57 / 92.55 ±1.90 / 96.10 ms │         91.07 / 92.61 ±1.87 / 96.29 ms │     no change │
│ QQuery 49 │        69.26 / 70.38 ±1.86 / 74.08 ms │         69.32 / 69.76 ±0.52 / 70.76 ms │     no change │
│ QQuery 50 │        58.59 / 60.04 ±2.14 / 64.24 ms │         57.85 / 58.23 ±0.31 / 58.58 ms │     no change │
│ QQuery 51 │        93.34 / 95.19 ±1.59 / 97.85 ms │         93.39 / 95.01 ±2.11 / 99.10 ms │     no change │
│ QQuery 52 │        24.10 / 24.45 ±0.36 / 25.05 ms │         23.97 / 24.43 ±0.71 / 25.84 ms │     no change │
│ QQuery 53 │        29.31 / 29.58 ±0.36 / 30.29 ms │         28.86 / 29.25 ±0.21 / 29.48 ms │     no change │
│ QQuery 54 │        52.93 / 54.87 ±3.19 / 61.23 ms │         53.19 / 54.26 ±1.84 / 57.93 ms │     no change │
│ QQuery 55 │        23.36 / 23.90 ±0.50 / 24.61 ms │         23.68 / 24.16 ±0.63 / 25.32 ms │     no change │
│ QQuery 56 │        39.10 / 39.44 ±0.18 / 39.65 ms │         38.39 / 38.72 ±0.28 / 39.18 ms │     no change │
│ QQuery 57 │     151.79 / 153.39 ±1.48 / 155.66 ms │      152.21 / 153.94 ±2.62 / 159.00 ms │     no change │
│ QQuery 58 │     103.30 / 104.08 ±0.86 / 105.58 ms │      102.50 / 104.12 ±1.16 / 105.91 ms │     no change │
│ QQuery 59 │     106.69 / 108.25 ±1.60 / 111.28 ms │      107.41 / 108.88 ±1.91 / 112.54 ms │     no change │
│ QQuery 60 │        43.13 / 43.69 ±0.66 / 44.56 ms │         42.93 / 43.33 ±0.46 / 44.15 ms │     no change │
│ QQuery 61 │          9.50 / 9.68 ±0.25 / 10.18 ms │           9.74 / 9.88 ±0.15 / 10.10 ms │     no change │
│ QQuery 62 │        37.64 / 39.11 ±1.51 / 41.89 ms │         38.46 / 39.02 ±0.74 / 40.44 ms │     no change │
│ QQuery 63 │        29.32 / 29.47 ±0.15 / 29.74 ms │         29.02 / 29.46 ±0.31 / 29.88 ms │     no change │
│ QQuery 64 │     365.10 / 369.52 ±4.31 / 377.47 ms │      365.12 / 367.83 ±2.51 / 372.20 ms │     no change │
│ QQuery 65 │     132.61 / 135.47 ±3.16 / 141.24 ms │      134.31 / 134.96 ±0.49 / 135.76 ms │     no change │
│ QQuery 66 │        70.53 / 72.56 ±2.00 / 76.32 ms │         70.95 / 71.54 ±0.55 / 72.56 ms │     no change │
│ QQuery 67 │     228.36 / 233.25 ±3.47 / 237.87 ms │      231.04 / 236.52 ±3.66 / 241.45 ms │     no change │
│ QQuery 68 │        11.42 / 11.64 ±0.17 / 11.92 ms │         11.46 / 11.60 ±0.12 / 11.82 ms │     no change │
│ QQuery 69 │        56.93 / 59.54 ±4.61 / 68.74 ms │         56.82 / 58.54 ±2.78 / 64.04 ms │     no change │
│ QQuery 70 │       98.90 / 99.37 ±0.67 / 100.69 ms │       98.60 / 100.91 ±2.19 / 104.99 ms │     no change │
│ QQuery 71 │        34.55 / 35.06 ±0.30 / 35.38 ms │         34.51 / 34.96 ±0.32 / 35.51 ms │     no change │
│ QQuery 72 │ 1638.63 / 1730.23 ±67.75 / 1845.38 ms │  1563.81 / 1620.18 ±48.11 / 1704.10 ms │ +1.07x faster │
│ QQuery 73 │         9.57 / 10.03 ±0.62 / 11.23 ms │            9.43 / 9.58 ±0.20 / 9.99 ms │     no change │
│ QQuery 74 │     248.94 / 251.97 ±2.96 / 256.80 ms │      244.83 / 249.24 ±3.45 / 255.47 ms │     no change │
│ QQuery 75 │     138.07 / 141.52 ±5.03 / 151.50 ms │      137.54 / 138.94 ±1.15 / 140.92 ms │     no change │
│ QQuery 76 │        34.18 / 35.10 ±1.31 / 37.70 ms │         34.03 / 35.82 ±2.36 / 40.43 ms │     no change │
│ QQuery 77 │        59.45 / 59.71 ±0.27 / 60.23 ms │         59.35 / 59.72 ±0.23 / 59.96 ms │     no change │
│ QQuery 78 │     160.96 / 163.46 ±2.64 / 168.37 ms │      161.50 / 163.78 ±1.86 / 166.71 ms │     no change │
│ QQuery 79 │        66.43 / 68.37 ±3.11 / 74.57 ms │         66.80 / 68.00 ±1.53 / 71.02 ms │     no change │
│ QQuery 80 │        94.39 / 95.81 ±1.07 / 97.55 ms │         93.33 / 95.54 ±1.34 / 97.19 ms │     no change │
│ QQuery 81 │        28.54 / 28.83 ±0.22 / 29.09 ms │         28.62 / 30.20 ±2.46 / 35.05 ms │     no change │
│ QQuery 82 │        17.12 / 17.26 ±0.16 / 17.57 ms │         17.30 / 17.59 ±0.22 / 17.80 ms │     no change │
│ QQuery 83 │        32.40 / 32.63 ±0.14 / 32.81 ms │         32.18 / 32.49 ±0.23 / 32.86 ms │     no change │
│ QQuery 84 │        31.14 / 33.33 ±3.38 / 39.99 ms │         31.24 / 31.43 ±0.15 / 31.65 ms │ +1.06x faster │
│ QQuery 85 │      99.82 / 100.88 ±1.01 / 102.52 ms │      100.16 / 101.99 ±1.96 / 105.32 ms │     no change │
│ QQuery 86 │        25.08 / 25.18 ±0.07 / 25.25 ms │         25.04 / 25.32 ±0.18 / 25.58 ms │     no change │
│ QQuery 87 │        65.05 / 65.74 ±0.62 / 66.83 ms │         65.47 / 66.00 ±0.39 / 66.54 ms │     no change │
│ QQuery 88 │        58.75 / 59.21 ±0.43 / 59.74 ms │         59.17 / 60.05 ±0.68 / 61.20 ms │     no change │
│ QQuery 89 │        35.21 / 36.02 ±0.97 / 37.92 ms │         35.44 / 35.87 ±0.30 / 36.20 ms │     no change │
│ QQuery 90 │        16.16 / 16.45 ±0.20 / 16.71 ms │         15.89 / 17.18 ±1.99 / 21.15 ms │     no change │
│ QQuery 91 │        40.54 / 40.71 ±0.13 / 40.91 ms │         41.27 / 41.74 ±0.44 / 42.26 ms │     no change │
│ QQuery 92 │        28.76 / 29.13 ±0.29 / 29.54 ms │         29.19 / 30.05 ±0.70 / 31.14 ms │     no change │
│ QQuery 93 │        48.50 / 49.66 ±0.95 / 51.07 ms │         48.60 / 49.85 ±1.03 / 51.22 ms │     no change │
│ QQuery 94 │        38.21 / 38.47 ±0.25 / 38.87 ms │         37.81 / 38.33 ±0.47 / 39.09 ms │     no change │
│ QQuery 95 │        79.48 / 80.25 ±0.63 / 81.03 ms │         80.19 / 81.49 ±0.95 / 82.74 ms │     no change │
│ QQuery 96 │        23.66 / 23.97 ±0.35 / 24.63 ms │         23.83 / 23.93 ±0.11 / 24.12 ms │     no change │
│ QQuery 97 │        50.06 / 52.28 ±2.88 / 57.91 ms │         50.19 / 51.37 ±0.66 / 52.07 ms │     no change │
│ QQuery 98 │        38.21 / 38.47 ±0.27 / 38.96 ms │         37.64 / 38.20 ±0.47 / 39.04 ms │     no change │
│ QQuery 99 │        56.32 / 56.70 ±0.22 / 57.00 ms │         56.68 / 58.17 ±2.17 / 62.41 ms │     no change │
└───────────┴───────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 9665.83ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 9535.72ms │
│ Average Time (HEAD)                                   │   97.63ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │   96.32ms │
│ Queries Faster                                        │         5 │
│ Queries Slower                                        │         0 │
│ Queries with No Change                                │        94 │
│ Queries with Failure                                  │         0 │
└───────────────────────────────────────────────────────┴───────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 926901f9001977e85df617b96328801b42f6af4a | Changed: a7aa6d133e1b9ab56c2a81dec061ed6e847178ee

tpcds — tpcds_sf1

Query	Base	Changed	Change
Query 1	197.5 KiB	377.6 KiB	+91.2%
Query 2	90.9 MiB	91.9 MiB	+1.1%
Query 3	1.9 MiB	2.1 MiB	+10.5%
Query 4	79.7 MiB	79.7 MiB	-0.0%
Query 5	13.0 MiB	13.0 MiB	+0.0%
Query 6	4.4 MiB	4.4 MiB	+0.2%
Query 7	3.7 MiB	4.0 MiB	+10.1%
Query 8	1.7 MiB	1.8 MiB	+6.5%
Query 9	32.1 KiB	32.1 KiB	+0.0%
Query 10	2.0 MiB	2.0 MiB	+0.0%
Query 11	62.0 MiB	62.0 MiB	-0.1%
Query 12	114.4 MiB	114.4 MiB	+0.0%
Query 13	13.7 MiB	13.7 MiB	+0.0%
Query 14	170.9 MiB	171.8 MiB	+0.5%
Query 15	4.2 MiB	4.2 MiB	-0.1%
Query 16	118.6 KiB	118.6 KiB	+0.0%
Query 17	41.0 MiB	41.0 MiB	+0.0%
Query 18	21.3 MiB	21.2 MiB	-0.1%
Query 19	2.1 MiB	1.9 MiB	-10.6%
Query 20	114.6 MiB	124.6 MiB	+8.7%
Query 21	3.7 MiB	3.3 MiB	-11.3%
Query 22	61.8 MiB	61.8 MiB	+0.0%
Query 23	211.6 MiB	213.1 MiB	+0.7%
Query 24	38.7 MiB	38.7 MiB	+0.0%
Query 25	99.9 MiB	99.9 MiB	+0.0%
Query 26	3.5 MiB	3.6 MiB	+2.4%
Query 27	1.6 MiB	1.6 MiB	+0.0%
Query 28	1.3 MiB	1.3 MiB	+0.0%
Query 29	104.7 MiB	104.7 MiB	+0.0%
Query 30	3.7 MiB	3.7 MiB	-1.2%
Query 31	13.4 MiB	13.4 MiB	+0.0%
Query 32	2.5 MiB	2.7 MiB	+8.9%
Query 33	4.6 MiB	4.2 MiB	-7.6%
Query 34	6.0 MiB	6.0 MiB	+0.0%
Query 35	16.6 MiB	16.6 MiB	+0.0%
Query 36	10.3 MiB	10.3 MiB	+0.0%
Query 37	3.3 MiB	3.3 MiB	+0.0%
Query 38	10.4 MiB	10.4 MiB	+0.0%
Query 39	32.4 MiB	73.7 MiB	+127.4%
Query 40	12.2 MiB	12.2 MiB	+0.0%
Query 41	2.1 MiB	2.1 MiB	+0.0%
Query 42	1.4 MiB	1.1 MiB	-25.0%
Query 43	275.0 KiB	275.0 KiB	+0.0%
Query 44	233.2 KiB	233.2 KiB	+0.0%
Query 45	4.5 MiB	4.5 MiB	+0.0%
Query 46	6.2 MiB	6.2 MiB	+0.0%
Query 47	128.5 MiB	128.5 MiB	-0.0%
Query 48	11.8 MiB	11.8 MiB	+0.0%
Query 49	42.8 MiB	53.2 MiB	+24.3%
Query 50	16.9 MiB	17.3 MiB	+2.2%
Query 51	178.0 MiB	177.7 MiB	-0.2%
Query 52	1.5 MiB	1.5 MiB	+0.7%
Query 53	30.6 MiB	30.6 MiB	+0.0%
Query 54	2.2 MiB	2.1 MiB	-4.2%
Query 55	1.5 MiB	1.5 MiB	+0.0%
Query 56	3.3 MiB	3.0 MiB	-6.7%
Query 57	120.8 MiB	120.8 MiB	-0.0%
Query 58	4.4 MiB	4.4 MiB	+0.0%
Query 59	11.7 MiB	11.7 MiB	+0.0%
Query 60	7.2 MiB	7.2 MiB	+0.0%
Query 61	2.5 KiB	2.4 KiB	-3.2%
Query 62	3.6 MiB	3.6 MiB	+0.0%
Query 63	30.6 MiB	20.6 MiB	-32.7%
Query 64	94.7 MiB	94.7 MiB	+0.0%
Query 65	29.7 MiB	29.7 MiB	+0.0%
Query 66	14.9 MiB	14.9 MiB	+0.0%
Query 67	138.2 MiB	144.3 MiB	+4.4%
Query 68	6.2 MiB	6.2 MiB	+0.0%
Query 69	3.9 MiB	3.9 MiB	-0.0%
Query 70	44.0 MiB	44.0 MiB	+0.0%
Query 71	40.4 MiB	40.6 MiB	+0.5%
Query 72	97.2 MiB	97.2 MiB	+0.0%
Query 73	6.0 MiB	6.0 MiB	+0.0%
Query 74	34.6 MiB	34.6 MiB	-0.0%
Query 75	20.2 MiB	20.7 MiB	+2.6%
Query 76	6.9 MiB	7.9 MiB	+14.7%
Query 77	4.1 MiB	4.2 MiB	+3.3%
Query 78	109.4 MiB	109.8 MiB	+0.4%
Query 79	8.0 MiB	8.0 MiB	+0.0%
Query 80	42.8 MiB	42.0 MiB	-1.9%
Query 81	4.1 MiB	4.1 MiB	+0.3%
Query 82	4.9 MiB	4.9 MiB	+0.0%
Query 83	9.0 MiB	8.9 MiB	-1.2%
Query 84	11.9 MiB	11.9 MiB	+0.0%
Query 85	17.0 MiB	17.0 MiB	+0.0%
Query 86	82.1 MiB	91.9 MiB	+12.0%
Query 87	13.5 MiB	12.9 MiB	-4.7%
Query 88	1.8 MiB	1.5 MiB	-16.3%
Query 89	112.2 MiB	112.2 MiB	+0.0%
Query 90	348.6 KiB	351.7 KiB	+0.9%
Query 91	21.1 MiB	30.9 MiB	+46.7%
Query 92	2.7 MiB	2.7 MiB	+0.0%
Query 93	14.9 MiB	15.4 MiB	+3.4%
Query 94	3.7 MiB	3.4 MiB	-7.6%
Query 95	18.8 MiB	18.3 MiB	-2.7%
Query 96	224.7 KiB	224.7 KiB	+0.0%
Query 97	33.1 MiB	33.1 MiB	+0.2%
Query 98	114.7 MiB	124.4 MiB	+8.4%
Query 99	1.9 MiB	1.6 MiB	-13.2%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark	Side	Max pool peak	Peak RSS	Gap	RSS / pool
tpcds	base (`926901f9001977e85df617b96328801b42f6af4a`)	211.6 MiB	1.6 GiB	1.4 GiB	7.9×
tpcds	changed (`a7aa6d133e1b9ab56c2a81dec061ed6e847178ee`)	213.1 MiB	1.9 GiB	1.7 GiB	9.2×

Resource Usage

tpcds — base (merge-base)

Metric	Value
Wall time	50.0s
Peak memory	1.6 GiB
Avg memory	1.2 GiB
CPU user	196.3s
CPU sys	5.1s
Peak spill	0 B

tpcds — branch

Metric	Value
Wall time	50.0s
Peak memory	1.9 GiB
Avg memory	1.4 GiB
CPU user	189.3s
CPU sys	5.1s
Peak spill	0 B

File an issue against this benchmark runner

adriangbot · 2026-10-30T19:37:08Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing a7aa6d1 (a7aa6d1) to 926901f diff

Run configuration

run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "926901f9001977e85df617b96328801b42f6af4a"
changed:
  ref: "a7aa6d133e1b9ab56c2a81dec061ed6e847178ee"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     1.12 ms │                                1.13 ms │     no change │
│ QQuery 1  │    10.91 ms │                               10.85 ms │     no change │
│ QQuery 2  │    35.46 ms │                               35.09 ms │     no change │
│ QQuery 3  │    31.02 ms │                               31.24 ms │     no change │
│ QQuery 4  │   236.84 ms │                              229.99 ms │     no change │
│ QQuery 5  │  1464.24 ms │                             1517.98 ms │     no change │
│ QQuery 6  │     1.17 ms │                                1.18 ms │     no change │
│ QQuery 7  │    12.21 ms │                               12.12 ms │     no change │
│ QQuery 8  │   338.38 ms │                              338.37 ms │     no change │
│ QQuery 9  │   471.13 ms │                              490.40 ms │     no change │
│ QQuery 10 │   197.79 ms │                              198.62 ms │     no change │
│ QQuery 11 │   208.50 ms │                              210.82 ms │     no change │
│ QQuery 12 │  1495.32 ms │                             1605.42 ms │  1.07x slower │
│ QQuery 13 │  2231.94 ms │                             2201.55 ms │     no change │
│ QQuery 14 │  1618.16 ms │                             1645.57 ms │     no change │
│ QQuery 15 │   286.57 ms │                              287.32 ms │     no change │
│ QQuery 16 │  3799.48 ms │                             3683.78 ms │     no change │
│ QQuery 17 │  3781.62 ms │                             3907.03 ms │     no change │
│ QQuery 18 │  6896.52 ms │                             7112.15 ms │     no change │
│ QQuery 19 │    26.28 ms │                               26.47 ms │     no change │
│ QQuery 20 │  1039.23 ms │                             1038.84 ms │     no change │
│ QQuery 21 │  1357.69 ms │                             1364.02 ms │     no change │
│ QQuery 22 │  3199.36 ms │                             3262.76 ms │     no change │
│ QQuery 23 │  9108.80 ms │                             9353.49 ms │     no change │
│ QQuery 24 │    95.71 ms │                               98.18 ms │     no change │
│ QQuery 25 │   351.20 ms │                              356.63 ms │     no change │
│ QQuery 26 │    97.76 ms │                               99.93 ms │     no change │
│ QQuery 27 │  1141.47 ms │                             1162.09 ms │     no change │
│ QQuery 28 │  2827.31 ms │                             2854.17 ms │     no change │
│ QQuery 29 │    37.74 ms │                               37.76 ms │     no change │
│ QQuery 30 │   562.74 ms │                              542.80 ms │     no change │
│ QQuery 31 │   533.98 ms │                              537.60 ms │     no change │
│ QQuery 32 │  1063.31 ms │                             1057.75 ms │     no change │
│ QQuery 33 │ 14123.79 ms │                            13730.88 ms │     no change │
│ QQuery 34 │ 14034.17 ms │                            13722.85 ms │     no change │
│ QQuery 35 │   310.91 ms │                              293.52 ms │ +1.06x faster │
│ QQuery 36 │   172.53 ms │                              192.31 ms │  1.11x slower │
│ QQuery 37 │    95.33 ms │                              102.35 ms │  1.07x slower │
│ QQuery 38 │    81.06 ms │                               78.24 ms │     no change │
│ QQuery 39 │   397.10 ms │                              433.71 ms │  1.09x slower │
│ QQuery 40 │    14.07 ms │                               14.77 ms │     no change │
│ QQuery 41 │    13.60 ms │                               13.77 ms │     no change │
│ QQuery 42 │    13.27 ms │                               13.82 ms │     no change │
└───────────┴─────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 73816.80ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 73909.34ms │
│ Average Time (HEAD)                                   │  1716.67ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  1718.82ms │
│ Queries Faster                                        │          1 │
│ Queries Slower                                        │          4 │
│ Queries with No Change                                │         38 │
│ Queries with Failure                                  │          0 │
└───────────────────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and rich-T-kid_dictionary-inlist-predicate
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                      HEAD ┃    rich-T-kid_dictionary-inlist-predicate ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │              1.12 / 4.11 ±5.91 / 15.94 ms │              1.13 / 4.07 ±5.82 / 15.72 ms │     no change │
│ QQuery 1  │            10.91 / 11.10 ±0.13 / 11.30 ms │            10.85 / 11.03 ±0.14 / 11.23 ms │     no change │
│ QQuery 2  │            35.46 / 35.76 ±0.20 / 36.04 ms │            35.09 / 35.29 ±0.20 / 35.65 ms │     no change │
│ QQuery 3  │            31.02 / 31.82 ±0.88 / 33.21 ms │            31.24 / 32.22 ±0.84 / 33.41 ms │     no change │
│ QQuery 4  │         236.84 / 241.65 ±5.03 / 251.26 ms │         229.99 / 239.16 ±7.93 / 253.43 ms │     no change │
│ QQuery 5  │     1464.24 / 1485.89 ±25.35 / 1521.62 ms │     1517.98 / 1584.47 ±75.59 / 1725.18 ms │  1.07x slower │
│ QQuery 6  │               1.17 / 1.32 ±0.24 / 1.80 ms │               1.18 / 1.35 ±0.24 / 1.82 ms │     no change │
│ QQuery 7  │            12.21 / 12.36 ±0.17 / 12.70 ms │            12.12 / 12.15 ±0.04 / 12.24 ms │     no change │
│ QQuery 8  │        338.38 / 361.97 ±12.13 / 372.29 ms │        338.37 / 350.64 ±11.94 / 368.62 ms │     no change │
│ QQuery 9  │        471.13 / 509.34 ±24.29 / 538.34 ms │        490.40 / 512.78 ±16.01 / 537.23 ms │     no change │
│ QQuery 10 │         197.79 / 200.20 ±1.42 / 202.02 ms │         198.62 / 201.30 ±2.65 / 205.04 ms │     no change │
│ QQuery 11 │         208.50 / 210.15 ±1.53 / 212.23 ms │         210.82 / 213.89 ±2.93 / 217.56 ms │     no change │
│ QQuery 12 │     1495.32 / 1553.21 ±38.07 / 1586.46 ms │     1605.42 / 1639.78 ±22.36 / 1672.07 ms │  1.06x slower │
│ QQuery 13 │    2231.94 / 2527.57 ±330.06 / 2963.38 ms │    2201.55 / 2602.34 ±486.19 / 3527.73 ms │     no change │
│ QQuery 14 │     1618.16 / 1702.80 ±63.06 / 1799.98 ms │     1645.57 / 1725.51 ±66.24 / 1833.47 ms │     no change │
│ QQuery 15 │        286.57 / 306.80 ±16.03 / 325.26 ms │         287.32 / 301.01 ±9.39 / 313.96 ms │     no change │
│ QQuery 16 │    3799.48 / 4021.05 ±174.06 / 4225.06 ms │     3683.78 / 3745.12 ±41.19 / 3812.93 ms │ +1.07x faster │
│ QQuery 17 │     3781.62 / 3901.21 ±85.53 / 4036.96 ms │     3907.03 / 3986.83 ±82.23 / 4131.46 ms │     no change │
│ QQuery 18 │    6896.52 / 7357.56 ±287.40 / 7783.44 ms │    7112.15 / 7343.42 ±156.60 / 7560.57 ms │     no change │
│ QQuery 19 │            26.28 / 28.74 ±3.70 / 36.10 ms │            26.47 / 26.97 ±0.55 / 28.04 ms │ +1.07x faster │
│ QQuery 20 │     1039.23 / 1055.74 ±11.44 / 1070.45 ms │      1038.84 / 1044.35 ±4.66 / 1049.85 ms │     no change │
│ QQuery 21 │     1357.69 / 1387.16 ±26.65 / 1432.69 ms │     1364.02 / 1378.74 ±16.24 / 1408.74 ms │     no change │
│ QQuery 22 │     3199.36 / 3286.75 ±51.73 / 3342.27 ms │     3262.76 / 3306.72 ±34.01 / 3351.75 ms │     no change │
│ QQuery 23 │    9108.80 / 9469.53 ±181.94 / 9596.54 ms │     9353.49 / 9441.77 ±96.13 / 9614.37 ms │     no change │
│ QQuery 24 │         95.71 / 105.64 ±11.36 / 122.23 ms │        98.18 / 171.48 ±145.12 / 461.73 ms │  1.62x slower │
│ QQuery 25 │         351.20 / 355.91 ±3.69 / 362.44 ms │         356.63 / 363.73 ±5.22 / 369.92 ms │     no change │
│ QQuery 26 │          97.76 / 100.31 ±2.81 / 105.71 ms │          99.93 / 101.56 ±1.55 / 104.34 ms │     no change │
│ QQuery 27 │     1141.47 / 1165.63 ±17.94 / 1192.38 ms │     1162.09 / 1200.65 ±26.00 / 1242.98 ms │     no change │
│ QQuery 28 │     2827.31 / 2878.27 ±38.55 / 2943.99 ms │     2854.17 / 2939.76 ±57.84 / 3021.63 ms │     no change │
│ QQuery 29 │            37.74 / 41.15 ±4.53 / 49.67 ms │            37.76 / 42.63 ±5.93 / 51.34 ms │     no change │
│ QQuery 30 │        562.74 / 576.54 ±24.00 / 624.32 ms │        542.80 / 553.70 ±11.07 / 570.15 ms │     no change │
│ QQuery 31 │        533.98 / 563.02 ±25.78 / 602.64 ms │        537.60 / 551.62 ±13.35 / 576.99 ms │     no change │
│ QQuery 32 │     1063.31 / 1131.30 ±80.52 / 1278.24 ms │     1057.75 / 1148.21 ±78.36 / 1264.32 ms │     no change │
│ QQuery 33 │ 14123.79 / 15016.38 ±827.70 / 16382.45 ms │ 13730.88 / 14808.37 ±587.21 / 15484.80 ms │     no change │
│ QQuery 34 │ 14034.17 / 14540.39 ±439.39 / 15286.35 ms │ 13722.85 / 14303.22 ±400.22 / 14878.91 ms │     no change │
│ QQuery 35 │         310.91 / 320.45 ±8.05 / 331.40 ms │        293.52 / 351.46 ±46.64 / 433.68 ms │  1.10x slower │
│ QQuery 36 │         172.53 / 180.53 ±6.69 / 192.08 ms │         192.31 / 208.09 ±9.09 / 220.21 ms │  1.15x slower │
│ QQuery 37 │          95.33 / 103.10 ±9.38 / 121.33 ms │         102.35 / 107.29 ±4.52 / 114.83 ms │     no change │
│ QQuery 38 │            81.06 / 83.66 ±2.15 / 86.69 ms │            78.24 / 82.43 ±2.96 / 87.10 ms │     no change │
│ QQuery 39 │        397.10 / 410.65 ±15.91 / 436.54 ms │         433.71 / 449.44 ±8.77 / 456.82 ms │  1.09x slower │
│ QQuery 40 │            14.07 / 14.72 ±0.54 / 15.42 ms │            14.77 / 17.64 ±3.44 / 24.35 ms │  1.20x slower │
│ QQuery 41 │            13.60 / 13.78 ±0.17 / 14.09 ms │            13.77 / 19.00 ±9.65 / 38.29 ms │  1.38x slower │
│ QQuery 42 │            13.27 / 18.20 ±8.16 / 34.40 ms │            13.82 / 14.23 ±0.36 / 14.69 ms │ +1.28x faster │
└───────────┴───────────────────────────────────────────┴───────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                                     ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                                     │ 77323.43ms │
│ Total Time (rich-T-kid_dictionary-inlist-predicate)   │ 77175.42ms │
│ Average Time (HEAD)                                   │  1798.22ms │
│ Average Time (rich-T-kid_dictionary-inlist-predicate) │  1794.78ms │
│ Queries Faster                                        │          3 │
│ Queries Slower                                        │          8 │
│ Queries with No Change                                │         32 │
│ Queries with Failure                                  │          0 │
└───────────────────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 926901f9001977e85df617b96328801b42f6af4a | Changed: a7aa6d133e1b9ab56c2a81dec061ed6e847178ee

clickbench_partitioned

Query	Base	Changed	Change
Query 0	0 B	0 B	0.0%
Query 1	104 B	104 B	+0.0%
Query 2	936 B	936 B	+0.0%
Query 3	312 B	312 B	+0.0%
Query 4	826.2 MiB	828.3 MiB	+0.3%
Query 5	1.5 GiB	1.5 GiB	+0.3%
Query 6	0 B	0 B	0.0%
Query 7	50.2 MiB	50.2 MiB	-0.0%
Query 8	944.0 MiB	945.2 MiB	+0.1%
Query 9	594.0 MiB	594.0 MiB	+0.0%
Query 10	72.6 MiB	72.8 MiB	+0.3%
Query 11	78.5 MiB	76.5 MiB	-2.5%
Query 12	1.6 GiB	1.6 GiB	+1.1%
Query 13	1.9 GiB	1.8 GiB	-0.4%
Query 14	1.9 GiB	1.9 GiB	+0.0%
Query 15	1.2 GiB	1.2 GiB	-3.1%
Query 16	3.3 GiB	3.4 GiB	+3.3%
Query 17	3.3 GiB	3.3 GiB	-0.8%
Query 18	7.2 GiB	7.4 GiB	+2.6%
Query 19	0 B	0 B	0.0%
Query 20	104 B	104 B	+0.0%
Query 21	1.3 MiB	1.4 MiB	+4.7%
Query 22	2.2 MiB	2.4 MiB	+6.8%
Query 23	13.1 MiB	12.3 MiB	-6.6%
Query 24	6.9 MiB	6.8 MiB	-1.5%
Query 25	8.1 MiB	8.1 MiB	+0.0%
Query 26	7.9 MiB	8.0 MiB	+0.5%
Query 27	2.5 MiB	2.5 MiB	+0.0%
Query 28	1.9 GiB	2.0 GiB	+2.1%
Query 29	624 B	624 B	+0.0%
Query 30	680.3 MiB	630.8 MiB	-7.3%
Query 31	1.5 GiB	1.6 GiB	+12.8%
Query 32	7.1 GiB	7.1 GiB	+0.2%
Query 33	12.7 GiB	12.7 GiB	-0.1%
Query 34	12.7 GiB	12.7 GiB	-0.0%
Query 35	674.4 MiB	668.8 MiB	-0.8%
Query 36	474.1 MiB	474.1 MiB	+0.0%
Query 37	8.1 MiB	8.4 MiB	+3.1%
Query 38	5.3 MiB	4.7 MiB	-11.5%
Query 39	450.8 MiB	460.4 MiB	+2.1%
Query 40	2.3 MiB	2.3 MiB	-1.8%
Query 41	3.3 MiB	3.3 MiB	+0.0%
Query 42	1.8 MiB	1.8 MiB	-1.8%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark	Side	Max pool peak	Peak RSS	Gap	RSS / pool
clickbench_partitioned	base (`926901f9001977e85df617b96328801b42f6af4a`)	12.7 GiB	13.1 GiB	416.7 MiB	1.0×
clickbench_partitioned	changed (`a7aa6d133e1b9ab56c2a81dec061ed6e847178ee`)	12.7 GiB	13.3 GiB	603.6 MiB	1.0×

Resource Usage

clickbench_partitioned — base (merge-base)

Metric	Value
Wall time	390.1s
Peak memory	13.1 GiB
Avg memory	5.9 GiB
CPU user	3368.1s
CPU sys	204.1s
Peak spill	0 B

clickbench_partitioned — branch

Metric	Value
Wall time	390.1s
Peak memory	13.3 GiB
Avg memory	5.9 GiB
CPU user	3369.6s
CPU sys	209.0s
Peak spill	0 B

File an issue against this benchmark runner

Preserve short IN lists for dictionary strings

841faaf

github-actions Bot added the optimizer Optimizer rules label Oct 30, 2026

Rich-T-kid closed this Oct 30, 2026

Rich-T-kid deleted the rich-T-kid/dictionary-inlist-predicate branch October 30, 2026 19:05

Rich-T-kid restored the rich-T-kid/dictionary-inlist-predicate branch October 30, 2026 19:06

Rich-T-kid reopened this Oct 30, 2026

Test dictionary string IN list preservation

a7aa6d1

Preserve short IN lists for dictionary binary values

29f9fee

Rich-T-kid changed the title ~~Preserve short IN lists for dictionary strings~~ Preserve short IN lists for dictionary strings and binary values Oct 30, 2026

Conversation

Rich-T-kid commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Uh oh!

Rich-T-kid commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

Rich-T-kid commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

adriangbot commented Oct 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Rich-T-kid commented Oct 30, 2026 •

edited

Loading