Repository navigation
[AQUMV] Extend AQUMV to support materialized views on partitioned tables. - #965
Merged
Merged
Conversation
…les.
This commit introduces support for partitioned tables in the AQUMV
(Answer Query Using Materialized Views) feature. Users can now create
materialized views based on partitioned tables, which may include
multiple levels of child partitions. When a query is executed on the
root partition table, AQUMV will automatically rewrite the query to
utilize the materialized view, thereby improving query performance by
avoiding direct access to the partitioned tables.
Example:
We have a partition table par with children tables of multiple level.
select * from pg_partition_tree('par');
relid | parentrelid | isleaf | level
---------------------+-------------+--------+-------
par | | f | 0
par_1_prt_1 | par | f | 1
par_1_prt_2 | par | f | 1
par_1_prt_1_2_prt_1 | par_1_prt_1 | t | 2
par_1_prt_1_2_prt_2 | par_1_prt_1 | t | 2
par_1_prt_2_2_prt_1 | par_1_prt_2 | t | 2
par_1_prt_2_2_prt_2 | par_1_prt_2 | t | 2
create materialized view mv_par as select count(*) as column_1 from par;
When execute:
select count(*) from par;
AQUMV will rewrite the SQL to:
select column_1 from mv_par;
Answer the results from materialized view mv_par instead of querying on
par and its partitions.
explain(costs off, verbose) select count(*) from par;
QUERY PLAN
---------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3)
Output: column_1
-> Seq Scan on public.mv_par
Output: column_1
Settings: enable_answer_query_using_materialized_views = 'on'
Optimizer: Postgres query optimizer
(6 rows)
This enhancement significantly boosts query performance in OLAP
environments where partitioned tables are extensively used, allowing
users to fully leverage the benefits of materialized views.
Authored-by: Zhang Mingli avamingli@gmail.com
gfphoenix78
reviewed
Mar 10, 2025
gfphoenix78
reviewed
Mar 10, 2025
yjhjstz
reviewed
Mar 11, 2025
yjhjstz
reviewed
Mar 11, 2025
yjhjstz
approved these changes
Mar 11, 2025
my-ship-it
approved these changes
Mar 12, 2025
lss602726449
added a commit
to lss602726449/cloudberrydb
that referenced
this pull request
Oct 8, 2026
…gineering/dev_collaboration#965
## Summary
Fixes field-engineering/dev_collaboration#965: a `count(1)` over a 281 column
append-only **row** table, qualified on three date columns, was 6x slower than the same
data in a heap table and 14x slower than AOCS.
### Root cause
`AppendOnlyExecutorReadBlock_ProcessTuple()` called `memtuple_deform()` and then
`ExecStoreVirtualTuple()`, and `appendonly_slot_callbacks()` returned `TTSOpsVirtual`.
A virtual slot promises its values are already in `tts_values[]`, so
`ExecStoreVirtualTuple()` sets `tts_nvalid = natts` and `slot_getattr()` never reaches
`getsomeattrs` — the scan has no choice but to expand **every** column of **every**
tuple. Heap tables are unaffected because `TTSOpsBufferHeapTuple` deforms incrementally
up to the highest attribute number the query references.
Scan cost therefore grew with the **width of the table** instead of the columns actually
read. Measured on a release build (`-O3`, no cassert), 500k rows, query touching only
attno 2:
| table columns | AO row before | heap |
|---|---|---|
| 20 | 46.9 ms | 35.7 ms |
| 100 | 114.2 ms | 36.8 ms |
| 281 | **236.0 ms** | 39.1 ms |
This is not how the format is meant to be read. A MemTuple keeps precomputed attribute
offsets in its binding, so any attribute is reachable in constant time, and before the
PostgreSQL 12 merge (`19cd1cf4b6`) the slot carried that binding and fetched attributes
on demand. That merge dropped the MemTuple slot shape — which at the time could only
exist as extra fields in the shared `TupleTableSlot` struct — and converted the tuple
eagerly instead. PostgreSQL 12 also introduced `TupleTableSlotOps` precisely so an access
method can keep tuples in its native format, which is what this MR finally uses.
**The same code is present in GPDB 7 and Apache Cloudberry main**, both verified to show
the identical regression (GPDB 7 release: 193.8 ms vs heap 28.9 ms at 281 columns). This
is an inherited PG12-merge artifact, not something introduced by Lightning.
### Change
Add `TTSOpsMemTuple`, a slot holding the MemTuple together with the binding needed to
interpret it, expanding attributes in `getsomeattrs`. Because offsets are precomputed,
`tts_nvalid` alone is the resume point — no separate deforming cursor (the `off` field
heap and minimal tuple slots carry) is needed. `memtuple_deform_range()` reuses the
existing inlinable internal helper so full-column access keeps the old bulk-deform speed.
Two details are easy to miss when adding a slot type here, both found by running the full
regression suite:
- `gp_segment_id` and `gp_foreign_server` are answered by the slot even though no tuple
format carries them, so `getsysattr` must keep the special cases that
`tts_virtual_getsysattr` documents. Omitting them broke ~59 tests.
- Not every slot reaches the scan through `table_slot_create()`.
`AppendOnlySegmentFileFullCompaction()` built a virtual slot directly and passed it to
`appendonly_getnextslot()`; in a release build the assertion is compiled out and the
larger slot struct would be written past the end of the smaller one. It now asks the
access method for the slot type, and `ProcessTuple` additionally falls back to expanding
the tuple up front for any non-MemTuple slot, so a caller bringing its own slot cannot
be handed one it cannot read.
## Test plan
Release build (`-O3`, `debug_assertions=off`), demo cluster, 500k rows.
The two tables below pin one variable each. Query is
`SELECT count(1) FROM t WHERE c2 > 0`, i.e. it references attno 2 only, unless stated.
**1. Cost no longer depends on how wide the table is.** Query fixed on attno 2, table width varied:
| table columns | before | after | heap |
|---|---|---|---|
| 20 | 46.9 ms | 35.9 ms | 35.7 ms |
| 50 | 67.3 ms | 36.9 ms | 38.4 ms |
| 100 | 114.2 ms | 36.0 ms | 36.8 ms |
| 200 | 171.1 ms | 37.6 ms | 39.0 ms |
| **281** | **236.0 ms** | **41.4 ms** | 39.1 ms |
| 281, all columns populated | 415.0 ms | 112.7 ms | 103.6 ms |
**2. Cost now depends on the highest attribute number referenced, exactly as heap does.**
Table fixed at 281 columns, referenced column varied (`count(cN)`):
| referenced up to | after | heap |
|---|---|---|
| c2 | 37.3 ms | 36.1 ms |
| c50 | 68.2 ms | 56.4 ms |
| c140 | 121.5 ms | 95.4 ms |
| c281 | 212.4 ms | 156.2 ms |
Slopes: AO 0.63 ms per column per 500k rows, heap 0.43 ms. So the win is not that MemTuple
deforms faster than heap — it does not — it is that unread columns are no longer deformed at
all. A query reaching the last column costs what it always cost; the flat line in table 1 is
a consequence of referencing attno 2, not an unconditional property.
The reported query references attno 9, 12 and 13, so it now expands 13 columns instead of
281. `getsomeattrs` fills a prefix, so the gap columns (1-8, 10, 11) are still expanded;
skipping those would need per-column projection, which `tts_nvalid` cannot express (it is a
single watermark, so validity is necessarily a prefix) and which would mean claiming
unfilled columns are valid. Not attempted here.
Full-column access is not penalised: `count(c281)` 217.7 ms (before the change a bare scan
already cost 236.0 ms because it deformed everything), `sum(c281)` on fully populated data
415.9 ms vs 415.0 ms.
**Correctness**
- [x] `SELECT * FROM ao EXCEPT SELECT * FROM heap` and the reverse both return 0 rows, over
all 281 columns, for sparse and fully populated data
- [x] System columns unchanged: `ctid` / `tableoid` / `gp_segment_id` work, the four MVCC
columns still error (a MemTuple carries no MVCC fields; visibility lives in the visimap)
- [x] Materialize-dependent paths: CTE MATERIALIZED, ORDER BY, CTAS round trip, cursor
across statements
- [x] `make installcheck` A/B on the same cluster: **baseline 36/723 failed, with this
change 36/723 failed, failure sets identical**. The 36 are environmental in this
build (no pax_storage / iceberg AM, ORCA disabled).
- [x] Builds clean under `-O3 -Werror`
**Not done**
- [ ] No regression test added for the wide-table-narrow-projection behaviour. Both bugs
above surfaced only from the full suite, so this path is under-covered; a test
exercising a wide table plus the VACUUM / CLUSTER / CREATE INDEX paths would be
worth adding. Happy to add it to this MR if preferred.
- [ ] Not yet verified on GPDB 7 / upstream Cloudberry, though the patch applies to the
identical code there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
(cherry picked from commit 47a358712540acfa3ecd7ee1cb778ed01d5298f4)
lss602726449
added a commit
to lss602726449/cloudberrydb
that referenced
this pull request
Oct 8, 2026
…gineering/dev_collaboration#965
## Summary
Fixes field-engineering/dev_collaboration#965: a `count(1)` over a 281 column
append-only **row** table, qualified on three date columns, was 6x slower than the same
data in a heap table and 14x slower than AOCS.
### Root cause
`AppendOnlyExecutorReadBlock_ProcessTuple()` called `memtuple_deform()` and then
`ExecStoreVirtualTuple()`, and `appendonly_slot_callbacks()` returned `TTSOpsVirtual`.
A virtual slot promises its values are already in `tts_values[]`, so
`ExecStoreVirtualTuple()` sets `tts_nvalid = natts` and `slot_getattr()` never reaches
`getsomeattrs` — the scan has no choice but to expand **every** column of **every**
tuple. Heap tables are unaffected because `TTSOpsBufferHeapTuple` deforms incrementally
up to the highest attribute number the query references.
Scan cost therefore grew with the **width of the table** instead of the columns actually
read. Measured on a release build (`-O3`, no cassert), 500k rows, query touching only
attno 2:
| table columns | AO row before | heap |
|---|---|---|
| 20 | 46.9 ms | 35.7 ms |
| 100 | 114.2 ms | 36.8 ms |
| 281 | **236.0 ms** | 39.1 ms |
This is not how the format is meant to be read. A MemTuple keeps precomputed attribute
offsets in its binding, so any attribute is reachable in constant time, and before the
PostgreSQL 12 merge (`19cd1cf4b6`) the slot carried that binding and fetched attributes
on demand. That merge dropped the MemTuple slot shape — which at the time could only
exist as extra fields in the shared `TupleTableSlot` struct — and converted the tuple
eagerly instead. PostgreSQL 12 also introduced `TupleTableSlotOps` precisely so an access
method can keep tuples in its native format, which is what this MR finally uses.
**The same code is present in GPDB 7 and Apache Cloudberry main**, both verified to show
the identical regression (GPDB 7 release: 193.8 ms vs heap 28.9 ms at 281 columns). This
is an inherited PG12-merge artifact, not something introduced by Lightning.
### Change
Add `TTSOpsMemTuple`, a slot holding the MemTuple together with the binding needed to
interpret it, expanding attributes in `getsomeattrs`. Because offsets are precomputed,
`tts_nvalid` alone is the resume point — no separate deforming cursor (the `off` field
heap and minimal tuple slots carry) is needed. `memtuple_deform_range()` reuses the
existing inlinable internal helper so full-column access keeps the old bulk-deform speed.
Two details are easy to miss when adding a slot type here, both found by running the full
regression suite:
- `gp_segment_id` and `gp_foreign_server` are answered by the slot even though no tuple
format carries them, so `getsysattr` must keep the special cases that
`tts_virtual_getsysattr` documents. Omitting them broke ~59 tests.
- Not every slot reaches the scan through `table_slot_create()`.
`AppendOnlySegmentFileFullCompaction()` built a virtual slot directly and passed it to
`appendonly_getnextslot()`; in a release build the assertion is compiled out and the
larger slot struct would be written past the end of the smaller one. It now asks the
access method for the slot type, and `ProcessTuple` additionally falls back to expanding
the tuple up front for any non-MemTuple slot, so a caller bringing its own slot cannot
be handed one it cannot read.
## Test plan
Release build (`-O3`, `debug_assertions=off`), demo cluster, 500k rows.
The two tables below pin one variable each. Query is
`SELECT count(1) FROM t WHERE c2 > 0`, i.e. it references attno 2 only, unless stated.
**1. Cost no longer depends on how wide the table is.** Query fixed on attno 2, table width varied:
| table columns | before | after | heap |
|---|---|---|---|
| 20 | 46.9 ms | 35.9 ms | 35.7 ms |
| 50 | 67.3 ms | 36.9 ms | 38.4 ms |
| 100 | 114.2 ms | 36.0 ms | 36.8 ms |
| 200 | 171.1 ms | 37.6 ms | 39.0 ms |
| **281** | **236.0 ms** | **41.4 ms** | 39.1 ms |
| 281, all columns populated | 415.0 ms | 112.7 ms | 103.6 ms |
**2. Cost now depends on the highest attribute number referenced, exactly as heap does.**
Table fixed at 281 columns, referenced column varied (`count(cN)`):
| referenced up to | after | heap |
|---|---|---|
| c2 | 37.3 ms | 36.1 ms |
| c50 | 68.2 ms | 56.4 ms |
| c140 | 121.5 ms | 95.4 ms |
| c281 | 212.4 ms | 156.2 ms |
Slopes: AO 0.63 ms per column per 500k rows, heap 0.43 ms. So the win is not that MemTuple
deforms faster than heap — it does not — it is that unread columns are no longer deformed at
all. A query reaching the last column costs what it always cost; the flat line in table 1 is
a consequence of referencing attno 2, not an unconditional property.
The reported query references attno 9, 12 and 13, so it now expands 13 columns instead of
281. `getsomeattrs` fills a prefix, so the gap columns (1-8, 10, 11) are still expanded;
skipping those would need per-column projection, which `tts_nvalid` cannot express (it is a
single watermark, so validity is necessarily a prefix) and which would mean claiming
unfilled columns are valid. Not attempted here.
Full-column access is not penalised: `count(c281)` 217.7 ms (before the change a bare scan
already cost 236.0 ms because it deformed everything), `sum(c281)` on fully populated data
415.9 ms vs 415.0 ms.
**Correctness**
- [x] `SELECT * FROM ao EXCEPT SELECT * FROM heap` and the reverse both return 0 rows, over
all 281 columns, for sparse and fully populated data
- [x] System columns unchanged: `ctid` / `tableoid` / `gp_segment_id` work, the four MVCC
columns still error (a MemTuple carries no MVCC fields; visibility lives in the visimap)
- [x] Materialize-dependent paths: CTE MATERIALIZED, ORDER BY, CTAS round trip, cursor
across statements
- [x] `make installcheck` A/B on the same cluster: **baseline 36/723 failed, with this
change 36/723 failed, failure sets identical**. The 36 are environmental in this
build (no pax_storage / iceberg AM, ORCA disabled).
- [x] Builds clean under `-O3 -Werror`
**Not done**
- [ ] No regression test added for the wide-table-narrow-projection behaviour. Both bugs
above surfaced only from the full suite, so this path is under-covered; a test
exercising a wide table plus the VACUUM / CLUSTER / CREATE INDEX paths would be
worth adding. Happy to add it to this MR if preferred.
- [ ] Not yet verified on GPDB 7 / upstream Cloudberry, though the patch applies to the
identical code there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
(cherry picked from commit 47a358712540acfa3ecd7ee1cb778ed01d5298f4)
lss602726449
added a commit
to lss602726449/cloudberrydb
that referenced
this pull request
Oct 9, 2026
…gineering/dev_collaboration#965
## Summary
Fixes field-engineering/dev_collaboration#965: a `count(1)` over a 281 column
append-only **row** table, qualified on three date columns, was 6x slower than the same
data in a heap table and 14x slower than AOCS.
### Root cause
`AppendOnlyExecutorReadBlock_ProcessTuple()` called `memtuple_deform()` and then
`ExecStoreVirtualTuple()`, and `appendonly_slot_callbacks()` returned `TTSOpsVirtual`.
A virtual slot promises its values are already in `tts_values[]`, so
`ExecStoreVirtualTuple()` sets `tts_nvalid = natts` and `slot_getattr()` never reaches
`getsomeattrs` — the scan has no choice but to expand **every** column of **every**
tuple. Heap tables are unaffected because `TTSOpsBufferHeapTuple` deforms incrementally
up to the highest attribute number the query references.
Scan cost therefore grew with the **width of the table** instead of the columns actually
read. Measured on a release build (`-O3`, no cassert), 500k rows, query touching only
attno 2:
| table columns | AO row before | heap |
|---|---|---|
| 20 | 46.9 ms | 35.7 ms |
| 100 | 114.2 ms | 36.8 ms |
| 281 | **236.0 ms** | 39.1 ms |
This is not how the format is meant to be read. A MemTuple keeps precomputed attribute
offsets in its binding, so any attribute is reachable in constant time, and before the
PostgreSQL 12 merge (`19cd1cf4b6`) the slot carried that binding and fetched attributes
on demand. That merge dropped the MemTuple slot shape — which at the time could only
exist as extra fields in the shared `TupleTableSlot` struct — and converted the tuple
eagerly instead. PostgreSQL 12 also introduced `TupleTableSlotOps` precisely so an access
method can keep tuples in its native format, which is what this MR finally uses.
**The same code is present in GPDB 7 and Apache Cloudberry main**, both verified to show
the identical regression (GPDB 7 release: 193.8 ms vs heap 28.9 ms at 281 columns). This
is an inherited PG12-merge artifact, not something introduced by Lightning.
### Change
Add `TTSOpsMemTuple`, a slot holding the MemTuple together with the binding needed to
interpret it, expanding attributes in `getsomeattrs`. Because offsets are precomputed,
`tts_nvalid` alone is the resume point — no separate deforming cursor (the `off` field
heap and minimal tuple slots carry) is needed. `memtuple_deform_range()` reuses the
existing inlinable internal helper so full-column access keeps the old bulk-deform speed.
Two details are easy to miss when adding a slot type here, both found by running the full
regression suite:
- `gp_segment_id` and `gp_foreign_server` are answered by the slot even though no tuple
format carries them, so `getsysattr` must keep the special cases that
`tts_virtual_getsysattr` documents. Omitting them broke ~59 tests.
- Not every slot reaches the scan through `table_slot_create()`.
`AppendOnlySegmentFileFullCompaction()` built a virtual slot directly and passed it to
`appendonly_getnextslot()`; in a release build the assertion is compiled out and the
larger slot struct would be written past the end of the smaller one. It now asks the
access method for the slot type, and `ProcessTuple` additionally falls back to expanding
the tuple up front for any non-MemTuple slot, so a caller bringing its own slot cannot
be handed one it cannot read.
## Test plan
Release build (`-O3`, `debug_assertions=off`), demo cluster, 500k rows.
The two tables below pin one variable each. Query is
`SELECT count(1) FROM t WHERE c2 > 0`, i.e. it references attno 2 only, unless stated.
**1. Cost no longer depends on how wide the table is.** Query fixed on attno 2, table width varied:
| table columns | before | after | heap |
|---|---|---|---|
| 20 | 46.9 ms | 35.9 ms | 35.7 ms |
| 50 | 67.3 ms | 36.9 ms | 38.4 ms |
| 100 | 114.2 ms | 36.0 ms | 36.8 ms |
| 200 | 171.1 ms | 37.6 ms | 39.0 ms |
| **281** | **236.0 ms** | **41.4 ms** | 39.1 ms |
| 281, all columns populated | 415.0 ms | 112.7 ms | 103.6 ms |
**2. Cost now depends on the highest attribute number referenced, exactly as heap does.**
Table fixed at 281 columns, referenced column varied (`count(cN)`):
| referenced up to | after | heap |
|---|---|---|
| c2 | 37.3 ms | 36.1 ms |
| c50 | 68.2 ms | 56.4 ms |
| c140 | 121.5 ms | 95.4 ms |
| c281 | 212.4 ms | 156.2 ms |
Slopes: AO 0.63 ms per column per 500k rows, heap 0.43 ms. So the win is not that MemTuple
deforms faster than heap — it does not — it is that unread columns are no longer deformed at
all. A query reaching the last column costs what it always cost; the flat line in table 1 is
a consequence of referencing attno 2, not an unconditional property.
The reported query references attno 9, 12 and 13, so it now expands 13 columns instead of
281. `getsomeattrs` fills a prefix, so the gap columns (1-8, 10, 11) are still expanded;
skipping those would need per-column projection, which `tts_nvalid` cannot express (it is a
single watermark, so validity is necessarily a prefix) and which would mean claiming
unfilled columns are valid. Not attempted here.
Full-column access is not penalised: `count(c281)` 217.7 ms (before the change a bare scan
already cost 236.0 ms because it deformed everything), `sum(c281)` on fully populated data
415.9 ms vs 415.0 ms.
**Correctness**
- [x] `SELECT * FROM ao EXCEPT SELECT * FROM heap` and the reverse both return 0 rows, over
all 281 columns, for sparse and fully populated data
- [x] System columns unchanged: `ctid` / `tableoid` / `gp_segment_id` work, the four MVCC
columns still error (a MemTuple carries no MVCC fields; visibility lives in the visimap)
- [x] Materialize-dependent paths: CTE MATERIALIZED, ORDER BY, CTAS round trip, cursor
across statements
- [x] `make installcheck` A/B on the same cluster: **baseline 36/723 failed, with this
change 36/723 failed, failure sets identical**. The 36 are environmental in this
build (no pax_storage / iceberg AM, ORCA disabled).
- [x] Builds clean under `-O3 -Werror`
**Not done**
- [ ] No regression test added for the wide-table-narrow-projection behaviour. Both bugs
above surfaced only from the full suite, so this path is under-covered; a test
exercising a wide table plus the VACUUM / CLUSTER / CREATE INDEX paths would be
worth adding. Happy to add it to this MR if preferred.
- [ ] Not yet verified on GPDB 7 / upstream Cloudberry, though the patch applies to the
identical code there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
(cherry picked from commit 47a358712540acfa3ecd7ee1cb778ed01d5298f4)
tuhaihe
pushed a commit
to lss602726449/cloudberrydb
that referenced
this pull request
Oct 16, 2026
…gineering/dev_collaboration#965
## Summary
Fixes field-engineering/dev_collaboration#965: a `count(1)` over a 281 column
append-only **row** table, qualified on three date columns, was 6x slower than the same
data in a heap table and 14x slower than AOCS.
### Root cause
`AppendOnlyExecutorReadBlock_ProcessTuple()` called `memtuple_deform()` and then
`ExecStoreVirtualTuple()`, and `appendonly_slot_callbacks()` returned `TTSOpsVirtual`.
A virtual slot promises its values are already in `tts_values[]`, so
`ExecStoreVirtualTuple()` sets `tts_nvalid = natts` and `slot_getattr()` never reaches
`getsomeattrs` — the scan has no choice but to expand **every** column of **every**
tuple. Heap tables are unaffected because `TTSOpsBufferHeapTuple` deforms incrementally
up to the highest attribute number the query references.
Scan cost therefore grew with the **width of the table** instead of the columns actually
read. Measured on a release build (`-O3`, no cassert), 500k rows, query touching only
attno 2:
| table columns | AO row before | heap |
|---|---|---|
| 20 | 46.9 ms | 35.7 ms |
| 100 | 114.2 ms | 36.8 ms |
| 281 | **236.0 ms** | 39.1 ms |
This is not how the format is meant to be read. A MemTuple keeps precomputed attribute
offsets in its binding, so any attribute is reachable in constant time, and before the
PostgreSQL 12 merge (`19cd1cf4b6`) the slot carried that binding and fetched attributes
on demand. That merge dropped the MemTuple slot shape — which at the time could only
exist as extra fields in the shared `TupleTableSlot` struct — and converted the tuple
eagerly instead. PostgreSQL 12 also introduced `TupleTableSlotOps` precisely so an access
method can keep tuples in its native format, which is what this MR finally uses.
**The same code is present in GPDB 7 and Apache Cloudberry main**, both verified to show
the identical regression (GPDB 7 release: 193.8 ms vs heap 28.9 ms at 281 columns). This
is an inherited PG12-merge artifact, not something introduced by Lightning.
### Change
Add `TTSOpsMemTuple`, a slot holding the MemTuple together with the binding needed to
interpret it, expanding attributes in `getsomeattrs`. Because offsets are precomputed,
`tts_nvalid` alone is the resume point — no separate deforming cursor (the `off` field
heap and minimal tuple slots carry) is needed. `memtuple_deform_range()` reuses the
existing inlinable internal helper so full-column access keeps the old bulk-deform speed.
Two details are easy to miss when adding a slot type here, both found by running the full
regression suite:
- `gp_segment_id` and `gp_foreign_server` are answered by the slot even though no tuple
format carries them, so `getsysattr` must keep the special cases that
`tts_virtual_getsysattr` documents. Omitting them broke ~59 tests.
- Not every slot reaches the scan through `table_slot_create()`.
`AppendOnlySegmentFileFullCompaction()` built a virtual slot directly and passed it to
`appendonly_getnextslot()`; in a release build the assertion is compiled out and the
larger slot struct would be written past the end of the smaller one. It now asks the
access method for the slot type, and `ProcessTuple` additionally falls back to expanding
the tuple up front for any non-MemTuple slot, so a caller bringing its own slot cannot
be handed one it cannot read.
## Test plan
Release build (`-O3`, `debug_assertions=off`), demo cluster, 500k rows.
The two tables below pin one variable each. Query is
`SELECT count(1) FROM t WHERE c2 > 0`, i.e. it references attno 2 only, unless stated.
**1. Cost no longer depends on how wide the table is.** Query fixed on attno 2, table width varied:
| table columns | before | after | heap |
|---|---|---|---|
| 20 | 46.9 ms | 35.9 ms | 35.7 ms |
| 50 | 67.3 ms | 36.9 ms | 38.4 ms |
| 100 | 114.2 ms | 36.0 ms | 36.8 ms |
| 200 | 171.1 ms | 37.6 ms | 39.0 ms |
| **281** | **236.0 ms** | **41.4 ms** | 39.1 ms |
| 281, all columns populated | 415.0 ms | 112.7 ms | 103.6 ms |
**2. Cost now depends on the highest attribute number referenced, exactly as heap does.**
Table fixed at 281 columns, referenced column varied (`count(cN)`):
| referenced up to | after | heap |
|---|---|---|
| c2 | 37.3 ms | 36.1 ms |
| c50 | 68.2 ms | 56.4 ms |
| c140 | 121.5 ms | 95.4 ms |
| c281 | 212.4 ms | 156.2 ms |
Slopes: AO 0.63 ms per column per 500k rows, heap 0.43 ms. So the win is not that MemTuple
deforms faster than heap — it does not — it is that unread columns are no longer deformed at
all. A query reaching the last column costs what it always cost; the flat line in table 1 is
a consequence of referencing attno 2, not an unconditional property.
The reported query references attno 9, 12 and 13, so it now expands 13 columns instead of
281. `getsomeattrs` fills a prefix, so the gap columns (1-8, 10, 11) are still expanded;
skipping those would need per-column projection, which `tts_nvalid` cannot express (it is a
single watermark, so validity is necessarily a prefix) and which would mean claiming
unfilled columns are valid. Not attempted here.
Full-column access is not penalised: `count(c281)` 217.7 ms (before the change a bare scan
already cost 236.0 ms because it deformed everything), `sum(c281)` on fully populated data
415.9 ms vs 415.0 ms.
**Correctness**
- [x] `SELECT * FROM ao EXCEPT SELECT * FROM heap` and the reverse both return 0 rows, over
all 281 columns, for sparse and fully populated data
- [x] System columns unchanged: `ctid` / `tableoid` / `gp_segment_id` work, the four MVCC
columns still error (a MemTuple carries no MVCC fields; visibility lives in the visimap)
- [x] Materialize-dependent paths: CTE MATERIALIZED, ORDER BY, CTAS round trip, cursor
across statements
- [x] `make installcheck` A/B on the same cluster: **baseline 36/723 failed, with this
change 36/723 failed, failure sets identical**. The 36 are environmental in this
build (no pax_storage / iceberg AM, ORCA disabled).
- [x] Builds clean under `-O3 -Werror`
**Not done**
- [ ] No regression test added for the wide-table-narrow-projection behaviour. Both bugs
above surfaced only from the full suite, so this path is under-covered; a test
exercising a wide table plus the VACUUM / CLUSTER / CREATE INDEX paths would be
worth adding. Happy to add it to this MR if preferred.
- [ ] Not yet verified on GPDB 7 / upstream Cloudberry, though the patch applies to the
identical code there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
(cherry picked from commit 47a358712540acfa3ecd7ee1cb778ed01d5298f4)
tuhaihe
pushed a commit
that referenced
this pull request
Oct 16, 2026
…ing/dev_collaboration#965
## Summary
Fixes field-engineering/dev_collaboration#965: a `count(1)` over a 281 column
append-only **row** table, qualified on three date columns, was 6x slower than the same
data in a heap table and 14x slower than AOCS.
### Root cause
`AppendOnlyExecutorReadBlock_ProcessTuple()` called `memtuple_deform()` and then
`ExecStoreVirtualTuple()`, and `appendonly_slot_callbacks()` returned `TTSOpsVirtual`.
A virtual slot promises its values are already in `tts_values[]`, so
`ExecStoreVirtualTuple()` sets `tts_nvalid = natts` and `slot_getattr()` never reaches
`getsomeattrs` — the scan has no choice but to expand **every** column of **every**
tuple. Heap tables are unaffected because `TTSOpsBufferHeapTuple` deforms incrementally
up to the highest attribute number the query references.
Scan cost therefore grew with the **width of the table** instead of the columns actually
read. Measured on a release build (`-O3`, no cassert), 500k rows, query touching only
attno 2:
| table columns | AO row before | heap |
|---|---|---|
| 20 | 46.9 ms | 35.7 ms |
| 100 | 114.2 ms | 36.8 ms |
| 281 | **236.0 ms** | 39.1 ms |
This is not how the format is meant to be read. A MemTuple keeps precomputed attribute
offsets in its binding, so any attribute is reachable in constant time, and before the
PostgreSQL 12 merge (`19cd1cf4b6`) the slot carried that binding and fetched attributes
on demand. That merge dropped the MemTuple slot shape — which at the time could only
exist as extra fields in the shared `TupleTableSlot` struct — and converted the tuple
eagerly instead. PostgreSQL 12 also introduced `TupleTableSlotOps` precisely so an access
method can keep tuples in its native format, which is what this MR finally uses.
**The same code is present in GPDB 7 and Apache Cloudberry main**, both verified to show
the identical regression (GPDB 7 release: 193.8 ms vs heap 28.9 ms at 281 columns). This
is an inherited PG12-merge artifact, not something introduced by Lightning.
### Change
Add `TTSOpsMemTuple`, a slot holding the MemTuple together with the binding needed to
interpret it, expanding attributes in `getsomeattrs`. Because offsets are precomputed,
`tts_nvalid` alone is the resume point — no separate deforming cursor (the `off` field
heap and minimal tuple slots carry) is needed. `memtuple_deform_range()` reuses the
existing inlinable internal helper so full-column access keeps the old bulk-deform speed.
Two details are easy to miss when adding a slot type here, both found by running the full
regression suite:
- `gp_segment_id` and `gp_foreign_server` are answered by the slot even though no tuple
format carries them, so `getsysattr` must keep the special cases that
`tts_virtual_getsysattr` documents. Omitting them broke ~59 tests.
- Not every slot reaches the scan through `table_slot_create()`.
`AppendOnlySegmentFileFullCompaction()` built a virtual slot directly and passed it to
`appendonly_getnextslot()`; in a release build the assertion is compiled out and the
larger slot struct would be written past the end of the smaller one. It now asks the
access method for the slot type, and `ProcessTuple` additionally falls back to expanding
the tuple up front for any non-MemTuple slot, so a caller bringing its own slot cannot
be handed one it cannot read.
## Test plan
Release build (`-O3`, `debug_assertions=off`), demo cluster, 500k rows.
The two tables below pin one variable each. Query is
`SELECT count(1) FROM t WHERE c2 > 0`, i.e. it references attno 2 only, unless stated.
**1. Cost no longer depends on how wide the table is.** Query fixed on attno 2, table width varied:
| table columns | before | after | heap |
|---|---|---|---|
| 20 | 46.9 ms | 35.9 ms | 35.7 ms |
| 50 | 67.3 ms | 36.9 ms | 38.4 ms |
| 100 | 114.2 ms | 36.0 ms | 36.8 ms |
| 200 | 171.1 ms | 37.6 ms | 39.0 ms |
| **281** | **236.0 ms** | **41.4 ms** | 39.1 ms |
| 281, all columns populated | 415.0 ms | 112.7 ms | 103.6 ms |
**2. Cost now depends on the highest attribute number referenced, exactly as heap does.**
Table fixed at 281 columns, referenced column varied (`count(cN)`):
| referenced up to | after | heap |
|---|---|---|
| c2 | 37.3 ms | 36.1 ms |
| c50 | 68.2 ms | 56.4 ms |
| c140 | 121.5 ms | 95.4 ms |
| c281 | 212.4 ms | 156.2 ms |
Slopes: AO 0.63 ms per column per 500k rows, heap 0.43 ms. So the win is not that MemTuple
deforms faster than heap — it does not — it is that unread columns are no longer deformed at
all. A query reaching the last column costs what it always cost; the flat line in table 1 is
a consequence of referencing attno 2, not an unconditional property.
The reported query references attno 9, 12 and 13, so it now expands 13 columns instead of
281. `getsomeattrs` fills a prefix, so the gap columns (1-8, 10, 11) are still expanded;
skipping those would need per-column projection, which `tts_nvalid` cannot express (it is a
single watermark, so validity is necessarily a prefix) and which would mean claiming
unfilled columns are valid. Not attempted here.
Full-column access is not penalised: `count(c281)` 217.7 ms (before the change a bare scan
already cost 236.0 ms because it deformed everything), `sum(c281)` on fully populated data
415.9 ms vs 415.0 ms.
**Correctness**
- [x] `SELECT * FROM ao EXCEPT SELECT * FROM heap` and the reverse both return 0 rows, over
all 281 columns, for sparse and fully populated data
- [x] System columns unchanged: `ctid` / `tableoid` / `gp_segment_id` work, the four MVCC
columns still error (a MemTuple carries no MVCC fields; visibility lives in the visimap)
- [x] Materialize-dependent paths: CTE MATERIALIZED, ORDER BY, CTAS round trip, cursor
across statements
- [x] `make installcheck` A/B on the same cluster: **baseline 36/723 failed, with this
change 36/723 failed, failure sets identical**. The 36 are environmental in this
build (no pax_storage / iceberg AM, ORCA disabled).
- [x] Builds clean under `-O3 -Werror`
**Not done**
- [ ] No regression test added for the wide-table-narrow-projection behaviour. Both bugs
above surfaced only from the full suite, so this path is under-covered; a test
exercising a wide table plus the VACUUM / CLUSTER / CREATE INDEX paths would be
worth adding. Happy to add it to this MR if preferred.
- [ ] Not yet verified on GPDB 7 / upstream Cloudberry, though the patch applies to the
identical code there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
(cherry picked from commit 47a358712540acfa3ecd7ee1cb778ed01d5298f4)
roseduan
pushed a commit
to roseduan/cloudberrydb
that referenced
this pull request
Oct 23, 2026
…gineering/dev_collaboration#965
## Summary
Fixes field-engineering/dev_collaboration#965: a `count(1)` over a 281 column
append-only **row** table, qualified on three date columns, was 6x slower than the same
data in a heap table and 14x slower than AOCS.
### Root cause
`AppendOnlyExecutorReadBlock_ProcessTuple()` called `memtuple_deform()` and then
`ExecStoreVirtualTuple()`, and `appendonly_slot_callbacks()` returned `TTSOpsVirtual`.
A virtual slot promises its values are already in `tts_values[]`, so
`ExecStoreVirtualTuple()` sets `tts_nvalid = natts` and `slot_getattr()` never reaches
`getsomeattrs` — the scan has no choice but to expand **every** column of **every**
tuple. Heap tables are unaffected because `TTSOpsBufferHeapTuple` deforms incrementally
up to the highest attribute number the query references.
Scan cost therefore grew with the **width of the table** instead of the columns actually
read. Measured on a release build (`-O3`, no cassert), 500k rows, query touching only
attno 2:
| table columns | AO row before | heap |
|---|---|---|
| 20 | 46.9 ms | 35.7 ms |
| 100 | 114.2 ms | 36.8 ms |
| 281 | **236.0 ms** | 39.1 ms |
This is not how the format is meant to be read. A MemTuple keeps precomputed attribute
offsets in its binding, so any attribute is reachable in constant time, and before the
PostgreSQL 12 merge (`19cd1cf4b6`) the slot carried that binding and fetched attributes
on demand. That merge dropped the MemTuple slot shape — which at the time could only
exist as extra fields in the shared `TupleTableSlot` struct — and converted the tuple
eagerly instead. PostgreSQL 12 also introduced `TupleTableSlotOps` precisely so an access
method can keep tuples in its native format, which is what this MR finally uses.
**The same code is present in GPDB 7 and Apache Cloudberry main**, both verified to show
the identical regression (GPDB 7 release: 193.8 ms vs heap 28.9 ms at 281 columns). This
is an inherited PG12-merge artifact, not something introduced by Lightning.
### Change
Add `TTSOpsMemTuple`, a slot holding the MemTuple together with the binding needed to
interpret it, expanding attributes in `getsomeattrs`. Because offsets are precomputed,
`tts_nvalid` alone is the resume point — no separate deforming cursor (the `off` field
heap and minimal tuple slots carry) is needed. `memtuple_deform_range()` reuses the
existing inlinable internal helper so full-column access keeps the old bulk-deform speed.
Two details are easy to miss when adding a slot type here, both found by running the full
regression suite:
- `gp_segment_id` and `gp_foreign_server` are answered by the slot even though no tuple
format carries them, so `getsysattr` must keep the special cases that
`tts_virtual_getsysattr` documents. Omitting them broke ~59 tests.
- Not every slot reaches the scan through `table_slot_create()`.
`AppendOnlySegmentFileFullCompaction()` built a virtual slot directly and passed it to
`appendonly_getnextslot()`; in a release build the assertion is compiled out and the
larger slot struct would be written past the end of the smaller one. It now asks the
access method for the slot type, and `ProcessTuple` additionally falls back to expanding
the tuple up front for any non-MemTuple slot, so a caller bringing its own slot cannot
be handed one it cannot read.
## Test plan
Release build (`-O3`, `debug_assertions=off`), demo cluster, 500k rows.
The two tables below pin one variable each. Query is
`SELECT count(1) FROM t WHERE c2 > 0`, i.e. it references attno 2 only, unless stated.
**1. Cost no longer depends on how wide the table is.** Query fixed on attno 2, table width varied:
| table columns | before | after | heap |
|---|---|---|---|
| 20 | 46.9 ms | 35.9 ms | 35.7 ms |
| 50 | 67.3 ms | 36.9 ms | 38.4 ms |
| 100 | 114.2 ms | 36.0 ms | 36.8 ms |
| 200 | 171.1 ms | 37.6 ms | 39.0 ms |
| **281** | **236.0 ms** | **41.4 ms** | 39.1 ms |
| 281, all columns populated | 415.0 ms | 112.7 ms | 103.6 ms |
**2. Cost now depends on the highest attribute number referenced, exactly as heap does.**
Table fixed at 281 columns, referenced column varied (`count(cN)`):
| referenced up to | after | heap |
|---|---|---|
| c2 | 37.3 ms | 36.1 ms |
| c50 | 68.2 ms | 56.4 ms |
| c140 | 121.5 ms | 95.4 ms |
| c281 | 212.4 ms | 156.2 ms |
Slopes: AO 0.63 ms per column per 500k rows, heap 0.43 ms. So the win is not that MemTuple
deforms faster than heap — it does not — it is that unread columns are no longer deformed at
all. A query reaching the last column costs what it always cost; the flat line in table 1 is
a consequence of referencing attno 2, not an unconditional property.
The reported query references attno 9, 12 and 13, so it now expands 13 columns instead of
281. `getsomeattrs` fills a prefix, so the gap columns (1-8, 10, 11) are still expanded;
skipping those would need per-column projection, which `tts_nvalid` cannot express (it is a
single watermark, so validity is necessarily a prefix) and which would mean claiming
unfilled columns are valid. Not attempted here.
Full-column access is not penalised: `count(c281)` 217.7 ms (before the change a bare scan
already cost 236.0 ms because it deformed everything), `sum(c281)` on fully populated data
415.9 ms vs 415.0 ms.
**Correctness**
- [x] `SELECT * FROM ao EXCEPT SELECT * FROM heap` and the reverse both return 0 rows, over
all 281 columns, for sparse and fully populated data
- [x] System columns unchanged: `ctid` / `tableoid` / `gp_segment_id` work, the four MVCC
columns still error (a MemTuple carries no MVCC fields; visibility lives in the visimap)
- [x] Materialize-dependent paths: CTE MATERIALIZED, ORDER BY, CTAS round trip, cursor
across statements
- [x] `make installcheck` A/B on the same cluster: **baseline 36/723 failed, with this
change 36/723 failed, failure sets identical**. The 36 are environmental in this
build (no pax_storage / iceberg AM, ORCA disabled).
- [x] Builds clean under `-O3 -Werror`
**Not done**
- [ ] No regression test added for the wide-table-narrow-projection behaviour. Both bugs
above surfaced only from the full suite, so this path is under-covered; a test
exercising a wide table plus the VACUUM / CLUSTER / CREATE INDEX paths would be
worth adding. Happy to add it to this MR if preferred.
- [ ] Not yet verified on GPDB 7 / upstream Cloudberry, though the patch applies to the
identical code there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
(cherry picked from commit 47a358712540acfa3ecd7ee1cb778ed01d5298f4)
Alena0704
pushed a commit
to open-gpdb/cloudberry
that referenced
this pull request
Oct 23, 2026
…gineering/dev_collaboration#965
## Summary
Fixes field-engineering/dev_collaboration#965: a `count(1)` over a 281 column
append-only **row** table, qualified on three date columns, was 6x slower than the same
data in a heap table and 14x slower than AOCS.
### Root cause
`AppendOnlyExecutorReadBlock_ProcessTuple()` called `memtuple_deform()` and then
`ExecStoreVirtualTuple()`, and `appendonly_slot_callbacks()` returned `TTSOpsVirtual`.
A virtual slot promises its values are already in `tts_values[]`, so
`ExecStoreVirtualTuple()` sets `tts_nvalid = natts` and `slot_getattr()` never reaches
`getsomeattrs` — the scan has no choice but to expand **every** column of **every**
tuple. Heap tables are unaffected because `TTSOpsBufferHeapTuple` deforms incrementally
up to the highest attribute number the query references.
Scan cost therefore grew with the **width of the table** instead of the columns actually
read. Measured on a release build (`-O3`, no cassert), 500k rows, query touching only
attno 2:
| table columns | AO row before | heap |
|---|---|---|
| 20 | 46.9 ms | 35.7 ms |
| 100 | 114.2 ms | 36.8 ms |
| 281 | **236.0 ms** | 39.1 ms |
This is not how the format is meant to be read. A MemTuple keeps precomputed attribute
offsets in its binding, so any attribute is reachable in constant time, and before the
PostgreSQL 12 merge (`19cd1cf4b6`) the slot carried that binding and fetched attributes
on demand. That merge dropped the MemTuple slot shape — which at the time could only
exist as extra fields in the shared `TupleTableSlot` struct — and converted the tuple
eagerly instead. PostgreSQL 12 also introduced `TupleTableSlotOps` precisely so an access
method can keep tuples in its native format, which is what this MR finally uses.
**The same code is present in GPDB 7 and Apache Cloudberry main**, both verified to show
the identical regression (GPDB 7 release: 193.8 ms vs heap 28.9 ms at 281 columns). This
is an inherited PG12-merge artifact, not something introduced by Lightning.
### Change
Add `TTSOpsMemTuple`, a slot holding the MemTuple together with the binding needed to
interpret it, expanding attributes in `getsomeattrs`. Because offsets are precomputed,
`tts_nvalid` alone is the resume point — no separate deforming cursor (the `off` field
heap and minimal tuple slots carry) is needed. `memtuple_deform_range()` reuses the
existing inlinable internal helper so full-column access keeps the old bulk-deform speed.
Two details are easy to miss when adding a slot type here, both found by running the full
regression suite:
- `gp_segment_id` and `gp_foreign_server` are answered by the slot even though no tuple
format carries them, so `getsysattr` must keep the special cases that
`tts_virtual_getsysattr` documents. Omitting them broke ~59 tests.
- Not every slot reaches the scan through `table_slot_create()`.
`AppendOnlySegmentFileFullCompaction()` built a virtual slot directly and passed it to
`appendonly_getnextslot()`; in a release build the assertion is compiled out and the
larger slot struct would be written past the end of the smaller one. It now asks the
access method for the slot type, and `ProcessTuple` additionally falls back to expanding
the tuple up front for any non-MemTuple slot, so a caller bringing its own slot cannot
be handed one it cannot read.
## Test plan
Release build (`-O3`, `debug_assertions=off`), demo cluster, 500k rows.
The two tables below pin one variable each. Query is
`SELECT count(1) FROM t WHERE c2 > 0`, i.e. it references attno 2 only, unless stated.
**1. Cost no longer depends on how wide the table is.** Query fixed on attno 2, table width varied:
| table columns | before | after | heap |
|---|---|---|---|
| 20 | 46.9 ms | 35.9 ms | 35.7 ms |
| 50 | 67.3 ms | 36.9 ms | 38.4 ms |
| 100 | 114.2 ms | 36.0 ms | 36.8 ms |
| 200 | 171.1 ms | 37.6 ms | 39.0 ms |
| **281** | **236.0 ms** | **41.4 ms** | 39.1 ms |
| 281, all columns populated | 415.0 ms | 112.7 ms | 103.6 ms |
**2. Cost now depends on the highest attribute number referenced, exactly as heap does.**
Table fixed at 281 columns, referenced column varied (`count(cN)`):
| referenced up to | after | heap |
|---|---|---|
| c2 | 37.3 ms | 36.1 ms |
| c50 | 68.2 ms | 56.4 ms |
| c140 | 121.5 ms | 95.4 ms |
| c281 | 212.4 ms | 156.2 ms |
Slopes: AO 0.63 ms per column per 500k rows, heap 0.43 ms. So the win is not that MemTuple
deforms faster than heap — it does not — it is that unread columns are no longer deformed at
all. A query reaching the last column costs what it always cost; the flat line in table 1 is
a consequence of referencing attno 2, not an unconditional property.
The reported query references attno 9, 12 and 13, so it now expands 13 columns instead of
281. `getsomeattrs` fills a prefix, so the gap columns (1-8, 10, 11) are still expanded;
skipping those would need per-column projection, which `tts_nvalid` cannot express (it is a
single watermark, so validity is necessarily a prefix) and which would mean claiming
unfilled columns are valid. Not attempted here.
Full-column access is not penalised: `count(c281)` 217.7 ms (before the change a bare scan
already cost 236.0 ms because it deformed everything), `sum(c281)` on fully populated data
415.9 ms vs 415.0 ms.
**Correctness**
- [x] `SELECT * FROM ao EXCEPT SELECT * FROM heap` and the reverse both return 0 rows, over
all 281 columns, for sparse and fully populated data
- [x] System columns unchanged: `ctid` / `tableoid` / `gp_segment_id` work, the four MVCC
columns still error (a MemTuple carries no MVCC fields; visibility lives in the visimap)
- [x] Materialize-dependent paths: CTE MATERIALIZED, ORDER BY, CTAS round trip, cursor
across statements
- [x] `make installcheck` A/B on the same cluster: **baseline 36/723 failed, with this
change 36/723 failed, failure sets identical**. The 36 are environmental in this
build (no pax_storage / iceberg AM, ORCA disabled).
- [x] Builds clean under `-O3 -Werror`
**Not done**
- [ ] No regression test added for the wide-table-narrow-projection behaviour. Both bugs
above surfaced only from the full suite, so this path is under-covered; a test
exercising a wide table plus the VACUUM / CLUSTER / CREATE INDEX paths would be
worth adding. Happy to add it to this MR if preferred.
- [ ] Not yet verified on GPDB 7 / upstream Cloudberry, though the patch applies to the
identical code there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
(cherry picked from commit 47a358712540acfa3ecd7ee1cb778ed01d5298f4)
(cherry picked from commit 342c89d)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit introduces support for partitioned tables in the AQUMV (Answer Query Using Materialized Views) feature. Users can now create materialized views based on partitioned tables, which may include multiple levels of child partitions. When a query is executed on the root partition table, AQUMV will automatically rewrite the query to utilize the materialized view, thereby improving query performance by avoiding direct access to the partitioned tables.
Example:
We have a partition table par with children tables of multiple level.
create materialized view mv_par as select count(*) as column_1 from par;
When execute:
AQUMV will rewrite the SQL to:
Answer the results from materialized view mv_par instead of querying on par and its partitions.
This enhancement significantly boosts query performance in OLAP environments where partitioned tables are extensively used, allowing users to fully leverage the benefits of materialized views.
Authored-by: Zhang Mingli avamingli@gmail.com
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions