Repository navigation
Conversation
For a better user and developer experience, update the Issue and PR templates. Files changed: * Add bug-report.yml * Modify pull_request_template.md * Delete bug-report.md * Delete enhancement.md * Delete feature-request.md
HuSen8891
added a commit
to HuSen8891/cloudberrydb
that referenced
this pull request
Mar 1, 2024
For test case:
create table t0(c0 inet) distributed randomly;
create table t2(c0 inet) distributed randomly;
create table t3(c0 inet) distributed randomly;
SELECT ALL t2.c0, t3.c0, t0.c0 FROM t0, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE (((('0.5496844753539182')||(t3.c0)))LIKE(CAST((0.13292931)::MONEY AS VARCHAR(971))))
UNION ALL SELECT t2.c0, t3.c0, t0.c0 FROM t0, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE NOT ((((('0.5496844753539182')||(t3.c0)))LIKE((CAST(0.13292931 AS MONEY))::VARCHAR(971))))
UNION ALL SELECT ALL t2.c0, t3.c0, t0.c0 FROM t0*, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE ((((('0.5496844753539182')||(t3.c0)))LIKE((CAST(0.13292931 AS MONEY))::VARCHAR(971)))) ISNULL;
will cause crash because of assert failure in 'create_plan_recurse'.
'apache#3 0x00007fe94eccf476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
apache#4 0x00007fe94ecb57f3 in __GI_abort () at ./stdlib/abort.c:79
apache#5 0x00007fe94fcdd548 in ExceptionalCondition (conditionName=0x7fe95043dcd0 "best_path->parallel_workers == best_path->locus.parallel_workers",
errorType=0x7fe95043db06 "FailedAssertion", fileName=0x7fe95043dbdb "createplan.c", lineNumber=623) at assert.c:48
apache#6 0x00007fe94f94918f in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0380, flags=1) at createplan.c:623
apache#7 0x00007fe94f94a1f8 in create_append_plan (root=0x55d7cbe96f78, best_path=0x55d7cbec0700, flags=1) at createplan.c:1380
apache#8 0x00007fe94f948d37 in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0700, flags=1) at createplan.c:481
apache#9 0x00007fe94f94e2d1 in create_motion_plan (root=0x55d7cbe96f78, path=0x55d7cbec0e50) at createplan.c:3316
#10 0x00007fe94f9490dc in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0e50, flags=1) at createplan.c:608
apache#11 0x00007fe94f948ba3 in create_plan (root=0x55d7cbe96f78, best_path=0x55d7cbec0e50, curSlice=0x55d7cbe96f20) at createplan.c:392'
The parallel_workers should be set to zero because parallel full join is not supported yet.
my-ship-it
pushed a commit
that referenced
this pull request
Mar 4, 2024
For test case:
create table t0(c0 inet) distributed randomly;
create table t2(c0 inet) distributed randomly;
create table t3(c0 inet) distributed randomly;
SELECT ALL t2.c0, t3.c0, t0.c0 FROM t0, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE (((('0.5496844753539182')||(t3.c0)))LIKE(CAST((0.13292931)::MONEY AS VARCHAR(971))))
UNION ALL SELECT t2.c0, t3.c0, t0.c0 FROM t0, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE NOT ((((('0.5496844753539182')||(t3.c0)))LIKE((CAST(0.13292931 AS MONEY))::VARCHAR(971))))
UNION ALL SELECT ALL t2.c0, t3.c0, t0.c0 FROM t0*, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE ((((('0.5496844753539182')||(t3.c0)))LIKE((CAST(0.13292931 AS MONEY))::VARCHAR(971)))) ISNULL;
will cause crash because of assert failure in 'create_plan_recurse'.
'#3 0x00007fe94eccf476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4 0x00007fe94ecb57f3 in __GI_abort () at ./stdlib/abort.c:79
#5 0x00007fe94fcdd548 in ExceptionalCondition (conditionName=0x7fe95043dcd0 "best_path->parallel_workers == best_path->locus.parallel_workers",
errorType=0x7fe95043db06 "FailedAssertion", fileName=0x7fe95043dbdb "createplan.c", lineNumber=623) at assert.c:48
#6 0x00007fe94f94918f in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0380, flags=1) at createplan.c:623
#7 0x00007fe94f94a1f8 in create_append_plan (root=0x55d7cbe96f78, best_path=0x55d7cbec0700, flags=1) at createplan.c:1380
#8 0x00007fe94f948d37 in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0700, flags=1) at createplan.c:481
#9 0x00007fe94f94e2d1 in create_motion_plan (root=0x55d7cbe96f78, path=0x55d7cbec0e50) at createplan.c:3316
#10 0x00007fe94f9490dc in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0e50, flags=1) at createplan.c:608
#11 0x00007fe94f948ba3 in create_plan (root=0x55d7cbe96f78, best_path=0x55d7cbec0e50, curSlice=0x55d7cbe96f20) at createplan.c:392'
The parallel_workers should be set to zero because parallel full join is not supported yet.
foreyes
pushed a commit
to foreyes/cloudberrydb
that referenced
this pull request
Jul 8, 2024
For test case:
create table t0(c0 inet) distributed randomly;
create table t2(c0 inet) distributed randomly;
create table t3(c0 inet) distributed randomly;
SELECT ALL t2.c0, t3.c0, t0.c0 FROM t0, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE (((('0.5496844753539182')||(t3.c0)))LIKE(CAST((0.13292931)::MONEY AS VARCHAR(971))))
UNION ALL SELECT t2.c0, t3.c0, t0.c0 FROM t0, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE NOT ((((('0.5496844753539182')||(t3.c0)))LIKE((CAST(0.13292931 AS MONEY))::VARCHAR(971))))
UNION ALL SELECT ALL t2.c0, t3.c0, t0.c0 FROM t0*, ONLY t3 FULL OUTER JOIN t2 ON ((t2.c0)=(t3.c0))
WHERE ((((('0.5496844753539182')||(t3.c0)))LIKE((CAST(0.13292931 AS MONEY))::VARCHAR(971)))) ISNULL;
will cause crash because of assert failure in 'create_plan_recurse'.
'apache#3 0x00007fe94eccf476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
apache#4 0x00007fe94ecb57f3 in __GI_abort () at ./stdlib/abort.c:79
apache#5 0x00007fe94fcdd548 in ExceptionalCondition (conditionName=0x7fe95043dcd0 "best_path->parallel_workers == best_path->locus.parallel_workers",
errorType=0x7fe95043db06 "FailedAssertion", fileName=0x7fe95043dbdb "createplan.c", lineNumber=623) at assert.c:48
apache#6 0x00007fe94f94918f in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0380, flags=1) at createplan.c:623
apache#7 0x00007fe94f94a1f8 in create_append_plan (root=0x55d7cbe96f78, best_path=0x55d7cbec0700, flags=1) at createplan.c:1380
apache#8 0x00007fe94f948d37 in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0700, flags=1) at createplan.c:481
apache#9 0x00007fe94f94e2d1 in create_motion_plan (root=0x55d7cbe96f78, path=0x55d7cbec0e50) at createplan.c:3316
#10 0x00007fe94f9490dc in create_plan_recurse (root=0x55d7cbe96f78, best_path=0x55d7cbec0e50, flags=1) at createplan.c:608
apache#11 0x00007fe94f948ba3 in create_plan (root=0x55d7cbe96f78, best_path=0x55d7cbec0e50, curSlice=0x55d7cbe96f20) at createplan.c:392'
The parallel_workers should be set to zero because parallel full join is not supported yet.
robozmey
added a commit
to robozmey/cloudberrydb
that referenced
this pull request
Dec 13, 2024
Dumps CdbExplain_NodeSummary struct in EXPLAIN ANALYZE
`set gp_enable_explain_node_summary=on;`
`drop table if exists tt; create table tt (a int, b int) distributed randomly;`
`explain (analyze,verbose) insert into tt select * from generate_series(1,1000)a,generate_series(1,1000)b;`
`explain (analyze,verbose) select * from tt where a > b;`
```
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (actual time=33.612..78.159 rows=1000 loops=1)
Output: a, b
Node Summary: vmax vsum vcnt imax
ntuples: 1000 1000 1 -1
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 44592 44592 1 -1
totalPartTableScanned: 0 0 0 0
segindex0: -1
ninst: 1
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg-1) 241 0.00 0.03 0.08 1000.00 1.00 0 0 0 0 0 0 44592.00 0 0 0 0 0 0 0
-> Seq Scan on public.tt (cost=0.00..431.00 rows=1 width=8) (actual time=2.512..77.640 rows=353 loops=1)
Output: a, b
Filter: (tt.a = 100)
Node Summary: vmax vsum vcnt imax
ntuples: 353 1000 3 0
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 19072 57216 3 0
totalPartTableScanned: 0 0 0 0
segindex0: 0
ninst: 3
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg0) 211 0.00 0.00 0.08 353.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg1) 211 0.00 0.00 0.03 330.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg2) 211 0.00 0.00 0.05 317.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
Planning time: 4.514 ms
(slice0) Executor memory: 119K bytes.
(slice1) Executor memory: 27K bytes avg x 3 workers, 27K bytes max (seg0).
Memory used: 128000kB
Optimizer: Pivotal Optimizer (GPORCA)
Execution time: 78.584 ms
(39 rows)
```
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Dec 20, 2024
We've heard a couple of reports of people having trouble with multi-gigabyte-sized query-texts files. It occurred to me that on 32-bit platforms, there could be an issue with integer overflow of calculations associated with the total query text size. Address that with several changes: 1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX. The hashtable code will bound it to that anyway unless "long" is 64 bits. We still need overflow guards on its use, but this helps. 2. Add a check to prevent extending the query-texts file to more than MaxAllocHugeSize. If it got that big, qtext_load_file would certainly fail, so there's not much point in allowing it. Without this, we'd need to consider whether extent, query_offset, and related variables shouldn't be off_t not size_t. 3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit arithmetic on all platforms. It appears possible that under duress those multiplications could overflow 32 bits, yielding a false conclusion that we need to garbage-collect the texts file, which could lead to repeatedly garbage-collecting after every hash table insertion. Per report from Bruno da Silva. I'm not convinced that these issues fully explain his problem; there may be some other bug that's contributing to the query-texts file becoming so large in the first place. But it did get that big, so apache#2 is a reasonable defense, and apache#3 could explain the reported performance difficulties. (See also commit 8bbe4cb, which addressed some related bugs. The second Discussion: link is the thread that led up to that.) This issue is old, and is primarily a problem for old platforms, so back-patch. Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Dec 22, 2024
We've heard a couple of reports of people having trouble with multi-gigabyte-sized query-texts files. It occurred to me that on 32-bit platforms, there could be an issue with integer overflow of calculations associated with the total query text size. Address that with several changes: 1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX. The hashtable code will bound it to that anyway unless "long" is 64 bits. We still need overflow guards on its use, but this helps. 2. Add a check to prevent extending the query-texts file to more than MaxAllocHugeSize. If it got that big, qtext_load_file would certainly fail, so there's not much point in allowing it. Without this, we'd need to consider whether extent, query_offset, and related variables shouldn't be off_t not size_t. 3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit arithmetic on all platforms. It appears possible that under duress those multiplications could overflow 32 bits, yielding a false conclusion that we need to garbage-collect the texts file, which could lead to repeatedly garbage-collecting after every hash table insertion. Per report from Bruno da Silva. I'm not convinced that these issues fully explain his problem; there may be some other bug that's contributing to the query-texts file becoming so large in the first place. But it did get that big, so apache#2 is a reasonable defense, and apache#3 could explain the reported performance difficulties. (See also commit 8bbe4cb, which addressed some related bugs. The second Discussion: link is the thread that led up to that.) This issue is old, and is primarily a problem for old platforms, so back-patch. Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Dec 24, 2024
We've heard a couple of reports of people having trouble with multi-gigabyte-sized query-texts files. It occurred to me that on 32-bit platforms, there could be an issue with integer overflow of calculations associated with the total query text size. Address that with several changes: 1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX. The hashtable code will bound it to that anyway unless "long" is 64 bits. We still need overflow guards on its use, but this helps. 2. Add a check to prevent extending the query-texts file to more than MaxAllocHugeSize. If it got that big, qtext_load_file would certainly fail, so there's not much point in allowing it. Without this, we'd need to consider whether extent, query_offset, and related variables shouldn't be off_t not size_t. 3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit arithmetic on all platforms. It appears possible that under duress those multiplications could overflow 32 bits, yielding a false conclusion that we need to garbage-collect the texts file, which could lead to repeatedly garbage-collecting after every hash table insertion. Per report from Bruno da Silva. I'm not convinced that these issues fully explain his problem; there may be some other bug that's contributing to the query-texts file becoming so large in the first place. But it did get that big, so apache#2 is a reasonable defense, and apache#3 could explain the reported performance difficulties. (See also commit 8bbe4cb, which addressed some related bugs. The second Discussion: link is the thread that led up to that.) This issue is old, and is primarily a problem for old platforms, so back-patch. Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
my-ship-it
pushed a commit
that referenced
this pull request
Dec 24, 2024
We've heard a couple of reports of people having trouble with multi-gigabyte-sized query-texts files. It occurred to me that on 32-bit platforms, there could be an issue with integer overflow of calculations associated with the total query text size. Address that with several changes: 1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX. The hashtable code will bound it to that anyway unless "long" is 64 bits. We still need overflow guards on its use, but this helps. 2. Add a check to prevent extending the query-texts file to more than MaxAllocHugeSize. If it got that big, qtext_load_file would certainly fail, so there's not much point in allowing it. Without this, we'd need to consider whether extent, query_offset, and related variables shouldn't be off_t not size_t. 3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit arithmetic on all platforms. It appears possible that under duress those multiplications could overflow 32 bits, yielding a false conclusion that we need to garbage-collect the texts file, which could lead to repeatedly garbage-collecting after every hash table insertion. Per report from Bruno da Silva. I'm not convinced that these issues fully explain his problem; there may be some other bug that's contributing to the query-texts file becoming so large in the first place. But it did get that big, so #2 is a reasonable defense, and #3 could explain the reported performance difficulties. (See also commit 8bbe4cb, which addressed some related bugs. The second Discussion: link is the thread that led up to that.) This issue is old, and is primarily a problem for old platforms, so back-patch. Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Dec 24, 2024
We've heard a couple of reports of people having trouble with multi-gigabyte-sized query-texts files. It occurred to me that on 32-bit platforms, there could be an issue with integer overflow of calculations associated with the total query text size. Address that with several changes: 1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX. The hashtable code will bound it to that anyway unless "long" is 64 bits. We still need overflow guards on its use, but this helps. 2. Add a check to prevent extending the query-texts file to more than MaxAllocHugeSize. If it got that big, qtext_load_file would certainly fail, so there's not much point in allowing it. Without this, we'd need to consider whether extent, query_offset, and related variables shouldn't be off_t not size_t. 3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit arithmetic on all platforms. It appears possible that under duress those multiplications could overflow 32 bits, yielding a false conclusion that we need to garbage-collect the texts file, which could lead to repeatedly garbage-collecting after every hash table insertion. Per report from Bruno da Silva. I'm not convinced that these issues fully explain his problem; there may be some other bug that's contributing to the query-texts file becoming so large in the first place. But it did get that big, so apache#2 is a reasonable defense, and apache#3 could explain the reported performance difficulties. (See also commit 8bbe4cb, which addressed some related bugs. The second Discussion: link is the thread that led up to that.) This issue is old, and is primarily a problem for old platforms, so back-patch. Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Dec 24, 2024
We've heard a couple of reports of people having trouble with multi-gigabyte-sized query-texts files. It occurred to me that on 32-bit platforms, there could be an issue with integer overflow of calculations associated with the total query text size. Address that with several changes: 1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX. The hashtable code will bound it to that anyway unless "long" is 64 bits. We still need overflow guards on its use, but this helps. 2. Add a check to prevent extending the query-texts file to more than MaxAllocHugeSize. If it got that big, qtext_load_file would certainly fail, so there's not much point in allowing it. Without this, we'd need to consider whether extent, query_offset, and related variables shouldn't be off_t not size_t. 3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit arithmetic on all platforms. It appears possible that under duress those multiplications could overflow 32 bits, yielding a false conclusion that we need to garbage-collect the texts file, which could lead to repeatedly garbage-collecting after every hash table insertion. Per report from Bruno da Silva. I'm not convinced that these issues fully explain his problem; there may be some other bug that's contributing to the query-texts file becoming so large in the first place. But it did get that big, so apache#2 is a reasonable defense, and apache#3 could explain the reported performance difficulties. (See also commit 8bbe4cb, which addressed some related bugs. The second Discussion: link is the thread that led up to that.) This issue is old, and is primarily a problem for old platforms, so back-patch. Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
avamingli
pushed a commit
that referenced
this pull request
Jan 23, 2025
## Problem
An error occurs in python lib when a plpython function is executed.
After our analysis, in the user's cluster, a plpython UDF
was running with the unstable network, and got a timeout error:
`failed to acquire resources on one or more segments`.
Then a plpython UDF was run in the same session, and the UDF
failed with GC error.
Here is the core dump:
```
2023-11-24 10:15:18.945507 CST,,,p2705198,th2081832064,,,,0,,,seg-1,,,,,"LOG","00000","3rd party error log:
#0 0x7f7c68b6d55b in frame_dealloc /home/cc/repo/cpython/Objects/frameobject.c:509:5
#1 0x7f7c68b5109d in gen_send_ex /home/cc/repo/cpython/Objects/genobject.c:108:9
#2 0x7f7c68af9ddd in PyIter_Next /home/cc/repo/cpython/Objects/abstract.c:3118:14
#3 0x7f7c78caa5c0 in PLy_exec_function /home/cc/repo/gpdb6/src/pl/plpython/plpy_exec.c:134:11
#4 0x7f7c78cb5ffb in plpython_call_handler /home/cc/repo/gpdb6/src/pl/plpython/plpy_main.c:387:13
#5 0x562f5e008bb5 in ExecMakeTableFunctionResult /home/cc/repo/gpdb6/src/backend/executor/execQual.c:2395:13
#6 0x562f5e0dddec in FunctionNext_guts /home/cc/repo/gpdb6/src/backend/executor/nodeFunctionscan.c:142:5
#7 0x562f5e0da094 in FunctionNext /home/cc/repo/gpdb6/src/backend/executor/nodeFunctionscan.c:350:11
#8 0x562f5e03d4b0 in ExecScanFetch /home/cc/repo/gpdb6/src/backend/executor/execScan.c:84:9
#9 0x562f5e03cd8f in ExecScan /home/cc/repo/gpdb6/src/backend/executor/execScan.c:154:10
#10 0x562f5e0da072 in ExecFunctionScan /home/cc/repo/gpdb6/src/backend/executor/nodeFunctionscan.c:380:9
#11 0x562f5e001a1c in ExecProcNode /home/cc/repo/gpdb6/src/backend/executor/execProcnode.c:1071:13
#12 0x562f5dfe6377 in ExecutePlan /home/cc/repo/gpdb6/src/backend/executor/execMain.c:3202:10
#13 0x562f5dfe5bf4 in standard_ExecutorRun /home/cc/repo/gpdb6/src/backend/executor/execMain.c:1171:5
#14 0x562f5dfe4877 in ExecutorRun /home/cc/repo/gpdb6/src/backend/executor/execMain.c:992:4
#15 0x562f5e857e69 in PortalRunSelect /home/cc/repo/gpdb6/src/backend/tcop/pquery.c:1164:4
#16 0x562f5e856d3f in PortalRun /home/cc/repo/gpdb6/src/backend/tcop/pquery.c:1005:18
#17 0x562f5e84607a in exec_simple_query /home/cc/repo/gpdb6/src/backend/tcop/postgres.c:1848:10
```
## Reproduce
We can use a simple procedure to reproduce the above problem:
- set timeout GUC: `gpconfig -c gp_segment_connect_timeout -v 5` and `gpstop -ari`
- prepare function:
```
CREATE EXTENSION plpythonu;
CREATE OR REPLACE FUNCTION test_func() RETURNS SETOF int AS
$$
plpy.execute("select pg_backend_pid()")
for i in range(0, 5):
yield (i)
$$ LANGUAGE plpythonu;
```
- exit from the current psql session.
- stop the postmaster of segment: `gdb -p "the pid of segment postmaster"`
- enter a psql session.
- call `SELECT test_func();` and get error
```
gpadmin=# select test_func();
ERROR: function "test_func" error fetching next item from iterator (plpy_elog.c:121)
DETAIL: Exception: failed to acquire resources on one or more segments
CONTEXT: Traceback (most recent call last):
PL/Python function "test_func"
```
- quit gdb and make postmaster runnable.
- call `SELECT test_func();` again and get panic
```
gpadmin=# SELECT test_func();
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>
```
## Analysis
- There is an SPI call in test_func(): `plpy.execute()`.
- Then coordinator will start a subtransaction by PLy_spi_subtransaction_begin();
- Meanwhile, if the segment cannot receive the instruction from the coordinator,
the subtransaction beginning procedure return fails.
- BUT! The Python processor does not know whether an error happened and
does not clean its environment.
- Then the next plpython UDF in the same session will fail due to the wrong
Python environment.
## Solution
- Use try-catch to catch the exception caused by PLy_spi_subtransaction_begin()
- set the python error indicator by PLy_spi_exception_set()
Co-authored-by: Chen Mulong <chenmulong@gmail.com>
robozmey
added a commit
to robozmey/cloudberrydb
that referenced
this pull request
Mar 25, 2025
Dumps CdbExplain_NodeSummary struct in EXPLAIN ANALYZE
`set gp_enable_explain_node_summary=on;`
`drop table if exists tt; create table tt (a int, b int) distributed randomly;`
`explain (analyze,verbose) insert into tt select * from generate_series(1,1000)a,generate_series(1,1000)b;`
`explain (analyze,verbose) select * from tt where a > b;`
```
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (actual time=33.612..78.159 rows=1000 loops=1)
Output: a, b
Node Summary: vmax vsum vcnt imax
ntuples: 1000 1000 1 -1
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 44592 44592 1 -1
totalPartTableScanned: 0 0 0 0
segindex0: -1
ninst: 1
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg-1) 241 0.00 0.03 0.08 1000.00 1.00 0 0 0 0 0 0 44592.00 0 0 0 0 0 0 0
-> Seq Scan on public.tt (cost=0.00..431.00 rows=1 width=8) (actual time=2.512..77.640 rows=353 loops=1)
Output: a, b
Filter: (tt.a = 100)
Node Summary: vmax vsum vcnt imax
ntuples: 353 1000 3 0
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 19072 57216 3 0
totalPartTableScanned: 0 0 0 0
segindex0: 0
ninst: 3
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg0) 211 0.00 0.00 0.08 353.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg1) 211 0.00 0.00 0.03 330.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg2) 211 0.00 0.00 0.05 317.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
Planning time: 4.514 ms
(slice0) Executor memory: 119K bytes.
(slice1) Executor memory: 27K bytes avg x 3 workers, 27K bytes max (seg0).
Memory used: 128000kB
Optimizer: Pivotal Optimizer (GPORCA)
Execution time: 78.584 ms
(39 rows)
```
robozmey
added a commit
to robozmey/cloudberrydb
that referenced
this pull request
Apr 8, 2025
Dumps CdbExplain_NodeSummary struct in EXPLAIN ANALYZE
`set gp_enable_explain_node_summary=on;`
`drop table if exists tt; create table tt (a int, b int) distributed randomly;`
`explain (analyze,verbose) insert into tt select * from generate_series(1,1000)a,generate_series(1,1000)b;`
`explain (analyze,verbose) select * from tt where a > b;`
```
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (actual time=33.612..78.159 rows=1000 loops=1)
Output: a, b
Node Summary: vmax vsum vcnt imax
ntuples: 1000 1000 1 -1
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 44592 44592 1 -1
totalPartTableScanned: 0 0 0 0
segindex0: -1
ninst: 1
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg-1) 241 0.00 0.03 0.08 1000.00 1.00 0 0 0 0 0 0 44592.00 0 0 0 0 0 0 0
-> Seq Scan on public.tt (cost=0.00..431.00 rows=1 width=8) (actual time=2.512..77.640 rows=353 loops=1)
Output: a, b
Filter: (tt.a = 100)
Node Summary: vmax vsum vcnt imax
ntuples: 353 1000 3 0
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 19072 57216 3 0
totalPartTableScanned: 0 0 0 0
segindex0: 0
ninst: 3
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg0) 211 0.00 0.00 0.08 353.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg1) 211 0.00 0.00 0.03 330.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg2) 211 0.00 0.00 0.05 317.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
Planning time: 4.514 ms
(slice0) Executor memory: 119K bytes.
(slice1) Executor memory: 27K bytes avg x 3 workers, 27K bytes max (seg0).
Memory used: 128000kB
Optimizer: Pivotal Optimizer (GPORCA)
Execution time: 78.584 ms
(39 rows)
```
robozmey
added a commit
to robozmey/cloudberrydb
that referenced
this pull request
May 19, 2025
Dumps CdbExplain_NodeSummary struct in EXPLAIN ANALYZE
`set gp_enable_explain_node_summary=on;`
`drop table if exists tt; create table tt (a int, b int) distributed randomly;`
`explain (analyze,verbose) insert into tt select * from generate_series(1,1000)a,generate_series(1,1000)b;`
`explain (analyze,verbose) select * from tt where a > b;`
```
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (actual time=33.612..78.159 rows=1000 loops=1)
Output: a, b
Node Summary: vmax vsum vcnt imax
ntuples: 1000 1000 1 -1
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 44592 44592 1 -1
totalPartTableScanned: 0 0 0 0
segindex0: -1
ninst: 1
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg-1) 241 0.00 0.03 0.08 1000.00 1.00 0 0 0 0 0 0 44592.00 0 0 0 0 0 0 0
-> Seq Scan on public.tt (cost=0.00..431.00 rows=1 width=8) (actual time=2.512..77.640 rows=353 loops=1)
Output: a, b
Filter: (tt.a = 100)
Node Summary: vmax vsum vcnt imax
ntuples: 353 1000 3 0
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 19072 57216 3 0
totalPartTableScanned: 0 0 0 0
segindex0: 0
ninst: 3
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg0) 211 0.00 0.00 0.08 353.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg1) 211 0.00 0.00 0.03 330.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg2) 211 0.00 0.00 0.05 317.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
Planning time: 4.514 ms
(slice0) Executor memory: 119K bytes.
(slice1) Executor memory: 27K bytes avg x 3 workers, 27K bytes max (seg0).
Memory used: 128000kB
Optimizer: Pivotal Optimizer (GPORCA)
Execution time: 78.584 ms
(39 rows)
```
robozmey
added a commit
to robozmey/cloudberrydb
that referenced
this pull request
Jun 11, 2025
Dumps CdbExplain_NodeSummary struct in EXPLAIN ANALYZE
`set gp_enable_explain_node_summary=on;`
`drop table if exists tt; create table tt (a int, b int) distributed randomly;`
`explain (analyze,verbose) insert into tt select * from generate_series(1,1000)a,generate_series(1,1000)b;`
`explain (analyze,verbose) select * from tt where a > b;`
```
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (actual time=33.612..78.159 rows=1000 loops=1)
Output: a, b
Node Summary: vmax vsum vcnt imax
ntuples: 1000 1000 1 -1
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 44592 44592 1 -1
totalPartTableScanned: 0 0 0 0
segindex0: -1
ninst: 1
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg-1) 241 0.00 0.03 0.08 1000.00 1.00 0 0 0 0 0 0 44592.00 0 0 0 0 0 0 0
-> Seq Scan on public.tt (cost=0.00..431.00 rows=1 width=8) (actual time=2.512..77.640 rows=353 loops=1)
Output: a, b
Filter: (tt.a = 100)
Node Summary: vmax vsum vcnt imax
ntuples: 353 1000 3 0
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 19072 57216 3 0
totalPartTableScanned: 0 0 0 0
segindex0: 0
ninst: 3
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg0) 211 0.00 0.00 0.08 353.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg1) 211 0.00 0.00 0.03 330.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg2) 211 0.00 0.00 0.05 317.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
Planning time: 4.514 ms
(slice0) Executor memory: 119K bytes.
(slice1) Executor memory: 27K bytes avg x 3 workers, 27K bytes max (seg0).
Memory used: 128000kB
Optimizer: Pivotal Optimizer (GPORCA)
Execution time: 78.584 ms
(39 rows)
```
robozmey
added a commit
to robozmey/cloudberrydb
that referenced
this pull request
Jul 16, 2025
Dumps CdbExplain_NodeSummary struct in EXPLAIN ANALYZE
`set gp_enable_explain_node_summary=on;`
`drop table if exists tt; create table tt (a int, b int) distributed randomly;`
`explain (analyze,verbose) insert into tt select * from generate_series(1,1000)a,generate_series(1,1000)b;`
`explain (analyze,verbose) select * from tt where a > b;`
```
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (actual time=33.612..78.159 rows=1000 loops=1)
Output: a, b
Node Summary: vmax vsum vcnt imax
ntuples: 1000 1000 1 -1
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 44592 44592 1 -1
totalPartTableScanned: 0 0 0 0
segindex0: -1
ninst: 1
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg-1) 241 0.00 0.03 0.08 1000.00 1.00 0 0 0 0 0 0 44592.00 0 0 0 0 0 0 0
-> Seq Scan on public.tt (cost=0.00..431.00 rows=1 width=8) (actual time=2.512..77.640 rows=353 loops=1)
Output: a, b
Filter: (tt.a = 100)
Node Summary: vmax vsum vcnt imax
ntuples: 353 1000 3 0
execmemused: 0 0 0 0
workmemused: 0 0 0 0
workmemwanted: 0 0 0 0
totalWorkfileCreated: 0 0 0 0
peakMemBalance: 19072 57216 3 0
totalPartTableScanned: 0 0 0 0
segindex0: 0
ninst: 3
StatInsts:
(segN) pstype starttime counter firsttuple startup total ntuples nloops execmemused workmemused workmemwanted workfileCreated firststart peakMemBalance numPartScanned sortMethod sortSpaceType sortSpaceUsed bnotes enotes
(seg0) 211 0.00 0.00 0.08 353.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg1) 211 0.00 0.00 0.03 330.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
(seg2) 211 0.00 0.00 0.05 317.00 1.00 0 0 0 0 0 0 19072.00 0 0 0 0 0 0 0
Planning time: 4.514 ms
(slice0) Executor memory: 119K bytes.
(slice1) Executor memory: 27K bytes avg x 3 workers, 27K bytes max (seg0).
Memory used: 128000kB
Optimizer: Pivotal Optimizer (GPORCA)
Execution time: 78.584 ms
(39 rows)
```
12 tasks
reshke
added a commit
that referenced
this pull request
Sep 19, 2025
leborchuk
added a commit
to leborchuk/cloudberry
that referenced
this pull request
Sep 2, 2026
Upstream PostgreSQL commit "Remove configure probe for sockaddr_in6 and require AF_INET6." (bcc8b14) deleted the HAVE_IPV6 probe and stripped the #ifdef HAVE_IPV6 guards from its own code, since AF_INET6 is now always available. Cloudberry-specific code still gated IPv6 handling behind #ifdef HAVE_IPV6, so after the PG16 merge those blocks became dead code: HAVE_IPV6 is never defined on non-Windows builds. The practical effect: on an IPv6-only cluster, getDnsCachedAddress() never populates its cache entry (the IPv6 branch was compiled out), then returns e->hostinfo with e == NULL -- a bogus non-NULL pointer (offsetof key[]) -- which the caller passes to pstrdup(), crashing in strlen(). This shows up as a coordinator/FtsProbe SIGSEGV: #0 __strlen_evex #1 MemoryContextStrdup #2 getCdbComponentInfo apache#3 cdbcomponent_getCdbComponents apache#4 FtsProbeMain Remove the leftover #ifdef HAVE_IPV6 guards so the IPv6 paths compile unconditionally, matching what upstream did to its own files. Also guard the cache return against a NULL entry so an unresolvable segment logs a clean "cannot resolve network address" error instead of segfaulting. Files: cdbutil.c (both getDnsCachedAddress copies), auth.c, and the interconnect listener setup (ic_common.c, ic_tcp.c, ic_udpifc.c). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leborchuk
added a commit
that referenced
this pull request
Sep 11, 2026
Upstream PostgreSQL commit "Remove configure probe for sockaddr_in6 and require AF_INET6." (bcc8b14) deleted the HAVE_IPV6 probe and stripped the #ifdef HAVE_IPV6 guards from its own code, since AF_INET6 is now always available. Cloudberry-specific code still gated IPv6 handling behind #ifdef HAVE_IPV6, so after the PG16 merge those blocks became dead code: HAVE_IPV6 is never defined on non-Windows builds. The practical effect: on an IPv6-only cluster, getDnsCachedAddress() never populates its cache entry (the IPv6 branch was compiled out), then returns e->hostinfo with e == NULL -- a bogus non-NULL pointer (offsetof key[]) -- which the caller passes to pstrdup(), crashing in strlen(). This shows up as a coordinator/FtsProbe SIGSEGV: #0 __strlen_evex #1 MemoryContextStrdup #2 getCdbComponentInfo #3 cdbcomponent_getCdbComponents #4 FtsProbeMain Remove the leftover #ifdef HAVE_IPV6 guards so the IPv6 paths compile unconditionally, matching what upstream did to its own files. Also guard the cache return against a NULL entry so an unresolvable segment logs a clean "cannot resolve network address" error instead of segfaulting. Files: cdbutil.c (both getDnsCachedAddress copies), auth.c, and the interconnect listener setup (ic_common.c, ic_tcp.c, ic_udpifc.c). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leborchuk
pushed a commit
that referenced
this pull request
Sep 29, 2026
checkExtensionMembership() set the DUMP_COMPONENT_SECLABEL and DUMP_COMPONENT_POLICY flags for extension member objects, even though we lack any infrastructure for tracking extensions' initial settings of these properties. This is not OK. The result was that a dump would always include commands to set these properties for extension objects that have them, with at least three negative consequences: 1. The restoring user might not have privilege to set these properties on these objects. 2. The properties might be incorrect/irrelevant for the version of the extension that's installed in the destination database. 3. The dump itself might fail, in the case of RLS properties attached to extension tables that the dumping user lacks privilege to LOCK. (That's because we must get at least AccessShareLock to ensure that we don't fail while trying to decompile the RLS expressions.) When and if somebody cares to invent initial-state infrastructure for extensions' RLS policies and security labels, we could think about finding another way around problem #3. But in the absence of such infrastructure, this whole thing is just wrong and we shouldn't do it. (Note: this applies only to ordinary dumps; binary-upgrade dumps still dump and restore extension member objects separately, with all properties.) Tom Lane and Jacob Champion. Back-patch to all supported branches. Discussion: https://postgr.es/m/00d46a48-3324-d9a0-49bf-e7f0f11d1038@timescale.com (cherry picked from commit 15439205d8cec6fff71e77f0fdd935129120edd0)
1 of 2 tasks
13 tasks
13 tasks
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Oct 27, 2026
(cherry picked from commit c0730c2f0c58af4a7b12d626c21596e8b4f35864) Sampling a CO table walks every projected column to the target row using one shared logical counter, which assumes each column has a datum for every row. Missing-mode ADD COLUMN breaks that: the new column's file starts after its lastrownum, so the walk overshoots and runs off the end of the segfile. Take the row number the anchor column produced and fetch the other columns at that same physical row instead of recomputing a target per column. This is the hazard docs/gp7-avoid-table-rewrite-commit-correlation.md flagged as the easiest one to miss, and it is real: ANALYZE on an AOCO table whose added column is *partially* present crashes the segment. CREATE TABLE t(a int, b int) USING ao_column; -- 200000 rows ALTER TABLE t ADD COLUMN c int DEFAULT 7; INSERT INTO t SELECT i, i%100, i FROM generate_series(200001,210000) i; ANALYZE t; -- QE dies The core shows exactly the coordinate mismatch, at the pg_unreachable() in aocs_gettuple(): apache#2 aocs_gettuple (targrow=69896) at aocsam.c:1339 apache#3 aocs_get_target_tuple apache#4 aoco_acquire_sample_rows (gdb) p startrow -> 69893 (gdb) p phyrow -> 69944 (gdb) p scan->columnScanInfo.ds[2]->blockFirstRowNum -> 66701 (gdb) p attnum_to_rownum[2 * 128 + segno] -> 66700 startrow/targrow are logical positions driven by the anchor, phyrow is a physical row number, and column c's file only begins at row 66701 -- so walking c by the logical counter looks for a row 66700 rows past its end. The case upstream's own test covers -- a column added and never written to afterwards -- does not crash, because then every row is missing and the check added by 3986f92616 returns the missing value before any walk happens. It takes a *partially* present column to reach the walk. Ported to the PG16 kernel: - aocs_gettuple_column() keeps cbdb's AOTupleId-by-value local (that difference predates this commit); only upstream's rowstoprocess initialisation is taken, which the new branch needs. - the two restructured call sites in aocs_gettuple() take upstream's shape, with cbdb's "goto out" restored -- it has to update the scan's tracking variables before returning. - pgstat_count_buffer_read_ao() was dropped from all three places the merge reintroduced it; it belongs to 67da2c369b, which cbdb does not carry. The test additions needed one cbdb-specific correction: gp_acquire_sample_rows() has NUM_SAMPLE_FIXED_COLS = 4 here, not 3 -- cbdb returns an extra stadistinct_array column -- so every "as (totalrows, totaldeadrows, oversized_cols_length, ...)" list in the file gains it. Without that even a plain three-column ao_row table fails with "Returned row contains 7 attributes, but query expects 6". The output template was rebuilt from an ao_row run and verified against both the _row and _column variants. Verified: AO/AOCS regression 24/24, AO isolation2 136/136 plus ao_partial_scan. ANALYZE on the reproducer above now reports reltuples = 210000 exactly, and pg_stats for the text column added the same way has n_distinct = 2, matching the data.
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Oct 27, 2026
(cherry picked from commit c0730c2f0c58af4a7b12d626c21596e8b4f35864) Sampling a CO table walks every projected column to the target row using one shared logical counter, which assumes each column has a datum for every row. Missing-mode ADD COLUMN breaks that: the new column's file starts after its lastrownum, so the walk overshoots and runs off the end of the segfile. Take the row number the anchor column produced and fetch the other columns at that same physical row instead of recomputing a target per column. This is the hazard docs/gp7-avoid-table-rewrite-commit-correlation.md flagged as the easiest one to miss, and it is real: ANALYZE on an AOCO table whose added column is *partially* present crashes the segment. CREATE TABLE t(a int, b int) USING ao_column; -- 200000 rows ALTER TABLE t ADD COLUMN c int DEFAULT 7; INSERT INTO t SELECT i, i%100, i FROM generate_series(200001,210000) i; ANALYZE t; -- QE dies The core shows exactly the coordinate mismatch, at the pg_unreachable() in aocs_gettuple(): apache#2 aocs_gettuple (targrow=69896) at aocsam.c:1339 apache#3 aocs_get_target_tuple apache#4 aoco_acquire_sample_rows (gdb) p startrow -> 69893 (gdb) p phyrow -> 69944 (gdb) p scan->columnScanInfo.ds[2]->blockFirstRowNum -> 66701 (gdb) p attnum_to_rownum[2 * 128 + segno] -> 66700 startrow/targrow are logical positions driven by the anchor, phyrow is a physical row number, and column c's file only begins at row 66701 -- so walking c by the logical counter looks for a row 66700 rows past its end. The case upstream's own test covers -- a column added and never written to afterwards -- does not crash, because then every row is missing and the check added by 3986f92616 returns the missing value before any walk happens. It takes a *partially* present column to reach the walk. Ported to the PG16 kernel: - aocs_gettuple_column() keeps cbdb's AOTupleId-by-value local (that difference predates this commit); only upstream's rowstoprocess initialisation is taken, which the new branch needs. - the two restructured call sites in aocs_gettuple() take upstream's shape, with cbdb's "goto out" restored -- it has to update the scan's tracking variables before returning. - pgstat_count_buffer_read_ao() was dropped from all three places the merge reintroduced it; it belongs to 67da2c369b, which cbdb does not carry. The test additions needed one cbdb-specific correction: gp_acquire_sample_rows() has NUM_SAMPLE_FIXED_COLS = 4 here, not 3 -- cbdb returns an extra stadistinct_array column -- so every "as (totalrows, totaldeadrows, oversized_cols_length, ...)" list in the file gains it. Without that even a plain three-column ao_row table fails with "Returned row contains 7 attributes, but query expects 6". The output template was rebuilt from an ao_row run and verified against both the _row and _column variants. Verified: AO/AOCS regression 24/24, AO isolation2 136/136 plus ao_partial_scan. ANALYZE on the reproducer above now reports reltuples = 210000 exactly, and pg_stats for the text column added the same way has n_distinct = 2, matching the data.
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Oct 27, 2026
(cherry picked from commit c0730c2f0c58af4a7b12d626c21596e8b4f35864) Sampling a CO table walks every projected column to the target row using one shared logical counter, which assumes each column has a datum for every row. Missing-mode ADD COLUMN breaks that: the new column's file starts after its lastrownum, so the walk overshoots and runs off the end of the segfile. Take the row number the anchor column produced and fetch the other columns at that same physical row instead of recomputing a target per column. This is the hazard docs/gp7-avoid-table-rewrite-commit-correlation.md flagged as the easiest one to miss, and it is real: ANALYZE on an AOCO table whose added column is *partially* present crashes the segment. CREATE TABLE t(a int, b int) USING ao_column; -- 200000 rows ALTER TABLE t ADD COLUMN c int DEFAULT 7; INSERT INTO t SELECT i, i%100, i FROM generate_series(200001,210000) i; ANALYZE t; -- QE dies The core shows exactly the coordinate mismatch, at the pg_unreachable() in aocs_gettuple(): apache#2 aocs_gettuple (targrow=69896) at aocsam.c:1339 apache#3 aocs_get_target_tuple apache#4 aoco_acquire_sample_rows (gdb) p startrow -> 69893 (gdb) p phyrow -> 69944 (gdb) p scan->columnScanInfo.ds[2]->blockFirstRowNum -> 66701 (gdb) p attnum_to_rownum[2 * 128 + segno] -> 66700 startrow/targrow are logical positions driven by the anchor, phyrow is a physical row number, and column c's file only begins at row 66701 -- so walking c by the logical counter looks for a row 66700 rows past its end. The case upstream's own test covers -- a column added and never written to afterwards -- does not crash, because then every row is missing and the check added by 3986f92616 returns the missing value before any walk happens. It takes a *partially* present column to reach the walk. Ported to the PG16 kernel: - aocs_gettuple_column() keeps cbdb's AOTupleId-by-value local (that difference predates this commit); only upstream's rowstoprocess initialisation is taken, which the new branch needs. - the two restructured call sites in aocs_gettuple() take upstream's shape, with cbdb's "goto out" restored -- it has to update the scan's tracking variables before returning. - pgstat_count_buffer_read_ao() was dropped from all three places the merge reintroduced it; it belongs to 67da2c369b, which cbdb does not carry. The test additions needed one cbdb-specific correction: gp_acquire_sample_rows() has NUM_SAMPLE_FIXED_COLS = 4 here, not 3 -- cbdb returns an extra stadistinct_array column -- so every "as (totalrows, totaldeadrows, oversized_cols_length, ...)" list in the file gains it. Without that even a plain three-column ao_row table fails with "Returned row contains 7 attributes, but query expects 6". The output template was rebuilt from an ao_row run and verified against both the _row and _column variants. Verified: AO/AOCS regression 24/24, AO isolation2 136/136 plus ao_partial_scan. ANALYZE on the reproducer above now reports reltuples = 210000 exactly, and pg_stats for the text column added the same way has n_distinct = 2, matching the data.
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Oct 27, 2026
(cherry picked from commit c0730c2f0c58af4a7b12d626c21596e8b4f35864) Sampling a CO table walks every projected column to the target row using one shared logical counter, which assumes each column has a datum for every row. Missing-mode ADD COLUMN breaks that: the new column's file starts after its lastrownum, so the walk overshoots and runs off the end of the segfile. Take the row number the anchor column produced and fetch the other columns at that same physical row instead of recomputing a target per column. This is the hazard docs/gp7-avoid-table-rewrite-commit-correlation.md flagged as the easiest one to miss, and it is real: ANALYZE on an AOCO table whose added column is *partially* present crashes the segment. CREATE TABLE t(a int, b int) USING ao_column; -- 200000 rows ALTER TABLE t ADD COLUMN c int DEFAULT 7; INSERT INTO t SELECT i, i%100, i FROM generate_series(200001,210000) i; ANALYZE t; -- QE dies The core shows exactly the coordinate mismatch, at the pg_unreachable() in aocs_gettuple(): apache#2 aocs_gettuple (targrow=69896) at aocsam.c:1339 apache#3 aocs_get_target_tuple apache#4 aoco_acquire_sample_rows (gdb) p startrow -> 69893 (gdb) p phyrow -> 69944 (gdb) p scan->columnScanInfo.ds[2]->blockFirstRowNum -> 66701 (gdb) p attnum_to_rownum[2 * 128 + segno] -> 66700 startrow/targrow are logical positions driven by the anchor, phyrow is a physical row number, and column c's file only begins at row 66701 -- so walking c by the logical counter looks for a row 66700 rows past its end. The case upstream's own test covers -- a column added and never written to afterwards -- does not crash, because then every row is missing and the check added by 3986f92616 returns the missing value before any walk happens. It takes a *partially* present column to reach the walk. Ported to the PG16 kernel: - aocs_gettuple_column() keeps cbdb's AOTupleId-by-value local (that difference predates this commit); only upstream's rowstoprocess initialisation is taken, which the new branch needs. - the two restructured call sites in aocs_gettuple() take upstream's shape, with cbdb's "goto out" restored -- it has to update the scan's tracking variables before returning. - pgstat_count_buffer_read_ao() was dropped from all three places the merge reintroduced it; it belongs to 67da2c369b, which cbdb does not carry. The test additions needed one cbdb-specific correction: gp_acquire_sample_rows() has NUM_SAMPLE_FIXED_COLS = 4 here, not 3 -- cbdb returns an extra stadistinct_array column -- so every "as (totalrows, totaldeadrows, oversized_cols_length, ...)" list in the file gains it. Without that even a plain three-column ao_row table fails with "Returned row contains 7 attributes, but query expects 6". The output template was rebuilt from an ao_row run and verified against both the _row and _column variants. Verified: AO/AOCS regression 24/24, AO isolation2 136/136 plus ao_partial_scan. ANALYZE on the reproducer above now reports reltuples = 210000 exactly, and pg_stats for the text column added the same way has n_distinct = 2, matching the data.
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Oct 27, 2026
(cherry picked from commit c0730c2f0c58af4a7b12d626c21596e8b4f35864) Sampling a CO table walks every projected column to the target row using one shared logical counter, which assumes each column has a datum for every row. Missing-mode ADD COLUMN breaks that: the new column's file starts after its lastrownum, so the walk overshoots and runs off the end of the segfile. Take the row number the anchor column produced and fetch the other columns at that same physical row instead of recomputing a target per column. This is the hazard docs/gp7-avoid-table-rewrite-commit-correlation.md flagged as the easiest one to miss, and it is real: ANALYZE on an AOCO table whose added column is *partially* present crashes the segment. CREATE TABLE t(a int, b int) USING ao_column; -- 200000 rows ALTER TABLE t ADD COLUMN c int DEFAULT 7; INSERT INTO t SELECT i, i%100, i FROM generate_series(200001,210000) i; ANALYZE t; -- QE dies The core shows exactly the coordinate mismatch, at the pg_unreachable() in aocs_gettuple(): apache#2 aocs_gettuple (targrow=69896) at aocsam.c:1339 apache#3 aocs_get_target_tuple apache#4 aoco_acquire_sample_rows (gdb) p startrow -> 69893 (gdb) p phyrow -> 69944 (gdb) p scan->columnScanInfo.ds[2]->blockFirstRowNum -> 66701 (gdb) p attnum_to_rownum[2 * 128 + segno] -> 66700 startrow/targrow are logical positions driven by the anchor, phyrow is a physical row number, and column c's file only begins at row 66701 -- so walking c by the logical counter looks for a row 66700 rows past its end. The case upstream's own test covers -- a column added and never written to afterwards -- does not crash, because then every row is missing and the check added by 3986f92616 returns the missing value before any walk happens. It takes a *partially* present column to reach the walk. Ported to the PG16 kernel: - aocs_gettuple_column() keeps cbdb's AOTupleId-by-value local (that difference predates this commit); only upstream's rowstoprocess initialisation is taken, which the new branch needs. - the two restructured call sites in aocs_gettuple() take upstream's shape, with cbdb's "goto out" restored -- it has to update the scan's tracking variables before returning. - pgstat_count_buffer_read_ao() was dropped from all three places the merge reintroduced it; it belongs to 67da2c369b, which cbdb does not carry. The test additions needed one cbdb-specific correction: gp_acquire_sample_rows() has NUM_SAMPLE_FIXED_COLS = 4 here, not 3 -- cbdb returns an extra stadistinct_array column -- so every "as (totalrows, totaldeadrows, oversized_cols_length, ...)" list in the file gains it. Without that even a plain three-column ao_row table fails with "Returned row contains 7 attributes, but query expects 6". The output template was rebuilt from an ao_row run and verified against both the _row and _column variants. Verified: AO/AOCS regression 24/24, AO isolation2 136/136 plus ao_partial_scan. ANALYZE on the reproducer above now reports reltuples = 210000 exactly, and pg_stats for the text column added the same way has n_distinct = 2, matching the data.
tuhaihe
pushed a commit
that referenced
this pull request
Oct 28, 2026
UBSan reported an unaligned uint16 load while parsing tuple chunks. Copy the length into an aligned local with memcpy before adding the header size, preserving the existing native-endian interpretation. Sanitizer report (UBSan, instrumented build, distributed query with Motion traffic over UDP): ic_udpifc.c:4572:39: runtime error: load of misaligned address 0x52500016fa59 for type 'uint16', which requires 2 byte alignment #0 RecvTupleChunkUDPIFC ic_udpifc.c:4572 #1 receiveChunksUDPIFCLoop ic_udpifc.c:4462 #2 receiveChunksUDPIFC ic_udpifc.c:4406 #3 RecvTupleChunkFromAnyUDPIFC ic_udpifc.c:4739 #4 processIncomingChunks cdb/cdbmotion.c:653 #5 RecvTupleFrom cdbmotion.c:607 #6 execMotionUnsortedReceiver executor/nodeMotion.c:356 Reached whenever bytesProcessed walks the receive buffer past an even offset while splitting a message into tuple chunks. Benign on x86_64, but undefined behavior on strictly aligned targets.
reshke
added a commit
that referenced
this pull request
Oct 28, 2026
UBSan reported an unaligned uint64_t load in GpMurmurHash64A. Read each word with memcpy so input need not have uint64_t alignment, while preserving the little-endian hash computation. Sanitizer report (UBSan, instrumented build, ANALYZE on a table with short text values): gp_hyperloglog.c:897:11: runtime error: load of misaligned address 0x529000093515 for type 'uint64_t', which requires 8 byte alignment #0 GpMurmurHash64A gp_hyperloglog.c:897 #1 gp_hll_add_element gp_hyperloglog.c:471 #2 gp_hyperloglog_add_item gp_hyperloglog.c:788 #3 compute_scalar_stats commands/analyze.c:3968 #4 do_analyze_rel commands/analyze.c:971 #5 analyze_rel_internal commands/analyze.c:429 Triggered by ordinary ndistinct estimation during ANALYZE: short varlena values hashed via VARDATA_ANY have a payload offset that is not a multiple of 8. Benign on x86_64, but undefined behavior on strictly aligned targets.
yjhjstz
pushed a commit
to yjhjstz/cloudberry
that referenced
this pull request
Oct 28, 2026
(cherry picked from commit c0730c2f0c58af4a7b12d626c21596e8b4f35864) Sampling a CO table walks every projected column to the target row using one shared logical counter, which assumes each column has a datum for every row. Missing-mode ADD COLUMN breaks that: the new column's file starts after its lastrownum, so the walk overshoots and runs off the end of the segfile. Take the row number the anchor column produced and fetch the other columns at that same physical row instead of recomputing a target per column. This is the hazard docs/gp7-avoid-table-rewrite-commit-correlation.md flagged as the easiest one to miss, and it is real: ANALYZE on an AOCO table whose added column is *partially* present crashes the segment. CREATE TABLE t(a int, b int) USING ao_column; -- 200000 rows ALTER TABLE t ADD COLUMN c int DEFAULT 7; INSERT INTO t SELECT i, i%100, i FROM generate_series(200001,210000) i; ANALYZE t; -- QE dies The core shows exactly the coordinate mismatch, at the pg_unreachable() in aocs_gettuple(): apache#2 aocs_gettuple (targrow=69896) at aocsam.c:1339 apache#3 aocs_get_target_tuple apache#4 aoco_acquire_sample_rows (gdb) p startrow -> 69893 (gdb) p phyrow -> 69944 (gdb) p scan->columnScanInfo.ds[2]->blockFirstRowNum -> 66701 (gdb) p attnum_to_rownum[2 * 128 + segno] -> 66700 startrow/targrow are logical positions driven by the anchor, phyrow is a physical row number, and column c's file only begins at row 66701 -- so walking c by the logical counter looks for a row 66700 rows past its end. The case upstream's own test covers -- a column added and never written to afterwards -- does not crash, because then every row is missing and the check added by 3986f92616 returns the missing value before any walk happens. It takes a *partially* present column to reach the walk. Ported to the PG16 kernel: - aocs_gettuple_column() keeps cbdb's AOTupleId-by-value local (that difference predates this commit); only upstream's rowstoprocess initialisation is taken, which the new branch needs. - the two restructured call sites in aocs_gettuple() take upstream's shape, with cbdb's "goto out" restored -- it has to update the scan's tracking variables before returning. - pgstat_count_buffer_read_ao() was dropped from all three places the merge reintroduced it; it belongs to 67da2c369b, which cbdb does not carry. The test additions needed one cbdb-specific correction: gp_acquire_sample_rows() has NUM_SAMPLE_FIXED_COLS = 4 here, not 3 -- cbdb returns an extra stadistinct_array column -- so every "as (totalrows, totaldeadrows, oversized_cols_length, ...)" list in the file gains it. Without that even a plain three-column ao_row table fails with "Returned row contains 7 attributes, but query expects 6". The output template was rebuilt from an ao_row run and verified against both the _row and _column variants. Verified: AO/AOCS regression 24/24, AO isolation2 136/136 plus ao_partial_scan. ANALYZE on the reproducer above now reports reltuples = 210000 exactly, and pg_stats for the text column added the same way has n_distinct = 2, matching the data.
yjhjstz
pushed a commit
that referenced
this pull request
Oct 31, 2026
UBSan reported a misaligned member access when decoding the distributed-transaction field of commit WAL records during crash-recovery redo. The WAL record payload has no alignment guarantee; the neighboring xl_origin branch already copies onto the stack for exactly this reason. Do the same for xl_xact_distrib. Sanitizer report (UBSan, instrumented build, crash recovery): xactdesc.c:155:34: runtime error: member access within misaligned address 0x7b39d104de44 for type 'struct xl_xact_distrib', which requires 8 byte alignment #0 ParseCommitRecord xactdesc.c:155 #1 xact_redo xactdesc.c:... #2 ApplyWalRecord xlogrecovery.c:2023 #3 PerformWalRecovery xlogrecovery.c:1845 Undefined behavior at recovery time; harmless on x86_64 but incorrect on strictly aligned targets.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For a better user and developer experience, update the Issue and PR templates.
Files changed: