[Waloid24]

Overview

cdbexplain_depositStatsToNode() receives execution statistics from the executors and uses them to populate the dispatcher’s Instrumentation object.

For each plan node, the function works with four related objects:

• rsh is the CdbExplain_StatHdr for the current executor statistics message. It contains the segment index, the array of per-node statistics, worker-level statistics, and extra-text metadata.
• rsi points to the CdbExplain_StatInst for the current plan node inside rsh:

  rsi = &rsh->inst[ctx->iStatInst];

It identifies the statistics of one specific executor or parallel worker. The received message remains valid while the statistics are being deposited, so rsi is a stable pointer to that worker’s record.

• ns is the CdbExplain_NodeSummary allocated for the current plan node and attached to the dispatcher’s Instrumentation. It contains aggregate statistics and an insts array with one slot per segment.
• nsi points to the entry in ns->insts corresponding to rsh->segindex:

  nsi = &ns->insts[rsh->segindex - ns->segindex0];

Unlike rsi, nsi identifies a segment slot, not a particular worker.

The function iterates over all received executor statistics messages:

for (imsgptr = 0; imsgptr < ctx->nmsgptr; imsgptr++)
{
    rsh = ctx->msgptrs[imsgptr];
    rsi = &rsh->inst[ctx->iStatInst];

    nsi = &ns->insts[rsh->segindex - ns->segindex0];
    *nsi = *rsi;

    ...
}

For non-parallel execution, there is normally one statistics record per segment, so copying rsi into the segment’s nsi slot is unambiguous.

With parallel execution, several workers can run on the same segment. All of their messages have the same rsh->segindex, so they resolve to the same nsi address:

segment 0 / worker 0 ──┐
                       ├──> ns->insts[0]
segment 0 / worker 1 ──┘

Each *nsi = *rsi therefore overwrites the statistics copied from the previous worker on that segment.

At the same time, the function maintains several accumulators. When an accumulator finds a new maximum, it records both:

• rsimax: the stable received CdbExplain_StatInst of the winning worker.
• nsimax: the per-segment ns->insts slot into which that worker was copied.

The problem is that rsimax continues to identify the selected worker, while nsimax can subsequently be overwritten by another worker from the same segment.

After all messages have been processed, the old code copied the winner’s instrumentation from nsimax:

instr->ntuples = nodeAcc->nsimax->ntuples;
instr->total = nodeAcc->nsimax->total;
...

Consequently, the accumulator could select one worker as the winner, but the displayed rows, loops, timing, filtering, and memory values could come from a later worker that reused the same segment slot.

In other words, nsimax identifies the winner’s segment slot, but it does not reliably identify the winner itself.

Reproduction

The issue can be reproduced with multiple parallel workers per segment:

CREATE TABLE explain_parallel_worker_stats(id int)
WITH (parallel_workers=2) DISTRIBUTED RANDOMLY;

INSERT INTO explain_parallel_worker_stats
SELECT generate_series(1, 100000);

SET min_parallel_table_scan_size = 0;
SET parallel_setup_cost = 0;
SET parallel_tuple_cost = 0;
SET max_parallel_workers_per_gather = 2;
SET enable_parallel = on;
SET gp_enable_explain_allstat = on;

EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, SUMMARY OFF)
SELECT * FROM explain_parallel_worker_stats;
                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Gather Motion 4:1  (slice1; segments: 4) (actual time=1.183..16.288 rows=100000 loops=1)
   ->  Parallel Seq Scan on explain_parallel_worker_stats (actual time=0.049..2.645 rows=25452 loops=1)
         allstat: seg_firststart_total_ntuples/seg0_1.091 ms_2.640 ms_25429/seg1_1.446 ms_2.645 ms_25452//end
 Planning:
   Buffers: shared hit=39 read=1
 Optimizer: GPORCA
(6 rows)

EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, SUMMARY OFF)
SELECT * FROM explain_parallel_worker_stats;
                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Gather Motion 4:1  (slice1; segments: 4) (actual time=0.961..16.745 rows=100000 loops=1)
   ->  Parallel Seq Scan on explain_parallel_worker_stats (actual time=0.025..2.696 rows=24576 loops=1)
         allstat: seg_firststart_total_ntuples/seg0_0.933 ms_2.674 ms_25429/seg1_1.200 ms_2.696 ms_24576//end
 Planning:
   Buffers: shared hit=5
 Optimizer: GPORCA
(6 rows)

The same you can observe if you use PostgreSQL planner:

EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, SUMMARY OFF)
SELECT * FROM explain_parallel_worker_stats;
                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Gather Motion 4:1  (slice1; segments: 4) (actual time=1.907..18.237 rows=100000 loops=1)
   ->  Parallel Seq Scan on explain_parallel_worker_stats (actual time=0.019..2.481 rows=25452 loops=1)
         allstat: seg_firststart_total_ntuples/seg0_1.998 ms_2.342 ms_24543/seg1_1.951 ms_2.481 ms_25452//end
 Planning:
   Buffers: shared hit=97 read=3
 Optimizer: Postgres query optimizer
(6 rows)

EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, SUMMARY OFF)
SELECT * FROM explain_parallel_worker_stats;
                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Gather Motion 4:1  (slice1; segments: 4) (actual time=1.029..17.477 rows=100000 loops=1)
   ->  Parallel Seq Scan on explain_parallel_worker_stats (actual time=0.025..2.478 rows=24576 loops=1)
         allstat: seg_firststart_total_ntuples/seg0_1.077 ms_2.549 ms_25429/seg1_2.009 ms_2.478 ms_24576//end
 Planning:
   Buffers: shared hit=22
 Optimizer: Postgres query optimizer
(6 rows)

What does this PR do?

Copy the winner's instrumentation from its stable received statistics message (rsi) rather than from the reusable per-segment summary slot (nsi).

Type of Change

• Bug fix (non-breaking change)

Test Plan

• Unit tests added/updated
• Integration tests added/updated
• Passed make installcheck
• Passed make -C src/test installcheck-cbdb-parallel

Impact

EXPLAIN outputs the maximum value across workers.

Checklist

• Followed contribution guide
• Added/updated documentation
• Reviewed code for security implications
• This PR contains AI-assisted code generation
• Requested review from cloudberry committers

Additional Context

CI Skip Instructions

---

[Waloid24]

There is a bug in gp_enable_explain_allstat output too. It writes that seg0 and seg1 have approximately 25k rows while actually they have 50k. Again, it happens because it uses 1 slot per segment which is reused across parallel workers. I'll fix it in the follow up commit.

I suggest to do it in a separate PR because it's logically independent fix.

[Alena0704]

The winner-statistics fix looks correct, but allstat still loses parallel-worker statistics: four QEs execute the scan, yet only two records appear. Workers on the same segment overwrite the same ns->insts slot, leaving only the last worker’s statistics for each segment.

reproduction on a cluster with two primary segments:

SET optimizer = on;
SET enable_parallel = on;
SET min_parallel_table_scan_size = 0;
SET parallel_setup_cost = 0;
SET parallel_tuple_cost = 0;
SET max_parallel_workers_per_gather = 2;
SET gp_enable_explain_allstat = on;

CREATE TABLE explain_parallel_worker_stats(id int)
WITH (parallel_workers = 2)
DISTRIBUTED RANDOMLY;

INSERT INTO explain_parallel_worker_stats
SELECT generate_series(1, 100000);

ANALYZE explain_parallel_worker_stats;

EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, SUMMARY OFF)
SELECT * FROM explain_parallel_worker_stats;

                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Gather Motion 4:1  (slice1; segments: 4) (actual time=1.183..16.288 rows=100000 loops=1)
   ->  Parallel Seq Scan on explain_parallel_worker_stats (actual time=0.049..2.645 rows=25452 loops=1)
         allstat: seg_firststart_total_ntuples/seg0_1.091 ms_2.640 ms_25429/seg1_1.446 ms_2.645 ms_25452//end
 Planning:
   Buffers: shared hit=39 read=1
 Optimizer: GPORCA
(6 rows)

[Waloid24]

The winner-statistics fix looks correct, but allstat still loses parallel-worker statistics: four QEs execute the scan, yet only two records appear. Workers on the same segment overwrite the same ns->insts slot, leaving only the last worker’s statistics for each segment.

reproduction on a cluster with two primary segments:

SET optimizer = on;
SET enable_parallel = on;
SET min_parallel_table_scan_size = 0;
SET parallel_setup_cost = 0;
SET parallel_tuple_cost = 0;
SET max_parallel_workers_per_gather = 2;
SET gp_enable_explain_allstat = on;

CREATE TABLE explain_parallel_worker_stats(id int)
WITH (parallel_workers = 2)
DISTRIBUTED RANDOMLY;

INSERT INTO explain_parallel_worker_stats
SELECT generate_series(1, 100000);

ANALYZE explain_parallel_worker_stats;

EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, SUMMARY OFF)
SELECT * FROM explain_parallel_worker_stats;

                                                  QUERY PLAN
--------------------------------------------------------------------------------------------------------------
 Gather Motion 4:1  (slice1; segments: 4) (actual time=1.183..16.288 rows=100000 loops=1)
   ->  Parallel Seq Scan on explain_parallel_worker_stats (actual time=0.049..2.645 rows=25452 loops=1)
         allstat: seg_firststart_total_ntuples/seg0_1.091 ms_2.640 ms_25429/seg1_1.446 ms_2.645 ms_25452//end
 Planning:
   Buffers: shared hit=39 read=1
 Optimizer: GPORCA
(6 rows)

Yes, I mentioned this problem in my comment above.

I suggest to patch it in an independent PR -- look at the comment here #2209 (comment).