Skip to content

Fix: no need to do the qual-clause twice - #396

Merged
my-ship-it merged 1 commit into
apache:mainfrom
jiaqizho:fix-pushdown-qual-clause
Aug 19, 2024
Merged

Fix: no need to do the qual-clause twice#396
my-ship-it merged 1 commit into
apache:mainfrom
jiaqizho:fix-pushdown-qual-clause

Conversation

@jiaqizho

@jiaqizho jiaqizho commented Mar 14, 2024

Copy link
Copy Markdown
Contributor

fix #ISSUE_Number

Change logs

CBDB already support pushdown qual-clause. If current qual-clause have been process in access method, Then ExprState inside PlanState should reset to NULL, In this time, ExecScan should not exec it again.

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Contributor's Checklist

Here are some reminders and checklists before/when submitting your pull request, please check them:

  • Make sure your Pull Request has a clear title and commit message. You can take git-commit template as a reference.
  • Sign the Contributor License Agreement as prompted for your first-time contribution(One-time setup).
  • Learn the coding contribution guide, including our code conventions, workflow and more.
  • List your communication in the GitHub Issues or Discussions (if has or needed).
  • Document changes.
  • Add tests for the change
  • Pass make installcheck
  • Pass make -C src/test installcheck-cbdb-parallel
  • Feel free to request cloudberrydb/dev team for review and approval when your PR is ready🥳

@my-ship-it

my-ship-it commented Mar 14, 2024

Copy link
Copy Markdown
Contributor

This is my plan for heap table:

postgres=# explain analyze verbose select * from int8_tbl where q1 in (select c1 from inner_text);
                                                                  QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3)  (cost=0.00..1.05 rows=2 width=16) (actual time=2.100..2.103 rows=2 loops=1)
   Output: int8_tbl.q1, int8_tbl.q2
   ->  Seq Scan on public.int8_tbl  (cost=0.00..1.02 rows=1 width=16) (actual time=1.257..1.262 rows=2 loops=1)
         Output: int8_tbl.q1, int8_tbl.q2
         Filter: (hashed SubPlan 1)
         SubPlan 1
           ->  Broadcast Motion 3:3  (slice2; segments: 3)  (cost=0.00..150.00 rows=11600 width=32) (actual time=1.031..1.033 rows=2 loops=1)
                 Output: inner_text.c1
                 ->  Seq Scan on public.inner_text  (cost=0.00..150.00 rows=11600 width=32) (actual time=0.098..0.099 rows=1 loops=1)
                       Output: inner_text.c1
 Settings: optimizer = 'off'
 Planning Time: 0.471 ms
   (slice0)    Executor memory: 117K bytes.
   (slice1)    Executor memory: 316K bytes avg x 3x(0) workers, 414K bytes max (seg0).
   (slice2)    Executor memory: 111K bytes avg x 3x(0) workers, 111K bytes max (seg0).
 Memory used:  128000kB
 Optimizer: Postgres query optimizer
 Execution Time: 2.809 ms

@my-ship-it

my-ship-it commented Mar 14, 2024

Copy link
Copy Markdown
Contributor

So, should we handle hash Subplan correctly in Pax?

@jiaqizho jiaqizho changed the title Fix: no need to do the qual-clause twice [DNM]Fix: no need to do the qual-clause twice Mar 14, 2024
gfphoenix78
gfphoenix78 approved these changes Apr 12, 2024
View reviewed changes

@gfphoenix78 gfphoenix78 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiaqizho jiaqizho changed the title [DNM]Fix: no need to do the qual-clause twice Fix: no need to do the qual-clause twice Aug 14, 2024
@jiaqizho
jiaqizho force-pushed the fix-pushdown-qual-clause branch from 8adcd8f to 8d136ac Compare August 16, 2024 03:13
@my-ship-it
my-ship-it merged commit 4b7f9d2 into apache:main Aug 19, 2024
@jiaqizho
jiaqizho deleted the fix-pushdown-qual-clause branch August 20, 2024 03:20
Alena0704 added a commit to Alena0704/cloudberry that referenced this pull request Aug 25, 2026
@Alena0704
Adapt cross-slice Shared Scan subplan fix from open-gpdb PR apache#396
9d51425 
Port of Greengage PR apache#342 / open-gpdb PR apache#396 ("Fix cross-slice
interaction detection for subplans") to Cloudberry.

The code part of the original fix is not needed here. In GPDB 6 the
shareinput walkers processed glob->subplans separately from the main
plan tree, so the motion stack contained only the default motion id (0)
and a Shared Scan consumer inside a SubPlan was never marked as
cross-slice. Cloudberry (GPDB 7 based) already tracks the slice of
every SubPlan reference point: the planner records it in
glob->subplan_sliceIds (build_slice_table_walker dives into SubPlans in
place), ORCA records it in CContextDXLToPlStmt::AddSubplan, and
apply_shareinput_xslice pushes that slice id onto the motion stack
before walking each subplan. The executor-side ExecSliceDependencyNode
change does not apply either: cross-slice Shared Scan synchronization
was rewritten to use shared-memory IPC in nodeShareInputScan.c.
Likewise, glob->subroots stays aligned with glob->subplans when
subplans are duplicated (adjust_appendrel_attrs_mutator), so the
fixup_subplan_walker change has no counterpart. Verified on a demo
cluster: both scenarios from the original patch produce the same plan
shape (Shared Scan consumer under a SubPlan in a different slice than
the producer), are correctly marked cross-slice and return correct
results under both planners.

What is ported is the regression coverage, to guard against future
regressions in this area:

- both test queries from the original patch (planner and ORCA answer
  files), with tables renamed to xslice_t1/xslice_t2 to avoid clashing
  with t1/t2 already created by this test;
- a new fault injection point shareinput_writer_pre_freeze, the
  Cloudberry analog of material_pre_tuplestore_flush from the original
  patch: it delays the producer right before it publishes the
  tuplestore, so a consumer that fails to wait (e.g. because it was not
  marked cross-slice) yields a stable wrong result instead of a flaky
  one.

The pax_storage copy of shared_scan is intentionally left untouched,
consistent with how the previous shared_scan addition was handled.
Alena0704 added a commit to Alena0704/cloudberry that referenced this pull request Aug 25, 2026
@Alena0704
Adapt cross-slice Shared Scan subplan fix from open-gpdb PR apache#396
7e3c590 
Port of Greengage PR apache#342 / open-gpdb PR apache#396 ("Fix cross-slice
interaction detection for subplans") to Cloudberry.

The code part of the original fix is not needed here. In GPDB 6 the
shareinput walkers processed glob->subplans separately from the main
plan tree, so the motion stack contained only the default motion id (0)
and a Shared Scan consumer inside a SubPlan was never marked as
cross-slice. Cloudberry (GPDB 7 based) already tracks the slice of
every SubPlan reference point: the planner records it in
glob->subplan_sliceIds (build_slice_table_walker dives into SubPlans in
place), ORCA records it in CContextDXLToPlStmt::AddSubplan, and
apply_shareinput_xslice pushes that slice id onto the motion stack
before walking each subplan. The executor-side ExecSliceDependencyNode
change does not apply either: cross-slice Shared Scan synchronization
was rewritten to use shared-memory IPC in nodeShareInputScan.c.
Likewise, glob->subroots stays aligned with glob->subplans when
subplans are duplicated (adjust_appendrel_attrs_mutator), so the
fixup_subplan_walker change has no counterpart. Verified on a demo
cluster: both scenarios from the original patch produce the same plan
shape (Shared Scan consumer under a SubPlan in a different slice than
the producer), are correctly marked cross-slice and return correct
results under both planners.

What is ported is the regression coverage, to guard against future
regressions in this area:

- both test queries from the original patch (planner and ORCA answer
  files), with tables renamed to xslice_t1/xslice_t2 to avoid clashing
  with t1/t2 already created by this test;
- a new fault injection point shareinput_writer_pre_freeze, the
  Cloudberry analog of material_pre_tuplestore_flush from the original
  patch: it delays the producer right before it publishes the
  tuplestore, so a consumer that fails to wait (e.g. because it was not
  marked cross-slice) yields a stable wrong result instead of a flaky
  one.

The pax_storage copy of shared_scan is intentionally left untouched,
consistent with how the previous shared_scan addition was handled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yjhjstz yjhjstz yjhjstz approved these changes
@gfphoenix78 gfphoenix78 gfphoenix78 approved these changes
@my-ship-it my-ship-it my-ship-it approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jiaqizho @my-ship-it @yjhjstz @gfphoenix78