Skip to content

CI: run the pg_dump TAP tests - #2243

Open
tuhaihe wants to merge 8 commits into
apache:mainfrom
tuhaihe:ci-pg-dump
Open

tuhaihe wants to merge 8 commits into
apache:mainfrom
tuhaihe:ci-pg-dump

Conversation

@tuhaihe

@tuhaihe tuhaihe commented Nov 4, 2026

Copy link
Copy Markdown
Member

CI does not run the TAP tests in src/bin/pg_dump, so nobody sees when pg_dump, pg_dumpall or pg_restore break.

Add an ic-pg-dump test job that runs them on Rocky 8, 9 and 10.

Assisted-by: Claude Code

Fixes #ISSUE_Number

What does this PR do?

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)
  • Breaking change (fix or feature with breaking changes)
  • Documentation update

Breaking Changes

Test Plan

  • Unit tests added/updated
  • Integration tests added/updated
  • Passed make installcheck
  • Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Additional Context

CI Skip Instructions


CI does not run the TAP tests in src/bin/pg_dump, so nobody sees when
pg_dump, pg_dumpall or pg_restore break.

Add an ic-pg-dump test job that runs them on Rocky 8, 9 and 10.

Assisted-by: Claude Code
getTypes() reads the default column encoding of each type from
pg_type_encoding into the typstorage column, and
dumpTypeStorageOptions() writes it as ALTER TYPE ... SET DEFAULT
ENCODING.  The REL_16_9 merge took the upstream query, which reads
pg_type.typstorage instead.  So pg_dump writes the storage letter as
an encoding for each base type, and the restore fails:

  ALTER TYPE dump_test.int42 SET DEFAULT ENCODING (p);
  ERROR:  type attribute "p" not recognized

Join pg_type_encoding again, as before the merge.  Now
004_pg_dump_parallel.pl passes 4 of 4, instead of 2.

Assisted-by: Claude Code
The REL_16_9 merge put the Cloudberry column aggrepsafeexec after the
last upstream column.  aggmfinalmodify has no comma after it, and
aggrepsafeexec has one before FROM:

  aggmfinalmodify
  aggrepsafeexec,
  FROM pg_catalog.pg_aggregate a, pg_catalog.pg_proc p ...

So pg_dump and pg_dumpall fail with 'syntax error at or near "FROM"'
for each database that has a user-defined aggregate.

Put the commas back where they were before the merge.  Also read
aggrepsafeexec only from Cloudberry 2 and later again, because other
servers do not have this column.

Assisted-by: Claude Code
PostgreSQL 15 moved the CREATE SCHEMA of dumpNamespace() into a check
of nspinfo->create, which is false for the public schema.  The REL_16_9
merge kept this, and also the older CREATE SCHEMA before the check.
So the dump creates each schema two times, and it also creates public:

  CREATE SCHEMA dump_test_second_schema;
  CREATE SCHEMA dump_test_second_schema;
  ...
  CREATE SCHEMA public;
  -- *not* creating schema, since initdb creates it

A restore of such a dump fails with "schema ... already exists".
Remove the older CREATE SCHEMA.

Assisted-by: Claude Code
_getObjectDescription() does not know the DYNAMIC TABLE type.  Before
the REL_16_9 merge this was only a warning, and the restored dynamic
table did not get its owner.  PostgreSQL 16 makes it an error, so each
dump with a dynamic table in a non-plain format, or with --clean, now
fails:

  pg_restore: error: don't know how to set owner for object type
  "DYNAMIC TABLE"

There is no ALTER DYNAMIC TABLE.  A dynamic table is a materialized
view, so set its owner with ALTER MATERIALIZED VIEW.

Assisted-by: Claude Code
setup_privileges() records the privileges of the catalog objects in
pg_init_privs.  setup_cdb_schema() runs cdb_schema.sql later, so the
GRANTs on gp_distributed_log, gp_distributed_xacts, gp_pgdatabase and
gp_transaction_log are not recorded.  pg_dump then sees them as
changes to the initial privileges, and dumps them for each database:

  GRANT SELECT ON TABLE pg_catalog.gp_distributed_log TO PUBLIC;

After the cdb_init.d scripts, record the privileges of the pg_catalog
relations that pg_init_privs does not have yet.

Backpatch-through: REL_2_STABLE
Assisted-by: Claude Code
The REL_16_9 merge took new upstream cases that Cloudberry does not
support, and the upstream output changed the expected text of some
Cloudberry cases.  Because of the first setup error, the test ran no
checks at all.

- Disable 'LO create (with no data)'.  Cloudberry does not support
  large objects, and the other large object cases are disabled too.
- Set allow_system_table_mods before the UPDATE of pg_database.
- The *_dir compression runs compress blobs.toc by hand, but without
  large objects there is no blobs.toc.  Do not do that.
- PostgreSQL 16 does not qualify the columns of a query on one table,
  so expect "SELECT col1" for matview_ivm and dynamic_table.
- pg_dump writes an AO table with SET default_table_access_method, so
  its WITH clause does not have appendonly = true.
- The merge kept two 'COPY test_third_table' cases, and the older one
  replaced the newer one.  Remove the older one.
- The new only_dump_measurement run does not dump the Cloudberry cases
  in dump_test.
- CREATE SCHEMA public is not dumped in binary upgrade mode any more.

Now the pg_dump TAP tests pass 11420 of 11420 on Rocky 8, 9 and 10.

Assisted-by: Claude Code
@tuhaihe tuhaihe moved this to In Progress in Merge postgres 16 Nov 4, 2026
@tuhaihe

tuhaihe commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Note: This separate PR addresses the failing test and is easy to review; once fixed, I’ll consolidate the similar tests into one.

The comments that the earlier commits of this change added name the
project as "GPDB".  Use "Cloudberry" instead.  Only comments change.

Assisted-by: Claude Code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant