Repository navigation
Conversation
The description of the "index" filter keyword referred to the --index option as --indexes, which is a typo. Oversight in commit a5cf808be5. Author: Taras Kloba <taras.kloba@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Discussion: https://postgr.es/m/CAODqpgofaVQBZ73rmPoj83Y1-6gbGhLiBKHD=82b1wiNkoF-AQ@mail.gmail.com Backpatch-through: 17
Base backups call stat() on each file while scanning the data directory, and later send exactly that many bytes. If the file is replaced by for example a concurrent rename(), the backup contains the new file truncated or zero-padded to the size of the old one. This is harmless for relation files as WAL replay repairs them. However, it causes harm for non-relation files manipulated as temporary file renamed in place. Here is a list of non-relation files that use a temporary file followed by a final rename() (I may be missing some): * postgresql.auto.conf * pg_logical/replorigin_checkpoint (a damaged copy fails its CRC check and prevents startup) * pg_prewarm's autoprewarm.blocks * pg_stash_advice.tsv. When sending files that are not relation files, fstat() is called again on the opened file and the refreshed stats are used for the tar header and the backup manifest. Since the rename is atomic, the opened file is always complete. Relation files keep using the earlier stat, to avoid copying data that WAL replay would restore anyway (because that's cheaper). Back-patch down to v15. v14 is going to be EOL soon, so I have decided to leave it out of the picture (risk/reward thing). Author: David Steele <david@pgbackrest.org> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Shihao Zhong <zhong950419@gmail.com> Discussion: https://postgr.es/m/CAGRkXqS_On-pLgcv6xjH8mvGhdcPmfWtythW3r8pYKUG+hsiuQ@mail.gmail.com Backpatch-through: 15
This was blocked during normal CREATE TABLE, but cases such as CREATE TABLE (LIKE) would copy the source table's NOT NULL NO INHERIT constraints and apply them to the partitioned table being created. This could mean that partitions didn't correctly receive the NOT NULL constraint when they were created, thus could contain NULL values. cacef17 added a test to ensure CREATE TABLE (LIKE) PARTITION BY fails when the source table has any CHECK NO INHERIT constraints. Here we do the same for NOT NULL constraints. It's probably unlikely that people have commonly done this, but the release notes may wish to note that people should check if this has happened to them. The following query can used each database to identify problematic constraints: SELECT conrelid::regclass, conname, connoinherit FROM pg_constraint WHERE contype = 'n' AND connoinherit = true AND conrelid IN (SELECT oid FROM pg_class WHERE relkind = 'p'); If no results are shown, the queried database isn't affected. To resolve these issues, unwanted constraints can simply be dropped from the partitioned table with: ALTER TABLE ... DROP CONSTRAINT ...; To remove the NO INHERIT flag, use: ALTER TABLE ... ALTER CONSTRAINT ... INHERIT; This will fail if any partition contains a NULL value in the given column. Finding the cause of failures is made more difficult by the planner trusting that "WHERE col IS NULL" cannot be true because of the presence of a NOT NULL constraint. Affected users may need to resort to tricks like WHERE col+0 IS NULL to find these. If it was thought to be a common problem, then we could adjust get_relation_notnullatts(), and the equivalent functions on older branches to ignore constraints marked as NO INHERIT on partitioned tables, but currently that would require an expensive lookup to find the pg_constraint row. Backpatch to v18 where NOT NULL NO INHERIT constraints were added by 14e87ffa5. Bug: #19770 Reported-by: Ann C <anncalla@163.com> Author: David Rowley <dgrowleyml@gmail.com> Author: Nitin Motiani <nitinmotiani@google.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Nitin Motiani <nitinmotiani@google.com> Discussion: https://postgr.es/m/19770-7c6e75fc30df7e06@postgresql.org Discussion: https://postgr.es/m/CAH5HC96YCzOy+9JZLQ4n0sMchVnharTOXrr4sbYuh=_d+ffjAA@mail.gmail.com Backpatch-through: 18
ruleutils.c's looks_like_function() assumed that all COERCE_SQL_SYNTAX cases produce output that looks like a function call (or at least like one of the syntaxes the func_expr_windowless production accepts). But AT TIME ZONE, AT LOCAL, OVERLAPS and IS xxx NORMALIZED print as other sorts of a_expr, so those cases would produce invalid SQL syntax when used in a FROM clause. Extend the logic to detect which specific functions are safe or unsafe. Author: Henson Choi <assam258@gmail.com> Reviewed-by: Prashant Chinnam <crprashant@hotmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/CAAAe_zC68s-DwVX1o7C1HVcRk+Z+RTS6Vw19auZFfz=KaX8tZA@mail.gmail.com Backpatch-through: 14
…M-clause functions.)
Base backups read the control file (pg_control) from disk without any protection against concurrent writes by the server, making it exposed to torn copies. Attempting to use such a control file at startup would cause a failure with a CRC not matching. Base backups now read the control file while holding ControlFileLock. c558e6fd92f has done the same for some SQL functions. The contents of the file are saved in memory and sent over the wire as-is. Backpatch down to v15. This issue is older than v15 but v14 is left out of the picture as it is going to be EOL soon, so it does not seem worth taking a risk for. Author: David Steele <david@pgbackrest.org> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/d1697de1-eb7f-41bf-a4b7-bba7203df11f@pgbackrest.org Backpatch-through: 15
RelationReloadIndexInfo() re-reads the mutable pg_index fields (indisready, indisvalid, indislive, etc.) only for indexes that are not system relations. The TOAST index of a user table lives in pg_toast and counts as one, but REINDEX CONCURRENTLY changes these fields on it like on any other index. A backend that had the TOAST indexes open when the invalidation arrived kept indisready = false for the new index, and did not insert its chunks there. Once the old index is dropped, reading such a value fails with "missing chunk number 0 for toast value N in pg_toast_X". The chunks remain in the TOAST table, only the index references were gone, so a rebuild of the TOAST index would fix this issue. This issue is fixed by re-reading the pg_index row for TOAST indexes, filling in their relcache fields. This is the same as for an index on a user table. Previouly, IsSystemRelation() was used, meaning the pg_index data was not re-read for a TOAST relation or a relation with a pinned OID (created by initdb). This check is meant for indexes whose values never change, so IsCatalogRelation() is fine, implying that only indexes created by initdb should be skipped (REINDEX CONCURRENTLY does not work on catalogs). This code is as old as 8ec9438, and was only used for CREATE INDEX CONCURRENTLY. 5dc92b8, that has introduced REINDEX CONCURRENTLY, has made possible to reach the problem with TOAST indexes due to these getting concurrently built, validated and then swapped. A test is added, able to reproduce the issue. It forces an insert before a swap, with a second session that maintains the TOAST indexes open while the new index becomes ready. This has been slightly simplified by me compared to what the author has sent. Oversight in 5dc92b8. Reported-by: Saicharanreddy Chinthareddy <saicharh@amazon.com> Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/CALj2ACUAv=7wjOV3pRFhQP4WHME_JazuhoCr01i4_F2XA=JKog@mail.gmail.com Backpatch-through: 14
The DEFAULT expression of an ON ERROR or ON EMPTY clause is compiled
with the JsonExpr's ErrorSaveContext in effect, so that any error
raised while evaluating it is saved rather than thrown and can be
reported later as "could not coerce ON ERROR expression (DEFAULT) to
the RETURNING type". That reporting is done by an
EEOP_JSONEXPR_COERCION_FINISH step, but ExecInitJsonExpr only added
that step when the DEFAULT expression needed a runtime coercion, or
when its top-level node was a CoerceViaIO or a CoerceToDomain. Those
are not the only nodes that can save an error: the ErrorSaveContext
applies to the whole expression tree, so a cast nested inside, say,
an OpExpr saves its error just the same. In that case there was
nothing to rethrow it and the query silently returned NULL instead
of failing, for example:
SELECT JSON_VALUE('"a"', '$' RETURNING int
DEFAULT (('a' || (random() * 0)::int)::int + 0)
ON ERROR);
Fix by adding the COERCION_FINISH step after every DEFAULT expression
that is evaluated softly, that is, whenever the ON ERROR behavior is
not ERROR.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxHDK39zF+1zZF6prH_wtn1SDt74_mrqYk1Oo82cCu44ng@mail.gmail.com
Backpatch-through: 17
As reported by buildfarm members thorntail and billbug, both running SPARC, 4a92c8c29bd0 is making the base backup code unstable on such platforms. The only suspicion I have at this stage is a stack frame issue, pointing to something unrelated to 4a92c8c29bd0, that the addition of an array worth a size of PG_CONTROL_FILE_SIZE within perform_base_backup() is able to trigger. thorntail complained with a SIGBUS, pointing at misaligned access, with a sporadic failure. billbug complained with a SIGSEGV (siglongjmp issue with PG_ENSURE_ERROR_CLEANUP perhaps?) in a more stable fashion. The code is changed to use a palloc0() instead, which should cool down the problem with the stack, hopefully. Unfortunately, I could not test this hypothesis directly, lacking an environment able to do so. Discussion: https://postgr.es/m/auGYf3jXtzLrr4gA@paquier.xyz Backpatch-through: 15
SQL/JSON query functions could throw an error, or report the wrong one, in cases where the ON ERROR or ON EMPTY clause said what to do instead. This happened when the RETURNING type, or a DEFAULT expression, involved a domain: if the value failed to convert to the domain's base type, that failure was saved to be handled by the ON ERROR behavior, but the domain's constraints were then checked anyway, against the dummy NULL left behind by the failed conversion. A CHECK expression can throw a hard error on that NULL, which ON ERROR cannot catch, and a NOT NULL constraint reports a violation that overwrites the saved error, so the error message ends up describing the wrong problem. For example, this failed with "division by zero" instead of returning 11: CREATE DOMAIN d AS int CHECK (1 / coalesce(VALUE, 0) = 0); SELECT JSON_VALUE(jsonb '"x"', '$' RETURNING d DEFAULT 11 ON ERROR); Fix by stopping as soon as a soft error has been saved, in each of the places that kept going: * The domain constraint checks of a CoerceToDomain in an ON ERROR or ON EMPTY DEFAULT expression, which is the only case where they are evaluated softly. ExecInitCoerceToDomain now emits a jump before each constraint check that skips the remaining checks if an error has already been saved. * populate_domain, called from json_populate_type when coercing to a domain. Return as soon as the conversion to the base type fails, instead of going on to check the domain's constraints. * populate_record, likewise. Stop populating the remaining columns as soon as one of them fails. * ExecEvalJsonCoercion. Return at once if the DEFAULT expression whose result it would coerce has already failed, and leave throwing the error to the COERCION_FINISH step that follows. Reported-by: jian he <jian.universality@gmail.com> Author: jian he <jian.universality@gmail.com> Discussion: https://postgr.es/m/CACJufxHDK39zF+1zZF6prH_wtn1SDt74_mrqYk1Oo82cCu44ng@mail.gmail.com Backpatch-through: 17
…domain fails softly)
transformJsonTable() handed the raw PASSING argument list both to the JsonFuncExpr built for the row pattern and, via transformJsonTableColumn(), to the JsonFuncExpr built for each column other than FOR ORDINALITY, so the same raw expressions were run through transformExpr() once for the row pattern and again for every such column. That doesn't work for sublinks: transformSubLink() replaces the SubLink's raw subselect with the analyzed Query, so the second pass failed with "unexpected non-SELECT command in SubLink". Other nodes that are transformed in place, such as NullTest, failed with "unrecognized node type". Any such PASSING argument therefore made JSON_TABLE fail unless all of its columns were FOR ORDINALITY. Fix by giving each transformation its own copy of the raw PASSING arguments. Bug: #19786 Reported-by: Alperen Keles <alpkeles99@gmail.com> Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Co-authored-by: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/19786-70758103a4b2bff2@postgresql.org Backpatch-through: 17
A buffer registered with REGBUF_NO_CHANGE is not modified by the record and, in the case of the hash index AM, may be held with only a pin (no content lock). The hash index code registers the primary bucket page this way in hashbucketcleanup(), _hash_freeovflpage() and _hash_squeezebucket(), solely to force a cleanup lock on that block during replay. When wal_consistency_checking includes "hash", XLogRecordAssemble() included a full-page image for block only pinned, because XLR_CHECK_CONSISTENCY forces an image regardless of REGBUF_NO_IMAGE. The record CRC is then computed over the image while a concurrent backend, which legitimately holds nothing stronger than a pin on that page, could modify it before the record is copied into the WAL buffers. This would result is an intermittent "incorrect resource manager data checksum in record" failure on a standby. As of this commit, never take an image for a REGBUF_NO_CHANGE buffer, even under XLR_CHECK_CONSISTENCY. There is no need to do a consistency check on a block not changed by a record. verifyBackupPageConsistency() is taught to skip a registered block that carries no image rather than asserting one should be present. REGBUF_NO_CHANGE exists since 00d7fb5e2e39, so backpatch down to v17. On older branches, the CRC problem can be reproduced as well, but we lack the mechanism to check how a buffer is handled by the code assembling a record. Per discussion, the long term picture should be to fix the hash index AM so as REGBUF_NO_CHANGE is removed. This would be a HEAD-only change. Reported-by: Alexander Lakhin <exclusion@gmail.com> Diagnosed-by: Thomas Munro <thomas.munro@gmail.com> Author: Greg Burd <greg@burd.me> Discussion: https://postgr.es/m/ats4pi-RM0Uom0T9@floki Backpatch-through: 17
When reconsider_outer_join_clauses() finds an outer-join clause OUTERVAR = INNERVAR whose OUTERVAR is known to equal a constant, it pushes INNERVAR = CONSTANT into the inner side and replaces the clause with constant TRUE. The replacement assumes that OUTERVAR cannot have gone to NULL where the clause is evaluated. That assumption fails for a clone made for outer join identity 3. When an outer join is pushed into the RHS of a lower outer join, its relid is not added to the joinrel until the commuted pair is complete, so a version of a clause that lacks that join's nulling bit can be applied above it, after it has null-extended OUTERVAR. This is fine for the original clause, because it is strict, which is what allowed the commutation in the first place; but it is not fine for constant TRUE. As a result, inner rows could be joined to null-extended outer rows, producing wrong query results. To fix, throw back a clone as-is rather than the TRUE replacement, as v15 and earlier did. The derived INNERVAR = CONSTANT clause remains valid and is still pushed down. This can leave a redundant join filter in some plans, as seen in an existing regression test. To keep the thrown-back clause from affecting selectivity estimates, including for parameterized scans, where the cached selectivity is not used, restore the redundancy marker removed by b448f1c8d: a norm_selec greater than 1 marks a redundant clause, for which clause_selectivity_ext() always returns 1.0. Back-patch to v16, where clone clauses were introduced. Bug: #19765 Reported-by: Viktor Leis <leis@in.tum.de> Diagnosed-by: Tender Wang <tndrwang@gmail.com> Author: Richard Guo <guofenglinux@gmail.com> Co-authored-by: Nitin Motiani <nitinmotiani@google.com> Reviewed-by: Tender Wang <tndrwang@gmail.com> Discussion: https://postgr.es/m/19765-5778b18918040583@postgresql.org Backpatch-through: 16
It's not valid for a RelOptInfo to have a non-empty partial_pathlist when its consider_parallel is true. In set_append_rel_pathlist() when we see the parent rel doesn't have consider_parallel set, we set that property to false for the child. It is possible that the child already has some partial paths at that point, so we need to nullify that list as well as set the consider_parallel flag to false. For the reported problem case, the partial paths had been created during set_subquery_pathlist(). These not being empty was causing an Assert failure on cassert builds. I think this could only have gone on to result in some wasted planner effort in non-cassert builds, but cannot confirm if anything worse could have happened. Bug: #19803 Reported-by: Gowtham Reddy <gowthamdoma18@gmail.com> Author: David Rowley <dgrowleyml@gmail.com> Tested-by: Gowtham Reddy <gowthamdoma18@gmail.com> Discussion: https://postgr.es/m/19803-1fcc1df108c54ab6@postgresql.org Backpatch-through: 14
A NestLoop can supply a PlaceHolderVar as a parameter if the PHV can be evaluated from the lefthand rels plus the required-outer rels of the join. A PHV's ph_eval_at can include outer-join relids, but required-outer sets need not: create_lateral_join_info() keeps outer-join relids out of the lateral dependencies it derives from ph_lateral. So when an outer join within ph_eval_at had been computed by an enclosing nestloop's outer side, the test in identify_current_nestloop_params() failed and no plan node claimed the parameter. This resulted in "failed to assign all NestLoopParams to plan nodes", or, if an unrelated upper NestLoop happened to cover ph_eval_at and claimed the parameter, "plan should not reference subplan's variable". To fix, have create_nestloop_plan() also count the outer joins that have already been computed by the outer rels in scope. Back-patch to v18, where commit a16ef313f made it possible to compute a PHV that is used as a nestloop parameter in the outer side of the NestLoop. v16 and v17 can fail the same way, since ph_eval_at has included outer-join relids since v16, but they lack that machinery. Fixing them would mean adding join-order restrictions to have_dangerous_phv() that could change plans that work today, which does not seem worth the risk for a failure this unlikely. Author: Richard Guo <guofenglinux@gmail.com> Discussion: https://postgr.es/m/CAMbWs4-fcAULTpWthZMHGOFK1J+Hz=Dvr1TO-M20cC317rOGOA@mail.gmail.com Backpatch-through: 18
…ated above outer joins)
The loop above from where the Assert is removed handles assigning the same attnum to any duplicate expressions, so asserting there are no duplicate attnums just after purposefully making duplicate ones isn't the correct thing to do. The Assert was added by a4d75c8, so backpatch to v14 Author: BHUVAN CHILUKUTI <c.bhuvan@iitg.ac.in> Reviewed-by: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/PN0P287MB0903563C70D8E5B81912E855A7602@PN0P287MB0903.INDP287.PROD.OUTLOOK.COM Backpatch-through: 14
When the centroid lies outside the query box, spg_quad_inner_consistent() keeps only the quadrants that contain a corner of the box. It meant to test all four corners, but the last step set x back to low.x instead of y to low.y, so it tested (low.x, high.y) twice and never (high.x, low.y). With exact comparisons the missing corner would not matter: a box that does not contain the centroid lies on one side of it on at least one axis, so (high.x, low.y) shares a quadrant with another corner, but getQuadrant() compares with FPeq(), which tiebreaks to the lowest numbered quadrant. When a box edge lies within EPSILON of the centroid, the corner on that edge counts as being on the centroid's line, and (high.x, low.y) may be the only corner in its quadrant. That quadrant was then skipped. Fix the missing corner test. Discussion: https://postgr.es/m/CANWCAZZZpkwrX0kC4Jo6KM27J6dyBdCDir-PbjEYAS06r%3D4-ag%40mail.gmail.com Backpatch-through: 14
The buildfarm member bushmaster reported failures in the final test query in 041_checkpoint_at_promote.pl with "the database system is in recovery mode". After killing a backend to trigger crash recovery, the test should wait for the postmaster to terminate the remaining server processes and restart the server, then wait for recovery to finish and connections to be accepted before running the final query. Previously, the test assumed that the killed backend's connection closing meant the postmaster had noticed the crash and begun restarting. However, the connection can close before the postmaster notices the crash. The subsequent connection check can then succeed before recovery starts, causing the test to mistake this for recovery having completed. The final query could then run during recovery and fail. Fix this by waiting for the postmaster to log that all server processes have terminated and it is reinitializing before checking that connections are accepted again. This ensures that the postmaster has noticed the crash and begun restarting before the test waits for recovery to finish. Backpatch to v17, where this test was introduced. Reported-by: Alexander Lakhin <exclusion@gmail.com> Author: Fujii Masao <masao.fujii@gmail.com> Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Discussion: https://postgr.es/m/e7abd282-655d-495e-94c9-e38a58917ebf@gmail.com Backpatch-through: 17
Previously, strtoint() set errno to ERANGE on overflow but returned the result of casting the long value to int. When the value is outside the range of int, this produces an implementation-defined result that cannot reliably indicate the direction of overflow. POSIX requires strtol() to return exactly LONG_MAX or LONG_MIN on overflow, which permits safe sign-testing or range-testing. By analogy, make strtoint() return INT_MAX or INT_MIN on overflow. Back-patch, since future bug fixes may expect this behavior. Author: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/A33EE17C-D014-4980-B3DD-AC51588A06EC@gmail.com Backpatch-through: 14
pg_GSS_have_cred_cache() always acquires new GSS credentials. If we already had some, they were just leaked due to overwriting conn->gcred, and that's a nontrivial amount of memory (around 16K according to reporter). Unfortunately, pg_GSS_continue() did this each time through, and it is typically called more than once in a GSSAPI connection cycle. So repeated connections using GSS authentication could leak substantial amounts of memory. IMO the source of this problem is really the poorly-designed API of pg_GSS_have_cred_cache(): it's unclear that it always gets new credentials. Let's rename it and remove the unnecessary pointer parameter to make its behavior clearer. To add insult to injury, pg_GSS_continue() shouldn't even be trying to acquire credentials unless GSS delegation is enabled, which it is not by default; so the whole effort is often wasted. Problems evidently introduced by 6633cfb21, so back-patch to v16. Bug: #19411 Reported-by: Matt Carter <Matt.Carter@twosigma.com> Diagnosed-by: Matt Carter <Matt.Carter@twosigma.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19411-0440c8897a04b638@postgresql.org Backpatch-through: 16
Test EPQ handling in presence of dropped columns and resjunk elements in the targetlist. Development versions of an upcoming fix broke those cases without breaking any extant tests. Security: CVE-2026-6460 Backpatch-through: 14
Test a trigger on a partition table where a columns NOT NULLness differs between parent and child. Development versions of an upcoming fix broke those cases without breaking any extant tests. Security: CVE-2026-6460 Backpatch-through: 14
make_expanded_record_from_datum() / DatumGetExpandedRecord() / PG_GETARG_EXPANDED_RECORD() were unused. I could not find any extensions using it either. It seemed too likely that one of the subsequent fixes in this series would break them, therefore this adds some tests (written with AI help). Security: CVE-2026-6460 Backpatch-through: 14
An upcoming fix touches related code. It turns out there were a number of cases there were not or only incidentally tested. In particular, we didn't leave tables with combinations of dropped columns to be tested by 002_pg_upgrade.pl Security: CVE-2026-6460 Backpatch-through: 14
Most of these show the problem by displaying 'FIXME' in test output in case of problems, for more details about that aspect see rowtypes_alter.sql. A large percentage of these tests were converted from a larger set of tests that were not mergeable in their former shape. Author: Andres Freund <andres@anarazel.de> Author: Heikki Linnakangas <heikki.linnakangas@iki.fi> (in different form) Security: CVE-2026-6460 Backpatch-through: 14
Otherwise we can assign tuples not matching the target tupledescriptor, e.g. because one of the tuple descriptors has been acquired before/after some DDL. It may seem tempting to just not enter the surrounding "if" if the er_tupdesc_id doesn't match (in fact both Heikki and I (Andres) were previously tempted), but the fieldwise-assignment path differs behaviorally, due to not copying the ctid along. As the tupdesc_id can increment due to invalidation processing, that would lead to unpredictable behavior. Tests were added in previous commits that would show that behavior. Note that there are some cases where this will cause failures that are not required for correctness, when assigning a variable of a changed type to a previously used variable. It'd add more complexity to make that work; for a bugfix this seems good enough. As also noted in a comment, this perhaps should be done in the expanded record layer. But that'd require a few more changes and would make it harder to emit a somewhat understandable error message. Reviewed-by: Tomas Vondra <tomas@vondra.me> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Security: CVE-2026-6460 Backpatch-through: 14
Only make_expanded_record_from_datum() deferred tuple descriptor lookup, and it has no direct/indirect users in core nor in known open source extensions (checked via pgxn and github). The other ERH constructors already acquire a descriptor eagerly. The main reason for this change is that subsequent commits are simplified by it. This also allows implementing the datum constructor using make_expanded_record_from_typeid(), removing a good bit of duplicated code. This preserves ABI compatibility, to allow for backpatching. Reviewed-by: Tomas Vondra <tomas@vondra.me> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Security: CVE-2026-6460 Backpatch-through: 14
Using a tuple descriptor that does not match the "shape" of a composite datum can lead to crashes, spurious errors and can be used for exploits. While postgres has code that prevents such mismatches, there are a number of weaknesses in those protections that are nontrivial to fix, particularly in the back-branches. This commit introduces a new safety layer to prevent the worst of the negative consequences. From now on, composite datums are marked as such and named composite datums are versioned, allowing detection of mismatches between a tuple descriptor and the composite during deforming. In case of mismatches, errors are raised. For details about how this works, see src/backend/catalog/README.attver.md. Note that, over time, we should prevent most of the things that the safety layer detects at a higher level. This fixes a number of security relevant composite type confusions. This is a significant piece of infrastructure grafted onto the existing code, in a back-patchable way. There unfortunately still are some compatibility issues due to this commit: - To get the full protection, extensions need to be rebuilt, due to DatumGetHeapTupleHeader() not being a macro anymore (but an out-of-line function). That also means that some extensions built against a minor version with this change will fail with a linker failure when loaded in an earlier minor version. Extensions that have not been recompiled can observe HeapTupleHeaderGetTypMod() returning parts of the version instead of -1. - Upgrading to a minor version with this fix, performing DDL, downgrading and then upgrading again can cause errors for composite datums formed while downgraded; DDL while downgraded can additionally break the version invariants. - Code looking at pg_attribute.attoptions will see the new pg_attver key, which could lead to external applications setting this new key via DDL. To reduce the impact of that, ALTER COLUMN ... SET (pg_attver = ...) is ignored. - Dumps of a server with this change, made by a pg_dump without it, contain ALTER COLUMN ... SET (pg_attver = ...), which servers without this change reject. As pg_upgrade uses the target's pg_dump, upgrading from a minor version with this change to a minor version of a later major version without it fails. - Use of TupleDescCopy() is forbidden with tupledescs for named composites. The new TupleDescCopySize()/TupleDescCopySized() can copy any descriptor, and verify that the destination is large enough. - Code that forms datums and then modifies fields in t_choice.t_datum without removing HEAP_IS_DATUM can trigger spurious errors. To reduce the impact of that, the version check skips some known cases of such modifications (see list below for the known effects on extensions). - DatumGetHeapTupleHeader() will be slower when pre-upgrade tuples are passed, due to copying those datums before modifying them. - Tuples returned by heap_form_tuple() are now marked as datum tuples and, for named composites, version-checked when deformed. Code that writes such tuples to its own storage needs to clear the marking (via HeapTupleHeaderClearDatum()). - heap_modify_tuple() used to copy t_ctid, which is not possible anymore, due to that containing the datum version. No callers could be found that need the copy. At least some of the above should be mentioned in the release notes. Investigation via github codesearch, a dump of all the extensions on pgxn, extensions from various distributions, etc, suggests the following effects on extensions: - PostGIS does some quite broken things on the upgrade from < 3.6 to 3.6, including corrupting the catalog. This is more noticeable with these changes. In particular WARNINGs will be emitted and pg_upgrade will fail, until PostGIS is fixed. See https://trac.osgeo.org/postgis/ticket/6142 - PL/R (trigger arguments), pg_variables (stored records) and pljs copy tuples and set the composite datum header fields by hand, which leads to spurious errors when the result is read. They should use heap_copy_tuple_as_datum() instead. - ddlx fails for columns with an explicitly set attribute option, as it expects at most one option per column. pgcontext refuses to convert versioned columns, as it rejects columns with any attribute options. - firebird_fdw spuriously errors out in some cases, due to overwriting the xmax (but not xmin). Other FDWs clear out xmin, which this commit uses as a signal that the version information is not usable, but there's no easy way to detect this in the firebird_fdw case. The fix would be a one-liner. - storage_engine's rowcompress table AM stores composite datum-style tuples, which will raise errors on read if the table schema is subsequently changed. The fix is trivial. - pgrx-based extensions will not be protected, even after recompiling, due to pgrx copying postgres code like heap_getattr(). pgrx will need to update that copied code. - pg_strom's arrow FDW bypasses core functionality for more than 100 columns and inserts attributes itself. That does not set the attribute versions correctly and thus will trigger spurious errors if composites of that FDW are used. - timescaledb has a bug causing crashes during chunk creation as part of DDL commands if attribute options are present. With this change every new / altered attribute has an attribute option. A later commit will work around these crashes. - Citus' distributed aggregation path theoretically can hit the new error in TupleDescCopy(). However, it looks like Citus' planner does not ever generate such plans - but the ERROR can be reached by a user issuing a query with calls to the relevant internal Citus functions. The fix again is trivial, should it be needed (use CreateTupleDescCopy() instead of open-coding it). The new composite_type_mismatch developer option allows superusers to turn the version mismatch error into a warning, or to disable the check. Both settings are unsafe. With this infrastructure in place, additional checks can be performed (e.g. in heap_datum_check(), heap_datum_update_type()), at the cost of some complexity and runtime overhead. A subsequent commit adds those. Reported-by: Heikki Linnakangas <heikki.linnakangas@iki.fi> Reported-by: Andres Freund <andres@anarazel.de> Reported-by: Andrew Gierth <andrew@tao11.riddles.org.uk> Reported-by: Alexander Lakhin <exclusion@gmail.com> Reported-by: Francesco Verardi <frevadiscor89@gmail.com> Reported-by: Hcamael <baiyjrh@gmail.com> Reported-by: Tomer Fichman <tomer@irregular.com> Reported-by: OpenAI Security Research Team Reported-by: Peter Geoghegan <pg@bowt.ie> Reported-by: Mehmet Ince @mdisec <mehmet@mehmetince.net> Reported-by: Bryan Green <dbryan.green@gmail.com> Reported-by: Sumedh Pathak <sumedhpathak@microsoft.com> Reported-by: Nicholas Carlini <npc@anthropic.com> Reported-by: Sergey Shinderuk <s.shinderuk@postgrespro.ru> Reported-by: Yuxiao Guo <dllggyx@outlook.com> Reported-by: Nikhil Sontakke <nikhil@planetscale.com> Reported-by: Stephan Lipp <Stephan@Adalogics.com> Reported-by: Daniel Bakker <daniel@jackds.nl> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Tomas Vondra <tomas@vondra.me> Reviewed-by: Noah Misch <noah@leadboat.com> Reviewed-by: Robert Haas <robertmhaas@gmail.com> (parts) Security: CVE-2026-6460 Backpatch-through: 14 # Conflicts: # src/backend/access/common/heaptuple.c # src/backend/access/common/tupdesc.c # src/backend/access/heap/heapam.c # src/backend/commands/tablecmds.c # src/include/catalog/pg_attribute.h
Some of these were originally written with AI contributions, but they have been heavily rewritten since. The most important thing this tests is pg_upgrade's handling of attribute versions, including verification of composite columns staying properly readable across upgrades. Security: CVE-2026-6460 Backpatch-through: 14 # Conflicts: # contrib/test_decoding/Makefile
The previously added attribute versioning checks prevent cases of deforming a datum tuple with an incompatible tuple descriptor for the same type. However, we have a number of cases where we deform a datum tuple with a descriptor for a different relation. The additional checks introduces here add verification for such cases, by checking that the tuple descriptors are compatible. The following cases are now checked during deforming: - Deforming a versioned tuple with a RECORDOID tupledesc This can happen with triggers and EvalPlanQual rechecks of function scans. - Deforming a versioned tuple with a tupledesc for another relation This happens a bunch for partitioned tables. Additionally, when changing the type of a tuple, the old and new types are compared for compatibility. This currently happens in: - heap_copy_tuple_as_datum() and via that in places like ExecEvalConvertRowtype(), ExecFetchSlotHeapTupleDatum(), ... - plpgsql's coerce_function_result_tuple() In a perfect world these extra checks would not be needed. But... It is perhaps worth noting that this knowingly emits error messages in an untranslatable way. That was considered acceptable for now, these error checks should not normally be hit. Note that these checks do add nontrivial overhead. This is ameliorated by the fact that the common ways to reach the checks are part of already expensive operations (like trigger invocations during DML into a partitioned table), which are expensive enough to make the added overhead comparatively small. Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Tomas Vondra <tomas@vondra.me> Reviewed-by: Noah Misch <noah@leadboat.com> Security: CVE-2026-6460 Backpatch-through: 14
TimescaleDB crashes if an attribute option is present during chunk creation as part of a DDL command, due to calling AlterTableInternal() without having set up event trigger state. With the introduction of attribute versioning, which stores the version in an internal attribute option in the back-branches, attribute options are much more common. The breakage is easy enough to work around, so do that. This will only be applied to release branches, as this is clearly a bug in TimescaleDB. Security: CVE-2026-6460 Backpatch-through: 14-19
Security: CVE-2026-6460 Backpatch-through: 14-18
Security: CVE-2026-6460 and many more.
…_5.7 # Conflicts: # src/oracle_test/modules/gin/expected/gin_incomplete_splits.out # src/oracle_test/modules/test_extensions/expected/test_extensions.out # src/oracle_test/regress/parallel_schedule
Contributor
|
Important Review skippedToo many files! This PR contains 752 files, which is 602 over the limit of 150. To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR synchronizes IvorySQL v5.7 with upstream PostgreSQL 18.7 release
This pull request synchronizes IvorySQL IVORY_REL_5_STABLE with PostgreSQL REL_18_STABLE branch over the period
2026-08-11 to 2026-11-09, incorporating a total of 571 commits:
"Tweak allocation rule for tzload()'s "union local_storage" variable." to the latest included commit "Stamp 18.7."
upstream changes.
All integrated code has been thoroughly validated and successfully passes four core regression test
suites(check-world; oracle-check-world; oracle-pg-check; oracle-check), ensuring functional correctness and stability
after the sync.
close #3343