Repository navigation
Run PRAGMA after earlier statements in one Python query - #640
Closed
t-kalinowski wants to merge 1 commit into
Closed
t-kalinowski wants to merge 1 commit into
t-kalinowski wants to merge 1 commit into
Conversation
evertlammerts
added a commit
to evertlammerts/duckdb-python
that referenced
this pull request
Oct 17, 2026
duckdb#678) Fixes the execute() half of the bug behind duckdb#640: a PRAGMA or other expanding statement after earlier statements in one string was expanded before those statements ran, so `CREATE TABLE t ...; PRAGMA table_info('t')` failed, and a later syntax error stopped the earlier statements from running. With the engine now expanding a statement when it is submitted (duckdb/duckdb#27108), execute(), sql() and executemany() drive the engine's statement iterator and submit each statement as parsed. The client adds what the engine does not do for a Python call: - Statements are parsed lazily, so a syntax error in a later statement is reported after the earlier ones ran, with the engine's own caret context and errors_as_json. This also fixes two long-standing expected failures in the JSON logging tests. - The last statement that returns rows gives execute()'s result; a statement after it that returns none (CREATE, INSERT, SET, COPY) no longer replaces the rows with its count. Rows of an earlier row-returning statement are held until a later row-returning statement is submitted. - A parse error aborts an open transaction the way a failed statement does. - Parameters are read before anything runs, and are refused when no statement takes them; executemany validates every parameter set before running. - A result that failed while it was being completed keeps its error, so an interrupted last statement ends its query before the connection is free again instead of deadlocking. Behavior changes: - A statement that expands to nothing (an IMPORT of an empty export) is a statement with an empty result, as in the engine and the CLI. - extract_statements returns statements as parsed; a PRAGMA is no longer expanded there. - sql() of a raw PIVOT or PRAGMA is evaluated when sql() returns; the lazy relation exists for a plain SELECT. Known limitation: rows of an earlier row-returning statement stay in memory while the statements after it run.
Member
|
Thanks for the PR and for tracking this down. The same fix landed in #678, which runs each statement of a query through the engine's statement iterator before the next one is expanded, and it shipped in 2.0.0. I checked 2.0.0 and main: EXPORT, DROP and PRAGMA import_database in one string now round trip for execute, sql and executemany, and CREATE TABLE followed by PRAGMA table_info sees the new table. The regression tests are in tests/fast/api/test_statement_order.py. Closing as superseded. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A
CREATE TABLEfollowed by a catalog-dependentPRAGMAin one Python query string failed because statement extraction expanded the PRAGMA before the table was created. Multi-statement calls now execute earlier statements before expanding the next one, soPRAGMA create_fts_index(...)can see the new table.Internal changes
Use DuckDB's statement iterator in the shared Python query path and keep the final statement for its existing result and parameter handling. A regression uses
PRAGMA import_databaseto cover the same ordering requirement without a downloadable extension.Related: duckdb/duckdb#26198.