Skip to content

feat(query): add query_arrow for Arrow results in one request - #189

Merged
eddietejeda merged 2 commits into
mainfrom
feat/query-arrow
Nov 19, 2026
Merged

eddietejeda merged 2 commits into
mainfrom
feat/query-arrow

Conversation

@eddietejeda

@eddietejeda eddietejeda commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds Client::query_arrow and Client::query_arrow_with (behind the arrow feature), which run a query and return the rows as Arrow in one request. Mirrors QueryApi.query_arrow in the Python SDK (hotdata-dev/sdk-python#238).

POST /v1/query already answers with an Arrow IPC stream when asked, and sends the values the JSON body would carry as response headers. query_to_arrow does not use that: it runs the query, then fetches the stored result in further requests.

Behaviour

  • Returns ArrowQueryResult: batches, schema, query_run_id, result_id, total_row_count, execution_time_ms, warning, rollup (RollupUsage), followed.
  • dialect and the other QueryRequest fields work as in query.
  • HTTP 429 and pre-response connection resets are retried on the same RetryPolicy as query.
  • A result too large to send inline (202 with X-Result-Id) is awaited and read from the Results API off the socket, asking for at most max_auto_rows + 1 rows. max_auto_bytes applies too, measured on the decoded batches' data; past either guard it returns ResultError::TooLarge. With auto_follow off it returns the new ResultError::NotInline, which carries the ids.
  • async = true is rejected with QueryError::AsyncRequested; a 202 that names no result is QueryError::Async.
  • Other errors (including the 408 query timeout) are QueryError::Submit with the status, as in query.
  • New QueryError::Arrow variant (feature-gated) for read/decode failures. Both error enums are #[non_exhaustive], and so is ArrowQueryResult.
  • A 200 that is not Arrow is returned as ArrowError::Http with its body, not as a decode error, so the run id of a query that already ran is kept.

query_to_arrow is unchanged: it pages with offset / limit and has its own error type.

Tests

  • 15 unit tests in src/query_arrow.rs (wiremock, real Arrow IPC bytes).
  • Integration scenario tests/query_arrow.rs: inline result, ClickHouse dialect, large-result follow, and the auto_follow-off refusal. Passes against a live workspace.

POST /v1/query can answer with an Arrow IPC stream and carries the run id,
result id, row count, timing, warning and rollup outcome as response
headers. Add Client::query_arrow / query_arrow_with, which make that one
request and return an ArrowQueryResult.

429 is retried as in query(). A result too large to send inline (202 with
X-Result-Id) is read from the Results API off the socket, under the
max_auto_rows guard; with auto_follow off it returns
ResultError::NotInline. Mirrors QueryApi.query_arrow in the Python SDK.
@eddietejeda
eddietejeda requested a review from a team as a code owner November 19, 2026 19:25
@eddietejeda
eddietejeda requested review from zfarrell and removed request for a team November 19, 2026 19:25
claude[bot]
claude Bot previously approved these changes Nov 19, 2026
- A 200 that is not Arrow returns its body (ArrowError::Http) instead of a
  decode error, so the run id of a query that already ran is not lost.
- The follow path enforces max_auto_bytes as well as max_auto_rows, on the
  size of each batch's own data.
- ArrowQueryResult is #[non_exhaustive], so fields can be added later.
- Tests for a non-Arrow 200, a cut-off Arrow body, and the byte guard.
@eddietejeda
eddietejeda merged commit bd055c1 into main Nov 19, 2026
4 checks passed
@eddietejeda
eddietejeda deleted the feat/query-arrow branch November 19, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant