Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
f6d3413
Rename CI image labels to Ubuntu 26.04 (#10934)
robert3005 Nov 1, 2026
65c992d
Point the Ubuntu 26.04 CI image labels at the Ubuntu 26.04 AMIs (#11075)
robert3005 Nov 1, 2026
9085458
Run the remaining benchmarks on the V2 scan (#11061)
joseph-isaacs Nov 2, 2026
085c871
Better READMEs and user docs (#10960)
myrrc Nov 2, 2026
a0068f2
Lock file maintenance (#11078)
renovate[bot] Nov 2, 2026
51454e0
Lock file maintenance (#11079)
renovate[bot] Nov 2, 2026
e3e7a4e
fix(python): report the scan's row count and columns from the Ray dat…
jackylee-ch Nov 2, 2026
e750985
Lock file maintenance (#11080)
renovate[bot] Nov 2, 2026
4b1dcc3
Run all Windows CI checks in one job (#11077)
robert3005 Nov 2, 2026
c6cca0d
fix(jni): release the schema export in VortexWriter.build() (#11081)
gitRasheed Nov 2, 2026
7c9bb58
fix(python): read pyarrow files positionally in open_readable (#11074)
jackylee-ch Nov 2, 2026
7e8fb3a
ci: disable FSST LIKE fuzzer in scheduled fuzz workflow (#11094)
joseph-isaacs Nov 2, 2026
ea94807
Separate the read and write IDs of array plugins (#11057)
connortsui20 Nov 2, 2026
75a4b92
bench: run JOB in post-commit develop benchmarks (#11095)
joseph-isaacs Nov 2, 2026
ec6c7fb
fix(spark): let abort close every writer after a failed write (#11084)
gitRasheed Nov 2, 2026
b9b02f6
Change extension dtype nullability without validating again (#11012)
connortsui20 Nov 2, 2026
921cb17
Refactor scan version configuration to use allowlist pattern (#11097)
joseph-isaacs Nov 2, 2026
23bb75b
fix(spark): resolve pruned nested fields against each file's struct (…
gitRasheed Nov 2, 2026
8448fa6
Record whether an array has been optimized (#11089)
robert3005 Nov 2, 2026
c95e777
Select editions from JNI and Python, and write files for readers befo…
robert3005 Nov 2, 2026
a606596
Read Vortex file column statistics through vortex-jni (#10863)
robert3005 Nov 2, 2026
dafc832
Require matching dtypes for extension comparisons (#11016)
connortsui20 Nov 2, 2026
3f12ae4
Preserve extension literals in DataFusion comparisons (#11067)
connortsui20 Nov 2, 2026
581ace7
fix(python): keep the scheme of object store paths in the Ray datasou…
jackylee-ch Nov 2, 2026
38dc319
Validate extension storage and return scalar read errors (#11014)
connortsui20 Nov 2, 2026
ca2d97b
perf(sequence): answer ordering and inequality comparisons directly (…
robert3005 Nov 2, 2026
0a43e94
perf(alp): push nullable and patched comparisons to the encoded ints …
robert3005 Nov 2, 2026
62e8814
Document temporal storage and binary comparison execution (#11099)
connortsui20 Nov 2, 2026
172ea59
fix(jni): attach native worker threads to the JVM as daemons (#11085)
gitRasheed Nov 2, 2026
3039ba1
ci: shard CodSpeed simulation benchmarks dynamically
joseph-isaacs Nov 2, 2026
9ba7f6e
Add ScalarFnVTable::execute_step for iterative scalar fn execution (#…
joseph-isaacs Nov 2, 2026
d8e7467
fix(duckdb): escape LIKE metacharacters in pushed string filters (#11…
jackylee-ch Nov 2, 2026
7d00329
Stabilize CodSpeed microbenchmarks (#11110)
connortsui20 Nov 2, 2026
955f4b6
Merge upstream develop into QC develop
ChristopherWirt Nov 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/runs-on.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
_extends: .github-private

images:
ubuntu24-full-x64-pre-v2:
ubuntu26-full-x64-pre-v2:
platform: "linux"
arch: "x64"
name: "vortex-ci-v2-x64-*"
name: "vortex-ci-v2-ubuntu26-x64-*"
owner: "245040174862"
ubuntu24-full-arm64-pre-v2:
ubuntu26-full-arm64-pre-v2:
platform: "linux"
arch: "arm64"
name: "vortex-ci-v2-arm64-*"
name: "vortex-ci-v2-ubuntu26-arm64-*"
owner: "245040174862"
windows25-full-x64-pre:
platform: "windows"
Expand Down
10 changes: 5 additions & 5 deletions .github/vercel-cli/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

197 changes: 96 additions & 101 deletions .github/workflows/ci.yml

Large diffs are not rendered by default.

271 changes: 153 additions & 118 deletions .github/workflows/codspeed.yml

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions .github/workflows/compat-gen-upload.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
dry-run:
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=compat-gen-dry-run', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=compat-gen-dry-run', github.run_id)
|| 'ubuntu-latest' }}
timeout-minutes: 30
permissions:
Expand Down Expand Up @@ -78,7 +78,7 @@ jobs:
if: inputs.confirm_upload == 'yes'
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=compat-gen-upload', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=compat-gen-upload', github.run_id)
|| 'ubuntu-latest' }}
timeout-minutes: 30
environment: compat-upload
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/compat-validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
compat-test:
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=compat-validation', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=compat-validation', github.run_id)
|| 'ubuntu-latest' }}
timeout-minutes: 30
steps:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/develop-bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
# forever. With the long `family` list and the longest benchmark id, this label is close.
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.8xlarge+c8g.8xlarge+c8gn.8xlarge+m8gd.8xlarge+m8g.8xlarge+r8gd.8xlarge+r8g.8xlarge+x8g.8xlarge+i8g.8xlarge/image=ubuntu24-full-arm64-pre-v2/extras=s3-cache/tag={1}', github.run_id, matrix.benchmark.id)
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.8xlarge+c8g.8xlarge+c8gn.8xlarge+m8gd.8xlarge+m8g.8xlarge+r8gd.8xlarge+r8g.8xlarge+x8g.8xlarge+i8g.8xlarge/image=ubuntu26-full-arm64-pre-v2/extras=s3-cache/tag={1}', github.run_id, matrix.benchmark.id)
|| 'ubuntu-latest' }}
strategy:
fail-fast: false
Expand Down Expand Up @@ -68,7 +68,7 @@ jobs:
timeout-minutes: 120
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.metal-24xl/image=ubuntu24-full-arm64-pre-v2/extras=s3-cache/tag={1}{2}', github.run_id, matrix.benchmark.id, matrix.benchmark.variant_id)
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.metal-24xl/image=ubuntu26-full-arm64-pre-v2/extras=s3-cache/tag={1}{2}', github.run_id, matrix.benchmark.id, matrix.benchmark.variant_id)
|| 'ubuntu-latest' }}
strategy:
fail-fast: false
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/fuzz.yml
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,8 @@ jobs:
fsst_like_fuzz:
name: "FSST LIKE Fuzz"
needs: build_cpu_fuzzers
# Temporarily disabled. The report and fix jobs below are skipped with it.
if: false
uses: ./.github/workflows/run-fuzzer.yml
with:
fuzz_target: fsst_like
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/musl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
timeout-minutes: 90
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-large/image=ubuntu24-full-x64-pre-v2/tag=rust-build-musl', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-large/image=ubuntu26-full-x64-pre-v2/tag=rust-build-musl', github.run_id)
|| 'ubuntu-latest' }}
container:
image: ghcr.io/rust-lang/rust:1-alpine3.24
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -182,9 +182,9 @@ jobs:
matrix:
include:
- target: x86_64-unknown-linux-gnu
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=prepare-java-linux-amd64
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=prepare-java-linux-amd64
- target: aarch64-unknown-linux-gnu
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=arm64-medium/image=ubuntu24-full-arm64-pre-v2/extras=s3-cache/tag=prepare-java-linux-arm64
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=arm64-medium/image=ubuntu26-full-arm64-pre-v2/extras=s3-cache/tag=prepare-java-linux-arm64
runs-on: ${{ matrix.runner }}
steps:
- uses: runs-on/action@v2
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pr-bench-runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
# forever. With the long `family` list and the longest benchmark id, this label is close.
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.8xlarge+c8g.8xlarge+c8gn.8xlarge+m8gd.8xlarge+m8g.8xlarge+r8gd.8xlarge+r8g.8xlarge+x8g.8xlarge+i8g.8xlarge/image=ubuntu24-full-arm64-pre-v2/tag={1}{2}', github.run_id, inputs.benchmark_id, github.event.pull_request.head.repo.fork == false && '/extras=s3-cache' || '')
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.8xlarge+c8g.8xlarge+c8gn.8xlarge+m8gd.8xlarge+m8g.8xlarge+r8gd.8xlarge+r8g.8xlarge+x8g.8xlarge+i8g.8xlarge/image=ubuntu26-full-arm64-pre-v2/tag={1}{2}', github.run_id, inputs.benchmark_id, github.event.pull_request.head.repo.fork == false && '/extras=s3-cache' || '')
|| 'ubuntu-latest' }}
steps:
- uses: runs-on/action@v2
Expand Down Expand Up @@ -93,7 +93,7 @@ jobs:
timeout-minutes: 120
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.metal-24xl/image=ubuntu24-full-arm64-pre-v2/tag={1}{2}', github.run_id, inputs.benchmark_id, github.event.pull_request.head.repo.fork == false && '/extras=s3-cache' || '')
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.metal-24xl/image=ubuntu26-full-arm64-pre-v2/tag={1}{2}', github.run_id, inputs.benchmark_id, github.event.pull_request.head.repo.fork == false && '/extras=s3-cache' || '')
|| 'ubuntu-latest' }}
steps:
# `extras=s3-cache` points ACTIONS_RESULTS_URL at the RunsOn artifact proxy, but only
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/publish-dry-runs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
if: github.event.pull_request.draft != true
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-ci-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=python-wheel-build', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-ci-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=python-wheel-build', github.run_id)
|| 'ubuntu-latest' }}
timeout-minutes: 30
steps:
Expand Down Expand Up @@ -82,9 +82,9 @@ jobs:
matrix:
include:
- target: x86_64-unknown-linux-gnu
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=check-java-publish-build-amd64
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=check-java-publish-build-amd64
- target: aarch64-unknown-linux-gnu
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=arm64-medium/image=ubuntu24-full-arm64-pre-v2/extras=s3-cache/tag=check-java-publish-build-arm64
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=arm64-medium/image=ubuntu26-full-arm64-pre-v2/extras=s3-cache/tag=check-java-publish-build-arm64
runs-on: ${{ matrix.runner }}
steps:
- parallel:
Expand Down Expand Up @@ -123,7 +123,7 @@ jobs:
timeout-minutes: 30
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-xsmall/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=rust-publish-dry-run', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-xsmall/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=rust-publish-dry-run', github.run_id)
|| 'ubuntu-latest' }}
steps:
- parallel:
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/release-binaries.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ jobs:
# cargo-zigbuild with glibc 2.31 pinning, matching the Python wheel
# and JNI builds.
- target: aarch64-unknown-linux-gnu
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=release-vx-aarch64-linux
fallback_runner: ubuntu-24.04
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=release-vx-aarch64-linux
fallback_runner: ubuntu-26.04
archive: tgz
- target: x86_64-unknown-linux-gnu
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=release-vx-amd64-linux
fallback_runner: ubuntu-24.04
runner: runs-on=${{ github.run_id }}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=release-vx-amd64-linux
fallback_runner: ubuntu-26.04
archive: tgz
steps:
- uses: runs-on/action@v2
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/rust-instrumented.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
- suite: tests
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-large/volume=200gb/image=ubuntu24-full-x64-pre-v2/tag=rust-coverage-suite-{1}', github.run_id, matrix.suite)
&& format('runs-on={0}/region=us-east-1/runner=amd64-large/volume=200gb/image=ubuntu26-full-x64-pre-v2/tag=rust-coverage-suite-{1}', github.run_id, matrix.suite)
|| 'ubuntu-latest' }}
env:
RUSTFLAGS: "-Cinstrument-coverage -A warnings"
Expand Down Expand Up @@ -113,7 +113,7 @@ jobs:
id-token: write
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=cxx-coverage', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=cxx-coverage', github.run_id)
|| 'ubuntu-latest' }}
steps:
- parallel:
Expand Down Expand Up @@ -158,7 +158,7 @@ jobs:
name: "Rust tests (${{ matrix.sanitizer }})"
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=rust-test-sanitizer', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/extras=s3-cache/tag=rust-test-sanitizer', github.run_id)
|| 'ubuntu-latest' }}
timeout-minutes: 30
env:
Expand Down Expand Up @@ -238,7 +238,7 @@ jobs:
RUST_LIB_BACKTRACE: 0
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/tag=rust-ffi-test-sanitizer', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/tag=rust-ffi-test-sanitizer', github.run_id)
|| 'ubuntu-latest' }}
steps:
- parallel:
Expand Down Expand Up @@ -287,7 +287,7 @@ jobs:
if: github.event.pull_request.draft != true
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/tag=rust-miri', github.run_id)
&& format('runs-on={0}/region=us-east-1/runner=amd64-medium/image=ubuntu26-full-x64-pre-v2/tag=rust-miri', github.run_id)
|| 'ubuntu-latest' }}
timeout-minutes: 30
env:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/s3-measurements.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ concurrency:
jobs:
collect:
timeout-minutes: 780
runs-on: runs-on=${{ github.run_id }}/runner=bench-dedicated/family=c8gd.16xlarge/spot=false/image=ubuntu24-full-arm64-pre-v2/tag=s3-measure/extras=s3-cache
runs-on: runs-on=${{ github.run_id }}/runner=bench-dedicated/family=c8gd.16xlarge/spot=false/image=ubuntu26-full-arm64-pre-v2/tag=s3-measure/extras=s3-cache
env:
AWS_REGION: us-east-1
REMOTE_ROOT: s3://vortex-ci-benchmark-datasets/s3-measurements/${{ github.run_id }}-${{ github.run_attempt }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/sql-bench-matrix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
FLAT_LAYOUT_INLINE_ARRAY_NODE: "1"
runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.8xlarge+c8g.8xlarge+c8gn.8xlarge+m8gd.8xlarge+m8g.8xlarge+r8gd.8xlarge+r8g.8xlarge+x8g.8xlarge+i8g.8xlarge/image=ubuntu24-full-arm64-pre-v2/tag=build{1}', github.run_id, (inputs.mode != 'pr' || github.event.pull_request.head.repo.fork == false) && '/extras=s3-cache' || '')
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family=c8gd.8xlarge+c8g.8xlarge+c8gn.8xlarge+m8gd.8xlarge+m8g.8xlarge+r8gd.8xlarge+r8g.8xlarge+x8g.8xlarge+i8g.8xlarge/image=ubuntu26-full-arm64-pre-v2/tag=build{1}', github.run_id, (inputs.mode != 'pr' || github.event.pull_request.head.repo.fork == false) && '/extras=s3-cache' || '')
|| 'ubuntu-latest' }}
steps:
- uses: runs-on/action@v2
Expand Down Expand Up @@ -112,7 +112,7 @@ jobs:

runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family={1}/image=ubuntu24-full-arm64-pre-v2/tag={2}{3}', github.run_id, inputs.machine_type, matrix.id, (inputs.mode != 'pr' || github.event.pull_request.head.repo.fork == false) && '/extras=s3-cache' || '')
&& format('runs-on={0}/region=us-east-1/runner=bench-dedicated/family={1}/image=ubuntu26-full-arm64-pre-v2/tag={2}{3}', github.run_id, inputs.machine_type, matrix.id, (inputs.mode != 'pr' || github.event.pull_request.head.repo.fork == false) && '/extras=s3-cache' || '')
|| 'ubuntu-latest' }}
steps:
- uses: runs-on/action@v2
Expand Down
Loading
Loading