Skip to content

Exclude Markdown from ruff format - #8940

Merged
robert3005 merged 1 commit into
developfrom
rk/ruff-md-format-exclude
Jul 23, 2026
Merged

robert3005 merged 1 commit into
developfrom
rk/ruff-md-format-exclude

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

Rationale for this change

The Python (lint) CI job is failing on develop: CI installs ruff unpinned via uvx, and ruff 0.16 started formatting Python code blocks inside Markdown files by default. Three docs (docs/user-guide/spark.md, java/vortex-spark/README.md, vortex-python-cuda/README.md) use deliberate comment alignment in their examples and now fail ruff format --check.

What changes are included in this PR?

Adds exclude = ["*.md"] under [tool.ruff.format], restoring the pre-0.16 behavior (Python sources only — 137 files — are formatted; ruff check is unaffected). Verified locally with ruff 0.16.0: uvx ruff format --check . and uvx ruff check . both pass.

What APIs are changed? Are there any user-facing changes?

None — lint configuration only.

🤖 Generated with Claude Code

ruff 0.16 (installed unpinned via uvx in CI) started formatting Python
code blocks inside Markdown files by default, breaking the Python lint
job on three docs that use deliberate comment alignment. Keep Markdown
out of the formatter to restore the pre-0.16 behavior; ruff check is
unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005 robert3005 added the changelog/chore A trivial change label Jul 23, 2026
@robert3005
robert3005 enabled auto-merge (squash) July 23, 2026 22:48
@robert3005
robert3005 merged commit 2794f89 into develop Jul 23, 2026
82 of 83 checks passed
@robert3005
robert3005 deleted the rk/ruff-md-format-exclude branch July 23, 2026 22:49
dimitarvdimitrov pushed a commit that referenced this pull request Jul 28, 2026
## Rationale for this change

The `Python (lint)` CI job is failing on `develop`: CI installs ruff
unpinned via `uvx`, and ruff 0.16 started formatting Python code blocks
inside Markdown files by default. Three docs
(`docs/user-guide/spark.md`, `java/vortex-spark/README.md`,
`vortex-python-cuda/README.md`) use deliberate comment alignment in
their examples and now fail `ruff format --check`.

## What changes are included in this PR?

Adds `exclude = ["*.md"]` under `[tool.ruff.format]`, restoring the
pre-0.16 behavior (Python sources only — 137 files — are formatted;
`ruff check` is unaffected). Verified locally with ruff 0.16.0: `uvx
ruff format --check .` and `uvx ruff check .` both pass.

## What APIs are changed? Are there any user-facing changes?

None — lint configuration only.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Robert Kruszewski <github@robertk.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
phillipleblanc added a commit to spiceai/vortex that referenced this pull request Aug 5, 2026
* chore: remove committed Java class artifacts and ignore java/bin

The REUSE compliance check fails because 30 compiled .class files are
tracked under java/vortex-jni/bin/. Build outputs carry no SPDX
copyright or licensing headers, so reuse-check reports the project as
non-compliant (2851 of 2881 files annotated).

The Gradle build writes to build/, which is already ignored; bin/ is the
Eclipse/VS Code Java output directory and is not referenced by any
Gradle configuration. Remove the tracked artifacts and ignore bin/ so
they cannot be committed again.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* chore: ignore IntelliJ out/ and stray class files under java/

Ignoring only bin/ prevents the exact path that tracked the removed
artifacts. Compiled output can still reach the tree through IntelliJ's
out/ directory or a stray .class drop, each of which fails reuse-check
the same way, since REUSE requires an SPDX header on every tracked file.

Gradle writes classes under build/, so no tracked file matches these
patterns.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* fix: restore size-based file splitting in VortexSink

VortexFormat configures target_file_size_mb (default 128MB), computes a
byte target, and passes it to VortexSink::new, but the sink accepts three
arguments and holds no target, so vortex-datafusion does not compile:

    error[E0061]: this function takes 3 arguments but 4 arguments were supplied
      --> vortex-datafusion/src/persistent/format.rs:737:29

Restore the sink's implementation of the option: the target_file_size
field, the compression estimate used to predict a file's compressed size,
the active-writer bookkeeping that rolls to a new file once the target is
reached, and the direct write path that keeps one coordinated writer per
statement for non-partitioned writes. Partitioned writes continue to go
through DataFusion's demuxer.

ArrowSessionExt is imported from vortex_arrow, matching its current home.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* fix: clean up and abort VortexSink writers on failure and cancellation

Three related defects on the sink's non-partitioned write path.

A file that fails while being finalized is already out of active_writer
and never reaches results, so cleanup saw neither and left the object
behind while deleting its successfully rotated siblings. Track every path
a writer was opened for and clean up from that instead.

ActiveFileWriter held a bare JoinHandle, which keeps running when
dropped. A cancelled write could therefore let the writer task finish its
multipart upload and publish a file the caller never accounted for, and on
stores that do not discard incomplete multipart uploads the upload was
left to a lifecycle rule. Abort the task on drop.

finish_file_writer took the handle out of the writer before awaiting it,
which reintroduced that same detached handle for the duration of the join,
and the join is when the file is published. Join through the handle the
writer still owns so cancellation aborts it, and clear it once the task
has completed.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* ci: install build tools on forks

Every job selects upstream's prebuilt runner image only when
github.repository is vortex-data/vortex, and otherwise falls back to a
bare GitHub-hosted runner. That image supplies cargo-hack, cargo-nextest,
cargo-minimal-versions, grcov, uv and flatc, so on a fork these jobs fail
immediately with 'no such command' or 'command not found' before they
compile or test anything.

setup-prebuild already owns this split, so the two tools that many jobs
need, cargo-hack and cargo-nextest, are installed there on the fallback
path. A job that starts using either one gets it without having to know
about forks. This also replaces the Rust lint job's cargo install of
cargo-hack, which compiled it from source on every fork run.

Tools that only one job needs stay with that job: cargo-minimal-versions,
grcov, flatc, and uv.

setup-flatc extracts to /tmp rather than the working directory, so the
generated-files job's clean-tree assertion cannot trip over a stray
binary, and installs with sudo when available, matching setup-protoc.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* ci: exclude Markdown from ruff format

Newer ruff formats Python code blocks inside Markdown, which reports three
documentation files as unformatted. The examples in them are deliberately
aligned across several lines and reformatting collapses each onto one long
line.

Take the exclusion upstream added for the same reason in 2794f89
("Exclude Markdown from ruff format", vortex-data#8940), which is not on this release
branch. Upstream does not hit the failure because its prebuilt image
supplies uv with a warm tool cache, while the fork installs ruff fresh.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* ci: fix schema back-compat base and restrict benchmarks to upstream

The flatbuffer back-compat step always compared against develop. A release
branch is behind develop by construction, so conforming this branch against
it reports every field develop has added since as a deleted field, which is
branch topology rather than a change in the pull request. Compare against
the ref being merged into instead, which asks the question that matters on
any branch, including develop itself.

Reporting a Codspeed run needs CODSPEED_TOKEN, which only upstream holds, so
the benchmark shards cannot report anywhere else. The CUDA benchmark jobs in
the same file are already restricted this way.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* ci: give the Java publish dry-run jobs a runner on forks

The matrix pins runners from upstream's pool, so on a fork both jobs queue
forever and the workflow never finishes. Every other job that pins those
runners is restricted to upstream; this one is not, and its steps already
branch on the repository, so it is meant to run here.

Fall back to a GitHub-hosted runner. The target comes from cross-compiling
with zigbuild, so an x64 runner serves the aarch64 entry too.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* ci: raise timeouts that assume a warm prebuild image

Three jobs are budgeted for upstream's prebuild image and its compile cache,
and are cancelled at the limit on a fork that builds from scratch: the C++ API
job and the two FFI sanitizer jobs at five minutes, and the Rust sanitizer
matrix at thirty, where -Zbuild-std rebuilds core and std as well.

Raise the limit only where the image is unavailable, so upstream keeps the
tighter budget and still notices if one of these gets slower.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* fix: make fork CI reach the same result as upstream

The coverage job now runs and one restored test fails there. It requires every
file after the first to be near the target size, but the last file holds
whatever remained when the input ended, so its size depends on how the input
divides rather than on the target. Check the files between the first and the
last, and sort them so the comparison does not depend on listing order.

Looking for llvm-symbolizer only on the rustup path and in PATH found nothing
on a GitHub-hosted runner, so the sanitizer jobs failed at the check that is
supposed to guarantee suppressions can match. Search the toolchain, the
versioned system binaries, and install llvm if none of those has it.

Benchmark setup still aborted, at the loop that disables background units:
irqbalance is absent on a GitHub-hosted runner and the redirect hid its output
but not its exit status.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* fix: give each thread its own row-count out-param, and align slow test and doc expectations

The FFI scan test spawns a thread per file and each one passes the enclosing
vx_estimate to vx_partition_row_count, so every thread writes the same struct
on the main thread's stack. ThreadSanitizer reports the race, and the
assertions are unsound for the same reason: a thread can compare against a row
count another thread produced. Give each thread its own out-param.

compress_large_int is already listed in the nextest config as slow. Under
coverage instrumentation on a runner without a compile cache it reaches the
default cap and is terminated, so raise the cap for the two tests that entry
already covers.

The io write-options docstring states exact file sizes. This branch writes
eight bytes more than the sizes recorded there, so the doctest fails once it is
reachable; record what this branch produces.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* ci: raise the coverage job timeout on forks

An instrumented build and run of the whole workspace was cancelled just past
thirty minutes on a GitHub-hosted runner. Upstream fits the budget because it
starts from the prebuild image with a warm compile cache.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

* ci: scope the raised test cap to the coverage job, and stop the schema check self-comparing

The longer slow-test cap was on the default nextest profile, so every run,
including upstream's, tolerated ten minutes of stall before killing a test
rather than two and a half. Only the coverage job needs it: instrumented,
compress_large_int runs close to the cap, while the Windows and musl runs of the
same test finish in about a minute. Move it to a profile that job selects, so
everything else keeps the tighter cap and the hang signal it provides.

The flatbuffer base ref fell back to the current branch when no pull request
base was available, which made a manual run compare a branch against itself and
always pass. Fall back to develop instead, and use the current branch only for
pushes, which is where it is the branch being merged into.

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>

---------

Signed-off-by: "Phillip LeBlanc" <879445+phillipleblanc@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants