Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 12 additions & 66 deletions .github/workflows/ci-and-release-tag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ name: CI and Release
on:
pull_request:
push:
# Releases are manual: run this workflow from `main` in the Actions tab.
workflow_dispatch:

env:
CARGO_TERM_COLOR: always
Expand Down Expand Up @@ -46,7 +48,7 @@ jobs:
release-tag:
name: Release Tag
needs: ci
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.event.deleted == false
if: github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
Expand All @@ -61,72 +63,20 @@ jobs:
fetch-depth: 0
fetch-tags: true

- name: Compare release dependency versions
id: versions
shell: bash
env:
BEFORE: ${{ github.event.before }}
run: |
set -euo pipefail

current_tag="$(python3 scripts/version_pair.py tag)"
zero_sha="0000000000000000000000000000000000000000"
if [[ "${BEFORE}" != "${zero_sha}" ]] && ! git cat-file -e "${BEFORE}^{commit}" 2>/dev/null; then
git fetch --no-tags --depth=1 origin "${BEFORE}" || true
fi

if [[ "${BEFORE}" == "${zero_sha}" ]] || ! git cat-file -e "${BEFORE}:Cargo.lock" 2>/dev/null; then
echo "initial release tag: ${current_tag}"
{
echo "release_needed=true"
echo "old_tag="
echo "new_tag=${current_tag}"
echo "changes=initial"
} >> "$GITHUB_OUTPUT"
else
git show "${BEFORE}:Cargo.lock" > /tmp/previous-Cargo.lock
version_output="$(mktemp)"
python3 scripts/version_pair.py changed \
--old /tmp/previous-Cargo.lock \
--new Cargo.lock \
--github-output "${version_output}"

release_needed="$(awk -F= '$1 == "release_needed" { print $2 }' "${version_output}")"
if [[ "${release_needed}" == "false" ]] && ! git rev-parse -q --verify "refs/tags/${current_tag}" >/dev/null; then
echo "current release tag missing: ${current_tag}"
{
echo "release_needed=true"
echo "old_tag=${current_tag}"
echo "new_tag=${current_tag}"
echo "changes=missing-tag"
} >> "$GITHUB_OUTPUT"
else
cat "${version_output}" >> "$GITHUB_OUTPUT"
fi
fi

- name: Create release tag
if: steps.versions.outputs.release_needed == 'true'
shell: bash
env:
TAG: ${{ steps.versions.outputs.new_tag }}
run: |
set -euo pipefail

head_commit="$(git rev-parse HEAD)"
if git rev-parse -q --verify "refs/tags/${TAG}" >/dev/null; then
tag_commit="$(git rev-list -n 1 "${TAG}")"
if [[ "${tag_commit}" == "${head_commit}" ]]; then
echo "Tag ${TAG} already exists at HEAD."
exit 0
fi

echo "Tag ${TAG} already exists at ${tag_commit}, not HEAD ${head_commit}." >&2
exit 1
tag="$(python3 scripts/version_pair.py tag)"
if git rev-parse -q --verify "refs/tags/${tag}" >/dev/null; then
# Keep an existing tag where it is so it always names the same source.
echo "Tag ${tag} already exists at $(git rev-list -n 1 "${tag}"); reusing it."
exit 0
fi

git tag "${TAG}"
git push origin "refs/tags/${TAG}"
git tag "${tag}"
git push origin "refs/tags/${tag}"

- name: Select binary release
id: release
Expand All @@ -148,11 +98,7 @@ jobs:
release-build:
name: Release build (${{ matrix.target }})
needs: release-tag
# Tag pushes made with GITHUB_TOKEN do not trigger another workflow.
# Exercise the same packaging on PRs without publishing a release.
if: >-
always() && !cancelled() &&
(github.event_name == 'pull_request' || needs.release-tag.outputs.publish == 'true')
if: needs.release-tag.outputs.publish == 'true'
strategy:
fail-fast: false
matrix:
Expand All @@ -170,7 +116,7 @@ jobs:
- name: Checkout release source
uses: actions/checkout@v6
with:
ref: ${{ needs.release-tag.outputs.commit || github.sha }}
ref: ${{ needs.release-tag.outputs.commit }}
persist-credentials: false

- name: Install Rust
Expand Down
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,6 @@ For reproducible benchmarks, download a specific [release](https://github.com/vo

Release tags use the format `<vortex-version>-<df-version>`, where `vortex-version` is the resolved `vortex-datafusion` crate version and `df-version` is the resolved `datafusion`/`datafusion-cli` crate version.

Pushes to `main` create the current tag if it does not already exist. Later merges to `main` create a new tag only when `Cargo.lock` changes either of those resolved versions. Code, docs, or unrelated dependency changes still run CI, but do not create release tags once the current tag exists.

After CI passes on `main`, the workflow builds binaries from the current tag's commit and publishes a GitHub Release with both archives and their SHA-256 checksums. An existing tag without a published release is also packaged, so the first release does not require a dependency update. Published releases are left unchanged. Failed builds or draft uploads can be retried by rerunning the workflow; publication waits for both platforms to succeed. Pull requests build and smoke-test the same archives without publishing them.
Pull requests and pushes run CI only. Releases are manual: in the Actions tab, run the **CI and Release** workflow on `main`. After CI passes it computes the tag from `Cargo.lock`, creates it at the `main` commit if it does not exist yet (an existing tag is kept where it is), then builds binaries from the tagged commit and publishes a GitHub Release with both archives and their SHA-256 checksums. Published releases are left unchanged, so re-running is safe; a failed build or draft upload can be retried by running the workflow again, and publication waits for both platforms to succeed.

[Renovate](https://docs.renovatebot.com/) updates Cargo manifests, `Cargo.lock`, and GitHub Actions workflow action versions. Vortex, DataFusion, and DataFusion CLI updates are grouped together, including major upgrades, so their integration uses compatible DataFusion versions.
Loading