Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,25 @@ jobs:
name: Resolve versions
runs-on: ubuntu-latest
outputs:
version: ${{ steps.meta.outputs.version }}
sdk: ${{ steps.meta.outputs.sdk }}
ruff: ${{ steps.meta.outputs.ruff }}
ruff: ${{ steps.pyproject.outputs.ruff }}
steps:
- uses: actions/checkout@v5

- uses: actions/setup-python@v6
with:
python-version: '3.11'

- name: Read pyproject.toml
id: meta
run: python scripts/release_meta.py >> "$GITHUB_OUTPUT"
id: pyproject
run: |
python3 - <<'PY' >> "$GITHUB_OUTPUT"
import tomllib
project = tomllib.load(open("pyproject.toml", "rb"))["project"]
print(f"version={project['version']}")
print("ruff=" + next(s for s in project["optional-dependencies"]["dev"] if s.startswith("ruff")))
PY

- id: meta
uses: cubert-hyperspectral/cuvis.docker/.github/actions/release-meta@main
with:
version: ${{ steps.pyproject.outputs.version }}

lint:
name: Lint
Expand Down Expand Up @@ -60,12 +66,7 @@ jobs:
with:
fetch-depth: 0

- uses: actions/setup-python@v6
with:
python-version: '3.11'

- name: Validate CHANGELOG.md structure
run: python scripts/check_changelog.py
- uses: cubert-hyperspectral/cuvis.docker/.github/actions/changelog@main

- name: Require a changelog entry
if: >
Expand Down
130 changes: 75 additions & 55 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
name: Release

# Pushing a vMAJOR.MINOR.PATCH.TWEAK tag on main releases that commit.
# workflow_dispatch runs the same validation, build and TestPyPI publish as a
# dry run, and stops before PyPI.
# Pushing a vGENERATION.MAJOR.MINOR.PATCH tag on main releases that commit: sdist and wheel to
# TestPyPI, then PyPI, then a GitHub Release with the CHANGELOG.md section as notes.
# A pre-release tag (v3.6.0.0rc1) publishes to PyPI as a pre-release and creates no GitHub Release.
on:
push:
tags: ['v*.*.*.*']
workflow_dispatch:

concurrency:
group: release
Expand All @@ -22,8 +21,8 @@ jobs:
outputs:
version: ${{ steps.meta.outputs.version }}
sdk: ${{ steps.meta.outputs.sdk }}
ruff: ${{ steps.meta.outputs.ruff }}
is_tag: ${{ startsWith(github.ref, 'refs/tags/') }}
prerelease: ${{ steps.meta.outputs.prerelease }}
ruff: ${{ steps.pyproject.outputs.ruff }}
steps:
- uses: actions/checkout@v5
with:
Expand All @@ -33,44 +32,30 @@ jobs:
with:
python-version: '3.11'

- name: Restrict manual dry runs to repository admins
if: github.event_name == 'workflow_dispatch'
uses: actions/github-script@v7
with:
script: |
const { data } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: context.actor,
});
if (data.permission !== "admin") {
core.setFailed(`Only repository admins may publish. (${context.actor} has: ${data.permission})`);
}

- name: Read pyproject.toml
id: meta
run: python scripts/release_meta.py >> "$GITHUB_OUTPUT"

- name: Verify the tag points at a commit on main
if: startsWith(github.ref, 'refs/tags/')
id: pyproject
run: |
git fetch --no-tags origin main
if ! git merge-base --is-ancestor HEAD origin/main; then
echo "Tag ${GITHUB_REF_NAME} is not an ancestor of main."
echo "Releases are cut from main; merge the release pull request first."
exit 1
fi
python - <<'PY' >> "$GITHUB_OUTPUT"
import tomllib
project = tomllib.load(open("pyproject.toml", "rb"))["project"]
print(f"version={project['version']}")
print("ruff=" + next(s for s in project["optional-dependencies"]["dev"] if s.startswith("ruff")))
PY

- id: meta
uses: cubert-hyperspectral/cuvis.docker/.github/actions/release-meta@main
with:
version: ${{ steps.pyproject.outputs.version }}
tag: ${{ github.ref_name }}

- name: Verify tag, version and changelog agree
if: startsWith(github.ref, 'refs/tags/')
run: python scripts/check_changelog.py --tag "${GITHUB_REF_NAME}"
- uses: cubert-hyperspectral/cuvis.docker/.github/actions/tag-on-main@main

- name: Validate CHANGELOG.md structure
if: github.event_name == 'workflow_dispatch'
run: python scripts/check_changelog.py
- uses: cubert-hyperspectral/cuvis.docker/.github/actions/changelog@main
with:
version: ${{ steps.meta.outputs.base_version }}
prerelease: ${{ steps.meta.outputs.prerelease }}

- name: Verify the version is not published yet
if: startsWith(github.ref, 'refs/tags/')
env:
VERSION: ${{ steps.meta.outputs.version }}
run: |
Expand All @@ -89,13 +74,53 @@ jobs:
if released.get(version):
sys.exit(
f"cuvis=={version} is already on PyPI and cannot be replaced. "
"Bump TWEAK, move the CHANGELOG.md section, and tag again."
"Bump PATCH, move the CHANGELOG.md section, and tag again."
)
print(f"cuvis=={version} is not published yet.")
PY

- name: Verify the cuvis-il requirement agrees with the version
env:
VERSION: ${{ steps.meta.outputs.version }}
SDK: ${{ steps.meta.outputs.sdk }}
run: |
python -m pip install packaging
python - <<'PY'
import os, sys, tomllib
from packaging.requirements import Requirement
from packaging.version import Version

version, sdk = Version(os.environ["VERSION"]), os.environ["SDK"]
project = tomllib.load(open("pyproject.toml", "rb"))["project"]
spec = next(Requirement(d) for d in project["dependencies"] if d.startswith("cuvis-il")).specifier

if Version(sdk) not in spec:
sys.exit(f"cuvis-il{spec} does not accept {sdk}, the SDK this release targets.")

# PEP 440 sorts 3.6.0.0rc1 below 3.6.0, so only a bound that names a pre-release
# can reach one; naming one also opens the whole range to pre-releases, which a
# final release must not ship.
if bool(spec.prereleases) != version.is_prerelease:
opens = "opens" if spec.prereleases else "does not open"
sys.exit(
f"cuvis-il{spec} {opens} to pre-releases but {version} is "
f"{'a pre-release' if version.is_prerelease else 'final'}. "
f"A pre-release needs a bound naming one, as in >={sdk}a0; a final needs >={sdk}."
)
print(f"cuvis-il{spec} agrees with {version}.")
PY

- name: Verify the cuvis_pyil image for this SDK exists
env:
IMAGE: cubertgmbh/cuvis_pyil:${{ steps.meta.outputs.sdk }}-ubuntu24.04
run: |
docker manifest inspect "$IMAGE" > /dev/null || {
echo "${IMAGE} does not exist. Release cuvis.pyil for SDK ${{ steps.meta.outputs.sdk }} first."
exit 1
}

- name: Install ruff
run: python -m pip install "${{ steps.meta.outputs.ruff }}"
run: python -m pip install "${{ steps.pyproject.outputs.ruff }}"

- name: Check formatting and lint rules
run: |
Expand Down Expand Up @@ -184,24 +209,25 @@ jobs:
python-version: '3.11'

- name: Confirm the version is downloadable from TestPyPI
env:
VERSION: ${{ needs.validate.outputs.version }}
run: |
for attempt in 1 2 3 4 5 6; do
if python -m pip download --no-deps --dest /tmp/smoke \
--index-url https://test.pypi.org/simple/ \
"cuvis==${{ needs.validate.outputs.version }}"; then
"cuvis==${VERSION}"; then
ls -lh /tmp/smoke
exit 0
fi
echo "Not indexed yet, retrying in 30s (attempt ${attempt}/6)."
sleep 30
done
echo "cuvis==${{ needs.validate.outputs.version }} never appeared on TestPyPI."
echo "cuvis==${VERSION} never appeared on TestPyPI."
exit 1

publish-pypi:
name: Publish to PyPI
needs: [validate, publish-testpypi]
if: needs.validate.outputs.is_tag == 'true'
runs-on: ubuntu-latest
environment: pypi
permissions:
Expand All @@ -217,29 +243,23 @@ jobs:
github-release:
name: Create GitHub release
needs: [validate, publish-pypi]
if: needs.validate.outputs.prerelease == 'false'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v5

- uses: actions/setup-python@v6
with:
python-version: '3.11'

- uses: actions/download-artifact@v4
with:
name: distributions
path: dist/

- name: Extract the changelog section
id: notes
run: |
{
echo 'body<<CHANGELOG_SECTION'
python scripts/check_changelog.py --extract "${{ needs.validate.outputs.version }}"
echo CHANGELOG_SECTION
} >> "$GITHUB_OUTPUT"
- id: notes
uses: cubert-hyperspectral/cuvis.docker/.github/actions/changelog@main
with:
mode: extract
version: ${{ needs.validate.outputs.version }}

- uses: softprops/action-gh-release@v2
with:
Expand Down
11 changes: 7 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ All notable changes to the `cuvis` Python wrapper are documented here.
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
Entry wording follows the conventions in [CONTRIBUTING.md](CONTRIBUTING.md#changelog-conventions) - every API entry names the fully qualified symbol first, then states the change with one of the fixed predicates.

Versions are `MAJOR.MINOR.PATCH.TWEAK`.
`MAJOR.MINOR.PATCH` is the cuvis SDK release the wrapper targets; `TWEAK` counts wrapper-only revisions against that same SDK.
Versions are `GENERATION.MAJOR.MINOR.PATCH`.
`GENERATION.MAJOR.MINOR` is the cuvis SDK release the wrapper targets; `PATCH` counts wrapper-only revisions against that same SDK.
See [CONTRIBUTING.md](CONTRIBUTING.md#version-scheme) for the full scheme.

Entries for versions released before this file existed were reconstructed from the published PyPI artifacts and from an AST-level diff of the public `cuvis` API surface between the corresponding commits.
Expand All @@ -16,8 +16,10 @@ Pre-releases (`b*`, `rc*`) are not listed.
### Added

- `CI` - `.github/workflows/ci.yml` runs the test suite and a lint job enforcing `ruff check` and `ruff format --check` on every pull request and on every push to `develop` and `main`.
- `CI` - `.github/workflows/release.yml` is driven by `v*.*.*.*` tags: it validates the tag against `pyproject.toml` and against this file, builds, publishes to TestPyPI, and publishes to PyPI plus a GitHub Release after manual approval.
- `CI` - `scripts/check_changelog.py` validates this file's structure (header format, allowed section names, descending versions) and the tag/version/changelog agreement at release time.
- `CI` - `.github/workflows/release.yml` is driven by `v*.*.*.*` tags: it validates the tag against `pyproject.toml`, this file and the `cuvis_pyil` image, builds, publishes to TestPyPI, and publishes to PyPI plus a GitHub Release after manual approval.
Pre-release tags (`a`, `b`, `rc` suffix) publish to PyPI as pre-releases and create no GitHub Release.
- `CI` - the changelog structure and the tag/version/changelog agreement are validated by the shared actions in [cuvis.docker](https://github.com/cubert-hyperspectral/cuvis.docker).
- `CI` - the release refuses to publish when the `cuvis-il` requirement disagrees with the version: it must accept the SDK the version targets, and it may open to pre-releases only when the release is itself a pre-release.
- `CONTRIBUTING.md` - documents the branch model, the version scheme, the changelog conventions and the release checklist.
- `cuvis.BindingInfo` - new frozen dataclass with the fields `built_against: str`, `library_version: str`, `library_path: str` and `missing_symbols: Tuple[str, ...]`, the read-only property `is_complete: bool`, and a `__str__` rendering a report fit for a bug report.
- `cuvis.SdkSettings` - new class, a `MutableMapping` of setting id to value that writes the SDK's `cuvis.settings` file, so the SDK configuration can be built in Python instead of maintained by hand.
Expand All @@ -36,6 +38,7 @@ Pre-releases (`b*`, `rc*`) are not listed.
### Changed

- Whole tree reformatted with `ruff format`; no behaviour change.
- `pyproject.toml` - the `cuvis-il` requirement moved from `>=3.5.0,<3.6.0` to `>=3.6.0a0,<3.7.0`, so the wrapper targets cuvis SDK 3.6 and accepts the pre-releases of that SDK line.
- `README.md` - documents the version scheme, and lists Python 3.14 among the supported interpreters as `pyproject.toml` already did.
- `prebuild.py` - writes `cuvis/git-hash.txt` instead of `git-hash.txt` at the repository root, so the file lands inside the package that declares it as package data.
- `cuvis.General.init` - parameter `settings_path` type changed from `str` to `Union[str, Path, SdkSettings]`.
Expand Down
35 changes: 23 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ A pull request into `develop` or `main` must pass the `ci.yml` lint and test job

## Version scheme

Versions are `MAJOR.MINOR.PATCH.TWEAK`, always with all four components.
Versions are `GENERATION.MAJOR.MINOR.PATCH`, always with all four components, optionally followed by a pre-release suffix `a1`, `b1` or `rc1`.

- `MAJOR.MINOR.PATCH` is the cuvis SDK release this wrapper targets.
- `GENERATION.MAJOR.MINOR` is the cuvis SDK release this wrapper targets.
It is not chosen by the wrapper; it follows the SDK.
- `TWEAK` counts wrapper-only revisions against that same SDK release, starting at `0`.
- `PATCH` counts wrapper-only revisions against that same SDK release, starting at `0`.

Examples:

Expand All @@ -40,8 +40,15 @@ Two consequences worth knowing:

- PEP 440 treats `3.5.3.0` and `3.5.3` as the same version, so only one of the two forms may ever be published for a given release.
Tags created before this scheme was written down use the three-component form (`v3.5.3` is release `3.5.3.0`); everything from `v3.5.3.2` onward is four-component.
- A `TWEAK` bump never widens or narrows the `cuvis-il` requirement in `pyproject.toml`.
If the interface layer requirement changes, the SDK it targets changed, so the change belongs in a `MAJOR.MINOR.PATCH` release.
- A `PATCH` bump never widens or narrows the `cuvis-il` requirement in `pyproject.toml`.
If the interface layer requirement changes, the SDK it targets changed, so the change belongs in a `GENERATION.MAJOR.MINOR` release.
The one exception is the pre-release bound below, which is tied to the release being a pre-release rather than to the SDK.
- A pre-release opens the `cuvis-il` requirement at `a0`, as in `>=3.6.0a0,<3.7.0`; the final release closes it again to `>=3.6.0,<3.7.0`.
The `a0` is load-bearing.
PEP 440 sorts `3.6.0.0rc1` below `3.6.0`, so a `>=3.6.0` bound cannot reach a pre-release of that SDK at all, and naming a pre-release in the bound is also what permits `pip` to consider pre-releases without `--pre`.
The cost is that `pip` then prefers a pre-release of a later SDK in the same line, `3.6.1.0rc1`, over the current final `3.6.0.0`.
- A pre-release (`3.6.0.0rc1`) runs the whole release pipeline and reaches PyPI as a pre-release, which `pip` ignores unless asked for with `--pre` or an exact pin.
It creates no GitHub Release; its changelog entries stay under `## [Unreleased]` until the final version, whose section then covers everything since the previous final release.

The version lives in exactly one place: `[project].version` in `pyproject.toml`.
The git tag is `v` followed by that value, and the release workflow refuses to publish when the two disagree.
Expand Down Expand Up @@ -70,7 +77,7 @@ Widening it is a separate, self-contained pull request, never a side effect of a
## Changelog conventions

Every user-visible change is recorded in `CHANGELOG.md` under `## [Unreleased]` in the same pull request that makes the change.
The file follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and is validated by `scripts/check_changelog.py`, which CI runs on every pull request.
The file follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and is validated on every pull request by the shared `changelog` action from [cuvis.docker](https://github.com/cubert-hyperspectral/cuvis.docker).

### Structure

Expand Down Expand Up @@ -123,6 +130,9 @@ The git history already records those, and they say nothing about the API.

## Releasing

Releases run in a fixed order across repositories; see the [cuvis.docker README](https://github.com/cubert-hyperspectral/cuvis.docker#release-order).
The `cuvis_pyil:<sdk>-ubuntu24.04` image must exist before cuvis.python can release for that SDK; the workflow checks and stops otherwise.

### One-time repository setup

The release workflow depends on settings that live outside the repository:
Expand All @@ -139,10 +149,11 @@ The release workflow depends on settings that live outside the repository:
### Regular release from `develop`

1. On `develop`, confirm which SDK version the wrapper targets and that `cuvis-il` in `pyproject.toml` matches it.
2. Rename `## [Unreleased]` to `## [X.Y.Z.W] - <today>` and add the SDK statement lines beneath it.
2. For a final version rename `## [Unreleased]` to `## [X.Y.Z.W] - <today>` and add the SDK statement lines beneath it.
Add a fresh empty `## [Unreleased]` above it.
3. Set `[project].version` in `pyproject.toml` to `X.Y.Z.W`.
4. Run `ruff format --check . && ruff check . && pytest && python scripts/check_changelog.py`.
For a pre-release leave the entries under `## [Unreleased]`.
3. Set `[project].version` in `pyproject.toml` to `X.Y.Z.W` (or `X.Y.Z.WrcN`).
4. Run `ruff format --check . && ruff check . && pytest`.
5. Open a pull request `develop` -> `main` titled `release: vX.Y.Z.W` and merge it once CI is green.
6. Tag the merge commit on `main` and push the tag:

Expand All @@ -152,15 +163,15 @@ The release workflow depends on settings that live outside the repository:
git push origin vX.Y.Z.W
```

7. `release.yml` validates the tag, builds, publishes to TestPyPI, and then waits for approval on the `pypi` environment before publishing to PyPI and creating the GitHub Release.
7. `release.yml` validates the tag, builds, publishes to TestPyPI, and then waits for approval on the `pypi` environment before publishing to PyPI and, for a final version, creating the GitHub Release.
8. Merge `main` back into `develop` so the release commit is an ancestor of both.

### Hotfix release from `main`

Same as above, except the branch is `hotfix/<slug>` cut from `main`, the pull request targets `main` directly, only `TWEAK` increases, and step 8 becomes mandatory rather than tidy-up.
Same as above, except the branch is `hotfix/<slug>` cut from `main`, the pull request targets `main` directly, only `PATCH` increases, and step 8 becomes mandatory rather than tidy-up.

### If a release goes wrong

A published PyPI version cannot be replaced.
Fix forward with the next `TWEAK`; yank on PyPI only when the artifact is actively harmful.
Fix forward with the next `PATCH`; yank on PyPI only when the artifact is actively harmful.
Delete the tag and re-tag only while the release workflow has not yet published anything.
Loading