Skip to content

build: modernize packaging and development tooling - #511

Open
itskyf wants to merge 5 commits into
NVIDIA:mainfrom
itskyf:chore/modernize-packaging-tooling
Open

itskyf wants to merge 5 commits into
NVIDIA:mainfrom
itskyf:chore/modernize-packaging-tooling

Conversation

@itskyf

@itskyf itskyf commented Oct 12, 2026 •

Copy link
Copy Markdown

What does this PR do?

Modernize packaging, dependency management, and development tooling while keeping tests working.

  • Migrate to pyproject.toml and uv for dependency management, builds, and CI.
  • Replace Black, isort, and Flake8 with Ruff; add prek and Tombi.
  • Update dependencies and replace the unmaintained editdistance package with RapidFuzz.
  • Fix Hindi and Kinyarwanda cache handling and the Sparrowhawk Docker build to ensure tests run correctly.
  • Update installation instructions and require Python 3.11+.
  • Edit to unstale.

Pre-checks

  • All commits are signed off.
  • Pytest tests completed successfully.
  • Sparrowhawk tests completed successfully.

- Migrate build configuration from setup.py, setup.cfg, and requirements/
  to PEP 517/PEP 621 pyproject.toml with setuptools build backend.
- Configure standalone pytest.toml for pytest configuration.
- Format TOML files with tombi.
- Set requires-python = ">=3.11".
- Upgrade dependencies to latest versions and specify explicit ">=" bounds.
- Prune obsolete, redundant, and unused dependencies (wget, wrapt,
  pytest-runner, ruamel-yaml, wandb, duplicate pynini, sphinx packages).
- Remove stray root __init__.py and properly scope package discovery to
  nemo_text_processing to prevent shipping test suite in wheels.
- Update Jenkinsfile with triple single-quoted block using pip install --upgrade uv==0.12.17
  and uv sync --locked.
- Update README installation instructions to use uv.
- Update pre-commit hook versions and remove requirements-txt-fixer.
- Commit uv.lock.

Signed-off-by: Kỳ <40734986+itskyf@users.noreply.github.com>
- Replace Black, isort, and Flake8 with Ruff using standalone ruff.toml.
- Set line-length = 119 to preserve existing formatting style.
- Set lint.select = ["I", "W"] to check import sorting and whitespace warnings.
- Set format.quote-style = "preserve" to retain existing string quotation conventions.
- Update .pre-commit-config.yaml to use astral-sh/ruff-pre-commit.
- Update pyproject.toml test dependencies to include ruff and remove black/isort.
- Perform harmless cleanup of trailing whitespace and blank lines to satisfy I and W rules without broad reformatting.
- Format TOML files with tombi.
- Update uv.lock.

Signed-off-by: Kỳ <40734986+itskyf@users.noreply.github.com>
- Add prek to dev dependencies in pyproject.toml.
- Add tombi-toml/tombi-pre-commit (v1.5.5) hook to .pre-commit-config.yaml.
- Run prek update to update configured pre-commit repositories.
- Format TOML files with tombi.
- Update uv.lock.

Signed-off-by: Kỳ <40734986+itskyf@users.noreply.github.com>
- Replace unmaintained editdistance dependency with rapidfuzz >= 3.14.6.
- Update calculate_cer in normalize_with_audio.py to use rapidfuzz.distance.Levenshtein.distance.
- Format TOML files with tombi.
- Update uv.lock.

Signed-off-by: Kỳ <40734986+itskyf@users.noreply.github.com>
- Fall back to 'ALL' key when restoring verbalizer cache in Kinyarwanda (rw)

- Fix verbalizer cache filename prefix typo from en_tn to hi_tn in Hindi (hi)

- Add --no-same-owner to tar in Sparrowhawk Dockerfile to fix rootless build error

Signed-off-by: Kỳ <40734986+itskyf@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@github-actions github-actions Bot added Stale and removed Stale labels Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant