Skip to content

Script cleanup #950

Closed
Closed

Description

@lovelydinosaur

Here's the scripts we use from our workflows...

Test Suite

Publish

  • scripts/install
  • scripts/build
  • scripts/publish

There's a handful of scripts left that we don't actually use in the workflows...

  • scripts/docs-build
  • scripts/docs-serve
  • scripts/clean
  • scripts/lint

I'd like to clean those up a bit, but figured I'd start this as a discussion rather than a PR.

  1. Given we've now got a nice build step, we could clean the two docs scripts into just scripts/docs, to use for serving the docs during development. That'll error on build failures anyway, and it feels like nicer naming. Potentially we could drop it altogether and just use mkdocs serve, but it's nice to be consistent here.
  2. We could consider dropping scripts/clean? I've personally now got a global gitclean command that I use in repos to tidy up everything out of source control or "venv". Is the existing script useful for other contributors/maintainers? If we do keep it then I think we should change it to git clean -d -x --force --exclude=venv which is a much cleaner simpler approach to cleaning out the gumpf, and will work across all our encode projects.
  3. The scripts/lint one is a bit odd, since it just replicates scripts/check but using in-place fix flags on the various commands. Is there a neater way for us to do this? Eg. Should we be running scripts/check --fix instead? Is that less convenient? Are there other good options here? If scripts/test fails the linting locally, perhaps we should just prompt the user with Running 'test/check' failed. I can fix this for you automatically. Do you want me to run 'scripts/check --fix' and retry? [y|n].

Activity

  1. florimondmanca commented on May 15, 2020

    @florimondmanca
    Contributor
    • Agreed with 1/
    • Agreed with 2/ too - scripts/clean sounds less useful now that we publish from CI, where the env is always "clean" anyway. (Also, I personally don't use the clean script.)
    • 3/ A --fix flag is what I've seen in a lot of places (eg in the JS ecosystem). One alternative I thought about is scripts/lint [--check], but scripts/check [--fix] fits my brain better for some reason. Dunno how easy ternary operators are in bash but if most tools have some sort of flag to switch them from "check" to "fix in place", then we could use that. :-)
  2. StephenBrown2 commented on May 15, 2020

    @StephenBrown2
    Contributor

    Chiming in, it probably doesn't matter too much, but I believe linters/formatters most often have a --check flag, which would map down easily from scripts/lint [--check] (Pass down --check to contained programs)

    But it may be beneficial to be non-destructive at first, and require an explicit flag to modify the system.

  3. florimondmanca commented on May 15, 2020

    @florimondmanca
    Contributor

    It's actually a mix and depends on the tool:

    Tool Check invocation Fix invocation
    seed-isort-config N/A seed-isort-config
    autoflake N/A autoflake --in-place
    flake8 flake8 N/A
    mypy mypy N/A
    Black black --check black
    isort isort --check --diff isort --apply

    So I suppose we could end up with a scripts/check [--fix] something like this…

    #!/bin/sh -e
    
    export PREFIX=""
    if [ -d 'venv' ] ; then
        export PREFIX="venv/bin/"
    fi
    
    export FIX=""; [ "$1" == --fix ] && FIX=true
    export SOURCE_FILES="httpx tests"
    
    BLACK_OPTS="--check --diff"; [ $FIX ] && BLACK_OPTS=""
    ISORT_OPTS="--check --diff"; [ $FIX ] && ISORT_OPTS="--apply"
    
    set -x
    
    if [ $FIX ]; then
        ${PREFIX}seed-isort-config --application-directories=httpx
        ${PREFIX}autoflake --in-place --recursive $SOURCE_FILES
    else
        ${PREFIX}flake8 $SOURCE_FILES
        ${PREFIX}mypy httpx
    fi
    
    ${PREFIX}black $BLACK_OPTS --target-version=py38 $SOURCE_FILES
    ${PREFIX}isort $ISORT_OPTS --project=httpx --recursive $SOURCE_FILES
  4. StephenBrown2 commented on May 15, 2020

    @StephenBrown2
    Contributor

    Ah, I see. black and isort were indeed the formatters I was thinking of, and I knew flake8 didn't fix anything with or without a flag.

    However, that shows that scripts/lint doesn't just replicate scripts/check, but instead runs different things with different flags. Nothing saying that couldn't be implemented as demonstrated though, since it would be only one place to modify a linter or formatter if needed, rather than two.

    Personally I would rename scripts/lint to scripts/format and scripts/check to scripts/lint, but as I said, having the flag is also 👍 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    toolingChanges to our CI/CD, tests setup, etc.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions