Repository navigation
Enforce 100% test coverage #316
Description
As previously commented we should enfoce a 100% test coverage while testing locally and in CI.
As previously commented we should enfoce a 100% test coverage while testing locally and in CI.
Activity
lovelydinosaur commentedon Dec 3, 2019
I'm really, really keen on us getting this back.
Dropping the 1.0 milestone tho, since it's not strictly required for us to get to a point where we consider HTTPX to be API-stable.
yeraydiazdiaz commentedon May 13, 2020
I'd like us to revisit this, IMO we should at least aim to have 100% and if in some situations we can't then we mark as
nocoverand explain why.Thoughts?
lovelydinosaur commentedon May 13, 2020
Yup, v keen on this.
yeraydiazdiaz commentedon May 15, 2020
In order to get this going I think we should start enforcing the coverage not to drop on each commit.
I seem to remember we used to have a codecov but I'm not sure when and why we dropped that?
lovelydinosaur commentedon May 15, 2020
We used to have 100% enforced, and somewhere way back that seemed to get accidentally dropped.
After that we didn't have any enforcement, and I probably dropped codecov when switching across to GitHub actions, as a potential "follow up" action, that I didn't want to block us being able to move to GitHub actions.
I'd probably be quite keen on instead enforcing it ourselves instead. It's not ideal while we're at <100% because we can probably only enforce "this percent and no less", but it does mean that we can make sure that running the tests locally ensure compliance, rather than having to wait for the cycle of "submit PR", "test suite fails", "maintainer points out why", "contributor adds coverage".
Also it's one less moving part, and I've found codecov to just end up being a bit invasive compared to what I'd like to see.
Not absolute on this, so could consider it, or could consider having both local compliance plus "send the results to codecov on CI".
yeraydiazdiaz commentedon May 15, 2020
I agree it would be best to have it locally.
I guess it's not too bad to enforce it via pytest-cov's
--cov-fail-under=$CURRENT_COV? We could bump the value manually ourselves as the coverage increases. A quick search hasn't yielded any obvious way of doing this automatically.lovelydinosaur commentedon May 15, 2020
A quick note on how I'd recommend we loop this all in...
Our GitHub actions have the lint stage nicely separated from the test stage, so we have
scripts/lintandscripts/teststeps, which helps us see at a glance which of the linting or the tests have failed.However we also want to ensure that contributors are having the linting checked when just running
scripts/test(It's too easy to forget to check the linting otherwise) so we automatically run the linting from the test script if we're not on GitHub actions...scripts/test:
I think we should do the same for coverage enforcement, so that we can have a GitHub actions step that just runs
scripts/coverage, help us see when it's the coverage step that's failed.Our test script will then look like this...
lovelydinosaur commentedon May 15, 2020
At a later point
scripts/coveragecould also collate results to CodeCov if it's running in CI, but either way we'll have the structure setup the way we want it.lovelydinosaur commentedon Jul 28, 2020
We're there now, excepting
URLLib3Transport, which we'd like to push into a third party repo.