Skip to content

Test failures when hypothesis is available but cannot reach GitHub #132316

Description

@kulikjak

Bug report

Bug description:

Hi, we recently started seeing several tests (namely test_binascii, test_math_property, test_tools and test_zoneinfo) to fail due to hypothesis warnings (which were recently turned into errors with #131802):

hypothesis.errors.HypothesisWarning: Authorization failed when trying to download artifact from GitHub. Check that you have a valid GITHUB_TOKEN set in your environment.`

or alternatively with

hypothesis.errors.HypothesisWarning: Could not connect to GitHub to get the latest artifact.

when I restrict the access to the internet.

I see that running with "CI" set in the environment fixes the issue as the problematic profile with GitHubArtifactDatabase is not registered, but that seems more like a lucky workaround.

I am not sure about the solution - maybe it can check whether "GITHUB_TOKEN" is also available? Or maybe the warning should be ignored and not cause the tests to fail.

I also think it should honor -urlfetch and -network options.

CPython versions tested on:

3.13

Operating systems tested on:

Other

Linked PRs

Activity

  1. added
    infraCI, GitHub Actions, buildbots, Dependabot, etc.
    on Apr 9, 2025
  2. self-assigned this
    on Apr 9, 2025
  3. sobolevn commented on Apr 9, 2025

    @sobolevn
    Member

    Thanks for the report! Working on it.

  4. hugovk commented on Apr 9, 2025

    @hugovk
    Member

    If it's useful, GitHub Actions also sets a GITHUB_ACTIONS variable:

    Always set to true when GitHub Actions is running the workflow. You can use this variable to differentiate when tests are being run locally or by GitHub Actions.

    https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/store-information-in-variables#default-environment-variables

  5. sobolevn commented on Apr 10, 2025

    @sobolevn
    Member

    Docs right now state: https://hypothesis.readthedocs.io/en/latest/reference/api.html#hypothesis.database.GitHubArtifactDatabase

    You must provide GITHUB_TOKEN as an environment variable. In CI, Github Actions provides this automatically, but it needs to be set manually for local usage. In a developer machine, this would usually be a Personal Access Token. If the repository is private, it’s necessary for the token to have repo scope in the case of a classic token, or actions:read in the case of a fine-grained token.

    We need GITHUB_TOKEN for hypothesis to work properly, otherwise it may fail with:

    { 
      "message":"API rate limit exceeded for YOUR_API. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
      "documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"
    }
  6. added 2 commits that reference this issue on Apr 10, 2025
  7. hugovk commented on Apr 10, 2025

    @hugovk
    Member

    Thanks both for the report and the PR!

  8. kulikjak commented on Apr 10, 2025

    @kulikjak
    ContributorAuthor

    Thank you for the fix!

    I verified that with this patch all the previously failing tests pass again.

  9. added a commit that references this issue on Apr 10, 2025
  10. added a commit that references this issue on Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

infraCI, GitHub Actions, buildbots, Dependabot, etc.type-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions