Skip to content

Run SSL tests on each CI run #97923

Closed
Closed
Run SSL tests on each CI run#97923
Labels
testsTests in the Lib/test dirtopic-SSLtype-bugAn unexpected behavior, bug, or error

Description

@sobolevn
sobolevn
opened on Oct 5, 2022

Idea: #93369 (comment)

Recent PR broke SSL tests: ce8fc18

It is not the first when this happens.
The idea is to run SSL tests on each pull request, instead of the current scheme, when tests are run for each push event on main, but not in PRs.

I will send a PR with the fix today :)

Activity

  1. sobolevn
    added
    type-bugAn unexpected behavior, bug, or error
    topic-SSL
    on Oct 5, 2022
  2. AlexWaygood
    added
    testsTests in the Lib/test dir
    on Oct 5, 2022
  3. sobolevn
    added a commit that references this issue on Oct 5, 2022

    pythongh-97923: Always run Ubuntu SSL tests with others in CI

    37d4f5e

  4. vstinner commented on Oct 6, 2022

    @vstinner
    vstinner
    on Oct 6, 2022
    Member

    If I recall correctly, skipping SSL tests was decided to make the workflow faster, and to not waste CI resources.

  5. ambv commented on Oct 7, 2022

    @ambv
    ambv
    on Oct 7, 2022
    Contributor

    Let's try it. Missing SSL test failures in PRs is a bigger problem than wasted resources. If the queues get too long we can always revert to what we have now.

  6. ambv
    added a commit that references this issue on Oct 7, 2022

    gh-97923: Always run Ubuntu SSL tests with others in CI (#97940)

    3108fc1

  7. kumaraditya303 commented on Oct 8, 2022

    @kumaraditya303
    kumaraditya303
    on Oct 8, 2022
    Contributor

    What's the reasoning behind running the tests with warnings turned into exceptions? What is the benefit of it? It causes a lot of issue when adding any warning in asyncio. No other tests are run with -Werror, IMO it should be removed.

    cc @gvanrossum

  8. sobolevn commented on Oct 8, 2022

    @sobolevn
    sobolevn
    on Oct 8, 2022
    MemberAuthor

    СС @tiran

  9. arhadthedev commented on Oct 8, 2022

    @arhadthedev
    arhadthedev
    on Oct 8, 2022
    Member

    It causes a lot of issue when adding any warning in asyncio. No other tests are run with -Werror, IMO it should be removed.

    In my case, a deprecation warning revealed that I accidentally broke a test in such a way that it gave no exception:

    DeprecationWarning: It is deprecated to return a value!=None from a test case (<bound method StreamTests2.test_eof_feed_when_closing_writer of <test.test_asyncio.test_streams.StreamTests2 testMethod=test_eof_feed_when_closing_writer>>)

    So we probably should do the opposite: investigate and fix all flukes then enable -Werror for all tests.

  10. carljm
    added a commit that references this issue on Oct 8, 2022

    Merge branch 'main' into typewatch

    27a4980
  11. mpage
    added a commit that references this issue on Oct 11, 2022

    pythongh-97923: Always run Ubuntu SSL tests with others in CI (python…

    dc2c10e

  12. kumaraditya303 commented on Nov 29, 2022

    @kumaraditya303
    kumaraditya303
    on Nov 29, 2022
    Contributor

    Fixed by #97940

  13. kumaraditya303
    closed this as completedon Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtopic-SSLtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions