Repository navigation
Switching to anyio as the default backend when running under asyncio. #344
Description
Having been bitten by some rough edges in asyncio's SSL support, and having taken a look over anyio's TLSStream implementation. (Which just plain makes sense *) I'm now very warm on the idea of us switching over to anyio as the default backend for the asyncio case.
We could consider deprecating and later removing the native curio.py and asyncio.py modules, but we don't necessarily need to do that to start with.
First step here would be switching the default, and issuing a release, in order to make sure we hit any unexpected issues that might crop up as a result.
Paging @agronholm. 😀
*: In contrast to this, what the heck is loop.start_tls? Why is that a property of the event loop? Etc.
Activity
agronholm commentedon May 24, 2021
Hooray! 💯
As for curio support, you would have to keep that in anyways since AnyIO has dropped the curio backend (at the request of curio's author).
Is there anything you need from me?
lovelydinosaur commentedon May 24, 2021
Okay. Just for context, is that a private request, or on a public thread?
Up to you. I'm happy to put in the PR myself, since I've got paid OSS time which I can use to do it, but equally if you were keen to get it in because it's your baby, then you're also welcome to take it on yourself. 🤷♂️😁
Aside: It's interesting to me that Trio's TLSStream implementation has so much more complex locking than the anyio case. I started to have a bit of a deeper dig into that, but just mentioning it here for now.
agronholm commentedon May 24, 2021
agronholm/anyio#185
I looked at the trio implementation before I made my own; I just couldn't figure out why it was so damn complex.
agronholm commentedon May 24, 2021
I'll try to get a PR done this week.
lovelydinosaur commentedon May 24, 2021
Noting it here while I remember - we almost certainly want to set Trio's
https_compatible=Trueand AnyIO'sstandard_compatible=Falseto theSSLStreamconstructors.Also
AsyncHTTPProxyis not currently passing though thebackendargument toAsyncHTTPConnectioninstances, so setting eg.AsyncHTTPProxy(..., backend="anyio|auto|asyncio")does not currently work as expected. Nobody has bumped into that because, well, everyone's just using"auto", because of course they are.agronholm commentedon May 24, 2021
The AnyIO backend has done this from the start IIRC, but the trio backend currently doesn't.
agronholm commentedon May 24, 2021
One question remains – is it okay to require the latest anyio as a hard dependency? if so, I'll remove the compatibility code from the backend then.
agronholm commentedon May 24, 2021
One other thing I noticed – all the async tests seem to be running only on trio, or am I missing something?
agronholm commentedon May 24, 2021
Just most of them actually – they're marked with
@pytest.mark.triowhile others are marked@pytest.mark.anyio.agronholm commentedon May 24, 2021
Is there any reason at all to use
pytest-triohere?lovelydinosaur commentedon May 25, 2021
Yes. Tho question - are there any API differences between 3.0 and 3.1 that impact us? If we can depend on 3.x then let's do that by preference, otherwise, sure 3.1+.
Not sure without digging, but let's not change anything there as part of any initial PR here.
agronholm commentedon May 25, 2021
AnyIO uses semantic versioning so v3.1 is API compatible with v3.0. It's that httpcore's AnyIO backend was made for AnyIO v2.0 originally, and if anybody requires the older major release then this bump might negatively affect them. My other main concern was adding AnyIO as a hard dependency, regardless of version.
If indeed most tests are only run on trio and not the other backends then this needs to be fixed ASAP, even if in a separate PR.
lovelydinosaur commentedon May 25, 2021
Well, yes and no. We don't always need to exercise down to "repeat this test with each individual backend". Eg. if we're testing some connection pooling behaviour, or some part of HTTP/2 fiddlieness, or suchlike, then it's not a terrible thing to just test that against one given backend, so long as we do also have sufficient coverage over testing each of the backends in general. So, mixed bag, but in either case let's treat any convo there as a separate thing.