Drop certifi, use system trust store by default

This is an issue that many people have been trying to tackle for Python but hasn't been done yet. I think we're in a pretty good position to try to tackle this in a way that can be available to everyone.

Linux/OpenSSL should use the default system CA bundle path compiled into OpenSSL.
Windows should use SChannel which is capable of fetching updated CA certs if they're not in the trust store.
macOS should use SecureTransport

This should probably be implemented as a separate library, maybe use a stripped-down oscrypto project as a starting point.

Reasons to do this:

HTTPX will use the same CA certs as the system
Easier configuration and deployment to corporate settings
If a system is shipped with outdated OpenSSL (Windows, macOS) we don't care because the system is more likely to be up to date than whatever was shipped with Python.
Windows and macOS automatically update their certs and CRLs.

It's been a while since this was raised has been brought up recently a couple of times, could we go into detail of what's needed here?

For reference oscrypto lists other related libraries, would any of those fit our needs? (I'd look myself but I just don't know enough about the subject 🙂 )

Hey!

So considering the amount of 👍's on this issue I thought it would make sense to take some kind of decision on this as part of 1.0 - #947 (comment)

I'm personally on the side of dropping certifi as well, mainly because 1/ it's an extra dependency and 2/ it's probably better for certifi to be opt-in (e.g. we can hint it in the Requests migration guide) rather than opt-out. Other reasons mentioned by Seth in the issue description also apply.

I'm pretty much a n00b in terms of TLS implementations, let alone system compatibility, so let me ask what I feel might be a n00b question but maybe not after all…

What if we just used ssl.create_default_context()?

From the docs (emphasis mine):

Return a new SSLContext object with default settings for the given purpose. The settings are chosen by the ssl module, and usually represent a higher security level than when calling the SSLContext constructor directly. (So it's supposed to be secure enough choice, right?)

cafile, capath, cadata represent optional CA certificates to trust for certificate verification, as in SSLContext.load_verify_locations(). If all three are None, this function can choose to trust the system’s default CA certificates instead.

TL;DR: it looks to me ssl.create_default_context() should* create an ssl_context that uses the system defaults. Is there anything else we'd need? Or would we not be considering this because it's not the always most secure choice everywhere?

(*The usage of "can" in the docs looks suspicious to me. Are there cases when Python wouldn't trust the default system CA? And what happens if it doesn't? Would the default ssl_context then be a "no verify" context?)

I'm wondering if this wouldn't be an okay default, mainly because two notable projects use and/or recommend the use of ssl.create_default_context() for configuring TLS:

@Lukasa

Not my core expertise, but pretty sure it's not a solved issue.

I sat down with @Lukasa last year briefly, and one of the technical issues we talked about was dropping certifi, and the complexities of accessing the system trust store on different O/S's.

I'm sure @sethmlarson has a much better handle on the current state of affairs here. I think either @tiran's or @glyph's name might also have come up in my conversation with Cory, wrt. folks having done some work in this area, but I might be getting that wrong. Kinda a PyCon type thing that would've benefited from being able to chat it out with relevant folks, but...

In any case, assuming I have got the landscape correct here, I think "use system trust store by default" sits firmly in "make this work in an independent, tightly-scoped third party package", rather than specifically "solve this in httpx".

If and when such a package exists, then yup we can take a look at using it, otherwise it's in the realm of "this isn't a resolved issue in the Python ecosystem, certifi is the best we can do until then".

Just found out about this discussion starting back from 2016 in the Requests repo, loads of good background there: psf/requests#2966

Edit, after reading it:

it's in the realm of "this isn't a resolved issue in the Python ecosystem, certifi is the best we can do until then".

Agreed.

(Also I think I found an answer there to "Wouldn't create_default_context() be enough?", which is "yes, but only on *nix, and only if OpenSSL is linked correctly". We already support passing a preconfigured ssl_context as verify, so we probably cover enough ground there already.)

I need to finish my prototype...

Another data point in favor of us keeping certifi at this time: aiohttp not bundling certs by default seems to have been causing users a lot of pain… https://github.com/aio-libs/aiohttp/issues?q=is%3Aissue+verify+failed+

In a lot of these issues the solutions are often a mix of "disable cert validation" or "use certifi".

Eg aio-libs/aiohttp#955

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960869 should help to solve a bunch of problems. Fedora / RHEL should have root CA certificates installed. I'll verify UBI mini tomorrow.

@tiran

@tiran How far along is that prototype? 😄

A short-term solution might be to support pulling a CA bundle from environment variables (requests also supports this). This gives a knob for system administrators to make httpx behave.

On the prior art pile, urllib3 supports SecureTransport, but not SChannel.

EDIT: Oh, they stole code from https://github.com/wbond/oscrypto which looks a lot more complete

Since this hasn't been mentioned in the thread, TLS has several knobs with regards to security, the big ones being algorithm selection and server certificate policies. Per the docs:

The settings are: PROTOCOL_TLS, OP_NO_SSLv2, and OP_NO_SSLv3 with high encryption cipher suites without RC4 and without unauthenticated cipher suites. Passing SERVER_AUTH as purpose sets verify_mode to CERT_REQUIRED and either loads CA certificates (when at least one of cafile, capath or cadata is given) or uses SSLContext.load_default_certs() to load default CA certificates.

This is basically drops the horribly insecure options and presents an ok default. (I am not enough of a security expert to say how good these defaults are.) In general, OpenSSL is complex, probably more complex than necessary.

Also, a moment of silence for PEP 543.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@sethmlarson

So @sethmlarson's truststore looks like it's mature now.
Worth considering the switch, perhaps?

Truststore is used by pip and PDM, so should be good to adopt if you are still interested. Still requires Python 3.10 or later, though. We also already have integration tests for HTTPX specifically: https://github.com/sethmlarson/truststore/blob/main/tests/test_inject.py#L116

Yup, very much interested.

Still requires Python 3.10 or later, though.

Okay, that's a consideration for us, tho we could potentially have a fallback to certifi for 3.8 and 3.9.

Truststore has also been integrated into pip 24.2+ as the default method for verifying HTTPS certificates

I knew truststore was an option in pip, I didn't realise it was now on-by-default. Impressive.

I would love if it became the default on 3.10+ for v1.0! It's what I plan to do for Hatch anyway so it would save me time and it's what is desirable everywhere I've ever worked.

It's worth noting that the Rust community is also going in the direction of making system certificate verification the default with this new package that everyone is transitioning to https://github.com/rustls/rustls-platform-verifier

It's worth noting that the Rust community is also going in the direction of making system certificate verification the default...

Hm interesting. uv does not use system certificates by default (and we don't really get complaints about it).

It looks like you do astral-sh/uv#1512

Edit: ah by default you mean but is supported. I assume by not getting complaints is because you have support for it via an option

If httpx does not chose to use truststore by default it would be helpful to allow end-users to force the use of native certs if an environmental variable is set. This would ensure a package using httpx under the hood could avoid adding logic to their api allowing their users to override the ssl_context if a method is using httpx.

Use system trust store by default (#328)

Relevant: https://www.digicert.com/blog/tls-certificate-lifetimes-will-officially-reduce-to-47-days

Really gets my goat when package maintainers ship their own cert store.

It's my machine, I'll trust the certs I want to trust, not a bundle of random CAs that you trust. Shipping your own trust store is anti-user, and achieves nothing for you as the package maintainer: I have to download your trust store via an SSL connection established using my trust store, which evidently you don't trust because you're trying to override it with your own. You've created a chicken and egg problem where you have no way to verify the integrity of the certs that you want to use to verify the integrity of every subsequent request.

And then there's the endless headaches that it causes for downstream consumers running in corporate environments with MITM SSL inspection...

Please use the system trust store by default.

HTTPX is used internally by many other python packages: Users usually don't have direct control over the args/kwargs passed to httpx when it is used as a transitive dependency. Thankfully there are at least some envvars that can be set to regain control over the CAs used in this situation (as long as the package hasn't specified trust_env=False) and make packages that depend on HTTPX usable in corporate environments, but the system trust store should really be the default.

@judilsteve

Shipping your own trust store is anti-user, and achieves nothing for you as the package maintainer

@judilsteve, package maintainers ship their own cert bundles because integrating with the OS crypto libraries properly is really annoyingly difficult and subtle and often provokes more problems than it solves. Everyone here wants system trust roots to be trusted by default. Yelling at maintainers to "just make good software, it does no good to make bad software" does not create progress.

Please actually read the comments here, and on the relevant PR #3409 to understand where the holdup might be and see where you might help out.

Luckily for you, one of the big blockers according to @lovelydinosaur was that the project needed to wait for 2025-10 to drop python3.9 support first, since the relevant library (truststore) doesn't have support back that far, and hey look what day it is. If you want to increase the urgency, perhaps bump that issue with information about the relevant constraint (i.e.: python3.9 EOL happened 4 days ago), you could ask for the patch already suggested by @cclauss to be applied. Or you could ask what you could do to move this forward.

@glyph

@glyph It seems the actual change to HTTPX's code on that PR is very simple (2 lines, the rest is tests/docs/pyproject.toml/requirements.txt updates). Could the python 3.9 issue not be worked around with a conditional dependency and some graceful import handling?

# pyproject.toml
dependencies = [
    'certifi; python_version < "3.10"',
    'truststore; python_version >= "3.10"',
    ...
]

# _config.py

def create_ssl_context(verify: ssl.SSLContext | bool = True) -> ssl.SSLContext:
    import ssl

    try:
        import truststore
    except ModuleNotFoundError:
        # Fall back to `certifi` for py3.9 and below
        import certifi
        return ssl.create_default_context(cafile=certifi.where())
    else:
        return truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)

Could the python 3.9 issue not be worked around with a conditional dependency and some graceful import handling?

I'm not an httpx maintainer so I can't answer that personally. You should probably go ask on the PR. But my guess is that the workaround is to wait until 4 days ago, and drop 3.9 support. Maybe write a PR that just drops that so it can get landed as a prerequisite.

Navigation Menu

Repository navigation

Drop certifi, use system trust store by default #302

19 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Drop certifi, use system trust store by default #302

Description

Activity

yeraydiazdiaz commented on Mar 2, 2020

florimondmanca commented on May 16, 2020

lovelydinosaur commented on May 17, 2020

florimondmanca commented on May 17, 2020

tiran commented on May 17, 2020

florimondmanca commented on May 17, 2020

tiran commented on May 17, 2020

ofek commented on May 31, 2020

AstraLuma commented on Feb 4, 2021

AstraLuma commented on Feb 4, 2021

stale commented on Feb 20, 2022

19 remaining items

lovelydinosaur commented on Oct 24, 2024

sethmlarson commented on Oct 24, 2024

lovelydinosaur commented on Oct 24, 2024

ofek commented on Oct 24, 2024

ofek commented on Oct 24, 2024

zanieb commented on Oct 25, 2024

ofek commented on Oct 25, 2024

omdaniel commented on Dec 18, 2024

ofek commented on Apr 17, 2025

judilsteve commented on Nov 4, 2025

glyph commented on Nov 4, 2025

judilsteve commented on Nov 4, 2025

glyph commented on Nov 4, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions