Introduce an Intersection

@JukkaL

This question has already been discussed in #18 long time ago, but now I stumble on this in a practical question: How to annotate something that subclasses two ABC's. Currently, a workaround is to introduce a "mix" class:

from typing import Iterable, Container

class IterableContainer(Iterable[int], Container[int]):
    ...

def f(x: IterableContainer) -> None: ...

class Test(IterableContainer):
    def __iter__(self): ...
    def __contains__(self, item: int) -> bool: ...

f(Test())

but mypy complains about this

error: Argument 1 of "__contains__" incompatible with supertype "Container"

But then I have found this code snippet in #18

def assertIn(item: T, thing: Intersection[Iterable[T], Container[T]]) -> None:
    if item not in thing:
        # Debug output
        for it in thing:
            print(it)

Which is exactly what I want, and it is also much cleaner than introducing an auxiliary "mix" class. Maybe then introducing Intersection is a good idea, @JukkaL is it easy to implement it in mypy?

Mypy complains about your code because __contains__ should accept an argument of type object. It's debatable whether this is the right thing to do, but that's how it's specified in typeshed, and it allows Container to be covariant.

I'm worried that intersection types would be tricky to implement in mypy, though conceptually it should be feasible. I'd prefer supporting structural subtyping / protocols -- they would support your use case, as IterableContainer could be defined as a protocol (the final syntax might be different):

from typing import Iterable, Container

class IterableContainer(Iterable[int], Container[int], Protocol):
    ...

def f(x: IterableContainer) -> None: ...

class Test:
    def __iter__(self): ...
    def __contains__(self, item: int) -> bool: ...

f(Test())  # should be fine (except for the __contains__ argument type bit)

It would be really cool to implement protocols. Still, in this case intersection could be added as a "syntactic sugar", since there would be a certain asymmetry: Assume you want a type alias for something that implements either protocol, then you write:
IterableOrContainer = Union[Iterable[int], Container[int]]
But if you want a type alias for something that implements both, you would write:
class IterableContainer(Iterable[int], Container[int], Protocol): ...
I imagine such asymmetry could confuse a novice. Intersection could then be added (very roughly) as:

class _Intersection:
    def __getitem__(self, bases):
        full_bases = bases+(Protocol,)
        class Inter(*full_bases): ...
        return Inter

Intersection = _Intersection()

then one could write:
IterableContainer = Intersection[Iterable[int], Container[int]]

Intersection[...] gets tricky once you consider type variables, callable types and all the other more special types as items. An intersection type that only supports protocols would be too special purpose to include, as it's not even clear how useful protocols would be.

I understand what you mean. That could be indeed tricky in general case.

Concerning protocols, I think structural subtyping would be quite natural for Python users, but only practice could show whether it will be useful. I think it will be useful.

This keeps coming up, in particular when people have code that they want to support both sets and sequences -- there is no good common type, and many people believe Iterable is the solution, but it isn't (it doesn't support __len__).

@JukkaL

I think Intersection is a very natural thing (at least if one thinks about types as sets, as I usually do). Also, it naturally appears when one wants to support several ABCs/interfaces/protocols.

I don't think that one needs to choose between protocols and Intersection, on the contrary they will work very well in combination. For example, if one wants to have something that supports either "old-style" reversible protocol (i.e. has __len__ and __iter__ methods) or "new-style" (3.6+) reversible protocol (i.e. has __reversed__ method), then the corresponding type is Union[Reversible, Intersection[Sized, Iterable]].

It is easy to add Intersection to PEP 484 (it is already mentioned in PEP 483) and to typing.py, the more difficult part is to implement it in mypy (although @JukkaL mentioned this is feasible).

For cross-reference from #2702, this would be useful for type variables, e.g. T = TypeVar('T', bound=Intersection[t1, t2]).

Intersection[FooClass, BarMixin] is something I found myself missing today

If we had an intersection class in typing.py, what would we call it?

Intersection is linguistically symmetric with Union, but it's also rather long.
Intersect is shorter, but it's a verb. Meet is the type-theoretic version and also nice and short, but, again, you'd expect Union to be called Join if you call Intersection Meet.

As a data point, I first looked for Intersection in the docs.

Just as a random idea I was thinking about All (it would be more clear if Union would be called Any, but that name is already taken). In general, I don't think long name is a big issue, I have seen people writing from typing import Optional as Opt or even Optional as O depending on their taste. Also generic aliases help in such cases:

T = TypeVar('T')
CBack = Optional[Callable[[T], None]]

def process(data: bytes, on_error: CBack[bytes]) -> None:
    ...

I just opened #483 hoping for exactly the same thing. I literally named it the same. I would be all for Intersection or All to allow to require a list of base classes.

@JukkaL

Requests for Intersection appear here and there, maybe we should go ahead and support it in mypy? It can be first put in mypy_extensions or typing_extensions. It is a large piece of work, but should not be too hard. @JukkaL @gvanrossum what do you think?

I think we should note the use cases but not act on it immediately -- there are other tasks that IMO are more important.

@ivan-klass

@ivan-klass Your code contains type errors. To see them, add a type annotation for use_a_or_b:

def example(name: str, a: A, b: B, use_a_or_b: Callable[[A], float] | Callable[[B], float]):
    try:
        print(use_a_or_b(a))
    except:
        print(use_a_or_b(b))

Now, we get two type errors: "A" is not assignable to "B" and "B" is not assignable to "A". That's exactly what prevents us from passing to use_a_or_b something that is not A and B at the same time. The key here is that Python's union type is untagged, so you can't distinguish between the two cases (A -> float and B -> float) and you're forced to give it A & B.

@LeeeeT

@LeeeeT

@ivan-klass Your code contains type errors

I know, I've added explicit cast to epmhasize the intention - we can't distinguish, but we can treat a code like "either A" or "either B", but that doesn't imply "A and B".

def example(name: str, a: A, b: B, use_a_or_b): # (A -> float) | (B -> float)
    print(f"Test Case: {name}")
    try:
        use_as_a = cast(Callable[[A], float], use_a_or_b)
        print(use_as_a(a))
    except:
        use_as_b = cast(Callable[[B], float], use_a_or_b)
        print(use_as_b(b))

@ivan-klass

I think @ivan-klass's point is that (A -> float) | (B -> float), from the programmer's perspective, means that "the function will either use A's properties, or B's properties, but not both", while (A & B) -> float means "the function will use both A's and B's properties"

But @LeeeeT is pointing out that, from the typechecker's perspective, they are both the same, because the typechecker only allows you to write code that works for all possible values. The only way to call a (A -> float) | (B -> float) function that always works for any possible function is to only pass A & B arguments. That's why that code is impossible to write without casts or implicit Any arguments. (And when you use casts or Any, you waive any guarantees that there won't be runtime errors.)

This is an incorrect simplification that only appears correct from a limited perspective and has significant issues when applied more broadly. The specific safe applicative domain equivalence is different from type equivalence. While within the shared domain they have a shared resulting type, they must be treated as distinct types that are not equivalent because of various interactions beyond simply calling a function. Functions may be passed as arguments, and function types can be narrowed via TypeIs functions.

Within the existing framework python has for typing, The simplification would only be valid in the absence of further narrowing of the function's type, while also in the absence of gradual types, while also having both functions have the same return type, while also disallowing the function from being used as a generic parameter, while also not making the simplification when checking if the function, when passed as a value, is compatible with the required type for that value.

There are many significant false assumptions baked into the simplification that seem to stem from attempting to borrow something that may work in other type systems or in other languages with other restrictions on validity.

@max-kamps

@max-kamps

And when you use casts or Any, you waive any guarantees that there won't be runtime errors.

No, that's wrong. When you use casts or Any you are implying that you have out-of-band information that is not reflected in the type system, not waiving away correctness entirely. And you can even tell the difference out-of-band today, by using the inspect module to obtain the signature of the function argument at run-time. A & B -> T is fundamentally different from (A -> T) | (B -> T).

Moreover, it would be a mistake to design what ought to be an orthogonal feature of the type system based on assumptions about current limitations of orthogonal parts of the type system remaining unaddressed.

I'm sorry but I can't figure out what the current state is.

As seen from the size of the issue and the number of issues within the PEP prepare project https://github.com/CarliJoy/intersection_examples/issues/ this is not a small problem scope.

The current state of an RFC for a PEP is tracked here atm.

Naive question: is there some reason we can't follow ty's lead? Or do I have it backwards and they just followed the PEP?

Context: for my project, Intersection is essential to enable automatic injection of type checked functions. Ty simply worked but my case may be simplistic or pathological. (Though I suspect not.)

@jvdillon

@jvdillon Glad to hear that ty's implementation of intersections worked for your case!

ty's implementation and the current work on a proto-PEP kind of evolved in tandem; the core understandings are all shared, so I don't expect a future PEP to stray too far from what ty does. But it may not go as far (e.g. one area of debate is whether we need user-spellable negation types or not.)

@carljm

Hi @carljm! Thanks for your speedy and thorough response!

That's great to hear about tandem development--and yes I can confirm ty worked wonderfully for my case. I am ofc biased, but I think my project is a great demonstration of why this feature is essential. The tl;dr is auto creation of factories necessitates Intersection.

I also made a table comparing ty and basedpyright which you might find interesting (though you surely know all this).

Finally, I'd like to highlight my hack to workaround the hetereogeneous support for Intersection. I made a fake ty-extensions package which I use for non ty to define Interesection of A and B as just A. While this allows me to have code that otherwise pretends Intersection exists (albeit approximated as the first only) it will be a good day when I can remove this hack. :)

@jvdillon

@jvdillon From the other side of the question, and the perspective of working on iterating to a point of proposal and finding what needs specification language so that library authors can get consistent results across typecheckers, ty's implementation of intersections, at least assuming various open issues and known things about ty that I've seen stated are planned remain planned and are eventually implemented, are the closest aligned to what is likely to be proposed as far as already existing implementations go, and I don't believe anything proposed will require ty change their implementation of intersections if the proposal is accepted.

If you'd like to see what other type checkers do for "an intersection-like" currently, as well as for negation, you can do a few things with TypeIs to poke at the internal representations without having to dive into the implementations themselves.

Notably however, some current typecheckers do things we've already found reasons to say shouldn't be the behavior of intersections in TypeIs (which was specified in such a way that when Intersections are specified, the behavior of TypeIs should naturally reflect Intersections and negation) that aren't what should happen, such as mypy erasing Any from intersections produced with TypeIs.

Depending on how you poke at it, you may also find some neat special understandings that aren't specified, like that pyright can infer the type of the output of using type to construct a new type from multiple existing bases.

@mikeshardmind

Hi @mikeshardmind! Tysm for you speedy and thorough answer! You guys are awesome!

Regarding TypeIs, I did play with this extensively and even managed to resuscitate the half-dozen or so ideas I could conceive to achieve my needs without using Intersection. (I tried many more but these are the ones I bothered to document.) If there's a trick I missed, I'd sure love to learn it!

I didn't quite follow your third paragraph but I suspect we're aligned and agreed in the sense that (a) I think TypeIs is useful and well defined (which I also use in my project) and (b) Intersection can and should be a distinct concept. For (b) in particular, I highlight my project as it touches on many (if not most) of the more sophisticated aspects of typing. And furthermore, since this fills a real need (at least, per my professional opinion as an ML scientist training foundational models) then it could serve as a nice motivating example...

@jvdillon

@jvdillon

I didn't quite follow your third paragraph but I suspect we're aligned and agreed in the sense that (a) I think TypeIs is useful and well defined (which I also use in my project) and (b) Intersection can and should be a distinct concept. For (b) in particular, I highlight my project as it touches on many (if not most) of the more sophisticated aspects of typing. And furthermore, since this fills a real need (at least, per my professional opinion as an ML scientist training foundational models) then it could serve as a nice motivating example...

TypeIs is already specified in terms of intersections and negation. It's a little loose right now and was left up to each typechecker due to neither Intersections nor Negations being specified.

With Intersections and Negations specified, I expect slight changes to the behavior of TypeIs to happen automatically for some current typecheckers. If anyone is concerned about this, I doubt much existing code will have new errors. Any proposal that results in new type errors around TypeIs won't be creating those errors, only specifying enough that they are uniformly detected by typecheckers, and it's difficult to provoke some of the known places where typecheckers have a false negative here currently.

Intersections definitely don't remove the need for TypeIs though, this informs typecheckers that a function's purpose is a narrowing constructs that the results of are built with intersections and negation.

It should make it possible to call those functions less in fully typed codebases though since you can narrow once, then call a function that needs that narrowing to be true, rather than that function also needing to use that guard.

I see; makes sense! I guess what your saying is that TypeIs can be (conceivably) built from Intersections but the converse is not true. If indeed this is what you're saying then my own experimentation corroborates this as I was unable to "fake" the absence of Intersection as a first-class citizen. (But again, I'd be thrilled if someone looked at my experiments and could inform me of a technique I overlooked.)

another victim of this is functools.cache: it therefore just replaces the signature with *args: Hashable, **kw: Hashable (python/typeshed#11280)

Navigation Menu

Repository navigation

Introduce an Intersection #213

253 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Introduce an Intersection #213

Description

Activity

JukkaL commented on May 6, 2016

ilevkivskyi commented on May 6, 2016

JukkaL commented on May 6, 2016

ilevkivskyi commented on May 6, 2016

gvanrossum commented on Jul 21, 2016

ilevkivskyi commented on Jul 23, 2016

gvanrossum commented on Jan 17, 2017

jeffkaufman commented on Jun 9, 2017

matthiaskramm commented on Jun 27, 2017

jeffkaufman commented on Jun 27, 2017

ilevkivskyi commented on Jun 28, 2017

mitar commented on Oct 18, 2017

ilevkivskyi commented on Oct 21, 2017

gvanrossum commented on Oct 21, 2017

253 remaining items

LeeeeT commented on Dec 4, 2025

ivan-klass commented on Dec 4, 2025

max-kamps commented on Dec 4, 2025

mikeshardmind commented on Dec 4, 2025

bcmills commented on Dec 5, 2025

Kroppeb commented on Feb 25, 2026

CarliJoy commented on Feb 25, 2026

jvdillon commented on Mar 3, 2026

carljm commented on Mar 3, 2026

jvdillon commented on Mar 3, 2026

mikeshardmind commented on Mar 3, 2026

jvdillon commented on Mar 3, 2026

mikeshardmind commented on Mar 3, 2026

jvdillon commented on Mar 3, 2026

flying-sheep commented on Mar 27, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions