Nondeterminism observed on pandas in typeshed primer #21445

Activity

1. hauntsaninja

   added

   bugmypy got something wrongmypy got something wrong

   on May 9, 2026
2. JelleZijlstra commented on May 9, 2026

   

   JelleZijlstra

   on May 9, 2026

   Member

   More actions

   I set Codex on this a while ago but didn't submit its PR yet because I didn't feel like it was addressing the root cause. Codex thought it was because operator and _operator are part of an SCC and some types are different dependent on the order in which the files in the SCC are processed.

   Depending on the order, some dict in pandas gets inferred as either dict[str, Overload(...)] or dict[str, function], and one of the two leads to errors. Codex proposed to fix that by changing how we union those two types, which felt like a bandaid.

   If you want the LLM's full verbose output:

   Details

   Title

   Nondeterministic inference for operator functions can make dict(zip(...)) alternate between precise callable types and builtins.function

   Summary

   I observed nondeterministic mypy output in mypy-primer on pandas. The reported diff was that this pandas error sometimes appeared and sometimes disappeared with identical code:

   pandas/core/computation/ops.py:328: error: Need type annotation for "_binary_ops_dict"  [var-annotated]
   

   The code is roughly:

   import operator
   
   ARITH_OPS_SYMS = ("+", "-", "*", "/", "**", "//", "%")
   _arith_ops_funcs = (
       operator.add,
       operator.sub,
       operator.mul,
       operator.truediv,
       operator.pow,
       operator.floordiv,
       operator.mod,
   )
   _arith_ops_dict = dict(zip(ARITH_OPS_SYMS, _arith_ops_funcs, strict=True))
   
   _binary_ops_dict = {}
   
   for d in (_cmp_ops_dict, _bool_ops_dict, _arith_ops_dict):
       _binary_ops_dict.update(d)

   Repeated runs against the same checkout sometimes infer _arith_ops_dict as a precise overloaded-function dict, and sometimes as a bare builtins.function dict. That then changes the loop item type for d.

   Observed shapes

   In one run:

   for d in (...): d
   # dict[str, def (Any, Any) -> Any]
   

   In another run:

   for d in (...): d
   # dict[str, def (Any, Any) -> Any] | dict[str, function]
   

   The latter shape prevents partial dict inference for _binary_ops_dict = {}, so mypy emits the annotation error.

   Likely source of nondeterminism

   The root appears to be interaction between typeshed’s operator / _operator stubs and mypy’s SCC processing.

   In current typeshed, there is an import cycle:

   # operator.pyi
   from _operator import add, sub, mul, ...
   
   # _operator.pyi
   from operator import attrgetter, itemgetter, methodcaller

   So operator and _operator are in the same SCC. Mypy’s build graph / SCC code passes around sets in a few places, for example SCCs and ready groups. Depending on processing order within that SCC, operator.add and friends may be seen as precise overloaded functions or as the coarser fallback type builtins.function.

   That gives _arith_ops_dict two possible inferred shapes:

   dict[str, Overload(...)]
   

   or:

   dict[str, function]
   

   Then later union simplification preserves this difference, which leaks into pandas’ _binary_ops_dict inference.

   Notes

   • This was observed with mypy-primer using compiled mypy.
   • The pandas/typeshed code was unchanged between runs.
   • The typeshed PR that exposed it was unrelated (sys._jit), so it likely just perturbed processing enough to reveal an existing nondeterminism.
   • A local mitigation was to make union simplification prefer a precise function-like type over bare builtins.function when the latter is only a fallback inside the same container shape. That makes the pandas loop infer one deterministic type, but it does not fully explain/fix the earlier operator/_operator shape flip.
3. hauntsaninja

   changed the title [-][possible 1.20 regression] Nondeterminism observed on pandas in typeshed primer[/-] [+]Nondeterminism observed on pandas in typeshed primer[/+] on May 9, 2026
4. hauntsaninja commented on May 9, 2026

   

   hauntsaninja

   on May 9, 2026

   CollaboratorAuthor

   More actions

   Hm I kind of doubt that explanation... I will ask my Codex to investigate
5. hauntsaninja commented on May 10, 2026

   

   hauntsaninja

   on May 10, 2026

   CollaboratorAuthor

   More actions

   Yeah, I was right. It is similar to some bugs that I fixed a while back, caused by non-associative operations that get exposed because of set iteration order.

   My Codex kind of got there but it needed help. #21455 is the PR
6. hauntsaninja commented on May 10, 2026

   

   hauntsaninja

   on May 10, 2026

   CollaboratorAuthor

   More actions

   (We could just use dict.fromkeys as a heavy hammer, but the interesting thing is that some orders of operations are better for users so we should try to pick those. Also we really want operations to at least be commutative. So I kind of don't mind learning about it and having chosen behaviour instead of arbitrary behaviour)
7. hauntsaninja

   added a commit that references this issue on May 10, 2026

   Fix nondeterminism from nonassociativity of overload joins (#21455)

   ...

   This issue will close once commit 010f0a2 is merged into the 'master' branch.

   010f0a2
8. hauntsaninja

   closed this as completedin #21455on May 10, 2026
9. JukkaL

   added a commit that references this issue on May 10, 2026

   Fix nondeterminism from nonassociativity of overload joins (#21455)

   ...

   This issue will close once commit 116d60b is merged into the 'master' branch.

   116d60b