Skip to content

Change backoff counter to have larger range and backoff more quickly. #141498

Description

@markshannon

The backoff counter, _Py_BackoffCounter, supports exponential backoff and a value up to 4095.
Backoff is exponential in powers 2.
There are two issues with this:

  • It is a bit slow to backoff: specialization and JIT optimizer failures are rarely transient, so we are wasting effort with backoffs like 2, 4, 8,...
  • It has a relatively low limit of 4095 forcing re-specialization/trace recording every 4095 iteration at most.

The fix is simple, increase the power from 2 to 4. This means we only need 3 bits for the backoff and we get a higher maximum of 8191.
We can also use a lookup table, to ensure that the number of iteration until the next specialization/jit is a prime number

Backoff Value Iterations (value + 1)
0 1 2
1 6 7
6 8190 8191
7 UNREACHABLE_BACKOFF

Linked PRs

Activity

  1. self-assigned this
    on Nov 13, 2025
  2. efimov-mikhail commented on Nov 15, 2025

    @efimov-mikhail
    Member

    I'd like to work on this, if you're not against it, @markshannon.

  3. efimov-mikhail commented on Nov 15, 2025

    @efimov-mikhail
    Member

    I've created a PR with minimal fix: 12 + 4 bits -> 13 + 3 bits, value = 2**backoff - 1 -> value = 2**(2*backoff+1) - 1.
    I can add using prime numbers to the same PR, if we want to.

    CC @Fidget-Spinner

  4. markshannon commented on Nov 19, 2025

    @markshannon
    MemberAuthor

    I'd like to work on this, if you're not against it, @markshannon.

    I did some work on it, but I'm happy for you to do it. It looks like you already are 🙂

  5. added a commit that references this issue on Nov 21, 2025
  6. added a commit that references this issue on Dec 6, 2025
  7. added a commit that references this issue on Dec 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.15bugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usagetopic-JIT

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions