Repository navigation
Increase fraction of code executed by tier 2. #118093
Description
Activity
By my count we're currently hovering around 54% of code executed in tier two (our benchmarks run about 266 billion tier one instructions on normal builds and 122 billion instructions on JIT builds). I've identified a few strategies for improving this (based on stats and tracing through how we execute a bunch of the benchmarks) and will start landing PRs soon. No magic bullets here, just chipping away at things:
- Add specializations for
CALL_KWandCALL_FUNCTION_EX. We probably don't need to do anything too crazy here early on in terms of optimizing calls... we can start by just adding a handful of specializations that allow us to trace through them instead of ending the trace. - Add tier two support to several other instructions that are prematurely ending traces. Some of these are easy (
CALL_LIST_APPEND,IMPORT_NAME,LOAD_NAME,BUILD_SET,SEND_GEN, andIMPORT_FROM), and some are harder (LOAD_ATTR_PROPERTY,BINARY_SUBSCR_GETITEM,CALL_ALLOC_AND_ENTER_INIT,RAISE_VARARGS, andBINARY_OP_INPLACE_ADD_UNICODE). - Specialize
SEND_ASYNC_GEN_ANEXT, using a similar shim frame asCALL_ALLOC_AND_ENTER_INIT. - Handle underflow, either dynamically (with
DYNAMIC_EXIT) or statically (by using the current stack when projecting to infer callers). A more radical idea could be to start recording traces instead of projecting them. That simplifies a lot of things, but is a big rewrite of some pretty core stuff. - Turn more
DEOPT_IFs intoEXIT_IFs for better handling of control flow and polymorphism._FOR_ITER_TIER_TWOis an obvious candidate here, but there are others, too. - Allow shorter traces (it's tricky to do this while still requiring progress, but doable).
- Better handling of polymorphism (our current progress requirement inhibits this, but that can be relaxed without too much trouble).
- Remove invalid traces from side exits (currently we just remove them from the bytecode, and side exits not only keep the invalid trace alive and continually deopting, but also prevent new traces from taking their place).
- Be better about closing loops in one trace, by allowing a single jump to occur anywhere in a trace, rather than always at the start.
My motivation for this is to make JIT improvements more pronounced. We currently spend less than 10% of our time in the JIT (vs ~25% of our time in tier one), which means that we need to improve the performance of JIT code by over 10% just to see a 1% improvement on the benchmarks. My (probably ambitious) goal is to get the fraction of code executed in tier two up to around 80% (meaning, in the neighborhood of 25%-30% of the total time spent running the benchmarks) in the next couple of weeks. Then the improvements can be easier to measure and iterate on.
Reacted by Maxim Martynov and Erlend E. Aasland- Add specializations for
It's also worth noting that our stats are currently broken on benchmarks that use C extensions or spawn subprocesses. So the actual numbers may vary a bit right now, but probably aren't heavily biased one way or another.
- added a commit that references this issue
on Aug 16, 2024 - added a commit that references this issue
on Aug 20, 2024 - added 4 commits that reference this issue
on Aug 22, 2024 - added a commit that references this issue
on Aug 22, 2024 There are still improvements that can be made, but those can be done in separate, smaller issues
According to stats and profiling only about 40% of bytecode instructions are executed by tier 2 and the remaining 60% by tier 1.
We the expected improvements to the JIT and tier 2 optimizer we expect tier 2 (with JIT) to have a significantly faster than tier 1.
It therefore make sense to get the fraction of instructions executed by tier 2 up from 40% to nearer 90%.
To do that we need to:
Linked PRs
BINARY_OP_INPLACE_ADD_UNICODE#122253LOAD_ATTR_PROPERTY#122283DEOPT_IFs intoEXIT_IFs #122998CALL_KW#123006CALL_FUNCTION_EX#123034CALL_ALLOC_AND_ENTER_INITsuitable for tier 2. #123140CALL_NON_PY_GENERAL#123212