#143158 describes hot-cold splitting in the JIT, but that is too late for optimizations that sink code onto exits.
To support those optimizations, we need to split uops as defined into bytecodes.c into "hot" and "cold" ops.
From #143158 (comment)
For example, we want to transform this:
op (_GUARD_IS_NONE_POP, (val -- )) {
int is_none = PyStackRef_IsNone(val);
if (!is_none) {
PyStackRef_CLOSE(val);
AT_END_EXIT_IF(1);
}
DEAD(val);
}into this:
op (_GUARD_IS_NONE_POP, (val -- )) {
int is_none = PyStackRef_IsNone(val);
if (!is_none) {
goto _GUARD_IS_NONE_POP_COLD; // tail call
}
DEAD(val);
}
op (_GUARD_IS_NONE_POP_COLD, (val -- )) {
PyStackRef_CLOSE(val);
AT_END_EXIT_IF(1);
}
Currently, during lowering of the uop trace to a form suitable for execution we need to insert an _EXIT_TRACE uop as the target for exit jumps in the uop. We want to replace those with the "cold" uop followed by the exit, allowing compensation code to be more easily sunk onto side exits.
While it is appealing to do this automatically, it might make more sense to do the splitting manually, and add support for custom exits to the tooling.
We could mark ops as exit and explicitly state which exit is being used (with _EXIT_TRACE as the default exit).
So _GUARD_IS_NONE_POP would be rewritten as:
op (_GUARD_IS_NONE_POP, (val -- )) {
int is_none = PyStackRef_IsNone(val);
if (!is_none) {
EXIT_TO(POP_TOP_EXIT);
}
DEAD(val);
}
exit op (POP_TOP_EXIT, (val -- )) {
PyStackRef_CLOSE(val);
}
Likewise, _ITER_NEXT_INLINE would be rewritten as:
tier2 op(_ITER_NEXT_INLINE, (iternext_fn/4, iter, null_or_index -- iter, null_or_index, next)) {
assert(sizeof(iternextfunc) == sizeof(uintptr_t));
volatile iternextfunc iternext_v = (iternextfunc)iternext_fn;
PyObject *item = iternext_v(PyStackRef_AsPyObjectBorrow(iter));
if (item == NULL) {
EXIT_TO(_ITER_NEXT_INLINE_EXIT);
}
STAT_INC(FOR_ITER, hit);
next = PyStackRef_FromPyObjectSteal(item);
}
exit op(_ITER_NEXT_INLINE_EXIT, ( -- )) {
if (_PyErr_Occurred(tstate)) {
if (_PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) {
_PyEval_MonitorRaise(tstate, frame, frame->instr_ptr);
_PyErr_Clear(tstate);
}
else {
ERROR_NO_POP();
}
}
}
#143158 describes hot-cold splitting in the JIT, but that is too late for optimizations that sink code onto exits.
To support those optimizations, we need to split uops as defined into bytecodes.c into "hot" and "cold" ops.
From #143158 (comment)
For example, we want to transform this:
into this:
Currently, during lowering of the uop trace to a form suitable for execution we need to insert an
_EXIT_TRACEuop as the target for exit jumps in the uop. We want to replace those with the "cold" uop followed by the exit, allowing compensation code to be more easily sunk onto side exits.While it is appealing to do this automatically, it might make more sense to do the splitting manually, and add support for custom exits to the tooling.
We could mark
ops asexitand explicitly state which exit is being used (with_EXIT_TRACEas the default exit).So
_GUARD_IS_NONE_POPwould be rewritten as:Likewise,
_ITER_NEXT_INLINEwould be rewritten as: