Repository navigation
Behavior change for opcode trace after PEP 669 #103615
Description
Before PEP 669, the opcode trace is togglable after trace function being set. So something like
import sys
def opcode_trace_func(frame, event, arg):
print(frame, event)
return opcode_trace_func
sys.settrace(opcode_trace_func)
sys._getframe().f_trace = opcode_trace_func
sys._getframe().f_trace_opcodes = True
def f():
a = [1, 2, 3]
return a[0]
f()would work.
After PEP 669, the opcode event is set on sys.settrace() so it's impossible to enable it after enabling the trace.
sys._getframe().f_trace_opcodes = True
sys.settrace(opcode_trace_func)
sys._getframe().f_trace = opcode_trace_funckind of works but it's super intuitive.
For the current implementation, if we enabled opcode before we do sys.settrace(), it would be "enabled" forever. Meaning the event will be always on, the callback will be always triggered, but frame.f_trace_opcode will be checked in the callback function. Can we always have opcode callback on? It would be a performance hit(extra call for each opcode), but that happens if we want any opcode event at all, and I think that's basically how the old tracing system does it.
Another option would be making f_trace_opcode a descriptor and toggle(actually, probable just enable) opcode events there.
If we don't want to do that at all and just want to break the backward-compatibility because no one is doing insturction trace anyway (I only realized this when syncing #103050 to the lastest change), we should probably document it - make sure the current frame's f_trace_opcode is set before calling sys.settrace() if you want opcode events.
BTW we would've caught this if we already had the instruction level debugging :)
Activity
mdboom commentedon Apr 20, 2023
Cc: @markshannon
gaogaotiantian commentedon May 12, 2023
Hi @markshannon , any idea on this issue?
brandtbucher commentedon Jun 15, 2023
Yeah, but this is globally enabling a more expensive mechanism, forever. No thanks. ;)
This (enabling opcode tracing for a single code object via a descriptor on the frame) definitely seems like the right approach. It's a shame that we probably can't disable it locally too when the descriptor is set to false (or the frame dies) without a counter somewhere (maybe we can aggressively disable events for each instruction somehow?), but I think that this approach seems much better than enabling super-expensive opcode tracing globally forever just because one function somewhere got clever. :)
gaogaotiantian commentedon Jun 16, 2023
Can we check whether tracing is enabled in
sys_trace_instruction_func(the INSTRUCTION callback)? It already checks forframe->f_trace_opcodesanyway. If we can do that (tstate->interp->sys_tracing_threads > 0?), we can get rid of the global instruction events(INSTRUCTION level tracing is toggled per frame anyway). If the user turned off the tracing, it can returnDISABLEso it won't be triggered anymore.brandtbucher commentedon Jun 22, 2023
Feel free to try it, but my understanding is that enabling these events globally is incredibly expensive (in terms of the work that needs to be done/undone for every code object that is currently executing or will be executed, as well as wiping out all specializations, allocating arrays for instruction-level metadata, etc.).
Toggling it per-code object and then checking if the frame has opcode tracing enabled in the callback seems a lot slicker to me, but you also seem to have a better idea of how all of the different pieces fit together. I could definitely be missing something. :)
markshannon commentedon Sep 27, 2023
FTR,
f_trace_opcodewas added in #75525 specifically to add a test for #74174. In fact,f_trace_opcodewas not used in the tests for the fix, #18334.I consider adding
f_trace_opcodea bit of a mistake, but it is there, so we have to support it.It will go away when
sys.settraceis removed. Hopefully in the not too distant future.gh-103615: Use local events for opcode tracing (GH-109472)
pythongh-103615: Use local events for opcode tracing (pythonGH-109472)
pythongh-103615: Use local events for opcode tracing (pythonGH-109472)
Move tests to tests/, add pytest config, fix BINARY_OP for Python 3.12+