Repository navigation
dis.dis() reports line numbers where there are none. #107932
Description
Take this example function from #107901:
def f():
for i in range(30000000):
if not i%1000000:
pass
dis.dis produces the following output:
1 0 RESUME 0
2 2 LOAD_GLOBAL 1 (range + NULL)
12 LOAD_CONST 1 (30000000)
14 CALL 1
22 GET_ITER
>> 24 FOR_ITER 14 (to 56)
28 STORE_FAST 0 (i)
3 30 LOAD_FAST 0 (i)
32 LOAD_CONST 2 (1000000)
34 BINARY_OP 6 (%)
38 TO_BOOL
46 POP_JUMP_IF_FALSE 2 (to 52)
48 JUMP_BACKWARD 14 (to 24)
4 >> 52 JUMP_BACKWARD 16 (to 24)
2 >> 56 END_FOR
58 RETURN_CONST 0 (None)
Which has incorrect line numbers.
The issue is not that the line numbers are wrong, but that you can't tell from the dis output.
The whole point of dis is show what is going on at the bytecode level, so it is failing if it gives wrong line numbers.
The actual line numbers can be see from co_lines()
>>> list(f.__code__.co_lines())
[(0, 2, 1), (2, 30, 2), (30, 48, 3), (48, 52, None), (52, 56, 4), (56, 60, 2)]
The correct output should be:
1 0 RESUME 0
2 2 LOAD_GLOBAL 1 (range + NULL)
12 LOAD_CONST 1 (30000000)
14 CALL 1
22 GET_ITER
>> 24 FOR_ITER 14 (to 56)
28 STORE_FAST 0 (i)
3 30 LOAD_FAST 0 (i)
32 LOAD_CONST 2 (1000000)
34 BINARY_OP 6 (%)
38 TO_BOOL
46 POP_JUMP_IF_FALSE 2 (to 52)
None 48 JUMP_BACKWARD 14 (to 24)
4 >> 52 JUMP_BACKWARD 16 (to 24)
2 >> 56 END_FOR
58 RETURN_CONST 0 (None)
Activity
markshannon commentedon Aug 14, 2023
I've submitted this as a "bug", but it could be argued it is a behavior change, so I think it shouldn't be backported.
@iritkatriel thoughts?
iritkatriel commentedon Aug 14, 2023
I think I'd call it a bug and backport.
CorvinM commentedon Aug 14, 2023
Are bytecodes that do not map back to any source line intended behavior? Older versions of python seem to not generate the bytecode that has no source line for this fragment (tested on 3.10)
Was working on a PR for this but most of dis.py makes the assumption of every bytecode has a source line so the behavior change is getting vague and messy quickly. For example findlinestarts() ignores line=None so dis() (and many of the other exported functions that depend on it) group these lines in with the previous block of bytecodes.
I'm happy to keep working on the patch but looking for some assurance the actual "bug" is disassembly related rather than in the bytecode generator before continuing further.
carljm commentedon Aug 15, 2023
It is definitely intentional that the compiler can generate instructions that are "synthetic" and don't map directly to any line number in the source. And yes, this is a change in recent Python versions, that didn't used to be possible. So it's not too surprising that
dis.pymay have "every instruction has a line number" assumptions built into it, and it may not be a trivial change (and may require some design decisions) to untangle those assumptions correctly.Fix dis module for bytecode that does not have an associated source line
WIP Merge branch 'main' into pythongh-107932
gh-107932: Fix dis module for bytecode that does not have an associat…
pythongh-107932: Fix test_dis
serhiy-storchaka commentedon Aug 25, 2023
It broke the tests.
AlexWaygood commentedon Aug 25, 2023
They will be fixed by #108478 @serhiy-storchaka
gh-107932: Fix merge conflict in test_dis (GH-108478)
AlexWaygood commentedon Aug 25, 2023
now merged