Repository navigation
The code generated for the interpreter(s) is too fragile #111485
Description
Changing the name of an instruction, or slight changes in cache size can cause the generated code to be incorrect in odd ways.
This is a result of there being too many special cases and undocumented assumptions built in to the code generator.
This was a natural result of getting the thing working and was, until now, acceptable technical debt.
However, as we want to generate abstract interpreters, a new combiner tier-1/tier-2 interpreter and possible tail-calling interprtet/JIT compiler pair, things are too fragile.
We need to:
- Reduce the number of ad-hoc special cases in the tools to zero.
- Minimize and very clearly specify any special markers and variables we use in
bytecodes.c - Clarify the in-memory state and transitions of execution.
- Document the representation of that state in the various tiers and components.
Linked PRs
- GH-111485: Increment
next_instrconsistently at the start of the instruction. #111486 - GH-111485: Remove some special cases from the code generator and bytecodes.c #111540
- GH-111485: Use micro-ops to split specialization code from base action #111547
- GH-111485: Use micro-ops to split specialization code from base action #111561
- GH-111485: Silence warnings in Python/executor_cases.c.h #111619
- GH-111485: Allow arbitrary annotations on instructions and micro-ops. #111697
- GH-111485: Make
BEFORE_WITHa uop #111812 - GH-111485: Separate out parsing, analysis and code-gen phases #112299
- GH-111485: Sort cases #112315
- GH-111485: Factor out opcode ID generator. #112831
- GH-111485: Factor out generation of uop IDs from cases generator. #112877
- GH-111485: Factor out tier 2 code generation from the rest of the interpreter code generator #112968
- GH-111485: Sort metadata tables for easier checking of future diffs #113101
- GH-111485: Mark some instructions as
TIER_ONE_ONLY#113155 - GH-111485: Break up instructions with unused cache entries into component micro-ops #113169
- GH-111485: Test new cases generator #113252
- GH-111485: Generate instruction and uop metadata #113287
- GH-111485: Generate
TARGETtable for computed goto dispatch. #113319 - GH-111485: Delete the old generator code. #113321
- GH-111485: Fix DEFAULT_OUTPUT in opcode_metadata_generator.py #113324
- GH-111485: Fix handling of
FOR_ITERin tier2 #113394
Activity
GH-111485: Increment `next_instr` consistently at the start of the in…
GH-111485: Remove some special cases from the code generator and byte…
GH-111485: Use micro-ops to split specialization code from base action (
GH-111485: Silence warnings in Python/executor_cases.c.h (#111619)
pythonGH-111485: Increment `next_instr` consistently at the start of …
pythonGH-111485: Remove some special cases from the code generator an…
pythonGH-111485: Use micro-ops to split specialization code from base…
pythonGH-111485: Silence warnings in Python/executor_cases.c.h (pytho…
GH-111485: Make BEFORE_WITH a uop (GH-111812)
GH-111485: Allow arbitrary annotations on instructions and micro-ops. (…
pythonGH-111485: Make BEFORE_WITH a uop (pythonGH-111812)
pythonGH-111485: Allow arbitrary annotations on instructions and micr…
neonene commentedon Nov 21, 2023
Maybe
INSTRUMENTED_RESUMEhas an unintentional edit (mixed replacement) since d27acd4?cpython/Python/bytecodes.c
Lines 187 to 189 in d27acd4
Previously, line 189 was
next_instr = frame->instr_ptr;gvanrossum commentedon Nov 21, 2023
@markshannon ^^
markshannon commentedon Nov 21, 2023
Indeed. That doesn't look right.
57 remaining items
pythonGH-111485: Allow arbitrary annotations on instructions and micr…
pythonGH-111485: Sort cases in the case generator output (pythonGH-11…
pythonGH-111485: Separate out parsing, analysis and code-gen phases o…
pythonGH-111485: Factor out opcode ID generator from the main cases g…
pythonGH-111485: Factor out generation of uop IDs from cases generato…
pythonGH-111485: Factor out tier 2 code generation from the rest of t…
pythonGH-111485: Sort metadata tables for easier checking of future d…
pythonGH-111485: Mark some instructions as `TIER_ONE_ONLY` (pythonGH-…
pythonGH-111485: Test the new cases generator (pythonGH-113252)
pythonGH-111485: Break up instructions with unused cache entries into…
pythonGH-111485: Generate instruction and uop metadata (pythonGH-113287)
pythonGH-111485: Generate `TARGET` table for computed goto dispatch. (p…
pythonGH-111485: Fix DEFAULT_OUTPUT in opcode_metadata_generator.py (p…
pythonGH-111485: Delete the old generator code. (pythonGH-113321)
pythonGH-111485: Fix handling of FOR_ITER in Tier 2 (pythonGH-113394)