Skip to content

[WIP] kprove: apply the specification steps only to the spec's own modules - #4943

Draft
RaoulSchaffranek wants to merge 2528 commits into
developfrom
raoul/kprove-spec-steps
Draft

RaoulSchaffranek wants to merge 2528 commits into
developfrom
raoul/kprove-spec-steps

Conversation

@RaoulSchaffranek

@RaoulSchaffranek RaoulSchaffranek commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Work in progress. Not ready for review. This draft is open to discuss the approach and to try it in KEVM's CI (runtimeverification/evm-semantics#2883).

What is slow

kevm prove loads the claims of each spec file with kprove --dry-run --emit-json-spec (through pyk's ClaimLoader). On KEVM's CI this takes 18–37 s per spec file (median 24 s), about 4,000 s in total over KEVM's 159 spec files. A kprove --verbose profile of one spec puts about three quarters of that time in the specification steps (KoreBackend.specificationSteps). These passes run on the spec module and on every module it imports, which for KEVM is the whole semantics. Right afterwards, every module of the kompiled definition is replaced by its kompiled version. So most of this work is thrown away.

What this changes

  • The passes up to and including propagateMacroToRules still run on all modules. Macro expansion and ResolveFunctionWithConfig read the definition's macro rules and function rules in the form these passes give them. The kompiled form of those rules is different (for example, after constant folding, or after a function got its configuration argument), so it cannot be used instead.
  • Macro expansion and the passes after it run only on the modules that are not part of the kompiled definition, i.e. the spec's own modules.
  • The definition modules are replaced by their kompiled versions right after macro expansion, instead of at the end.
  • ModuleTransformer gets a skipping predicate. A skipped module is returned unchanged, together with its imports.

A first version replaced the definition modules before all passes. That changed the claims: a macro's right-hand side came out constant-folded, and calls to functions that use the configuration lost their configuration argument. It also changed the claims of one KEVM spec. This version keeps those passes on the definition modules.

What was checked

  • KEVM, 159 spec files (all 7 proof suites, 21 kompiled definitions). Unpatched and patched kprove --dry-run --emit-json-spec --allow-rules, called as pyk calls it. For 138 specs the JSON is byte-identical. For the other 21, the claims are identical but come out in a different order. Unpatched kprove already varies here: two unpatched runs give different claim orders for 42 of the 159 specs. The likely cause is that Att.Key hashes a set of Class objects, whose hash codes change between JVM runs, and sentences are kept in hash sets.
  • K's regression-new. All proof tests pass with the patched kprove, as with the unpatched one: the 39 test directories with spec files that make proofs reaches, plus the pl-tutorial IMP++ proof (proof-tests is disabled). Their 244 kprove calls also emit the same claims (233 byte-identical, 11 in a different order).
  • A small definition with a constant-folded macro, an alias, a function that reads the configuration, a fresh variable, and spec-own simplification rules and rules: identical claims and identical spec.kore.
  • k-frontend unit tests (mvn -pl k-frontend test): 317 run, no failures, 2 skipped.

Time per spec file

Sequential runs on an 8-core machine with a load average of 3–5. Wall time of kprove --dry-run --emit-json-spec, two runs each:

spec unpatched patched specification steps (unpatched → patched)
benchmarks/address00 37.7 s, 41.5 s 11.9 s, 13.3 s 28–31 s → 2.4–2.8 s
erc20/ds/approve-success 35.5 s, 39.4 s 12.2 s, 12.3 s 26–29 s → 2.1–2.3 s
mcd/dai-adduu-fail-rough 39.1 s, 41.0 s 11.8 s, 12.9 s 29–31 s → 2.5–2.8 s
functional/lemmas 42.3 s, 48.3 s 15.9 s, 18.0 s 29–32 s → 3.0–3.4 s
benchmarks/functional 40.3 s, 39.7 s 12.8 s, 12.8 s 28 s → 2.9–3.1 s

Still open

  • The passes before macro expansion still run on the whole definition. In one run of address00 on a busy machine, the specification steps took 7.5 s with this version and 1.5 s with the first version (46 s unpatched). Running these passes only on the rules that macro expansion and ResolveFunctionWithConfig read could recover part of that, but needs a careful argument.
  • Not run: the full regression-new suite (only the directories with proof tests), pyk's tests, and other users of kprove (Kontrol, KMIR).
  • Measured on one machine, not yet in CI. The KEVM PR ([WIP] Try a K frontend that loads claims faster (k#4943) evm-semantics#2883) uses this branch.
  • The claim order of --emit-json-spec is not deterministic, with or without this change. This could be fixed separately.

🤖 Generated with Claude Code

rv-auditor and others added 28 commits May 27, 2026 19:03
kprove applies the specification steps to the spec module and to every
module it imports, which includes the whole kompiled definition. The
definition modules are then replaced by their kompiled versions, so most
of that work is discarded.

The passes up to macro expansion still run on all modules, because macro
expansion and ResolveFunctionWithConfig read the definition's macro and
function rules in the form these passes give them, not in their kompiled
form. Macro expansion and the later passes now skip the definition
modules, which are replaced by their kompiled versions right after macro
expansion.

ModuleTransformer gets a `skipping` predicate: modules for which it holds
are returned unchanged, together with their imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants