Repository navigation
feat(explain): add evaluation event hooks to the RVM - #898
Open
snehabandla wants to merge 2 commits into
Open
snehabandla wants to merge 2 commits into
snehabandla wants to merge 2 commits into
Conversation
In run-to-completion mode, a rule body that failed with an error inside a loop or comprehension left its entries on the rule's loop and comprehension stacks. The next body of the same rule then routed its guards to the stale entry. For example, an `else` branch evaluated after an error in a comprehension looped until the instruction limit. Clear both stacks before each body. The stacks are already swapped per rule call, so this only drops entries left by the previous body. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add `ExplanationRecorder`, a trait that receives the RVM's evaluation events, behind the default-off `explanations` feature. Install one with `RegoVM::set_explanation_recorder`. It receives: - one check per executed condition (comparison, membership, truthiness, negation, `every`, undefined values), with its operands and outcome; - the end of each rule body, with the value of a complete rule or function; - the values emitted by partial set and object rules; - builtin and arithmetic operator errors; - the start of comprehensions, whose checks are not attributed to the enclosing body. Events carry the program counter, rule, definition and body indices, call depth and whether they occur inside a comprehension. Both execution modes deliver the same events. Without a recorder, each hook site costs one `is_some()` test; with the feature off, the hooks are compiled out. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The explanation tests fail to compile without arc because their value-capturing recorder cannot satisfy the required Send + Sync bounds.
5 open issues
What changed in this PR
Adds opt-in evaluation events to Regorus’s RVM, building on the existing explanation report types. Recorders receive borrowed values and execution context; source details remain in the compiled program.
Changes:
- Introduces the recorder trait and public event types.
- Adds feature-gated hooks across both RVM execution modes.
- Clears loop and comprehension stacks between rule bodies and adds regression coverage.
| File | Description |
|---|---|
tests/rvm/rego/mod.rs |
Runs Rego suites with a no-op recorder installed. |
tests/rvm/rego/cases/rule_body_isolation.yaml |
Adds a regression for stale state after comprehension errors. |
tests/rvm/mod.rs |
Registers the hook test module. |
tests/rvm/explain_hooks.rs |
Tests event payloads, context, lifecycle, errors, and execution-mode parity. |
tests/explanation_corpus/mod.rs |
Runs corpus evaluations with a no-op recorder. |
src/rvm/vm/state.rs |
Resets the recorder for each evaluation. |
src/rvm/vm/rules.rs |
Reports body completion, excludes parameter checks, and clears per-body stacks. |
src/rvm/vm/mod.rs |
Feature-gates the VM hook module. |
src/rvm/vm/machine.rs |
Stores and initializes recorder state. |
src/rvm/vm/loops.rs |
Reports iteration over undefined collections. |
src/rvm/vm/functions.rs |
Reports builtin errors. |
src/rvm/vm/explain.rs |
Implements recorder management, context, event decoding, and delivery. |
src/rvm/vm/dispatch.rs |
Adds hooks for checks, arithmetic errors, emissions, and comprehensions. |
src/explain/recorder.rs |
Defines and documents the recorder contract and event types. |
src/explain/mod.rs |
Re-exports the public recorder API. |
Review effort: Max
| //! Events delivered to an `ExplanationRecorder` by the RVM, in both | ||
| //! execution modes. | ||
|
|
||
| #![cfg(feature = "explanations")] |
| }; | ||
| #[cfg(feature = "explanations")] | ||
| if self.explain_enabled() { | ||
| self.explain_guard(program, register, mode, passed); |
| ObjectSet { obj, key, value } => { | ||
| #[cfg(feature = "explanations")] | ||
| if self.explain_enabled() { | ||
| self.explain_emission(Some(key), value); |
Comment on lines
+182
to
+184
| if (mode == GuardMode::NotUndefined && passed) || self.explain_in_parameters() { | ||
| return; | ||
| } |
Comment on lines
+109
to
+112
| #[cfg(feature = "explanations")] | ||
| if self.explain_enabled() { | ||
| self.explain_reset(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
Adds
regorus::explain::ExplanationRecorder, a trait that receives the RVM's evaluation events, behind the default-offexplanationsfeature. Install one withRegoVM::set_explanation_recorderand remove it withRegoVM::take_explanation_recorder. The recorder receives:check: each executed comparison, membership, truthiness, negation,everyquantifier, and failed existence check (an undefined value that had to be defined, including the collection ofsome x in xsorxs[_]). Each event has the operator, the operand values and the outcome (Success,FailureorUndefined);body_end: the end of each rule body, with its outcome and, for complete rules and functions, the produced value;emission: each value produced by a partial set or partial object rule;builtin_error: builtin and arithmetic operator errors, with the builtin name, a message without a source location, and whether the error was raised;not_captured: constructs whose contributions are not tracked yet (comprehensions);reset: the start of each evaluation.Events identify their instruction by program counter, rule, definition, body and call depth (
HookContext). They carry no source text: that is decoded from the compiled program when a report is built. Events borrow the VM's values; a recorder copies what it keeps.This PR also contains a separate fix commit,
fix(rvm): clear loop and comprehension stacks before each rule body. In run-to-completion mode, a rule body that failed with an error inside a loop or comprehension left entries on the rule's loop and comprehension stacks. The next body of the same rule then sent its guards to the stale entry. For example, anelsebody evaluated after an error in a comprehension looped until the instruction limit. Each body now starts with empty stacks.Details
#[cfg(feature = "explanations")]and checks whether a recorder is installed before doing any work.recordermodule docs list where events differ from the policy text:operand 1 must be number but got stringordivide by zero.|and&are reported asorandand. Errors that end the evaluation, such as exceeded limits or numeric overflow, are not reported.HookContext::in_comprehensionset. The comprehension itself is reported asnot_captured.ExplanationRecorderrequiresSend + Sync, soRegoVMstaysSend + Syncwith the defaultarcfeature. Withoutarc,Valueis notSend, so a recorder that stores values cannot satisfy the bound.allOf/anyOf/notcombinations are not reported.everyworkloads, in both execution modes):main. Run-to-completion adds at most 20 instructions per evaluation (≤ 0.09%), all from the stack-clear fix.Compatibility
New public API, available only with the
explanationsfeature. With the feature off, the only change is the stack-clear fix, which changes run-to-completion results only for rules whose earlier body failed with an error inside a loop or comprehension.Related
ConditionKind,OperatorandOutcome.Testing
tests/rvm/explain_hooks.rscover:every;RegoVMstayingSend + Syncwitharc.tests/rvm/rego/cases/rule_body_isolation.yaml. It fails without the stack-clear fix by reaching the instruction limit.cargo fmt --all -- --check(including bindings);cargo clippy --all-targets --all-features -- -Dwarnings;cargo clippy --no-default-features -- -Dwarnings; clippy for the ffi, java, python and wasm bindings.cargo test,cargo test --features rego-extensions, andcargo test --test aci/--test katawith and withoutrego-extensions;cargo test --doc.cargo test --features explanations,azure_policy --lib --test mod: 839 lib tests and 378 integration tests passed, including the explanation corpus.cargo test --features opa-testutil,serde_json/arbitrary_precision,rego-extensions --test opaovertests/opa.passing(2875 passed, 0 failed).tests/ensure_no_stdbuilds forthumbv7m-none-eabi;cargo build --all-features.