Skip to content

bug(prover): prove() doc comment incorrectly references severity grades #2394

Closed

Description

@eviehoward

Agent Diagnostic

  • No relevant skills loaded (pure code investigation)
  • OpenShell version tested: v0.0.88 (latest release, 2026-07-21)
  • Latest release checked: yes — bug present in v0.0.88
  • Possible duplicates searched: no existing issues found for "severity", "prover", "exit code", or "doc comment"
  • Findings: lib.rs:31-32 uses severity language ("critical/high") that contradicts four other locations in the same crate and the actual exit code logic in report.rs
  • Agent could not self-resolve: the doc comment is wrong and needs correcting in source

Description

The prove() function in crates/openshell-prover/src/lib.rs has an incorrect doc comment at lines
31–32:

/// - `0` — pass (no critical/high findings, or all accepted)
/// - `1` — fail (critical or high findings present)

The prover has no severity grades — findings are categorical (yes/no per formal query). Exit code 1
fires on any unaccepted finding regardless of category. The correct semantics are:

  /// - `0` — pass (no findings, or all accepted)
  /// - `1` — fail (one or more unaccepted findings present)

Four other locations in the same crate all explicitly state the prover is not severity-graded
(src/finding.rs:8, src/report.rs:6, src/queries.rs:9, README.md:19). The exit code logic in
src/report.rs:111–124 confirms no severity check is performed — it branches solely on path_count > 0
where path_count counts all unaccepted finding paths.

Expected behavior: Doc comment accurately reflects that exit code 1 fires on any unaccepted finding,
not only high/critical ones.

Reproduction Steps

  1. Read crates/openshell-prover/src/lib.rs:31-32.
  2. Compare against src/report.rs:111-124 (exit code logic) and the four references above.

Environment

  • OpenShell: v0.0.88
  • Introduced: commit e98ea3e (feat(policy): add agentic approval loop)
  • Affected file: crates/openshell-prover/src/lib.rs:31-32

Logs

Agent-First Checklist

  • I pointed my agent at the repo and had it investigate this issue
  • I loaded relevant skills (e.g., debug-openshell-cluster, debug-inference, openshell-cli)
  • I checked the latest OpenShell release and either reproduced the issue there or explained why I cannot upgrade/test it
  • I searched existing issues for possible duplicates or explained why I could not
  • My agent could not resolve this — the diagnostic above explains why

Activity

  1. added theissue type on Jul 21, 2026
  2. matthewgrossman commented on Jul 21, 2026

    @matthewgrossman
    Member

    📋 triage-agent

    Triage Assessment

    Classification: bug-confirmed

    Summary

    The report is valid against OpenShell v0.0.88 and current main. Prover findings are categorical, and the renderer returns failure when unaccepted finding paths remain; it does not gate failure on critical or high severity grades.

    Investigation

    crates/openshell-prover/src/lib.rs:31-32 conflicts with the finding model documented in finding.rs, queries.rs, the crate README, and the policy advisor docs. The JSON and shorthand renderers in report.rs filter accepted findings and return 1 when remaining finding paths are nonzero. PR #2395 contains the minimal correction for the reported lines. Review also found adjacent comments in report.rs that say any finding returns 1, although accepted findings return 0, and lib.rs documents a 2 return even though prove() propagates input errors as Err.

    Recommendation

    Route PR #2395 for review and include the adjacent acceptance-aware renderer comments and impossible 2 return documentation in the final doc correction. No published documentation change is needed because the README and policy advisor already describe the behavior correctly.

  3. added
    area:policyPolicy engine and policy lifecycle work
    and removed
    state:triage-neededOpened without agent diagnostics and needs triage
    on Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:policyPolicy engine and policy lifecycle work

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions