Skip to content

knowledge(testing): a changed permission set needs a test that runs an allowed operation under it - #279

Open
Michael Dieringer (MichaelDieringer) wants to merge 3 commits into
microsoft:mainfrom
Curabis:community-contribution/testing-permission-set-allow-side
Open

Michael Dieringer (MichaelDieringer) wants to merge 3 commits into
microsoft:mainfrom
Curabis:community-contribution/testing-permission-set-allow-side

Conversation

@MichaelDieringer

Copy link
Copy Markdown
Contributor

What this changes

This extends microsoft/knowledge/testing/permission-tests-must-lower-the-execution-context with the other direction of the same gap. A permission set can grant too little: a missing tabledata line, or an RIMD that should have the I. A suite that never lowers the execution context never runs the set, so it cannot see that. The defect then shows up first at a customer, as a permission error for a user who holds exactly that set.

  • Best Practice: when a change adds or modifies a permissionset or permissionsetextension, add or update a test that does all of the following:

    • lowers the context to that set, or to a set or role that includes it;
    • runs an operation the set is meant to allow;
    • asserts the result within what the set grants.

    Removing a grant that the operation needs must make that test fail.

  • Anti Pattern: one sentence naming the gap from the allow side. The existing "do not report the mirror image" clause stays unchanged.

  • Good sample: a second, non-assignable set, CUSTOMER EDITOR (tabledata Customer = RI), and a test, EditorCanRunSetup, that runs the existing Protected Setup Action Test under it and checks the inserted row with Assert.RecordCount. The check reads, so it stays within R.

The trigger is diff-based on purpose: the rule fires only when a permission set is in the change.

Why the model gets this wrong without it

The current article teaches the deny side: lower the context and assert that a limited user is refused. A reviewer that applies it as written passes a change that only adds or edits a permission set, because there is no deny assertion to check. A missing grant is invisible to the compiler and to AppSourceCop. A full-access test run passes too.

Evidence

This comes from a static read of production AL test suites in four apps (2026-11-18). In the three apps that were counted, none of 16 shipped permission sets was run in a lowered context by any test.

Two details checked against BCApps

  • The sample does not call PermissionsMock.Stop(). The test runner starts and stops the mock around every test method (TestRunnerMgt, the StartStopPermissionMock calls before and after each method). We also tested whether a Stop() inside the test inverts the runner's toggle, using Run-TestsInBcContainer with a positive control. It did not. Leaving it out keeps the sample independent of that detail.
  • The text states that "Permissions Mock".Set assigns the set together with execute rights on all objects. That is what Set does in the Permissions Mock codeunit (131006). So the test exercises the set's data permissions, which is what the added paragraph is about.

Checks

Check Result
validate_frontmatter.py 0 errors, 0 warnings
Test-ReviewFixtures.ps1 passed
Test-ReviewContract.ps1 passed

Applies to all BC versions that have "Permissions Mock". The domain is testing, the same as the article it extends.

…n allowed operation under it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new rule targets permission-set-only changes, but al-testing-review still skips a change with no test-code surface. Please extend its scope and permission cue so these changes reach the rule, checking existing test coverage rather than requiring a test edit just to trigger review.

The changed-path fixture check also fails because this article is not selected for evaluation. Please add meaningful coverage for the new scenario and fix the examples before selecting them: both lack BDD tags, and the failing example has a bare asserterror without checking the expected error. Those extra findings would otherwise invalidate the intended test results.

…icle for evaluation

- al-testing-review now applies when a change adds or modifies a
  permissionset or permissionsetextension, even with no test code in the
  diff. The cue checks existing coverage in the test app first and does
  not require a test edit when a test already runs the granted operation
  under the set.
- The article is selected in the testing fixtures with a test-app context.
  The bad sample is a permission-set change with no test that runs the
  granted operation under it; the good sample adds that test.
- Both samples now carry BDD tags. The bare asserterror is gone, and the
  denial test checks WritePermission() with Assert.IsFalse instead of a
  partial error fragment.
- The context regression check accepts this article's own context and
  still rejects context leaking to any other testing article.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MichaelDieringer

Copy link
Copy Markdown
Contributor Author

Thanks, Jesper. Addressed in 6c35c8c (after merging current main):

  • al-testing-review now applies when a change adds or modifies a permissionset or permissionsetextension, even with no test code in the diff. Before reporting, the cue checks for an existing test in the test app that runs the granted operation under the set, and does not require a test edit when one already exists.
  • The article is selected in the testing fixtures with its own context. The bad sample is the new scenario: a permission set that gains Insert, with no test that runs the operation under it. The good sample adds that test.
  • Both samples carry BDD tags. The bare asserterror is gone, and the denial test now checks WritePermission() with Assert.IsFalse instead of a partial error fragment, so the ExpectedError rule from knowledge(testing): ExpectedError must identify the intended error #280 does not fire on it.
  • tools/Test-ReviewFixtureContexts.ps1 assumed only the license article has a testing context. It now accepts this article's own context and still rejects context leaking to any other testing article.

One trade-off: the fixture now covers the permission-set side; the original TestPermissions = Disabled case remains in the article text but no longer has its own sample. If you would rather have both covered, I can split the permission-set side into its own article.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants