Repository navigation
knowledge(testing): a changed permission set needs a test that runs an allowed operation under it - #279
Conversation
…n allowed operation under it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jesper Schulz-Wedde (JesperSchulz)
left a comment
There was a problem hiding this comment.
The new rule targets permission-set-only changes, but al-testing-review still skips a change with no test-code surface. Please extend its scope and permission cue so these changes reach the rule, checking existing test coverage rather than requiring a test edit just to trigger review.
The changed-path fixture check also fails because this article is not selected for evaluation. Please add meaningful coverage for the new scenario and fix the examples before selecting them: both lack BDD tags, and the failing example has a bare asserterror without checking the expected error. Those extra findings would otherwise invalidate the intended test results.
…icle for evaluation - al-testing-review now applies when a change adds or modifies a permissionset or permissionsetextension, even with no test code in the diff. The cue checks existing coverage in the test app first and does not require a test edit when a test already runs the granted operation under the set. - The article is selected in the testing fixtures with a test-app context. The bad sample is a permission-set change with no test that runs the granted operation under it; the good sample adds that test. - Both samples now carry BDD tags. The bare asserterror is gone, and the denial test checks WritePermission() with Assert.IsFalse instead of a partial error fragment. - The context regression check accepts this article's own context and still rejects context leaking to any other testing article. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, Jesper. Addressed in 6c35c8c (after merging current main):
One trade-off: the fixture now covers the permission-set side; the original |
What this changes
This extends
microsoft/knowledge/testing/permission-tests-must-lower-the-execution-contextwith the other direction of the same gap. A permission set can grant too little: a missingtabledataline, or anRIMDthat should have theI. A suite that never lowers the execution context never runs the set, so it cannot see that. The defect then shows up first at a customer, as a permission error for a user who holds exactly that set.Best Practice: when a change adds or modifies a
permissionsetorpermissionsetextension, add or update a test that does all of the following:Removing a grant that the operation needs must make that test fail.
Anti Pattern: one sentence naming the gap from the allow side. The existing "do not report the mirror image" clause stays unchanged.
Good sample: a second, non-assignable set,
CUSTOMER EDITOR(tabledata Customer = RI), and a test,EditorCanRunSetup, that runs the existingProtected Setup Action Testunder it and checks the inserted row withAssert.RecordCount. The check reads, so it stays withinR.The trigger is diff-based on purpose: the rule fires only when a permission set is in the change.
Why the model gets this wrong without it
The current article teaches the deny side: lower the context and assert that a limited user is refused. A reviewer that applies it as written passes a change that only adds or edits a permission set, because there is no deny assertion to check. A missing grant is invisible to the compiler and to AppSourceCop. A full-access test run passes too.
Evidence
This comes from a static read of production AL test suites in four apps (2026-11-18). In the three apps that were counted, none of 16 shipped permission sets was run in a lowered context by any test.
Two details checked against BCApps
PermissionsMock.Stop(). The test runner starts and stops the mock around every test method (TestRunnerMgt, theStartStopPermissionMockcalls before and after each method). We also tested whether aStop()inside the test inverts the runner's toggle, usingRun-TestsInBcContainerwith a positive control. It did not. Leaving it out keeps the sample independent of that detail."Permissions Mock".Setassigns the set together with execute rights on all objects. That is whatSetdoes in the Permissions Mock codeunit (131006). So the test exercises the set's data permissions, which is what the added paragraph is about.Checks
validate_frontmatter.pyTest-ReviewFixtures.ps1Test-ReviewContract.ps1Applies to all BC versions that have
"Permissions Mock". The domain is testing, the same as the article it extends.