Skip to content

knowledge(testing): ExpectedError must identify the intended error - #280

Merged
Jesper Schulz-Wedde (JesperSchulz) merged 1 commit into
microsoft:mainfrom
Curabis:community-contribution/testing-expected-error-identifies-error
Nov 19, 2026
Merged

Jesper Schulz-Wedde (JesperSchulz) merged 1 commit into
microsoft:mainfrom
Curabis:community-contribution/testing-expected-error-identifies-error

Conversation

@MichaelDieringer

Copy link
Copy Markdown
Contributor

What this changes

This sharpens microsoft/knowledge/testing/asserterror-needs-expectederror-and-code. The article only edits Markdown; the samples are unchanged.

The current text accepts Assert.ExpectedError "with a stable substring". Many error tests take that literally, and the test stays green when the wrong error fires. The change has two parts.

Best Practice now asks for text that only the intended error produces. Ideally that is the full Label with its arguments, built with StrSubstNo. It also states two facts:

  • Both Assert.ExpectedError and Assert.ExpectedErrorCode match by substring.
  • ExpectedErrorCode on its own identifies only the kind of error.

Anti Pattern adds three checks that look like verification but cannot tell one error from another:

  • ExpectedErrorCode alone;
  • a non-empty GetLastErrorText();
  • an ExpectedError fragment that several messages on the same code path contain.

In each case, the test stays green when an earlier validation starts to fire instead of the one under test.

Facts checked against BCApps

  • Substring matching: in Library Assert (Test Framework/Test Libraries/Assert/src/LibraryAssert.Codeunit.al), ExpectedError tests StrPos(GetLastErrorText(), Expected) = 0, and ExpectedErrorCode tests StrPos(GetLastErrorCode(), Expected) = 0.
  • Errors raised with Error share one code: this is not stated on Microsoft Learn. BCApps' own W1 tests rely on it, with ExpectedErrorCode('Dialog') after Error-raised failures (159 occurrences in 63 files). The text says "identifies only the kind of error" and does not name the code value.

Evidence

This comes from a static read of production AL test suites (2026-11-18). In two apps, a large share of error tests (about 55 of 142, and 9 of 31) used one of the three weak forms above, or a bare asserterror. The third app had too few error tests to count.

Checks

Check Result
validate_frontmatter.py 0 errors, 0 warnings
Test-ReviewFixtures.ps1 passed
Test-ReviewContract.ps1 passed

Applies to all BC versions. The domain is testing, the same as the article it sharpens.

…ubstring matching and code-only checks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good: the assertion now has to distinguish the intended error, while reusable helpers and the rollback-sentinel exception remain supported.

@JesperSchulz
Jesper Schulz-Wedde (JesperSchulz) merged commit b4613ea into microsoft:main Nov 19, 2026
7 checks passed
@MichaelDieringer
Michael Dieringer (MichaelDieringer) deleted the community-contribution/testing-expected-error-identifies-error branch November 20, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants