Improve --update-data #15265
Copy link
Copy link
Closed
Labels
Description
Current --update-data is somewhat crude:
- works by search and replace of substrings
- handles only errors, not warnings or notices
- only changes existing notes, does not add/remove notes
- does not handle multiline notes
While handling all sorts of data suites is a large undertaking, I think we can do a better job for 'check' tests. I have some rudimentary code (that I used privately for a PR that caused a lot of changes) and I wanted to gauge interest on contributing it.
p.s. I agree "snapshots tests", at their worst, can result in cavalier approach to test failures. I hope code reviews can compensate for this. The downside to not having an efficient update mechanism is withholding beneficial refactors due to the cost associated.
Reactions are currently unavailable
Activity
JelleZijlstra commentedon May 18, 2023
Sounds like a good idea! PRs that improve error messages often need a lot of tweaks to tests, so it's good to make that easier.