Manual testing of SARIF report visualizing on Problems tab

[alisevych]

Initial set-up

Check that the IntelliJ Idea UTBot plugin can be successfully installed

• Get plugin build from #1364
• Download plugin
• Check downloaded zip-file size < 100 MB
  Bug reproducing: - Reduce size of plugin zip #1333
• Open IntelliJ IDE
• Remove previously installed UTBot plugin
• Clone or reuse UTBot project (https://github.com/UnitTestBot/UTBotJava.git)
• Open the project in the IDE
• Install the downloaded plugin
• Turn on Problems view in Settings -> Tools -> UnitTestBot

IDE

• IDEA 2022.1.4
• IDEA 2022.2.3
• EAP 223

Samples to check

• Nested classes
• inner calls
  New bug: - Incorrect Problems visualizing for InnerCalls.doubleCallFactorial #1381
• recursion
• Sandboxing
  New bug: - Incorrect line is addressed from Problems tool window for the sandboxed Socket.connect #1383
• objects package
• mixed.Overload (with Integer - NPE)
• mixed.StaticMethodExamples (same - with wrapper types)
• enums
• casts
• natives

Settings to check

• math.OverflowExamples - check with settings Treat Overflow as errors
• Recursion - check with settings Treat Overflow as errors
• Sandboxing - check with permissions
• Timeout failing tests
• Mocks on / off
• Problems view + Generate and Run
• Parameterized tests - not supported by SARIF and visualizing on Problems tool window
• View generated test is addressing to the first line of test class, Stack trace doesn't contain test code calls.

[alisevych]

Bugs found:

• Incorrect Problems visualizing for InnerCalls.doubleCallFactorial #1381
• Incorrect line is addressed from Problems tool window for the sandboxed Socket.connect #1383