Manual testing of Release 2022.10 #1102

Initial set-up

Check that the IntelliJ Idea UTBot plugin can be successfully installed on IDEA 2022.1.4

• Plugin from unit-test-bot/rc2992022 build with latest changes
• Download plugin
• Check downloaded zip-file size < 100 MB
• Open IntelliJ IDE 2022.1.4
• Remove previously installed UTBot plugin
• Clone or reuse UTBot project (https://github.com/UnitTestBot/UTBotJava.git)
• Open the project in the IDE
• Install the downloaded plugin

Check that the IntelliJ Idea UTBot plugin can be successfully installed on IDEA 2022.2.2

• Open IntelliJ IDE 2022.2.2
• Remove previously installed UTBot plugin
• Clone or reuse UTBot project (https://github.com/UnitTestBot/UTBotJava.git)
• Open the project in the IDE
• Install the downloaded plugin

Go through manual scenarios on IDEA 2022.1.4

Manual scenario #1

• Use default plugin settings
• Open the utbot-sample/src/main/java/org/utbot/examples/algorithms/ArraysQuickSort.java file
• Generate tests for the class
• Remove results
• Generate and Run test for a method

Manual scenario #2

• Use default plugin settings
• Open the utbot-sample/src/main/java/org/utbot/examples/mock/CommonMocksExample.java file
• Generate tests with different Mocking options combinations

Go through manual scenarios on IDEA 2022.2.2

Manual scenario #1

• Use default plugin settings
• Open the utbot-sample/src/main/java/org/utbot/examples/algorithms/ArraysQuickSort.java file
• Generate tests for the class
• Remove results
• Generate and Run test for a method

Manual scenario #2

• Use default plugin settings
• Open the utbot-sample/src/main/java/org/utbot/examples/mock/CommonMocksExample.java file
• Generate tests with different Mocking options combinations

@alisevych

Check following in IDEA 2022.1.4

Manual scenario #3

• Create a new Gradle project with JDK 8
• Add a simple java file to test
• Generate a test in the existing test root

Manual scenario #4

• Create a new Maven project with JDK 8
• Add a simple java file to test
• Generate a test with existing test sources root

Manual scenario #5

• Create a new Idea project with JDK 11
• Add a simple java file to test
• Generate tests for several classes in a new test sources root

@tyuldashev

Check following in IDEA 2022.2.2

Manual scenario #3

• Create a new Gradle project with JDK 8
• Add a simple java file to test
• Generate a test in the existing test root

Manual scenario #4

• Create a new Maven project with JDK 8
• Add a simple java file to test
• Generate a test with existing test sources root

Manual scenario #5

• Create a new Idea project with JDK 11
• Add a simple java file to test
• Generate tests for several classes in a new test sources root

Regression testing

@alisevych

IDEA 2022.1.4

• Maven with JDK 11 : make following checks
• Maven with JDK 8 : make following checks
• Test frameworks JUnit4/ JUnit5/TestNG
• - with Parametrization on / off
• - with mocks on /off
• - UtUtils class (with mocks off / on) - check it's regenerated when mocks are on for the first time
• String Java 8 - with JDK 8
• String Java 9 - 11 - with JDK 11

Project with JDK 11:

• timeout changes - 1/10/60 sec
• cancellation
• regions: Symbolic first, Fuzzing, error suite (NPE), explicitly thrown exceptions, caught exceptions (successful)
• exceptions - check regions for exceptions
• context menu / shortcut
• call from method / class / package - check Members list
• math
• make.symbolic
• inner
• collections
• Generics - with Fuzzing
• Sandbox - smoke

@tyuldashev

IDEA 2022.2.2

• Gradle project with JDK 8 : make following checks
• Gradle project with JDK 11 : make following checks
• Test frameworks JUnit4/ JUnit5/TestNG
• - with Parametrization on / off
• - with mocks on /off
• - UtUtils class (with mocks off / on) - check it's regenerated when mocks are on for the first time
• String Java 8 - with JDK 8
• String Java 9 - 11 - with JDK 11

Project with JDK 11:

• timeout changes - 1/10/60 sec
• cancellation
• regions: Symbolic first, Fuzzing, error suite (NPE), explicitly thrown exceptions, caught exceptions (successful)
• exceptions - check regions for exceptions
• context menu / shortcut
• call from method / class / package - check Members list
• Date - check
• Arrays
• annotations
• Generics - with Fuzzing
• Sandbox - smoke