StringExamples generation exceeds timeout in 2-3 times

@alisevych

Description

Test generation timeout is exceeded 2-3 times for every StringExamples.

To Reproduce

Run the 'UTBotJava' project in IntelliJ Idea 2022.2.2
Install RC plugin build
Open test generation dialog for utbot-sample/src/main/java/org/utbot/examples/strings/StringExamples.java
Select both classes: IntPair and StringExamples, timeout is 60 sec
Check when test generation finishes.

Expected behavior

Tests are supposed to be generated during the defined timeout of 60 sec.

Actual behavior

Tests are generation for about 4 minutes 45 sec - up to 5 minutes

Environment

Windows 10 Pro
RAM 32.0 GB
Processor Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz 3.60 GHz
IntelliJ IDEA 2022.1.4
Heap memory allocated for IDEA is 2 GB

Additional context

Originally posted by @alisevych in #1102 (comment)

@dtim has made some investigation on the Release 2022.10 build

probably related to other performance issues:
#1028
#1027

I just tried to reproduce this issue locally (I used a debug IDE instead of the RC plugin because the linked RC plugin does not contain the fix for running child processes in Linux, so no tests can be generated at all). For me (Ubuntu 20.04.5 LTS, main branch with last commit 5af1f4888cbcf5d96ab935af37234b9dc0515962), test generations correctly stops in 60 seconds. Of course, most of tests are fuzzed, because the symbolic engine requires more time to process all methods). Method coverage is 100%, line coverage is 85%.

I also tried to reproduce in on W10, 312e34795e18decee752e98cc2f3e3499e21194e, also 60sec

Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz   2.11 GHz
16.0 GB (15.8 GB usable)

With fuzzer turned off in settings.properties, test generation also stops in 60 seconds for me. I tried to generate tests in the empty test root, and not I am unable to actually run these tests, but it seems that it is related to some AWT issues under Linux in the debug Idea itself. I'll check it out one more time when I can run the plugin binary in my main Idea.

Hardware specs:

Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz (8 cores)
Total online memory:    31,9G

Repro step-by-step:

open UTBotJava project in Idea 2022.2.3
install utbot plugin
if utbot was installed - reload idea. Otherwise do not
Generate tests for StringExamples.kt

Also following exception is generated

com.intellij.openapi.project.IndexNotReadyException: Please change caller according to com.intellij.openapi.project.IndexNotReadyException documentation
	at com.intellij.openapi.project.IndexNotReadyException.create(IndexNotReadyException.java:67)
	at com.intellij.util.indexing.FileBasedIndexImpl.handleDumbMode(FileBasedIndexImpl.java:844)
	at com.intellij.util.indexing.FileBasedIndexImpl.ensureUpToDate(FileBasedIndexImpl.java:792)
	at com.intellij.psi.stubs.StubIndexEx.getContainingIds(StubIndexEx.java:350)
	at com.intellij.psi.stubs.StubIndexEx.processElements(StubIndexEx.java:157)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:99)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:87)
	at com.intellij.psi.impl.java.stubs.index.JavaFullClassNameIndex.get(JavaFullClassNameIndex.java:34)
	at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.doFindClasses(JavaFileManagerImpl.java:88)
	at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.findClass(JavaFileManagerImpl.java:111)
	at com.intellij.psi.impl.PsiElementFinderImpl.findClass(PsiElementFinderImpl.java:40)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:93)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:71)
	at com.intellij.psi.impl.PsiClassImplUtil.findSpecialSuperClass(PsiClassImplUtil.java:748)
	at com.intellij.psi.impl.PsiClassImplUtil.getSuperClass(PsiClassImplUtil.java:740)
	at com.intellij.psi.impl.PsiClassImplUtil.getSupersInner(PsiClassImplUtil.java:792)
	at com.intellij.psi.impl.PsiClassImplUtil.getSupers(PsiClassImplUtil.java:752)
	at com.intellij.psi.impl.source.PsiClassImpl.getSupers(PsiClassImpl.java:215)
	at com.intellij.psi.impl.PsiClassImplUtil$MemberCache.lambda$new$1(PsiClassImplUtil.java:356)
	at com.intellij.util.containers.FilteredTraverserBase$Meta.childrenImpl(FilteredTraverserBase.java:437)
	at com.intellij.util.containers.FilteredTraverserBase$Meta.children(FilteredTraverserBase.java:423)
	at com.intellij.util.containers.FilteredTraverserBase.children(FilteredTraverserBase.java:304)
	at com.intellij.util.containers.TreeTraversal$2$1WrappedTree.fun(TreeTraversal.java:121)
	at com.intellij.util.containers.TreeTraversal$2$1WrappedTree.fun(TreeTraversal.java:108)
	at com.intellij.util.containers.TreeTraversal$P.iterable(TreeTraversal.java:805)
	at com.intellij.util.containers.TreeTraversal$P.iterator(TreeTraversal.java:799)
	at com.intellij.util.containers.TreeTraversal$BiOrderIt.nextImpl(TreeTraversal.java:540)
	at com.intellij.util.containers.JBIterator.peekNext(JBIterator.java:150)
	at com.intellij.util.containers.JBIterator.hasNext(JBIterator.java:110)
	at com.intellij.util.containers.ContainerUtil.newArrayList(ContainerUtil.java:133)
	at com.intellij.util.containers.JBIterable.toList(JBIterable.java:793)
	at com.intellij.util.containers.FilteredTraverserBase.toList(FilteredTraverserBase.java:309)
	at com.intellij.psi.impl.PsiClassImplUtil$MemberCache.<init>(PsiClassImplUtil.java:359)
	at com.intellij.psi.impl.PsiClassImplUtil.lambda$getMap$1(PsiClassImplUtil.java:194)
	at com.intellij.util.containers.ConcurrentFactoryMap$2.create(ConcurrentFactoryMap.java:174)
	at com.intellij.util.containers.ConcurrentFactoryMap.get(ConcurrentFactoryMap.java:40)
	at com.intellij.psi.impl.PsiClassImplUtil.getMap(PsiClassImplUtil.java:194)
	at com.intellij.psi.impl.PsiClassImplUtil.processCachedMembersByName(PsiClassImplUtil.java:532)
	at com.intellij.psi.impl.PsiClassImplUtil.processDeclarationsInClass(PsiClassImplUtil.java:444)
	at com.intellij.psi.impl.PsiClassImplUtil.processDeclarationsInClass(PsiClassImplUtil.java:421)
	at com.intellij.psi.impl.source.PsiClassImpl.processDeclarations(PsiClassImpl.java:479)
	at com.intellij.psi.scope.util.PsiScopesUtil.treeWalkUp(PsiScopesUtil.java:61)
	at com.intellij.psi.scope.util.PsiScopesUtil.treeWalkUp(PsiScopesUtil.java:40)
	at com.intellij.psi.scope.util.PsiScopesUtil.resolveAndWalk(PsiScopesUtil.java:207)
	at com.intellij.psi.scope.util.PsiScopesUtil.resolveAndWalk(PsiScopesUtil.java:136)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.resolve(PsiJavaCodeReferenceElementImpl.java:493)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.access$100(PsiJavaCodeReferenceElementImpl.java:44)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl$OurGenericsResolver.resolve(PsiJavaCodeReferenceElementImpl.java:379)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl$OurGenericsResolver.resolve(PsiJavaCodeReferenceElementImpl.java:372)
	at com.intellij.psi.impl.source.resolve.ResolveCache.lambda$resolveWithCaching$2(ResolveCache.java:184)
	at com.intellij.openapi.util.Computable.get(Computable.java:16)
	at com.intellij.psi.impl.source.resolve.ResolveCache.lambda$loggingResolver$4(ResolveCache.java:260)
	at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:114)
	at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:44)
	at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
	at com.intellij.psi.impl.source.resolve.ResolveCache.resolve(ResolveCache.java:237)
	at com.intellij.psi.impl.source.resolve.ResolveCache.resolveWithCaching(ResolveCache.java:184)
	at com.intellij.psi.impl.PsiImplUtil.multiResolveImpl(PsiImplUtil.java:798)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.getCanonicalText(PsiJavaCodeReferenceElementImpl.java:294)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.getCanonicalText(PsiJavaCodeReferenceElementImpl.java:284)
	at com.intellij.psi.impl.source.PsiClassReferenceType.getText(PsiClassReferenceType.java:246)
	at com.intellij.psi.impl.source.PsiClassReferenceType.getCanonicalText(PsiClassReferenceType.java:233)
	at com.intellij.psi.PsiClassType$Stub.getCanonicalText(PsiClassType.java:352)
	at org.utbot.intellij.plugin.util.SignaturesHelperKt.signature(SignaturesHelper.kt:17)
	at org.utbot.intellij.plugin.util.SignaturesHelperKt.signature(SignaturesHelper.kt:13)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor.findMethodsInClassMatchingSelected(UtTestsDialogProcessor.kt:350)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor.access$findMethodsInClassMatchingSelected(UtTestsDialogProcessor.kt:67)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1$1.compute(UtTestsDialogProcessor.kt:177)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1$1.compute(UtTestsDialogProcessor.kt:175)
	at com.intellij.openapi.project.DumbService.lambda$runReadActionInSmartMode$0(DumbService.java:113)
	at com.intellij.openapi.project.DumbService.runReadActionInSmartMode(DumbService.java:144)
	at com.intellij.openapi.project.DumbService.runReadActionInSmartMode(DumbService.java:113)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1.call(UtTestsDialogProcessor.kt:175)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1.call(UtTestsDialogProcessor.kt:161)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.insideReadAction(NonBlockingReadActionImpl.java:536)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$attemptComputation$3(NonBlockingReadActionImpl.java:501)
	at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1154)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:75)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runActionAndCancelBeforeWrite(ProgressIndicatorUtils.java:158)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:115)
	at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:66)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:66)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:112)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:75)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.attemptComputation(NonBlockingReadActionImpl.java:501)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.executeSynchronously(NonBlockingReadActionImpl.java:428)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl.executeSynchronously(NonBlockingReadActionImpl.java:186)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1.run(UtTestsDialogProcessor.kt:180)
	at com.intellij.openapi.progress.impl.CoreProgressManager.startTask(CoreProgressManager.java:442)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.startTask(ProgressManagerImpl.java:114)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcessWithProgressAsynchronously$5(CoreProgressManager.java:493)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$3(ProgressRunner.java:252)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$4(ProgressRunner.java:252)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
	at java.base/java.lang.Thread.run(Thread.java:833)
	```

Verified on the build from latest main - Not reproducing.

Navigation Menu

Repository navigation

StringExamples generation exceeds timeout in 2-3 times #1141

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

StringExamples generation exceeds timeout in 2-3 times #1141

Description

Activity

alisevych commented on Oct 11, 2022

dtim commented on Oct 12, 2022

Domonion commented on Oct 12, 2022

dtim commented on Oct 12, 2022

Domonion commented on Oct 12, 2022

Domonion commented on Oct 12, 2022

alisevych commented on Oct 17, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions