Repository navigation
Hanging test timeout is ignored by Symbolic execution #1095
Copy link
Copy link
Open
Labels
ctg-bugIssue is a bugIssue is a bug
Description
Description
When a hanging test is generated by Symbolic execution, it doesn't have Timeout annotation.
To Reproduce
- Install UTBot plugin on IntelliJ Idea
- Check Symbolic execution is enabled and Hanging test timeout is 1000 (default)
- Create a project and add the following class:
public class A {
public int hangForSeconds(int seconds) {
for (int i = 1; i < seconds; i++) {
try {
Thread.sleep(1001);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
return seconds;
}
}- Open the generated test
Expected behavior
Tests whose generation lasts for more than 1000 millisec should fail with TimeoutException.
Actual behavior
Test generated by Symbolic execution, lasts for , it doesn't have Timeout annotation.
Tests generated by Fuzzing has correct annotation.
Visual proofs (screenshots, logs, images)
public class ATest {
///region Test suites for executable org.example.A.hangForSeconds
///region SYMBOLIC EXECUTION: SUCCESSFUL EXECUTIONS for method hangForSeconds(int)
/**
* <pre>
* Test iterates the loop {@code for(int i = 1; i < seconds; i++) } once.
* Test afterwards returns from: {@code return seconds; }
* </pre>
*/
@Test
@DisplayName("hangForSeconds: ThreadSleep -> return seconds")
public void testHangForSeconds_ThreadSleep() {
A a = new A();
int actual = a.hangForSeconds(2);
assertEquals(2, actual);
}
///endregion
///region FUZZER: SUCCESSFUL EXECUTIONS for method hangForSeconds(int)
@Test
@DisplayName("hangForSeconds: seconds > 0 -> return 1")
public void testHangForSecondsReturnsOne() {
A a = new A();
int actual = a.hangForSeconds(1);
assertEquals(1, actual);
}
///endregion
///region FUZZER: EXPLICITLY_THROWN_UNCHECKED_EXCEPTIONS for method hangForSeconds(int)
@Test
@DisplayName("hangForSeconds: seconds = 134217729 (mutated from 1)")
@Timeout(value = 1000L, unit = TimeUnit.MILLISECONDS)
public void testHangForSeconds() {
A a = new A();
/* This execution may take longer than the 1000 ms timeout
and therefore fail due to exceeding the timeout. */
assertTimeoutPreemptively(Duration.ofMillis(1000L), () -> a.hangForSeconds(134217729));
}
///endregion
///endregion
}Environment
Windows 10 Pro
IntelliJ IDEA 2022.2.2 / 2022.1.4
Additional context
Also Fuzzing test generation mode throws exception in Concrete execution logs:
Exception in thread "executor" java.lang.IllegalMonitorStateException
at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:406)
at org.utbot.common.ThreadBasedExecutor$invokeWithTimeout$1.invoke(ThreadUtil.kt:46)
at org.utbot.common.ThreadBasedExecutor$invokeWithTimeout$1.invoke(ThreadUtil.kt:43)
at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
ctg-bugIssue is a bugIssue is a bug
Type
Projects
- StatusShow more project fieldsTodo
Activity
Damtev commentedon Oct 7, 2022
It looks like a corner case for the child process timeout - as such as the generated test takes
1000 + epsilonmilliseconds to execute, a child process does not detect such overflow ofhanging tests timeout. Tests are generated as expected in case of decreasing thehanging tests timeoutsetting or increasing sleeping time. @sergeypospelov what do you think?alisevych commentedon Oct 7, 2022
@Damtev @sergeypospelov Sorry, corrected description, I've used 1001 milliseconds.