[JUnit-Rule] Fail tests on exceptions/failed assertions in other threads
I use JUnit a lot. But obviously it has its problems. Fortunately rules are a great way to solve a lot of those problems. At least one can work around them in most of the cases…
On speciallity that got me more than once:
JUnit ignores exceptions/assertions in other threads
JUnit tests only fail on exceptions that are thrown within the “main” thread. Exceptions (and also failed Assertions!) in all other Threads are simply ignored.
Simple test case:
@Test
public void ignoredAssertion() throws Exception {
Thread thread = new Thread( new Runnable() {
@Override
public void run() {
//Of course only one of the following lines is executed. Just comment one of those lines out...
assertFalse(true); //will not be reported!
throw new RuntimeException( "This one is ignored by JUnit, too" );
}
} );
thread.start();
thread.join();
}
This test will be run successfully. (I don’t wanna discuss whether this behavior is correct or not. I know there are a lot of use cases where this is an important feature). And that is quite surprising for a lot of developers.
More important: It is very easy to miss some failed assertions if your code under test is multi threaded…
One rule to catch them all…
I have written a small rule that catches all exceptions on all threads and fails the test if at least one exception has been caught.
How to use the rule
Just add those lines to your test class:
@Rule public CatchAllExceptionsRule catchAllExceptionsRule = new CatchAllExceptionsRule();
And it is guaranteed that your test fails on all (really all) uncaught exceptions.
The rule is deployed to Maven Central:
<dependency>
<groupId>com.cedarsoft.commons</groupId>
<artifactId>test-utils</artifactId>
<version>5.0.9</version>
</dependency>
The rule itself:
/**
* This rule catches exceptions on all threads and fails the test if such exceptions are caught
*
* @author Johannes Schneider (<a href="mailto:js@cedarsoft.com">js@cedarsoft.com</a>)
*/
public class CatchAllExceptionsRule implements TestRule {
@Nullable
private Thread.UncaughtExceptionHandler oldHandler;
@Override
public Statement apply( final Statement base, Description description ) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
before();
try {
base.evaluate();
} catch ( Throwable t ) {
afterFailing();
throw t;
}
afterSuccess();
}
};
}
private void before() {
oldHandler = Thread.getDefaultUncaughtExceptionHandler();
Thread.setDefaultUncaughtExceptionHandler( new Thread.UncaughtExceptionHandler() {
@Override
public void uncaughtException( Thread t, Throwable e ) {
caught.add( e );
if ( oldHandler != null ) {
oldHandler.uncaughtException( t, e );
}
}
} );
}
@Nonnull
private final List<Throwable> caught = new ArrayList<Throwable>();
private void afterSuccess() {
Thread.setDefaultUncaughtExceptionHandler( oldHandler );
if ( caught.isEmpty() ) {
return;
}
throw new AssertionError( buildMessage() );
}
private String buildMessage() {
StringBuilder builder = new StringBuilder();
builder.append( caught.size() ).append( " exceptions thrown but not caught in other threads:\n" );
for ( Throwable throwable : caught ) {
builder.append( "---------------------\n" );
StringWriter out = new StringWriter();
throwable.printStackTrace( new PrintWriter( out ) );
builder.append( out.toString() );
}
builder.append( "---------------------\n" );
return builder.toString();
}
private void afterFailing() {
Thread.setDefaultUncaughtExceptionHandler( oldHandler );
}
}
You need some code to get this fixed.