Just Java

This happens more often that it should:

Unit tests run in isolation. But when you start the hole buch, some of the tests fail randomly.

Static initialization

This is the main reason for this behavior. Something is configured somewhere. And since static fields are involved, that (mis)configuration will have an impact on other tests. Very hard to discover. Good luck on that.

Remaining Threads

Another typical problem when running lots of unit tests are remaining threads.

When a unit test has finished, nobody checks for remaining threads. So those threads sill run, do some work, throw Exceptions that end up on the console, print debug statements, interrupt at break points during debugging…

Combined with some static initialization they guarantee for a lot of fun…

How to detect them?

I have written a short rule that detects whether some threads have been left after a unit test has finished. That rule stores all threads running at the beginning of the test. Then this set is compared with the set of all running tests at the end of the of a unit test.

While it does not detect all errors in all cases, it is very helpful to find some remaining ExecutorServices or BackgroundJobs. Just give it a try:

The rule can be used like all other rule (must be public):

  @Rule
  public ThreadRule threadRule = new ThreadRule();

If there are threads left after each test, an exception is thrown with the stack traces of each of the remaining threads:

java.lang.IllegalStateException: Some threads have been left:
// Remaining Threads:
-----------------------
---
Thread[Thread-0,5,main]
	at java.lang.Thread.sleep(Native Method)
	at com.cedarsoft.test.utils.ThreadRuleTest$2.run(ThreadRuleTest.java:34)
	at java.lang.Thread.run(Thread.java:662)
---
Thread[Thread-1,5,main]
	at java.lang.Thread.sleep(Native Method)
	at com.cedarsoft.test.utils.ThreadRuleTest$3.run(ThreadRuleTest.java:44)
	at java.lang.Thread.run(Thread.java:662)
-----------------------

	at com.cedarsoft.test.utils.ThreadRule.after(ThreadRule.java:67)
	at com.cedarsoft.test.utils.ThreadRule.access$200(ThreadRule.java:18)
	at com.cedarsoft.test.utils.ThreadRule$1.evaluate(ThreadRule.java:34)
	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
[...]

The code

The rule is available as part of cedarsoft test-utils deployed to Maven Central. Just give it a try:

<dependency>
    <groupId>com.cedarsoft.commons</groupId>
    <artifactId>test-utils</artifactId>
    <version>5.0.8</version>
</dependency>

com.cedarsoft.test.utils.ThreadRule

For easy copy/pasta:

import com.google.common.base.Joiner;

import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Set;

import javax.annotation.Nonnull;

import org.junit.rules.*;
import org.junit.runner.*;
import org.junit.runners.model.*;

public class ThreadRule implements TestRule {

  public static final String STACK_TRACE_ELEMENT_SEPARATOR = "\n\tat ";

  @Override
  public Statement apply( final Statement base, Description description ) {
    return new Statement() {
      @Override
      public void evaluate() throws Throwable {
        before();
        try {
          base.evaluate();
        } catch ( Throwable t ) {
          afterFailing();
          throw t;
        }
        after();
      }
    };
  }

  private Collection<Thread> initialThreads;

  private void before() {
    if ( initialThreads != null ) {
      throw new IllegalStateException( "???" );
    }

    initialThreads = Thread.getAllStackTraces().keySet();
  }

  @Nonnull
  public Collection<? extends Thread> getInitialThreads() {
    if ( initialThreads == null ) {
      throw new IllegalStateException( "not initialized yet" );
    }
    return Collections.unmodifiableCollection( initialThreads );
  }

  private void afterFailing() {
    Set<? extends Thread> remainingThreads = getRemainingThreads();
    if ( !remainingThreads.isEmpty() ) {
      System.err.print( "Some threads have been left:\n" + buildMessage( remainingThreads ) );
    }
  }

  private void after() {
    Set<? extends Thread> remainingThreads = getRemainingThreads();
    if ( !remainingThreads.isEmpty() ) {
      throw new IllegalStateException( "Some threads have been left:\n" + buildMessage( remainingThreads ) );
    }
  }

  @Nonnull
  private Set<? extends Thread> getRemainingThreads() {
    Collection<Thread> threadsNow = Thread.getAllStackTraces().keySet();

    Set<Thread> remainingThreads = new HashSet<Thread>( threadsNow );
    remainingThreads.removeAll( initialThreads );

    for ( Iterator<Thread> iterator = remainingThreads.iterator(); iterator.hasNext(); ) {
      Thread remainingThread = iterator.next();
      if ( !remainingThread.isAlive() ) {
        iterator.remove();
      }

      //Give the thread a very(!) short time to die off
      try {
        Thread.sleep( 10 );
      } catch ( InterruptedException ignore ) {
      }

      //Second try
      if ( !remainingThread.isAlive() ) {
        iterator.remove();
      }
    }
    return remainingThreads;
  }

  @Nonnull
  private String buildMessage( @Nonnull Set<? extends Thread> remainingThreads ) {
    StringBuilder builder = new StringBuilder();

    builder.append( "// Remaining Threads:" ).append( "\n" );
    builder.append( "-----------------------" ).append( "\n" );
    for ( Thread remainingThread : remainingThreads ) {
      builder.append( "---" );
      builder.append( "\n" );
      builder.append( remainingThread );
      builder.append( STACK_TRACE_ELEMENT_SEPARATOR );
      builder.append( Joiner.on( STACK_TRACE_ELEMENT_SEPARATOR ).join( remainingThread.getStackTrace() ) );
      builder.append( "\n" );
    }
    builder.append( "-----------------------" ).append( "\n" );

    return builder.toString();
  }
}