I've created a custom runner by extending Suite
:
public class MyRunner extends Suite {
public MyRunner(Class<?> klass, RunnerBuilder builder) throws InitializationError {
super(klass, builder);
}
@Override
public void run(RunNotifier notifier) {
notifier.addListener(new MyRunListener());
notifier.fireTestRunStarted(getDescription());
super.run(notifier);
}
}
As seen in run
this registeres a MyRunListener
which looks like this:
class MyRunListener extends RunListener {
@Override
public void testRunStarted(Description description) {
System.err.println("1: run started");
}
@Override
public void testStarted(Description description) {
System.err.println("2: test started");
}
@Override
public void testFinished(Description description) {
System.err.println("3: test finished");
}
@Override
public void testRunFinished(Result result) {
System.err.println("4: run finished");
}
}
I've added 1:
, 2:
, 3:
and 4:
according to the order in which I expect these methods to be called.
I've then created a test suite that looks as follows:
@RunWith(MyRunner.class)
@Suite.SuiteClasses({ MyTest.class })
public class MyTestSuite {
}
And the following test:
public class MyTest {
@BeforeClass
public static void beforeClass() {
System.err.println("A: beforeClass");
}
@Before
public void before() {
System.err.println("B: before");
}
@Test
public void test() {
System.err.println("C: Running actual test...");
}
@After
public void after() {
System.err.println("D: after");
}
@AfterClass
public static void afterClass() {
System.err.println("E: afterClass");
}
}
Again, output labeled according to the order I expect.
Here's the output I get when I run MyTestSuite
through IntelliJ (as a JUnit run configuration):
2: test started
B: before
C: Running actual test...
D: after
3: test finished
E: afterClass
4: run finished
1: run started
A: beforeClass
Why on earth am I getting that output order? I must be doing something very wrong in my runner, but I've just implemented it according to tutorials on the web. (Using JUnit 4.12)
As @DmitryB suggests in the comments, it seems to be some form of output buffering. The various parts of the output is then stitched together in a weird way after the test has finished.
I added Thread.sleep(1000)
after each printout and this is how it looks when I run it in IntelliJ:
The workaround for now is to use the gradle task run configuration rather than the JUnit one.
Filed this ticket at Jetbrains.