The Order of Tests in JUnit

1. Overview

By default, JUnit runs tests using a deterministic, but unpredictable order (MethodSorters.DEFAULT).

In most cases, that behavior is perfectly fine and acceptable; but there’re cases when we need to enforce a specific ordering.

2. Test Order in JUnit 5

In JUnit 5, we can use @TestMethodOrder to control the execution order of tests.

We can use our own MethodOrderer, as we’ll see later, or we can select one of three built-in orderers:

  1. @Order Annotation
  2. Alphanumeric Order
  3. Random Order

2.1. Using the @Order Annotation

We can use the @Order annotation to enforce tests to run in a specific order.

In the following example, the methods will run in this order — firstTest(), then secondTest(), and finally, thirdTest():

@TestMethodOrder(OrderAnnotation.class)
public class OrderAnnotationUnitTest {
    private static StringBuilder output = new StringBuilder("");
    
    @Test
    @Order(1)    
    public void firstTest() {
        output.append("a");
    }
    
    @Test
    @Order(2)    
    public void secondTest() {
        output.append("b");
    }
 
    @Test
    @Order(3)    
    public void thirdTest() {
        output.append("c");
    }
 
    @AfterAll
    public static void assertOutput() {
        assertEquals(output.toString(), "abc");
    }
}

2.2. Using Alphanumeric Order

We can also run tests based on their names in alphanumeric order:

@TestMethodOrder(Alphanumeric.class)
public class AlphanumericOrderUnitTest {
    private static StringBuilder output = new StringBuilder("");
    
    @Test
    public void myATest() {
        output.append("A");
    }
    
    @Test
    public void myBTest() {
        output.append("B");        
    }
    
    @Test
    public void myaTest() {
        output.append("a");
    }
 
    @AfterAll
    public static void assertOutput() {
        assertEquals(output.toString(), "ABa");
    }
}

Note that alphanumeric order is case sensitive, so uppercase characters come first then lowercase ones.

The tests will run in this order: myATest()myBTest() and finally myaTest().

2.3. Using a Custom Order

Finally, we can use our own custom order by implementing the MethodOrderer interface.

In our CustomOrder, we’ll order the tests based on their names in a case insensitive alphanumeric order:

public class CustomOrder implements MethodOrderer {
    @Override
    public void orderMethods(MethodOrdererContext context) {
        context.getMethodDescriptors().sort(
         (MethodDescriptor m1, MethodDescriptor m2)->
           m1.getMethod().getName().compareToIgnoreCase(m2.getMethod().getName()));
    }
}

Then, we’ll use CustomOrder to run the same tests from our previous example in this order — myATest()myaTest(), and finally, myBTest():

@TestMethodOrder(CustomOrder.class)
public class CustomOrderUnitTest {

    // ...
 
    @AfterAll
    public static void assertOutput() {
        assertEquals(output.toString(), "AaB");
    }
}

3. Test Order in JUnit 4

If you’re still using JUnit 4, the APIs for ordering tests are slightly different.

Let’s go through the options to achieve this in previous versions as well.

3.1. Using MethodSorters.DEFAULT

This default strategy compares test methods using their hashcodes. In case of a hash collision, the lexicographical order is used:

@FixMethodOrder(MethodSorters.DEFAULT)
public class DefaultOrderOfExecutionTest {
    private static StringBuilder output = new StringBuilder("");

    @Test
    public void secondTest() {
        output.append("b");
    }

    @Test
    public void thirdTest() {
        output.append("c");
    }

    @Test
    public void firstTest() {
        output.append("a");
    }

    @AfterClass
    public static void assertOutput() {
        assertEquals(output.toString(), "cab");
    }
}

When we execute the tests in the class above, we will see that they all pass, including assertOutput().

3.2. Using MethodSorters.JVM

Another ordering strategy is MethodSorters.JVM – this strategy utilizes the natural JVM ordering – which can be different for each run:

@FixMethodOrder(MethodSorters.JVM)
public class JVMOrderOfExecutionTest {    
    // same as above
}

Each time we execute the tests in this class, we get a different result.

3.3. Using MethodSorters.NAME_ASCENDING

Finally, this strategy can be used for running test in their lexicographic order:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class NameAscendingOrderOfExecutionTest {
    // same as above
    
    @AfterClass
    public static void assertOutput() {
        assertEquals(output.toString(), "abc");
    }
}

Similarly, when we execute the tests in this class, we see that they all pass, including assertOutput(), which confirms the execution order that we set with the annotation.

4. Conclusion

In this quick tutorial, we went through the ways of setting the execution order available in JUnit.

And, as always, the examples used in this article can be found over on GitHub.

Related posts:

Java Program to Find Location of a Point Placed in Three Dimensions Using K-D Trees
Java Program to Implement Ternary Heap
Collect a Java Stream to an Immutable Collection
Spring Data JPA @Query
Java Program to Implement Dijkstra’s Algorithm using Queue
Display Auto-Configuration Report in Spring Boot
Hướng dẫn Java Design Pattern – Memento
Semaphore trong Java
Java Program to Implement Interpolation Search Algorithm
HttpClient 4 – Send Custom Cookie
Period and Duration in Java
Java Program to Implement an Algorithm to Find the Global min Cut in a Graph
So sánh HashMap và Hashtable trong Java
Retrieve User Information in Spring Security
Java Program to Implement the linear congruential generator for Pseudo Random Number Generation
Java Program to Represent Linear Equations in Matrix Form
Java Program to Check Cycle in a Graph using Graph traversal
An Intro to Spring Cloud Task
Using Spring ResponseEntity to Manipulate the HTTP Response
Tiêu chuẩn coding trong Java (Coding Standards)
HashMap trong Java hoạt động như thế nào?
Java Program to Check if a Directed Graph is a Tree or Not Using DFS
Java Program to Perform the Sorting Using Counting Sort
Tránh lỗi ConcurrentModificationException trong Java như thế nào?
Overflow and Underflow in Java
Java Program to Solve the 0-1 Knapsack Problem
Java Program to Implement the RSA Algorithm
Create Java Applet to Simulate Any Sorting Technique
Java – Reader to String
Pagination and Sorting using Spring Data JPA
Circular Dependencies in Spring
Các kiểu dữ liệu trong java