The JUnit 5 Engine Test Kit provides a way to programmatically run test plans and collect detailed insights about their execution. It lets us see summary statistics, such as how many tests succeeded, failed, or were skipped. Additionally, we can inspect individual test results to verify whether each outcome matches our expectations. In this article, we’ll explore what the Engine Test Kit is, how it works, and how to use it effectively in our applications.
1. Setting Up the Engine Test Kit
To start using the Engine Test Kit, add the following dependency to your project.
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-testkit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.junit</groupId>
<artifactId>junit-bom</artifactId>
<version>5.14.1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
2. Discovering and Executing Tests
Here’s a simple example of using the Engine Test Kit to execute a test class.
public class EngineTestKitCalculatorTest {
@Test
void shouldExecuteTestsAndVerifyStatistics() {
EngineTestKit.engine("junit-jupiter")
.selectors(selectClass(CalculatorTests.class))
.execute()
.testEvents()
.assertStatistics(stats -> stats
.started(4)
.finished(4)
.succeeded(2)
.failed(1)
.aborted(1));
}
}
In this example, EngineTestKit.engine("junit-jupiter") targets the JUnit Jupiter test engine responsible for discovering and executing JUnit 5 tests, while DiscoverySelectors.selectClass(MyTests.class) instructs the engine to discover and run all tests defined in the specified test class. After execution, assertStatistics is used to validate the aggregated execution results by inspecting EventStatistics, which provides high-level metrics, such as the number of tests started, successfully completed, failed, aborted, or skipped.
Example Test Class
To see the Engine Test Kit in action, let’s create a sample test class:
class CalculatorTests {
@Test
void additionWorks() {
assertEquals(10, 7 + 3);
}
@Test
void subtractionWorks() {
assertEquals(4, 9 - 5);
}
@Test
void failingTest() {
assertEquals(10, 2 + 9);
}
@Test
void skippedTest() {
assumeTrue(false, "Skipping this test intentionally");
}
}
Using the Engine Test Kit, we can programmatically execute the CalculatorTests class and then verify the overall outcome of the test run, asserting that two tests complete successfully, one test fails as expected, and one test is aborted. This allows us to validate not only individual test behaviour but also the aggregate execution results, ensuring that the test suite behaves exactly as intended when processed by the JUnit Jupiter engine.
3. Choosing and Working with Different Test Engines
The Engine Test Kit gives us the flexibility to work directly with the test engine of our choice. In JUnit 5, the default engine is JUnit Jupiter, which is responsible for discovering and executing tests written with the JUnit 5 programming model. However, JUnit is designed to be extensible. It is possible to use other engines, such as JUnit Vintage for running JUnit 4 tests, or even to write and plug in a custom test engine tailored to a specific testing model or framework.
@Test
void shouldDiscoverTestsWithoutIssues() {
EngineDiscoveryResults discoveryResults = EngineTestKit.engine("junit-jupiter")
.selectors(selectClass(CalculatorTests.class))
.discover();
assertEquals(emptyList(), discoveryResults.getDiscoveryIssues());
}
}
In this example, EngineTestKit.engine("junit-jupiter") selects the JUnit Jupiter engine explicitly, while selectClass(CalculatorTests.class) tells the engine to scan a particular test class. The discover() method triggers only the discovery phase, returning an EngineDiscoveryResults object that contains information about any issues found during discovery.
The assertion assertEquals(emptyList(), discoveryResults.getDiscoveryIssues()) ensures that all tests in the class were discovered without errors. This approach can be adapted to use different engines, such as junit-vintage or a custom test engine, enabling meta-tests that validate discovery behaviour across multiple testing frameworks.
4. Verifying the Cause of a Test Failure
In addition to validating overall execution statistics, the Engine Test Kit allows us to verify why a specific test failed by inspecting the execution events produced by the test engine. This makes it possible to assert not only that a failure occurred, but that it happened for the expected reason.
@Test
void shouldFailWithAssertionException() {
EngineTestKit.engine("junit-jupiter")
.selectors(selectClass(CalculatorTests.class))
.execute()
.testEvents()
.assertThatEvents()
.haveExactly(1, event(test("failingTest"),
finishedWithFailure(
instanceOf(AssertionFailedError.class)
)));
}
In this example, the Engine Test Kit executes the CalculatorTests class using the JUnit Jupiter engine and captures all test execution events. The assertion then verifies that exactly one test method, identified by the name failingTest, finished with a failure and that the failure was caused by an AssertionFailedError. By asserting against the specific failure type, we ensure that the test failed for the intended reason, rather than due to an unexpected runtime or configuration issue.
5. Conclusion
In this article, we examined how the JUnit 5 Engine Test Kit can be used to discover and execute tests programmatically, analyze execution outcomes, and verify failure and abortion reasons. These capabilities make it well-suited for writing meta-tests that validate test suites, engines, and configurations beyond standard unit testing.
6. Download the Source Code
This guide provided an overview of the JUnit 5 Engine Test Kit.
You can download the full source code of this example here: JUnit 5 engine test kit
