TestNG is a powerful testing framework for Java that provides many options to organize and execute tests. Sometimes, you may not want to execute all tests in a suite but only a subset based on criteria like failing tests, specific groups, or individual methods. Let us delve into understanding the process to rerun only failed tests in Java TestNG.
1. Running Only Failed Tests with testng-failed.xml
After executing a TestNG test suite, TestNG automatically generates a testng-failed.xml file inside the test-output directory. This XML file includes only the tests that failed in the previous run, making it convenient to isolate and rerun just the failed tests without executing the entire suite again. To rerun the failed tests, use the following command:
java -cp "path_to_testng_jar:path_to_your_classes" org.testng.TestNG test-output/testng-failed.xml
Here’s what happens when you execute this command:
- Test Isolation: Only the failed test methods from the previous run are executed.
- Faster Feedback: You save time by not rerunning tests that already passed.
- Debugging Aid: Helps quickly verify whether a previously failing test has been fixed without going through the entire suite.
Note that testng-failed.xml is regenerated after each test suite execution, reflecting only the most recent failed tests. This mechanism is especially useful in large test suites or CI/CD pipelines where rerunning all tests can be time-consuming.
2. Selective Execution Using <include> in testng.xml
Using the <include> tag in testng.xml allows you to selectively run specific test methods within a class. This is particularly useful when you want to rerun only a subset of tests that are relevant to a certain feature or bug fix, without executing the entire test suite. The following example shows how to include only testMethod1 and testMethod3 from the class MyTest:
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="SelectiveSuite">
<test name="IncludeTest">
<classes>
<class name="com.example.tests.MyTest">
<methods>
<include name="testMethod1"/>
<include name="testMethod3"/>
</methods>
</class>
</classes>
</test>
</suite>
Using <include> provides several advantages:
- Allows precise control over which test methods are executed.
- Reduces test execution time when rerunning only specific methods.
- Makes it easier to focus on tests related to recent changes or bug fixes.
You can combine multiple <include> tags for the same class or even across multiple classes within the same test. This approach is often used in combination with the testng-failed.xml rerun strategy to selectively target only failed methods from previous runs.
3. Running Individual Tests via Maven Surefire
When using Maven to manage your Java projects, the Surefire plugin provides a convenient way to run individual test classes or even single test methods. This is useful when you want to quickly validate a specific method without executing the full test suite.
mvn -Dtest=MyTest#testMethod1 test
-Dtest=MyTest#testMethod1 specifies the fully qualified test class and the specific method to run, and test is the Maven lifecycle phase that triggers test execution. Advantages of running a single test method via Maven Surefire include faster feedback by only running the relevant method, usefulness during debugging or development of a specific feature, and the ability to combine with additional Maven parameters such as -DfailIfNoTests=false or -Dtest=MyTest#testMethod1+testMethod2 to run multiple specific methods.
4. Dynamic Test Control with @BeforeMethod
You can control test execution dynamically in TestNG by using @BeforeMethod in combination with metadata about the test method. This allows you to decide at runtime which tests should run based on conditions such as configuration, environment, or previous test results. The following example demonstrates a scenario where testMethod2 is executed only if a specific condition is met:
import org.testng.annotations.*;
import org.testng.SkipException;
import java.lang.reflect.Method;
public class ConditionalTest {
private boolean runTestMethod2 = false;
@BeforeMethod
public void setup(Method method) {
// Check the method name and set a flag to determine execution
if(method.getName().equals("testMethod2")) {
runTestMethod2 = true; // enable this test dynamically
}
}
@Test
public void testMethod1() {
System.out.println("Running testMethod1");
}
@Test
public void testMethod2() {
// Skip the test if the condition is not met
if(!runTestMethod2) {
throw new SkipException("Skipping testMethod2");
}
System.out.println("Running testMethod2");
}
}
This approach offers several advantages:
- Dynamically decides which tests to execute at runtime.
- Helps in scenarios where certain tests depend on the results of previous tests or external conditions.
- Useful in large test suites to avoid running unnecessary tests, saving execution time and resources.
- Integrates well with other selective execution techniques like
testng-failed.xmlor<include>tags for more granular control.
By leveraging @BeforeMethod, you can implement flexible test flows that adjust automatically based on the current test context.
5. Filtering Tests Programmatically Using IMethodInterceptor
You can implement IMethodInterceptor in TestNG to filter and control which test methods are executed programmatically before the test run begins. This is useful when you want to dynamically decide test execution based on naming conventions, annotations, runtime conditions, or external data sources. The following example demonstrates filtering tests so that only methods starting with “testMethod1” are executed:
import org.testng.*;
import org.testng.annotations.*;
import java.util.List;
import java.util.stream.Collectors;
public class MyInterceptor implements IMethodInterceptor {
@Override
public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
// Only execute methods starting with 'testMethod1'
return methods.stream()
.filter(m -> m.getMethod().getMethodName().startsWith("testMethod1"))
.collect(Collectors.toList());
}
}
@Listeners(MyInterceptor.class)
public class InterceptorTest {
@Test
public void testMethod1() { System.out.println("Running testMethod1"); }
@Test
public void testMethod2() { System.out.println("Running testMethod2"); }
}
Allows programmatic control over test execution before tests run, enables filtering based on method names, annotations, or custom runtime conditions, reduces test execution time by skipping irrelevant tests dynamically, and works well with other selective execution techniques like @BeforeMethod, testng-failed.xml, and <include> tags for more flexible control.
6. Quick Selective Execution Through IDEs
Most IDEs like Eclipse, IntelliJ IDEA, and NetBeans allow right-click execution of individual test classes or methods. Simply right-click the test or method → Run As → TestNG Test. This is a quick way to execute tests selectively without modifying XML or command-line scripts.
7. Conclusion
Selective test execution in TestNG provides developers with the flexibility to focus on specific tests, reduce execution time, and improve overall efficiency. Techniques like rerunning failed tests via testng-failed.xml, using <include> tags, Maven Surefire single method execution, @BeforeMethod conditional execution, IMethodInterceptor, and IDE-driven execution all serve different needs depending on the scenario. By combining these approaches, teams can streamline testing workflows, quickly validate fixes, and maintain faster feedback cycles in both development and CI/CD pipelines. Leveraging these strategies ensures a more targeted and efficient testing process, especially for large and complex test suites.
