Dynamic performance testing evaluates how an application performs in real-world scenarios, accounting for multiple users, variable data, and different workflows. Apache JMeter is a widely used open-source tool for this purpose. It allows testers to simulate user load, vary input data, and generate comprehensive reports, making it especially valuable for testing APIs and web services under concurrent stress. Let us delve into understanding how to build your first dynamic performance test in Apache JMeter.
1. What is Apache JMeter?
Apache JMeter is a powerful, open-source, Java-based performance testing tool developed by The Apache Software Foundation. Originally created for testing web applications, JMeter has evolved into a versatile framework capable of testing a wide range of services including REST APIs, SOAP, microservices, message queues, FTP servers, and databases.
JMeter allows testers, SREs, and developers to simulate real-world traffic by creating load test scenarios that mimic concurrent users interacting with applications. It supports GUI mode for designing tests and command-line (non-GUI) mode for running tests at scale, such as in CI/CD pipelines using JMeter Non-GUI mode.
Its extensibility is one of its strongest features. A rich plugin ecosystem exists via the JMeter Plugins Repository, enabling enhanced reporting, protocol support, graphs, listeners, and more.
1.1 Pros
- Open-source and free to use, backed by a strong community and an extensive plugins ecosystem.
- Supports multiple protocols including HTTP(S), JDBC, JMS, FTP, SMTP, TCP, and can extend to WebSockets via community plugins.
- Offers both GUI mode (for test creation) and Non-GUI mode for large-scale or automated performance tests, making it ideal for CI/CD pipelines using tools like Jenkins, GitHub Actions, or GitLab CI.
- Generates rich HTML dashboard reports, including detailed charts such as response time trends, latency, throughput, errors, and percentiles. See HTML Dashboard documentation.
1.2 Cons
- As a Java-based application, it may require JVM tuning for large-scale or distributed load tests. Heavy tests often require adjusting
-Xmsand-Xmxheap settings. - The GUI can be slow or memory-intensive with very large or complex test plans; JMX files can grow large and become hard to maintain.
- Complex dynamic scenarios sometimes require scripting using JSR223 + Groovy, which adds learning overhead.
1.3 Use Cases
- Load testing, stress testing, endurance testing, and spike testing of REST and SOAP APIs. Example protocols and components can be found in the Component Reference Guide.
- Performance testing of relational databases using the JDBC Sampler, including query benchmarking and connection pool behavior evaluation.
- Simulating hundreds or thousands of virtual users executing realistic user journeys (login → search → checkout). Excellent for microservices and web app testing.
- Running regression performance tests after code changes to ensure latency, throughput, and error percentages remain stable across builds.
- Integrating performance tests into CI/CD using CLI mode or through plugins such as Jenkins Performance Plugin.
2. Code Example
2.1 Maven Dependencies
<dependencies>
<dependency>
<groupId>org.apache.jmeter</groupId>
<artifactId>ApacheJMeter_core</artifactId>
<version>stable__jar__version</version>
</dependency>
<dependency>
<groupId>org.apache.jmeter</groupId>
<artifactId>ApacheJMeter_http</artifactId>
<version>stable__jar__version</version>
</dependency>
<dependency>
<groupId>org.apache.jmeter</groupId>
<artifactId>ApacheJMeter_components</artifactId>
<version>stable__jar__version</version>
</dependency>
<dependency>
<groupId>org.apache.jmeter</groupId>
<artifactId>jmeter-json</artifactId>
<version>stable__jar__version</version>
</dependency>
</dependencies>
2.2 Dynamic Performance Test Code Example
2.2.1 Pre-Requisite
Before running the Java-based JMeter test, ensure you have a users.csv file containing multiple user credentials that will be dynamically injected into the test during execution.
alice,password123 bob,pass456 charlie,admin789
2.2.2 Java Code
// DynamicJMeterTest.java
import org.apache.jmeter.config.CSVDataSet;
import org.apache.jmeter.config.Arguments;
import org.apache.jmeter.control.LoopController;
import org.apache.jmeter.engine.DistributedRunner;
import org.apache.jmeter.engine.StandardJMeterEngine;
import org.apache.jmeter.extractor.json.jsonpath.JSONPostProcessor;
import org.apache.jmeter.protocol.http.control.HeaderManager;
import org.apache.jmeter.protocol.http.control.gui.HttpTestSampleGui;
import org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy;
import org.apache.jmeter.reporters.Summariser;
import org.apache.jmeter.reporters.ResultCollector;
import org.apache.jmeter.save.SaveService;
import org.apache.jmeter.samplers.SampleSaveConfiguration;
import org.apache.jmeter.testelement.TestPlan;
import org.apache.jmeter.testelement.property.BooleanProperty;
import org.apache.jmeter.testelement.property.StringProperty;
import org.apache.jmeter.threads.ThreadGroup;
import org.apache.jmeter.util.JMeterUtils;
import org.apache.jmeter.assertions.ResponseAssertion;
import org.apache.jorphan.collections.HashTree;
import java.io.FileInputStream;
public class DynamicJMeterTest {
public static void main(String[] args) throws Exception {
// -----------------------------------------------------------
// 1. Initialize JMeter environment
// -----------------------------------------------------------
JMeterUtils.setJMeterHome("/path/to/apache-jmeter-5.6.3");
JMeterUtils.loadJMeterProperties("/path/to/apache-jmeter-5.6.3/bin/jmeter.properties");
JMeterUtils.initLocale();
StandardJMeterEngine jmeter = new StandardJMeterEngine();
// -----------------------------------------------------------
// 2. CSV Data Set Config
// -----------------------------------------------------------
CSVDataSet csvDataSet = new CSVDataSet();
csvDataSet.setProperty("filename", "users.csv");
csvDataSet.setProperty("variableNames", "username,password");
csvDataSet.setProperty("delimiter", ",");
csvDataSet.setProperty("recycle", "true");
csvDataSet.setProperty("stopThread", "false");
// -----------------------------------------------------------
// 3. HTTP Login Sampler (POST)
// -----------------------------------------------------------
HTTPSamplerProxy loginSampler = new HTTPSamplerProxy();
loginSampler.setDomain("api.example.com");
loginSampler.setPort(443);
loginSampler.setProtocol("https");
loginSampler.setPath("/api/login");
loginSampler.setMethod("POST");
loginSampler.setPostBodyRaw(true);
loginSampler.addNonEncodedArgument("",
"{\"username\":\"${username}\", \"password\":\"${password}\"}", "");
loginSampler.setName("POST Login");
loginSampler.setProperty(TestPlan.GUI_CLASS, HttpTestSampleGui.class.getName());
// -----------------------------------------------------------
// 4. Extract JSON Token
// -----------------------------------------------------------
JSONPostProcessor jsonExtractor = new JSONPostProcessor();
jsonExtractor.setName("Extract Token");
jsonExtractor.setRefNames("authToken");
jsonExtractor.setJsonPathExprs("$.token");
jsonExtractor.setDefaultValues("NOT_FOUND");
// -----------------------------------------------------------
// 5. Protected API Call (GET profile)
// -----------------------------------------------------------
HTTPSamplerProxy profileSampler = new HTTPSamplerProxy();
profileSampler.setDomain("api.example.com");
profileSampler.setPort(443);
profileSampler.setProtocol("https");
profileSampler.setPath("/api/profile");
profileSampler.setMethod("GET");
profileSampler.setName("GET Profile");
HeaderManager headerManager = new HeaderManager();
headerManager.add("Authorization", "Bearer ${authToken}");
profileSampler.setHeaderManager(headerManager);
// -----------------------------------------------------------
// 6. Response Assertion
// -----------------------------------------------------------
ResponseAssertion assertion = new ResponseAssertion();
assertion.setTestFieldResponseData();
assertion.addTestString("${username}");
assertion.setToContainsType();
assertion.setName("Profile Contains Username");
assertion.setProperty(new BooleanProperty(ResponseAssertion.ASSERTION_ISREGEX, false));
// -----------------------------------------------------------
// 7. Thread Group + Loop Controller
// -----------------------------------------------------------
LoopController loopController = new LoopController();
loopController.setLoops(1);
loopController.setFirst(true);
loopController.initialize();
ThreadGroup threadGroup = new ThreadGroup();
threadGroup.setName("Dynamic-Test-Users");
threadGroup.setNumThreads(3);
threadGroup.setRampUp(1);
threadGroup.setSamplerController(loopController);
// -----------------------------------------------------------
// 8. Test Plan
// -----------------------------------------------------------
TestPlan testPlan = new TestPlan("Dynamic JMeter Test Plan");
// -----------------------------------------------------------
// 9. Build Test Tree
// -----------------------------------------------------------
HashTree testPlanTree = new HashTree();
HashTree plan = testPlanTree.add(testPlan);
HashTree tg = plan.add(threadGroup);
tg.add(csvDataSet);
tg.add(loginSampler).add(jsonExtractor);
tg.add(profileSampler).add(assertion);
// -----------------------------------------------------------
// 10. Add Listener (JTL results)
// -----------------------------------------------------------
Summariser summariser = new Summariser("summary");
ResultCollector logger = new ResultCollector(summariser);
logger.setFilename("results.jtl");
plan.add(logger);
// -----------------------------------------------------------
// 11. Run the Test
// -----------------------------------------------------------
System.out.println("Running JMeter Test...");
jmeter.configure(testPlanTree);
jmeter.run();
System.out.println("Test completed. Results saved to results.jtl");
}
}
2.2.2.1 Code Explanation
The above Java code demonstrates a fully programmatic approach to building and executing a dynamic JMeter test plan, where each part of the script corresponds to a component traditionally created inside the JMeter GUI; it begins by initializing the JMeter environment using JMeterUtils.setJMeterHome() and loading the core configuration files such as jmeter.properties, which prepares the runtime engine (StandardJMeterEngine) required to run a headless test; next, a CSVDataSet configuration is created to allow dynamic user data (usernames and passwords) to be read from a users.csv file, enabling data-driven testing with variable substitution inside samplers; the script then builds an HTTP POST login request (HTTPSamplerProxy) pointing to api.example.com and uses raw JSON in the request body, where credentials are parameterized using ${username} and ${password}; after the login sampler, a JSONPostProcessor is attached to extract a token from the API response using JSONPath ($.token) and store it inside a variable authToken, which is then reused in the subsequent protected API call; the next sampler (GET /api/profile) dynamically adds an Authorization: Bearer ${authToken} header using a HeaderManager, proving how authentication chaining works in a dynamic performance test; to validate correctness, a ResponseAssertion is included to check that the profile response contains the same username that was taken from the CSV, ensuring that each virtual user fetches its own profile—this helps validate functional accuracy during a performance test; the code then sets up a LoopController (runs once) and a ThreadGroup with 3 threads and 1-second ramp-up, defining user concurrency for the test plan; all components are assembled into a hierarchical HashTree (a test structure used internally by JMeter), after which a ResultCollector listener is added to output results into results.jtl while also printing summarised metrics via Summariser; finally, the script configures the engine with the test tree and starts execution via jmeter.run(), printing console messages before and after execution—ultimately demonstrating how to build your first dynamic performance test in Apache JMeter entirely through Java code without using the GUI, enabling CI/CD automation, parameterization through CSV, authentication token extraction, request chaining, response validation, and results logging inside a single self-contained JMeter test harness.
2.2.2.2 Code Run and Output
When the Java program is executed using java DynamicJMeterTest, JMeter initializes its environment, loads the CSV file users.csv, and creates a headless performance test where each virtual user logs in using POST /api/login, extracts the authentication token from the JSON response, and then calls the protected API GET /api/profile using the extracted token. Each profile response is validated through a response assertion to ensure it contains the correct username. Once all threads complete execution, results are written into results.jtl and summary metrics are printed on the console.
Running JMeter Test... summary + 3 in 00:00:02 = 1.5/s Avg: 320 Min: 290 Max: 360 Err: 0 (0.00%) summary + 3 in 00:00:01 = 3.0/s Avg: 210 Min: 200 Max: 230 Err: 0 (0.00%) summary = 6 in 00:00:03 = 2.0/s Avg: 265 Min: 200 Max: 360 Err: 0 (0.00%) Test completed. Results saved to results.jtl
This output shows the throughput, average response times, minimum/maximum values, and error count for all samplers. The absence of errors (Err: 0) confirms test success.
<httpSample t="310" lt="5" ts="1732523321001" s="true" lb="POST Login" rc="200" rm="OK" tn="Dynamic-Test-Users 1-1">
<responseData>{"token":"abx92ue..."}</responseData>
</httpSample>
<httpSample t="215" lt="3" ts="1732523321210" s="true" lb="GET Profile" rc="200" rm="OK" tn="Dynamic-Test-Users 1-1">
<responseData>{"username":"john","email":"john@example.com"}</responseData>
</httpSample>
Each XML entry logs execution time (t), latency (lt), HTTP code (rc), response message (rm), and actual API response (<responseData>). A value of s="true" confirms a successful request and assertion.
3. Conclusion
Building a dynamic performance test with Apache JMeter enables developers and testers to effectively evaluate system behavior under various loads, using real data and validating complex workflows. JMeter’s flexibility, protocol support, and strong reporting make it ideal for a wide range of back-end performance scenarios, though users must address its resource intensity and monitoring limitations with appropriate planning and external integrations.
