Simple HTTP Load Testing with SLOs

Iter8’s command-line interface makes it simple and easy to set up load tests and visual reports for HTTP services with SLO specifications.

Feb 9th, 2022 10:27am by Srinivasan Parthasarathy

Featued image for: Simple HTTP Load Testing with SLOs

Featured image via Pixabay.

Srinivasan Parthasarathy

Sri is an applied machine learning researcher with a track record of creating scalable AI/ML/advanced optimization-based enterprise solutions for hybrid cloud, cybersecurity and data-exploration problem domains. A co-founder of Iter8, he has presented at Kubecon 2020 and 2021, and at community meetups like Knative and KFServing.

Load testing is an essential building block in the continuous integration and delivery (CI/CD) of HTTP services. During a load test, a stream of HTTP requests is sent to the target service and responses are evaluated for error and latency-related violations. From a developer’s perspective, setting up a load test involves three key dimensions — namely, i) the load-related characteristics of the request stream, such as the request rate; ii) the shape of the requests, in particular, whether the HTTP endpoint requires a payload to be sent as part of its requests; and iii) the service-level objectives (SLOs) used to validate the quality of the target service. This article shows how you can use Iter8, the open source Kubernetes-friendly release engineering platform, to flexibly set up a load test with precise control over all of the above, within seconds. Iter8’s command-line interface (CLI) makes it simple and easy to set up load tests for HTTP services with SLO specifications, verify that the target service meets the SLOs, and create a visual report of the load test as shown below. Screenshot of Iter8 Experiment Report

Prerequisites: Get Iter8 CLI and Run Sample App

Install the Iter8 CLI using one of the following methods.

- Using Brew

brew tap iter8-tools/iter8
brew install iter8

- Using precompiled binaries

Precompiled Iter8 binaries for many platforms are available here. Uncompress the iter8-X-Y.tar.gz archive for your platform, and move the iter8 binary to any folder in your PATH.

- Using Go 1.16+

go install github.com/iter8-tools/iter8@latest

You can now run iter8 from your gopath bin/ directory.

2. We will use httpbin as the target service. Run it in a separate terminal as follows.

docker run -p 80:80 kennethreitz/httpbin

Part 1: Basic Load Test with SLOs

Iter8 provides a predefined and highly customizable experiment chart for load testing. Similar to a Helm chart, Iter8’s experiment charts can be combined with values to create fully specified experiments that can be run by Iter8. Download the load test HTTP experiment chart as follows.

iter8 hub -e load-test-http
cd load-test-http

The sample app (httpbin) implements an HTTP endpoint http://127.0.0.1/get that accepts HTTP GET requests. The following example shows how to run the load test for this endpoint.

iter8 run --set url=http://127.0.0.1/get \
--set SLOs.error-rate=0 \
--set SLOs.latency-mean=50 \
--set SLOs.latency-p90=100 \
--set SLOs.latency-p'97\.5'=200

The above command sends a steady stream of HTTP GET requests to the specified URL and built-in error and latency-related metrics. By default, the following metrics are collected: error-count, error-rate, latency-mean, latency-max, latency-stddev, latency-min and latency percentiles in the list [50.0, 75.0, 90.0, 95.0, 99.0, 99.9]. In addition, any other latency percentiles that are specified as part of SLOs are also collected. In the above example:

Latency percentiles [25.0, 50.0, 75.0, 90.0, 95.0, 97.5, 99.0, 99.9] are collected and reported.
The following SLOs are validated.
1. error rate is 0
2. mean latency is under 50 msec
3. 90th percentile latency is under 100 msec
4. 97.5th percentile latency is under 200 msec

You can use the iter8 assert subcommand to assert that the experiment completed without any failures and all the SLOs are satisfied (as follows).

iter8 assert -c completed -c nofailure -c slos

Assuming the assertions are satisfied, the above subcommand exits with exit code 0. It exits with exit code 1 otherwise. You can use the iter8 report subcommand to view the results of an experiment in HTML or text formats. View a report of the experiment in HTML, as follows.

iter8 report -o html > report.html

Open report.html with a browser. In MacOS, you can run open report.html to do so.

Part 2: Load Characteristics

Iter8 makes it easy to control the load-related characteristics of the request stream that is generated by Iter8. In particular, you can specify the number of queries/duration of the load test, the number of queries sent per second and the number of parallel connections used to send requests. The following example shows how to set the total number of requests sent to 200 (numQueries), number of requests per second to 10 (qps) and the number of parallel connections to five (connections). Replace the iter8 run command used in Part 1 with the following.

iter8 run --set url=http://127.0.0.1/get \
--set SLOs.error-rate=0 \
--set SLOs.latency-mean=50 \
--set SLOs.latency-p90=100 \
--set SLOs.latency-p'97\.5'=200 \
--set numQueries=200 \
--set qps=10 \
--set connections=5

Part 3: Shape of the Requests

HTTP services with POST endpoints may accept payloads as part of requests. Iter8 makes it easy to send various types of content as payload during the load test. In the following example, Iter8 downloads a JSON object and uses the downloaded JSON as the payload in POST requests; the type of payload is set to “application/json”.

iter8 run --set url=http://127.0.0.1/post \
--set payloadURL=https://json-generator.com/ \
--set contentType="application/json" \
--set SLOs.error-rate=0 \
--set SLOs.latency-mean=50 \
--set SLOs.latency-p90=100 \
--set SLOs.latency-p'97\.5'=200

In the following example, Iter8 downloads an image from Pixabay and uses the downloaded image as the payload in POST requests; the type of payload is set to “image/jpeg”.

iter8 run --set url=http://127.0.0.1/post \
--set payloadURL=https://cdn.pixabay.com/photo/2021/09/08/17/58/poppy-6607526_1280.jpg \
--set contentType="image/jpeg" \
--set SLOs.error-rate=0 \
--set SLOs.latency-mean=50 \
--set SLOs.latency-p90=100 \
--set SLOs.latency-p'97\.5'=200

In the following example, the payload is not downloaded from a URL but is directly supplied as a string:

iter8 run --set url=http://127.0.0.1/post \
--set payloadStr="abc123" \
--set contentType="text/plain" \
--set SLOs.error-rate=0 \
--set SLOs.latency-mean=50 \
--set SLOs.latency-p90=100 \
--set SLOs.latency-p'97\.5'=200

Conclusion: Iter8

Iter8’s load test http experiment chart and its simple CLI enable you to rapidly set up load experiments and validate HTTP services. The experiments can be ad hoc, i.e., run on the command line manually. They are also scriptable and can be embedded easily into CI/CD/GitOps pipelines for greater automation, with the outcome of the iter8 assert command influencing the execution of the pipeline. Complete documentation for Iter8 is available.