Build your own trial image

Learn how to build your own Docker image for your next Trial

Optimize Pro has been deprecated.
But you can check out Optimize Live:

An Experiment enables you to explore the performance and behavior of your application in various configurations. A Trial represents a unique configuration within the space defined by your experiment to be tested by Optimize Pro. You can use trial pods to generate load or otherwise interact with your application to simulate desired conditions during a trial. StormForge provides a number of ready to use images for trials in the thestormforge/optimize-trials repository as well as examples for various applications.

Besides these provided images, you can build your own images - feel free to take the provided trial images as inspiration.

Requirements

The requirements for a trial image are simple, as anything that can be executed via a Kubernetes Job can be used:

  1. Exit with a non-zero exit code if the trial could not run or had a fatal errors.
  2. Export metrics to one of the supported metric types. This enables the Optimize controller to query for metrics and tune your application not only by its own metrics (such as CPU/memory usage) but also observed client performance, such as transactions per second or latency.

Best Practices

You’ll find it easier to use an image in experiments by following the best practices below.

Utilize the PUSHGATEWAY_URL environment variable

The PUSHGATEWAY_URL is an environment variable that the Optimize controller sets on the trial container, if the prometheus setupTask is configured. You can also configure this directly in the trial spec, if you want to set up the Pushgateway yourself.

A trial image should read this environment variable and export its metrics to the URL, if set. If your code in the trial image exports metrics as a .json file, you can transform the metrics to the prometheus text format via jq and upload via curl:

if [ -n "${PUSHGATEWAY_URL}" ]; then
    cat "/metrics.json" \
        | jq -r '.data.attributes.basic_statistics|keys[] as $k | "\($k) \(.[$k])"' \
        | curl --data-binary @- "${PUSHGATEWAY_URL}"
else
    echo "No PUSHGATEWAY_URL configured" 1>&2
fi

You may also log an error/exit, if the PUSHGATEWAY_URL environment variable is configured but the image doesn’t support it.

Make the trial image configurable via ENV and filesystem

A trial image should be configurable via the ENV and filesystem, thus you should rarely need to rebuild the image. Kubernetes enables you to configure the env of a trial container from various sources including ConfigMaps and Secrets. ConfigMaps can also be mounted into the filesystem, enabling the container to receive various configurations or test definitions like JMeter .jmx files.

Expressive logs

Logs help with debugging, especially on the first trials when you need to establish a baseline for your experiment. Make sure the logs help with understanding what happens while executing the trial. If your image has many configuration options, provide helpful error messages for missing options or invalid combinations of options.

Last modified February 12, 2024