Experiment Generation Tutorial

Prerequisites

This example will deploy the Voting Web App which requires more resources then the quick start example, therefore you will need something larger then a typical minikube cluster. We recommend using a cluster with 4 nodes, 16 vCPUs (4 on each node) and 32GB of memory (8 on each node).

Application File

In order to generate a StormForge Optimize experiment file, you can start with a basic declarative application configuration file, as we have done in our Locust metrics example. Experiment generation will scan the manifests provided to produce an experiment file including the necessary patches for resources and replicas parameters, metric queries, and load test configuration. You can optionally customize your experiment file further after generating it.

To write your own experiment generation file will need to define:

  • resources: The location of the manifests for the application you would like to tune. You can use files, directories, URLs, or git repositories.
  • scenarios: The name and location of your load test file. You can use either a StormForge Performance Test or Locust.
  • objectives: The metrics you want to optimize for. View github for a full list of available Locust metrics and StormForge performance test metrics. You can optionally define metric constraints and add non-optimized metrics.

Read more about the fields that are available in the application reference.

Generate an Experiment File

After cloning the thestormforge/examples repo, we can generate the experiment file and save it locally for our Locust metrics example by running:

$ redskyctl generate experiment -f app.yaml > experiment.yaml

Next, generate and apply the RBAC permissions to allow the controller to patch the deployment:

$ redskyctl generate rbac -f experiment.yaml | kubectl apply -f -

Finally, run your experiment by applying the voting web app resources and experiment file to your cluster:

$ kubectl apply -f <(kustomize build) -f experiment.yaml

Export a Configuration

After your experiment completes, you have the ability to export the manifests that are patched with the parameter values of your chosen configuration. Select the trial you would like to export and use the following command, replacing votingapp-100-clients-020 with the name of the trial you would like to export:

$ redskyctl export -f app.yaml votingapp-100-clients-020

Congrats! Now that you have run our experiment generation example, you can try generating an experiment file for your own app. Questions? Reach out to us via Slack or email.


Last modified December 18, 2020: fix export (04cbc13)