Secret Management

Secret Management

Performance testing sometimes requires private information that needs to be kept secret. Typically secrets include data for authentication. That can be for the entire test, using a special header, HTTP Basic Authentication or even TLS client certificates.

In this context we are referring to technical information that needs to kept secret and exclude personal information, which are not allowed to be used with our services at all.

What we are doing and what we are not doing

A test run generates a lot of data and logging information. In many cases parts of requests executed and responses received, as well as used data sources we get in contact with potentially sensitive secrets. Depending on test run size we are generating many Gigabytes of data.

Since there is no way for us to not to know and see this information, the only option is to hide secrets by scrubbing out the sensitive parts. We are currently not doing that either since it is very complicated to do that safely in all cases and guarantee that only a special group of people can see those. Among other restrictions unauthorized people would not be allowed to edit a test case, as you can change the target and expose secrets this way.

If you include sensitive secrets in your tests, everyone with access to that organization, our support and engineering teams will have access to it. We are applying industry best-practices like TLS and mutual authentication to minimize potential risk of exposure as good as possible.

Recommendation

We generally recommend the following when dealing with sensitive secrets in performance tests:

• use only short-lived secrets: If you know that a test is running for 30 minutes, limit the validity of secrets to approximately the same time frame. Using our CLI you can automatically inject secrets into the test case definition. Alternatively you can update your data sources just before launching a new test.
• use secrets that can be easily be invalidated: If you are able to revoke or invalidate your secrets easily, you can automatically rotate them after each test run, once or day, etc.
• create dedicated, single-purpose secrets: Secrets allowing access to your test systems should only be used for testing and not shared across different use-cases. This makes it a lot easier to invalidate secrets.

Depending on your thread model or policies additional measures like limiting access to your system under test with IP ranges can be applied. For special cases we might also be able to offer dedicated, exclusive IP addresses for customers. If you have any questions regarding secrets management, don’t hesitate to get in contact. We’re happy to help!