Advanced installation

Prerequisites

You’ll need:

• A StormForge Account
• Kubernetes version 1.24 or later
• Helm version 3.14.0 or later
• kubectl properly configured for your cluster
• Cluster-Admin access (for installation only)
• A Kubernetes cluster and the cluster name to install the Agent into

Be sure to:

• Configure a proxy, if needed
• If you use a private container registry, import the required containers
• Configure firewall rules, including granting Kubernetes clusters access to these StormForge services endpoints on port 443:
  • api.stormforge.io
  • in.stormforge.io
  • registry.stormforge.io*
    (*If you configure the Agent to use an internal registry, you don’t need to grant cluster access to this endpoint).

Prepare to install

Install the StormForge CLI tool

The StormForge CLI is used to authenticate, retrieve credentials, and register the StormForge Agent on your Kubernetes cluster.

Download and install the CLI tool:

• Linux
• Windows
• macOS
• Docker

# Automatically selects either AMD64 or ARM64 architecture, downloads
# the appropriate binary, then moves it to a location in PATH
{ [ "$(uname -sm)" = "Linux x86_64"  ] && curl -L https://downloads.stormforge.io/stormforge-cli/latest/stormforge_linux_amd64.tar.gz | tar -xz; } ||
{ [ "$(uname -sm)" = "Linux aarch64" ] && curl -L https://downloads.stormforge.io/stormforge-cli/latest/stormforge_linux_arm64.tar.gz | tar -xz; } &&
sudo mv stormforge /usr/local/bin/

sudo mv stormforge /usr/local/bin/

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
If ($env:PROCESSOR_ARCHITECTURE -eq "AMD64") { Invoke-WebRequest -Uri "https://downloads.stormforge.io/stormforge-cli/latest/stormforge_windows_amd64.zip" -Outfile "stormforge.zip" }
If ($env:PROCESSOR_ARCHITECTURE -eq "ARM64") { Invoke-WebRequest -Uri "https://downloads.stormforge.io/stormforge-cli/latest/stormforge_windows_arm64.zip" -Outfile "stormforge.zip" }
Expand-Archive "stormforge.zip" "." -WarningVariable $w; if ($w.Count -eq 0) { Remove-Item "stormforge.zip" } 

brew install thestormforge/tap/stormforge

# Automatically selects either AMD64 or ARM64 architecture, downloads
# the appropriate binary, then moves it to a location in PATH
{ [ "$(uname -sm)" = "Darwin x86_64" ] && curl -L https://downloads.stormforge.io/stormforge-cli/latest/stormforge_darwin_amd64.tar.gz | tar -xz; } ||
{ [ "$(uname -sm)" = "Darwin arm64"  ] && curl -L https://downloads.stormforge.io/stormforge-cli/latest/stormforge_darwin_arm64.tar.gz | tar -xz; } &&
sudo mv stormforge /usr/local/bin/

docker pull registry.stormforge.io/library/stormforge-cli

Log in

From the command line, log into your StormForge account:

stormforge login

Alternatively, you can get a one-time use code to enter into a browser from another device:

stormforge login --url

At any time, you can check to see that you can communicate with the API:

stormforge ping

After you log in, go back to your command line session.

Generate an access credential

If the Agent is already installed on a cluster, you can reuse the existing credential: locate the file that contains the credential, and skip to this section later in this topic.

If the Agent doesn’t exist on your estate, generate a credential by running the following command.

• Replacing CREDENTIAL_NAME with a name that will help you identify the file (such as the name of the cluster you’re installing on).
• Replace CREDENTIAL_FILE with a filename that will help you to identify the file that contans the credential (no filename extension is needed).

stormforge auth create CREDENTIAL_NAME > CREDENTIAL_FILE

The credentials file should look something like this:

stormforge: 
   address: https://api.stormforge.io/ 
authorization: 
   issuer: https://api.stormforge.io/ 
   clientID: CLIENT_ID 
   clientSecret: CLIENT_SECRET

Install the StormForge Agent - advanced scenarios

If you installed a 2.0.x version of the stormforge-agent Helm chart

You must delete that version before installing the latest version.

kubectl delete deploy stormforge-agent -n stormforge-system

If you have a proxy server

Before you install the Agent, create a proxy.yaml server file (shown below) for the Prometheus container, which is installed with the Agent. Replace PROXY_HOST with the appropriate hostname.

When you install the Agent (or the Applier), append the --values proxy.yaml argument to the Helm command.

---
proxyUrl: https://PROXY_HOST:3128
noProxy: 10.100.0.0/16,kubernetes.default.svc

Note: The no_proxy parameter should include the network range that enables direct communication with the cluster and nodes. For example, to get this cluster value, run the following command and append a subnet mask sized appropriately for your environment to the CLUSTER-IP value.

kubectl get svc kubernetes -n default

The output will look something like this:

NAME        TYPE      CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
kubernetes  ClusterIP 10.100.0.0     <none>       443/TCP   6d17h

Using this example, you could use a value of 10.100.0.0/16.

If you have a private container registry

From a terminal that has direct internet access, run the following commands to import the Agent and Prometheus containers and push them to your repository.

Important:

• In the docker tag command, make sure that the TARGET_IMAGE path uses the same slash-separated name components and tag as the SOURCE_IMAGE path (/optimize/workload-agent:2.13.0, /prometheus/prometheus:v2.53.1).
• Replace $HOSTNAME with the private container registry location.

$ docker pull registry.stormforge.io/optimize/workload-agent:2.13.0
$ docker tag registry.stormforge.io/optimize/workload-agent:2.13.0 $HOSTNAME/optimize/workload-agent:2.13.0
$ docker push $HOSTNAME/optimize/workload-agent:2.13.0
$ docker pull quay.io/prometheus/prometheus:v2.53.1
$ docker tag quay.io/prometheus/prometheus:v2.53.1 $HOSTNAME/prometheus/prometheus:v2.53.1
$ docker push $HOSTNAME/prometheus/prometheus:v2.53.1

When you install the Agent, append these arguments to the Helm install command:

--set workload.image.repository=$HOSTNAME/optimize/workload-agent \
--set prom.image.repository=$HOSTNAME/prometheus/prometheus \

Install the Agent

The oci://registry.stormforge.io/library/stormforge-agent Helm chart creates and uses a ServiceAccount called stormforge-agent and binds it to the Kubernetes view ClusterRole, granting read-only permissions to all resources in the cluster.

To install the Agent, run the following command:

• Replace CREDENTIAL_FILE with the name of the file that contains your access credential.

• Replace CLUSTER_NAME with the name of the cluster (lowercase, no underscore) you’re installing the Agent on.

• Include any extra arguments from the Agent configuration topic.

• Note: If you cannot run helm install, skip to this section later in this topic.

helm install stormforge-agent oci://registry.stormforge.io/library/stormforge-agent \ 
  --namespace stormforge-system \ 
  --create-namespace \
  --values CREDENTIAL_FILE \
  --set clusterName=CLUSTER_NAME

If the Agent didn’t install, see the Installation section in the Troubleshooting topic.

The Agent is now sending metrics to the cloud service. In approximately 10 minutes, a preliminary recommendation for each workload will be available for you to view in the Optimize Live UI.

Validate the installation

To see your cluster and workloads:

1. Log into https://app.stormforge.io with your StormForge account.
2. In the left navigation, click Workloads.
3. Optional: Refine your search by cluster, namespace, or workload name; press Enter, and review the results.

If you don’t see what you expect, try troubleshooting the Agent.

Should you install the StormForge Applier?

The StormForge Applier, an optional component of Optimize Live, applies the recommended CPU and memory settings to objects, mostly commonly workloads and HPAs, in the cluster.

Install the StormForge Applier if you plan to:

• Deploy recommendations on demand. You can apply a single recommendation in any environment as you experiment with recommendations or if you need to quickly deploy a recommendation outside of a schedule.
• Deploy recommendations automatically. Define a schedule to receive recommendations, and Optimize Live will generate and deploy them. This option enables you to immediately realize the benefits of right-sizing your workloads without manual intervention.

You do not need to install the StormForge Applier if you plan to:

• Deploy recommendations as part of a GitOps workflow (for example, in a staging or production environment)
• Export the patches manually from the Optimize Live web application.

The oci://registry.stormforge.io/library/stormforge-applier Helm chart creates and uses a ServiceAccount called stormforge-applier and binds it to the Kubernetes edit ClusterRole, granting update and patch permissions to all optimizable workloads (and HPA, if enabled). You can grant additional permissions by specifying additional RBAC in the Helm install command.

Continuous reconciliation and the Applier

You can control whether the Applier continuously reconciles workload requests or limits to the recommended settings by setting the Applier reconciliation.mode. For details, see Continuous reconciliation in the Applier configuration topic.

Install the Applier

First, make sure that the Agent is installed:

kubectl get pods -A | grep 'stormforge-agent'

For additional Applier configuration options, see the Applier configuration topic.

If you use a private container registry, import and tag the Applier container and push it to your repository (and don’t change the /optimize/applier:2.4.0 name component and tag):

• Replace $HOSTNAME with the private container registry location.

$ docker pull registry.stormforge.io/optimize/applier:2.4.0
$ docker tag registry.stormforge.io/optimize/applier:2.4.0 $HOSTNAME/optimize/applier:2.4.0
$ docker push $HOSTNAME/optimize/applier:2.4.0

To install the Applier, run the following command:

• If you use a proxy server, add the -f proxy.yaml flag after the repository name.
• Replace $HOSTNAME with the private container registry location.
• If you cannot run helm install, skip to this section.

$ helm install stormforge-applier \
oci://registry.stormforge.io/library/stormforge-applier \
  -n stormforge-system \
  --set image.repository=$HOSTNAME/optimize/applier

To validate the installation, run:

kubectl get pods -A | grep 'stormforge-applier'

The output should look something like this:

stormforge-system    stormforge-applier-8554758f5b-whbc4             1/1     Running   0              20s

Additional installation considerations

If you cannot run helm install

Some environments cannot run Helm installations. StormForge relies on Helm to generate the templates required for installation; therefore, Helm is a strict requirement. However, you can run Helm to generate the template files to be applied to your cluster using standard kubectl commands:

• If you use a proxy server, add the -f proxy.yaml flag before --output-dir ..
• Replace CREDENTIAL_FILE with the name of the file that contains your access credential.
• Replace CLUSTER_NAME with the name of the cluster (lowercase, no underscore) you’re installing the Agent on.
• Include any extra arguments from the Agent configuration topic.

helm template stormforge-agent \
  oci://registry.stormforge.io/library/stormforge-agent \
  --namespace stormforge-system \
  --values CREDENTIAL_FILE \
  --set clusterName=CLUSTER_NAME \
  --output-dir .

The helm template command above creates manifests in a stormforge-agent subdirectory. To apply the manifests to your cluster, run:

kubectl create ns stormforge-system; kubectl apply -f stormforge-agent --recursive=true

Workload discovery and metrics collection

In a default Agent installation:

• The Agent discovers and gathers metrics for all DaemonSet, HPA, Pod, ReplicaSet, ReplicationController, and StatefulSet types across all the namespaces on the cluster on which it’s installed.
• Custom resources are ignored and no metrics are collected on them. If you have custom resources in your environment or if you prefer not to have workloads discovered automatically, contact us for guidance on how to modify the Helm chart.

What happens after you install?

Optimize Live starts collecting metrics on workloads as soon as you install the Agent.

First 7 days: metrics collection and preliminary recommendations

Ten minutes after installation, after some initial metrics are collected, Optimize Live provides its first preliminary recommendation to give you an evolving preview of potential optimization impact. 

Preliminary recommendations are then generated:

• Hourly, for the first 24 hours after installing the Agent

• Once daily, on days 2 to 7 after installing the Agent

  Preliminary recommendations can be viewed, but should not be applied because they’re not based on a complete set of metrics.

It typically takes about 7 days to collect enough metrics to generate an optimal recommendation (referred to as a complete recommendation) for you to apply.

For this reason, we recommend waiting 7 days before applying recommendations.

After 7 days: Next steps

After we collect 7 days of metrics, you’ll receive a complete recommendation once daily (the default schedule).

By default, recommendations are not applied automatically, giving you control over how they’re applied.

When you are satisfied that the recommendations are safe to apply, you can enable automatic deployment for some or all of your workloads by setting the live.stormforge.io/auto-deploy annotation to "true" as described in the Auto-deploy topic.

You can also change the recommendation schedule based on how closely you want to track CPU and memory utilization and your tolerance for pod churn.

Review the optimization settings
Review the settings that Optimize Live uses to generate recommendations for a workload: In the StormForge UI, click Workloads, then click a workload name and go to the Settings tab.

Change the optimization settings (optional)
If you’re satisfied with the default optimization settings, you don’t need to change them. If you would like to change any values, it’s considered a best practice to configure your default optimization settings at the cluster level and then define exceptions at the namespace and workload levels. See the topics under Configure optimization settings for details.

Helm charts and troubleshooting

To learn more about the Agent and Applier Helm charts and steps to troubleshoot, you can run the following commands:

helm show readme oci://registry.stormforge.io/library/stormforge-agent

helm show readme oci://registry.stormforge.io/library/stormforge-applier

To learn about other parameters that you can configure, run the following commands:

helm show values oci://registry.stormforge.io/library/stormforge-agent

helm show values oci://registry.stormforge.io/library/stormforge-applier

Related topics

• Recommendations (generating on demand, viewing and applying)