View and apply recommendations

A recommendation includes the CPU and memory requests and limits values (and recommended HPA configuration, if an HPA is running) that the machine learning has determined to be optimal for each container in a workload.

By default:

• Recommendations are not applied automatically, giving you the opportunity to review their projected impact on your workloads. You can configure Optimize Live to deploy recommendations automatically, as described later in this topic.
• Recommendations are applied as patches. You can choose to configure Optimize Live to use a mutating admission webhook to apply recommendations, as described later in this topic.

Viewing the overall projected impact of a recommendation

To understand how a recommendation will right-size a workload, review the workload details page header:

• Optimization Score measures how well your current request settings align with Optimize Live’s recommendations. A score of 100 indicates perfect alignment.

• Recommendation Summary

  • Current Requests shows the current requests multiplied by the average number of replicas over the last 7 days.

  • Recommended shows the recommended requests multiplied by the average number of replicas over the last 7 days.

  • Net Impact shows the difference between the current and recommended total CPU and memory requests.

  Note: To change the CPU and memory cost/hr values used to calculate the estimated monthly costs, click Settings in the upper-right corner of the page.

A note about HPA recommendations

If a HorizontalPodAutoscaler (HPA) is enabled on the workload, Optimize Live detects the metric (or metrics) that the HPA is configured to scale on.

Scale-down events

When a workload is scaled down for any reason, by any actor, the duration of the event determines whether a recommendation is generated:

• 0 replicas for more than 75% of the time within a 7-day period: No recommendation provided. Optimize Live cannot collect sufficient metrics about that workload. When this occurs, you’ll see a message in the UI.
• 0 replicas for any shorter duration: Recommendation provided, and you’ll see a message in the UI.

Viewing recommendation details in the Optimize Live web application

To view the recommended CPU and memory settings for each container in a workload, navigate to a workload details page, click Recommendation, and then click View beside a container name in the Container Values section.

The container-level charts show the projected impact of using the recommended settings, averaged across the time period shown on the graph. Hover at any point on the graph to see details for a point in time.

Viewing recommendations from the command line

Run the following command, replacing WORKLOAD_NAME with your workload name:

$ stormforge get recommendations WORKLOAD_NAME -o json

Here’s an excerpt from the recommendation:

         "goal": "savings",
          "values": {
            "containerResources": [
              {
                "containerName": "respod",
                "requests": {
                  "cpu": "0",
                  "memory": "1099366"
                }
              }
            ],
            "autoScaling": {
              "metrics": [
                {
                  "resource": {
                    "name": "cpu",
                    "target": {
                      "averageUtilization": 90
                    }
                  }
                }
              ]
            }

Applying workload recommendations

During a workload’s learning period, the preliminary recommendations generated by Optimize Live can by applied manually only. However, a best practice is to wait until the learning period is complete before applying any recommendations.

After the learning period, when you’re satisfied with how the generated recommendations can right-size your workloads, you can deploy them in any of the ways described below.

Choosing an apply method - patch or webhook?

By default Optimize Live applies recommendations as patches to a workload.

You can choose to configure Optimize Live to apply recommendations directly to Pods via a mutating admission webhook. You might choose the webhook method if you use third-party tools to monitor for configuration drift or if you have many custom resources.

The Applier component of Optimize Live applies the recommendation using the configured apply method (either PatchWorkloadResources or DynamicAdmissionWebhook). For details, see the Apply method section in the Apply settings topic.

If you use the PatchWorkloadResources apply method, you can choose to configure the Applier to automatically reconcile workload drift, ensuring that recommended settings are maintained and not overwritten during CI/CD or deployment activity on the cluster. For details, see Continuous reconciliation in the Applier configuration topic.

Deploy recommendations on demand

Requires the StormForge Applier.

You can apply a single recommendation in any environment as you experiment with recommendations or if you need to quickly deploy a recommendation outside of a schedule.

1. In the left navigation of the Optimize Live web application, click Workloads and then click a workload name.
2. On the workload page, click Apply Now in the workload details page header.

The Applier applies the recommendation using the configured apply method (either PatchWorkloadResources or DynamicAdmissionWebhook). The next recommendation will be available to apply at the time shown in the Next scheduled field in the workload page header.

Deploy recommendations automatically

Requires the StormForge Applier.

This option enables you to immediately realize the benefits of right-sizing your workloads. It also saves you time and toil by relieving you of the task of manually reviewing recommendations on a schedule.

To enable automatic deployment, set the live.stormforge.io/auto-deploy annotation to "true" as described in the Apply settings topic.

The Applier applies the recommendation using the configured apply method (either PatchWorkloadResources or DynamicAdmissionWebhook).

By default, a new recommendation is applied once daily. You can change the recommendation schedule based on how closely you want to track CPU and memory utilization and your tolerance for pod churn.

When you enable automatic deployment, you can also configure these features:

• Deployment thresholds: To reduce pod churn from automatically applying low-impact recommendations, you can set deployment thresholds. The Applier will apply a recommendation only if the minimum change thresholds are met.
• Continuous reconciliation of workload drift: When enabled, recommended requests and limits values are maintained and not overwritten during CI/CD or deployment activity on the cluster, except when changes are made by field managers that you declare. For details, see Continuous reconciliation in the Applier configuration topic.
• Automatic memory bump-ups in response to out-of-memory (OOM) events: For each container in a workload, you can define the memory bump-up percentage, min and max bump-up amounts, and when Optimize Live applies the bump-up. For details, see the Reliability response topic.

Apply recommendations as patches as part of a CI/CD workflow

You can apply recommendations both automatically and on demand without the StormForge Applier.

• Automatically using the StormForge CLI:
  • Run the stormforge apply command with the appropriate flags. See the CLI reference docs for details.
     
• On demand using kubectl:

  1. On the Patches tab of the workload details page, download or copy the patches.

  2. Apply the patch using the kubectl command provided.

     Because this method doesn’t use the Applier, the apply method is ignored (if set).

Next steps

After applying a recommendation, notice the workload’s updated optimization score on the Workloads page or the workload details page.

The next recommendation will be available to apply at the interval specified in the Next scheduled field in the workload details page header.

If you’re satisfied with how often you receive recommendations, no further action is needed on this workload. Otherwise, you can change the recommendation schedule.

To roll back a recommendation, use Git or Kube-native tools.