Configure default optimization settings

Learn how to:

• Automatically deploy recommendations
• Set a schedule
• Set thresholds to skip low-impact recommendations
• Configure CPU and memory optimization goals
• Specify what to optimize for specific containers (set an optimization policy)
• Change the limit-to-request ratio
• Stop generating recommendations

Note: You can learn more about how Optimize Live generates recommendations in the Concepts topic.

Key Points

• Use the Optimize Live UI to configure optimization settings for one workload at a time, which can be helpful in testing workloads.

• Use annotations to configure optimization settings at scale at the workload, namespace, and cluster levels.

  This topic doesn’t discuss annotations in detail. If you plan to use annotations, take a few minutes to read the Configure by using annotations topic — it lists the supported annotations and will help you to understand where to place annotations to configure settings at the workload, namespace, and cluster levels.

• Workloads whose settings are managed in any way by annotations become view-only in the UI.

  To edit the workload in the UI, you must remove the cluster-defaults ConfigMap, namespace-level annotations, and workload-level annotations that pertain to the workload.

Automatically deploy recommendations

By default, Optimize Live doesn’t apply (deploy) recommendations automatically. See the best practices below.

Best practices:

• Review and selectively apply the first few recommendations to see what happens. Then, as you trust recommendations more, you can enable them to be applied automatically on more workloads.
• To reduce pod churn, you can set deployment thresholds. Optimize Live will apply a recommendation only if that threshold is met.

Steps

Check whether the StormForge Applier is installed. On the command line, run:

helm ls -a -n stormforge-system

If you don’t see the stormforge-applier-2.*.* chart listed in the CHART column of the output, install it.

Next, configure auto-deployment using one of these methods:
UI:

1. In the left navigation, click Optimize Live > Workloads, then find and click the name of the workload you want to work with.
2. On the workload details page, click Config, and set Automatic Deployment to On.
3. Click Update to save your changes.

Annotations:
Decide at which level to set the value and annotate accordingly (refer to Configure by using annotations for details):

• Workload or namespace level: Annotate the workload YAML manifest or the namespace YAML manifest with the live.stormforge.io/auto-deploy annotation. Example: live.stormforge.io/auto-deploy: "true"
• Cluster level: Add the autoDeploy: "VALUE" parameter:value pair to the clusterDefaultConfig values in a cluster-defaults.yaml file. Example: autoDeploy: "true".

Set a schedule

Specify how often you want Optimize Live to generate and apply recommendations for a workload.

Prerequisites

• The Applier must be installed.
• Automatic deployment must be enabled.

Key points:

• When automatic deployment is enabled, Optimize Live generates recommendations once daily (@daily) by default, which is considered a best practice in stable or production environments.
• Short intervals (such as hourly or every few hours) provide close tracking of utilization with quick-changing short-lived recommendations. Useful when in automatic deployment mode.
• Longer intervals (such as weekly) will produce longer-lived recommendations, which is useful for integrating with slower-moving CI or applying recommendations manually.
• To stop generating recommendations, set the live.stormforge.io/schedule annotation to "@never" or "P0D" as described in this section later in this topic.

Steps

To specify how often recommendations are generated and how long they’re valid for:
UI:

1. In the left navigation, click Optimize Live > Workloads, then find and click the name of the workload you want to work with.
2. On the Workload page, click Config.
3. In the Recommendation Schedule section, specify how often you want to receive recommendations.
4. To save your changes, click Update.

Annotations:
Use macros, ISO 8601 Duration strings like "P1D", or Cron format to specify a schedule. Decide at which level to set the value and annotate accordingly (refer to Configure by using annotations for details):

• Workload or namespace level: Annotate the workload YAML manifest or the namespace YAML manifest with the live.stormforge.io/schedule annotation. Example: live.stormforge.io/schedule: "@daily"
• Cluster level: Add the schedule: "VALUE" parameter:value pair to the clusterDefaultConfig values in a cluster-defaults.yaml file. Example: schedule: "P1D".

Set thresholds to skip low-impact recommendations

Optional. Setting unit-change and percent-change thresholds can help to reduce pod churn. When choosing threshold values, be sure to consider the types of containers in the workload.

Best practice: Set both a percent and a unit threshold.

If, for at least one container, the recommendation satisfies the thresholds for at least one resource — as in, CPU (unit AND percent threshold) OR memory (unit AND percent threshold) — the recommendation is applied automatically.

If you set a percent threshold only, small unit changes for small containers can be a large enough percentage to meet the percent threshold, thus deploying the recommendation and causing unnecessary churn. Setting percent and unit thresholds ensures small changes don’t trigger a deployment.

Key points:

• Setting thresholds at the cluster level provides the most value, compared to setting them on individual workloads.
• You can set thresholds only on workloads that have automatic deployment enabled.
• Important: If later you disable automatic deployment, thresholds are not preserved. For this reason, consider defining them in a YAML manifest.
• When setting a threshold, you must specify a positive integer value for both CPU and memory.

Example

Suppose you set percent thresholds only:

thresholds:
  cpu:
    minPercentChange: 8
  memory:
    minPercentChange: 5

Now consider the following current and recommended requests values for two containers in the same workload:

• 
  The small unit changes result in large percent changes: 10.53% for CPU requests and 11.33% for memory requests, which meet the threshold.
• 
  Even with larger unit changes, neither the 7.73% CPU requests change nor the 2.61% memory requests change meet the percent thresholds.

But because at least one container - in this case, apache - meets the percent thresholds, the recommendation will be deployed, which for this workload, will cause unnecessary churn.

However, setting percent and unit thresholds would not trigger a deployment:

thresholds:
  cpu:
    minUnitChange: 50m
    minPercentChange: 8
  memory:
    minUnitChange: 40Mi  
    minPercentChange: 5

For those same containers:

• apache: Only the unit change is satisfied, which is not sufficient to trigger deployment.
• php-fpm: Both the unit and percent thresholds are satisfied (and in this case, for both CPU and memory), which is sufficient to trigger deployment.

Steps

To configure thresholds:
UI:

1. In the left navigation, click Optimize Live > Workloads, then find and click the name of the workload you want to work with.
2. On the Workload page, click Config, and set Automatic Deployment to on if it isn’t already set.
3. Enable either the percent-change or unit-change threshold, or both, based on the amount of change you want to enforce.
4. For CPU and Memory, set the smallest amount of change to requests that would make it worthwhile to deploy the recommendation.
5. To save your changes, click Update.

To remove a threshold, click its toggle to the off position and then click Update.

Annotations:
Decide at which level to set the value and annotate accordingly (refer to Configure by using annotations for details). Remember to include the live.stormforge.io/auto-deploy: "true" annotation.

More detailed examples are shown in the Configure by using annotations topic.

• Workload or namespace level: Annotate the workload YAML manifest or the namespace YAML manifest with one or both of the following pairs of annotations. Replace the sample values with the values you need.

  Percent-change:

  live.stormforge.io/auto-deploy.thresholds.cpu.percent: "5"
  live.stormforge.io/auto-deploy.thresholds.memory.percent: "10"
  

  Unit-change:

  live.stormforge.io/auto-deploy.thresholds.cpu.unit: "10m"
  live.stormforge.io/auto-deploy.thresholds.memory.unit: "10Mi"
  

• Cluster level: Add either or both of the following parameter:value pairs to the clusterDefaultConfig values in a cluster-defaults.yaml file. Remember to include the autoDeploy: "true" pair. Replace the sample values with the values you need.

  Percent-change:

  autoDeployThresholdsCpuPercent: "5"
  autoDeployThresholdsMemoryPercent: "10"
  

  Unit-change:

  autoDeployThresholdsCpuUnit: "10m"
  autoDeployThresholdsMemoryUnit: "10Mi"  
  

CLI:
Run stormforge edit workload handlers with either or both pairs of arguments. Remember to include the --auto-deploy (or equivalent --autodeploy=true) argument. Replace the VALUE and VALUE_AND_UNIT with the values you need.

Percent-change arguments:

  --auto-deploy-thresholds-min-percent-change-cpu=VALUE    
  --auto-deploy-thresholds-min-percent-change-memory=VALUE

Unit-change arguments:

--auto-deploy-thresholds-min-unit-change-cpu=VALUE_AND_UNIT
--auto-deploy-thresholds-min-unit-change-memory=VALUE_AND_UNIT

Examples:

• Apply only recommendations that change CPU requests by at least 5% for at least one container. Suppose for this example, memory requests don’t matter to you - you must still set the memory threshold to a positive integer value.

  stormforge edit workload WORKLOAD_NAME handlers \
  --auto-deploy=true \
  --auto-deploy-thresholds-min-percent-change-cpu=5 \
  --auto-deploy-thresholds-min-percent-change-memory=1 \
  

• Apply only recommendations that satisfy either (or both) of the following conditions for at least one container:

  • CPU requests change by at least 5% and 10m

  • Memory requests change by at least 10% and 10Mi

    stormforge edit workload WORKLOAD_NAME handlers \
    --auto-deploy=true \
    --auto-deploy-thresholds-min-percent-change-cpu=5 \
    --auto-deploy-thresholds-min-percent-change-memory=10 \
    --auto-deploy-thresholds-min-unit-change-cpu=10m \
    --auto-deploy-thresholds-min-unit-change-memory=10Mi 
    

To remove a threshold, run stormforge edit workload handlers with the appropriate threshold values set to 0. For example, to remove the unit-change threshold, run the following command:

stormforge edit workload WORKLOAD_NAME handlers \
  --auto-deploy=true \
  --auto-deploy-thresholds-min-unit-change-cpu=0 \
  --auto-deploy-thresholds-min-unit-change-memory=0 

To see additional arguments (for example, for applying these settings to all workloads in a cluster or in a specific workspace), see stormforge edit workloads handlers or run stormforge edit workloads handlers --help.

Configure CPU and memory optimization goals

Based on your risk profile, choose a value for the CPU optimization goal and memory optimization goal:

• savings: Provides recommendations that are closer to actual CPU or memory usage. Consider this option when you want to maximize your resource savings.
• balanced: Default value. Provides a balanced approach to achieving cost savings and increased reliability.
• reliability: Minimizes the risk of hitting CPU or memory limits. Consider this option for business-critical applications.

You can set different CPU and memory optimization goals. For example, if your organization can tolerate throttling when containers exceed CPU limits but cannot tolerate restarts when containers exceed memory limits, you can set a savings goal for CPU and a reliability goal for memory.

Steps

To configure optimization goals:
UI:

1. In the left navigation, click Optimize Live > Workloads, then find and click the name of the workload you want to work with.
2. On the Workload page, click Config and choose your optimization goal.
3. To save your changes, click Update.

Annotations:
Decide at which level to set the value and annotate accordingly (refer to Configure by using annotations for details):

• Workload or namespace level: Annotate the workload YAML manifest or the namespace YAML manifest with one or both of the following annotations. Example:

  live.stormforge.io/cpu.optimization-goal: "Reliability"
  live.stormforge.io/memory.optimization-goal: "Savings"
  

• Cluster level: Add either or both of the following parameter:value pairs to the clusterDefaultConfig values in a cluster-defaults.yaml file. Example:

  cpuOptimizationGoal: "Reliability"
  memoryOptimizationGoal: "Savings"
  

Specify what to optimize (set an optimization policy): requests and limits, requests, or nothing

By default, only CPU and memory requests (not limits) are optimized for each container in a workload. You can adjust this value to optimize requests and limits; you can also exclude containers (such as sidecar containers) from optimization.

Steps

To specify what to optimize:
UI:

1. From the left navigation, click Optimize Live > Workloads, then find and click the name of the workload you want to work with.
2. On the Workload page, click Config.
3. In the Containers section, expand the container you want to work with.
4. In the Configure CPU and Configure Memory sections, select what you want to optimize.
   • To exclude a container (such as a sidecar container) from optimzation : In both the Configure CPU and Configure Memory sections, select Don’t optimize.
5. Repeat as needed for containers in the workload.
6. To save your changes, click Update.

Annotations:
Decide at which level to set the value and annotate accordingly (refer to Configure by using annotations for details):

• Workload or namespace level: Annotate the workload YAML manifest or the namespace YAML manifest with one or both of the following annotations. Example:

      live.stormforge.io/containers.cpu.optimization-policy: "RequestsAndLimits"
      live.stormforge.io/containers.memory.optimization-policy: "RequestsAndLimits"
  

• Cluster level: Add either or both of the following parameter:value pairs to the clusterDefaultConfig values in a cluster-defaults.yaml file. Example:

  containersCpuOptimizationPolicy: "RequestsAndLimits"
  containersMemoryOptimizationPolicy: "RequestsAndLimits"
  

Examples: Setting container-specific defaults

Suppose you want to set container-specific defaults, as in these examples:

• Default to optimizing CPU and memory requests only (and not limits). Several named containers are given exceptions: for the server and api containers, optimize requests and limits; for the sidecar container, do not optimize anything.

  • Workload or namespace level: In the workload YAML manifest or the namespace YAML manifest, add:

    live.stormforge.io/containers.cpu.optimization-policy: "RequestsOnly,server=RequestsAndLimits,api=RequestsAndLimits,sidecar=DoNotOptimize"
    live.stormforge.io/containers.memory.optimization-policy: "RequestsOnly,server=RequestsAndLimits,api=RequestsAndLimits,sidecar=DoNotOptimize"
    

  • Cluster level: In the clusterDefaultConfig values in the cluster-defaults.yaml file, add:

    containersCpuOptimizationPolicy: "RequestsOnly,server=RequestsAndLimits,api=RequestsAndLimits,sidecar=DoNotOptimize"
    containersMemoryOptimizationPolicy: "RequestsOnly,server=RequestsAndLimits,api=RequestsAndLimits,sidecar=DoNotOptimize"
    

• Assume all containers are set to optimize RequestsOnly (the default value). Suppose you want to optimize the server container CPU and memory requests and limits, and not optimize the sidecar container:

  • In the workload YAML manifest or the namespace YAML manifest, add:

    live.stormforge.io/containers.cpu.optimization-policy: server="RequestsAndLimits",sidecar="DoNotOptimize"
    live.stormforge.io/containers.memory.optimization-policy: server="RequestsAndLimits",sidecar="DoNotOptimize"
    

  • In the clusterDefaultConfig values in a cluster-defaults.yaml file, add:

    containersCpuOptimizationPolicy: server="RequestsAndLimits",sidecar="DoNotOptimize"
    containersMemoryOptimizationPolicy: server="RequestsAndLimits",sidecar="DoNotOptimize"
    

  In this example, the optimization policies are updated for the server and sidecar containers only.

Change the limit-to-request ratio (limitRequestRatio)

Important: Change this container-level setting only if you need a specific limit-to-request ratio requirement to ensure resource consumption doesn’t exceed requests.

Concepts and examples

Optimize Live uses this ratio to calculate recommended CPU and memory limits:
Recommended limits = recommended requests * limitRequestRatio

The default limitRequestRatio is 2.0, which means that the recommended limit will be double the recommended requests. For example, if the recommended CPU requests value is 100m, then the recommended limit would be 200m.

Optimize Live always calculates recommended CPU and memory limits values, but doesn’t apply them if a container is configured to optimize requests only.

You can configure the limitRequestRatio based on your container needs:

• 1.0 provides Guaranteed Quality of Service
• 2.0 is the default value
• A custom value equal to or greater than 1.0 (to two decimal places) provides more room for spikes or changes in consumption

Using limitRequestRatio in conjunction with resource limits
You can use both the limitRequestRatio and resource limits values together — they are not mutually exclusive. Optimize Live calculates the recommended limits value and then adjusts it if needed.

Suppose:

• You want to ensure a workload has at least 2 cores for startup requirements.
• You want a limitRequestRatio of 1.33.

Your container settings might look something like this:

containerSettings: 
  - cpu:
      requests:
        min: 20m
        max: 2000m
      limits:
        min: 2000m
        max: 16000m
        limitRequestRatio: 1.33

If Optimize Live recommends a cpu.requests value of 100m, then the calculated cpu.limits value is 133m (100 x 1.33), which is lower than cpu.limits.min of 2000m. Optimize Live would adjust the recommended cpu.limits value to 2000m to respect the cpu.limits.min=2000m setting.

Steps

To change the limitRequestRatio setting:
UI:

1. Navigate to the workload details page and click Config.
2. Expand the container you want to work with.
3. In both the CPU and Memory sections, set the Limit Request Ratio to a value equal to or greater than 1.0 (up to two decimal places).
   Tip:
   • 1.0 provides Guaranteed Quality of Service
   • 2.0 is the default value
4. Repeat as needed for other containers.
5. To save your changes, click Update.

Annotations:

Decide at which level to set the value and annotate accordingly (refer to Configure by using annotations for details):

• Workload or namespace level: Annotate the workload YAML manifest or the namespace YAML manifest with one or both of the following annotations. Example:

  live.stormforge.io/containers.cpu.limits.limit-request-ratio: "1.33"
  live.stormforge.io/containers.memory.limits.limit-request-ratio: "1.33"
  

• Cluster level: Add either or both of the following parameter:value pairs to the clusterDefaultConfig values in a cluster-defaults.yaml file. Example:

  containersCpuLimitsLimitRequestRatio: "1.33"
  containersMemoryLimitsLimitRequestRatio: "1.33"
  

Examples: Setting container-specific defaults

Suppose you want to set container-specific defaults, as in these examples:

• Set a ratio of 1.33 for all containers (overriding the default value of 2.0); specify exceptions for the server and api containers:

  • Workload or namespace level: In the workload YAML manifest or the namespace YAML manifest, add:

    live.stormforge.io/containers.cpu.limits.limit-request-ratio: "1.33,server=1.4,api=2.0"
    live.stormforge.io/containers.memory.limits.limit-request-ratio: "1.33,server=1.4,api=2.0"
    

  • Cluster level: In the clusterDefaultConfig values in the cluster-defaults.yaml file, add:

    containersCpuLimitsLimitRequestRatio: "1.33,server=1.4,api=2.0"
    containersMemoryLimitsLimitRequestRatio: "1.33,server=1.4,api=2.0"
    

• Override the current value for the server container only:

  • In the workload YAML manifest or the namespace YAML manifest, add:

    live.stormforge.io/containers.cpu.limits.limit-request-ratio: "server=1.4"
    live.stormforge.io/containers.memory.limits.limit-request-ratio: "server=1.4"
    

  • In the clusterDefaultConfig values in the cluster-defaults.yaml file, add:

    containersCpuLimitsLimitRequestRatio: "server=1.4"
    containersMemoryLimitsLimitRequestRatio: "server=1.4"
    

Stop generating recommendations

To stop generating recommendations for a workload, set the recommendation schedule to "@never" or "P0D".

• For all workloads in a cluster:
  Update the clusterDefaultConfig ConfigMap to include schedule: "@never" and then run helm install or helm upgrade to apply the changes. See this topic for detailed steps.

• For all workloads in a namespace, do one of the following:

  • Update the .metadata.annotations section of the namespace definition to include the live.stormforge.io/schedule: "@never" annotation. See this topic for detailed steps.
  • Upgrade the Agent, adding the --set 'workload.denyNamespaces={namespace1,namespace2} parameter. Replace namespace1,namespace2 with the appropriate namespaces.

• For a specific workload:
  Update the .metadata.annotations section of the workload definition to include the live.stormforge.io/schedule: "@never" annotation. See this topic for detailed steps.