Configure by using annotations

Optimize Live supports various resource annotations to control optimization behavior. You can annotate:

• Workloads (Deployments, StatefulSets, ReplicaSets), to customize optimization behavior for each individual workload
• Namespaces, to set defaults for all workloads inside the namespace

Using the same annotation keys and values as syntax, you can also:

• Edit the cluster-defaults ConfigMap, to set defaults for all workloads in the cluster

For example, platform teams can set organization-wide default workload values, and development teams can fine-tune settings depending on their application and environment (development, staging, test, production).

Key points:

• Think of settings at the cluster, namespace, and workload levels as a hierarchy of configuration.

  • Workload-level configuration has the highest precedence and cannot be overridden. You can set configuration for an individual workload by annotating it.
  • Namespace-level defaults apply to all workloads in a namespace. Default values set at this level fill in any settings not provided by annotations on the workloads themselves. You can set defaults for a namespace by annotating it.
  • Cluster-level defaults apply to all workloads in a cluster. Default values set at this level have the lowest precedence and can be overridden by values set at the namespace and workload levels. You can set defaults for the cluster in the cluster-defaults ConfigMap.

• You cannot use the UI to edit workload settings that are being managed in any way by annotations. These workloads will become view-only in the UI.

Topics:

• Available annotations
• Configure optimization settings for a specific workload
• Configure default optimization settings for all workloads in a namespace
• Configure default optimization settings for all workloads in a cluster
• Additional schedule examples

Available annotations

This table lists the resource annotations that Optimize Live supports to control optimization behavior. Enclose all values in double quotation marks ("").

Best practice: When editing container-level resources (items in the Container-level settings section in the table below, such as requests and limits), specify the value to apply to all or most containers first, followed by container-specific exceptions, as in these examples:

• When annotating a workload or namespace:
  live.stormforge.io/containers.cpu.optimization-policy: "RequestsAndLimits,sidecar=DoNotOptimize"
• In a cluster-default ConfigMap:
  containersCpuOptimizationPolicy: "RequestsAndLimits,sidecar=DoNotOptimize"

Configure optimization settings for a specific workload

Workspace-level defaults are defined by annotating the PodTemplateSpec metadata (spec.template.metadata.annotations) section of a Deployment or StatefulSet resource type. Be careful not to add them to the metadata for the workload itself (metadata.annotations).

The following example shows how to add annotations to a Deployment object. You can use this example as a template; just remember to replace APP_* and IMAGE_NAME with your specific application details.

Using YAML manifests

1. Get and open the workload definition in your favourite editor. This is commonly done using commands like:

   kubectl get deployment DEPLOYMENT -o yaml > FILENAME && $EDITOR FILENAME
   

   or:

   kubectl edit deployment DEPLOYMENT
   

2. You can copy the followingspec.template.metadata.annotations section from the example below and use it as a template. Include only the annotations you need, using the available annotations list above as your guide. Your definition should look something like this:

apiVersion: apps/v1 
kind: Deployment 
metadata:
  name: APP_NAME
  labels:
    app: APP_LABEL
spec:
  replicas: 2
  selector:
    matchLabels:
      app: APP_LABEL
  template:
    metadata:
      annotations:
        live.stormforge.io/schedule: "P1D"
        live.stormforge.io/auto-deploy: "false"
        live.stormforge.io/containers.cpu.requests.min: "20m"
        live.stormforge.io/containers.cpu.requests.max: "16000m"
        ...
    spec:
      containers:
      - name: server
        image: IMAGE_NAME
        ...
      - name: sidecar
        image: IMAGE_NAME
        ...

Configure default optimization settings for all workloads in a namespace

Namespace-level defaults are defined by annotating the namespace. You can either add annotations to the .metadata.annotations section of the namespace definition, or use kubectl annotate arguments.

apiVersion: v1
kind: Namespace
metadata:
  name: NAMESPACE
  annotations:
    live.stormforge.io/schedule: "P1D"
    live.stormforge.io/auto-deploy: "false"
    live.stormforge.io/containers.cpu.requests.min: "20m"
    live.stormforge.io/containers.cpu.requests.max: "16000m"
  ...

Note: When you set a parameter at the namespace level, its value overrides the equivalent value set at the cluster level, if it’s been set at the cluster level.

Using Yaml manifests

1. Get and open the namespace definition in your favorite editor. This is commonly done using commands like:

   kubectl get namespace NAMESPACE -o yaml > FILENAME && $EDITOR FILENAME
   

   or:

   kubectl edit namespace NAMESPACE
   

2. You can copy the .metadata.annotations list below and use it as a template. Delete what you don’t need, and set the appropriate values. Your namespace definition should look something like this:

   apiVersion: v1
   kind: Namespace
   metadata:
     name: NAMESPACE
     annotations:
       live.stormforge.io/schedule: "P1D"
       live.stormforge.io/auto-deploy: "false"
       live.stormforge.io/containers.cpu.requests.min: "20m"
       live.stormforge.io/containers.cpu.requests.max: "16000m"
       ... 
   

3. Save your changes. Apply them using kubectl apply if necessary.

   kubectl apply -f FILENAME
   

4. Optional: Check that the annotations are set at the namespace level. Run the following command; then, in the output, look for the annotations you added.

   kubectl get namespace NAMESPACE -o yaml
   

Using the kubectl annotate command

1. Run the kubectl annotate command. As an example, to set two individual optimization settings you might run something like the command below. Remember to replace NAMESPACE with the appropriate value.

   kubectl annotate namespace NAMESPACE \
     live.stormforge.io/schedule="P1D" \
     live.stormforge.io/auto-deploy="false" \
     live.stormforge.io/containers.cpu.requests.min: "20m" \
     live.stormforge.io/containers.cpu.requests.max: "16000m"
   

2. Optional: Check that the annotations are set at the namespace level. Run the following command; then, in the output, look for the annotations you added.

   kubectl get namespace NAMESPACE -o yaml
   

Configure default optimization settings for all workloads in a cluster

The cluster-level defaults are specified in a ConfigMap named cluster-defaults, in the stormforge-system namespace. The syntax used is to list setting annotation names and the desired value in the cluster-defaults.yaml data key.

apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-defaults
  namespace: stormforge-system
data:
  cluster-defaults.yaml: |
    live.stormforge.io/schedule: "P1D"
    live.stormforge.io/auto-deploy: "false"
    live.stormforge.io/containers.cpu.requests.min: "20m"
    live.stormforge.io/containers.cpu.requests.max: "16000m"
    ...    

You can create this ConfigMap yourself, but it may be easier to have Helm create and manage it for you.

Setting cluster default values using Helm

The stormforge-agent Helm chart has values you can pass to define each of the possible cluster-level default settings. If you define any of the clusterDefaultConfig values, Helm will create and manage the cluster-defaults ConfigMap for you.

Helm chart values for cluster default settings

1. Create a yaml values file (for example, cluster-defaults.yaml). You can copy the file below and use it as a template. Delete the parameter:value pairs that you don’t want to set.

   clusterDefaultConfig:
     schedule: "P1D"
     autoDeploy: "false"
     cpuOptimizationGoal: "balanced"
     memoryOptimizationGoal: "balanced"
     containersCpuOptimizationPolicy: "RequestsAndLimits"
     containersCpuRequestsMin: "20m"
     containersCpuRequestsMax: "16000m"
     containersCpuLimitsMin: "2000m"
     containersCpuLimitsMax: "16000m"
     containersCpuLimitsLimitRequestRatio: "1.33"
     containersMemoryOptimizationPolicy: "RequestsAndLimits"
     containersMemoryRequestsMin: "64Mi"
     containersMemoryRequestsMax: "32Gi"
     containersMemoryLimitsMin: "256Mi"
     containersMemoryLimitsMax: "64Gi"
     containersMemoryLimitsLimitRequestRatio: "1.33"
   

2. Pass in the values file you created using the --values flag when running the helm install or helm upgrade. For example:

   helm install stormforge-agent oci://registry.stormforge.io/library/stormforge-agent \
     --namespace stormforge-system \
     --set clusterName=CLUSTER_NAME \
     --values authorization.yaml \
     --values cluster-defaults.yaml
   

Additional schedule examples

Below are some additional examples of ways to set the live.stormforge.io/schedule (or schedule in a cluster-level ConfigMap) annotation using macros, ISO 8601 Duration, and Cron notation:

• Once daily (default value and best practice):
  "@daily", "P1D" or "H H * * *"
• Hourly (the shortest interval you can specify):
  "@hourly", "PT1H" or "H * * * *"
• Every 12 hours:
  "PT12H" or "H H/12 * * *"
• Every morning at approximately 0800h (exact time is not guaranteed):
  "00 08 * * *"
• Every Wednesday:
  "H H * * WED"