Configure applier alerts

The StormForge Applier can send an HTTP webhook notification each time it successfully applies a recommendation to a workload. You can configure one or more endpoints to receive these alerts, with optional custom payload formatting and bearer token authentication. Use this to connect StormForge to your existing observability or incident management workflows.

How alerts work

When the Applier applies a patch, it sends an HTTP POST to each configured endpoint. The request includes the header X-Event-Type: recommendation applied. The body contains either the full JSON recommendation object (default) or a custom payload that you can define using a Go template.

Alert payload

Default payload

If you don’t set payloadTemplate, the Applier sends the full recommendation object.

{
  "patches": [
    {
      "patch": {
        "apiVersion": "apps/v1",
        "kind": "Deployment",
        "metadata": {
          "name": "cartservice"
        },
        "spec": {
          "template": {
            "spec": {
              "containers": [
                {
                  "name": "server",
                  "resources": {
                    "requests": {
                      "cpu": "10m",
                      "memory": "88Mi"
                    }
                  }
                }
              ]
            }
          }
        }
      },
      "patchTarget": {
        "apiVersion": "apps/v1",
        "kind": "Deployment",
        "name": "cartservice"
      },
      "patchType": "application/strategic-merge-patch+json"
    }
  ],
  "recommendation": "https://api.stormforge.io/v3/workloads/my-cluster/hipster/deployments/cartservice/recommendations/1234567890",
  "workload": {
    "apiVersion": "apps/v1",
    "cluster": "my-cluster",
    "kind": "Deployment",
    "name": "cartservice",
    "namespace": "hipster",
    "resource": "deployments",
    "type": "deployment.apps"
  }
}

Custom payload template

The payloadTemplate field accepts a Go template string with the following variables:

You can also use the toPrettyJson helper function to render .patches[].patch as formatted JSON.

Example template:

payloadTemplate: |
  {
    "recommendation": "{{ .recommendation }}",
    "workload": "{{ .workload.name }}",
    "namespace": "{{ .workload.namespace }}",
    {{- if .patches }}
    "patches": [
      {{- range $index, $patch := .patches }}
      {
        "target": "{{ $patch.patchTarget.name }}",
        "kind": "{{ $patch.patchTarget.kind }}",
        "patch": "{{ toPrettyJson $patch.patch }}"
      },
      {{- end }}
    ]
    {{- end }}
  }

Configure the endpoint

Add an alerts block to your Applier values file (applier-values.yaml: create this if it doesn’t exist) with at least one endpoints block. Omit the authorization block if the endpoint accepts unauthenticated requests.

Example alerts block:

alerts:
  enabled: true
  endpoints:
    - name: my-webhook           # Identifier for this endpoint
      url: https://example.com/webhook
      method: POST
      contentType: application/json
      payloadTemplate: |         # Omit to send the full recommendation object
        { ... }
      authorization:
        type: Bearer
        credentialsFile: /etc/secrets/my-token  # Path to token value file

Store credentials securely

To use bearer token authentication, mount a Kubernetes secret into the Applier pod.

1. Create a Kubernetes secret in the stormforge-system namespace:

   kubectl -n stormforge-system create secret generic stormforge-alerts-token \
     --from-literal=token='YOUR_BEARER_TOKEN'
   

2. Add the following volume and volume mount to your Applier values file:

   additionalVolumes:
     - name: alerts-token
       secret:
         secretName: stormforge-alerts-token
         items:
           - key: token
             path: alerts-token
   
   additionalVolumeMounts:
     - name: alerts-token
       mountPath: /etc/secrets
       readOnly: true
   

3. Set credentialsFile in the endpoint config to /etc/secrets/alerts-token.

Integration examples

Slack

Use a Slack Incoming Webhook URL as the endpoint url. Omit the authorization block, as Slack embeds authentication in the URL.

The Slack payload format uses a text field or Slack’s Block Kit UI framework.

Example alerts block:

alerts:
  enabled: true
  endpoints:
    - name: slack
      url: https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK
      method: POST
      contentType: application/json
      payloadTemplate: |
        {
          "text": "StormForge applied a recommendation to *{{ .workload.name }}* ({{ .workload.namespace }})"
        }

Amazon Simple Notification Service

Amazon Simple Notification Service (SNS) requires AWS Signature Version 4 (SigV4)-signed API calls and doesn’t accept arbitrary HTTP POSTs directly. To bridge this, deploy an AWS API Gateway and a Lambda function as a webhook receiver. The Lambda function validates the bearer token and publishes the payload to an SNS topic.

Workflow: Applier → HTTPS (Bearer token) → API Gateway → Lambda → SNS topic → subscribers (for example, Amazon Simple Queue Service (SQS), email)

Provision AWS components

Create the following AWS resources:

• An SNS topic to receive messages
• A Lambda function that validates the bearer token and publishes to the SNS topic
• An API Gateway HTTP API with a POST /webhook route fronting the Lambda function
• (Optional) An SQS queue subscribed to the SNS topic, for consuming messages

Use the following settings as a starting point when provisioning these resources:

export AWS_REGION=us-east-1
export TOPIC_NAME=stormforge-alerts        # SNS topic name
export API_NAME=stormforge-sns-webhook     # API Gateway name
export FN_NAME=stormforge-sns-publisher    # Lambda function name
export TOKEN=$(openssl rand -hex 24)       # Bearer token (auto-generated)
export QUEUE_NAME=stormforge-alerts-queue  # Optional: SQS queue for consuming messages

After provisioning, record the Webhook URL (the API Gateway endpoint) and the Bearer token. You’ll need both to configure the Applier.

Create the Kubernetes secret

Create a secret in the stormforge-system namespace using the bearer token from the provisioning step:

kubectl -n stormforge-system create secret generic stormforge-alerts-token \
  --from-literal=token='YOUR_BEARER_TOKEN'

Configure the Applier

Add the following to your Applier values file, replacing the url with your API Gateway endpoint:

alerts:
  enabled: true
  endpoints:
    - name: sns-webhook
      url: https://YOUR_API_ID.execute-api.YOUR_REGION.amazonaws.com/prod/webhook
      method: POST
      contentType: application/json
      payloadTemplate: |
        {
          "recommendation": "{{ .recommendation }}",
          "workload": "{{ .workload.name }}",
          "namespace": "{{ .workload.namespace }}",
          {{- if .patches }}
          "patches": [
            {{- range $index, $patch := .patches }}
            {
              "target": "{{ $patch.patchTarget.name }}",
              "kind": "{{ $patch.patchTarget.kind }}"
            },
            {{- end }}
          ]
          {{- end }}
        }
      authorization:
        type: Bearer
        credentialsFile: /etc/secrets/alerts-token

additionalVolumes:
  - name: alerts-token
    secret:
      secretName: stormforge-alerts-token
      items:
        - key: token
          path: alerts-token

additionalVolumeMounts:
  - name: alerts-token
    mountPath: /etc/secrets
    readOnly: true

Test by consuming from SQS (optional)

If you set up the SQS queue, poll it to verify messages arrive after the Applier applies a recommendation:

aws sqs receive-message \
  --region YOUR_REGION \
  --queue-url https://sqs.YOUR_REGION.amazonaws.com/YOUR_ACCOUNT_ID/stormforge-alerts-queue \
  --max-number-of-messages 10 \
  --wait-time-seconds 20

Apply the configuration

helm upgrade --install stormforge-applier \
  oci://registry.stormforge.io/library/stormforge-applier \
  --namespace stormforge-system \
  -f applier-values.yaml \
  --reset-then-reuse-values

Verify

1. Enable debug mode in your Applier values file (debug: true) and redeploy:

   helm upgrade --install stormforge-applier  \
   oci://registry.stormforge.io/library/stormforge-applier \
   --namespace stormforge-system \
   --set debug=true \
   -f applier-values.yaml \
   --reset-then-reuse-values
   

2. Trigger an apply (via Apply Now in the UI or stormforge apply in the CLI).

3. Check the Applier logs:

   kubectl logs -n stormforge-system -l component=applier --tail=-1
   

   Look for this log entry:

   {"level":"debug","message":"alerting endpoints of recommendation applied"}
   

   Followed by:

   {"level":"info","message":"workload is healthy after patching"}