# Optimize job-type workloads

> Configuration and recommendation behavior for CronJobs, Jobs, and Spark applications

---

Site [llms.txt](/llms.txt)

---

If you run CronJobs, Jobs, or Spark applications, StormForge doesn't recognize them as workloads until you declare them in `workloadResourceKinds`. The Agent doesn't watch these kinds by default, because job-type workloads have requirements beyond those of long-running workloads. You can declare four kinds: Kubernetes CronJobs and Jobs, and the Spark operator's ScheduledSparkApplications and SparkApplications.

StormForge generates recommendations for all four kinds, but the Applier applies them only to the scheduled kinds: CronJobs and ScheduledSparkApplications. For the one-off kinds, Jobs and SparkApplications, you apply the recommended values yourself.

## Declare the kinds

The Agent can only see the kinds you declare. For the full field reference, see [Custom workload kinds][custom-workload-kinds].

### CronJob

```yaml
workloadResourceKinds:
- apiGroup: batch
  kind: CronJob
  resource: cronjobs
```

Also declare `Job`, as described under [Job](#job). The Agent reaches a CronJob through the Jobs it creates, so without a `Job` entry it can't find the CronJob.

### ScheduledSparkApplication

```yaml
workloadResourceKinds:
- apiGroup: sparkoperator.k8s.io
  kind: ScheduledSparkApplication
  resource: scheduledsparkapplications
```

As with CronJobs, also declare the kind it creates: `SparkApplication`, as described under [SparkApplication](#sparkapplication).

### Job

```yaml
workloadResourceKinds:
- apiGroup: batch
  kind: Job
  resource: jobs
  role: conditionalWorkload
```

`conditionalWorkload` makes a Job act as a workload when it stands alone, and defer to its owner when a CronJob created it.

<div class="alert alert-warning" role="alert"><div class="h4 alert-heading" role="heading">Always set conditionalWorkload on Job</div>


Declared as a plain workload, each per-execution Job becomes a workload in its own right and takes ownership of the Pods its CronJob created. The CronJob is then left with no Pods and no usage, and its recommendations fail with `ShortBatchJobs` even when the runs meet the [run requirements](#run-requirements).
</div>


To have recommendations applied to Jobs, run them from a CronJob, or declare the longer-lived object that creates them, as described under [Apply the pattern to other job-like kinds](#apply-the-pattern-to-other-job-like-kinds). For standalone Jobs, see [Group standalone runs](#group-standalone-runs).

### SparkApplication

```yaml
workloadResourceKinds:
- apiGroup: sparkoperator.k8s.io
  kind: SparkApplication
  resource: sparkapplications
  role: conditionalWorkload
```

The same pattern as `Job`: a workload when submitted directly, deferring to its owner when a `ScheduledSparkApplication` created it. To have recommendations applied to Spark runs, schedule them through a `ScheduledSparkApplication`. For SparkApplications submitted directly, see [Group standalone runs](#group-standalone-runs).

## Optimize scheduled jobs

A CronJob keeps one name, and each run is created from its job template, so its usage accumulates across runs and it stays available as a patch target between them. Keep the default [apply method][apply-method]: the patch lands on the job template and takes effect on the next run. StormForge knows where a CronJob holds its Pod template, so no `patchTargetDefaults` is needed.

A ScheduledSparkApplication behaves the same way: it keeps a stable name, is patched between runs, and uses the default apply method.

StormForge optimizes only the executor (the `spark-kubernetes-executor` container) and leaves the driver and any sidecars unchanged. It sets the executor's `coreRequest`, `coreLimit`, and `memory`, but not a memory limit, because the SparkApplication executor spec has no memory limit field.

### Enable auto-deploy

Enable auto-deploy to have the Applier apply recommendations to CronJobs and ScheduledSparkApplications automatically:

```yaml
live.stormforge.io/auto-deploy: "Enabled"
```

Set it at the cluster, namespace, or workload level. Automatic deployment begins once the workload's [learning period][learning-period] is complete. For the full setting reference, see [Auto-deploy][auto-deploy].

### Run requirements

StormForge treats only CronJobs and ScheduledSparkApplications as recurring batch jobs. These kinds produce a recommendation only when each run is observed for long enough, and when successive runs accumulate into a single history.

Runs shorter than 10 minutes often don't produce enough usage data to work from. There is no exact cutoff, and a single run of 10 minutes may not be enough on its own, because usage accumulates across runs.

When runs aren't observed for long enough, the recommendation fails to generate with the reason `ShortBatchJobs`, and the workload is reported as Unsupported.

The time between runs isn't held against these kinds. Usage is collected from the Pods a workload runs, so periods when it has no Pods contribute nothing either way. A job that runs for fifteen minutes once an hour is measured on those fifteen minutes. What has to add up is observed running time, not elapsed wall-clock time.

Temporary memory bump-ups after an OOM event aren't available for CronJobs or ScheduledSparkApplications, as described in [Reliability][reliability].

## Group standalone runs

StormForge can't apply recommendations to standalone Jobs or SparkApplications. Instead, group their runs and then apply the values yourself.

Each standalone `Job` or `SparkApplication` runs its Pods exactly once. A recommendation can only be produced after StormForge has observed that run, by which point the Pods have already started and the object that created them will never create any more. A later run is a separate workload under a new name.

A Job is immutable once created, so StormForge resolves no patch path for `batch/v1` Jobs and marks the recommendation unpatchable.

### How standalone runs are measured

The guidance under [Run requirements](#run-requirements) applies only to CronJobs and ScheduledSparkApplications. StormForge treats standalone Jobs and SparkApplications the same way as every other workload kind, such as Deployments, so they never show `ShortBatchJobs`. Each run still needs to last long enough for StormForge to collect usage data.

### Size later runs

Without a workload group, a recommendation reflects a single run: each run creates a workload with a new name and no history, so nothing accumulates. A group pools the Pods from every matching run, so the recommendation reflects usage across many runs rather than one.

1. Create a workload group that matches the runs. Groups can match on labels, on name patterns, or on explicit workload identity. [Workload groups][workload-groups] covers the full set of options, and [Ephemeral workloads][ephemeral-workloads] explains how to configure runs that get a new name each time.
2. After a run completes, check the group's recommendation.
3. Copy the recommended requests and limits into the manifest, Helm chart, or pipeline that defines the Jobs' Pod template.

## Troubleshoot job workloads

To see how StormForge has resolved a workload, and which workloads its group contains:

```sh
stormforge describe workload <workload>
```

| Symptom | Cause | Action |
|---|---|---|
| Permission errors in the Agent log naming a job kind | A kind in the Pod's ownership chain isn't declared. The Agent finds a workload by walking up a Pod's `ownerReferences` chain, but it holds Kubernetes RBAC only for the kinds you declare, so an undeclared kind stops the walk. | Declare the kind named in the error. |
| The workload shows Unsupported | The run is too short to measure. | See [Run requirements](#run-requirements). |
| The workload shows Learning (Waiting for Data) | It hasn't run since StormForge started observing it. | None. This resolves on its own once a run completes. See [Workload statuses][statuses]. |
| A standalone Job shows Patch Configuration warnings about a missing `patchPath` | Each run creates its Pods once, so by the time a recommendation exists there's nothing left to patch. | None. This is expected. See [Group standalone runs](#group-standalone-runs). |
| A CronJob reports `ShortBatchJobs` even though its runs are long enough | The recommendation was generated before a long enough run had completed, or the CronJob's Pods are being attributed to the per-execution Jobs instead. | Wait for a run to complete, then check again. If `ShortBatchJobs` remains, check that your `Job` entry sets `role: conditionalWorkload`. |
| Per-execution Job workloads remain after you correct the `Job` entry | Workloads created before the change stay until they go inactive and age out. Setting `role: conditionalWorkload` stops new ones appearing. | None. |

## Apply the pattern to other job-like kinds

The same scheduled vs. one-off split applies to any workload that runs to completion. Ask whether the kind owns Pods directly, or is templated by a longer-lived object that outlives each run:

- **A scheduler or template object** keeps a stable identity and accumulates usage. Declare it `workload` and keep the default apply method.
- **A per-run object** exists only while the run does, and creates its Pods once. Declare it `conditionalWorkload` and group its runs. Expect these to be observable only, for the reasons described under [Group standalone runs](#group-standalone-runs).

The measurement rules don't carry over. StormForge applies the recurring batch job rules to CronJob and ScheduledSparkApplication only, so any other kind is measured like a long-running workload whichever role you give it. See [Run requirements](#run-requirements).



[apply-method]:          /docs/configure/settings/apply-settings/#apply-method
[auto-deploy]:           /docs/configure/settings/apply-settings/#auto-deploy
[custom-workload-kinds]: /docs/recommendations/custom-workload-kinds/
[ephemeral-workloads]:   /docs/recommendations/workload-groups/#ephemeral-workloads
[learning-period]:       /docs/configure/settings/apply-settings/#learning-period
[reliability]:           /docs/configure/settings/reliability/
[statuses]:              /docs/reference/statuses/
[workload-groups]:       /docs/recommendations/workload-groups/