Managing jobs and workloads

Red Hat build of Kueue does not directly manipulate jobs that are created by users. Instead, Kueue manages Workload objects that represent the resource requirements of a job. Red Hat build of Kueue automatically creates a workload for each job, and syncs any decisions and statuses between the two objects.

Labeling namespaces to allow Red Hat build of Kueue to manage jobs

The Red Hat build of Kueue Operator uses an opt-in webhook mechanism to ensure that policies are only enforced for the jobs and namespaces that it is expected to target.

You must label the namespaces where you want Red Hat build of Kueue to manage jobs with the kueue.openshift.io/managed=true label.

Prerequisites

You have cluster administrator permissions.
The Red Hat build of Kueue Operator is installed on your cluster, and you have created a Kueue custom resource (CR).
You have installed the OpenShift CLI (oc).

Procedure

Add the kueue.openshift.io/managed=true label to a namespace by running the following command:
```
$ oc label namespace <namespace> kueue.openshift.io/managed=true
```

When you add this label, you instruct the Red Hat build of Kueue Operator that the namespace is managed by its webhook admission controllers. As a result, any Red Hat build of Kueue resources within that namespace are properly validated and mutated.

Configuring label policies for jobs

The spec.config.workloadManagement.labelPolicy spec in the Kueue custom resource (CR) is an optional field that controls how Red Hat build of Kueue decides whether to manage or ignore different jobs. The allowed values are QueueName, None and empty ("").

If the labelPolicy setting is omitted or empty (""), the default policy is that Red Hat build of Kueue manages jobs that have a kueue.x-k8s.io/queue-name label, and ignores jobs that do not have the kueue.x-k8s.io/queue-name label. This is the same workflow as if the labelPolicy is set to QueueName.

If the labelPolicy setting is set to None, jobs are managed by Red Hat build of Kueue even if they do not have the kueue.x-k8s.io/queue-name label.

Example workloadManagement spec configuration

apiVersion: kueue.openshift.io/v1
kind: Kueue
metadata:
  labels:
    app.kubernetes.io/name: kueue-operator
    app.kubernetes.io/managed-by: kustomize
  name: cluster
  namespace: openshift-kueue-operator
spec:
  config:
    workloadManagement:
      labelPolicy: QueueName
# ...

Example user-created Job object containing the kueue.x-k8s.io/queue-name label

apiVersion: batch/v1
kind: Job
metadata:
  generateName: sample-job-
  namespace: my-namespace
  labels:
    kueue.x-k8s.io/queue-name: user-queue
spec:
# ...