Skip to content

Release notes

Red Hat build of Kueue is released as an Operator that is supported on OpenShift Container Platform.

Compatible environments

Before you install Red Hat build of Kueue, review this section to ensure that your cluster meets the requirements.

Supported architectures

Red Hat build of Kueue version 1.1 and later is supported on the following architectures:

  • ARM64

  • 64-bit x86

  • ppc64le (IBM Power®)

  • s390x (IBM Z®)

Supported platforms

Red Hat build of Kueue version 1.1 and later is supported on the following platforms:

  • OpenShift Container Platform

  • Hosted control planes for OpenShift Container Platform

Important

Currently, Red Hat build of Kueue is not supported on Red Hat build of MicroShift (MicroShift).

Release notes for Red Hat build of Kueue version 1.2

Red Hat build of Kueue version 1.2 is a generally available release that is supported on OpenShift Container Platform versions 4.18 and later. Red Hat build of Kueue version 1.2 uses Kueue version 0.14.

New features and enhancements

Monitoring of pending workloads

Red Hat build of Kueue version 1.2 provides the VisibilityOnDemand feature to monitor the pipeline of pending jobs in the cluster queue and the local queue, and help users to estimate when their jobs will start. For more information, see Monitoring pending workloads.

Fixed issues

Custom resources are not deleted properly when you uninstall Red Hat build of Kueue

After you uninstall the Red Hat Build of Kueue Operator using the Delete all operand instances for this operator option in the OpenShift Container Platform web console, Red Hat build of Kueue custom resources were attempted to be deleted. With this release, they are not considered for deletion.

(OCPBUGS-62254)

Documentation error in previous versions of Red Hat build of Kueue

In Creating a Kueue custom resource, the optional workload types Pod, Deployment, StatefulSet were omitted. They are now included.

(OCPBUGS-62877)

Red Hat build of Kueue metrics were not being exposed to Prometheus from version 1.1

Prometheus was not scraping metrics from the Operator’s controller, even though the ServiceMonitor and RBAC resources were successfully created as part of the Operator installation. As a result, none of the Kueue metrics were available in the cluster monitoring stack.

The metrics service added during the installation was configured with an incorrect port reference, causing Prometheus to fail in scraping metrics from the Kueue endpoint. The port name has been updated with the correct port name.

(OCPBUGS-63441)

Known issues

Reconcile jobs only in opt-in namespaces

OpenShift Container Platform allows reconciliation of Job resources that have the kueue.x-k8s.io/queue-name label, even if these resources are in namespaces which are not configured to opt in to being managed by OpenShift Container Platform. This is inconsistent with the behavior for other core integrations like pods, deployments, and stateful sets, which are only reconciled if they are in namespaces which have been configured to opt in to being managed by OpenShift Container Platform by adding the kueue.openshift.io/managed=true.

(OCPBUGS-58205)

Kueue CR description reads as "Not available" in the OpenShift Container Platform web console

After installing Red Hat build of Kueue, in the Operator details view, the description for the Kueue CR reads as "Not available". This issue does not affect or degrade the Red Hat build of Kueue Operator functionality.

(OCPBUGS-62185)

Release notes for Red Hat build of Kueue version 1.1

Red Hat build of Kueue version 1.1 is a generally available release that is supported on OpenShift Container Platform versions 4.18 and later. Red Hat build of Kueue version 1.1 uses Kueue version 0.12.

Important

If you have a previously installed version of Red Hat build of Kueue on your cluster, you must uninstall the Operator and manually install version 1.1. For information see Upgrading Red Hat build of Kueue.

New features and enhancements

Configure a default local queue

A default local queue serves as the local queue for newly created jobs that do not have the kueue.x-k8s.io/queue-name label. After you create a default local queue, any new jobs created in the namespace without a kueue.x-k8s.io/queue-name label automatically update to have the kueue.x-k8s.io/queue-name: default label.

(RFE-7615)

Multi-architecture and Hosted control planes support

With this release, Red Hat build of Kueue is supported on multiple different architectures, including ARM64, 64-bit x86, ppc64le (IBM Power®), and s390x (IBM Z®), as well as on Hosted control planes for OpenShift Container Platform.

(OCPSTRAT-2103)

(OCPSTRAT-2106)

Fixed issues

You can create a Kueue custom resource by using the OpenShift Container Platform web console

Before this update, if you tried to use the OpenShift Container Platform web console to create a Kueue custom resource (CR) by using the form view, the web console showed an error and the resource could not be created. With this release, the default namespace was removed from the Kueue CR template. As a result, you can use the OpenShift Container Platform web console to create a Kueue CR by using the form view.

(OCPBUGS-58118)

Known issues

Kueue CR description reads as "Not available" in the OpenShift Container Platform web console

After you install Red Hat build of Kueue, in the Operator details view, the description for the Kueue CR reads as "Not available". This issue does not affect or degrade the Red Hat build of Kueue Operator functionality.

(OCPBUGS-62185)

Custom resources are not deleted properly when you uninstall Red Hat build of Kueue

After you uninstall the Red Hat Build of Kueue Operator using the Delete all operand instances for this operator option in the OpenShift Container Platform web console, some Red Hat build of Kueue custom resources are not fully deleted. These resources can be viewed in the Installed Operators view with the status Resource is being deleted. As a workaround, you can manually delete the resource finalizers to remove them fully.

(OCPBUGS-62254)

Release notes for Red Hat build of Kueue version 1.0.1

Red Hat build of Kueue version 1.0.1 is a patch release that is supported on OpenShift Container Platform versions 4.18 and 4.19 on the 64-bit x86 architecture.

Red Hat build of Kueue version 1.0.1 uses Kueue version 0.11.

Bug fixes in Red Hat build of Kueue version 1.0.1

  • Previously, leader election for Red Hat build of Kueue was not configured to tolerate disruption, which resulted in frequent crashing. With this release, the leader election values for Red Hat build of Kueue have been updated to match the durations recommended for OpenShift Container Platform. (OCPBUGS-58496)

  • Previously, the ReadyReplicas count was not set in the reconciler, which meant that the Red Hat build of Kueue Operator status would report that there were no replicas ready. With this release, the ReadyReplicas count is based on the number of ready replicas for the deployment, which ensures that the Operator shows as ready in the OpenShift Container Platform console when the kueue-controller-manager pods are ready. (OCPBUGS-59261)

  • Previously, when the Kueue custom resource (CR) was deleted from the openshift-kueue-operator namespace, the kueue-manager-config config map was not deleted automatically and could remain in the namespace. With this release, the kueue-manager-config config map, kueue-webhook-server-cert secret, and metrics-server-cert secret are deleted automatically when the Kueue CR is deleted. (OCPBUGS-57960)

Release notes for Red Hat build of Kueue version 1.0

Red Hat build of Kueue version 1.0 is a generally available release that is supported on OpenShift Container Platform versions 4.18 and 4.19 on the 64-bit x86 architecture. Red Hat build of Kueue version 1.0 uses Kueue version 0.11.

New features and enhancements

Role-based access control (RBAC)

Role-based access control (RBAC) enables you to control which types of users can create which types of Red Hat build of Kueue resources.

Configure resource quotas

Configuring resource quotas by creating cluster queues, resource flavors, and local queues enables you to control the amount of resources used by user-submitted jobs and workloads.

Control job and workload management

Labeling namespaces and configuring label policies enable you to control which jobs and workloads are managed by Red Hat build of Kueue.

Share borrowable resources between queues

Configuring cohorts, fair sharing, and gang scheduling settings enable you to share unused, borrowable resources between queues.

Known issues

Jobs in all namespaces are reconciled if they have the kueue.x-k8s.io/queue-name label

Red Hat build of Kueue uses the managedJobsNamespaceSelector configuration field, so that administrators can configure which namespaces opt in to be managed by Red Hat build of Kueue. Because namespaces must be manually configured to opt in to being managed by Red Hat build of Kueue, resources in system or third-party namespaces are not impacted or managed by Red Hat build of Kueue.

The behavior in Red Hat build of Kueue 1.0 allows reconciliation of Job resources that have the kueue.x-k8s.io/queue-name label, even if these resources are in namespaces that are not configured to opt in to being managed by Red Hat build of Kueue. This is inconsistent with the behavior for other core integrations like pods, deployments, and stateful sets, which are only reconciled if they are in namespaces that have been configured to opt in to being managed by Red Hat build of Kueue.

(OCPBUGS-58205)

You cannot create a Kueue custom resource by using the OpenShift Container Platform web console

If you try to use the OpenShift Container Platform web console to create a Kueue custom resource (CR) by using the form view, the web console shows an error and the resource cannot be created. As a workaround, use the YAML view to create a Kueue CR instead.

(OCPBUGS-58118)