Configuring shared volumes for virtual machines

You can configure shared disks to allow multiple virtual machines (VMs) to share the same underlying storage. A shared disk’s volume must be block mode.

You configure disk sharing by exposing the storage as either of these types:

An ordinary VM disk
A logical unit number (LUN) disk with an SCSI connection and raw device mapping, as required for Windows Failover Clustering for shared volumes

In addition to configuring disk sharing, you can also set an error policy for each ordinary VM disk or LUN disk. The error policy controls how the hypervisor behaves when an input/output error occurs on a disk Read or Write.

Configuring disk sharing by using virtual machine disks

You can configure block volumes so that multiple virtual machines (VMs) can share storage.

The application running on the guest operating system determines the storage option you must configure for the VM. A disk of type disk exposes the volume as an ordinary disk to the VM.

You can set an error policy for each disk. The error policy controls how the hypervisor behaves when an input/output error occurs while a disk is being written to or read. The default behavior stops the VM and generates a Kubernetes event.

You can accept the default behavior, or you can set the error policy to one of the following options:

report, which reports the error in the guest.
ignore, which ignores the error. The Read or Write failure is undetected.
enospace, which produces an error indicating that there is not enough disk space.

Prerequisites

The volume access mode must be ReadWriteMany (RWX) if the VMs that are sharing disks are running on different nodes.

If the VMs that are sharing disks are running on the same node, ReadWriteOnce (RWO) volume access mode is sufficient.
The storage provider must support the required Container Storage Interface (CSI) driver.

Procedure

Create the VirtualMachine manifest for your VM to set the required values, as shown in the following example:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: <vm_name>
spec:
  template:
# ...
    spec:
      domain:
        devices:
          disks:
          - disk:
              bus: virtio
            name: rootdisk
            errorPolicy: report 
          - disk:
              bus: virtio
            name: cluster
            shareable: true 
          interfaces:
          - masquerade: {}
            name: default

Identifies the error policy.
Identifies a shared disk.

Save the VirtualMachine manifest file to apply your changes.

To secure data on your VM from outside access, you can enable SCSI persistent reservation and configure a LUN-backed virtual machine disk to be shared among multiple virtual machines.

By enabling the shared option, you can use advanced SCSI commands, such as those required for a Windows failover clustering implementation, for managing the underlying storage.

When a storage volume is configured as the LUN disk type, a VM can use the volume as a logical unit number (LUN) device. As a result, the VM can deploy and manage the disk by using SCSI commands.

You reserve a LUN through the SCSI persistent reserve options. To enable the reservation:

Configure the feature gate option
Activate the feature gate option on the LUN disk to issue SCSI device-specific input and output controls (IOCTLs) that the VM requires.

You can set an error policy for each LUN disk. The error policy controls how the hypervisor behaves when an input/output error occurs on a disk Read or Write. The default behavior stops the guest and generates a Kubernetes event.

For a LUN disk with an SCSi connection and a persistent reservation, as required for Windows Failover Clustering for shared volumes, you set the error policy to report.

Important

OpenShift Virtualization does not currently support SCSI-3 Persistent Reservations (SCSI-3 PR) over multipath storage. As a workaround, disable multipath or ensure the Windows Server Failover Clustering (WSFC) shared disk is setup from a single device and not part of multipath.

Prerequisites

You must have cluster administrator privileges to configure the feature gate option.
The volume access mode must be ReadWriteMany (RWX) if the VMs that are sharing disks are running on different nodes.

If the VMs that are sharing disks are running on the same node, ReadWriteOnce (RWO) volume access mode is sufficient.
The storage provider must support a Container Storage Interface (CSI) driver that uses Fibre Channel (FC), Fibre Channel over Ethernet (FCoE), or iSCSI storage protocols.
If you are a cluster administrator and intend to configure disk sharing by using LUN, you must enable the cluster’s feature gate on the HyperConverged custom resource (CR).
Disks that you want to share must be in block mode.

Procedure

Edit or create the VirtualMachine manifest for your VM to set the required values, as shown in the following example:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: vm-0
spec:
  template:
    spec:
      domain:
        devices:
          disks:
          - disk:
              bus: sata
            name: rootdisk
          - errorPolicy: report 
            lun: 
              bus: scsi
              reservation: true 
            name: na-shared
            serial: shared1234
      volumes:
      - dataVolume:
          name: vm-0
        name: rootdisk
      - name: na-shared
        persistentVolumeClaim:
          claimName: pvc-na-share

Identifies the error policy.
Identifies a LUN disk.
Identifies that the persistent reservation is enabled.

Save the VirtualMachine manifest file to apply your changes.

You can use the OpenShift Container Platform web console to configure disk sharing by using LUN.

Prerequisites

The cluster administrator must enable the persistentreservation feature gate setting.

Procedure

Click Virtualization → VirtualMachines in the web console.
Select a VM to open the VirtualMachine details page.
Expand Storage.
On the Disks tab, click Add disk.
Specify the Name, Source, Size, Interface, and Storage Class.
Select LUN as the Type.
Select Shared access (RWX) as the Access Mode.
Select Block as the Volume Mode.
Expand Advanced Settings, and select both checkboxes.
Click Save.

You can use the command line to configure disk sharing by using LUN.

Procedure

Edit or create the VirtualMachine manifest for your VM to set the required values, as shown in the following example:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: vm-0
spec:
  template:
    spec:
      domain:
        devices:
          disks:
          - disk:
              bus: sata
            name: rootdisk
          - errorPolicy: report
            lun: 
              bus: scsi
              reservation: true 
            name: na-shared
            serial: shared1234
      volumes:
      - dataVolume:
          name: vm-0
        name: rootdisk
      - name: na-shared
        persistentVolumeClaim:
          claimName: pvc-na-share

Identifies a LUN disk.
Identifies that the persistent reservation is enabled.

Save the VirtualMachine manifest file to apply your changes.

Enabling the PersistentReservation feature gate

You can enable the SCSI persistentReservation feature gate and allow a LUN-backed block mode virtual machine (VM) disk to be shared among multiple virtual machines.

The persistentReservation feature gate is disabled by default. You can enable the persistentReservation feature gate by using the web console or the command line.

Prerequisites

Cluster administrator privileges are required.
The volume access mode ReadWriteMany (RWX) is required if the VMs that are sharing disks are running on different nodes. If the VMs that are sharing disks are running on the same node, the ReadWriteOnce (RWO) volume access mode is sufficient.
The storage provider must support a Container Storage Interface (CSI) driver that uses Fibre Channel (FC), Fibre Channel over Ethernet (FCoE), or iSCSI storage protocols.

Enabling the PersistentReservation feature gate by using the web console

You must enable the PersistentReservation feature gate to allow a LUN-backed block mode virtual machine (VM) disk to be shared among multiple virtual machines. Enabling the feature gate requires cluster administrator privileges.

Procedure

Click Virtualization → Overview in the web console.
Click the Settings tab.
Select Cluster.
Expand SCSI persistent reservation and set Enable persistent reservation to on.

Enabling the PersistentReservation feature gate by using the CLI

You can enable the persistentReservation feature gate by using the command line. Enabling the feature gate requires cluster administrator privileges.

Prerequisites

You have installed the OpenShift CLI (oc).

Procedure

Enable the persistentReservation feature gate by running the following command:

$ oc patch hyperconverged kubevirt-hyperconverged -n openshift-cnv --type json -p \
'[{"op":"replace","path":"/spec/featureGates/persistentReservation", "value": true}]'

Additional resources

Configuring shared volumes for virtual machines

Configuring disk sharing by using virtual machine disks

Configuring disk sharing by using LUN

Configuring disk sharing by using LUN and the web console

Configuring disk sharing by using LUN and the CLI

Enabling the PersistentReservation feature gate

Enabling the PersistentReservation feature gate by using the web console

Enabling the PersistentReservation feature gate by using the CLI