Skip to content

Understanding the Custom Resource Definitions

The Compliance Operator in the OpenShift Container Platform provides you with several Custom Resource Definitions (CRDs) to accomplish the compliance scans. To run a compliance scan, it leverages the predefined security policies, which are derived from the ComplianceAsCode community project. The Compliance Operator converts these security policies into CRDs, which you can use to run compliance scans and get remediations for the issues found.

CRDs workflow

The CRD provides you the following workflow to complete the compliance scans:

  1. Define your compliance scan requirements

  2. Configure the compliance scan settings

  3. Process compliance requirements with compliance scans settings

  4. Monitor the compliance scans

  5. Check the compliance scan results

Defining the compliance scan requirements

By default, the Compliance Operator CRDs include ProfileBundle and Profile objects, in which you can define and set the rules for your compliance scan requirements. You can also customize the default profiles by using a TailoredProfile object.

ProfileBundle object

When you install the Compliance Operator, it includes ready-to-run ProfileBundle objects. The Compliance Operator parses the ProfileBundle object and creates a Profile object for each profile in the bundle. It also parses Rule and Variable objects, which are used by the Profile object.

Example ProfileBundle object
apiVersion: compliance.openshift.io/v1alpha1
kind: ProfileBundle
  name: <profile bundle name>
  namespace: openshift-compliance
status:
  dataStreamStatus: VALID
  1. Indicates whether the Compliance Operator was able to parse the content files.

Note

When the contentFile fails, an errorMessage attribute appears, which provides details of the error that occurred.

Troubleshooting

When you roll back to a known content image from an invalid image, the ProfileBundle object stops responding and displays PENDING state. As a workaround, you can move to a different image than the previous one. Alternatively, you can delete and re-create the ProfileBundle object to return to the working state.

Profile object

The Profile object defines the rules and variables that can be evaluated for a certain compliance standard. It contains parsed out details about an OpenSCAP profile, such as its XCCDF identifier and profile checks for a Node or Platform type. You can either directly use the Profile object or further customize it using a TailorProfile object.

Note

You cannot create or modify the Profile object manually because it is derived from a single ProfileBundle object. Typically, a single ProfileBundle object can include several Profile objects.

Example Profile object
apiVersion: compliance.openshift.io/v1alpha1
description: <description of the profile>
id: xccdf_org.ssgproject.content_profile_moderate 
kind: Profile
metadata:
  annotations:
    compliance.openshift.io/product: <product name>
    compliance.openshift.io/product-type: Node 
  creationTimestamp: "YYYY-MM-DDTMM:HH:SSZ"
  generation: 1
  labels:
    compliance.openshift.io/profile-bundle: <profile bundle name>
  name: rhcos4-moderate
  namespace: openshift-compliance
  ownerReferences:
  - apiVersion: compliance.openshift.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ProfileBundle
    name: <profile bundle name>
    uid: <uid string>
  resourceVersion: "<version number>"
  selfLink: /apis/compliance.openshift.io/v1alpha1/namespaces/openshift-compliance/profiles/rhcos4-moderate
  uid: <uid string>
rules: 
- rhcos4-account-disable-post-pw-expiration
- rhcos4-accounts-no-uid-except-zero
- rhcos4-audit-rules-dac-modification-chmod
- rhcos4-audit-rules-dac-modification-chown
title: <title of the profile>
  1. Specify the XCCDF name of the profile. Use this identifier when you define a ComplianceScan object as the value of the profile attribute of the scan.
  2. Specify either a Node or Platform. Node profiles scan the cluster nodes and platform profiles scan the Kubernetes platform.
  3. Specify the list of rules for the profile. Each rule corresponds to a single check.

Rule object

The Rule object, which forms the profiles, are also exposed as objects. Use the Rule object to define your compliance check requirements and specify how it could be fixed.

Example Rule object
    apiVersion: compliance.openshift.io/v1alpha1
    checkType: Platform 
    description: <description of the rule>
    id: xccdf_org.ssgproject.content_rule_configure_network_policies_namespaces 
    instructions: <manual instructions for the scan>
    kind: Rule
    metadata:
      annotations:
        compliance.openshift.io/rule: configure-network-policies-namespaces
        control.compliance.openshift.io/CIS-OCP: 5.3.2
        control.compliance.openshift.io/NERC-CIP: CIP-003-3 R4;CIP-003-3 R4.2;CIP-003-3
          R5;CIP-003-3 R6;CIP-004-3 R2.2.4;CIP-004-3 R3;CIP-007-3 R2;CIP-007-3 R2.1;CIP-007-3
          R2.2;CIP-007-3 R2.3;CIP-007-3 R5.1;CIP-007-3 R6.1
        control.compliance.openshift.io/NIST-800-53: AC-4;AC-4(21);CA-3(5);CM-6;CM-6(1);CM-7;CM-7(1);SC-7;SC-7(3);SC-7(5);SC-7(8);SC-7(12);SC-7(13);SC-7(18)
      labels:
        compliance.openshift.io/profile-bundle: ocp4
      name: ocp4-configure-network-policies-namespaces
      namespace: openshift-compliance
    rationale: <description of why this rule is checked>
    severity: high 
    title: <summary of the rule>
  1. Specify the type of check this rule executes. Node profiles scan the cluster nodes and Platform profiles scan the Kubernetes platform. An empty value indicates there is no automated check.
  2. Specify the XCCDF name of the rule, which is parsed directly from the datastream.
  3. Specify the severity of the rule when it fails.

Note

The Rule object gets an appropriate label for an easy identification of the associated ProfileBundle object. The ProfileBundle also gets specified in the OwnerReferences of this object.

TailoredProfile object

Use the TailoredProfile object to modify the default Profile object based on your organization requirements. You can enable or disable rules, set variable values, and provide justification for the customization. After validation, the TailoredProfile object creates a ConfigMap, which can be referenced by a ComplianceScan object.

Tip

You can use the TailoredProfile object by referencing it in a ScanSettingBinding object. For more information about ScanSettingBinding, see ScanSettingBinding object.

Example TailoredProfile object
apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: rhcos4-with-usb
spec:
  extends: rhcos4-moderate 
  title: <title of the tailored profile>
  disableRules:
    - name: <name of a rule object to be disabled>
      rationale: <description of why this rule is checked>
status:
  id: xccdf_compliance.openshift.io_profile_rhcos4-with-usb 
  outputRef:
    name: rhcos4-with-usb-tp 
    namespace: openshift-compliance
  state: READY
  1. This is optional. Name of the Profile object upon which the TailoredProfile is built. If no value is set, a new profile is created from the enableRules list.
  2. Specifies the XCCDF name of the tailored profile.
  3. Specifies the ConfigMap name, which can be used as the value of the tailoringConfigMap.name attribute of a ComplianceScan.
  4. Shows the state of the object such as READY, PENDING, and FAILURE. If the state of the object is ERROR, then the attribute status.errorMessage provides the reason for the failure.

With the TailoredProfile object, it is possible to create a new Profile object using the TailoredProfile construct. To create a new Profile, set the following configuration parameters :

  • an appropriate title

  • extends value must be empty

  • scan type annotation on the TailoredProfile object:

    compliance.openshift.io/product-type: Platform/Node

    Note

    If you have not set the product-type annotation, the Compliance Operator defaults to Platform scan type. Adding the -node suffix to the name of the TailoredProfile object results in node scan type.

Configuring the compliance scan settings

After you have defined the requirements of the compliance scan, you can configure it by specifying the type of the scan, occurrence of the scan, and location of the scan. To do so, Compliance Operator provides you with a ScanSetting object.

ScanSetting object

Use the ScanSetting object to define and reuse the operational policies to run your scans. By default, the Compliance Operator creates the following ScanSetting objects:

  • default - it runs a scan every day at 1 AM on both master and worker nodes using a 1Gi Persistent Volume (PV) and keeps the last three results. Remediation is neither applied nor updated automatically.

  • default-auto-apply - it runs a scan every day at 1AM on both control plane and worker nodes using a 1Gi Persistent Volume (PV) and keeps the last three results. Both autoApplyRemediations and autoUpdateRemediations are set to true.

Example ScanSetting object
apiVersion: compliance.openshift.io/v1alpha1
autoApplyRemediations: true 
autoUpdateRemediations: true 
kind: ScanSetting
maxRetryOnTimeout: 3
metadata:
  creationTimestamp: "2022-10-18T20:21:00Z"
  generation: 1
  name: default-auto-apply
  namespace: openshift-compliance
  resourceVersion: "38840"
  uid: 8cb0967d-05e0-4d7a-ac1c-08a7f7e89e84
rawResultStorage:
  nodeSelector:
    node-role.kubernetes.io/master: ""
  pvAccessModes:
  - ReadWriteOnce
  rotation: 3 
  size: 1Gi 
  tolerations:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
    operator: Exists
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  - effect: NoSchedule
    key: node.kubernetes.io/memory-pressure
    operator: Exists
roles: 
- master
- worker
scanTolerations:
- operator: Exists
schedule: 0 1 * * * 
showNotApplicable: false
strictNodeScan: true
timeout: 30m
  1. Set to true to enable auto remediations. Set to false to disable auto remediations.
  2. Set to true to enable auto remediations for content updates. Set to false to disable auto remediations for content updates.
  3. Specify the number of stored scans in the raw result format. The default value is 3. As the older results get rotated, the administrator must store the results elsewhere before the rotation happens.
  4. Specify the storage size that should be created for the scan to store the raw results. The default value is 1Gi
  5. Specify how often the scan should be run in cron format.

    Note

    To disable the rotation policy, set the value to 0.

  6. Specify the node-role.kubernetes.io label value to schedule the scan for Node type. This value has to match the name of a MachineConfigPool.

Processing the compliance scan requirements with compliance scans settings

When you have defined the compliance scan requirements and configured the settings to run the scans, then the Compliance Operator processes it using the ScanSettingBinding object.

ScanSettingBinding object

Use the ScanSettingBinding object to specify your compliance requirements with reference to the Profile or TailoredProfile object. It is then linked to a ScanSetting object, which provides the operational constraints for the scan. Then the Compliance Operator generates the ComplianceSuite object based on the ScanSetting and ScanSettingBinding objects.

Example ScanSettingBinding object
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: <name of the scan>
profiles: 
  # Node checks
  - name: rhcos4-with-usb
    kind: TailoredProfile
    apiGroup: compliance.openshift.io/v1alpha1
  # Cluster checks
  - name: ocp4-moderate
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef: 
  name: my-companys-constraints
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
  1. Specify the details of Profile or TailoredProfile object to scan your environment.
  2. Specify the operational constraints, such as schedule and storage size.

The creation of ScanSetting and ScanSettingBinding objects results in the compliance suite. To get the list of compliance suite, run the following command:

$ oc get compliancesuites

Important

If you delete ScanSettingBinding, then compliance suite also is deleted.

Tracking the compliance scans

After the creation of compliance suite, you can monitor the status of the deployed scans using the ComplianceSuite object.

ComplianceSuite object

The ComplianceSuite object helps you keep track of the state of the scans. It contains the raw settings to create scans and the overall result.

For Node type scans, you should map the scan to the MachineConfigPool, since it contains the remediations for any issues. If you specify a label, ensure it directly applies to a pool.

Example ComplianceSuite object
apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceSuite
metadata:
  name: <name_of_the_suite>
spec:
  autoApplyRemediations: false 
  schedule: "0 1 * * *" 
  scans: 
    - name: workers-scan
      scanType: Node
      profile: xccdf_org.ssgproject.content_profile_moderate
      content: ssg-rhcos4-ds.xml
      contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc...
      rule: "xccdf_org.ssgproject.content_rule_no_netrc_files"
      nodeSelector:
        node-role.kubernetes.io/worker: ""
status:
  Phase: DONE 
  Result: NON-COMPLIANT 
  scanStatuses:
  - name: workers-scan
    phase: DONE
    result: NON-COMPLIANT
  1. Set to true to enable auto remediations. Set to false to disable auto remediations.
  2. Specify how often the scan should be run in cron format.
  3. Specify a list of scan specifications to run in the cluster.
  4. Indicates the progress of the scans.
  5. Indicates the overall verdict of the suite.

The suite in the background creates the ComplianceScan object based on the scans parameter. You can programmatically fetch the ComplianceSuites events. To get the events for the suite, run the following command:

$ oc get events --field-selector involvedObject.kind=ComplianceSuite,involvedObject.name=<name of the suite>

Important

You might create errors when you manually define the ComplianceSuite, since it contains the XCCDF attributes.

Advanced ComplianceScan Object

The Compliance Operator includes options for advanced users for debugging or integrating with existing tooling. While it is recommended that you not create a ComplianceScan object directly, you can instead manage it using a ComplianceSuite object.

Example Advanced ComplianceScan object
apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceScan
metadata:
  name: <name_of_the_compliance_scan>
spec:
  scanType: Node 
  profile: xccdf_org.ssgproject.content_profile_moderate 
  content: ssg-ocp4-ds.xml
  contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc... 
  rule: "xccdf_org.ssgproject.content_rule_no_netrc_files" 
  nodeSelector: 
    node-role.kubernetes.io/worker: ""
status:
  phase: DONE 
  result: NON-COMPLIANT
  1. Specify either Node or Platform. Node profiles scan the cluster nodes and platform profiles scan the Kubernetes platform.
  2. Specify the XCCDF identifier of the profile that you want to run.
  3. Specify the container image that encapsulates the profile files.
  4. It is optional. Specify the scan to run a single rule. This rule has to be identified with the XCCDF ID, and has to belong to the specified profile.

    Note

    If you skip the rule parameter, then scan runs for all the available rules of the specified profile.

  5. If you are on the OpenShift Container Platform and wants to generate a remediation, then nodeSelector label has to match the MachineConfigPool label.

    Note

    If you do not specify nodeSelector parameter or match the MachineConfig label, scan will still run, but it will not create remediation.

  6. Indicates the current phase of the scan.
  7. Indicates the verdict of the scan.

Important

If you delete a ComplianceSuite object, then all the associated scans get deleted.

When the scan is complete, it generates the result as Custom Resources of the ComplianceCheckResult object. However, the raw results are available in ARF format. These results are stored in a Persistent Volume (PV), which has a Persistent Volume Claim (PVC) associated with the name of the scan. You can programmatically fetch the ComplianceScans events. To generate events for the suite, run the following command:

oc get events --field-selector involvedObject.kind=ComplianceScan,involvedObject.name=<name_of_the_compliance_scan>

Viewing the compliance results

When the compliance suite reaches the DONE phase, you can view the scan results and possible remediations.

ComplianceCheckResult object

When you run a scan with a specific profile, several rules in the profiles are verified. For each of these rules, a ComplianceCheckResult object is created, which provides the state of the cluster for a specific rule.

Example ComplianceCheckResult object
apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceCheckResult
metadata:
  labels:
    compliance.openshift.io/check-severity: medium
    compliance.openshift.io/check-status: FAIL
    compliance.openshift.io/suite: example-compliancesuite
    compliance.openshift.io/scan-name: workers-scan
  name: workers-scan-no-direct-root-logins
  namespace: openshift-compliance
  ownerReferences:
  - apiVersion: compliance.openshift.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ComplianceScan
    name: workers-scan
description: <description of scan check>
instructions: <manual instructions for the scan>
id: xccdf_org.ssgproject.content_rule_no_direct_root_logins
severity: medium 
status: FAIL
  1. Describes the severity of the scan check.
  2. Describes the result of the check. The possible values are:

    • PASS: check was successful.

    • FAIL: check was unsuccessful.

    • INFO: check was successful and found something not severe enough to be considered an error.

    • MANUAL: check cannot automatically assess the status and manual check is required.

    • INCONSISTENT: different nodes report different results.

    • ERROR: check run successfully, but could not complete.

    • NOTAPPLICABLE: check did not run as it is not applicable.

To get all the check results from a suite, run the following command:

oc get compliancecheckresults \
-l compliance.openshift.io/suite=workers-compliancesuite

ComplianceRemediation object

For a specific check you can have a datastream specified fix. However, if a Kubernetes fix is available, then the Compliance Operator creates a ComplianceRemediation object.

Example ComplianceRemediation object
apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceRemediation
metadata:
  labels:
    compliance.openshift.io/suite: example-compliancesuite
    compliance.openshift.io/scan-name: workers-scan
    machineconfiguration.openshift.io/role: worker
  name: workers-scan-disable-users-coredumps
  namespace: openshift-compliance
  ownerReferences:
  - apiVersion: compliance.openshift.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ComplianceCheckResult
    name: workers-scan-disable-users-coredumps
    uid: <UID>
spec:
  apply: false 
  object:
    current: 
       apiVersion: machineconfiguration.openshift.io/v1
       kind: MachineConfig
       spec:
         config:
           ignition:
             version: 2.2.0
           storage:
             files:
             - contents:
                 source: data:,%2A%20%20%20%20%20hard%20%20%20core%20%20%20%200
               filesystem: root
               mode: 420
               path: /etc/security/limits.d/75-disable_users_coredumps.conf
    outdated: {}
  1. true indicates the remediation was applied. false indicates the remediation was not applied.
  2. Includes the definition of the remediation.
  3. Indicates remediation that was previously parsed from an earlier version of the content. The Compliance Operator still retains the outdated objects to give the administrator a chance to review the new remediations before applying them.

To get all the remediations from a suite, run the following command:

oc get complianceremediations \
-l compliance.openshift.io/suite=workers-compliancesuite

To list all failing checks that can be remediated automatically, run the following command:

oc get compliancecheckresults \
-l 'compliance.openshift.io/check-status in (FAIL),compliance.openshift.io/automated-remediation'

To list all failing checks that can be remediated manually, run the following command:

oc get compliancecheckresults \
-l 'compliance.openshift.io/check-status in (FAIL),!compliance.openshift.io/automated-remediation'