- Performing advanced Compliance Operator tasks
- Using the ComplianceSuite and ComplianceScan objects directly
- Setting
PriorityClass
forScanSetting
scans - Using raw tailored profiles
- Performing a rescan
- Setting custom storage size for results
- Applying remediations generated by suite scans
- Automatically update remediations
- Creating a custom SCC for the Compliance Operator
- Additional resources
Performing advanced Compliance Operator tasks
The Compliance Operator includes options for advanced users for the purpose of debugging or integration with existing tooling.
Using the ComplianceSuite and ComplianceScan objects directly
While it is recommended that users take advantage of the ScanSetting
and ScanSettingBinding
objects to define the suites and scans, there are valid use cases to define the ComplianceSuite
objects directly:
Specifying only a single rule to scan. This can be useful for debugging together with the
debug: true
attribute which increases the OpenSCAP scanner verbosity, as the debug mode tends to get quite verbose otherwise. Limiting the test to one rule helps to lower the amount of debug information.Providing a custom nodeSelector. In order for a remediation to be applicable, the nodeSelector must match a pool.
Pointing the Scan to a bespoke config map with a tailoring file.
For testing or development when the overhead of parsing profiles from bundles is not required.
The following example shows a ComplianceSuite
that scans the worker machines with only a single rule:
apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceSuite
metadata:
name: workers-compliancesuite
spec:
scans:
- name: workers-scan
profile: xccdf_org.ssgproject.content_profile_moderate
content: ssg-rhcos4-ds.xml
contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc...
debug: true
rule: xccdf_org.ssgproject.content_rule_no_direct_root_logins
nodeSelector:
node-role.kubernetes.io/worker: ""
The ComplianceSuite
object and the ComplianceScan
objects referred to above specify several attributes in a format that OpenSCAP expects.
To find out the profile, content, or rule values, you can start by creating a similar Suite from ScanSetting
and ScanSettingBinding
or inspect the objects parsed from the ProfileBundle
objects like rules or profiles. Those objects contain the xccdf_org
identifiers you can use to refer to them from a ComplianceSuite
.
Setting PriorityClass
for ScanSetting
scans
In large scale environments, the default PriorityClass
object can be too low to guarantee Pods execute scans on time. For clusters that must maintain compliance or guarantee automated scanning, it is recommended to set the PriorityClass
variable to ensure the Compliance Operator is always given priority in resource constrained situations.
Procedure
Set the
PriorityClass
variable:apiVersion: compliance.openshift.io/v1alpha1
strictNodeScan: true
metadata:
name: default
namespace: openshift-compliance
priorityClass: compliance-high-priority (1)
kind: ScanSetting
showNotApplicable: false
rawResultStorage:
nodeSelector:
node-role.kubernetes.io/master: ''
pvAccessModes:
- ReadWriteOnce
rotation: 3
size: 1Gi
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
- effect: NoSchedule
key: node.kubernetes.io/memory-pressure
operator: Exists
schedule: 0 1 * * *
roles:
- master
- worker
scanTolerations:
- operator: Exists
1 If the PriorityClass
referenced in theScanSetting
cannot be found, the Operator will leave thePriorityClass
empty, issue a warning, and continue scheduling scans without aPriorityClass
.
Using raw tailored profiles
While the TailoredProfile
CR enables the most common tailoring operations, the XCCDF standard allows even more flexibility in tailoring OpenSCAP profiles. In addition, if your organization has been using OpenScap previously, you may have an existing XCCDF tailoring file and can reuse it.
The ComplianceSuite
object contains an optional TailoringConfigMap
attribute that you can point to a custom tailoring file. The value of the TailoringConfigMap
attribute is a name of a config map which must contain a key called tailoring.xml
and the value of this key is the tailoring contents.
Procedure
Create the
ConfigMap
object from a file:$ oc -n openshift-compliance \
create configmap nist-moderate-modified \
--from-file=tailoring.xml=/path/to/the/tailoringFile.xml
Reference the tailoring file in a scan that belongs to a suite:
apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceSuite
metadata:
name: workers-compliancesuite
spec:
debug: true
scans:
- name: workers-scan
profile: xccdf_org.ssgproject.content_profile_moderate
content: ssg-rhcos4-ds.xml
contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:45dc...
debug: true
tailoringConfigMap:
name: nist-moderate-modified
nodeSelector:
node-role.kubernetes.io/worker: ""
Performing a rescan
Typically you will want to re-run a scan on a defined schedule, like every Monday or daily. It can also be useful to re-run a scan once after fixing a problem on a node. To perform a single scan, annotate the scan with the compliance.openshift.io/rescan=
option:
$ oc -n openshift-compliance \
annotate compliancescans/rhcos4-e8-worker compliance.openshift.io/rescan=
A rescan generates four additional mc
for rhcos-moderate
profile:
$ oc get mc
Example output
75-worker-scan-chronyd-or-ntpd-specify-remote-server
75-worker-scan-configure-usbguard-auditbackend
75-worker-scan-service-usbguard-enabled
75-worker-scan-usbguard-allow-hid-and-hub
When the scan setting |
Setting custom storage size for results
While the custom resources such as ComplianceCheckResult
represent an aggregated result of one check across all scanned nodes, it can be useful to review the raw results as produced by the scanner. The raw results are produced in the ARF format and can be large (tens of megabytes per node), it is impractical to store them in a Kubernetes resource backed by the etcd
key-value store. Instead, every scan creates a persistent volume (PV) which defaults to 1GB size. Depending on your environment, you may want to increase the PV size accordingly. This is done using the rawResultStorage.size
attribute that is exposed in both the ScanSetting
and ComplianceScan
resources.
A related parameter is rawResultStorage.rotation
which controls how many scans are retained in the PV before the older scans are rotated. The default value is 3, setting the rotation policy to 0 disables the rotation. Given the default rotation policy and an estimate of 100MB per a raw ARF scan report, you can calculate the right PV size for your environment.
Using custom result storage values
Because OKD can be deployed in a variety of public clouds or bare metal, the Compliance Operator cannot determine available storage configurations. By default, the Compliance Operator will try to create the PV for storing results using the default storage class of the cluster, but a custom storage class can be configured using the rawResultStorage.StorageClassName
attribute.
If your cluster does not specify a default storage class, this attribute must be set. |
Configure the ScanSetting
custom resource to use a standard storage class and create persistent volumes that are 10GB in size and keep the last 10 results:
Example ScanSetting
CR
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSetting
metadata:
name: default
namespace: openshift-compliance
rawResultStorage:
storageClassName: standard
rotation: 10
size: 10Gi
roles:
- worker
- master
scanTolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
schedule: '0 1 * * *'
Applying remediations generated by suite scans
Although you can use the autoApplyRemediations
boolean parameter in a ComplianceSuite
object, you can alternatively annotate the object with compliance.openshift.io/apply-remediations
. This allows the Operator to apply all of the created remediations.
Procedure
- Apply the
compliance.openshift.io/apply-remediations
annotation by running:
$ oc -n openshift-compliance \
annotate compliancesuites/workers-compliancesuite compliance.openshift.io/apply-remediations=
Automatically update remediations
In some cases, a scan with newer content might mark remediations as OUTDATED
. As an administrator, you can apply the compliance.openshift.io/remove-outdated
annotation to apply new remediations and remove the outdated ones.
Procedure
- Apply the
compliance.openshift.io/remove-outdated
annotation:
$ oc -n openshift-compliance \
annotate compliancesuites/workers-compliancesuite compliance.openshift.io/remove-outdated=
Alternatively, set the autoUpdateRemediations
flag in a ScanSetting
or ComplianceSuite
object to update the remediations automatically.
Creating a custom SCC for the Compliance Operator
In some environments, you must create a custom Security Context Constraints (SCC) file to ensure the correct permissions are available to the Compliance Operator api-resource-collector
.
Prerequisites
- You must have
admin
privileges.
Procedure
Define the SCC in a YAML file named
restricted-adjusted-compliance.yaml
:SecurityContextConstraints
object definitionallowHostDirVolumePlugin: false
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegeEscalation: true
allowPrivilegedContainer: false
allowedCapabilities: null
apiVersion: security.openshift.io/v1
defaultAddCapabilities: null
fsGroup:
type: MustRunAs
kind: SecurityContextConstraints
metadata:
name: restricted-adjusted-compliance
priority: 30 (1)
readOnlyRootFilesystem: false
requiredDropCapabilities:
- KILL
- SETUID
- SETGID
- MKNOD
runAsUser:
type: MustRunAsRange
seLinuxContext:
type: MustRunAs
supplementalGroups:
type: RunAsAny
users:
- system:serviceaccount:openshift-compliance:api-resource-collector (2)
volumes:
- configMap
- downwardAPI
- emptyDir
- persistentVolumeClaim
- projected
- secret
1 The priority of this SCC must be higher than any other SCC that applies to the system:authenticated
group.2 Service Account used by Compliance Operator Scanner pod. Create the SCC:
$ oc create -n openshift-compliance -f restricted-adjusted-compliance.yaml
Example output
securitycontextconstraints.security.openshift.io/restricted-adjusted-compliance created
Verification
Verify the SCC was created:
$ oc get -n openshift-compliance scc restricted-adjusted-compliance
Example output
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP PRIORITY READONLYROOTFS VOLUMES
restricted-adjusted-compliance false <no value> MustRunAs MustRunAsRange MustRunAs RunAsAny 30 false ["configMap","downwardAPI","emptyDir","persistentVolumeClaim","projected","secret"]