Enabling the API Audit Log in Downstream Clusters

Kubernetes auditing provides a security-relevant chronological set of records about a cluster. Kube-apiserver performs auditing. Requests generate an event at each stage of its execution, which is then preprocessed according to a certain policy and written to a backend. The policy determines what’s recorded and the backend persists the records.

You might want to configure the audit log as part of compliance with the Center for Internet Security (CIS) Kubernetes Benchmark controls.

For configuration details, refer to the official Kubernetes documentation.

  • RKE2
  • K3s
  • RKE1

You can set audit-policy-file in the configuration file. Rancher delivers the file to the path /var/lib/rancher/rke2/etc/config-files/audit-policy-file in control plane nodes, and sets the proper options in the RKE2 server.

Example:

  1. apiVersion: provisioning.cattle.io/v1
  2. kind: Cluster
  3. spec:
  4. rkeConfig:
  5. machineGlobalConfig:
  6. audit-policy-file: |
  7. apiVersion: audit.k8s.io/v1
  8. kind: Policy
  9. rules:
  10. - level: RequestResponse
  11. resources:
  12. - group: ""
  13. resources:
  14. - pods

Method 2: Use the Directives, machineSelectorFiles and machineGlobalConfig

Enabling the API Audit Log in Downstream Clusters - 图1note

This feature is available in Rancher v2.7.2 and later.

You can use machineSelectorFiles to deliver the audit policy file to the control plane nodes, and machineGlobalConfig to set the options on kube-apiserver.

As a prerequisite, you must create a secret or configmap to be the source of the audit policy.

The secret or configmap must meet the following requirements:

  1. It must be in the fleet-default namespace where the Cluster object exists.
  2. It must have the annotation rke.cattle.io/object-authorized-for-clusters: <cluster-name1>,<cluster-name2> which permits the target clusters to use it.

Enabling the API Audit Log in Downstream Clusters - 图2tip

Rancher Dashboard provides an easy-to-use form for creating the secret or configmap.

Example:

  1. apiVersion: v1
  2. data:
  3. audit-policy: >-
  4. IyBMb2cgYWxsIHJlcXVlc3RzIGF0IHRoZSBNZXRhZGF0YSBsZXZlbC4KYXBpVmVyc2lvbjogYXVkaXQuazhzLmlvL3YxCmtpbmQ6IFBvbGljeQpydWxlczoKLSBsZXZlbDogTWV0YWRhdGE=
  5. kind: Secret
  6. metadata:
  7. annotations:
  8. rke.cattle.io/object-authorized-for-clusters: cluster1
  9. name: <name1>
  10. namespace: fleet-default

Enable and configure the audit log by editing the cluster in YAML, and utilizing the machineSelectorFiles and machineGlobalConfig directives.

Example:

  1. apiVersion: provisioning.cattle.io/v1
  2. kind: Cluster
  3. spec:
  4. rkeConfig:
  5. machineGlobalConfig:
  6. kube-apiserver-arg:
  7. - audit-policy-file=<customized-path>/dev-audit-policy.yaml
  8. - audit-log-path=<customized-path>/dev-audit.logs
  9. machineSelectorFiles:
  10. - fileSources:
  11. - configMap:
  12. name: ''
  13. secret:
  14. items:
  15. - key: audit-policy
  16. path: <customized-path>/dev-audit-policy.yaml
  17. name: dev-audit-policy
  18. machineLabelSelector:
  19. matchLabels:
  20. rke.cattle.io/control-plane-role: 'true'

Enabling the API Audit Log in Downstream Clusters - 图3tip

You can also use the directive machineSelectorConfig with proper machineLabelSelectors to achieve the same effect.

For more information about cluster configuration, refer to the RKE2 cluster configuration reference pages.

Enabling the API Audit Log in Downstream Clusters - 图4note

This feature is available in Rancher v2.7.2 and later.

You can use machineSelectorFiles to deliver the audit policy file to the control plane nodes, and machineGlobalConfig to set the options on kube-apiserver.

As a prerequisite, you must create a secret or configmap to be the source of the audit policy.

The secret or configmap must meet the following requirements:

  1. It must be in the fleet-default namespace where the Cluster object exists.
  2. It must have the annotation rke.cattle.io/object-authorized-for-clusters: <cluster-name1>,<cluster-name2> which permits the target clusters to use it.

Enabling the API Audit Log in Downstream Clusters - 图5tip

Rancher Dashboard provides an easy-to-use form for creating the secret or configmap.

Example:

  1. apiVersion: v1
  2. data:
  3. audit-policy: >-
  4. IyBMb2cgYWxsIHJlcXVlc3RzIGF0IHRoZSBNZXRhZGF0YSBsZXZlbC4KYXBpVmVyc2lvbjogYXVkaXQuazhzLmlvL3YxCmtpbmQ6IFBvbGljeQpydWxlczoKLSBsZXZlbDogTWV0YWRhdGE=
  5. kind: Secret
  6. metadata:
  7. annotations:
  8. rke.cattle.io/object-authorized-for-clusters: cluster1
  9. name: <name1>
  10. namespace: fleet-default

Enable and configure the audit log by editing the cluster in YAML, and utilizing the machineSelectorFiles and machineGlobalConfig directives.

Example:

  1. apiVersion: provisioning.cattle.io/v1
  2. kind: Cluster
  3. spec:
  4. rkeConfig:
  5. machineGlobalConfig:
  6. kube-apiserver-arg:
  7. - audit-policy-file=<customized-path>/dev-audit-policy.yaml
  8. - audit-log-path=<customized-path>/dev-audit.logs
  9. machineSelectorFiles:
  10. - fileSources:
  11. - configMap:
  12. name: ''
  13. secret:
  14. items:
  15. - key: audit-policy
  16. path: <customized-path>/dev-audit-policy.yaml
  17. name: dev-audit-policy
  18. machineLabelSelector:
  19. matchLabels:
  20. rke.cattle.io/control-plane-role: 'true'

Enabling the API Audit Log in Downstream Clusters - 图6tip

You can also use the directive machineSelectorConfig with proper machineLabelSelectors to achieve the same effect.

For more information about cluster configuration, refer to the K3s cluster configuration reference pages.

The audit log can be enabled and configured by editing the cluster with YAML.

When the audit log is enabled, RKE1 default values will be applied.

  1. #
  2. # Rancher Config
  3. #
  4. rancher_kubernetes_engine_config:
  5. services:
  6. kube-api:
  7. audit_log:
  8. enabled: true

You can customize the audit log by using the configuration directive.

  1. #
  2. # Rancher Config
  3. #
  4. rancher_kubernetes_engine_config:
  5. services:
  6. kube-api:
  7. audit_log:
  8. enabled: true
  9. configuration:
  10. max_age: 6
  11. max_backup: 6
  12. max_size: 110
  13. path: /var/log/kube-audit/audit-log.json
  14. format: json
  15. policy:
  16. apiVersion: audit.k8s.io/v1 # This is required.
  17. kind: Policy
  18. omitStages:
  19. - "RequestReceived"
  20. rules:
  21. # Log pod changes at RequestResponse level
  22. - level: RequestResponse
  23. resources:
  24. - group: ""
  25. # Resource "pods" doesn't match requests to any subresource of pods,
  26. # which is consistent with the RBAC policy.
  27. resources: ["pods"]
  28. # Log "pods/log", "pods/status" at Metadata level
  29. - level: Metadata
  30. resources:
  31. - group: ""
  32. resources: ["pods/log", "pods/status"]

For configuration details, refer to the official RKE1 documentation.