Add Fluentd as Receiver (aka Collector)

KubeSphere supports using Elasticsearch, Kafka and Fluentd as log receivers. This doc will demonstrate:

  • How to deploy Fluentd as deployment and create corresponding service and configmap.
  • How to add Fluentd as a log receiver to receive logs sent from Fluent Bit and then output to stdout.
  • How to verify if Fluentd receives logs successfully.

Prerequisites

  • Before adding a log receiver, you need to enable any of the logging, events or auditing components following Enable Pluggable Components. The logging component is enabled as an example in this doc.
  • To configure log collection, you should use an account of platform-admin role.

Step 1: Deploy Fluentd as a deployment

Usually, Fluentd is deployed as a daemonset in K8s to collect container logs on each node. KubeSphere chooses Fluent Bit for this purpose because of its low memory footprint. Besides, Fluentd features numerous output plugins. Hence, KubeSphere chooses to deploy Fluentd as a deployment to forward logs it receives from Fluent Bit to more destinations such as S3, MongoDB, Cassandra, MySQL, syslog and Splunk.

To deploy Fluentd as a deployment, you simply need to open the kubectl console in KubeSphere Toolbox and run the following command:

Note

  • The following command will deploy Fluentd deployment, service and configmap into the default namespace and add a filter to Fluentd configmap to exclude logs from the default namespace to avoid Fluent Bit and Fluentd loop logs collection.
  • You’ll need to change all these default to the namespace you selected if you want to deploy to a different namespace.
  1. cat <<EOF | kubectl apply -f -
  2. apiVersion: v1
  3. kind: ConfigMap
  4. metadata:
  5.   name: fluentd-config
  6.   namespace: default
  7. data:
  8.   fluent.conf: |-
  9.     # Receive logs sent from Fluent Bit on port 24224
  10.     <source>
  11.       @type forward
  12.       port 24224
  13.     </source>
  15.     # Because this will send logs Fluentd received to stdout,
  16.     # to avoid Fluent Bit and Fluentd loop logs collection,
  17.     # add a filter here to avoid sending logs from the default namespace to stdout again
  18.     <filter **>
  19.       @type grep
  20.       <exclude>
  21.         key $.kubernetes.namespace_name
  22.         pattern /^default$/
  23.       </exclude>
  24.     </filter>
  26.     # Send received logs to stdout for demo/test purpose only
  27.     # Various output plugins are supported to output logs to S3, MongoDB, Cassandra, MySQL, syslog and Splunk etc.
  28.     <match **>
  29.       @type stdout
  30.     </match>
  31. ---
  32. apiVersion: apps/v1
  33. kind: Deployment
  34. metadata:
  35.   labels:
  36.     app: fluentd
  37.   name: fluentd
  38.   namespace: default
  39. spec:
  40.   replicas: 1
  41.   selector:
  42.     matchLabels:
  43.       app: fluentd
  44.   template:
  45.     metadata:
  46.       labels:
  47.         app: fluentd
  48.     spec:
  49.       containers:
  50.       - image: fluentd:v1.9.1-1.0
  51.         imagePullPolicy: IfNotPresent
  52.         name: fluentd
  53.         ports:
  54.         - containerPort: 24224
  55.           name: forward
  56.           protocol: TCP
  57.         - containerPort: 5140
  58.           name: syslog
  59.           protocol: TCP
  60.         volumeMounts:
  61.         - mountPath: /fluentd/etc
  62.           name: config
  63.           readOnly: true
  64.       volumes:
  65.       - configMap:
  66.           defaultMode: 420
  67.           name: fluentd-config
  68.         name: config
  69. ---
  70. apiVersion: v1
  71. kind: Service
  72. metadata:
  73.   labels:
  74.     app: fluentd-svc
  75.   name: fluentd-svc
  76.   namespace: default
  77. spec:
  78.   ports:
  79.   - name: forward
  80.     port: 24224
  81.     protocol: TCP
  82.     targetPort: forward
  83.   selector:
  84.     app: fluentd
  85.   sessionAffinity: None
  86.   type: ClusterIP
  87. EOF

Step 2: Add Fluentd as log receiver (aka collector)

  1. To add a log receiver:
  • Login KubeSphere with an account of platform-admin role
  • Click Platform -> Clusters Management
  • Select a cluster if multiple clusters exist
  • Click Cluster Settings -> Log Collections
  • Log receivers can be added by clicking Add Log Collector

Add receiver

  1. Choose Fluentd and fill in the Fluentd service address and port like below:

Add Fluentd

  1. Fluentd appears in the receiver list of Log Collections UI and its status shows Collecting.

Receiver List

  1. Verify whether Fluentd is receiving logs sent from Fluent Bit:
  • Click Application Workloads in the Cluster Management UI.
  • Select Workloads and then select the default namespace in the Workload - Deployments tab
  • Click the fluentd item and then click the fluentd-xxxxxxxxx-xxxxx pod
  • Click the fluentd container
  • In the fluentd container page, select the Container Logs tab

You’ll see logs begin to scroll up continuously.

Container Logs