Logs Collected in Sidecar Method

Although it is not recommended to use the sidecar method to collect container logs by default, in some limited scenarios, only the sidecar method can be used to collect container logs. Here we give an example for reference.

General Idea

As shown below:

Loggie and the business container are deployed in the same Pod, and the same log file volume needs to be mounted. In addition, the configuration of Loggie can be mounted to the container through configMap. Loggie collects the logs of the container according to the provided configuration file in configMap, and send them to the backend.

Injecte Loggie Sidecar

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: tomcat
  name: tomcat
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels:
        app: tomcat
    spec:
      containers:
      - name: tomcat
        image: tomcat
        volumeMounts:
        - mountPath: /usr/local/tomcat/logs
          name: log
      - name: loggie
        args:
        - -config.system=/opt/loggie/loggie.yml
        - -config.pipeline=/opt/loggie/pipeline.yml
        image: loggieio/loggie:main
        volumeMounts:
        # same log volume
        - mountPath: /usr/local/tomcat/logs
          name: log
        # loggie configuration configMap
        - mountPath: /opt/loggie
          name: loggie-config
        # loggie data
        - mountPath: /data
          name: registry
      volumes:
      - emptyDir: {}
        name: log
      - emptyDir: {}
        name: registry
      - name: loggie-config
        configMap:
          name: tomcat-loggie-config
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: tomcat-loggie-config
  namespace: default
data:
  loggie.yml: |
    loggie:
      reload:
        enabled: true
        period: 10s
      monitor:
        logger:
          period: 30s
          enabled: true
        listeners:
          filesource: ~
          filewatcher: ~
          reload: ~
          sink: ~
      http:
        enabled: true
        port: 9196
  pipeline.yml: |
    pipelines:
    - name: "tomcat"
      sources:
        - type: "file"
          name: "tomcatlog"
          paths:
            - "/usr/local/tomcat/logs/*.log"
          fields:
            namespace: default
            deployment: tomcat
          fieldsFromEnv:
            podname: HOSTNAME
      sink:
        type: "dev"
        printEvents: true
        codec:
          pretty: true

You can use the loggie sidecar to collect container logs by referring to the above exapmle. Node:

• At present, it is not recommended to enable kubernetes discovery in the configuration file. Since Kubernetes will be requested after opening, when there are many Pods, it will cause certain pressure on Kubernetes. Therefore, LogConfig CRD cannot be used, and configMap needs to be used to mount the configuration file.
• Since Kubernetes discovery is not used, the fields here will not be automatically added with the Pod’s meta information, and need to be obtained from the Pod’s environment variables using the fieldsFromEnv.

Tips

• Fixed information such as namespace can be configured in fields, or referenced to env by using the downward API.

• The Env environment variable obtained by fieldsFromEnv is not limited to the env field configured in the Pod yaml, but any environment variable in the Loggie container. We can execute the env command in the container to view it.

• Modifying the parameters in the configMap will take a period of time to be refreshed to the Pod. If you want it to take effect immediately, you need to rebuild the Pod. Please pay attention to whether it will affect the business.

Info

In the future, Loggie will support automatic Sidecar injection and automatic generation of ConfigMap mounts through LogConfig, so as to achieve the same experience as using DaemonSet.