Quick Start: Collecting Pod Logs in Kubernetes

The following article will show you how to quickly collect Pod logs by creating a LogConfig CRD in a Kubernetes cluster.

1. Prepare Kubernetes

You can use an existing Kubernetes cluster, or deploy Kubernetes. It is recommended to use Kind to build a Kubernetes cluster locally.

The operations need to be used locally:

Make sure you have kubectl and helm executable locally.

2. Deploy Loggie DaemonSet

You can view all released deployment charts on the installation page.

You can choose:

Download the chart and deploy it

  1. VERSION=v1.3.0
  2. helm pull https://github.com/loggie-io/installation/releases/download/${VERSION}/loggie-${VERSION}.tgz && tar xvzf loggie-${VERSION}.tgz

Try to modify values.yaml in it. Please replace the <VERSION> above with the specific version number.

Deploy:

  1. helm install loggie ./loggie -nloggie --create-namespace

You can also:

Deploy directly：

  1. helm install loggie -nloggie --create-namespace https://github.com/loggie-io/installation/releases/download/${VERSION}/loggie-${VERSION}.tgz

Please replace the <VERSION> above with the specific version number.

Want to use image of another version?

In order to facilitate the experience of the latest fixes and features, we provide the image version of the main branch after each merge, which can be selected here.
At the same time, you can add to the helm install command --set image=loggieio/loggie:vX.Y.Z to specify a specific Loggie image.

Problems with deployment?

If you have problems when trying to deploy, or the following demonstration operation fails in your environment, please refer to deploy Loggie in Kubernetes and modify the relevant configuration.

3. Collect Logs

Loggie defines Kubernetes CRD LogConfig. A LogConfig represents a log collection task for collecting a set of Pods.

3.1 Create Pods to Be Collected

We first create a Pod for log collection.

  1. kubectl create deploy nginx --image=nginx

Next, the standard output log “stdout” of this Nginx Pod will be collected.

3.2 Define the Sink

Next, we create a Sink instance (CRD defined by Loggie), indicating the backend for log sending. For the convenience of demonstration, here we send the log to the log of the Loggie Agent itself and print it.

  1. cat << EOF | kubectl apply -f -
  2. apiVersion: loggie.io/v1beta1
  3. kind: Sink
  4. metadata:
  5.   name: default
  6. spec:
  7.   sink: |
  8.     type: dev
  9.     printEvents: true
  10. EOF

You can use kubectl get sink to view the Sink that has been created.

3.3 Define Collection Tasks

Loggie defines CRD LogConfig, which represents a log collection task. We create a LogConfig example as follows:

  1. cat << EOF | kubectl apply -f -
  2. apiVersion: loggie.io/v1beta1
  3. kind: LogConfig
  4. metadata:
  5.   name: nginx
  6.   namespace: default
  7. spec:
  8.   selector:
  9.     type: pod
  10.     labelSelector:
  11.       app: nginx
  12.   pipeline:
  13.     sources: |
  14.       - type: file
  15.         name: mylog
  16.         paths:
  17.         - stdout
  18.     sinkRef: default
  19. EOF

As you can see, the sinkRef uses the reference just created sink default CR. we can also use the sink field directly in Logconfig:

  1. cat << EOF | kubectl apply -f -
  2. apiVersion: loggie.io/v1beta1
  3. kind: LogConfig
  4. metadata:
  5.   name: nginx
  6.   namespace: default
  7. spec:
  8.   selector:
  9.     type: pod
  10.     labelSelector:
  11.       app: nginx
  12.   pipeline:
  13.     sources: |
  14.       - type: file
  15.         name: mylog
  16.         paths:
  17.         - stdout
  18.     sink: |
  19.       type: dev
  20.       printEvents: true
  21.       codec:
  22.         type: json
  23.         pretty: true
  24. EOF

After creation, we can use kubectl get lgc to view the created CRD instance.

At the same time, we can also use kubectl describe lgc nginx to get the latest status by viewing the events of LogConfig.

  1. Events:
  2.   Type    Reason       Age   From                       Message
  3.   ----    ------       ----  ----                       -------
  4.   Normal  syncSuccess  52s   loggie/kind-control-plane  Sync type pod [nginx-6799fc88d8-5cb67] success

The above nginx LogConfig chooses Pods to collect log through the spec.selector. Here we use app: nginx to choose the nginx Pod that was just created. spec.pipeline represents Loggie’s Pipeline configuration. We only collect logs from the standard output of the container, so we fill stdout in paths.

4. View Log

First find the nginx pod node where it is located:

  1. kubectl get po -owide -l app=nginx

Then we find the Loggie of that node:

  1. kubectl -nloggie get po -owide |grep ${node}

Use

  1. kubectl -nloggie logs -f ${logge-pod}

to check the log printed by Loggie, which shows the collected nginx log.

More

The above is just a simple demonstration. If you have problems with the deployment or want to know more about how to use Loggie in Kubernetes…