Calico node status

A Calico node status resource (CalicoNodeStatus) represents a collection of status information for a node that Calico reports back to the user for use during troubleshooting.

As of today, status of BGP agents, BGP sessions and routes exposed to BGP agents are collected from Linux nodes only. Windows nodes are not supported at this time. Calico node status resource is only valid when Calico BGP networking is in use.

Notes

The updating of CalicoNodeStatus will have a small performance impact on CPU/Memory usage of the node as well as adding load to kubernetes apiserver.

In our testing on a ten node, full mesh cluster, a CalicoNodeStatus resource was created for each node where the update interval was set to ten seconds. On each node, this resulted in an increase in CPU use of 5% of a vCPU and an increase of 4MB of memory. The control plane node recorded an increase in CPU usage of 5% of a vCPU for these 10 nodes.

caution

The implementation of CalicoNodeStatus is designed to handle a small number of nodes (less than 10 is recommended) reporting back status in the same time. If CalicoNodeStatus are created for a large number of nodes, and with short update interval, the kubernetes apiserver may become slower and less responsive. You should create CalicoNodeStatus for the node you are interested in and for debugging purpose only. CalicoNodeStatus resource should be deleted upon the completion of the debugging process.

Sample YAML

To use this function, the user creates a CalicoNodeStatus object for the node, specifying the information to collect and the interval it should be collected at. This example collects information for node “my-kadm-node-0” with an update interval of 10 seconds.

kubectl apply -f -<<EOF
apiVersion: projectcalico.org/v3
kind: CalicoNodeStatus
metadata:
  name: my-caliconodestatus-1
spec:
  classes:
    - Agent
    - BGP
    - Routes
  node: my-kadm-node-0
  updatePeriodSeconds: 10
EOF

The user then reads back the same resource using the command kubectl get caliconodestatus my-caliconodestatus-1 -o yaml, and the information collected can be found in the status field. For example:

apiVersion: projectcalico.org/v3
kind: CalicoNodeStatus
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"crd.projectcalico.org/v1","kind":"CalicoNodeStatus","metadata":{"annotations":{},"name":"my-caliconodestatus-1"},"spec":{"classes":["Agent","BGP","Routes"],"node":"my-kadm-node-0","updatePeriodSeconds":10}}
    projectcalico.org/metadata: '{"uid":"6e3b676b-546a-41db-aeda-ad44f25ff73f","generation":2,"creationTimestamp":"2021-10-18T10:30:43Z","managedFields":[{"manager":"kubectl-client-side-apply","operation":"Update","apiVersion":"projectcalico.org/v3","time":"2021-10-18T10:30:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:node":{},"f:updatePeriodSeconds":{}}}},{"manager":"kubectl-edit","operation":"Update","apiVersion":"crd.projectcalico.org/v1","time":"2021-10-18T12:59:25Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:classes":{}}}}]}'
  creationTimestamp: '2021-10-18T10:30:43Z'
  generation: 139
  name: my-caliconodestatus-1
  resourceVersion: '203255'
  uid: 6e3b676b-546a-41db-aeda-ad44f25ff73f
spec:
  classes:
    - Agent
    - BGP
    - Routes
  node: my-kadm-node-0
  updatePeriodSeconds: 10
status:
  agent:
    birdV4:
      lastBootTime: '2021-10-18 09:23:19'
      lastReconfigurationTime: '2021-10-18 09:23:19'
      routerID: 10.128.0.240
      state: Ready
      version: v0.3.3+birdv1.6.8
    birdV6:
      lastBootTime: '2021-10-18 09:23:19'
      lastReconfigurationTime: '2021-10-18 09:23:19'
      routerID: 10.128.0.240
      state: Ready
      version: v0.3.3+birdv1.6.8
  bgp:
    numberEstablishedV4: 2
    numberEstablishedV6: 0
    numberNotEstablishedV4: 1
    numberNotEstablishedV6: 0
    peersV4:
      - peerIP: 10.128.0.242
        since: '09:23:22'
        state: Established
        type: NodeMesh
      - peerIP: 10.128.0.200
        since: '09:23:21'
        state: Established
        type: NodeMesh
      - peerIP: 10.128.0.241
        since: '09:23:22'
        state: OpenSent
        type: NodeMesh
  lastUpdated: '2021-10-18T13:22:03Z'
  routes:
    routesV4:
      - destination: 0.0.0.0/0
        gateway: 172.18.0.1
        interface: eth0
        learnedFrom:
          sourceType: Kernel
        type: FIB
      - destination: 192.168.82.0/26
        gateway: 10.128.0.200
        interface: eth0
        learnedFrom:
          peerIP: 10.128.0.200
          sourceType: NodeMesh
        type: FIB
      - destination: 192.168.36.0/26
        gateway: 10.128.0.242
        interface: eth0
        learnedFrom:
          peerIP: 10.128.0.242
          sourceType: NodeMesh
        type: FIB
      - destination: 10.128.0.5/16
        gateway: N/A
        interface: eth0
        learnedFrom:
          sourceType: Direct
        type: FIB

Calico node status definition

Metadata

Spec

Status

Classes

BGPDaemonStatus

BGPStatus

RouteStatus

BGPPeer

Route

LearnedFrom