Kubernetes annotations and labels
- Labels
- Annotations

Kubernetes annotations and labels

This page provides a complete list of all the annotations you can specify when you run Kuma in Kubernetes mode.

Labels

`kuma.io/sidecar-injection`

Enable or disable sidecar injection.

Example

Used on the namespace it will inject the sidecar in all pods created in the namespace:

apiVersion: v1
kind: Namespace
metadata:
 name: default
 labels:
   kuma.io/sidecar-injection: enabled
[...]

Used on a deployment using pod template it will inject the sidecar in all pods managed by this deployment:

apiVersion: v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  template:
    metadata:
      labels:
        kuma.io/sidecar-injection: enabled
[...]

Labeling pods or deployments will take precedence on the namespace annotation.

`kuma.io/mesh`

Associate Pods with a particular Mesh. Label value must be the name of a Mesh resource.

Example

It can be used on an entire namespace:

apiVersion: v1
kind: Namespace
metadata:
 name: default
 labels:
   kuma.io/mesh: default
[...]

It can be used on a pod:

apiVersion: v1
kind: Pod
metadata:
 name: backend
 labels:
   kuma.io/mesh: default
[...]

Labeling pods or deployments will take precedence on the namespace annotation.

`kuma.io/system-namespace`

This label is used to indicate the Namespace that Kuma stores its secrets in. It’s automatically set on the Namespace the Helm chart is installed into by a Job started by Helm.

Annotations

`kuma.io/gateway`

Lets you specify the Pod should run in gateway mode. Inbound listeners are not generated.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gateway
spec:
  selector:
    matchLabels:
      app: gateway
  template:
    metadata:
      labels:
        app: gateway
      annotations:
        kuma.io/gateway: enabled
[...]

`kuma.io/ingress`

Marks the Pod as the Zone Ingress. Needed for multizone communication – provides the entry point for traffic from other zones.

Example

apiVersion: v1
kind: Pod
metadata:
 name: zone-ingress
 annotations:
   kuma.io/ingress: enabled
[...]

`kuma.io/ingress-public-address`

Specifies the public address for Ingress. If not provided, Kuma picks the address from the Ingress Service.

Example

apiVersion: v1
kind: Pod
metadata:
 name: zone-ingress
 annotations:
   kuma.io/ingress: enabled
   kuma.io/ingress-public-address: custom-address.com
[...]

`kuma.io/ingress-public-port`

Specifies the public port for Ingress. If not provided, Kuma picks the port from the Ingress Service.

Example

apiVersion: v1
kind: Pod
metadata:
 name: zone-ingress
 annotations:
   kuma.io/ingress: enabled
   kuma.io/ingress-public-port: "1234"
[...]

`kuma.io/direct-access-services`

Defines a comma-separated list of Services that can be accessed directly.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/direct-access-services: test-app_playground_svc_80,test-app_playground_svc_443
    kuma.io/transparent-proxying: enabled
    kuma.io/transparent-proxying-inbound-port: [...]
    kuma.io/transparent-proxying-outbound-port: [...]

When you provide this annotation, Kuma generates a listener for each IP address and redirects traffic through a direct-access cluster that’s configured to encrypt connections.

These listeners are needed because transparent proxy and mTLS assume a single IP per cluster (for example, the ClusterIP of a Kubernetes Service). If you pass requests to direct IP addresses, Envoy considers them unknown destinations and manages them in passthrough mode – which means they’re not encrypted with mTLS. The direct-access cluster enables encryption anyway.

WARNING: You should specify this annotation only if you really need it. Generating listeners for every endpoint makes the xDS snapshot very large.

`kuma.io/application-probe-proxy-port`

Specifies the port on which “Application Probe Proxy” listens. Application Probe Proxy coverts HTTPGet, TCPSocket and gRPC probes in the pod to HTTPGet probes and converts back to their original types before sending to the application when actual probe requests are received.

Application Probe Proxy by default listens on port 9001 and it suppresses the “Virtual Probes” feature. By setting it to 0, you can disable this feature and activate “Virtual Probes” unless it’s also disabled.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/application-probe-proxy-port: "9001"
[...]

`kuma.io/virtual-probes`

Enables automatic converting of HttpGet probes to virtual probes. The virtual probe is served on a sub-path of the insecure port specified with kuma.io/virtual-probes-port – for example, :8080/health/readiness -> :9000/8080/health/readiness, where 9000 is the value of the kuma.io/virtual-probes-port annotation.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/virtual-probes: enabled
    kuma.io/virtual-probes-port: "9000"
[...]

`kuma.io/virtual-probes-port`

Specifies the insecure port for listening on virtual probes.

`kuma.io/sidecar-env-vars`

Semicolon (;) separated list of environment variables for the Kuma sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/sidecar-env-vars: TEST1=1;TEST2=2

`kuma.io/container-patches`

Specifies the list of names of ContainerPatch resources to be applied on kuma-init and kuma-sidecar containers.

More information about how to use ContainerPatch you can find at Custom Container Configuration.

Example

It can be used on a resource describing workload (i.e. Deployment, DaemonSet or Pod):

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: kuma-system
  name: example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        kuma.io/container-patches: container-patch-1,container-patch-2
    spec: [...]

`prometheus.metrics.kuma.io/port`

Lets you override the Mesh-wide default port that Prometheus should scrape metrics from.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/port: "1234"

`prometheus.metrics.kuma.io/path`

Lets you override the Mesh-wide default path that Prometheus should scrape metrics from.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/path: "/custom-metrics"

`kuma.io/builtindns`

Tells the sidecar to use its builtin DNS server.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/builtindns: enabled

`kuma.io/builtindnsport`

Port the builtin DNS server should listen on for DNS queries.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/builtindns: enabled
    kuma.io/builtindnsport: "15053"

`kuma.io/ignore`

A boolean to mark a resource as ignored by Kuma. It currently only works for services. This is useful when transitioning to Kuma or to temporarily ignore some entities.

Example

apiVersion: v1
kind: Service
metadata:
  name: example
  annotations:
    kuma.io/ignore: "true"

`traffic.kuma.io/exclude-inbound-ports`

List of inbound ports to exclude from traffic interception by the Kuma sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    traffic.kuma.io/exclude-inbound-ports: "1234,1235"

`traffic.kuma.io/exclude-outbound-ports`

List of outbound ports to exclude from traffic interception by the Kuma sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    traffic.kuma.io/exclude-outbound-ports: "1234,1235"

`kuma.io/transparent-proxying-experimental-engine`

Enable or disable experimental transparent proxy engine on Pod. Default is disabled.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/transparent-proxying-experimental-engine: enabled

`kuma.io/envoy-admin-port`

Specifies the port for Envoy Admin API. If not set, default admin port 9901 will be used.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/envoy-admin-port: "8801"

`kuma.io/envoy-log-level`

Specifies the log level for Envoy system logs to enable. The available log levels are trace, debug, info, warning/warn, error, critical, off. The default is info.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/envoy-log-level: "warning"

`kuma.io/envoy-component-log-level`

Specifies the log level for Envoy system logs to enable by components. See ALL_LOGGER_IDS in logger.h from Envoy source for a list of available components.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/envoy-component-log-level: "upstream:debug,connection:trace"

`kuma.io/service-account-token-volume`

Volume (specified in the pod spec) containing a service account token for Kuma to inject into the sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/service-account-token-volume: "token-vol"
spec:
  automountServiceAccountToken: false
  serviceAccount: example
  containers:
    - image: busybox
      name: busybox
  volumes:
    - name: token-vol
      projected:
        sources:
          - serviceAccountToken:
              expirationSeconds: 7200
              path: token
              audience: "https://kubernetes.default.svc"
          - configMap:
              items:
                - key: ca.crt
                  path: ca.crt
              name: kube-root-ca.crt
          - downwardAPI:
              items:
                - fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.namespace
                  path: namespace

`kuma.io/transparent-proxying-reachable-services`

A comma separated list of kuma.io/service to indicate which services this communicates with. For more details see the reachable services docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  ...
  template:
    metadata:
      ...
      annotations:
        # a comma separated list of kuma.io/service values
        kuma.io/transparent-proxying-reachable-services: "redis_kuma-demo_svc_6379,elastic_kuma-demo_svc_9200"
    spec:
      containers:
        ...

`kuma.io/transparent-proxying-ebpf`

When transparent proxy is installed with ebpf mode, you can disable it for particular workloads if necessary.

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf: disabled
    spec:
      containers:
        [...]

`kuma.io/transparent-proxying-ebpf-bpf-fs-path`

Path to BPF FS if different than default (/sys/fs/bpf)

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-bpf-fs-path: /custom/bpffs/path
    spec:
      containers:
        [...]

`kuma.io/transparent-proxying-ebpf-cgroup-path`

cgroup2 path if different than default (/sys/fs/cgroup)

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-cgroup-path: /custom/cgroup2/path
    spec:
      containers:
        [...]

`kuma.io/transparent-proxying-ebpf-programs-source-path`

Custom path for ebpf programs to be loaded when installing transparent proxy

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-programs-source-path: /custom/ebpf/programs/source/path
    spec:
      containers:
        [...]

`kuma.io/transparent-proxying-ebpf-tc-attach-iface`

Name of the network interface which should be used to attach to it TC-related eBPF programs. By default Kuma will use first, non-loopback interface it’ll find.

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-tc-attach-iface: eth3
    spec:
      containers:
        [...]

`kuma.io/wait-for-dataplane-ready`

Define if you want the kuma-sidecar container to wait for the dataplane to be ready before starting app container. Read relevant Data plane on Kubernetes section for more information.

`prometheus.metrics.kuma.io/aggregate-<name>-enabled`

Define if kuma-dp should scrape metrics from the application that has been defined in the Mesh configuration. Default value: true. For more details see the applications metrics docs

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/aggregate-app-enabled: "false"
spec: ...

`prometheus.metrics.kuma.io/aggregate-<name>-path`

Define path, which kuma-dp sidecar has to scrape for prometheus metrics. Default value: /metrics. For more details see the applications metrics docs

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/aggregate-app-path: "/stats"
spec: ...

`prometheus.metrics.kuma.io/aggregate-<name>-port`

Define port, which kuma-dp sidecar has to scrape for prometheus metrics. For more details see the applications metrics docs

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/aggregate-app-port: "1234"
spec: ...

`kuma.io/transparent-proxying-inbound-v6-port`

Define the port to use for IPv6 traffic. To turn off IPv6 set this to 0.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/transparent-proxying-inbound-v6-port: "0"
spec: ...

`kuma.io/sidecar-drain-time`

Allows specifying drain time of Kuma DP sidecar. The default value is 30s. The default could be changed using the control-plane configuration or KUMA_RUNTIME_KUBERNETES_INJECTOR_SIDECAR_CONTAINER_DRAIN_TIME env.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/sidecar-drain-time: "10s"
spec: ...

`kuma.io/init-first`

Allows specifying that the Kuma init container should run first (ahead of any other init containers). The default is false if omitted. Setting this to true may be desirable for security, as it would prevent network access for other init containers. The order is not guaranteed, as other mutating admission webhooks may further manipulate this ordering.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/init-first: "true"
spec: ...