Traefik & Kubernetes

The Kubernetes Ingress Controller, The Custom Resource Way.

Traefik used to support Kubernetes only through the Kubernetes Ingress provider, which is a Kubernetes Ingress controller in the strict sense of the term.

However, as the community expressed the need to benefit from Traefik features without resorting to (lots of) annotations,we ended up writing a Custom Resource Definition (alias CRD in the following) for an IngressRoute type, defined below, in order to provide a better way to configure access to a Kubernetes cluster.

Provider Configuration

endpoint

Optional, Default=empty

[providers.kubernetesCRD]
  endpoint = "http://localhost:8080"
  # ...

providers:
  kubernetesCRD:
    endpoint = "http://localhost:8080"
    # ...

--providers.kubernetescrd.endpoint="http://localhost:8080"

The Kubernetes server endpoint as URL.

When deployed into Kubernetes, Traefik will read the environment variables KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT or KUBECONFIG to construct the endpoint.

The access token will be looked up in /var/run/secrets/kubernetes.io/serviceaccount/token and the SSL CA certificate in /var/run/secrets/kubernetes.io/serviceaccount/ca.crt.Both are provided mounted automatically when deployed inside Kubernetes.

The endpoint may be specified to override the environment variable values inside a cluster.

When the environment variables are not found, Traefik will try to connect to the Kubernetes API server with an external-cluster client.In this case, the endpoint is required.Specifically, it may be set to the URL used by kubectl proxy to connect to a Kubernetes cluster using the granted authentication and authorization of the associated kubeconfig.

token

Optional, Default=empty

[providers.kubernetesCRD]
  token = "mytoken"
  # ...

providers:
  kubernetesCRD:
    token = "mytoken"
    # ...

--providers.kubernetescrd.token="mytoken"

Bearer token used for the Kubernetes client configuration.

certAuthFilePath

Optional, Default=empty

[providers.kubernetesCRD]
  certAuthFilePath = "/my/ca.crt"
  # ...

providers:
  kubernetesCRD:
    certAuthFilePath: "/my/ca.crt"
    # ...

--providers.kubernetescrd.certauthfilepath="/my/ca.crt"

Path to the certificate authority file.Used for the Kubernetes client configuration.

namespaces

Optional, Default: all namespaces (empty array)

[providers.kubernetesCRD]
  namespaces = ["default", "production"]
  # ...

providers:
  kubernetesCRD:
    namespaces:
    - "default"
    - "production"
    # ...

--providers.kubernetescrd.namespaces="default,production"

Array of namespaces to watch.

labelselector

Optional,Default: empty (process all Ingresses)

[providers.kubernetesCRD]
  labelselector = "A and not B"
  # ...

providers:
  kubernetesCRD:
    labelselector: "A and not B"
    # ...

--providers.kubernetescrd.labelselector="A and not B"

By default, Traefik processes all Ingress objects in the configured namespaces.A label selector can be defined to filter on specific Ingress objects only.

See label-selectors for details.

ingressClass

Optional, Default: empty

[providers.kubernetesCRD]
  ingressClass = "traefik-internal"
  # ...

providers:
  kubernetesCRD:
    ingressClass: "traefik-internal"
    # ...

--providers.kubernetescrd.ingressclass="traefik-internal"

Value of kubernetes.io/ingress.class annotation that identifies Ingress objects to be processed.

If the parameter is non-empty, only Ingresses containing an annotation with the same value are processed.Otherwise, Ingresses missing the annotation, having an empty value, or the value traefik are processed.

throttleDuration

Optional, Default: 0 (no throttling)

[providers.kubernetesCRD]
  throttleDuration = "10s"
  # ...

providers:
  kubernetesCRD:
    throttleDuration: "10s"
    # ...

--providers.kubernetescrd.throttleDuration="10s"

Resource Configuration

If you're in a hurry, maybe you'd rather go through the dynamic configuration reference.

Traefik IngressRoute definition

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

That IngressRoute kind can then be used to define an IngressRoute object, such as in:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutefoo
spec:
  entryPoints:
    - web
  routes:
  # Match is the rule corresponding to an underlying router.
  # Later on, match could be the simple form of a path prefix, e.g. just "/bar",
  # but for now we only support a traefik style matching rule.
  - match: Host(`foo.com`) && PathPrefix(`/bar`)
    # kind could eventually be one of "Rule", "Path", "Host", "Method", "Header",
    # "Parameter", etc, to support simpler forms of rule matching, but for now we
    # only support "Rule".
    kind: Rule
    # (optional) Priority disambiguates rules of the same length, for route matching.
    priority: 12
    services:
    - name: whoami
      port: 80
      # (default 1) A weight used by the weighted round-robin strategy (WRR).  
      weight: 1
      # (default true) PassHostHeader controls whether to leave the request's Host
      # Header as it was before it reached the proxy, or whether to let the proxy set it
      # to the destination (backend) host.
      passHostHeader: true
      responseForwarding:
        # (default 100ms) Interval between flushes of the buffered response body to the client.
        flushInterval: 100ms
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: ingressroutetcpfoo.crd
spec:
  entryPoints:
    - footcp
  routes:
  # Match is the rule corresponding to an underlying router.
  - match: HostSNI(`*`)
    services:
    - name: whoamitcp
      port: 8080

Middleware

Additionally, to allow for the use of middlewares in an IngressRoute, we defined the CRD below for the Middleware kind.

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

Once the Middleware kind has been registered with the Kubernetes cluster, it can then be used in IngressRoute definitions, such as:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: stripprefix
  namespace: foo
spec:
  stripPrefix:
    prefixes:
      - /stripit
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutebar
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`bar.com`) && PathPrefix(`/stripit`)
    kind: Rule
    services:
    - name: whoami
      port: 80
    middlewares:
    - name: stripprefix
      namespace: foo

Cross-provider namespace

As Kubernetes also has its own notion of namespace, one should not confuse the kubernetes namespace of a resource

(in the reference to the middleware) with the provider namespace,when the definition of the middleware is from another provider.In this context, specifying a namespace when referring to the resource does not make any sense, and will be ignored.

More information about available middlewares in the dedicated middlewares section.

TLS Option

Additionally, to allow for the use of TLS options in an IngressRoute, we defined the CRD below for the TLSOption kind.More information about TLS Options is available in the dedicated TLS Configuration Options.

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

Once the TLSOption kind has been registered with the Kubernetes cluster or defined in the File Provider, it can then be used in IngressRoute definitions, such as:

apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
  name: mytlsoption
  namespace: default
spec:
  minVersion: VersionTLS12
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutebar
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`bar.com`) && PathPrefix(`/stripit`)
    kind: Rule
    services:
    - name: whoami
      port: 80
  tls:
    options: 
      name: mytlsoption
      namespace: default

References and namespaces

If the optional namespace attribute is not set, the configuration will be applied with the namespace of the IngressRoute.

Additionally, when the definition of the TLS option is from another provider,

the cross-provider syntax ([email protected]) should be used to refer to the TLS option,just as in the middleware case.Specifying a namespace attribute in this case would not make any sense, and will be ignored.

TLS

To allow for TLS, we made use of the Secret kind, as it was already defined, and it can be directly used in an IngressRoute:

apiVersion: v1
kind: Secret
metadata:
  name: supersecret
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutetls
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`foo.com`) && PathPrefix(`/bar`)
    kind: Rule
    services:
    - name: whoami
      port: 443
  tls:
    secretName: supersecret

Further

Also see the full example with Let's Encrypt.