Installing Consul on Kubernetes

Consul can run directly on Kubernetes, both in server or client mode. For pure-Kubernetes workloads, this enables Consul to also exist purely within Kubernetes. For heterogeneous workloads, Consul agents can join a server running inside or outside of Kubernetes.

You can install Consul on Kubernetes using the following methods:

  1. Consul K8s CLI install
  2. Helm chart install

Refer to the architecture section to learn more about the general architecture of Consul on Kubernetes. For a hands-on experience with Consul as a service mesh for Kubernetes, follow the Getting Started with Consul service mesh tutorial.

Consul K8s CLI Installation

We recommend using the Consul K8s CLI to install Consul on Kubernetes for single-cluster deployments. You can install Consul on Kubernetes using the Consul K8s CLI tool after installing the CLI.

Before beginning the installation process, verify that kubectl is already configured to authenticate to the Kubernetes cluster using a valid kubeconfig file.

The Homebrew package manager is required to complete the following installation instructions.

NOTE: To deploy a previous version of Consul on Kubernetes via the CLI, you will need to first download the specific version of the CLI that matches the version of the control plane that you would like to deploy. Please follow Install a specific version of Consul K8s CLI.

  1. Install the HashiCorp tap, which is a repository of all Homebrew packages for HashiCorp:

    1. $ brew tap hashicorp/tap

  2. Install the Consul K8s CLI with the hashicorp/tap/consul formula.

    1. $ brew install hashicorp/tap/consul-k8s

  3. Issue the install subcommand to install Consul on Kubernetes. Refer to the Consul K8s CLI reference for details about all commands and available options. Without any additional options passed, the consul-k8s CLI will install Consul on Kubernetes by using the Consul Helm chart’s default values. Below is an example that installs Consul on Kubernetes with Service Mesh and CRDs enabled. If you did not set the -auto-approve option to true, you will be prompted to proceed with the installation if the pre-install checks pass.

    The pre-install checks may fail if existing PersistentVolumeClaims (PVC) are detected. Refer to the uninstall instructions for information about removing PVCs.

    1. $ consul-k8s install -set connectInject.enabled=true -set controller.enabled=true
    3. ==> Pre-Install Checks
    4. No existing installations found.
    5.  ✓ No previous persistent volume claims found
    6.  ✓ No previous secrets found
    8. ==> Consul Installation Summary
    9.     Installation name: consul
    10.     Namespace: consul
    11.     Overrides:
    12.     connectInject:
    13.       enabled: true
    14.     controller:
    15.       enabled: true
    17.     Proceed with installation? (y/N) y
    19. ==> Running Installation
    20.   ✓ Downloaded charts
    21.  --> creating 1 resource(s)
    22.  --> creating 45 resource(s)
    23.  --> beginning wait for 45 resources with timeout of 10m0s
    24.   ✓ Consul installed into namespace "consul"

  4. (Optional) Issue the consul-k8s status command to quickly glance at the status of the installed Consul cluster.

    1. $ consul-k8s status
    3. ==> Consul-K8s Status Summary
    4.    NAME  | NAMESPACE |  STATUS  | CHARTVERSION | APPVERSION | REVISION |      LAST UPDATED
    5. ---------+-----------+----------+--------------+------------+----------+--------------------------
    6.   consul | consul    | deployed | 0.40.0       | 1.11.2     |        1 | 2022/01/31 16:58:51 PST
    8. ==> Config:
    9.     connectInject:
    10.       enabled: true
    11.     controller:
    12.       enabled: true
    13.     global:
    14.       name: consul
    16.  ✓ Consul servers healthy (3/3)
    17.  ✓ Consul clients healthy (3/3)

Helm Chart Installation

We recommend using the Consul Helm chart to install Consul on Kubernetes for multi-cluster installations that involve cross-partition of cross datacenter communication. The Helm chart installs and configures all necessary components to run Consul. The configuration enables you to run a server cluster, a client cluster, or both.

Step-by-step tutorials for how to deploy Consul to Kubernetes, please see our Deploy to Kubernetes collection. This collection includes configuration caveats for single-node deployments.

The Helm chart exposes several useful configurations and automatically sets up complex resources, but it does not automatically operate Consul. You must still become familiar with how to monitor, backup, upgrade, etc. the Consul cluster.

The Helm chart has no required configuration and will install a Consul cluster with default configurations. We strongly recommend learning about the configuration options prior to going to production.

Security Warning: By default, the chart will install an insecure configuration of Consul. This provides a less complicated out-of-box experience for new users, but is not appropriate for a production setup. We strongly recommend using a properly-secured Kubernetes cluster or making sure that you understand and enable the recommended security features. Currently, some of these features are not supported in the Helm chart and require additional manual configuration.

Prerequisites

The Consul Helm only supports Helm 3.2+. Install the latest version of the Helm CLI here: Installing Helm.

Installing Consul

  1. Add the HashiCorp Helm Repository:

    1. $ helm repo add hashicorp https://helm.releases.hashicorp.com
    2.  "hashicorp" has been added to your repositories

  2. Verify that you have access to the consul chart:

    1. $ helm search repo hashicorp/consul
    2. NAME                CHART VERSION   APP VERSION DESCRIPTION
    3. hashicorp/consul    0.39.0          1.11.1      Official HashiCorp Consul Chart

  3. Prior to installing via Helm, ensure that the consul Kubernetes namespace does not exist, as installing on a dedicated namespace is recommended.

    1. $ kubectl get namespace
    2. NAME              STATUS   AGE
    3. default           Active   18h
    4. kube-node-lease   Active   18h
    5. kube-public       Active   18h
    6. kube-system       Active   18h

  4. Issue the following command to install Consul with the default configuration using Helm. You could also install Consul on a dedicated namespace of your choosing by modifying the value of the -n flag for the Helm install.

    1. $ helm install consul hashicorp/consul --set global.name=consul --create-namespace --namespace consul
    2. NAME: consul
    3. ...

The Helm chart does everything to set up a recommended Consul-on-Kubernetes deployment. After installation, a Consul cluster will be formed, a leader will be elected, and every node will have a running Consul agent.

Customizing Your Installation

If you want to customize your installation, create a config.yaml file to override the default settings. You can learn what settings are available by running helm inspect values hashicorp/consul or by reading the Helm Chart Reference.

Minimal config.yaml for Consul Service Mesh

The minimal settings to enable Consul Service Mesh) would be captured in the following config.yaml config file:

  1. global:
  2.   name: consul
  3. connectInject:
  4.   enabled: true
  5. controller:
  6.   enabled: true

Installing Consul on Kubernetes - 图1

config.yaml

Once you’ve created your config.yaml file, run helm install with the --values flag:

  1. $ helm install consul hashicorp/consul --create-namespace --namespace consul --values config.yaml
  2. NAME: consul
  3. ...

Enable Consul Service Mesh on select namespaces

By default, Consul Service Mesh is enabled on almost all namespaces (with the exception of kube-system and local-path-storage) within a Kubernetes cluster. You can restrict this to a subset of namespaces by specifying a namespaceSelector that matches a label attached to each namespace denoting whether to enable Consul service mesh. In order to default to enabling service mesh on select namespaces by label, the connectInject.default value must be set to true.

  1. global:
  2.   name: consul
  3. connectInject:
  4.   enabled: true
  5.   default: true
  6.   namespaceSelector: |
  7.     matchLabels:
  8.       connect-inject : enabled
  9. controller:
  10.   enabled: true

Installing Consul on Kubernetes - 图2

config.yaml

Label the namespace(s), where you would like to enable Consul Service Mesh.

  1. $ kubectl create ns foo 
  2. $ kubectl label namespace foo connect-inject=enabled

Next, run helm install with the --values flag:

  1. $ helm install consul hashicorp/consul --create-namespace --namespace consul --values config.yaml
  2. NAME: consul
  3. ...

Updating your Consul on Kubernetes configuration

If you’ve already installed Consul and want to make changes, you’ll need to run helm upgrade. See Upgrading for more details.

Viewing the Consul UI

The Consul UI is enabled by default when using the Helm chart. For security reasons, it isn’t exposed via a LoadBalancer Service by default so you must use kubectl port-forward to visit the UI.

TLS Disabled

If running with TLS disabled, the Consul UI will be accessible via http on port 8500:

  1. $ kubectl port-forward service/consul-server --namespace consul 8500:8500
  2. ...

Once the port is forwarded navigate to http://localhost:8500.

TLS Enabled

If running with TLS enabled, the Consul UI will be accessible via https on port 8501:

  1. $ kubectl port-forward service/consul-server --namespace consul 8501:8501
  2. ...

Once the port is forwarded navigate to https://localhost:8501.

You’ll need to click through an SSL warning from your browser because the Consul certificate authority is self-signed and not in the browser’s trust store.

ACLs Enabled

If ACLs are enabled, you will need to input an ACL token into the UI in order to see all resources and make modifications.

To retrieve the bootstrap token that has full permissions, run:

  1. $ kubectl get secrets/consul-bootstrap-acl-token --template='{{.data.token | base64decode }}'
  2. e7924dd1-dc3f-f644-da54-81a73ba0a178%

Then paste the token into the UI under the ACLs tab (without the %).

NOTE: If using multi-cluster federation, your kubectl context must be in the primary datacenter to retrieve the bootstrap token since secondary datacenters use a separate token with less permissions.

Exposing the UI via a service

If you want to expose the UI via a Kubernetes Service, configure the ui.service chart values. This service will allow requests to the Consul servers so it should not be open to the world.

Accessing the Consul HTTP API

The Consul HTTP API should be accessed by communicating to the local agent running on the same node. While technically any listening agent (client or server) can respond to the HTTP API, communicating with the local agent has important caching behavior, and allows you to use the simpler /agent endpoints for services and checks.

For Consul installed via the Helm chart, a client agent is installed on each Kubernetes node. This is explained in the architecture section. To access the agent, you may use the downward API.

An example pod specification is shown below. In addition to pods, anything with a pod template can also access the downward API and can therefore also access Consul: StatefulSets, Deployments, Jobs, etc.

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4.   name: consul-example
  5. spec:
  6.   containers:
  7.     - name: example
  8.       image: 'consul:latest'
  9.       env:
  10.         - name: HOST_IP
  11.           valueFrom:
  12.             fieldRef:
  13.               fieldPath: status.hostIP
  14.       command:
  15.         - '/bin/sh'
  16.         - '-ec'
  17.         - |
  18.           export CONSUL_HTTP_ADDR="${HOST_IP}:8500"
  19.           consul kv put hello world
  20.   restartPolicy: Never

An example Deployment is also shown below to show how the host IP can be accessed from nested pod specifications:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: consul-example-deployment
  5. spec:
  6.   replicas: 1
  7.   selector:
  8.     matchLabels:
  9.       app: consul-example
  10.   template:
  11.     metadata:
  12.       labels:
  13.         app: consul-example
  14.     spec:
  15.       containers:
  16.         - name: example
  17.           image: 'consul:latest'
  18.           env:
  19.             - name: HOST_IP
  20.               valueFrom:
  21.                 fieldRef:
  22.                   fieldPath: status.hostIP
  23.           command:
  24.             - '/bin/sh'
  25.             - '-ec'
  26.             - |
  27.               export CONSUL_HTTP_ADDR="${HOST_IP}:8500"
  28.               consul kv put hello world

Next Steps

If you are still considering a move to Kubernetes, or to Consul on Kubernetes specifically, our Migrate to Microservices with Consul Service Mesh on Kubernetes collection uses an example application written by a fictional company to illustrate why and how organizations can migrate from monolith to microservices using Consul service mesh on Kubernetes. The case study in this collection should provide information valuable for understanding how to develop services that leverage Consul during any stage of your microservices journey.