Prerequisites Needed to Run Consul-Terraform-Sync

Consul-Terraform-Sync requires a Terraform Provider, a Terraform Module, and a running Consul cluster outside of the consul-terraform-sync daemon. Practitioners can add support for their network infrastructure through Terraform providers. Once network infrastructure support exists, practitioners can add network integrations in the form of Terraform modules.

Run a Consul Cluster

Below are several steps towards a minimum Consul setup required for running Consul-Terraform-Sync.

Install Consul

Consul-Terraform-Sync is a daemon that runs alongside Consul, similar to other Consul ecosystem tools like Consul Template. Consul-Terraform-Sync is not included with the Consul binary and needs to be installed separately.

To install a local Consul agent, refer to the Getting Started: Install Consul Tutorial.

For information on compatible Consul versions, refer to the Consul compatibility matrix.

Run an Agent

The Consul agent must be running in order to dynamically update network devices. To run the local Consul agent, you can run Consul in development mode which can be started with consul agent -dev for simplicity. For more details on running Consul agent, refer to the Getting Started: Run the Consul Agent Tutorial.

When running a Consul agent with Consul-Terraform-Sync in production, we suggest to keep a few considerations in mind. Consul-Terraform-Sync uses blocking queries to monitor task dependencies, like changes to registered services. This results in multiple long running TCP connections between Consul-Terraform-Sync and the agent to poll changes for each dependency. Monitoring a high number of services may quickly hit the default Consul agent connection limits.

There are 2 ways to fix this issue. The first and recommended fix is to use HTTP/2 (requires HTTPS) to communicate between Consul-Terraform-Sync and the Consul agent. When using HTTP/2 only a single connection is made and reused for all communications. See the Consul Configuration section for more. The other option is to configure limits.http_max_conns_per_client for the agent to a reasonable value proportional to the number of services monitored by Consul-Terraform-Sync.

Register Services

Consul-Terraform-Sync monitors Consul catalog for service changes which lead to downstream changes to your network devices. Without services, your Consul-Terraform-Sync daemon will be operational but idle. You can register services with your Consul agent either by loading a service definition or by HTTP API request.

If you are running Consul in development mode, below is an example of registering a service by HTTP API request:

  1. $ echo '{
  2.     "ID": "web",
  3.     "Name": "web",
  4.     "Address": "10.10.10.10",
  5.     "Port": 8000
  6. }' > payload.json
  8. $ curl --request PUT --data @payload.json http://localhost:8500/v1/agent/service/register

The above example registers a service named “web” with your Consul agent. This represents a non-existent web service running at 10.10.10.10:8000. Your web service is now available for Consul-Terraform-Sync to consume. In Consul-Terraform-Sync, you can optionally configure the web service with a service block if it has any non-default values. You can also have Consul-Terraform-Sync monitor the web service to execute a task and update network device(s) by configuring “web” in task.services of a task block.

For more details on registering a service by HTTP API request, refer to the register service API docs.

For more details on registering a service by loading a service definition, refer to the Getting Started: Register a Service with Consul Service Discovery Tutorial.

Run a Cluster

The previous steps of installing and running a single Consul agent then registering a single service is sufficient to meaningfully start running Consul-Terraform-Sync.

If you would like to run a Consul cluster rather than a single agent, refer to Getting Started: Create a Local Consul Datacenter. This will walk you through the steps of running multiple Consul agents and then joining them together into a cluster.

Network Infrastructure (using a Terraform Provider)

Consul-Terraform-Sync integrations for the Terraform driver utilizes Terraform providers as plugins to interface with specific network infrastructure platforms. The Terraform driver of Consul-Terraform-Sync inherits the expansive collection of Terraform providers to integrate with, and with release of Terraform 0.13, this extends to include providers written by the community too by using provider source.

Finding Terraform Providers

To find providers for the infrastructure platforms you use, browse the providers section of the Terraform Registry.

How to Create a Provider

If there is no existing Terraform provider, a new Terraform provider can be created and published. The provider can then be used within a network integration task by authoring a compatible Terraform module.

Network Integration (using a Terraform Module)

The Terraform module for a task in Consul-Terraform-Sync is the core component of the integration. It declares which resources and how your infrastructure is dynamically updated.

Working with a Terraform provider, you can write an integration task for Consul-Terraform-Sync by creating a Terraform module that is compatible with the Terraform driver or use a module built by partners below.

Continue to the next page to get started with configuring Consul-Terraform-Sync and how to use Terraform providers and modules for tasks.

Partner Terraform Modules

The modules listed below are available to use and are compatible with Consul-Terraform-Sync.

A10 Networks

Avi Networks

  • Scale Up and Scale Down Pool and Pool Members (Servers): GitHub

AWS Application Load Balancer (ALB)

  • Create Listener Rule and Target Group for an AWS ALB, Forward Traffic to Consul Ingress Gateway: Terraform Registry / GitHub

Checkpoint

Cisco ACI

F5

NS1

Palo Alto Networks