Terraform Driver

Consul-Terraform-Sync (CTS) extends the Consul ecosystem to include Terraform as an officially supported tooling project. With the Terraform driver, CTS installs the Terraform CLI locally and runs Terraform commands based on monitored Consul changes. This page details how the Terraform driver operates using local workspaces and templated files.

Terraform CLI Automation

On startup, CTS:

1. Downloads and installs Terraform
2. Prepares local workspace directories. Terraform configuration and execution for each task is organized as separate Terraform workspaces. The state files for tasks are independent of each other.
3. Generates Terraform configuration files that make up the root module for each task.

Once all workspaces are set up, CTS monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform driver dynamically updates input variables for that task using a template to render them to a file named terraform.tfvars. This file is passed as a parameter to the Terraform CLI when executing terraform plan and terraform apply to update your network infrastructure with the latest Consul service details.

Local Workspaces

Within the CTS configuration for a task, practitioners can select the desired module to run for the task as well as set the condition to execute the task. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. CTS will manage concurrent execution of these tasks.

Autogenerated root modules for tasks are maintained in local subdirectories of the CTS working directory. Each subdirectory represents the local workspace for a task. By default, the working directory sync-tasks is created in the current directory. To configure where Terraform configuration files are stored, set working_dir to the desired path or configure the task.working_dir individually.

Note: Although Terraform state files for task workspaces are independent, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation.

Root Module

The root module proxies Consul information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from CTS configuration, information for task’s module to use as the automation playbook, and information from Consul such as service information.

A working directory with one task named “cts-example” would have the folder structure below when running with the Terraform driver.

$ tree sync-tasks/
sync-tasks/
└── cts-example/
    ├── main.tf
    ├── variables.tf
    ├── terraform.tfvars
    └── terraform.tfvars.tmpl

$ tree sync-tasks/
sync-tasks/
└── cts-example/
    ├── main.tf
    ├── variables.tf
    ├── terraform.tfvars
    └── terraform.tfvars.tmpl

The following files of the root module are generated for each task. An example of a root module created by CTS can be found in the project repository.

• main.tf - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
  • terraform block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module. The Terraform backend from the configuration is also templated here.
  • provider blocks - The provider blocks generated in the root module resemble the terraform_provider blocks from the configuration for CTS. They have identical arguments present and are set from the intermediate variable created per provider.
  • module block - The module block is where the task’s module is called as a child module. The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
• variables.tf - This file contains three types of variable declarations.
  • services input variable (required) determines module compatibility with Consul-Terraform Sync (read more on compatible Terraform modules for more details).
  • Any additional optional input variables provided by CTS that the module may use.
  • Various intermediate variables used to configure providers. Intermediate provider variables are interpolated from the provider blocks and arguments configured in the CTS configuration.
• variables.module.tf - This file is created if there are variables configured for the task and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
• providers.tfvars - This file is created if there are providers configured for the task and defined terraform_provider blocks. This file may contain sensitive information. To omit sensitive information from this file, you can securely configure Terraform providers for CTS using environment variables or templating.
• terraform.tfvars - The variable definitions file is where the services input variable and any optional CTS input variables are assigned values from Consul. It is periodically updated, typically when the task condition is met, to reflect the current state of Consul.
• terraform.tfvars.tmpl - The template file is used by CTS to template information from Consul by using the HashiCorp configuration and templating library (hashicorp/hcat).

Note: Generated template and Terraform configuration files are crucial for the automation of tasks. Any manual changes to the files may not be preserved and could be overwritten by a subsequent update. Unexpected manual changes to the format of the files may cause automation to error.