Configuration Options for Consul-Terraform-Sync

The Consul-Terraform-Sync daemon is configured using configuration files and supports HashiCorp Configuration Language (HCL) and JSON file formats.

Global Config Options

Top level options are reserved for configuring Consul-Terraform-Sync.

log_level   = "INFO"
working_dir = "sync-tasks"
port        = 8558
syslog {
  facility = "local2"
}
buffer_period {
  enabled = true
  min     = "5s"
  max     = "20s"
}
tls {
  enabled         = true
  cert            = "/path/to/cert.pem"
  key             = "/path/to/key.pem"
  verify_incoming = true
  ca_cert         = "/path/to/ca.pem"
}

• buffer_period - Configures the default buffer period for all dynamic tasks to dampen the effects of flapping services to downstream network devices. It defines the minimum and maximum amount of time to wait for the cluster to reach a consistent state and accumulate changes before triggering task executions. The default is enabled to reduce the number of times downstream infrastructure is updated within a short period of time. This is useful to enable in systems that have a lot of flapping. Buffer periods do not apply to scheduled tasks.
  • enabled - (bool: true) Enable or disable buffer periods globally. Specifying min will also enable it.
  • min - (string: “5s”) The minimum period of time to wait after changes are detected before triggering related tasks.
  • max - (string: “20s”) The maximum period of time to wait after changes are detected before triggering related tasks. If min is set, the default period for max is 4 times the value of min.
• log_level - (string: “INFO”) The log level to use for Consul-Terraform-Sync logging. This defaults to “INFO”. The available log levels are “TRACE”, “DEBUG”, “INFO”, “WARN”, and “ERR”.
• port - (int: 8558) The port for Consul-Terraform-Sync to use to serve API requests.
• syslog - Specifies the syslog server for logging.
  • enabled - (bool) Enable syslog logging. Specifying other option also enables syslog logging.
  • facility - (string: “local0”) Name of the syslog facility to log to.
  • name - (string: “consul-terraform-sync”) Name to use for the daemon process when logging to syslog.
• working_dir - (string: “sync-tasks”) Specifies the base working directory for managing the artifacts generated by Consul-Terraform-Sync for each task, including Terraform configuration files. Consul-Terraform-Sync will also generate a subdirectory for each task, e.g., ./sync-tasks/task-name. The subdirectory is the working directory for the task. Use the task.working_dir option to override task-level working directories.
• tls - Configure TLS on the Consul-Terraform-Sync API.
  • enabled - (bool: false) Enable TLS. Providing a value for any of the TLS options will enable this parameter implicitly.
  • cert - (string) The path to a PEM-encoded certificate file used for TLS connections to the CTS API.
  • key - (string) The path to the PEM-encoded private key file used with the certificate configured by cert.
  • verify_incoming - (bool: false) Enable mutual TLS. Requires all incoming connections to the CTS API to use a TLS connection and provide a certificate signed by a Certificate Authority specified by the ca_cert or ca_path.
  • ca_cert - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the incoming client connections to the CTS API when verify_incoming is set to true. Takes precedence over ca_path if both are configured.
  • ca_path - (string) The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the incoming client connections to the CTS API when verify_incoming is set to true.

• license_path

  Enterprise

  - (string: “”) Configures the path to the file containing the license. A license must be set to use enterprise features. You can also set the license by defining the CONSUL_LICENSE and CONSUL_LICENSE_PATH environment variables. See Setting the License for details.

Consul

The consul block is used to configure Consul-Terraform-Sync connection with a Consul agent to perform queries to the Consul Catalog and Consul KV pertaining to task execution.

Note: Use HTTP/2 to improve Consul-Terraform-Sync performance when communicating with the local Consul process. TLS/HTTPS must be configured for the local Consul with the cert_file and key_file parameters set. For the Consul-Terraform-Sync configuration, set tls.enabled = true and set the address parameter to the HTTPS URL, e.g., address = example.consul.com:8501. If using self-signed certificates for Consul, you will also need to set tls.verify = false or add the certificate to ca_cert or ca_path.

To read more on suggestions for configuring the Consul agent, see run an agent.

consul {
  address = "consul.example.com"
  auth {}
  tls {}
  token = null
  transport {}
}

• address - (string: “localhost:8500”) Address is the address of the Consul agent. It may be an IP or FQDN.
• auth - Auth is the HTTP basic authentication for communicating with Consul.
  • enabled - (bool)
  • username - (string)
  • password - (string)
• tls - Configure TLS to use a secure client connection with Consul. Using HTTP/2 can solve issues related to hitting Consul’s maximum connection limits, as well as improve efficiency when processing many blocking queries. This option is required for Consul-Terraform-Sync when connecting to a Consul agent with TLS verification enabled for HTTPS connections.
  • enabled - (bool) Enable TLS. Providing a value for any of the TLS options will enable this parameter implicitly.
  • verify - (bool: true) Enables TLS peer verification. The default is enabled, which will check the global certificate authority (CA) chain to make sure the certificates returned by Consul are valid.
    • If Consul is using a self-signed certificate that you have not added to the global CA chain, you can set this certificate with ca_cert or ca_path. Alternatively, you can disable SSL verification by setting verify to false. However, disabling verification is a potential security vulnerability.
  • ca_cert - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the connection to Consul over TLS. Can also be provided through the CONSUL_CACERT environment variable.
  • ca_path - (string) The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the connection to Consul over TLS. Can also be provided through the CONSUL_CAPATH environment variable.
  • cert - (string) The path to a PEM-encoded client certificate file provided to Consul over TLS in order for Consul to verify the authenticity of the connection from CTS. Required if Consul has verify_incoming set to true. Can also be provided through the CONSUL_CLIENT_CERT environment variable.
  • key - (string) The path to the PEM-encoded private key file used with the client certificate configured by cert. Required if Consul has verify_incoming set to true. Can also be provided through the CONSUL_CLIENT_KEY environment variable.
  • server_name - (string) The server name to use as the Server Name Indication (SNI) for Consul when connecting via TLS. Can also be provided through the CONSUL_TLS_SERVER_NAME environment variable.
• token - (string) The ACL token to use for client communication with the local Consul agent. The token can also be provided through the CONSUL_TOKEN or CONSUL_HTTP_TOKEN environment variables. More information on the required privileges required by Consul-Terraform-Sync are available in the Secure Consul-Terraform-Sync for Production tutorial
• transport - Transport configures the low-level network connection details.
  • dial_keep_alive - (string: “30s”) The amount of time for keep-alives.
  • dial_timeout - (string: “30s”) The amount of time to wait to establish a connection.
  • disable_keep_alives - (bool) Determines if keep-alives should be used. Disabling this significantly decreases performance.
  • idle_conn_timeout - (string: “5s”) The timeout for idle connections.
  • max_idle_conns - (int: 0) The maximum number of total idle connections across all hosts. The limit is disabled by default.
  • max_idle_conns_per_host - (int: 100) The maximum number of idle connections per remote host. The majority of connections are established with one host, the Consul agent.
    • To achieve the shortest latency between a Consul service update to a task execution, configure max_idle_conns_per_host equal to or greater than the number of services in automation across all tasks.
    • This value should be lower than the configured http_max_conns_per_client for the Consul agent. If max_idle_conns_per_host and the number of services in automation is greater than the Consul agent limit, Consul-Terraform-Sync may error due to connection limits (status code 429). You may increase the agent limit with caution. Note: requests to the Consul agent made by Terraform subprocesses or any other process on the same host as Consul-Terraform-Sync will contribute to the Consul agent connection limit.
  • tls_handshake_timeout - (string: “10s”) amount of time to wait to complete the TLS handshake.

Service

A service block is an optional block to explicitly define configuration of services that Consul-Terraform-Sync monitors. A service block is only necessary for services that have non-default values e.g. custom datacenter. Services that do not have a service block configured will assume default values. To configure multiple services, specify multiple service blocks. For services to be included in task automation, the service must be included in the task.services field of a task block. If a service block is configured, the service can be referred in task.services by service name or ID. If a service block is not configured, it can only be referred to by service name.

service {
  name        = "web"
  datacenter  = "dc1"
  description = "all instances of the service web in datacenter dc1"
}

• datacenter - (string) The datacenter the service is deployed in.
• description - (string) The human readable text to describe the service.
• id - (string) ID identifies the service for Consul-Terraform-Sync. This is used to explicitly identify the service config for a task to use. If no ID is provided, the service is identified by the service name within a task definition.
• name - (string: required) The Consul logical name of the service (required).

• namespace

  Enterprise

  - (string: “default”) The namespace of the service. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token, or default to the default namespace.

• filter - (string) Specifies the expression used to filter nodes for the service. For more details on supported filters, see the Consul documentation on filtering service nodes.
• cts_user_defined_meta - (map[string]) User-defined metadata is a map of strings that will be appended to the service input variable for compatible Terraform modules. Not all modules may use this value. To determine if your task uses metadata or what the expected keys and format are, reference documentation for the module(s) configured for your tasks.
  • If multiple tasks depend on the same service but require different metadata, you can declare different sets of metadata for the same service. Define multiple service blocks for the service with unique IDs (and identical names) for those blocks. The metadata can then be separated per task based on the service IDs.

Task

A task block configures which task to execute in automation. When the task should execute can be determined by the service block and condition block. The task block may be specified multiple times to configure multiple tasks.

task {
  name           = "taskA"
  description    = ""
  enabled        = true,
  providers      = []
  services       = ["web", "api"]
  source         = "org/example/module"
  version        = "1.0.0"
  variable_files = []
  condition "catalog-services" {
    regexp = ".*"
  }
}

• description - (string) The human readable text to describe the service.

• name - (string: required) Name is the unique name of the task (required). A task name must start with a letter or underscore and may contain only letters, digits, underscores, and dashes.

• enabled - (bool: true) Enable or disable a task from running and managing resources.

• providers - (list[string]) Providers is the list of provider names the task is dependent on. This is used to map Terraform provider configuration to the task.

• services - (list[string]) Required depending on condition configuration. Services is the list of logical service names or service IDs the task executes on. Consul-Terraform-Sync monitors the Consul Catalog for changes to these services and triggers the task to run. Any service value not explicitly defined by a service block with a matching ID is assumed to be a logical service name in the default namespace. Alternative to configuring services, a condition can be configured so that the task does not trigger on changes to services (default behavior) but instead trigger on a different condition. See Task Condition configuration for more details.

• source - (string: required) Source is the location the driver uses to discover the Terraform module used for automation. The source is the module path which can be local or remote on the Terraform Registry or private module registry. Read more on Terraform module source and other supported types here.

  • To use a private module with the terraform driver, run the command terraform login [hostname] to authenticate the local Terraform CLI prior to starting Consul-Terraform-Sync.

  • To use a private module with the terraform_cloud driver, no extra steps are needed.

    // local module example: "./terraform-cts-hello"
    source = "<PATH>"
    // public module example: "mkam/hello/cts"
    source = "<NAMESPACE>/<MODULE NAME>/<PROVIDER>"
    // private module example: "my.tfe.hostname.io/my-org/hello/cts"
    source = "<HOSTNAME>/<ORGANIZATION>/<MODULE NAME>/<PROVIDER>"

• variable_files - (list[string]) Specifies list of paths to Terraform variable definition files (.tfvars). The content of these files should consist of only variable name assignments. The variable assignments must match the corresponding variable declarations made available by the Terraform module for the task.

  • Variables are loaded in the order they appear in the files. Duplicate variables are overwritten with the later value. Unless specified by the module, configure arguments for Terraform providers using terraform_provider blocks.

    address_group = "consul-services"
    tags = [
      "consul-terraform-sync",
      "terraform"
    ]

    

    example.tfvars

• version - (string) The version of the provided source the task will use. For the Terraform driver, this is the module version. The latest version will be used as the default if omitted.

• working_dir - (string) The working directory to manage generated artifacts by Consul-Terraform-Sync for this task, including Terraform configuration files. By default, a working directory is created for each task as a subdirectory in the base working_dir, e.g. sync-tasks/task-name.

• buffer_period - Configures the buffer period for a dynamic task to dampen the effects of flapping services to downstream network devices. It defines the minimum and maximum amount of time to wait for the cluster to reach a consistent state and accumulate changes before triggering task execution. The default is inherited from the top level buffer_period block. If configured, these values will take precedence over the global buffer period. This is useful to enable for a task that is dependent on services that have a lot of flapping. Buffer periods do not apply to scheduled tasks.

  • enabled - (bool) Enable or disable buffer periods for this task. Specifying min will also enable it.
  • min - (string: “5s”) The minimum period of time to wait after changes are detected before triggering related tasks.
  • max - (string: “20s”) The maximum period of time to wait after changes are detected before triggering related tasks. If min is set, the default period for max is 4 times the value of min.

• condition - (obj) The requirement that, when met, triggers Consul-Terraform-Sync to execute the task. When unconfigured, the default condition is to trigger the task on changes in the services configured in services. Only one condition may be configured per task. Consul-Terraform-Sync supports different types of conditions, which each have their own configuration options. See Task Condition configuration for full details on configuration options for each condition type.

• source_input - (obj) Specifies a Consul object containing values or metadata to be provided to the Terraform Module. When source input is empty, source input will be determined by the condition or services list. Only one source_input may be configured per task. Consul-Terraform-Sync supports different types of source input, each source input has their own configuration options. The source input block is currently only supported with schedule condition. See Task Source Input configuration for full details on configuration options for each source input type.

• terraform_version - (string)

  Enterprise

  The version of Terraform to use for the Terraform Cloud workspace associated with the task. Defaults to the latest compatible version supported by the organization. This option is only available when used with the Terraform Cloud driver; otherwise, set the version within the Terraform driver.

Task Condition

A task block can be optionally configured with a condition block to set the conditions that should be met in order to execute that particular task. Below are the different types of conditions that Consul-Terraform-Sync supports.

Services Condition

This is the default type of condition when no condition block is configured for a task. This condition will trigger the task on either changes in services configured in task.services or changes in services that match the regular expression configured in regexp. The default behavior is to trigger on changes to task.services. The task.services option is required if regexp is not configured.

See Task Execution: Services Condition for more details on how tasks are triggered with a services condition.

task {
  name        = "services_condition_task"
  description = "execute on changes to services with names starting with web"
  providers   = ["my-provider"]
  source      = "path/to/services-condition-module"
  condition "services" {
    regexp = "^web.*"
  }
}

• regexp - (beta) (string) Only services that have a name which matches the regular expression are used by the task. If regexp is configured, then task.services must be omitted or empty. If both a list and a regex are needed, consider including the list as part of the regex or creating separate tasks.

Catalog-Services Condition

A catalog-services condition block configures a task to only execute on service registration and deregistration, more specifically on first service instance registration and last service instance deregistration respectively. The catalog-services condition has additional configuration options to specify the services that can trigger the task on registration and deregistration.

See Task Execution: Catalog Services Condition for more information on how tasks are triggered with a catalog-services condition.

task {
  name        = "catalog_service_condition_task"
  description = "execute on service de/registrations with name matching 'web.*'"
  source      = "path/to/catalog-services-module"
  providers   = ["my-provider"]
  // configure depending on module. provides detailed information for these
  // services but does not execute task. refer to module docs on how to configure.
  services = ["web-api"]
  condition "catalog-services" {
    datacenter          = "dc1"
    namespace           = "default"
    regexp              = "web.*"
    source_includes_var = false
    node_meta {
      key = "value"
    }
  }
}

• datacenter - (string) The datacenter of the services to query for the task. If not provided, the datacenter will default to the datacenter of the agent that Consul-Terraform-Sync queries.

• namespace

  Enterprise

  - (string) The namespace of the services to query for the task. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token or default to the default namespace.

• node_meta - (map[string]) The node metadata key/value pairs used to filter services. Only services registered at a node with the specified key/value pairs are used by the task.
• regexp - (string) Optional if task.services is configured. Either regexp or task.services or both must be configured. Only services that have a name which matches the regular expression are used by the task. If not provided, regexp will default to an exact match on task.services list. For example, if task.services = ["api", "web"], then regexp will default to ^api$|^web$. See Task Execution: Catalog Services Condition for more details on the relationship between task.services and regexp. Some resources for more information on regular expressions: regular expression syntax, try out regular expression string matching.
• source_includes_var - (bool: false) Whether or not the module configured at task.source includes the catalog_services variable. Please refer to the documentation of the selected module for guidance on how to configure this field. If configured inconsistently with the module, Consul-Terraform-Sync will error and exit.

Consul KV Condition

A consul-kv condition block configures a task to only execute on changes to a Consul KV entry. The condition can be configured for a single Consul KV entry or for any Consul KV entries that are prefixed with a given path.

See Task Execution: Consul KV Condition for more information on how tasks are triggered with a consul-kv condition.

task {
  name        = "consul_kv_condition_task"
  description = "execute on changes to Consul KV entry"
  source      = "path/to/consul-kv-module"
  providers   = ["my-provider"]
  services    = ["web-api"]
  condition "consul-kv" {
    path                = "my-key"
    recurse             = false
    datacenter          = "dc1"
    namespace           = "default"
    source_includes_var = false
  }
}

• path - (string: required) The path of the key that is used by the task.
• recurse - (bool: false) Setting to true instructs Consul-Terraform-Sync to treat the path as a prefix instead of a literal match.
• datacenter - (string) The datacenter of the services to query for the task. If not provided, the datacenter will default to the datacenter of the agent that Consul-Terraform-Sync queries.

• namespace

  Enterprise

  - (string) The namespace of the services to query for the task. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token or default to the default namespace.

• source_includes_var - (bool: false) If set to true, then Consul-Terraform-Sync will include the consul_kv variable as an input to the module specified in task.source. Refer to the documentation of the selected module for guidance on how to configure this field. If configured inconsistently with the module, Consul-Terraform-Sync will error and exit.

Schedule Condition

A scheduled task has a schedule condition block, which defines the schedule for executing the task. Unlike a dynamic task, a scheduled task does not dynamically trigger on changes in Consul.

Schedule tasks also rely on additional task configuration, separate from the condition block to determine the source input information to provide to the task module. See task.services or source_input block configuration for details on how to configure source input.

See Task Execution: Schedule Condition for more information on how tasks are triggered with schedule conditions.

See Terraform Module: Source Input for more information on source input options for a scheduled task.

task {
  name        = "scheduled_task"
  description = "execute every Monday using service information from web and db"
  services    = ["web", "db"]
  source      = "path/to/module"
  condition "schedule" {
    cron = "* * * * Mon"
  }
}

• cron - (string: required) The CRON expression that dictates the schedule to trigger the task. For more information on CRON expressions, see the cronexpr parsing library.

Task Source Input

You can add an optional source_input block to the task block. The source_input block specifies a Consul object containing values or metadata to be provided to the Terraform Module. Consul-Terraform-Sync supports the following source input types.

The source input block is currently only supported when using a schedule condition. Adding a source_input block alongside any other type of condition will result in an error. To accomplish a similar behavior with other condition blocks, use the source_includes_var field.

Services Source Input

This services source input object defines services registered to Consul whose metadata will be used as services source input to the Terraform Module. The following parameters are supported:

In the following example, the scheduled task queries all Consul services with web as the suffix. The metadata of matching services are provided to the Terraform module.

task {
  name        = "schedule_condition_task"
  description = "execute every Monday using information from service names starting with web"
  source      = "path/to/module"
  condition "schedule" {
    cron = "* * * * Mon"
  }
  source_input “services” {
    regexp = "^web.*"
  }
}

Consul KV Source Input

A Consul KV source input block defines changes to Consul KV that will be monitored. These changes will then be provided as Consul KV source input to the Terraform Module. The source input can be configured for a single Consul KV entry or for any Consul KV entries that are prefixed with a given path. The following parameters are supported:

In the following example, the scheduled task queries datacenter dc1 in the default namespace for changes to the value held by the key my-key.

task {
  name        = "schedule_condition_task_kv"
  description = "execute every Monday using information from Consul KV entry my-key"
  source      =  "path/to/module"
  condition "schedule" {
    cron = "* * * * Mon"
  }
  source_input "consul-kv" {
    path       = "my-key"
    recurse    = false
    datacenter = "dc1"
    namespace  = "default"
  }
}

Network Drivers

A driver is required for Consul-Terraform-Sync to propagate network infrastructure change. The driver block configures the subprocess that Consul-Terraform-Sync runs in automation. The default driver is the Terraform driver which automates Terraform as a local installation of the Terraform CLI.

Only one network driver can be configured per deployment of Consul-Terraform-Sync.

Terraform Driver

The Terraform driver block is used to configure Consul-Terraform-Sync for installing and automating Terraform locally. The driver block supports Terraform configuration to specify the backend used for state management and required_providers configuration used for provider discovery.

driver "terraform" {
  log         = false
  persist_log = false
  path        = ""
  backend "consul" {
    gzip = true
  }
  required_providers {
    myprovider = {
      source  = "namespace/myprovider"
      version = "1.3.0"
    }
  }
}

• backend - (obj) The backend stores Terraform state files for each task. This option is similar to the Terraform backend configuration. Consul-Terraform-Sync supports Terraform backends used as a state store.
  • Supported backend options: azurerm, consul, cos, gcs, kubernetes, local, manta, pg (Terraform v0.14+), s3. Visit the Terraform documentation links for details on backend configuration options.
  • If omitted, Consul-Terraform-Sync will generate default values and use configurations from the consul block to configure Consul as the backend, which stores Terraform statefiles in the Consul KV. The ACL token provided for Consul authentication is used to read and write to the KV store and requires Consul KV privileges. The Consul KV path is the base path to store state files for tasks. The full path of each state file will have the task identifier appended to the end of the path, e.g. consul-terraform-sync/terraform-env:task-name.
  • The remote enhanced backend is not supported with the Terraform driver to run operations in Terraform Cloud. Use the Terraform Cloud driver to integrate Consul-Terraform-Sync with Terraform Cloud for remote workspaces and remote operations.
• log - (bool) Enable all Terraform output (stderr and stdout) to be included in the Consul-Terraform-Sync log. This is useful for debugging and development purposes. It may be difficult to work with log aggregators that expect uniform log format.
• path - (string) The file path to install Terraform or discover an existing Terraform binary. If omitted, Terraform will be installed in the same directory as the Consul-Terraform-Sync daemon. To resolve an incompatible Terraform version or to change versions will require removing the existing binary or change to a different path.
• persist_log - (bool) Enable trace logging for each Terraform client to disk per task. This is equivalent to setting TF_LOG_PATH=<work_dir>/terraform.log. Trace log level results in verbose logging and may be useful for debugging and development purposes. We do not recommend enabling this for production. There is no log rotation and may quickly result in large files.
• required_providers - (obj: required) Declare each Terraform provider used across all tasks. This can be configured the same as how you would configure Terraform terraform.required_providers field to specify the source and version for each provider. Consul-Terraform-Sync will process these requirements when preparing each task that uses the provider.
• version - (string) The Terraform version to install and run in automation for task execution. If omitted, the driver will install the latest compatible release of Terraform. To change versions, remove the existing binary or change the path to install the desired version. Verify that the desired Terraform version is compatible across all Terraform modules used for Consul-Terraform-Sync automation.
• working_dir - (string: “sync-tasks”) Deprecated in Consul-Terraform-Sync 0.3.0, use the global working_dir option instead. The base working directory to manage Terraform configurations for all tasks. The full path of each working directory will have the task identifier appended to the end of the path, e.g. ./sync-tasks/task-name.

Terraform Cloud Driver

Enterprise

This feature requires Consul-Terraform-Sync Enterprise which is available with Consul Enterprise.

The Terraform Cloud driver enables Consul-Terraform-Sync Enterprise to integrate with Terraform Cloud, including both the self-hosted distribution and the managed service. With this driver, Consul-Terraform-Sync automates Terraform runs and remote operations for workspaces.

An overview of features enabled with Terraform Cloud can be viewed within the Network Drivers documentation.

Only one network driver can be configured per deployment of Consul-Terraform-Sync.

driver "terraform-cloud" {
  hostname     = "https://app.terraform.io"
  organization = "my-org"
  token        = "<TEAM_TOKEN>"
  // Optionally set the token to be securely queried from Vault instead of
  // written directly to the configuration file.
  // token = "{{ with secret \"secret/my/path\" }}{{ .Data.data.foo }}{{ end }}"
  workspaces {
    tags           = ["source:cts"]
    tags_allowlist = []
    tags_denylist  = []
  }
  required_providers {
    myprovider = {
      source  = "namespace/myprovider"
      version = "1.3.0"
    }
  }
}

• hostname - (string) The Terraform Cloud hostname to connect to. Can be overridden with the TFC_HOSTNAME environment variable.
• organization - (string) The Terraform Cloud organization that hosts the managed workspaces by Consul-Terraform-Sync. Can be overridden with the TFC_ORGANIZATION environment variable.
• token - (string) Required Team API token used for authentication with Terraform Cloud and workspace management. Only workspace permissions are needed for Consul-Terraform-Sync. The token can also be provided using the TFC_TOKEN environment variable.
  • We recommend creating a dedicated team and team API token to isolate automation by Consul-Terraform-Sync from other Terraform Cloud operations.
• workspace_prefix - (string) Specifies a prefix to prepend to the automatically-generated workspace names used for automation. This prefix will be used by all tasks that use this driver. By default, when no prefix is configured, the workspace name will be the task name. When a prefix is configured, the workspace name will be <workspace_prefix value><task name>. For example, if you configure the prefix as “cts_“, then a task with the name “task_firewall” will have the workspace name “cts_task_firewall”.
• workspaces - Configure Consul-Terraform-Sync management of Terraform Cloud workspaces.
  • tags - (list[string]) Tags for CTS to add to all automated workspaces when the workspace is first created or discovered. Tags are added to discovered workspaces only if the workspace meets automation requirements and satisfies the allowlist and denylist tag options. This option will not affect existing tags. Tags that were manually removed during runtime will be re-tagged when CTS restarts. Compatible with Terraform Cloud and Terraform Enterprise v202108-1+
  • tags_allowlist - (list[string]) Tag requirement to use as a provision check for CTS automation of workspaces. When configured, Terraform Cloud workspaces must have at least one tag from the allow list for CTS to automate the workspace and runs. Compatible with Terraform Cloud and Terraform Enterprise v202108-1+.
  • tags_denylist - (list[string]) Tag restriction to use as a provision check for CTS automation of workspaces. When configured, Terraform Cloud workspaces must not have any tag from the deny list for CTS to automate the workspace and runs. Denied tags have higher priority than tags set in the tags_allowlist option. Compatible with Terraform Cloud and Terraform Enterprise v202108-1+.
• required_providers - (obj: required) Declare each Terraform provider used across all tasks. This can be configured the same as how you would configure Terraform terraform.required_providers field to specify the source and version for each provider. Consul-Terraform-Sync will process these requirements when preparing each task that uses the provider.

• tls - Configure TLS to allow HTTPS connections to Terraform Enterprise.

  • enabled - (bool) Enable TLS. Providing a value for any of the TLS options will enable this parameter implicitly.
  • ca_cert - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the connection to Terraform Enterprise over TLS.
  • ca_path - (string) The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the connection to Terraform Enterprise over TLS.
  • cert - (string) The path to a PEM-encoded client certificate file provided to Terraform Enterprise over TLS in order for Terraform Enterprise to verify the authenticity of the connection from CTS.
  • key - (string) The path to the PEM-encoded private key file used with the client certificate configured by cert for communicating with Terraform Enterprise over TLS.
  • server_name - (string) The server name to use as the Server Name Indication (SNI) for Terraform Enterprise when connecting via TLS.

  • verify - (bool: true) Enables TLS peer verification. The default is enabled, which will check the global certificate authority (CA) chain to make sure the certificates returned by Terraform Enterprise are valid.

    • If Terraform Enterprise is using a self-signed certificate that you have not added to the global CA chain, you can set this certificate with ca_cert or ca_path. Alternatively, you can disable SSL verification by setting verify to false. However, disabling verification is a potential security vulnerability.

      tls {
        verify = false
      }

Consul-Terraform-Sync generates local artifacts to prepare configuration versions used for workspace runs. The location of the files created can be set with the working_dir option or configured per task. When a task is configured with a local module and is run with the Terraform Cloud driver, the local module is copied and uploaded as a part of the configuration version.

The version of Terraform to use for each workspace can also be set within the task configuration.

Terraform Provider

A terraform_provider block configures the options to interface with network infrastructure. Define a block for each provider required by the set of Terraform modules across all tasks. This block resembles provider blocks for Terraform configuration. To find details on how to configure a provider, refer to the corresponding documentation for the Terraform provider. The main directory of publicly available providers are hosted on the Terraform Registry.

The below configuration captures the general design of defining a provider using the AWS Terraform provider as an example.

driver "terraform" {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.33.0"
    }
  }
}
terraform_provider "aws" {
  // Configuration options
  region = "us-east-1"
}
task {
  source    = "some/source"
  providers = ["aws"]
  services  = ["web", "api"]
}

Note: Provider arguments configured in Consul-Terraform-Sync configuration files are written in plain text to the generated terraform.tfvars file for each Terraform workspace that references the provider. To exclude arguments or dynamic values from rendering to local files in plain text, use task_env in addition to using dynamic configuration.

Securely Configure Terraform Providers

The terraform_provider block supports dynamically loading arguments and the local environment from other sources. This can be used to securely configure your Terraform provider from the shell environment, Consul KV, or Vault. Using the task_env meta-argument and template syntax below, you can avoid exposing sensitive values or credentials in plain text within configuration files for Consul-Terraform-Sync.

task_env and the template syntax for dynamic values are only supported within the terraform_provider block.

Provider Environment Variables

Terraform providers may support shell environment variables as values for some of their arguments. When available, we recommend using environment variables as a way to keep credentials out of plain-text configuration files. Refer to the official provider docs hosted on the Terraform Registry to find supported environment variables for a provider. By default, Consul-Terraform-Sync enables all Terraform workspaces to inherit from its environment.

The task_env block is a meta-argument available for the terraform_provider block that can be used to rename or scope the available environment to a selected set of variables. Passing sensitive values as environment variables will scope the values to only the tasks that require the provider.

terraform_provider "foo" {
  // Direct assignment of provider arguments are rendered in plain-text within
  // the Consul-Terraform-Sync configuration and the generated terraform.tfvars
  // file for the corresponding Terraform workspaces.
  // token = "<token value>"
  // Instead of configuring the token argument directly for the provider,
  // use the provider's supported environment variable for the token argument.
  // For example,
  // $ export FOO_TOKEN = "<token value>"
  // Dynamically assign the task's environment from the shell env, Consul KV,
  // Vault.
  task_env {
    "FOO_TOKEN" = "{{ env \"CTS_FOO_TOKEN\" }}"
  }
}

Security note: Consul-Terraform-Sync does not prevent sensitive values from being written to Terraform state files. We recommend securing state files in addition to securely configuring Terraform providers. Options for securing state files can be set within driver.backend based on the backend used. For example, Consul KV is the default backend and can be secured with ACLs for KV path. For other backends, we recommend enabling encryption, if applicable.

Load Dynamic Values

Load dynamic values for Terraform providers with integrated template syntax.

Env

env reads the given environment variable accessible to Consul-Terraform-Sync.

terraform_provider "example" {
  address = "{{ env \"EXAMPLE_HOSTNAME\" }}"
}

Consul

key queries the key’s value in the KV store of the Consul server configured in the required consul block.

terraform_provider "example" {
  value = "{{ key \"path/example/key\" }}"
}

Vault

with secret queries the Vault KV secrets engine. Vault is an optional source that require operators to configure the Vault client with a vault block. Access the secret using template dot notation Data.data.<secret_key>.

vault {
  address = "vault.example.com"
}
terraform_provider "example" {
  token = "{{ with secret \"secret/my/path\" }}{{ .Data.data.foo }}{{ end }}"
}

Vault Configuration

• address - (string) The URI of the Vault server. This can also be set via the VAULT_ADDR environment variable.
• enabled - (bool) Enabled controls whether the Vault integration is active.
• namespace - (string) Namespace is the Vault namespace to use for reading secrets. This can also be set via the VAULT_NAMESPACE environment variable.
• renew_token - (bool) Renews the Vault token. This can also be set via the VAULT_RENEW_TOKEN environment variable.
• tls - (tls block) TLS indicates the client should use a secure connection while talking to Vault. Supports the environment variables:
  • VAULT_CACERT
  • VAULT_CAPATH
  • VAULT_CLIENT_CERT
  • VAULT_CLIENT_KEY
  • VAULT_SKIP_VERIFY
  • VAULT_TLS_SERVER_NAME
• token - (string) Token is the Vault token to communicate with for requests. It may be a wrapped token or a real token. This can also be set via the VAULT_TOKEN environment variable, or via the VaultAgentTokenFile.
• vault_agent_token_file - (string) The path of the file that contains a Vault Agent token. If this is specified, Consul-Terraform-Sync will not try to renew the Vault token.
• transport - (transport block) Transport configures the low-level network connection details.
• unwrap_token - (bool) Unwraps the provided Vault token as a wrapped token.

Note: Vault credentials are not accessible by tasks and the associated Terraform configurations, including automated Terraform modules. If the task requires Vault, you will need to separately configure the Vault provider and explicitly include it in the task.providers list.

Multiple Provider Configurations

Consul-Terraform-Sync supports the Terraform feature to define multiple configurations for the same provider by utilizing the alias meta-argument. Define multiple provider blocks with the same provider name and set the alias to a unique value across a given provider. Select which provider configuration to use for a task by specifying the configuration with the provider name and alias (<name>.<alias>) within the list of providers in the task.provider parameter. A task can use multiple providers, but only one provider instance of a provider is allowed per task.

The example Consul-Terraform-Sync configuration below defines two similar tasks executing the same module with different instances of the AWS provider.

terraform_provider "aws" {
  alias   = "a"
  profile = "team-a"
  task_env {
    "AWS_ACCESS_KEY_ID" = "{{ env \"CTS_AWS_ACCESS_KEY_ID_A\" }}"
  }
}
terraform_provider "aws" {
  alias   = "b"
  profile = "team-b"
  task_env {
    "AWS_ACCESS_KEY_ID" = "{{ env \"CTS_AWS_ACCESS_KEY_ID_B\" }}"
  }
}
terraform_provider "dns" {
  // ...
}
task {
  name      = "task-a"
  source    = "org/module"
  providers = ["aws.a", "dns"]
  // ...
}
task {
  name      = "task-b"
  source    = "org/module"
  providers = ["aws.b", "dns"]
  // ...
}