Extend Cloud Resources

To enable end users to provision and consume cloud resources, platform engineers need to prepare ComponentDefinitions for cloud resources if end users’ requirements are beyond the built-in capabilities.

Here is the guide to create Terraform typed ComponentDefinitions of cloud resources for cloud providers Alibaba Cloud, AWS and Azure.

Prerequisites

Create a Terraform resource or module for a cloud resource.

For example, we created a Terraform resource for AWS S3 bucket, and stored it in a local file named aws_s3_bucket.tf.

  1. resource "aws_s3_bucket" "bucket-acl" {
  2.   bucket = var.bucket
  3.   acl = var.acl
  4. }
  6. output "BUCKET_NAME" {
  7.   value = aws_s3_bucket.bucket-acl.bucket_domain_name
  8. }
  10. variable "bucket" {
  11.   description = "S3 bucket name"
  12.   default = "vela-website"
  13.   type = string
  14. }
  16. variable "acl" {
  17.   description = "S3 bucket ACL"
  18.   default = "private"
  19.   type = string
  20. }

We also created a Terraform module for Alibaba Cloud EIP, and stored it in GitHub repository https://github.com/oam-dev/terraform-alibaba-eip.git.

Please make sure this module is fine and ready for use. You’re recommended to debug these modules first with terraform command line and make sure they can work with parameters that should be shared with your end users.

By running vela def init command, we can generate a ComponentDefinition for a cloud resource based on Terraform resource or module either from a local file, or from a remote GitHub repository.

  1. $ vela def init -h
  3.       --git string             Specify which git repository the configuration(HCL) is stored in. Valid when --provider/-p is set.
  4.       --local string           Specify the local path of the configuration(HCL) file. Valid when --provider/-p is set.

We use --local to accept Terraform resource or module from a local file to generate a ComponentDefinition.

  1. $ vela def init s3 --type component --provider aws --desc "Terraform configuration for AWS S3" --local aws_s3_bucket.tf
  3. apiVersion: core.oam.dev/v1beta1
  4. kind: ComponentDefinition
  5. metadata:
  6.   annotations:
  7.     definition.oam.dev/description: Terraform configuration for AWS S3
  8.   creationTimestamp: null
  9.   labels:
  10.     type: terraform
  11.   name: aws-s3
  12.   namespace: vela-system
  13. spec:
  14.   schematic:
  15.     terraform:
  16.       configuration: |
  17.         resource "aws_s3_bucket" "bucket-acl" {
  18.           bucket = var.bucket
  19.           acl = var.acl
  20.         }
  22.         output "BUCKET_NAME" {
  23.           value = aws_s3_bucket.bucket-acl.bucket_domain_name
  24.         }
  26.         variable "bucket" {
  27.           description = "S3 bucket name"
  28.           default = "vela-website"
  29.           type = string
  30.         }
  32.         variable "acl" {
  33.           description = "S3 bucket ACL"
  34.           default = "private"
  35.           type = string
  36.         }
  37.   workload:
  38.     definition:
  39.       apiVersion: terraform.core.oam.dev/v1beta1
  40.       kind: Configuration
  41. status: {}

We use --git to accept Terraform module or resource from a remote GitHub repository to generate a ComponentDefinition.

  1. $ vela def init eip --type component --provider alibaba --desc "Terraform configuration for Alibaba Cloud Elastic IP" --git https://github.com/oam-dev/terraform-alibaba-eip.git
  3. apiVersion: core.oam.dev/v1beta1
  4. kind: ComponentDefinition
  5. metadata:
  6.   annotations:
  7.     definition.oam.dev/description: Terraform configuration for Alibaba Cloud Elastic
  8.       IP
  9.   creationTimestamp: null
  10.   labels:
  11.     type: terraform
  12.   name: alibaba-eip
  13.   namespace: vela-system
  14. spec:
  15.   schematic:
  16.     terraform:
  17.       configuration: https://github.com/oam-dev/terraform-alibaba-eip.git
  18.       type: remote
  19.   workload:
  20.     definition:
  21.       apiVersion: terraform.core.oam.dev/v1beta1
  22.       kind: Configuration
  23. status: {}

You are warmly welcome to contribute this extended cloud resource ComponentDefinition to oam-dev/catalog.

Write the generated ComponentDefinition into a file named terraform-<ComponentDefinition_NAME>.yaml and apply it to the running Kubernetes cluster.

  1. kubectl apply -f <FILENAME>

You can quickly verify the ComponentDefinition by command vela show. It may be a bit slow if you’re loading terraform module from remote git server for the first time. After that, vela will cache the data in $HOME/.vela/terraform folder. You may need to clean up the cached data from this folder if you want to update the module.

  1. $ vela show alibaba-eip
  2. # Properties
  3. +----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+
  4. |            NAME            |                                       DESCRIPTION                                        |                           TYPE                            | REQUIRED | DEFAULT |
  5. +----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+
  6. | name                       | Name to be used on all resources as prefix. Default to 'TF-Module-EIP'.                  | string                                                    | true     |         |
  7. | bandwidth                  | Maximum bandwidth to the elastic public network, measured in Mbps (Mega bit per second). | number                                                    | true     |         |
  8. | writeConnectionSecretToRef | The secret which the cloud resource connection will be written to                        | [writeConnectionSecretToRef](#writeConnectionSecretToRef) | false    |         |
  9. +----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+
  12. ## writeConnectionSecretToRef
  13. +-----------+-----------------------------------------------------------------------------+--------+----------+---------+
  14. |   NAME    |                                 DESCRIPTION                                 |  TYPE  | REQUIRED | DEFAULT |
  15. +-----------+-----------------------------------------------------------------------------+--------+----------+---------+
  16. | name      | The secret name which the cloud resource connection will be written to      | string | true     |         |
  17. | namespace | The secret namespace which the cloud resource connection will be written to | string | false    |         |
  18. +-----------+-----------------------------------------------------------------------------+--------+----------+---------+

If the tables display, the ComponentDefinition should work.

The end user will be able to use the component in an application like below, he must follow the spec as vela show:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4.   name: cloud-resource-example
  5. spec:
  6.   components:
  7.     - name: sample-eip
  8.       type: alibaba-eip
  9.       properties:
  10.         name: "my-eip"
  11.         bandwidth: 1
  12.         writeConnectionSecretToRef:
  13.           name: outputs-eip

Generally, the values in the properties are aligned with the variables defined in the terraform module, the terraform controller will help run terraform init, terraform plan automatically and finally use the parameters defined in the properties of application component to run terraform apply.

You can refer to scenario docs such as Provision cloud resources for more real use cases.

You are encouraged to generate the documentation for your ComponentDefinition and submit it to KubeVela official site.

By running vela def doc-gen command, we can generate related docs for the ComponentDefinition either from a local file, or from the running application on kubernetes cluster.

If you choose to generate docs from running cluster, you should provide the name of the ComponentDefinition and the namespace which it is running in.

  1. $ vela def doc-gen alibaba-eip --path ./kubevela.io/docs/end-user/components/cloud-services/terraform/
  2. Generated docs for alibaba-eip in ./kubevela.io/docs/end-user/components/cloud-services/terraform/alibaba-eip.md

If you choose to generate docs from local ComponentDefinition file, you should provide the file path.

  1. $ vela def doc-gen alibaba-eip.yaml

Move the file generated to kubevela/kubevela.io repo. Follow the contribution guide to submit the doc.

Last updated on Aug 4, 2023 by Daniel Higuero