Extend Cloud Resources

To enable end users to provision and consume cloud resources, platform engineers need to prepare ComponentDefinitions for cloud resources if end users’ requirements are beyond the built-in capabilities.

Here is the guide to create Terraform typed ComponentDefinitions of cloud resources for cloud providers Alibaba Cloud, AWS and Azure.

Prerequisites

Create a Terraform resource or module for a cloud resource.

For example, we created a Terraform resource for AWS S3 bucket, and stored it in a local file named aws_s3_bucket.tf.

  1. resource "aws_s3_bucket" "bucket-acl" {
  2. bucket = var.bucket
  3. acl = var.acl
  4. }
  5. output "BUCKET_NAME" {
  6. value = aws_s3_bucket.bucket-acl.bucket_domain_name
  7. }
  8. variable "bucket" {
  9. description = "S3 bucket name"
  10. default = "vela-website"
  11. type = string
  12. }
  13. variable "acl" {
  14. description = "S3 bucket ACL"
  15. default = "private"
  16. type = string
  17. }

We also created a Terraform module for Alibaba Cloud EIP, and stored it in GitHub repository https://github.com/oam-dev/terraform-alibaba-eip.git.

Please make sure this module is fine and ready for use. You’re recommended to debug these modules first with terraform command line and make sure they can work with parameters that should be shared with your end users.

By running vela def init command, we can generate a ComponentDefinition for a cloud resource based on Terraform resource or module either from a local file, or from a remote GitHub repository.

  1. $ vela def init -h
  2. --git string Specify which git repository the configuration(HCL) is stored in. Valid when --provider/-p is set.
  3. --local string Specify the local path of the configuration(HCL) file. Valid when --provider/-p is set.

We use --local to accept Terraform resource or module from a local file to generate a ComponentDefinition.

  1. $ vela def init s3 --type component --provider aws --desc "Terraform configuration for AWS S3" --local aws_s3_bucket.tf
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: ComponentDefinition
  4. metadata:
  5. annotations:
  6. definition.oam.dev/description: Terraform configuration for AWS S3
  7. creationTimestamp: null
  8. labels:
  9. type: terraform
  10. name: aws-s3
  11. namespace: vela-system
  12. spec:
  13. schematic:
  14. terraform:
  15. configuration: |
  16. resource "aws_s3_bucket" "bucket-acl" {
  17. bucket = var.bucket
  18. acl = var.acl
  19. }
  20. output "BUCKET_NAME" {
  21. value = aws_s3_bucket.bucket-acl.bucket_domain_name
  22. }
  23. variable "bucket" {
  24. description = "S3 bucket name"
  25. default = "vela-website"
  26. type = string
  27. }
  28. variable "acl" {
  29. description = "S3 bucket ACL"
  30. default = "private"
  31. type = string
  32. }
  33. workload:
  34. definition:
  35. apiVersion: terraform.core.oam.dev/v1beta1
  36. kind: Configuration
  37. status: {}

We use --git to accept Terraform module or resource from a remote GitHub repository to generate a ComponentDefinition.

  1. $ vela def init eip --type component --provider alibaba --desc "Terraform configuration for Alibaba Cloud Elastic IP" --git https://github.com/oam-dev/terraform-alibaba-eip.git
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: ComponentDefinition
  4. metadata:
  5. annotations:
  6. definition.oam.dev/description: Terraform configuration for Alibaba Cloud Elastic
  7. IP
  8. creationTimestamp: null
  9. labels:
  10. type: terraform
  11. name: alibaba-eip
  12. namespace: vela-system
  13. spec:
  14. schematic:
  15. terraform:
  16. configuration: https://github.com/oam-dev/terraform-alibaba-eip.git
  17. type: remote
  18. workload:
  19. definition:
  20. apiVersion: terraform.core.oam.dev/v1beta1
  21. kind: Configuration
  22. status: {}

You are warmly welcome to contribute this extended cloud resource ComponentDefinition to oam-dev/catalog.

Write the generated ComponentDefinition into a file named terraform-<ComponentDefinition_NAME>.yaml and apply it to the running Kubernetes cluster.

  1. kubectl apply -f <FILENAME>

You can quickly verify the ComponentDefinition by command vela show. It may be a bit slow if you’re loading terraform module from remote git server for the first time. After that, vela will cache the data in $HOME/.vela/terraform folder. You may need to clean up the cached data from this folder if you want to update the module.

  1. $ vela show alibaba-eip
  2. # Properties
  3. +----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+
  4. | NAME | DESCRIPTION | TYPE | REQUIRED | DEFAULT |
  5. +----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+
  6. | name | Name to be used on all resources as prefix. Default to 'TF-Module-EIP'. | string | true | |
  7. | bandwidth | Maximum bandwidth to the elastic public network, measured in Mbps (Mega bit per second). | number | true | |
  8. | writeConnectionSecretToRef | The secret which the cloud resource connection will be written to | [writeConnectionSecretToRef](#writeConnectionSecretToRef) | false | |
  9. +----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+
  10. ## writeConnectionSecretToRef
  11. +-----------+-----------------------------------------------------------------------------+--------+----------+---------+
  12. | NAME | DESCRIPTION | TYPE | REQUIRED | DEFAULT |
  13. +-----------+-----------------------------------------------------------------------------+--------+----------+---------+
  14. | name | The secret name which the cloud resource connection will be written to | string | true | |
  15. | namespace | The secret namespace which the cloud resource connection will be written to | string | false | |
  16. +-----------+-----------------------------------------------------------------------------+--------+----------+---------+

If the tables display, the ComponentDefinition should work.

The end user will be able to use the component in an application like below, he must follow the spec as vela show:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4. name: cloud-resource-example
  5. spec:
  6. components:
  7. - name: sample-eip
  8. type: alibaba-eip
  9. properties:
  10. name: "my-eip"
  11. bandwidth: 1
  12. writeConnectionSecretToRef:
  13. name: outputs-eip

Generally, the values in the properties are aligned with the variables defined in the terraform module, the terraform controller will help run terraform init, terraform plan automatically and finally use the parameters defined in the properties of application component to run terraform apply.

You can refer to scenario docs such as Provision cloud resources for more real use cases.

You are encouraged to generate the documentation for your ComponentDefinition and submit it to KubeVela official site.

By running vela def doc-gen command, we can generate related docs for the ComponentDefinition either from a local file, or from the running application on kubernetes cluster.

If you choose to generate docs from running cluster, you should provide the name of the ComponentDefinition and the namespace which it is running in.

  1. $ vela def doc-gen alibaba-eip --path ./kubevela.io/docs/end-user/components/cloud-services/terraform/
  2. Generated docs for alibaba-eip in ./kubevela.io/docs/end-user/components/cloud-services/terraform/alibaba-eip.md

If you choose to generate docs from local ComponentDefinition file, you should provide the file path.

  1. $ vela def doc-gen alibaba-eip.yaml

Move the file generated to kubevela/kubevela.io repo. Follow the contribution guide to submit the doc.

Last updated on Aug 4, 2023 by Daniel Higuero