Version: v1.8

GitOps with FluxCD

In this section, we will introduce how to use KubeVela to deliver in GitOps mode.

Please make sure you have enabled the fluxcd addon.

GitOps will automatically synchronize the configuration in the repository to the cluster. First, we need a repository that stores all the configuration files you need: such as some Kubernetes native resources Deployment, Secret, ConfigMap, etc. Also, you can store KubeVela’s Application in the repository.

Suppose in our repository, there is a folder called infrastructure, which has a KubeVela Application called server and a ConfigMap called server-config.

The directory structure of the repository is as follows:

  1. ├── infrastructure
  2.     ├── server-config.yaml
  3.     └── server.yaml

Deploy the following KubeVela GitOps application:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4.   name: infra-gitops
  5. spec:
  6.   components:
  7.   - name: database-config
  8.     type: kustomize
  9.     properties:
  10.       repoType: git
  11.       # replace it with your repo url
  12.       url: https://github.com/FogDong/KubeVela-GitOps-Infra-Demo
  13.       # replace it with your git secret if it's a private repo
  14.       # secretRef: git-secret
  15.       # the pull interval time, set to 10m since the infrastructure is steady
  16.       pullInterval: 10m
  17.       git:
  18.         # the branch name
  19.         branch: infra
  20.       # the path to sync
  21.       path: ./infrastructure

Check the status of this GitOps application:

  1. $ vela status infra-gitops
  2. About:
  4.   Name:         infra-gitops
  5.   Namespace:    default
  6.   Created at:   2022-06-30 14:52:33 +0800 CST
  7.   Status:       running
  9. Workflow:
  11.   mode: DAG
  12.   finished: true
  13.   Suspend: false
  14.   Terminated: false
  15.   Steps
  16.   - id:dgatat8jag
  17.     name:database-config
  18.     type:apply-component
  19.     phase:succeeded
  20.     message:
  22. Services:
  24.   - Name: database-config
  25.     Cluster: local  Namespace: default
  26.     Type: kustomize
  27.     Healthy
  28.     No trait applied

As you can see, the GitOps application is running successfully. At this point, the application continuously pulls the configuration from the repository and syncs across the cluster at 10-minute intervals.

Looking at the resources in the cluster, you can find that the server Application and the server-config ConfigMap have been automatically deployed.

  1. $ vela ls
  2. APP             COMPONENT       TYPE        TRAITS  PHASE   HEALTHY STATUS      CREATED-TIME
  3. infra-gitops    database-config kustomize           running healthy             2022-06-30 14:52:33 +0800 CST
  4. server          server          webservice          running healthy Ready:1/1   2022-06-30 14:52:35 +0800 CST
  6. $ kubectl get configmap
  7. NAME                                             DATA   AGE
  8. server-config                                    1      2m58s

GitOps can also watch your image repository, get the latest image version, and update the configuration in your code repository with the latest version, so as to achieve the purpose of automatically updating the image.

Suppose the directory structure of our repository is as follows:

  1. ├── application
  2.     └── my-app.yaml

The my-app.yaml in the application is as follows:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4.   name: my-app
  5.   namespace: default
  6. spec:
  7.   components:
  8.     - name: my-app
  9.       type: webservice
  10.       properties:
  11.         image: nginx # {"$imagepolicy": "default:image-gitops"}

Note that there is a # {"$imagepolicy": "default:image-gitops"} comment after the image field. KubeVela will update the corresponding image field through this annotation. default:image-gitops is the namespace and name of the GitOps application component we will deploy.

Deploy the following KubeVela GitOps application:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4.   name: image-gitops
  5. spec:
  6.   components:
  7.   - name: image-gitops
  8.     type: kustomize
  9.     properties:
  10.       repoType: git
  11.       # replace it with your repo url
  12.       url: https://github.com/FogDong/KubeVela-GitOps-Infra-Demo
  13.       # replace it with your git secret, GitOps will update the files in your repository with the latest image, which requires write permission
  14.       secretRef: git-secret
  15.       pullInterval: 1m
  16.       git:
  17.         # the branch name
  18.         branch: image
  19.       # the path to sync
  20.       path: ./application
  21.       imageRepository:
  22.         # replace it with your image url
  23.         image: ghcr.io/fogdong/test-fog
  24.         # if it's a private image registry, use `kubectl create secret docker-registry` to create the secret
  25.         # secretRef: imagesecret
  26.         filterTags:
  27.           # filter the image tag
  28.           pattern: '^master-[a-f0-9]+-(?P<ts>[0-9]+)'
  29.           extract: '$ts'
  30.         # use the policy to sort the latest image tag and update
  31.         policy:
  32.           numerical:
  33.             order: asc
  34.         # add more commit message
  35.         commitMessage: "Image: {{range .Updated.Images}}{{println .}}{{end}}"

After the application is deployed successfully, you can see that the my-app application has also been automatically deployed. At this point, the image in my-app is nginx:

  1. $ vela ls
  2. APP             COMPONENT       TYPE        TRAITS  PHASE           HEALTHY     STATUS      CREATED-TIME
  3. image-gitops    image-gitops    kustomize           running         healthy                 2022-06-30 15:16:30 +0800 CST
  4. my-app          my-app          webservice          running         healthy  Ready:1/1  2022-06-30 15:16:31 +0800 CST

After a period of time, the imageRepository we configured will automatically pull the latest image we want and update the application image in the repository.

At this point, you can see a commit from kubevelabot in the config repository, which replaces the nginx image with the latest image from our own repository. Commits are prefixed with Update image automatically.. You can also append the information you want in the commitMessage field with {{range .Updated.Images}}{{println .}}{{end}}.

alt

Note that if you want to put the code and config in the same repository, you need to filter out the commit from KubeVela in CI configuration like below to avoid the repeat build of pipeline.

  1. jobs:
  2.  publish:
  3.    if: "!contains(github.event.head_commit.message, 'Update image automatically')"

Re-check the Application in cluster, we can see that the image of the my-app has been updated after a while.

KubeVela polls the latest information from the code and image repo periodically (at an interval that can be customized):

  • When the Application file in the Git repository is updated, KubeVela will update the Application in the cluster based on the latest configuration.
  • When a new tag is added to the image registry, KubeVela will filter out the latest tag based on your policy and update it to Git repository. When the files in the repository are updated, KubeVela repeats the first step and updates the files in the cluster, thus achieving automatic deployment.

Every time the spec of Application differs, the Application will re-run its workflow to deploy. If you want to strictly control the Application runs with your expected version, you can use Publish Version. With it, you can also viewing the history revisions, checking the differences across revisions, rolling back to the latest succeeded revision and re-publishing past revisions.

GitOps with FluxCD - 图2note

If you’re using external policy or workflow in your Application, then your Application spec may not change if you have updated your policy or workflow. In this case, you can also use Publish Version to force re-trigger the deploy.

If you want to control the Application version in GitOps, you need to update the Publish Version in the Application in your CI to trigger the deploy.

GitOps with FluxCD - 图3note

Before update the PublishVersion, you need to setup a initial PublishVersion in your Application like:

  1. metadata:
  2.   name: my-app
  3.   annotations:
  4.     app.oam.dev/publishVersion: <initial version>

You can checkout the example repo or setup the CI workflow to update it automatically like below:

  1. name: Auto Commit
  2. on:
  3.   push:
  4.     branches:
  5.     - '*'
  7. jobs:
  8.   run:
  9.     name: Auto Commit
  10.     runs-on: ubuntu-latest
  11.     steps:
  12.       - name: Checkout repo
  13.         uses: actions/checkout@v2
  15.       - name: Update publish version
  16.         id: update
  17.         run: |
  18.           VERSION=${GITHUB_SHA::8}
  19.           echo ::set-output name=VERSION::${VERSION}
  20.           # replace app.yaml with your app file name
  21.           sed -i "s|app.oam.dev/publishVersion: .*|app.oam.dev/publishVersion: $VERSION|" app.yaml
  23.       - name: Commit changes
  24.         uses: EndBug/add-and-commit@v7
  25.         with:
  26.           default_author: github_actions
  27.           add: '.'
  28.           message: "[ci skip] deploy from ${{ steps.update.outputs.VERSION }}"
  29.           signoff: true
  30.           # specify the branch you want to commit if need
  31.           # branch: main

This CI will use GitHub SHA as the PublishVersion and update it every time when you push to the branch. Note that this is an example of updating PublishVersion in GitHub Action, if you’re using other CI tools, you can setup the CI like above.

You can also check out GitOps blog and video practice to better experience and use GitOps.

Last updated on May 6, 2023 by Tianxin Dong