Continuous Integration

To ensure the quality of their software, teams often apply Continuous Integration workflows, commonly known as CI. With CI, teams continuously run a suite of automated verifications against every change to the code-base. During CI, teams may run many kinds of verifications:

  • Compilation or build of the most recent version to make sure it isn’t broken.
  • Linting to enforce any accepted code-style standards.
  • Unit tests that verify individual components work as expected and that changes to the codebase do not cause regressions in other areas.
  • Security scans to make sure no known vulnerabilities are introduced to the codebase.
  • And much more!

From our discussions with the Ent community, we have learned that many teams using Ent already use CI and would like to enforce some Ent-specific verifications into their workflows.

To support the community with this effort we have started this guide which documents common best practices to verify in CI and introduces ent/contrib/ci a GitHub Action we maintain that codifies them.

Verify all generated files are checked in

Ent heavily relies on code generation. In our experience, generated code should always be checked into source control. This is done for two reasons:

  • If generated code is checked into source control, it can be read along with the main application code. Having generated code present when the code is reviewed or when a repository is browsed is essential to get a complete picture of how things work.
  • Differences in development environments between team members can easily be spotted and remedied. This further reduces the chance of “it works on my machine” type issues since everyone is running the same code.

If you’re using GitHub for source control, it’s easy to verify that all generated files are checked in with the ent/contrib/ci GitHub Action. Otherwise, we supply a simple bash script that you can integrate in your existing CI flow.

  • GitHub Action
  • Bash

Simply add a file named `.github/workflows/ent-ci.yaml` in your repository:

  1. name: EntCI
  2. on:
  3.   push:
  4.   # Run whenever code is changed in the master.
  5.     branches:
  6.       - master
  7.   # Run on PRs where something changed under the `ent/` directory.
  8.   pull_request:
  9.     paths:
  10.       - 'ent/*'
  11. jobs:
  12.   ent:
  13.     runs-on: ubuntu-latest
  14.     steps:
  15.       - uses: actions/checkout@v3.0.1
  16.       - uses: actions/setup-go@v3
  17.         with:
  18.           go-version-file: 'go.mod'
  19.       - uses: ent/contrib/ci@master
  1. go generate ./...
  2. status=$(git status --porcelain)
  3. if [ -n "$status" ]; then
  4.     echo "you need to run 'go generate ./...' and commit the changes"
  5.     echo "$status"
  6.     exit 1
  7. fi

Lint migration files

Changes to your project’s Ent schema almost always result in a modification of your database. If you are using Versioned Migrations to manage changes to your database schema, you can run migration linting as part of your continuous integration flow. This is done for multiple reasons:

  • Linting replays your migration directory on a database container to make sure all SQL statements are valid and in the correct order.
  • Migration directory integrity is enforced - ensuring that history wasn’t accidentally changed and that migrations that are planned in parallel are unified to a clean linear history.
  • Destructive changes are detected notifying you of any potential data loss that may be caused by your migrations way before they reach your production database.
  • Linting detects data-dependant changes that may fail upon deployment and require more careful review from your side.

If you’re using GitHub, you can use the Official Atlas Action to run migration linting during CI.

Add .github/workflows/atlas-ci.yaml to your repo with the following contents:

  • MySQL
  • MariaDB
  • PostgreSQL
  • SQLite
  1. name: Atlas CI
  2. on:
  3.   # Run whenever code is changed in the master branch,
  4.   # change this to your root branch.
  5.   push:
  6.     branches:
  7.       - master
  8.   # Run on PRs where something changed under the `ent/migrate/migrations/` directory.
  9.   pull_request:
  10.     paths:
  11.       - 'ent/migrate/migrations/*'
  12. jobs:
  13.   lint:
  14.     services:
  15.       # Spin up a mysql:8.0.29 container to be used as the dev-database for analysis.
  16.       mysql:
  17.         image: mysql:8.0.29
  18.         env:
  19.           MYSQL_ROOT_PASSWORD: pass
  20.           MYSQL_DATABASE: test
  21.         ports:
  22.           - 3306:3306
  23.         options: >-
  24.           --health-cmd "mysqladmin ping -ppass"
  25.           --health-interval 10s
  26.           --health-start-period 10s
  27.           --health-timeout 5s
  28.           --health-retries 10
  29.     runs-on: ubuntu-latest
  30.     steps:
  31.       - uses: actions/checkout@v3.0.1
  32.         with:
  33.           fetch-depth: 0 # Mandatory unless "latest" is set below.
  34.       - uses: ariga/atlas-action@v0
  35.         with:
  36.           dir: ent/migrate/migrations
  37.           dir-format: golang-migrate # Or: atlas, goose, dbmate
  38.           dev-url: mysql://root:pass@localhost:3306/test
  1. name: Atlas CI
  2. on:
  3.   # Run whenever code is changed in the master branch,
  4.   # change this to your root branch.
  5.   push:
  6.     branches:
  7.       - master
  8.   # Run on PRs where something changed under the `ent/migrate/migrations/` directory.
  9.   pull_request:
  10.     paths:
  11.       - 'ent/migrate/migrations/*'
  12. jobs:
  13.   lint:
  14.     services:
  15.       # Spin up a maria:10.7 container to be used as the dev-database for analysis.
  16.       maria:
  17.         image: mariadb:10.7
  18.         env:
  19.           MYSQL_DATABASE: test
  20.           MYSQL_ROOT_PASSWORD: pass
  21.         ports:
  22.           - 3306:3306
  23.         options: >-
  24.           --health-cmd "mysqladmin ping -ppass"
  25.           --health-interval 10s
  26.           --health-start-period 10s
  27.           --health-timeout 5s
  28.           --health-retries 10
  29.     runs-on: ubuntu-latest
  30.     steps:
  31.       - uses: actions/checkout@v3.0.1
  32.         with:
  33.           fetch-depth: 0 # Mandatory unless "latest" is set below.
  34.       - uses: ariga/atlas-action@v0
  35.         with:
  36.           dir: ent/migrate/migrations
  37.           dir-format: golang-migrate # Or: atlas, goose, dbmate
  38.           dev-url: maria://root:pass@localhost:3306/test
  1. name: Atlas CI
  2. on:
  3.   # Run whenever code is changed in the master branch,
  4.   # change this to your root branch.
  5.   push:
  6.     branches:
  7.       - master
  8.   # Run on PRs where something changed under the `ent/migrate/migrations/` directory.
  9.   pull_request:
  10.     paths:
  11.       - 'ent/migrate/migrations/*'
  12. jobs:
  13.   lint:
  14.     services:
  15.       # Spin up a postgres:10 container to be used as the dev-database for analysis.
  16.       postgres:
  17.         image: postgres:10
  18.         env:
  19.           POSTGRES_DB: test
  20.           POSTGRES_PASSWORD: pass
  21.         ports:
  22.           - 5432:5432
  23.         options: >-
  24.           --health-cmd pg_isready
  25.           --health-interval 10s
  26.           --health-timeout 5s
  27.           --health-retries 5
  28.     runs-on: ubuntu-latest
  29.     steps:
  30.       - uses: actions/checkout@v3.0.1
  31.         with:
  32.           fetch-depth: 0 # Mandatory unless "latest" is set below.
  33.       - uses: ariga/atlas-action@v0
  34.         with:
  35.           dir: ent/migrate/migrations
  36.           dir-format: golang-migrate # Or: atlas, goose, dbmate
  37.           dev-url: postgres://postgres:pass@localhost:5432/test?sslmode=disable
  1. name: Atlas CI
  2. on:
  3.   # Run whenever code is changed in the master branch,
  4.   # change this to your root branch.
  5.   push:
  6.     branches:
  7.       - master
  8.   # Run on PRs where something changed under the `ent/migrate/migrations/` directory.
  9.   pull_request:
  10.     paths:
  11.       - 'ent/migrate/migrations/*'
  12. jobs:
  13.   lint:
  14.     runs-on: ubuntu-latest
  15.     steps:
  16.       - uses: actions/checkout@v3.0.1
  17.         with:
  18.           fetch-depth: 0 # Mandatory unless "latest" is set below.
  19.       - uses: ariga/atlas-action@v0
  20.         with:
  21.           dir: ent/migrate/migrations
  22.           dir-format: golang-migrate # Or: atlas, goose, dbmate
  23.           dev-url: sqlite://./dev.db?_fk=1

Notice that running atlas migrate lint requires a clean dev-database which is provided by the services block in the example code above.