Release Checklist

A Maintainer’s Guide to Releasing Helm

Time for a new Helm release! As a Helm maintainer cutting a release, you are the best person to update this release checklist should your experiences vary from what’s documented here.

All releases will be of the form vX.Y.Z where X is the major version number, Y is the minor version number and Z is the patch release number. This project strictly follows semantic versioning so following this step is critical.

Helm announces in advance the date of its next minor release. Every effort should be made to respect the announced date. Furthermore, when starting the release process, the date for the next release should have been selected as it will be used in the release process.

These directions will cover initial configuration followed by the release process for three different kinds of releases:

  • Major Releases - released less frequently - have breaking changes
  • Minor Releases - released every 3 to 4 months - no breaking changes
  • Patch Releases - released monthly - do not require all steps in this guide

Initial Configuration

  1. Create the Release Branch
  2. Major/Minor releases: Change the Version Number in Git
  3. Major/Minor releases: Commit and Push the Release Branch
  4. Major/Minor releases: Create a Release Candidate
  5. Major/Minor releases: Iterate on Successive Release Candidates
  6. Finalize the Release
  7. Write the Release Notes
  8. PGP Sign the downloads
  9. Publish Release
  10. Update Docs
  11. Tell the Community

Initial Configuration

Set Up Git Remote

It is important to note that this document assumes that the git remote in your repository that corresponds to https://github.com/helm/helm is named “upstream”. If yours is not (for example, if you’ve chosen to name it “origin” or something similar instead), be sure to adjust the listed snippets for your local environment accordingly. If you are not sure what your upstream remote is named, use a command like git remote -v to find out.

If you don’t have an upstream remote , you can add one using something like:

  1. git remote add upstream git@github.com:helm/helm.git

Set Up Environment Variables

In this doc, we are going to reference a few environment variables as well, which you may want to set for convenience. For major/minor releases, use the following:

  1. export RELEASE_NAME=vX.Y.0
  2. export RELEASE_BRANCH_NAME="release-X.Y"
  3. export RELEASE_CANDIDATE_NAME="$RELEASE_NAME-rc.1"

If you are creating a patch release, use the following instead:

  1. export PREVIOUS_PATCH_RELEASE=vX.Y.Z
  2. export RELEASE_NAME=vX.Y.Z+1
  3. export RELEASE_BRANCH_NAME="release-X.Y"

Set Up Signing Key

We are also going to be adding security and verification of the release process by hashing the binaries and providing signature files. We perform this using GitHub and GPG. If you do not have GPG already setup you can follow these steps:

  1. Install GPG
  2. Generate GPG key
  3. Add key to GitHub account
  4. Set signing key in Git

Once you have a signing key you need to add it to the KEYS file at the root of the repository. The instructions for adding it to the KEYS file are in the file. If you have not done so already, you need to add your public key to the keyserver network. If you use GnuPG you can follow the instructions provided by Debian.

1. Create the Release Branch

Major/Minor Releases

Major releases are for new feature additions and behavioral changes that break backwards compatibility. Minor releases are for new feature additions that do not break backwards compatibility. To create a major or minor release, start by creating a release-X.Y branch from main.

  1. git fetch upstream
  2. git checkout upstream/main
  3. git checkout -b $RELEASE_BRANCH_NAME

This new branch is going to be the base for the release, which we are going to iterate upon later.

Verify that a helm/helm milestone for the release exists on GitHub (creating it if necessary). Make sure PRs and issues for this release are in this milestone.

For major & minor releases, move on to step 2: Major/Minor releases: Change the Version Number in Git.

Patch releases

Patch releases are a few critical cherry-picked fixes to existing releases. Start by creating a release-X.Y branch:

  1. git fetch upstream
  2. git checkout -b $RELEASE_BRANCH_NAME upstream/$RELEASE_BRANCH_NAME

From here, we can cherry-pick the commits we want to bring into the patch release:

  1. # get the commits ids we want to cherry-pick
  2. git log --oneline
  3. # cherry-pick the commits starting from the oldest one, without including merge commits
  4. git cherry-pick -x <commit-id>

After the commits have been cherry picked the release branch needs to be pushed.

  1. git push upstream $RELEASE_BRANCH_NAME

Pushing the branch will cause the tests to run. Make sure they pass prior to creating the tag. This new tag is going to be the base for the patch release.

Creating a helm/helm milestone is optional for patch releases.

Make sure to check helm on CircleCI to see that the release passed CI before proceeding. Patch releases can skip steps 2-5 and proceed to step 6 to Finalize the Release.

2. Major/Minor releases: Change the Version Number in Git

When doing a major or minor release, make sure to update internal/version/version.go with the new release version.

  1. $ git diff internal/version/version.go
  2. diff --git a/internal/version/version.go b/internal/version/version.go
  3. index 712aae64..c1ed191e 100644
  4. --- a/internal/version/version.go
  5. +++ b/internal/version/version.go
  6. @@ -30,7 +30,7 @@ var (
  7.         // Increment major number for new feature additions and behavioral changes.
  8.         // Increment minor number for bug fixes and performance enhancements.
  9.         // Increment patch number for critical fixes to existing releases.
  10. -       version = "v3.3"
  11. +       version = "v3.4"
  13.         // metadata is extra build time data
  14.         metadata = ""

In addition to updating the version within the version.go file, you will also need to update corresponding tests that are using that version number.

  • cmd/helm/testdata/output/version.txt
  • cmd/helm/testdata/output/version-client.txt
  • cmd/helm/testdata/output/version-client-shorthand.txt
  • cmd/helm/testdata/output/version-short.txt
  • cmd/helm/testdata/output/version-template.txt
  • pkg/chartutil/capabilities_test.go
  1. git add .
  2. git commit -m "bump version to $RELEASE_NAME"

This will update it for the $RELEASE_BRANCH_NAME only. You will also need to pull this change into the main branch for when the next release is being created, as in this example of 3.2 to 3.3, and add it to the milestone for the next release.

  1. # get the last commit id i.e. commit to bump the version
  2. git log --format="%H" -n 1
  4. # create new branch off main
  5. git checkout main
  6. git checkout -b bump-version-<release_version>
  8. # cherry pick the commit using id from first command
  9. git cherry-pick -x <commit-id>
  11. # commit the change
  12. git push origin bump-version-<release-version>

3. Major/Minor releases: Commit and Push the Release Branch

In order for others to start testing, we can now push the release branch upstream and start the test process.

  1. git push upstream $RELEASE_BRANCH_NAME

Make sure to check helm on CircleCI to see that the release passed CI before proceeding.

If anyone is available, let others peer-review the branch before continuing to ensure that all the proper changes have been made and all of the commits for the release are there.

4. Major/Minor releases: Create a Release Candidate

Now that the release branch is out and ready, it is time to start creating and iterating on release candidates.

  1. git tag --sign --annotate "${RELEASE_CANDIDATE_NAME}" --message "Helm release ${RELEASE_CANDIDATE_NAME}"
  2. git push upstream $RELEASE_CANDIDATE_NAME

CircleCI will automatically create a tagged release image and client binary to test with.

For testers, the process to start testing after CircleCI finishes building the artifacts involves the following steps to grab the client:

linux/amd64, using /bin/bash:

  1. wget https://get.helm.sh/helm-$RELEASE_CANDIDATE_NAME-linux-amd64.tar.gz

darwin/amd64, using Terminal.app:

  1. wget https://get.helm.sh/helm-$RELEASE_CANDIDATE_NAME-darwin-amd64.tar.gz

windows/amd64, using PowerShell:

  1. PS C:\> Invoke-WebRequest -Uri "https://get.helm.sh/helm-$RELEASE_CANDIDATE_NAME-windows-amd64.tar.gz" -OutFile "helm-$ReleaseCandidateName-windows-amd64.tar.gz"

Then, unpack and move the binary to somewhere on your $PATH, or move it somewhere and add it to your $PATH (e.g. /usr/local/bin/helm for linux/macOS, C:\Program Files\helm\helm.exe for Windows).

5. Major/Minor releases: Iterate on Successive Release Candidates

Spend several days explicitly investing time and resources to try and break helm in every possible way, documenting any findings pertinent to the release. This time should be spent testing and finding ways in which the release might have caused various features or upgrade environments to have issues, not coding. During this time, the release is in code freeze, and any additional code changes will be pushed out to the next release.

During this phase, the $RELEASE_BRANCH_NAME branch will keep evolving as you will produce new release candidates. The frequency of new candidates is up to the release manager: use your best judgement taking into account the severity of reported issues, testers’ availability, and the release deadline date. Generally speaking, it is better to let a release roll over the deadline than to ship a broken release.

Each time you’ll want to produce a new release candidate, you will start by adding commits to the branch by cherry-picking from main:

  1. git cherry-pick -x <commit_id>

You will also want to push the branch to GitHub and ensure it passes CI.

After that, tag it and notify users of the new release candidate:

  1. export RELEASE_CANDIDATE_NAME="$RELEASE_NAME-rc.2"
  2. git tag --sign --annotate "${RELEASE_CANDIDATE_NAME}" --message "Helm release ${RELEASE_CANDIDATE_NAME}"
  3. git push upstream $RELEASE_CANDIDATE_NAME

Once pushed to GitHub, check to ensure the branch with this tag builds in CI.

From here on just repeat this process, continuously testing until you’re happy with the release candidate. For a release candidate, we don’t write the full notes, but you can scaffold out some release notes.

6. Finalize the Release

When you’re finally happy with the quality of a release candidate, you can move on and create the real thing. Double-check one last time to make sure everything is in order, then finally push the release tag.

  1. git checkout $RELEASE_BRANCH_NAME
  2. git tag --sign --annotate "${RELEASE_NAME}" --message "Helm release ${RELEASE_NAME}"
  3. git push upstream $RELEASE_NAME

Verify that the release succeeded in CircleCI. If not, you will need to fix the release and push the release again.

As the CI job will take some time to run, you can move on to writing release notes while you wait for it to complete.

7. Write the Release Notes

We will auto-generate a changelog based on the commits that occurred during a release cycle, but it is usually more beneficial to the end-user if the release notes are hand-written by a human being/marketing team/dog.

If you’re releasing a major/minor release, listing notable user-facing features is usually sufficient. For patch releases, do the same, but make note of the symptoms and who is affected.

The release notes should include the version and planned date of the next release.

An example release note for a minor release would look like this:

  1. ## vX.Y.Z
  3. Helm vX.Y.Z is a feature release. This release, we focused on <insert focal point>. Users are encouraged to upgrade for the best experience.
  5. The community keeps growing, and we'd love to see you there!
  7. - Join the discussion in [Kubernetes Slack](https://kubernetes.slack.com):
  8.   - `#helm-users` for questions and just to hang out
  9.   - `#helm-dev` for discussing PRs, code, and bugs
  10. - Hang out at the Public Developer Call: Thursday, 9:30 Pacific via [Zoom](https://zoom.us/j/696660622)
  11. - Test, debug, and contribute charts: [Artifact Hub helm charts](https://artifacthub.io/packages/search?kind=0)
  13. ## Notable Changes
  15. - Kubernetes 1.16 is now supported including new manifest apiVersions
  16. - Sprig was upgraded to 2.22
  18. ## Installation and Upgrading
  20. Download Helm X.Y. The common platform binaries are here:
  22. - [MacOS amd64](https://get.helm.sh/helm-vX.Y.Z-darwin-amd64.tar.gz) ([checksum](https://get.helm.sh/helm-vX.Y.Z-darwin-amd64.tar.gz.sha256sum) / CHECKSUM_VAL)
  23. - [Linux amd64](https://get.helm.sh/helm-vX.Y.Z-linux-amd64.tar.gz) ([checksum](https://get.helm.sh/helm-vX.Y.Z-linux-amd64.tar.gz.sha256sum) / CHECKSUM_VAL)
  24. - [Linux arm](https://get.helm.sh/helm-vX.Y.Z-linux-arm.tar.gz) ([checksum](https://get.helm.sh/helm-vX.Y.Z-linux-arm.tar.gz.sha256) / CHECKSUM_VAL)
  25. - [Linux arm64](https://get.helm.sh/helm-vX.Y.Z-linux-arm64.tar.gz) ([checksum](https://get.helm.sh/helm-vX.Y.Z-linux-arm64.tar.gz.sha256sum) / CHECKSUM_VAL)
  26. - [Linux i386](https://get.helm.sh/helm-vX.Y.Z-linux-386.tar.gz) ([checksum](https://get.helm.sh/helm-vX.Y.Z-linux-386.tar.gz.sha256) / CHECKSUM_VAL)
  27. - [Linux ppc64le](https://get.helm.sh/helm-vX.Y.Z-linux-ppc64le.tar.gz) ([checksum](https://get.helm.sh/helm-vX.Y.Z-linux-ppc64le.tar.gz.sha256sum) / CHECKSUM_VAL)
  28. - [Linux s390x](https://get.helm.sh/helm-vX.Y.Z-linux-s390x.tar.gz) ([checksum](https://get.helm.sh/helm-vX.Y.Z-linux-s390x.tar.gz.sha256sum) / CHECKSUM_VAL)
  29. - [Windows amd64](https://get.helm.sh/helm-vX.Y.Z-windows-amd64.zip) ([checksum](https://get.helm.sh/helm-vX.Y.Z-windows-amd64.zip.sha256sum) / CHECKSUM_VAL)
  31. The [Quickstart Guide](https://docs.helm.sh/using_helm/#quickstart-guide) will get you going from there. For **upgrade instructions** or detailed installation notes, check the [install guide](https://docs.helm.sh/using_helm/#installing-helm). You can also use a [script to install](https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3) on any system with `bash`.
  33. ## What's Next
  35. - vX.Y.Z+1 will contain only bug fixes and is planned for <insert DATE>.
  36. - vX.Y+1.0 is the next feature release and is planned for <insert DATE>. This release will focus on ...
  38. ## Changelog
  40. - chore(*): bump version to v2.7.0 08c1144f5eb3e3b636d9775617287cc26e53dba4 (Adam Reese)
  41. - fix circle not building tags f4f932fabd197f7e6d608c8672b33a483b4b76fa (Matthew Fisher)

A partially completed set of release notes including the changelog can be created by running the following command:

  1. export VERSION="$RELEASE_NAME"
  2. export PREVIOUS_RELEASE=vX.Y.Z
  3. make clean
  4. make fetch-dist
  5. make release-notes

This will create a good baseline set of release notes to which you should just need to fill out the Notable Changes and What’s next sections.

Feel free to add your voice to the release notes; it’s nice for people to think we’re not all robots.

You should also double check the URLs and checksums are correct in the auto-generated release notes.

Once finished, go into GitHub to helm/helm releases and edit the release notes for the tagged release with the notes written here. For target branch, set to $RELEASE_BRANCH_NAME.

It is now worth getting other people to take a look at the release notes before the release is published. Send a request out to #helm-dev for review. It is always beneficial as it can be easy to miss something.

8. PGP Sign the downloads

While hashes provide a signature that the content of the downloads is what it was generated, signed packages provide traceability of where the package came from.

To do this, run the following make commands:

  1. export VERSION="$RELEASE_NAME"
  2. make clean        # if not already run
  3. make fetch-dist    # if not already run
  4. make sign

This will generate ascii armored signature files for each of the files pushed by CI.

All of the signature files (*.asc) need to be uploaded to the release on GitHub (attach binaries).

9. Publish Release

Time to make the release official!

After the release notes are saved on GitHub, the CI build is completed, and you’ve added the signature files to the release, you can hit “Publish” on the release. This publishes the release, listing it as “latest”, and shows this release on the front page of the helm/helm repo.

10. Update Docs

The Helm website docs section lists the Helm versions for the docs. Major, minor, and patch versions need to be updated on the site. The date for the next minor release is also published on the site and must be updated. To do that create a pull request against the helm-www repository. In the config.toml file find the proper params.versions section and update the Helm version, like in this example of updating the current version. In the same config.toml file, update the params.nextversion section.

Close the helm/helm milestone for the release, if applicable.

Update the version skew for major and minor releases.

Update the release calendar here:

  • create an entry for the next minor release with a reminder for that day at 5pm GMT
  • create an entry for the RC1 of the next minor release on the Monday of the week before the planned release, with a reminder for that day at 5pm GMT

11. Tell the Community

Congratulations! You’re done. Go grab yourself a $DRINK_OF_CHOICE. You’ve earned it.

After enjoying a nice $DRINK_OF_CHOICE, go forth and announce the new release in Slack and on Twitter with a link to the release on GitHub.

Optionally, write a blog post about the new release and showcase some of the new features on there!