Release Preparation

Releases of Apache projects must strictly follow the Apache Foundation release process. Related guidance and policies can be found at.

This document describes the main process and prep work for release. For specific Doris component release steps, you can refer to the respective documentation:

There are three main forms of releases for Apache projects.

  • Source Release: i.e. source release, this is mandatory.
  • Binary Release: e.g., release of a compiled executable. This is optional.
  • Convenience Binaries: Release to third-party platforms for user convenience, such as Maven, Docker, etc. This is also optional.

Release Process

Each project release requires a PMC member or Committer as the Release Manager.

The overall release process is as follows.

  1. Environment preparation
  2. Release preparation
    1. the community initiates DISCUSS and communicates with the community about the specific release plan
    2. create a branch for the release
    3. clean up the issue
    4. merge the necessary patches into the released branch
  3. verify the branch
    1. stability testing
    2. verify the compilation flow of the branch code
    3. Prepare Release Nodes
  4. prepare release materials
    1. Tagging
    2. upload the content to be released to the Apache Dev SVN repositoryRelease Preparation - 图4 (opens new window)
    3. preparation of other Convenience Binaries (e.g. upload to Maven Staging repositoryRelease Preparation - 图5 (opens new window))
  5. Community Release Polling Process
    1. Initiate a VOTE in the Doris Community Dev Mail Group.
    2. After the vote is approved, send a Result email in the Doris community.
    3. If it is an Incubator Project, you also need to
      1. initiate a VOTE in the Incubator General Mailing Group. 2.
      2. Send a Result email to general@incubator.apache.org.
  6. Complete the work
    1. Upload the signed packages to the Apache Release repositoryRelease Preparation - 图6 (opens new window) and generate the relevant links.
    2. Post the download links on the Doris website and github, and clean up the old packages on svn.
    3. Send an Announce email to dev@doris.apache.org
    4. If it is an Incuator Project, you also need to
      1. Send an Announce email to general@incubator.apache.org

Prepare signatures

If this is your first time as Release Manager, then you need to prepare the following tools in your environment

  1. Release SigningRelease Preparation - 图7 (opens new window)
  2. gpgRelease Preparation - 图8 (opens new window)
  3. svnRelease Preparation - 图9 (opens new window)

Prepare gpg key

Release manager needs to create its own signature public key before release, and upload it to the public key server, then you can use this public key to sign the package to be released. If your KEY already exists in KEYSRelease Preparation - 图10 (opens new window), then you can skip this step.

Installation and configuration of the signature software GnuPG

GnuPG

In 1991, programmer Phil Zimmermann developed the encryption software PGP in order to avoid government surveillance; it worked so well that it quickly spread and became an essential tool for many programmers. However, it was commercial software and could not be used freely. So, the Free Software Foundation decided to develop a replacement for PGP, named GnuPG, and that’s how GPG came to be.

installation configuration

CentOS installation command.

  1. yum install gnupg

After installation, the default configuration file gpg.conf will be placed in the home directory.

  1. ~/.gnupg/gpg.conf

If this directory or file does not exist, you can just create an empty file.

Apache recommends SHA512 for signatures, which can be done by configuring gpg. Edit gpg.conf, adding the following three lines.

  1. personal-digest-preferences SHA512
  2. cert-digest-algo SHA512
  3. default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed

Generate a new signature

Preparing a signature

Recommended settings for generating new signatures.

Here you must log in to the user account directly through a terminal such as SecureCRT, not through su - user or ssh, otherwise the password input box will not show up and an error will be reported.

First look at the version of gpg and whether it supports SHA512.

  1. $ gpg --version
  2. gpg (GnuPG) 2.0.22
  3. libgcrypt 1.5.3
  4. Copyright (C) 2013 Free Software Foundation, Inc.
  5. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  6. This is free software: you are free to change and redistribute it.
  7. There is NO WARRANTY, to the extent permitted by law.
  9. Home: ~/.gnupg
  10. Supported algorithms:
  11. Pubkey: RSA, ?, ?, ELG, DSA
  12. Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
  13.         CAMELLIA128, CAMELLIA192, CAMELLIA256
  14. Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
  15. Compression: Uncompressed, ZIP, ZLIB, BZIP2
Generate a new signature
  1. $ gpg --gen-key
  2. gpg (GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc.
  3. This is free software: you are free to change and redistribute it.
  4. There is NO WARRANTY, to the extent permitted by law.
  6. Please select what kind of key you want:
  7.    (1) RSA and RSA (default)
  8.    (2) DSA and Elgamal
  9.    (3) DSA (sign only)
  10.    (4) RSA (sign only)
  11. Your selection? 1
  12. RSA keys may be between 1024 and 4096 bits long.
  13. What keysize do you want? (2048) 4096
  14. Requested keysize is 4096 bits
  15. Please specify how long the key should be valid.
  16.          0 = key does not expire
  17.       <n>  = key expires in n days
  18.       <n>w = key expires in n weeks
  19.       <n>m = key expires in n months
  20.       <n>y = key expires in n years
  21. Key is valid for? (0)
  22. Key does not expire at all
  23. Is this correct? (y/N) y
  25. GnuPG needs to construct a user ID to identify your key.
  27. Real name: xxx
  28. Name must be at least 5 characters long
  29. Real name: xxx-yyy
  30. Email address: xxx@apache.org
  31. Comment: xxx's key
  32. You selected this USER-ID:
  33.     "xxx-yyy (xxx's key) <xxx@apache.org>"
  35. Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o

Real name should be the same as the id shown in id.apache.org. Email address is the apache email address.

Enter passphrase, twice, more than 8 characters.

The secret key here must be remembered, it will be used later when signing. It will also be used for publishing other components

View and output

The first line shows the public key file name (pubring.gpg), the second line shows the public key characteristics (4096 bits, hash string and generation time), the third line shows the “user ID”, comments, emails, and the fourth line shows the private key characteristics.

  1. $ gpg --list-keys
  2. /home/lide/.gnupg/pubring.gpg
  3. -----------------------------
  4. pub 4096R/33DBF2E0 2018-12-06
  5. uid xxx-yyyy (xxx's key) <xxx@apache.org>
  6. sub 4096R/0E8182E6 2018-12-06

where xxx-yyyy is the user ID.

  1. gpg --armor --output public-key.txt --export [user-id]
  1. $ gpg --armor --output public-key.txt --export xxx-yyyy
  2. The file 'public-key.txt' already exists. Is it overwritten? (y/N)y
  3. $ cat public-key.txt
  4. -----BEGIN PGP PUBLIC KEY BLOCK-----
  5. Version: GnuPG v2.0.22 (GNU/Linux)
  7. mQINBFwJEQ0BEACwqLluHfjBqD/RWZ4uoYxNYHlIzZvbvxAlwS2mn53BirLIU/G3
  8. 9opMWNplvmK+3+gNlRlFpiZ7EvHsF/YJOAP59HmI2Z...

Uploading Signed Public Keys

A public key server is a server on the network dedicated to storing the user’s public key. send-keys parameter can upload the public key to the server.

  1. gpg --send-keys xxxx --keyserver https://keyserver.ubuntu.com/

where xxxx is the string after pub in the -list-keys result of the previous step, e.g., 33DBF2E0

You can also upload the contents of the above public-key.txt via https://keyserver.ubuntu.com/Release Preparation - 图11 (opens new window).

After successful upload, you can query this https://keyserver.ubuntu.com/Release Preparation - 图12 (opens new window) by entering 0x33DBF2E0. (Note that it needs to start with 0x)

There is a delay in querying this website, you may need to wait for 1 hour.

generates fingerprint and uploads it to apache user information

Since the public key server has no checking mechanism, anyone can upload a public key in your name, so there is no way to guarantee the reliability of the public key on the server. Usually, you can publish a public key fingerprint on your website and let other people check whether the downloaded public key is genuine or not.

The fingerprint parameter generates a public key fingerprint.

  1. gpg --fingerprint [user-id]
  1. $ gpg --fingerprint xxx-yyyy
  2. pub 4096R/33DBF2E0 2018-12-06
  3.       Key fingerprint = 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0
  4. uid xxx-yyyy (xxx's key) <xxx@apache.org>
  5. sub 4096R/0E8182E6 2018-12-06

Paste the fingerprint above (i.e. 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0) into your own user information at

https://id.apache.org

OpenPGP Public Key Primary Fingerprint:

Note: Each person can have more than one Public Key.

generates keys

Be careful not to delete existing content in the KEYS file, it can only be added.

  1. svn co https://dist.apache.org/repos/dist/dev/incubator/doris/
  2. # edit doris/KEYS file
  3. gpg --list-sigs [user-id] >> doris/KEYS
  4. gpg --armor --export [user ID] >> doris/KEYS
  5. svn ci --username $ASF_USERNAME --password "$ASF_PASSWORD" -m "UPDATE KEYS"

Note that the KEYS file should also be published to the following svn library.

  1. svn co https://dist.apache.org/repos/dist/release/incubator/doris
  2. # edit doris/KEYS file
  3. svn ci --username $ASF_USERNAME --password "$ASF_PASSWORD" -m "UPDATE KEYS"

After that it will automatically sync to.

  1. https://downloads.apache.org/incubator/doris/KEYS

In subsequent release poll emails, use the KEYS file here in https://downloads.apache.org/incubator/doris/KEYS.

Maven Release Preparation

For components such as the Doris Connector, you need to use maven for the release.

  1. Generate a master password

    mvn --encrypt-master-password <password>

    This password is only used to encrypt other passwords that follow, and the output is something like {VSb+6+76djkH/43...} Then create the ~/.m2/settings-security.xml file with the following content

    1. <settingsSecurity>
    2.   <master>{VSb+6+76djkH/43...}</master>
    3. </settingsSecurity>

  2. Encrypt apache passwords

    mvn --encrypt-password <password>

    The password is the password for the apache account. The output is similar to {GRKbCylpwysHfV...}

    Add in ~/.m2/settings.xml

    1. <servers>
    2.   <!-- To publish a snapshot of your project -->
    3.   <server>
    4.     <id>apache.snapshots.https</id>
    5.     <username>yangzhg</username>
    6.     <password>{GRKbCylpwysHfV...}</password>
    7.   </server>
    8.   <!-- To stage a release of your project -->
    9.   <server>
    10.     <id>apache.releases.https</id>
    11.     <username>yangzhg</username>
    12.     <password>{GRKbCylpwysHfV...}</password>
    13.   </server>
    14. </servers>

Initiating DISCUSS in the community

DISCUSS is not a required process before a release, but it is highly recommended to start a discussion in the dev@doris mail group before a major release. Content includes, but is not limited to, descriptions of important features, bug fixes, etc.