Apache Doris 发布流程

Apache Doris 发布流程

Apache 的发布必须至少是 IPMC 成员，拥有 apache 邮箱的commiter，这个角色叫做 release manager。

发布的大致流程如下：

在社区发起 DISCUSS;
准备分支和打 tag;
将 tag 打包签名；
上传签名的软件包到 Apache SVN 的 DEV 目录
发社区投票邮件
投票通过后，发 Result 邮件
发邮件到 general@incubator.apache.org 进行投票
发 Result 邮件到 general@incubator.apache.org
上传签名的软件包到 Apache SVN 的 release 目录，并生成相关链接
准备 release note 并发 Announce 邮件到 general@incubator.apache.org
在 Doris 官网和 github 发布下载链接

Release manager 在发布前需要先生成自己的签名公钥，并上传到公钥服务器，之后就可以用这个公钥对准备发布的软件包进行签名。

1. 准备发布

1.1 在社区发起 DISCUSS

如果觉得已经修复了很多bug，开发了比较重要的 feature，任何 IPMC 成员都可以发起 DISCUSS 讨论发布新版本。可以发起一个标题为 [DISCUSS] x.y.z release 的邮件，在社区内部进行讨论，说明已经修复了哪些bug，开发了哪些 features。如果 DISCUSS 邮件得到大家支持就可以进行下一步。

1.2 准备分支

发布前需要先新建一个分支，这个分支要进行比较充分的测试，使得功能可用，bug收敛，重要bug都得到修复。

例如：

$ git checkout -b branch-0.9

1.3 打 tag

当上述分支已经比较稳定后，就可以在此分支上打 tag。记得在创建 tag 时，修改 gensrc/script/gen_build_version.sh 中的 build_version 变量。如 build_version="0.10.0-release"

例如：

$ git checkout branch-0.9
$ git tag -a 0.9.0-rc01 -m "0.9.0 release candidate 01"
$ git push origin 0.9.0-rc01
Counting objects: 1, done.
Writing objects: 100% (1/1), 165 bytes | 0 bytes/s, done.
Total 1 (delta 0), reused 0 (delta 0)
To git@github.com:apache/incubator-doris.git
 * [new tag]         0.9.0-rc01 -> 0.9.0-rc01
$ git tag

2. 签名软件 GnuPG 的安装配置

2.1 GnuPG

1991年，程序员Phil Zimmermann为了避开政府监视，开发了加密软件PGP。这个软件非常好用，迅速流传开来，成了许多程序员的必备工具。但是，它是商业软件，不能自由使用。所以，自由软件基金会决定，开发一个PGP的替代品，取名为GnuPG。这就是GPG的由来。

2.2 安装配置

CentOS 安装命令：

yum install gnupg

安装完成后，默认配置文件 gpg.conf 会放在 home 目录下。

~/.gnupg/gpg.conf

如果不存在这个目录或文件，可以直接创建一个空文件。编辑gpg.conf, 修改或者增加 keyserver 配置：

keyserver hkp://keys.gnupg.net

Apache 签名推荐 SHA512，可以通过配置 gpg 完成。编辑gpg.conf, 增加下面的三行：

personal-digest-preferences SHA512
cert-digest-algo SHA512
default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed

3. 生成新的签名

3.1 准备签名

推荐的生成新签名的设置：

这里必须通过 SecureCRT 等终端直接登录用户账户，不能通过 su - user 或者 ssh 转，否则密码输入 box 会显示不出来而报错。

先看下 gpg 的 version 以及是否支持 SHA512.

$ gpg --version
gpg (GnuPG) 2.0.22
libgcrypt 1.5.3
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Home: ~/.gnupg
Supported algorithms:
Pubkey: RSA, ?, ?, ELG, DSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

3.2 生成新的签名

$ gpg --gen-key
gpg (GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) y
GnuPG needs to construct a user ID to identify your key.
Real name: xxx
Name must be at least 5 characters long
Real name: xxx-yyy
Email address: xxx@apache.org
Comment: xxx's key
You selected this USER-ID:
    "xxx-yyy (xxx's key) <xxx@apache.org>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o

其中 Real name 需保持和 id.apache.org 中显示的 id 一致。 Email address 为 apache 的邮箱。

3.3 查看和输出

第一行显示公钥文件名（pubring.gpg），第二行显示公钥特征（4096位，Hash字符串和生成时间），第三行显示”用户ID”，第四行显示私钥特征。

$ gpg --list-keys
/home/lide/.gnupg/pubring.gpg
-----------------------------
pub   4096R/33DBF2E0 2018-12-06
uid                  xxx-yyy  (xxx's key) <xxx@apache.org>
sub   4096R/0E8182E6 2018-12-06

其中 xxx-yyy 就是用户ID。

gpg —armor —output public-key.txt —export [用户ID]

$ gpg --armor --output public-key.txt --export xxx-yyy
$ cat public-key.txt
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v2.0.22 (GNU/Linux)
mQINBFwJEQ0BEACwqLluHfjBqD/RWZ4uoYxNYHlIzZvbvxAlwS2mn53BirLIU/G3
9opMWNplvmK+3+gNlRlFpiZ7EvHsF/YJOAP59HmI2Z...

4. 上传签名公钥

公钥服务器是网络上专门储存用户公钥的服务器。send-keys 参数可以将公钥上传到服务器。

gpg —send-keys xxxx

其中 xxxx 为上一步 —list-keys 结果中 pub 后面的字符串，如上为：33DBF2E0

也可以通过下面的网址上传上述 public-key.txt 的内容：

http://keys.gnupg.net

上传成功之后，可以通过查询这个网站，输入 0x33DBF2E0 查询：

http://keys.gnupg.net

该网站查询有延迟，可能需要等1个小时。

5. 生成 fingerprint 并上传到 apache 用户信息中

由于公钥服务器没有检查机制，任何人都可以用你的名义上传公钥，所以没有办法保证服务器上的公钥的可靠性。通常，你可以在网站上公布一个公钥指纹，让其他人核对下载到的公钥是否为真。

fingerprint参数生成公钥指纹：

gpg --fingerprint [用户ID]

$ gpg --fingerprint xxx-yyy
pub   4096R/33DBF2E0 2018-12-06
      Key fingerprint = 07AA E690 B01D 1A4B 469B  0BEF 5E29 CE39 33DB F2E0
uid                  xxx-yyy (xxx's key) <xxx@apache.org>
sub   4096R/0E8182E6 2018-12-06

将上面的 fingerprint （即 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0）粘贴到自己的用户信息中：

https://id.apache.org OpenPGP Public Key Primary Fingerprint:

6. 生成 keys

新建一个名为 KEYS 的文件，写入如下内容（无需做任何修改）：

This file contains the PGP keys of various developers.
Users: pgp < KEYS
or
       gpg --import KEYS
Developers:
    pgp -kxa <your name> and append it to this file.
or
    (pgpk -ll <your name> && pgpk -xa <your name>) >> this file.
or
    (gpg --list-sigs <your name>
    && gpg --armor --export <your name>) >> this file.

然后生成将签名信息追加写入：

gpg --list-sigs [用户 ID] >> KEYS

最后，将 public key 追加导入：

gpg --armor --export [用户 ID] >> KEYS

7. 打包签名

如下步骤，也需要通过 SecureCRT 等终端直接登录用户账户，不能通过 su - user 或者 ssh 转，否则密码输入 box 会显示不出来而报错。

$ git checkout 0.9.0-rc01
$ git archive --format=tar 0.9.0-rc01 --prefix=apache-doris-0.9.0-incubating-src/ | gzip > apache-doris-0.9.0-incubating-src.tar.gz
$ gpg -u xxx@apache.org --armor --output apache-doris-0.9.0-incubating-src.tar.gz.asc --detach-sign apache-doris-0.9.0-incubating-src.tar.gz
$ gpg --verify apache-doris-0.9.0-incubating-src.tar.gz.asc apache-doris-0.9.0-incubating-src.tar.gz
$ sha512sum apache-doris-0.9.0-incubating-src.tar.gz > apache-doris-0.9.0-incubating-src.tar.gz.sha512
$ sha512sum --check apache-doris-0.9.0-incubating-src.tar.gz.sha512

8. 上传签名的软件包和 KEYS 文件到 DEV svn

首先，下载 svn 库：

svn co https://dist.apache.org/repos/dist/dev/incubator/doris/

将之前得到的全部文件组织成以下svn路径

./doris/
├── 0.9
│   └── 0.9.0-rc1
│       ├── apache-doris-0.9.0-incubating-src.tar.gz
│       ├── apache-doris-0.9.0-incubating-src.tar.gz.asc
│       ├── apache-doris-0.9.0-incubating-src.tar.gz.sha512
│       └── KEYS

上传这些文件

svn add 0.9.0-rc1
svn commit -m "Release Apache Doris (incubating) 0.9.0 rc1"

9. 发社区投票邮件

[VOTE] Release Apache Doris 0.9.0-incubating-rc01

Hi all,
Please review and vote on Apache Doris 0.9.0-incubating-rc01 release.
The release candidate has been tagged in GitHub as 0.9.0-rc01, available
here:
https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01
===== CHANGE LOG =====
New Features:
....
======================
Thanks to everyone who has contributed to this release.
The artifacts (source, signature and checksum) corresponding to this release
candidate can be found here:
https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc1/
This has been signed with PGP key 33DBF2E0, corresponding to
lide@apache.org.
KEYS file is available here:
https://dist.apache.org/repos/dist/dev/incubator/doris/KEYS
It is also listed here:
https://people.apache.org/keys/committer/lide.asc
To verify and build, you can refer to following wiki:
https://github.com/apache/incubator-doris/wiki/How-to-verify-Apache-Release
https://wiki.apache.org/incubator/IncubatorReleaseChecklist
The vote will be open for at least 72 hours.
[ ] +1 Approve the release
[ ] +0 No opinion
[ ] -1 Do not release this package because ...
Best Regards,
xxx

10. 投票通过后，发 Result 邮件

[Result][VOTE] Release Apache Doris 0.9.0-incubating-rc01

Thanks to everyone, and this vote is now closed.
It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
Binding:
+1 Zhao Chun
+1 xxx
+1 Li Chaoyong
+1 Mingyu Chen
Best Regards,
xxx

11. 发邮件到 general@incubator.apache.org 进行投票

[VOTE] Release Apache Doris 0.9.0-incubating-rc01

Hi all,
Please review and vote on Apache Doris 0.9.0-incubating-rc01 release.
Apache Doris is an MPP-based interactive SQL data warehousing for reporting and analysis.
The Apache Doris community has voted on and approved this release:
https://lists.apache.org/thread.html/d70f7c8a8ae448bf6680a15914646005c6483564464cfa15f4ddc2fc@%3Cdev.doris.apache.org%3E
The vote result email thread:
https://lists.apache.org/thread.html/64d229f0ba15d66adc83306bc8d7b7ccd5910ecb7e842718ce6a61da@%3Cdev.doris.apache.org%3E
The release candidate has been tagged in GitHub as 0.9.0-rc01, available here:
https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01
There is no CHANGE LOG file because this is the first release of Apache Doris.
Thanks to everyone who has contributed to this release, and there is a simple release notes can be found here:
https://github.com/apache/incubator-doris/issues/406
The artifacts (source, signature and checksum) corresponding to this release candidate can be found here:
https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/
This has been signed with PGP key 33DBF2E0, corresponding to lide@apache.org.
KEYS file is available here:
https://dist.apache.org/repos/dist/dev/incubator/doris/KEYS
It is also listed here:
https://people.apache.org/keys/committer/lide.asc
The vote will be open for at least 72 hours.
[ ] +1 Approve the release
[ ] +0 No opinion
[ ] -1 Do not release this package because ...
To verify and build, you can refer to following instruction:
Firstly, you must be install and start docker service, and then you could build Doris as following steps:
Step1: Pull the docker image with Doris building environment
$ docker pull apachedoris/doris-dev:build-env
You can check it by listing images, its size is about 3.28GB.
Step2: Run the Docker image
You can run image directly:
$ docker run -it apachedoris/doris-dev:build-env
Step3: Download Doris source
Now you should in docker environment, and you can download Doris source package.
(If you have downloaded source and it is not in image, you can map its path to image in Step2.)
$ wget https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/apache-doris-0.9.0.rc01-incubating-src.tar.gz
Step4: Build Doris
Now you can decompress and enter Doris source path and build Doris.
$ tar zxvf apache-doris-0.9.0.rc01-incubating-src.tar.gz
$ cd apache-doris-0.9.0.rc01-incubating-src
$ sh build.sh
Best Regards,
xxx

邮件的 thread 连接可以在这里找到：

https://lists.apache.org/list.html?dev@doris.apache.org

12. 发 Result 邮件到 general@incubator.apache.org

[RESULT][VOTE] Release Apache Doris 0.9.0-incubating-rc01

Hi,
Thanks to everyone, and the vote for releasing Apache Doris 0.9.0-incubating-rc01 is now closed.
It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
Binding:
+1 Willem Jiang
+1 Justin Mclean
+1 ShaoFeng Shi
+1 Makoto Yui
The vote thread:
https://lists.apache.org/thread.html/da05fdd8d84e35de527f27200b5690d7811a1e97d419d1ea66562130@%3Cgeneral.incubator.apache.org%3E
Best Regards,
xxx

13. 上传 package 到 release

当正式发布投票成功后，先发[Result]邮件，然后就准备 release package。将之前在dev下发布的对应rc文件夹下的源码包、签名文件和hash文件拷贝到另一个目录 0.9.0-incubating，注意文件名字中不要rcxx (可以rename，但不要重新计算签名，hash可以重新计算，结果不会变)

第一次发布的话 KEYS 文件也需要拷贝过来。然后add到svn release 下。


https://dist.apache.org/repos/dist/release/incubator/doris/0.9.0-incubating/
最终能在 apache 官网看到：
http://www.apache.org/dist/incubator/doris/0.9.0-incubating/

14. 发 Announce 邮件到 general@incubator.apache.org

Title:

[ANNOUNCE] Apache Doris (incubating) 0.9.0 Release

发送邮件组：

general@incubator.apache.org <general@incubator.apache.org>
dev@doris.apache.org <dev@doris.apache.org>

邮件正文：

Hi All,
We are pleased to announce the release of Apache Doris 0.9.0-incubating.
Apache Doris (incubating) is an MPP-based interactive SQL data warehousing for reporting and analysis.
The release is available at:
http://doris.apache.org/downloads.html
Thanks to everyone who has contributed to this release, and the release note can be found here:
https://github.com/apache/incubator-doris/releases
Best Regards,
On behalf of the Doris team,
xxx

15. 在 Doris 官网和 github 发布链接

15.1 创建下载链接

下载链接： http://www.apache.org/dyn/closer.cgi?filename=incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz&action=download

wget —trust-server-names “https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz“

原始位置: https://www.apache.org/dist/incubator/doris/0.9.0-incubating/

http://www.apache.org/dyn/closer.cgi/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz

源码包（source package）: http://www.apache.org/dyn/closer.cgi/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz

ASC: http://archive.apache.org/dist/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz.asc

sha512: http://archive.apache.org/dist/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz.sha512

KEYS: http://archive.apache.org/dist/incubator/doris/KEYS

refer to: http://www.apache.org/dev/release-download-pages#closer

15.2 准备 release note

需要修改如下两个地方：

1、Github 的 release 页面

https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01

2、Doris 官网下载页面

http://doris.apache.org/downloads.html