如何发布

本教程只适用于 Apache Kylin Committers。
以在 Mac OS X 上的 Shell 命令作为样例。
对于中国用户,请谨慎使用代理以避免潜在的防火墙问题。

建立账户

确保您有可使用的账号且对以下应用有权限:

软件要求

  • Java 8 or above;
  • Maven 3.5.3 或更高版本。
  • 如果你是用 Mac OS X 做发布, 请安装 GNU TAR, 按照 此文章.

设置 GPG 签名密钥

按照 http://www.apache.org/dev/release-signing 上的说明创建密钥对
安装 gpg (以 Mac OS X 为例):
brew install gpg

生成 gpg 密钥:
参考: https://www.gnupg.org/gph/en/manual/c14.html
生成的所有新 RSA 密钥应至少为 4096 位。不要生成新的 DSA 密钥
gpg --full-generate-key

验证您的密钥:
gpg --list-sigs YOUR_NAME

获取密钥的指纹:
gpg --fingerprint YOUR_NAME

它将显示指纹,如 “Key fingerprint = XXXX XXXX …“,然后在 https://id.apache.org/ 上的”OpenPGP Public Key Primary Fingerprint”字段处将指纹添加到您的 apache 帐户;等待几个小时,密钥将添加到 https://people.apache.org/keys/,例如:
https://people.apache.org/keys/committer/lukehan.asc

生成 ASCII Amromed 键:
gpg -a --export YOUR_MAIL_ADDRESS > YOUR_NAME.asc &

上传密钥到公共服务器:
gpg --send-keys YOUR_KEY_HASH

或通过 web 提交密钥:
打开并提交到 http://pool.sks-keyservers.net:11371 (您可以选择任意一个有效的公钥服务器)

一旦您的密钥提交到服务器,您可以通过使用以下命令验证:
gpg --recv-keys YOUR_KEY_HASH
举例:
gpg --recv-keys 027DC364

按照 KEYS 文件中的说明将公钥添加到 KEYS 文件:
KEYS 文件位于: ${kylin}/KEYS
例如:
(gpg --list-sigs YOURNAME && gpg --armor --export YOURNAME) >> KEYS

提交您的改动。

准备 release 的工件

开始前:

  • 如上所述设置签名密钥。
  • 确保您使用的是 JDK 1.8。
  • 确保您使用的是 GIT 2.7.2 或更高版本。
  • 确保您使用的是正确的 release 版本号。
  • 确保每个“resolved”的 JIRA 案例(包括重复案例)都分配了修复版本。
  • 确保你在干净的目录工作

在 Maven 中配置 Apache 存储库服务器
如果您是第一次发布,您需要在 ~/.m2/settings.xml 中服务器授权信息;如果该文件不存在,从 $M2_HOME/conf/settings.xml 拷贝一个模板;

在“服务器”部分中,确保添加以下服务器,并将 #YOUR_APACHE_ID#, #YOUR_APACHE_PWD#, #YOUR_GPG_PASSPHRASE# 替换为您的 ID,密码和口令:

  1. <servers>
  2. <!-- To publish a snapshot of some part of Maven -->
  3. <server>
  4. <id>apache.snapshots.https</id>
  5. <username>#YOUR_APACHE_ID#</username>
  6. <password>#YOUR_APACHE_PWD#</password>
  7. </server>
  8. <!-- To stage a release of some part of Maven -->
  9. <server>
  10. <id>apache.releases.https</id>
  11. <username>#YOUR_APACHE_ID#</username>
  12. <password>#YOUR_APACHE_PWD#</password>
  13. </server>
  14. <!-- To publish a website of some part of Maven -->
  15. <server>
  16. <id>apache.website</id>
  17. <username>#YOUR_APACHE_ID#</username>
  18. <password>#YOUR_APACHE_PWD#</password>
  19. <!-- Either
  20. <privateKey>...</privateKey>
  21. -->
  22. <filePermissions>664</filePermissions>
  23. <directoryPermissions>775</directoryPermissions>
  24. </server>
  25. <!-- To stage a website of some part of Maven -->
  26. <server>
  27. <id>stagingSite</id>
  28. <!-- must match hard-coded repository identifier in site:stage-deploy -->
  29. <username>#YOUR_APACHE_ID#</username>
  30. <filePermissions>664</filePermissions>
  31. <directoryPermissions>775</directoryPermissions>
  32. </server>
  33. <server>
  34. <id>gpg.passphrase</id>
  35. <passphrase>#YOUR_GPG_PASSPHRASE#</passphrase>
  36. </server>
  37. </servers>

修复许可证问题

  1. # Set passphrase variable without putting it into shell history
  2. $ read -s GPG_PASSPHRASE
  3. # Make sure that there are no junk files in the sandbox
  4. $ git clean -xf
  5. $ mvn clean
  6. # Make sure all unit tests are passed
  7. $ mvn test
  8. # Check the `org.apache.kylin.common.KylinVersion` class, ensure the value of `CURRENT_KYLIN_VERSION` is the release version.
  9. # Fix any license issues as reported by target/rat.txt
  10. $ mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_PASSPHRASE} install

可选的,当 dry-run 成功了,将安装变为部署:

  1. $ mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_PASSPHRASE} deploy

准备
检查并确保你可以 ssh 连接到 github:

  1. ssh -T git@github.com

基于要当前的开发分支,创建一个以 release 版本号命名的发布分支,例如,v2.5.0-release (注意分支名字不能与tag名字相同),并将其推到服务器端。

  1. $ git checkout -b vX.Y.Z-release
  2. $ git push -u origin vX.Y.Z-release

如果任何步骤失败,请清理(见下文),解决问题,然后从头重新开始。

  1. # Set passphrase variable without putting it into shell history
  2. $ read -s GPG_PASSPHRASE
  3. # Make sure that there are no junk files in the sandbox
  4. $ git clean -xf
  5. $ mvn clean
  6. # 可选的, do a dry run of the release:prepare step, which sets version numbers.
  7. $ mvn -DdryRun=true -DskipTests -DreleaseVersion=X.Y.Z -DdevelopmentVersion=(X.Y.Z+1)-SNAPSHOT -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests" release:prepare 2>&1 | tee /tmp/prepare-dry.log

查看 dry run 输出:

  • target 目录中应该是这 8 个文件,其中包括:
    • apache-kylin-X.Y.Z-SNAPSHOT-source-release.zip
    • apache-kylin-X.Y.Z-SNAPSHOT-source-release.zip.asc
    • apache-kylin-X.Y.Z-SNAPSHOT-source-release.zip.asc.sha256
    • apache-kylin-X.Y.Z-SNAPSHOT-source-release.zip.sha256
  • 移除 .zip.asc.sha256 文件因为不需要。
  • 注意文件名以 apache-kylin- 开始
  • 在源发行版 .zip 文件中,检查所有文件是否属于名为 apache-kylin-X.Y.Z-SNAPSHOT 的目录。
  • 该目录必须包含 NOTICE, LICENSE, README.md 文件
  • 检查 PGP。

运行真实的 release:
现在真正开始 release

  1. # 如果之前做了dry run,在真正开始release之前需要做清理
  2. # Prepare sets the version numbers, creates a tag, and pushes it to git.
  3. $ mvn -DskipTests -DreleaseVersion=X.Y.Z -DdevelopmentVersion=(X.Y.Z+1)-SNAPSHOT -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests" release:prepare
  4. # Perform checks out the tagged version, builds, and deploys to the staging repository
  5. $ mvn -DskipTests -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests" release:perform

一个失败的 release 尝试后进行清理:

  1. # Make sure that the tag you are about to generate does not already
  2. # exist (due to a failed release attempt)
  3. $ git tag
  4. # If the tag exists, delete it locally and remotely
  5. $ git tag -d kylin-X.Y.Z
  6. $ git push origin :refs/tags/kylin-X.Y.Z
  7. # Remove modified files
  8. $ mvn release:clean
  9. # Check whether there are modified files and if so, go back to the
  10. # original git commit
  11. $ git status
  12. $ git reset --hard HEAD

关闭 Nexus 仓库中的阶段性工件:

上传到临时区域:
通过 subversion 将工件上传到临时区域,https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-X.Y.Z-rcN:

  1. # Create a subversion workspace, if you haven't already
  2. $ mkdir -p ~/dist/dev
  3. $ pushd ~/dist/dev
  4. $ svn co https://dist.apache.org/repos/dist/dev/kylin
  5. $ popd
  6. ## Move the files into a directory
  7. $ cd target
  8. $ mkdir ~/dist/dev/kylin/apache-kylin-X.Y.Z-rcN
  9. $ mv apache-kylin-* ~/dist/dev/kylin/apache-kylin-X.Y.Z-rcN
  10. ## Remove the .zip.asc.sha256 file as it is not needed.
  11. $ rm ~/dist/dev/kylin/apache-kylin-X.Y.Z-rcN/apache-kylin-X.Y.Z-SNAPSHOT-source-release.zip.asc.sha256
  12. ## Check in
  13. $ cd ~/dist/dev/kylin
  14. $ svn add apache-kylin-X.Y.Z-rcN
  15. $ svn commit -m 'Upload release artifacts to staging' --username <YOUR_APACHE_ID>

验证 release

  1. # Check unit test
  2. $ mvn test
  3. # Check that the signing key (e.g. 2AD3FAE3) is pushed
  4. $ gpg --recv-keys key
  5. # Check keys
  6. $ curl -O https://dist.apache.org/repos/dist/release/kylin/KEYS
  7. # Sign/check sha256 hashes
  8. # (Assumes your O/S has a 'shasum' command.)
  9. function checkHash() {
  10. cd "$1"
  11. for i in *.{pom,gz}; do
  12. if [ ! -f $i ]; then
  13. continue
  14. fi
  15. if [ -f $i.sha256 ]; then
  16. if [ "$(cat $i.sha256)" = "$(shasum -a 256 $i)" ]; then
  17. echo $i.sha256 present and correct
  18. else
  19. echo $i.sha256 does not match
  20. fi
  21. else
  22. shasum -a 256 $i > $i.sha256
  23. echo $i.sha256 created
  24. fi
  25. done
  26. };
  27. $ checkHash apache-kylin-X.Y.Z-rcN

Apache 投票过程

在 Apache Kylin dev 邮件列表上投票
在 dev 邮件列表上进行 release 投票,使用由 Maven release plugin 生成的 commit id,其消息看起来像 “[maven-release-plugin] prepare release kylin-x.x.x”:

  1. To: dev@kylin.apache.org
  2. Subject: [VOTE] Release apache-kylin-X.Y.Z (RC[N])
  3. Hi all,
  4. I have created a build for Apache Kylin X.Y.Z, release candidate N.
  5. Changes highlights:
  6. ...
  7. Thanks to everyone who has contributed to this release.
  8. Heres release notes:
  9. https://github.com/apache/kylin/blob/XXX/docs/release_notes.md
  10. The commit to be voted upon:
  11. https://github.com/apache/kylin/commit/xxx
  12. Its hash is xxx.
  13. The artifacts to be voted on are located here:
  14. https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-X.Y.Z-rcN/
  15. The hash of the artifact is as follows:
  16. apache-kylin-X.Y.Z-source-release.zip.sha256 xxx
  17. A staged Maven repository is available for review at:
  18. https://repository.apache.org/content/repositories/orgapachekylin-XXXX/
  19. Release artifacts are signed with the following key:
  20. https://people.apache.org/keys/committer/lukehan.asc
  21. Please vote on releasing this package as Apache Kylin X.Y.Z.
  22. The vote is open for the next 72 hours and passes if a majority of
  23. at least three +1 PMC votes are cast.
  24. [ ] +1 Release this package as Apache Kylin X.Y.Z
  25. [ ] 0 I don't feel strongly about it, but I'm okay with the release
  26. [ ] -1 Do not release this package because...
  27. Here is my vote:
  28. +1 (binding)

投票完成后,发出结果:

  1. Subject: [RESULT][VOTE] Release apache-kylin-X.Y.Z (RC[N])
  2. To: dev@kylin.apache.org
  3. Thanks to everyone who has tested the release candidate and given
  4. their comments and votes.
  5. The tally is as follows.
  6. N binding +1s:
  7. N non-binding +1s:
  8. No 0s or -1s.
  9. Therefore I am delighted to announce that the proposal to release
  10. Apache-Kylin-X.Y.Z has passed.

发布

成功发布投票后,我们需要推动发行到镜像,以及其它任务。

在 JIRA 中,搜索
all issues resolved in this release,
并进行批量更新,将它们的状态更改为“关闭”,
并加上更改的评论
“Resolved in release X.Y.Z (YYYY-MM-DD)”
(填写适当的发布号和日期)。
取消 “Send mail for this update”。

标记 JIRA 系统中发布的版本,管理版本

推广分阶段的 nexus 工件。

  • 转到 https://repository.apache.org/ 并登陆
  • 在 “Build Promotion” 下点击 “Staging Repositories”
  • 在 “orgapachekylin-xxxx” 行中,选中框
  • 点击 “Release” 按钮

将工件检入 svn。

  1. # Get the release candidate.
  2. $ mkdir -p ~/dist/dev
  3. $ cd ~/dist/dev
  4. $ svn co https://dist.apache.org/repos/dist/dev/kylin
  5. # Copy the artifacts. Note that the copy does not have '-rcN' suffix.
  6. $ mkdir -p ~/dist/release
  7. $ cd ~/dist/release
  8. $ svn co https://dist.apache.org/repos/dist/release/kylin
  9. $ cd kylin
  10. $ mkdir apache-kylin-X.Y.Z
  11. $ cp -rp ../../dev/kylin/apache-kylin-X.Y.Z-rcN/apache-kylin* apache-kylin-X.Y.Z/
  12. $ svn add apache-kylin-X.Y.Z
  13. # Check in.
  14. svn commit -m 'checkin release artifacts'

Svnpubsub 将会发布到
https://dist.apache.org/repos/dist/release/kylin 并会在 24 小时内传播到
http://www.apache.org/dyn/closer.cgi/kylin

如果现在有超过 2 个版本,请清除最旧的版本:

  1. cd ~/dist/release/kylin
  2. svn rm apache-kylin-X.Y.Z
  3. svn commit -m 'Remove old release'

旧版本将保留在 release archive.

在 JIRA 中发布相同版本,检查最新发布版本的更改日志。

构建和上传二进制包

发布后,您需要生成二进制包并将它们放入到 VPN 发布库中;

  • 使用 git fetch --all --prune --tags 来同步您本地和远程的仓库。
  • Git 检出当前发布的标签;
  • 通过参考此文档制作二进制包;
  • 使用 gpg 对生成的二进制包进行签名,例如:
  1. gpg --armor --output apache-kylin-2.5.0-bin.tar.gz.asc --detach-sig apache-kylin-2.5.0-bin.tar.gz
  • 生成二进制包的 sha256 文件,例如:
  1. shasum -a 256 apache-kylin-2.5.0-bin.tar.gz > apache-kylin-2.5.0-bin.tar.gz.sha256
  2. on Linux:
  3. openssl sha256 apache-kylin-2.5.0-bin.tar.gz > apache-kylin-2.5.0-bin.tar.gz.sha256
  • 将二进制包,签名文件和 sha256 文件推送到 svn dev 仓库,然后运行 svn mv <files-in-dev> <files-in-release> 命令将他们移动到 svn release 仓库。
  • 对于不同的 Hadoop/HBase 版本,您可能需要上述步骤;
  • 添加文件,然后将更改提交 svn。

更新源码

发布后,您需要更新一些源代码:

  • 把分布分支,如v2.5.0-release,合并到开发分支中,如2.5.x,以便开始下个版本的开发。

  • 手动更新 KylinVersion 类,将 CURRENT_KYLIN_VERSION 的值更改为当前开发版本。

发布网站

更多细节参考如何写文档

发送通知邮件到邮件列表

发送一个邮件主题如 “[Announce] Apache Kylin x.y.z released” 到以下列表:

  • Apache Kylin Dev 邮箱列表: dev@kylin.apache.org
  • Apache Kylin User 邮箱列表: user@kylin.apache.org
  • Apache Announce 邮箱列表: announce@apache.org
    请注意始终使用您的 Apache 邮件地址发送;

这是一个公告电子邮件的样本(通过研究 Kafka):

  1. The Apache Kylin team is pleased to announce the immediate availability of the 2.5.0 release.
  2. This is a major release after 2.4, with more than 100 bug fixes and enhancements; All of the changes in this release can be found in:
  3. https://kylin.apache.org/docs/release_notes.html
  4. You can download the source release and binary packages from Apache Kylin's download page: https://kylin.apache.org/download/
  5. Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Apache Hadoop, supporting extremely large datasets.
  6. Apache Kylin lets you query massive data set at sub-second latency in 3 steps:
  7. 1. Identify a star schema or snowflake schema data set on Hadoop.
  8. 2. Build Cube on Hadoop.
  9. 3. Query data with ANSI-SQL and get results in sub-second, via ODBC, JDBC or RESTful API.
  10. Thanks everyone who have contributed to the 2.1.0 release.
  11. We welcome your help and feedback. For more information on how to
  12. report problems, and to get involved, visit the project website at
  13. https://kylin.apache.org/

感谢

本指南起草于 Apache Calcite Howto doc 的参考资料,非常感谢。