gs_upgradectl

背景信息

用户根据openGauss提供的新特性和数据库现状，确定是否对现有系统进行升级。

当前支持的升级模式为就地升级和灰度升级。升级方式的策略又分为大版本升级和小版本升级。

用户挑选升级方式后，系统会自动判断并选择合适的升级策略。

就地升级：升级期间需停止业务进行，一次性升级所有节点。

灰度升级：灰度升级支持全业务操作，也是一次性升级所有节点（openGauss1.1.0版本之后的版本支持该功能）。

滚动升级：滚动升级支持全业务操作，可先升级部分指定节点，在升级剩余节点（openGauss3.1.0版本之后的版本支持该功能）。

注意事项

  • 升级操作不能和扩容、缩容同时执行。
  • 不支持虚拟IP。
  • 升级过程中，不允许对wal_level、max_connections、max_prepared_transactions、max_locks_per_transaction这四个GUC参数的值进行修改。如果修改，会导致回滚后实例启动异常。
  • 建议在数据库系统空闲情况下进行升级，尽量避开业务繁忙的时间段（可按照经验判断，如节假日等）。
  • 升级前尽可能保证数据库正常。可以通过gs_om -t status查询，查询结果的cluster_state为Normal代表数据库正常。
  • 升级前保证数据库互信正常，可以在任意节点上，通过ssh hostname命令，连接另外一个节点进行验证。如果各机器间互连不用输入密码，说明互信正常（通常数据库状态正常时，互信一般都是正常的）。
  • 升级前后，数据库的部署方式（配置文件）不能发生变化。升级前会对部署方式进行校验，如果改变，会报错。
  • 升级前要保证操作系统处于健康状态，通过gs_checkos工具可以完成操作系统状态检查。
  • 就地升级需要停止业务，灰度升级支持全业务操作。
  • 数据库运行正常且主数据库节点的数据完全同步到备数据库节点。
  • 升级过程中不允许打开kerberos开关。
  • 请不要修改安装包中解压出来的version.cfg文件。
  • 如果升级过程中出现异常导致升级失败，需用户手动回滚，并且必须回滚成功后才能进行下一次升级。
  • 如果升级回滚成功后，再次升级成功，未提交阶段设置的GUC参数将失效。
  • 执行升级的过程中请不要手动设置GUC参数。
  • 灰度升级中，升级的时候都会产生不超过10s的业务中断。
  • 升级过程中，必须保持内核版本与om版本一致才可执行om操作。这里的一致是指，内核代码和om代码都来自同一个软件包。如果执行了升级包的前置脚本却没有升级，或者升级回滚后没有执行基线包的前置脚本，就会造成内核代码和om代码的不一致。
  • 升级过程中如果系统表新增了字段，升级后通过**\d命令将查看不到这些新增的字段。此时通过select**命令可以查到这些新增的字段。
  • 升级需要guc参数enable_stream_replication=on，该参数为off时不允许升级。
  • 灰度升级中， 业务并发要小于200并发读加200并发写的情况。
  • 建议数据库节点磁盘使用率低于80%时再执行升级操作。
  • 执行gs_upgradectl -t auto-upgrade 之后，没有提交之前，不能执行快照生成，即升级过程中不能执行快照生成。

语法

  • 显示帮助信息

    1. gs_upgradectl -? | --help

  • 显示版本号信息

    1. gs_upgradectl -V | --version

  • 选择升级策略

    1. gs_upgradectl -t chose-strategy [-l LOGFILE]

  • 自动升级openGauss

    1. gs_upgradectl -t auto-upgrade -X XMLFILE  [-l LOGFILE] [--grey]

  • 自动回滚升级

    1. gs_upgradectl -t auto-rollback -X XMLFILE [-l LOGFILE] [--force]

  • 升级提交

    1. gs_upgradectl -t commit-upgrade -X XMLFILE [-l LOGFILE]

    gs_upgradectl - 图1 说明：

    • 一旦提交操作完成，则不能再执行回滚操作。

参数说明

  • -t

    gs_upgradectl命令的类型。

    取值范围：chose-strategy、auto-upgrade、auto-rollback和commit-upgrade。

  • -l

    用于记录升级或回滚过程中的日志信息。

    取值范围：任意存在的可访问的绝对路径。

    默认值：/var/log/gaussdb/用户名/om/gs_upgradectl-YYYY-MM-DD_hhmmss.log

  • -?, –help

    显示帮助信息。

  • -V, –version

    显示版本号信息。

  • -X

    指定openGauss配置文件。

    取值范围：xml文件的存储路径。

  • –grey

    使用灰度升级方式来进行升级操作。

  • -h 升级部分指定节点，必须与–grey连用。

    Value range: 单节点 or 多节点

  • –continue

    升级剩余节点, 必须与–grey连用。

  • –force

    当openGauss状态不正常，无法支持正常回滚时，用此参数进行强制回滚操作。

示例

示例一：升级前使用新包进行前置操作。

  1. ./gs_preinstall -U roach -G users -X /data/xml/3node_3c3d_1m2s_etcd.xml 
  2. Parsing the configuration file.
  3. Successfully parsed the configuration file.
  4. Installing the tools on the local node.
  5. Successfully installed the tools on the local node.
  6. Setting pssh path
  7. Successfully set core path.
  8. Distributing package.
  9. Begin to distribute package to tool path.
  10. Successfully distribute package to tool path.
  11. Begin to distribute package to package path.
  12. Successfully distribute package to package path.
  13. Successfully distributed package.
  14. Installing the tools in the cluster.
  15. Successfully installed the tools in the cluster.
  16. Checking hostname mapping.
  17. Successfully checked hostname mapping.
  18. Checking OS version.
  19. Successfully checked OS version.
  20. Creating cluster's path.
  21. Successfully created cluster's path.
  22. Set and check OS parameter.
  23. Setting OS parameters.
  24. Successfully set OS parameters.
  25. Set and check OS parameter completed.
  26. Preparing CRON service.
  27. Successfully prepared CRON service.
  28. Preparing SSH service.
  29. Successfully prepared SSH service.
  30. Setting user environmental variables.
  31. Successfully set user environmental variables.
  32. Configuring alarms on the cluster nodes.
  33. Successfully configured alarms on the cluster nodes.
  34. Setting the dynamic link library.
  35. Successfully set the dynamic link library.
  36. Setting pssh path
  37. Successfully set pssh path.
  38. Setting Cgroup.
  39. Successfully set Cgroup.
  40. Set ARM Optimization.
  41. Successfully set ARM Optimization.
  42. Preinstalling secbox.
  43. Successfully installed secbox.
  44. Setting finish flag.
  45. Successfully set finish flag.
  46. Preinstallation succeeded.

**示例二：**使用gs_upgradectl脚本执行就地升级。

  1. gs_upgradectl -t upgrade -X /data/xml/3node_3c3d_1m2s_etcd.xml 
  2. Static configuration matched with old static configuration files.
  3. Performing inplace rollback.
  4. Rollback succeeded.
  5. Checking upgrade environment.
  6. Successfully checked upgrade environment.
  7. Successfully checked upgrade environment.
  8. Successfully started cluster.
  9. Start to do health check.
  10. Successfully checked cluster status.
  11. Backing up current application and configurations.
  12. Successfully backed up current application and configurations.
  13. Stopping the cluster.
  14. Successfully stopped cluster.
  15. Backing up cluster configuration.
  16. Successfully backup hotpatch config file.
  17. Successfully backed up cluster configuration.
  18. Installing new binary.
  19. Restoring cluster configuration.
  20. Successfully restored cluster configuration.
  21. Modifying the socket path.
  22. Successfully modified socket path.
  23. Switch symbolic link to new binary directory.
  24. Successfully switch symbolic link to new binary directory.
  25. Successfully started cluster.
  26. Start to do health check.
  27. Successfully checked cluster status.
  28. Upgrade main process has been finished, user can do some check now.
  29. Once the check done, please execute following command to commit upgrade:
  31.     gs_upgradectl -t commit-upgrade -X /data/xml/3node_3c3d_1m2s_etcd.xml

**示例三：**使用gs_upgradectl脚本执行自动回滚，撤销已经成功/失败的升级操作（升级回滚）。

  1. gs_upgradectl -t auto-rollback -X /data/xml/3node_3c3d_1m2s_etcd.xml
  2. Static configuration matched with old static configuration files.
  3. Performing inplace rollback.
  4. Checking static configuration files.
  5. Successfully checked static configuration files.
  6. Restoring cluster configuration.
  7. Successfully rollback hotpatch config file.
  8. Successfully restored cluster configuration.
  9. Switch symbolic link to old binary directory.
  10. Successfully switch symbolic link to old binary directory.
  11. Stopping the cluster.
  12. Successfully stopped cluster.
  13. Restoring application and configurations.
  14. Successfully restored application and configuration.
  15. Restoring cluster configuration.
  16. Successfully rollback hotpatch config file.
  17. Successfully restored cluster configuration.
  18. Successfully started cluster.
  19. Successfully cleaned new install path.
  20. Rollback succeeded.

**示例四：**使用gs_upgradectl脚本执行升级后提交（升级提交）。

  1. gs_upgradectl -t commit -X /data/xml/3node_3c3d_1m2s_etcd.xml
  2. Old cluster app path is /data/gauss/app_e67b8bcd
  3. Successfully Cleaned old install path.
  4. Commit binary upgrade succeeded.

**示例五：**使用gs_upgradectl脚本执行滚动升级。.

升级部分指定节点

  1. gs_upgradectl -t auto-upgrade -X /data/node2.xml --grey -h hostname0
  2. Static configuration matched with old static configuration files.
  3. Successfully set upgrade_mode to 0.
  4. Checking upgrade environment.
  5. Successfully checked upgrade environment.
  6. Start to do health check.
  7. Successfully checked cluster status.
  8. Upgrade nodes ['hostname0'].
  9. NOTICE: The directory /data/install/app_oldcommitid will be deleted after commit-upgrade, please make sure there is no personal data.
  10. Performing grey rollback.
  11. No need to rollback.
  12. The directory /data/install/app_oldcommitid will be deleted after commit-upgrade, please make sure there is no personal data.
  13. Installing new binary.
  14. copy certs from /data/install/app_oldcommitid to /data/install/app_newcommitid.
  15. Successfully copy certs from /data/install/app_oldcommitid to /data/install/app_newcommitid.
  16. Successfully backup hotpatch config file.
  17. Sync cluster configuration.
  18. Successfully synced cluster configuration.
  19. Switch symbolic link to new binary directory.
  20. Successfully switch symbolic link to new binary directory.
  21. Switching all db processes.
  22. Check cluster state.
  23. Create checkpoint before switching.
  24. Switching DN processes.
  25. Ready to grey start cluster.
  26. Grey start cluster successfully.
  27. Wait for the cluster status normal or degrade.
  28. Successfully switch all process version
  29. The nodes ['hostname0'] have been successfully upgraded to new version. Then do health check.
  30. Start to do health check.
  31. Successfully checked cluster status.
  32. The nodes ['hostname0']ve been successfully upgraded.Then can upgrade the remaining nodes.

升级剩余节点

  1. gs_upgradectl -t auto-upgrade -X /data/node2.xml --grey --continue
  2. Static configuration matched with old static configuration files.
  3. Checking upgrade environment.
  4. ['hostname0'] node have been upgrade, can upgrade the remaining nodes.
  5. Successfully checked upgrade environment.
  6. Start to do health check.
  7. Successfully checked cluster status.
  8. Successfully backup hotpatch config file.
  9. Sync cluster configuration.
  10. Successfully synced cluster configuration.
  11. Switch symbolic link to new binary directory.
  12. Successfully switch symbolic link to new binary directory.
  13. Switching all db processes.
  14. Check cluster state.
  15. Create checkpoint before switching.
  16. Switching DN processes.
  17. Ready to grey start cluster.
  18. Grey start cluster successfully.
  19. Wait for the cluster status normal or degrade.
  20. Successfully switch all process version
  21. The nodes ['hostname1'] have been successfully upgraded to new version. Then do health check.
  22. Start to do health check.
  23. Successfully checked cluster status.
  24. Waiting for the cluster status to become normal.
  25. .
  26. The cluster status is normal.
  27. Create checkpoint before switching.
  28. Upgrade main process has been finished, user can do some check now.
  29. Once the check done, please execute following command to commit upgrade:
  31. gs_upgradectl -t commit-upgrade -X /data/node2.xml
  32. Successfully upgrade all nodes.