检查办法

通过openGauss提供的gs_check工具可以开展openGauss健康状态检查。

注意事项

  • 扩容新节点检查只能在root用户下执行，其他场景都必须在omm用户下执行。
  • 必须指定-i或-e参数，-i会检查指定的单项，-e会检查对应场景配置中的多项。
  • 如果-i参数中不包含root类检查项或-e场景配置列表中没有root类检查项，则不需要交互输入root权限的用户及其密码。
  • 可使用–skip-root-items跳过检查项中包含的root类检查，以免需要输入root权限用户及密码。
  • 检查扩容新节点与现有节点之间的一致性，在现有节点执行gs_check命令指定–hosts参数进行检查，其中hosts文件中需要写入新节点ip。

操作步骤

方式1：

  1. 以操作系统用户omm登录数据库主节点。

  2. 执行如下命令对openGauss数据库状态进行检查。

    1. gs_check -i CheckClusterState

    其中，-i指定检查项，注意区分大小写。格式：-i CheckClusterState、-i CheckCPU或-i CheckClusterState，CheckCPU。

    取值范围为所有支持的检查项名称，详细列表请参见《openGauss 工具参考》中“服务端工具 > gs_checkos > openGauss状态检查表”，用户可以根据需求自己编写新检查项。

方式2：

  1. 以操作系统用户omm登录数据库主节点。

  2. 执行如下命令对openGauss数据库进行健康检查。

    1. gs_check -e inspect

    其中，-e指定场景名，注意区分大小写。格式：-e inspect或-e upgrade。

    取值范围为所有支持的巡检场景名称，默认列表包括：inspect（例行巡检）、upgrade（升级前巡检）、expand（扩容前巡检）、binary_upgrade（就地升级前巡检）、health（健康检查巡检），用户可以根据需求自己编写场景。

方式3：

  1. 以操作系统用户omm登录数据库主节点。
  2. 将巡检工具gs_check及相关目录inspection拷贝分发到所有扩容新节点。
  3. 将扩容新节点ip写到文件_ipListFile_中，以换行符进行分隔。

  4. 执行如下命令对扩容新节点进行扩容前检查。

    1. gs_check -e expand_new_node --hosts ipListFile

    -e必须为expand_new_node，即扩容前新节点检查。

openGauss巡检的主要作用是在openGauss运行过程中，检查整个openGauss状态是否正常，或者重大操作前（升级、扩容），确保openGauss满足操作所需的环境条件和状态条件。详细的巡检项目和场景请参见《openGauss 工具参考》中“服务端工具 > gs_checkos > openGauss状态检查表”。

示例

执行单项检查结果：

  1. perfadm@lfgp000700749:/opt/huawei/perfadm/tool/script> gs_check -i CheckCPU
  2. Parsing the check items config file successfully
  3. Distribute the context file to remote hosts successfully
  4. Start to health check for the cluster. Total Items:1 Nodes:3
  6. Checking...               [=========================] 1/1
  7. Start to analysis the check result
  8. CheckCPU....................................OK
  9. The item run on 3 nodes.  success: 3
  11. Analysis the check result successfully
  12. Success. All check items run completed. Total:1  Success:1  Failed:0
  13. For more information please refer to /opt/huawei/wisequery/script/gspylib/inspection/output/CheckReport_201902193704661604.tar.gz

本地执行结果：

  1. perfadm@lfgp000700749:/opt/huawei/perfadm/tool/script> gs_check -i CheckCPU -L
  3. 2017-12-29 17:09:29 [NAM] CheckCPU
  4. 2017-12-29 17:09:29 [STD] 检查主机CPU占用率，如果idle 大于30%并且iowait 小于 30%.则检查项通过，否则检查项不通过
  5. 2017-12-29 17:09:29 [RST] OK
  7. 2017-12-29 17:09:29 [RAW]
  8. Linux 4.4.21-69-default (lfgp000700749)  12/29/17  _x86_64_
  10. 17:09:24        CPU     %user     %nice   %system   %iowait    %steal     %idle
  11. 17:09:25        all      0.25      0.00      0.25      0.00      0.00     99.50
  12. 17:09:26        all      0.25      0.00      0.13      0.00      0.00     99.62
  13. 17:09:27        all      0.25      0.00      0.25      0.13      0.00     99.37
  14. 17:09:28        all      0.38      0.00      0.25      0.00      0.13     99.25
  15. 17:09:29        all      1.00      0.00      0.88      0.00      0.00     98.12
  16. Average:        all      0.43      0.00      0.35      0.03      0.03     99.17

执行场景检查结果：

  1. [perfadm@SIA1000131072 Check]$ gs_check -e inspect
  2. Parsing the check items config file successfully
  3. The below items require root privileges to execute:[CheckBlockdev CheckIOrequestqueue CheckIOConfigure CheckCheckMultiQueue CheckFirewall CheckSshdService CheckSshdConfig CheckCrondService CheckNoCheckSum CheckSctpSeProcMemory CheckBootItems CheckFilehandle CheckNICModel CheckDropCache]
  4. Please enter root privileges user[root]:root
  5. Please enter password for user[root]:
  6. Please enter password for user[root] on the node[10.244.57.240]:
  7. Check root password connection successfully
  8. Distribute the context file to remote hosts successfully
  9. Start to health check for the cluster. Total Items:59 Nodes:2
  11. Checking...               [                         ] 21/59
  12. Checking...               [=========================] 59/59
  13. Start to analysis the check result
  14. CheckClusterState...........................OK
  15. The item run on 2 nodes.  success: 2
  17. CheckDBParams...............................OK
  18. The item run on 1 nodes.  success: 1
  20. CheckDebugSwitch............................OK
  21. The item run on 2 nodes.  success: 2
  23. CheckDirPermissions.........................OK
  24. The item run on 2 nodes.  success: 2
  26. CheckReadonlyMode...........................OK
  27. The item run on 1 nodes.  success: 1
  29. CheckEnvProfile.............................OK
  30. The item run on 2 nodes.  success: 2  (consistent)
  31. The success on all nodes value:
  32. GAUSSHOME        /usr1/gaussdb/app
  33. LD_LIBRARY_PATH  /usr1/gaussdb/app/lib
  34. PATH             /usr1/gaussdb/app/bin
  37. CheckBlockdev...............................OK
  38. The item run on 2 nodes.  success: 2
  40. CheckCurConnCount...........................OK
  41. The item run on 1 nodes.  success: 1
  43. CheckCursorNum..............................OK
  44. The item run on 1 nodes.  success: 1
  46. CheckPgxcgroup..............................OK
  47. The item run on 1 nodes.  success: 1
  49. CheckDiskFormat.............................OK
  50. The item run on 2 nodes.  success: 2
  52. CheckSpaceUsage.............................OK
  53. The item run on 2 nodes.  success: 2
  55. CheckInodeUsage.............................OK
  56. The item run on 2 nodes.  success: 2
  58. CheckSwapMemory.............................OK
  59. The item run on 2 nodes.  success: 2
  61. CheckLogicalBlock...........................OK
  62. The item run on 2 nodes.  success: 2
  64. CheckIOrequestqueue.....................WARNING
  65. The item run on 2 nodes.  warning: 2
  66. The warning[host240,host157] value:
  67. On device (vdb) 'IO Request' RealValue '256' ExpectedValue '32768'
  68. On device (vda) 'IO Request' RealValue '256' ExpectedValue '32768'
  70. CheckMaxAsyIOrequests.......................OK
  71. The item run on 2 nodes.  success: 2
  73. CheckIOConfigure............................OK
  74. The item run on 2 nodes.  success: 2
  76. CheckMTU....................................OK
  77. The item run on 2 nodes.  success: 2  (consistent)
  78. The success on all nodes value:
  79. 1500
  81. CheckPing...................................OK
  82. The item run on 2 nodes.  success: 2
  84. CheckRXTX...................................NG
  85. The item run on 2 nodes.  ng: 2
  86. The ng[host240,host157] value:
  87. NetWork[eth0]
  88. RX: 256
  89. TX: 256
  92. CheckNetWorkDrop............................OK
  93. The item run on 2 nodes.  success: 2
  95. CheckMultiQueue.............................OK
  96. The item run on 2 nodes.  success: 2
  98. CheckEncoding...............................OK
  99. The item run on 2 nodes.  success: 2  (consistent)
  100. The success on all nodes value:
  101. LANG=en_US.UTF-8
  103. CheckFirewall...............................OK
  104. The item run on 2 nodes.  success: 2
  106. CheckKernelVer..............................OK
  107. The item run on 2 nodes.  success: 2  (consistent)
  108. The success on all nodes value:
  109. 3.10.0-957.el7.x86_64
  111. CheckMaxHandle..............................OK
  112. The item run on 2 nodes.  success: 2
  114. CheckNTPD...................................OK
  115. host240: NTPD service is running, 2020-06-02 17:00:28
  116. host157: NTPD service is running, 2020-06-02 17:00:06
  119. CheckOSVer..................................OK
  120. host240: The current OS is centos 7.6 64bit.
  121. host157: The current OS is centos 7.6 64bit.
  124. CheckSysParams..........................WARNING
  125. The item run on 2 nodes.  warning: 2
  126. The warning[host240,host157] value:
  127. Warning reason: variable 'net.ipv4.tcp_retries1' RealValue '3' ExpectedValue '5'.
  128. Warning reason: variable 'net.ipv4.tcp_syn_retries' RealValue '6' ExpectedValue '5'.
  129. Warning reason: variable 'net.sctp.path_max_retrans' RealValue '5' ExpectedValue '10'.
  130. Warning reason: variable 'net.sctp.max_init_retransmits' RealValue '8' ExpectedValue '10'.
  133. CheckTHP....................................OK
  134. The item run on 2 nodes.  success: 2
  136. CheckTimeZone...............................OK
  137. The item run on 2 nodes.  success: 2  (consistent)
  138. The success on all nodes value:
  139. +0800
  141. CheckCPU....................................OK
  142. The item run on 2 nodes.  success: 2
  144. CheckSshdService............................OK
  145. The item run on 2 nodes.  success: 2
  147. CheckSshdConfig.........................WARNING
  148. The item run on 2 nodes.  warning: 2
  149. The warning[host240,host157] value:
  151. Warning reason: UseDNS parameter is not set; expected: no
  153. CheckCrondService...........................OK
  154. The item run on 2 nodes.  success: 2
  156. CheckStack..................................OK
  157. The item run on 2 nodes.  success: 2  (consistent)
  158. The success on all nodes value:
  159. 8192
  161. CheckNoCheckSum.............................OK
  162. The item run on 2 nodes.  success: 2  (consistent)
  163. The success on all nodes value:
  164. Nochecksum value is N,Check items pass.
  166. CheckSysPortRange...........................OK
  167. The item run on 2 nodes.  success: 2
  169. CheckMemInfo................................OK
  170. The item run on 2 nodes.  success: 2  (consistent)
  171. The success on all nodes value:
  172. totalMem: 31.260929107666016G
  174. CheckHyperThread............................OK
  175. The item run on 2 nodes.  success: 2
  177. CheckTableSpace.............................OK
  178. The item run on 1 nodes.  success: 1
  180. CheckSctpService............................OK
  181. The item run on 2 nodes.  success: 2
  183. CheckSysadminUser...........................OK
  184. The item run on 1 nodes.  success: 1
  186. CheckGUCConsistent..........................OK
  187. All DN instance guc value is consistent.
  189. CheckMaxProcMemory..........................OK
  190. The item run on 1 nodes.  success: 1
  192. CheckBootItems..............................OK
  193. The item run on 2 nodes.  success: 2
  195. CheckHashIndex..............................OK
  196. The item run on 1 nodes.  success: 1
  198. CheckPgxcRedistb............................OK
  199. The item run on 1 nodes.  success: 1
  201. CheckNodeGroupName..........................OK
  202. The item run on 1 nodes.  success: 1
  204. CheckTDDate.................................OK
  205. The item run on 1 nodes.  success: 1
  207. CheckDilateSysTab...........................OK
  208. The item run on 1 nodes.  success: 1
  210. CheckKeyProAdj..............................OK
  211. The item run on 2 nodes.  success: 2
  213. CheckProStartTime.......................WARNING
  214. host157:
  215. STARTED COMMAND
  216. Tue Jun  2 16:57:18 2020 /usr1/dmuser/dmserver/metricdb1/server/bin/gaussdb --single_node -D /usr1/dmuser/dmb1/data -p 22204
  217. Mon Jun  1 16:15:15 2020 /usr1/gaussdb/app/bin/gaussdb -D /usr1/gaussdb/data/dn1 -M standby
  220. CheckFilehandle.............................OK
  221. The item run on 2 nodes.  success: 2
  223. CheckRouting................................OK
  224. The item run on 2 nodes.  success: 2
  226. CheckNICModel...............................OK
  227. The item run on 2 nodes.  success: 2  (consistent)
  228. The success on all nodes value:
  229. version: 1.0.0
  230. model: Red Hat, Inc. Virtio network device
  233. CheckDropCache..........................WARNING
  234. The item run on 2 nodes.  warning: 2
  235. The warning[host240,host157] value:
  236. No DropCache process is running
  238. CheckMpprcFile..............................NG
  239. The item run on 2 nodes.  ng: 2
  240. The ng[host240,host157] value:
  241. There is no mpprc file
  243. Analysis the check result successfully
  244. Failed. All check items run completed. Total:59   Success:52   Warning:5   NG:2
  245. For more information please refer to /usr1/gaussdb/tool/script/gspylib/inspection/output/CheckReport_inspect611.tar.gz