Validating an installation

You can check the status of an OKD cluster after an installation by following the procedures in this document.

Reviewing the installation log

You can review a summary of an installation in the OKD installation log. If an installation succeeds, the information required to access the cluster is included in the log.

Prerequisites

  • You have access to the installation host.

Procedure

  • Review the .openshift_install.log log file in the installation directory on your installation host:

    1. $ cat <install_dir>/.openshift_install.log

    Example output

    Cluster credentials are included at the end of the log if the installation is successful, as outlined in the following example:

    1. ...
    2. time="2020-12-03T09:50:47Z" level=info msg="Install complete!"
    3. time="2020-12-03T09:50:47Z" level=info msg="To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/myuser/install_dir/auth/kubeconfig'"
    4. time="2020-12-03T09:50:47Z" level=info msg="Access the OpenShift web-console here: https://console-openshift-console.apps.mycluster.example.com"
    5. time="2020-12-03T09:50:47Z" level=info msg="Login to the console with user: \"kubeadmin\", and password: \"6zYIx-ckbW3-4d2Ne-IWvDF\""
    6. time="2020-12-03T09:50:47Z" level=debug msg="Time elapsed per stage:"
    7. time="2020-12-03T09:50:47Z" level=debug msg="    Infrastructure: 6m45s"
    8. time="2020-12-03T09:50:47Z" level=debug msg="Bootstrap Complete: 11m30s"
    9. time="2020-12-03T09:50:47Z" level=debug msg=" Bootstrap Destroy: 1m5s"
    10. time="2020-12-03T09:50:47Z" level=debug msg=" Cluster Operators: 17m31s"
    11. time="2020-12-03T09:50:47Z" level=info msg="Time elapsed: 37m26s"

Viewing the image pull source

For clusters with unrestricted network connectivity, you can view the source of your pulled images by using a command on a node, such as crictl images.

However, for disconnected installations, to view the source of pulled images, you must review the CRI-O logs to locate the Trying to access log entry, as shown in the following procedure. Other methods to view the image pull source, such as the crictl images command, show the non-mirrored image name, even though the image is pulled from the mirrored location.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  • Review the CRI-O logs for a master or worker node:

    1. $  oc adm node-logs <node_name> -u crio

    Example output

    The Trying to access log entry indicates where the image is being pulled from.

    1. ...
    2. Mar 17 02:52:50 ip-10-0-138-140.ec2.internal crio[1366]: time="2021-08-05 10:33:21.594930907Z" level=info msg="Pulling image: quay.io/openshift-release-dev/ocp-release:4.10.0-ppc64le" id=abcd713b-d0e1-4844-ac1c-474c5b60c07c name=/runtime.v1alpha2.ImageService/PullImage
    3. Mar 17 02:52:50 ip-10-0-138-140.ec2.internal crio[1484]: time="2021-03-17 02:52:50.194341109Z" level=info msg="Trying to access \"li0317gcp1.mirror-registry.qe.gcp.devcluster.openshift.com:5000/ocp/release@sha256:1926eae7cacb9c00f142ec98b00628970e974284b6ddaf9a6a086cb9af7a6c31\""
    4. Mar 17 02:52:50 ip-10-0-138-140.ec2.internal crio[1484]: time="2021-03-17 02:52:50.226788351Z" level=info msg="Trying to access \"li0317gcp1.mirror-registry.qe.gcp.devcluster.openshift.com:5000/ocp/release@sha256:1926eae7cacb9c00f142ec98b00628970e974284b6ddaf9a6a086cb9af7a6c31\""
    5. ...

    The log might show the image pull source twice, as shown in the preceding example.

    If your ImageContentSourcePolicy object lists multiple mirrors, OKD attempts to pull the images in the order listed in the configuration, for example:

    1. Trying to access \"li0317gcp1.mirror-registry.qe.gcp.devcluster.openshift.com:5000/ocp/release@sha256:1926eae7cacb9c00f142ec98b00628970e974284b6ddaf9a6a086cb9af7a6c31\"
    2. Trying to access \"li0317gcp2.mirror-registry.qe.gcp.devcluster.openshift.com:5000/ocp/release@sha256:1926eae7cacb9c00f142ec98b00628970e974284b6ddaf9a6a086cb9af7a6c31\"

Getting cluster version, status, and update details

You can view the cluster version and status by running the oc get clusterversion command. If the status shows that the installation is still progressing, you can review the status of the Operators for more information.

You can also list the current update channel and review the available cluster updates.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

  • You have installed the OpenShift CLI (oc).

Procedure

  1. Obtain the cluster version and overall status:

    1. $ oc get clusterversion

    Example output

    1. NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
    2. version   4.6.4     True        False         6m25s   Cluster version is 4.6.4

    The example output indicates that the cluster has been installed successfully.

  2. If the cluster status indicates that the installation is still progressing, you can obtain more detailed progress information by checking the status of the Operators:

    1. $ oc get clusteroperators.config.openshift.io

  3. View a detailed summary of cluster specifications, update availability, and update history:

    1. $ oc describe clusterversion

  4. List the current update channel:

    1. $ oc get clusterversion -o jsonpath='{.items[0].spec}{"\n"}'

    Example output

    1. {"channel":"stable-4.6","clusterID":"245539c1-72a3-41aa-9cec-72ed8cf25c5c"}

  5. Review the available cluster updates:

    1. $ oc adm upgrade

    Example output

    1. Cluster version is 4.6.4
    3. Updates:
    5. VERSION IMAGE
    6. 4.6.6   quay.io/openshift-release-dev/ocp-release@sha256:c7e8f18e8116356701bd23ae3a23fb9892dd5ea66c8300662ef30563d7104f39

Additional resources

Querying the status of the cluster nodes by using the CLI

You can verify the status of the cluster nodes after an installation.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

  • You have installed the OpenShift CLI (oc).

Procedure

  1. List the status of the cluster nodes. Verify that the output lists all of the expected control plane and compute nodes and that each node has a Ready status:

    1. $ oc get nodes

    Example output

    1. NAME                          STATUS   ROLES    AGE   VERSION
    2. compute-1.example.com         Ready    worker   33m   v1.23.0
    3. control-plane-1.example.com   Ready    master   41m   v1.23.0
    4. control-plane-2.example.com   Ready    master   45m   v1.23.0
    5. compute-2.example.com         Ready    worker   38m   v1.23.0
    6. compute-3.example.com         Ready    worker   33m   v1.23.0
    7. control-plane-3.example.com   Ready    master   41m   v1.23.0

  2. Review CPU and memory resource availability for each cluster node:

    1. $ oc adm top nodes

    Example output

    1. NAME                          CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
    2. compute-1.example.com         128m         8%     1132Mi          16%
    3. control-plane-1.example.com   801m         22%    3471Mi          23%
    4. control-plane-2.example.com   1718m        49%    6085Mi          40%
    5. compute-2.example.com         935m         62%    5178Mi          75%
    6. compute-3.example.com         111m         7%     1131Mi          16%
    7. control-plane-3.example.com   942m         26%    4100Mi          27%

Additional resources

Reviewing the cluster status from the OKD web console

You can review the following information in the Overview page in the OKD web console:

  • The general status of your cluster

  • The status of the control plane, cluster Operators, and storage

  • CPU, memory, file system, network transfer, and pod availability

  • The API address of the cluster, the cluster ID, and the name of the provider

  • Cluster version information

  • Cluster update status, including details of the current update channel and available updates

  • A cluster inventory detailing node, pod, storage class, and persistent volume claim (PVC) information

  • A list of ongoing cluster activities and recent events

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  • In the Administrator perspective, navigate to Home → Overview.

Reviewing the cluster status from Red Hat OpenShift Cluster Manager

You can review detailed information about the status of your cluster in the OpenShift Cluster Manager.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  1. In the Administrator perspective, navigate to Home → Overview → Details → OpenShift Cluster Manager to open the Overview page for the cluster in OpenShift Cluster Manager.

    Alternatively, you can navigate to OpenShift Cluster Manager directly and select your cluster ID from the list of available clusters.

  2. In the Overview page, review the following information about your cluster:

    • vCPU and memory availability and resource usage

    • The cluster ID, status, type, location, and the provider name

    • Node counts by node type

    • Cluster version details, the creation date of the cluster, and the name of the cluster owner

    • The life cycle support status of the cluster

    • Subscription information, including the service level agreement (SLA) status, the subscription unit type, the production status of the cluster, the subscription obligation, and the service level

    • A cluster history

  3. Navigate to the Observe page to review the following information:

    • A list of any issues that have been detected

    • A list of alerts that are firing

    • The cluster Operator status and version

    • Cluster resource usage

  4. Navigate to the Insights page to review the following information provided by Red Hat Insights:

    • Potential issues that your cluster might be exposed to, categorized by risk level

    • Health-check status by category

Additional resources

Checking cluster resource availability and utilization

OKD provides a comprehensive set of monitoring dashboards that help you understand the state of cluster components.

In the Administrator perspective, you can access dashboards for core OKD components, including:

  • etcd

  • Kubernetes compute resources

  • Kubernetes network resources

  • Prometheus

  • Dashboards relating to cluster and node performance

monitoring dashboard compute resources

Figure 1. Example compute resources dashboard

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  1. In the Administrator perspective in the OKD web console, navigate to Observe → Dashboards.

  2. Choose a dashboard in the Dashboard list. Some dashboards, such as the etcd dashboard, produce additional sub-menus when selected.

  3. Optional: Select a time range for the graphs in the Time Range list.

    • Select a pre-defined time period.

    • Set a custom time range by selecting Custom time range in the Time Range list.

      1. Input or select the From and To dates and times.

      2. Click Save to save the custom time range.

  1. Optional: Select a Refresh Interval.

  2. Hover over each of the graphs within a dashboard to display detailed information about specific items.

Additional resources

Listing alerts that are firing

Alerts provide notifications when a set of defined conditions are true in an OKD cluster. You can review the alerts that are firing in your cluster by using the Alerting UI in the OKD web console.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  1. In the Administrator perspective, navigate to the Observe → Alerting → Alerts page.

  2. Review the alerts that are firing, including their Severity, State, and Source.

  3. Select an alert to view more detailed information in the Alert Details page.

Additional resources

Next steps