Machine phases and lifecycle

Machines move through a lifecycle that has several defined phases. Understanding the machine lifecycle and its phases can help you verify whether a procedure is complete or troubleshoot undesired behavior. In OKD, the machine lifecycle is consistent across all supported cloud providers.

Machine phases

As a machine moves through its lifecycle, it passes through different phases. Each phase is a basic representation of the state of the machine.

Provisioning

There is a request to provision a new machine. The machine does not yet exist and does not have an instance, a provider ID, or an address.

Provisioned

The machine exists and has a provider ID or an address. The cloud provider has created an instance for the machine. The machine has not yet become a node and the status.nodeRef section of the machine object is not yet populated.

Running

The machine exists and has a provider ID or address. Ignition has run successfully and the cluster machine approver has approved a certificate signing request (CSR). The machine has become a node and the status.nodeRef section of the machine object contains node details.

Deleting

There is a request to delete the machine. The machine object has a DeletionTimestamp field that indicates the time of the deletion request.

Failed

There is an unrecoverable problem with the machine. This can happen, for example, if the cloud provider deletes the instance for the machine.

The machine lifecycle

The lifecycle begins with the request to provision a machine and continues until the machine no longer exists.

The machine lifecycle proceeds in the following order. Interruptions due to errors or lifecycle hooks are not included in this overview.

  1. There is a request to provision a new machine for one of the following reasons:

    • A cluster administrator scales a machine set such that it requires additional machines.

    • An autoscaling policy scales machine set such that it requires additional machines.

    • A machine that is managed by a machine set fails or is deleted and the machine set creates a replacement to maintain the required number of machines.

  2. The machine enters the Provisioning phase.

  3. The infrastructure provider creates an instance for the machine.

  4. The machine has a provider ID or address and enters the Provisioned phase.

  5. The Ignition configuration file is processed.

  6. The kubelet issues a certificate signing request (CSR).

  7. The cluster machine approver approves the CSR.

  8. The machine becomes a node and enters the Running phase.

  9. An existing machine is slated for deletion for one of the following reasons:

    • A user with cluster-admin permissions uses the oc delete machine command.

    • The machine gets a machine.openshift.io/delete-machine annotation.

    • The machine set that manages the machine marks it for deletion to reduce the replica count as part of reconciliation.

    • The cluster autoscaler identifies a node that is unnecessary to meet the deployment needs of the cluster.

    • A machine health check is configured to replace an unhealthy machine.

  10. The machine enters the Deleting phase, in which it is marked for deletion but is still present in the API.

  11. The machine controller removes the instance from the infrastructure provider.

  12. The machine controller deletes the Node object.

Determining the phase of a machine

You can find the phase of a machine by using the OpenShift CLI (oc) or by using the web console. You can use this information to verify whether a procedure is complete or to troubleshoot undesired behavior.

Determining the phase of a machine by using the CLI

You can find the phase of a machine by using the OpenShift CLI (oc).

Prerequisites

  • You have access to an OKD cluster using an account with cluster-admin permissions.

  • You have installed the oc CLI.

Procedure

  • List the machines on the cluster by running the following command:

    1. $ oc get machine -n openshift-machine-api

    Example output

    1. NAME                                      PHASE     TYPE         REGION      ZONE         AGE
    2. mycluster-5kbsp-master-0                  Running   m6i.xlarge   us-west-1   us-west-1a   4h55m
    3. mycluster-5kbsp-master-1                  Running   m6i.xlarge   us-west-1   us-west-1b   4h55m
    4. mycluster-5kbsp-master-2                  Running   m6i.xlarge   us-west-1   us-west-1a   4h55m
    5. mycluster-5kbsp-worker-us-west-1a-fmx8t   Running   m6i.xlarge   us-west-1   us-west-1a   4h51m
    6. mycluster-5kbsp-worker-us-west-1a-m889l   Running   m6i.xlarge   us-west-1   us-west-1a   4h51m
    7. mycluster-5kbsp-worker-us-west-1b-c8qzm   Running   m6i.xlarge   us-west-1   us-west-1b   4h51m

    The PHASE column of the output contains the phase of each machine.

Determining the phase of a machine by using the web console

You can find the phase of a machine by using the OKD web console.

Prerequisites

  • You have access to an OKD cluster using an account with cluster-admin permissions.

Procedure

  1. Log in to the web console as a user with the cluster-admin role.

  2. Navigate to Compute → Machines.

  3. On the Machines page, select the name of the machine that you want to find the phase of.

  4. On the Machine details page, select the YAML tab.

  5. In the YAML block, find the value of the status.phase field.

    Example YAML snippet

    1. apiVersion: machine.openshift.io/v1beta1
    2. kind: Machine
    3. metadata:
    4.   name: mycluster-5kbsp-worker-us-west-1a-fmx8t
    5. # ...
    6. status:
    7.   phase: Running (1)
    1In this example, the phase is Running.

Additional resources