Kubeflow Deployment with kfctl_k8s_istio

Instructions for installing Kubeflow on your existing Kubernetes cluster using kfctl_k8s_istio config

This configuration creates a vanilla deployment of Kubeflow with all its core components without any external dependencies. The deployment can be customized based on your environment needs.

Before you start

This Kubeflow deployment requires a default StorageClass with a dynamic volume provisioner. Verify the provisioner field of your default StorageClass definition. If you don’t have a provisioner, ensure that you have configured volume provisioning in your Kubernetes cluster appropriately as mentioned below.

Configuring your installation with kfctl_k8s_istio.v1.2.0.yaml has an option you should consider:

  • Disabling istio installation - If your Kubernetes cluster has an existing Istio installation you may choose to not install Istio by removing the applications istio-crds and istio-install in the configuration file kfctl_k8s_istio.v1.2.0.yaml

Prepare your environment

Follow these steps to download the kfctl binary for the Kubeflow CLI and set some handy environment variables:

  1. Download the kfctl v1.2.0 release from the Kubeflow releases page.

  2. Unpack the tar ball:

    1. tar -xvf kfctl_v1.2.0_<platform>.tar.gz

  3. Create environment variables to make the deployment process easier:

    1. # The following command is optional. It adds the kfctl binary to your path.
    2. # If you don't add kfctl to your path, you must use the full path
    3. # each time you run kfctl.
    4. # Use only alphanumeric characters or - in the directory name.
    5. export PATH=$PATH:"<path-to-kfctl>"
    7. # Set KF_NAME to the name of your Kubeflow deployment. You also use this
    8. # value as directory name when creating your configuration directory.
    9. # For example, your deployment name can be 'my-kubeflow' or 'kf-test'.
    10. export KF_NAME=<your choice of name for the Kubeflow deployment>
    12. # Set the path to the base directory where you want to store one or more 
    13. # Kubeflow deployments. For example, /opt/.
    14. # Then set the Kubeflow application directory for this deployment.
    15. export BASE_DIR=<path to a base directory>
    16. export KF_DIR=${BASE_DIR}/${KF_NAME}
    18. # Set the configuration file to use when deploying Kubeflow.
    19. # The following configuration installs Istio by default. Comment out 
    20. # the Istio components in the config file to skip Istio installation. 
    21. # See https://github.com/kubeflow/kubeflow/pull/3663
    22. export CONFIG_URI="https://raw.githubusercontent.com/kubeflow/manifests/v1.2-branch/kfdef/kfctl_k8s_istio.v1.2.0.yaml"

Notes:

  • ${KF_NAME} - The name of your Kubeflow deployment. If you want a custom deployment name, specify that name here. For example, my-kubeflow or kf-test. The value of KF_NAME must consist of lower case alphanumeric characters or ‘-‘, and must start and end with an alphanumeric character. The value of this variable cannot be greater than 25 characters. It must contain just a name, not a directory path. You also use this value as directory name when creating the directory where your Kubeflow configurations are stored, that is, the Kubeflow application directory.

  • ${KF_DIR} - The full path to your Kubeflow application directory.

  • ${CONFIG_URI} - The GitHub address of the configuration YAML file that you want to use to deploy Kubeflow. The URI used in this guide is https://raw.githubusercontent.com/kubeflow/manifests/v1.2-branch/kfdef/kfctl_k8s_istio.v1.2.0.yaml. When you run kfctl apply or kfctl build (see the next step), kfctl creates a local version of the configuration YAML file which you can further customize if necessary.

Set up and deploy Kubeflow

To set up and deploy Kubeflow using the default settings, run the kfctl apply command:

  1. mkdir -p ${KF_DIR}
  2. cd ${KF_DIR}
  3. kfctl apply -V -f ${CONFIG_URI}

Check the resources deployed in namespace kubeflow:

  1. kubectl -n kubeflow get all

Alternatively, set up your configuration for later deployment

If you want to customize your configuration before deploying Kubeflow, you can set up your configuration files first, then edit the configuration, then deploy Kubeflow:

  1. Run the kfctl build command to set up your configuration:

    1. mkdir -p ${KF_DIR}
    2. cd ${KF_DIR}
    3. kfctl build -V -f ${CONFIG_URI}

  2. Edit the configuration files, as described in the guide to customizing your Kubeflow deployment.

  3. Set an environment variable pointing to your local configuration file:

    1. export CONFIG_FILE=${KF_DIR}/kfctl_k8s_istio.v1.2.0.yaml

  4. Run the kfctl apply command to deploy Kubeflow:

    1. kfctl apply -V -f ${CONFIG_FILE}

Access the Kubeflow user interface (UI)

After Kubeflow is deployed, the Kubeflow Dashboard can be accessed via istio-ingressgateway service. If loadbalancer is not available in your environment, NodePort or Port forwarding can be used to access the Kubeflow Dashboard. Refer to Ingress Gateway guide.

Delete Kubeflow

Run the following commands to delete your deployment and reclaim all resources:

  1. cd ${KF_DIR}
  2. # If you want to delete all the resources, run:
  3. kfctl delete -f ${CONFIG_FILE}

Understanding the deployment process

The kfctl deployment process includes the following commands:

  • kfctl build - (Optional) Creates configuration files defining the various resources in your deployment. You only need to run kfctl build if you want to edit the resources before running kfctl apply.
  • kfctl apply - Creates or updates the resources.
  • kfctl delete - Deletes the resources.

Application layout

Your Kubeflow application directory ${KF_DIR} contains the following files and directories:

  • ${CONFIG_FILE} is a YAML file that defines configurations related to your Kubeflow deployment.

  • kustomize is a directory that contains the kustomize packages for Kubeflow applications. See how Kubeflow uses kustomize.

    • The directory is created when you run kfctl build or kfctl apply.
    • You can customize the Kubernetes resources by modifying the manifests and running kfctl apply again.

We recommend that you check in the contents of your ${KF_DIR} directory into source control.

Provisioning of Persistent Volumes in Kubernetes

Note that you can skip this step if you have a dynamic volume provisioner already installed in your cluster.

If you don’t have one:

  • You can choose to create PVs manually after deployment of Kubeflow.
  • Or install a dynamic volume provisioner like Local Path Provisioner. Ensure that the StorageClass used by this provisioner is the default StorageClass.

Troubleshooting

Persistent Volume Claims are in Pending State

Check if PersistentVolumeClaims get Bound to PersistentVolumes.

  1. kubectl -n kubeflow get pvc

If the PersistentVolumeClaims (PVCs) are in Pending state after deployment and they are not bound to PersistentVolumes (PVs), you may have to either manually create PVs for each PVC in your Kubernetes Cluster or an alternative is to set up dynamic volume provisioning to create PVs on demand and redeploy Kubeflow after deleting existing PVCs.

Next steps

Last modified 20.04.2021: Apply Docs Restructure to `v1.2-branch` = update `v1.2-branch` to current `master` v2 (#2612) (4e2602bd)