Install Kubeflow

Instructions for deploying Kubeflow on AWS with the shell

This guide describes how to use the kfctl CLI todeploy Kubeflow on Amazon Web Services (AWS).

Prerequisites

EKS cluster

There’re many ways to provision EKS cluster, using AWS EKS CLI, CloudFormation or Terraform, AWS CDK or eksctl.Here, we highly recommend you to create an EKS cluster using eksctl.

You are required to have an existing Amazon Elastic Kubernetes Service (Amazon EKS) cluster before moving the next step.

The installation tool uses the eksctl command and doesn’t support the —profile option in that command.If you need to switch role, use the aws sts assume-role commands. See the AWS guide to using temporary security credentials to request access to AWS resources.

Prepare your environment

In order to deploy Kubeflow on your existing Amazon EKS cluster, you need to provide AWS_CLUSTER_NAME, cluster region and worker roles.

Follow these steps to download the kfctl binary for the Kubeflow CLI and setsome handy environment variables:

  1. tar -xvf kfctl_v0.7.1_<platform>.tar.gz
  • Create environment variables to make the deployment process easier:
  1. # Add kfctl to PATH, to make the kfctl binary easier to use.
  2. export PATH=$PATH:"<path to kfctl>"
  4. # Use the following kfctl configuration file for the standard AWS setup:
  5. export CONFIG_URI="https://raw.githubusercontent.com/kubeflow/manifests/v0.7-branch/kfdef/kfctl_aws.0.7.1.yaml"
  7. # Alternatively, use the following kfctl configuration if you want to enable
  8. # authentication:
  9. export CONFIG_URI="https://raw.githubusercontent.com/kubeflow/manifests/v0.7-branch/kfdef/kfctl_aws_cognito.0.7.1.yaml"
  11. # Set an environment variable for your AWS cluster name, and set the name
  12. # of the Kubeflow deployment to the same as the cluster name.
  13. export AWS_CLUSTER_NAME=<YOUR EKS CLUSTER NAME>
  14. export KF_NAME=${AWS_CLUSTER_NAME}
  16. # Set the path to the base directory where you want to store one or more
  17. # Kubeflow deployments. For example, /opt/.
  18. # Then set the Kubeflow application directory for this deployment.
  19. export BASE_DIR=<path to a base directory>
  20. export KF_DIR=${BASE_DIR}/${KF_NAME}

Notes:

  • ${CONFIG_URI} - The GitHub address of the configuration YAML file thatyou want to use to deploy Kubeflow. For AWS deployments, the followingconfigurations are available:

  • ${KF_NAME} - The name of your Kubeflow deployment.You should set this value to be the same as your AWS cluster name.The value of KF_NAME must consist of lower case alphanumeric characters or‘-’, and must start and end with an alphanumeric character.The value of this variable cannot be greater than 25 characters. It mustcontain just a name, not a directory path.This value also becomes the name of the directory where your Kubeflowconfigurations are stored, that is, the Kubeflow application directory.

  • ${KF_DIR} - The full path to your Kubeflow application directory.

Set up your Kubeflow configuration

Run kfctl to build your configuration files, so that you can edit theconfiguration before deploying Kubeflow:

  • Run the kfctl build command to set up your configuration:
  1. mkdir -p ${KF_DIR}
  2. cd ${KF_DIR}
  3. kfctl build -V -f ${CONFIG_URI}
  • Set an environment variable pointing to your local configuration file:
  1. export CONFIG_FILE=${KF_DIR}/kfctl_aws.0.7.1.yaml

Or:

  1. export CONFIG_FILE=${KF_DIR}/kfctl_aws_cognito.0.7.1.yaml

Customize your configuration

  • Replace the AWS cluster name in your ${CONFIG_FILE} file, by changingthe value kubeflow-aws to ${AWS_CLUSTER_NAME} in multiple locations inthe file. For example, use this sed command:
  1. sed -i'.bak' -e 's/kubeflow-aws/'"$AWS_CLUSTER_NAME"'/' ${CONFIG_FILE}
  • Retrieve the AWS Region and IAM role name for your worker nodes.To get the IAM role name for your Amazon EKS worker node, run the followingcommand:
  1. aws iam list-roles \
  2.   | jq -r ".Roles[] \
  3.   | select(.RoleName \
  4.   | startswith(\"eksctl-$AWS_CLUSTER_NAME\") and contains(\"NodeInstanceRole\")) \
  5.   .RoleName"
  7. eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6

Note: The above command assumes that you used eksctl to create yourcluster. If you use other provisioning tools to create your worker nodegroups, find the role that is associated with your worker nodes in theAmazon EC2 console.

  • Change cluster region and worker role names in your ${CONFIG_FILE} file:
  1. region: us-west-2
  2. roles:
  3.   - eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6

If you have multiple node groups, you will see corresponding number of node group roles. In that case, please provide the role names as an array.

Deploy Kubeflow

  • Run the following commands to initialize the Kubeflow cluster:
  1. cd ${KF_DIR}
  2. rm -rf kustomize/  # Remove kustomize folder and regenerate files after customization
  3. kfctl apply -V -f ${CONFIG_FILE}

Important!!! By default, these scripts create an AWS Application Load Balancer for Kubeflow that is open to public. This is good for development testing and for short term use, but we do not recommend that you use this configuration for production workloads.

To secure your installation, Follow the instructions to add authentication.

  • Wait for all the resources to become ready in the kubeflow namespace.
  1. kubectl -n kubeflow get all
  • Get Kubeflow service endpoint and copy link in browser.
  1. kubectl get ingress -n istio-system
  3. NAMESPACE      NAME            HOSTS   ADDRESS                                                             PORTS   AGE
  4. istio-system   istio-ingress   *       a743484b-istiosystem-istio-2af2-xxxxxx.us-west-2.elb.amazonaws.com   80      1h

This deployment may take 3-5 minutes to become ready. Verify that the address works by opening it in your preferred Internet browser. You can also run kubectl delete istio-ingress -n istio-system to remove the load balancer entirely.

Post Installation

Kubeflow 0.6 release brings multi-tenancy support and user are not able to create notebooks in kubeflow, default namespace. Instead, please create a Profile using kubectl apply -f profile.yaml and profile controller will create new namespace and service account which is allowed to create notebook in that namespace.

  1. apiVersion: kubeflow.org/v1beta1
  2. kind: Profile
  3. metadata:
  4.   name: aws-sample-user
  5. spec:
  6.   owner:
  7.     kind: User
  8.     name: aws-sample-user

Understanding the deployment process

The kfctl deployment process is controlled by the following commands:

  • kfctl build - (Optional) Creates configuration files defining the variousresources in your deployment. You only need to run kfctl build if you wantto edit the resources before running kfctl apply.
  • kfctl apply - Creates or updates the resources.
  • kfctl delete - Deletes the resources.

App layout

Your Kubeflow app directory ${KF_DIR} contains the following files and directories:

  • ${CONFIG_FILE} is a YAML file that defines configurations related to yourKubeflow deployment.

    • This file is a copy of the GitHub-based configuration YAML file thatyou used when deploying Kubeflow.
    • When you run kfctl apply or kfctl build, kfctl createsa local version of the configuration file, ${CONFIG_FILE},which you can further customize if necessary.

  • aws_config is a directory that contains a sample eksctl cluster configuration file that defines the AWS cluster and policy files to attach to your node group roles.

    • You can modify the cluster_config.yaml and cluster_features.yaml files to customize your AWS infrastructure.

  • kustomize is a directory that contains the kustomize packages for Kubeflow applications.

    • The directory is created when you run kfctl build or kfctl apply.
    • You can customize the Kubernetes resources (modify the manifests and run kfctl apply again).

The provisioning scripts can either bring up a new cluster and install Kubeflow on it, or you can install Kubeflow on your existing cluster. We recommend that you create a new cluster for better isolation.

If you experience any issues running these scripts, see the troubleshooting guidance for more information.