Deploy using MiniKF on GCP

Instructions for using the Google Cloud Marketplace to deploy MiniKF (mini Kubeflow) on Google Cloud Platform (GCP).

This guide describes how to launch a MiniKF virtual machine (VM) on GCP. MiniKFis a single VM solution on the Google Cloud Marketplace, and installs:

• Kubernetes (using Minikube)
• Kubeflow
• Kale, a tool to convert general purpose Jupyter Notebooks to KubeflowPipelines workflows (GitHub)
• Arrikto’s Rok, a data management software for data versioning andreproducibility

MiniKF installs Kubeflow 0.7.0 and supports NVIDIA GPUs.

Before you start

Before installing MiniKF, make sure that your GCP project meets the minimumrequirements described below:

• Select or create a project on the GCP Console.
• Make sure that you have the editor rolerole for the project. You can add a member to a project andgrant them a Cloud IAM role using the Cloud Console.
• Make sure that billing is enabled for your project. For more info, see theModify a Project’s Billing Settings guide.
• If you are using the GCP Free Tier or the 12-month trial period with $300credit, note that you can’t run the default GCP installation of MiniKF,because the free tier does not offer enough resources. You need to upgrade to a paid account.
• Read the GCP guide to resource quotas to understand the quotas onresource usage that Compute Engine enforces, and to learn how to check yourquota and how to request an increase in quota.

Installing MiniKF on GCP

To install MiniKF on GCP, follow the steps below:

• Go to the MiniKF page onGoogle Cloud Marketplace.

• Click on the Launch on Compute Engine button

• In the Configure & Deploy window, choose:

  • name: a name for your MiniKF instance
  • zone: the GCP zone that you will deploy MiniKF
  • machine type: a machine type with at least 2 vCPUs and 16 GB of RAM. Ifin doubt, use the default machine type, n1–standard-8, which has 8 vCPUs and30 GB of RAM.
  • boot disk: A boot disk. This is the disk that holds the images that MiniKFneeds. Use the default value of 200 GB SSD Persistent Disk.
  • data disk: A data disk. This is the disk that holds your snapshots. Usean SSD Persistent Disk of at least 500 GB capacity. Then click Deploy:

• Now wait for the VM to boot.

• When the VM is up, ensure that MiniKF is up and running. Before visitingthe MiniKF dashboard, ensure that Minikube, Kubeflow, and Rok (SnapshotStore) are up and running. To do so, click on the SSH button to connectto the MiniKF VM and follow the on-screen instructions.

• MiniKF dashboard, username, and password. Now that Minikube, Kubeflow,and Rok are up-and-running, close the SSH browser window, as you will notneed it anymore. On your current view, please find the MiniKF dashboardURL, MiniKF username, and MiniKF password. You will need them to login to Kubeflow and Rok. You should see a table like the one below:

• Log in to Kubeflow. Visit the MiniKF dashboard URL and log in to Kubeflowusing the MiniKF username and password.

• Log in to Snapshot Store (Rok). After logging in to Kubeflow, navigate tothe Snapshot Store:

And log in to Snapshot Store (Arrikto’s Rok) using the MiniKF username andpassword.

Congratulations! You have successfully deployed MiniKF on GCP. You can nowcreate notebooks, write your ML code, and run Kubeflow Pipelines.