Deploy Kubeflow using MiniKF on Google Cloud

Deploy Kubeflow with MiniKF (mini Kubeflow) via Google Cloud Marketplace

MiniKF latestversion

This guide describes how to launch a MiniKF virtual machine (VM) on Google Cloud. MiniKF is a single VM solution on the Google Cloud Marketplace, and installs:

  • Kubernetes (using Minikube)
  • Kubeflow
  • Kale, a tool to convert general purpose Jupyter Notebooks to Kubeflow Pipelines workflows (GitHub)
  • Arrikto’s Rok, a data management software for data versioning and reproducibility

MiniKF supports NVIDIA GPUs.

Before you start

Before installing MiniKF, make sure that your Google Cloud project meets the minimum requirements described below:

  1. Select or create a project on the Google Cloud Console.
  2. Make sure that you have the editor role role for the project. You can add a member to a project and grant them a Cloud IAM role using the Cloud Console.
  3. Make sure that billing is enabled for your project. For more info, see the Modify a Project’s Billing Settings guide.
  4. If you are using the Google Cloud Free Tier or the 12-month trial period with $300 credit, note that you can’t run the default Google Cloud installation of MiniKF, because the free tier does not offer enough resources. You need to upgrade to a paid account.
  5. Read the Google Cloud guide to resource quotas to understand the quotas on resource usage that Compute Engine enforces, and to learn how to check your quota and how to request an increase in quota.

Installing MiniKF on Google Cloud

To install MiniKF on Google Cloud, follow the steps below:

  1. Go to the MiniKF page on Google Cloud Marketplace.

  2. Click on the LAUNCH button

    Launch MiniKF on Google Cloud

  3. In the Configure & Deploy window, choose:

  • Deployment name: a name for your MiniKF instance

  • Zone: the Google Cloud zone that you will deploy MiniKF

  • Machine type: a machine type with at least 2 vCPUs and 16 GB of RAM. If in doubt, use the default machine type, n1–standard-8, which has 8 vCPUs and 30 GB of RAM.

  • Boot Disk: a boot disk. This is the disk that holds the images that MiniKF needs. Use the default value of 200 GB SSD Persistent Disk.

  • Data Disk: a data disk. This is the disk that holds your snapshots. Use an SSD Persistent Disk of at least 500 GB capacity.

  • GPUs: add one or more GPUs if you are running computationally intensive ML workloads.

    Then click Deploy:

    Configure and deploy MiniKF

  1. Now wait for the VM to boot.

    MiniKF on Google Cloud has been deployed

  2. When the VM is up, ensure that MiniKF is up and running. Before visiting the MiniKF dashboard, ensure that Minikube, Kubeflow, and Rok (Snapshot Store) are up and running. To do so, click on the SSH button to connect to the MiniKF VM and follow the on-screen instructions.

    Connect to the MiniKF VM

  3. MiniKF dashboard, username, and password. Now that Minikube, Kubeflow, and Rok are up-and-running, close the SSH browser window, as you will not need it anymore. On your current view, please find the MiniKF dashboard URL, MiniKF username, and MiniKF password. You will need them to log in to Kubeflow and Rok. You should see a table like the one below:

    MiniKF dashboard, username, and password

  4. Log in to MiniKF. Visit the MiniKF dashboard URL and log in to MiniKF using the MiniKF username and password.

    Log in to MiniKF

    Once you log in, you will see the Central Dashboard

    MiniKF Central Dashboard

Congratulations! You have successfully deployed MiniKF on Google Cloud. You can now create notebooks, write your ML code, and run Kubeflow Pipelines.

Last modified 26.10.2020: Remove the out-of-date banner on the MiniKF on GCP page (#2306) (aa07a903)