Train and Deploy on GCP from a Kubeflow Notebook

Use Kubeflow Fairing to train and deploy a model on Google Cloud Platform (GCP) from a notebook that is hosted on Kubeflow

This guide introduces you to using Kubeflow Fairing to train anddeploy a model to Kubeflow on Google Kubernetes Engine (GKE) andGoogle AI Platform Training.

Your Kubeflow deployment includes services for spawning and managing Jupyternotebooks. Kubeflow Fairing is preinstalled in the Kubeflow notebooks, alongwith a number of machine learning (ML) libraries.

Set up Kubeflow and access the Kubeflow notebook environment

Follow the Kubeflow notebook setup guideto install Kubeflow, access your Kubeflow hosted notebook environment, andcreate a new notebook server.

When selecting a Docker image and other settings for the baseline deploymentof your notebook server, you can leave all the settings at the default value.

Run the example notebook

As an example, this guide uses a notebook that is hosted on Kubeflowto demonstrate how to:

  • Train an XGBoost model in a notebook,
  • Use Kubeflow Fairing to train an XGBoost model remotely on Kubeflow,
  • Use Kubeflow Fairing to train an XGBoost model remotely onAI Platform Training,
  • Use Kubeflow Fairing to deploy a trained model to Kubeflow, and
  • Call the deployed endpoint for predictions.

Follow these instructions to run the XGBoost quickstart notebook:

  • Download the files used in this example and install the packages that theXGBoost quickstart notebook depends on.

    • On the Jupyter dashboard for your notebook server, click New andselect Terminal to start a new terminal session in your notebookenvironment. Use the terminal session to set up your notebookenvironment to run this example.

    • Clone the Kubeflow Fairing repository to download the files used inthis example.

  1. git clone https://github.com/kubeflow/fairing 
  1. - Install the Python dependencies for the XGBoost quickstart notebook.
  1. pip3 install -r fairing/examples/prediction/requirements.txt

  • Use the notebook user interface to open the XGBoost quickstart notebookat fairing/examples/prediction/xgboost-high-level-apis.ipynb.

  • Follow the instructions in the notebook to:

    • Train an XGBoost model in a notebook,
    • Use Kubeflow Fairing to train an XGBoost model remotely on Kubeflow,
    • Use Kubeflow Fairing to train an XGBoost model remotely on AI Platform Training,
    • Use Kubeflow Fairing to deploy a trained model to Kubeflow, and
    • Call the deployed endpoint for predictions.