Fairing on Azure

Documentation for Kubeflow Fairing on Microsoft Azure Kubernetes Service

This page documents how to run the Fairing prediction examplenotebook on Azure Kubernetes Service(AKS) in a notebook hosted on Kubeflow.

Prerequisites

Before you configure and use the Kubeflow Fairing Python SDK, you must have aKubeflow environment installed. The example notebook does in-cluster training.The example notebook has been tested to work in a cluster with 2 nodes andvirtual machines with Standard_D4_v3 size. It is not recommended to use thesmallest size virtual machines.

  • If you do not have a Kubeflow installation on Azure, follow the installationguide.
  • You must have the kubectl command line interface installed and configuredto use the Kubernetes cluster where Kubeflow is installed. In the aboveinstallation guide, the command az aks get-credentials configures kubectlto access your Kubernetes cluster.

Create Azure Container Registry and Storage

The example notebook uses Azure Container Registry (ACR) to host dockerimages for deployment and Azure Storage (Storage) is used as build context forin-cluster building.

You can re-use existing ACR and Storage resources or create new ones. For moreinformation, see the documentation for ACR andStorage.

After you have created your ACR and Storage resources, you must configure aservice principal with access to these resources. If you want to use the sameservice principal as your AKS cluster, you can get the service principal IDwith the following command:

  1. az aks show -g <resource-group-name> -n <name>

The above command has output like this:

  1.   "servicePrincipalProfile": {
  2.     "clientId": "<id>"
  3.   },

The role you grant must have at least read and write permissions to ACR andStorage.

To learn more about how to grant the service principal access, follow theAzure role-based authentication documentation.

Setting up credentials as Kubernetes secrets

Before running the notebook, you must first configure credentials so that thePython code running within the cluster can access the Azure resources requiredto train and deploy a model.

Run the following commands to set up your credentials as a Kubernetes secret.

  • Set environment variables to use in the following commands.
  1. export AZ_CLIENT_ID=<service-principal-client-id>
  2. export AZ_CLIENT_SECRET=<service-principal-client-secret>
  3. export AZ_TENANT_ID=<tenant-id>
  4. export AZ_SUBSCRIPTION_ID=<subscription-id>
  5. export TARGET_NAMESPACE=<target-namespace e.g. kubeflow-anonymous>
  6. export ACR_NAME=<acr-name>
  • AZ_CLIENT_ID: The service principal client ID. You can get theclient_id property from ~/.azure/aksServicePrincipal.json.
  • AZ_CLIENT_SECRET: The service principal secret. You can get theclient_secret property from ~/.azure/aksServicePrincipal.json.
  • AZ_TENANT_ID: The Azure Tenant ID of your account. You can get theTenant ID from the tenantId field in the output of az account show.
  • AZ_SUBSCRIPTION: The Azure Subscription ID of your account. You canget the Subscription ID from the id field in the output of az account show.
  • TARGET_NAMESPACE: Specify the namespace that your Notebook Server isin. For example, this guide recommends using kubeflow-anonymous.
  • ACR_NAME: The name of an ACR that the service principal can access.
    • Use the following command to create a secret that Kubeflow can use to accessAzure APIs.
  1. kubectl create secret generic -n ${TARGET_NAMESPACE} azcreds \
  2. --from-literal=AZ_CLIENT_ID=${AZ_CLIENT_ID} \
  3. --from-literal=AZ_CLIENT_SECRET=${AZ_CLIENT_SECRET} \
  4. --from-literal=AZ_TENANT_ID=${AZ_TENANT_ID} \
  5. --from-literal=AZ_SUBSCRIPTION_ID=${AZ_SUBSCRIPTION_ID}
  • Use the following command to create a secret that Kubeflow can use to accessACR.
  1. kubectl create secret docker-registry -n ${TARGET_NAMESPACE} acrcreds \
  2. --docker-server=${ACR_NAME}.azurecr.io \
  3. --docker-username=${AZ_CLIENT_ID} \
  4. --docker-password=${AZ_CLIENT_SECRET}
  5. kubectl patch serviceaccount default-editor -n ${TARGET_NAMESPACE} \
  6. -p "{\"imagePullSecrets\": [{\"name\": \"acrcreds\"}]}"

Creating a Notebook Server in Kubeflow

To create a notebook server, use your Web browser to access the KubeflowCentral Dashboard and select the Notebook Servers panel from the menu.

First, select the target namespace in which you want to host the server. In thedefault Kubeflow installation, there should be a namespace kubeflow-anonymousavailable in the namespace drop-down menu.

After the target namespace is selected, click NEW SERVER and fill in themandatory fields. The fields with default values can all be left as theyare and do not have to be modified to run the example notebook.

After launching the server, wait for the CONNECT button to appear and clickCONNECT to launch your Notebook Server. It may take up to a minute for theserver to be ready for connections.

Cloning the example notebook

Clone the Kubeflow Fairing repository to download the files used in this example.

  • Connect to your notebook server, then click the new terminal optionlike in the screenshot below:Creating new terminal after connecting to notebook server

  • Run the following command to clone the Kubeflow Fairing project:

  1. git clone https://github.com/kubeflow/fairing.git

This command clones the project including the example into your notebook server.

You can now close the terminal window, and you should now see the fairing folderin your notebooks server’s Files tab. Navigate to the example notebooks atfairing/examples/prediction/xgboost-high-level-apis.ipynb.

Executing the notebook

Before running the cells, read the following to learn how to configure Fairingto use your Azure resources.

In the xgboost-high-level-apis.ipynb notebook, find the cell tagged withparameters and update this cell to use Azure. Update following values toconfigure Fairing to use your Azure backend and Storage.

  1. export FAIRING_BACKEND = 'KubeflowAzureBackend'
  2. export DOCKER_REGISTRY = '<acr-name>.azurecr.io'
  3. export AZURE_REGION = None # This is only relevant if you haven't created a
  4.                            # Storage yet and let Fairing create it for you.
  5.                            # In that case, you can specify the region as
  6.                            # string, for example, 'NorthEurope'.
  7. export AZURE_RESOURCE_GROUP = '<storage-resource-group>'
  8. export AZURE_STORAGE_ACCOUNT = '<storage-name>'

After the above steps have been completed, you can run the example notebook.

You can also have a look at the CI pipeline that runs theexample notebook in AKS for steps involved to accomplish a successful runprogrammatically.