Install the Kubeflow Pipelines SDK

Setting up your Kubeflow Pipelines development environment

This guide tells you how to install theKubeflow Pipelines SDKwhich you can use to build machine learning pipelines. You can use the SDKto execute your pipeline, or alternatively you can upload the pipeline tothe Kubeflow Pipelines UI for execution.

All of the SDK’s classes and methods are described in the auto-generated SDK reference docs.

Set up Python

You need Python 3.5 or later to use the Kubeflow Pipelines SDK. Thisguide uses Python 3.7.

If you haven’t yet set up a Python 3 environment, do so now. This guiderecommends Miniconda, but you can usea virtual environment manager of your choice, such as virtualenv.

Follow the steps below to setup Python using Miniconda:

• Choose one of the following methods to install Miniconda, depending on yourenvironment:

  • Debian/Ubuntu/Cloud Shell:

apt-get update; apt-get install -y wget bzip2
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

- 

Windows: Download theinstallerand make sure you select the option toAdd Miniconda to my PATH environment variable during the installation.

- 

MacOS: Download theinstallerand run the following command:

bash Miniconda3-latest-MacOSX-x86_64.sh

• Check that the conda command is available:

which conda

If the conda command is not found, add Miniconda to your path:

export PATH=<YOUR_MINICONDA_PATH>/bin:$PATH

• Create a clean Python 3 environment with a name of your choosing. Thisexample uses Python 3.7 and an environment name of mlpipeline.:

conda create --name mlpipeline python=3.7
conda activate mlpipeline

Install the Kubeflow Pipelines SDK

Run the following command to install the Kubeflow Pipelines SDK:

pip3 install kfp --upgrade

Note: If you are not using a virtual environment, such as conda, when installing the Kubeflow Pipelines SDK, you may receive the following error:

ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/kfp-0.2.0.dist-info'
Consider using the `--user` option or check the permissions.

If you get this error, install kfp with the —user option:

pip3 install kfp --upgrade --user

This command installs the dsl-compile and kfp binaries under ~/.local/bin, which is not part of the PATH in some Linux distributions, such as Ubuntu. You can add ~/.local/bin to your PATH by appending the following to a new line at the end of your .bashrc file:

export PATH=$PATH:~/.local/bin

After successful installation, the command dsl-compile should be available.You can use this command to verify it:

which dsl-compile

The response should be something like this:

/<PATH_TO_YOUR_USER_BIN>/miniconda3/envs/mlpipeline/bin/dsl-compile

Next steps

• See how to use the SDK.
• Build a component and a pipeline.
• Get started with the UI.
• Understand pipeline concepts.

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.

Last modified 20.02.2020: Fix typo in install-sdk.md (#1708) (809485ff)