Set up Project
Creating a Google Cloud project for your Kubeflow deployment
Follow these steps to set up your Google Cloud project:
Select or create a project on the Google Cloud Console. If you plan to use different Google Cloud projects for Management Cluster and Kubeflow Clusters: create one Management project for Management Cluster, and create one or more Kubeflow projects for Kubeflow Clusters.
Make sure that you have the Owner role for the project in Cloud IAM (Identity and Access Management). The deployment process creates various service accounts with appropriate roles in order to enable seamless integration with Google Cloud services. This process requires that you have the owner role for the project in order to deploy Kubeflow.
Make sure that billing is enabled for your project. Refer to Enable billing for a project.
Open following pages on the Google Cloud Console and ensure that the specified APIs are enabled for all projects:
- Compute Engine API
- Kubernetes Engine API
- Identity and Access Management (IAM) API
- Service Management API
- Cloud Resource Manager API
- AI Platform Training & Prediction API
- Cloud Identity-Aware Proxy API
- Cloud Build API (It’s required if you plan to use Fairing in your Kubeflow cluster)
- Cloud SQL Admin API
You can also enable these APIs by running the following command in Cloud Shell:
gcloud services enable \
compute.googleapis.com \
container.googleapis.com \
iam.googleapis.com \
servicemanagement.googleapis.com \
cloudresourcemanager.googleapis.com \
ml.googleapis.com \
iap.googleapis.com \
sqladmin.googleapis.com \
meshconfig.googleapis.com
# Cloud Build API is optional, you need it if using Fairing.
# gcloud services enable cloudbuild.googleapis.com
If you are using the Google Cloud Free Program or the 12-month trial period with $300 credit, note that the free tier does not offer enough resources for default full Kubeflow installation. You need to upgrade to a paid account.
For more information, see the following issues:
- kubeflow/website #1065 reports the problem.
- kubeflow/kubeflow #3936 requests a Kubeflow configuration to work with a free trial project.
Read the Google Cloud Resource quotas to understand quotas on resource usage that Compute Engine enforces, and to learn how to check and increase your quotas.
Initialize your project to prepare it for Anthos Service Mesh installation:
PROJECT_ID=<YOUR_PROJECT_ID>
curl --request POST \
--header "Authorization: Bearer $(gcloud auth print-access-token)" \
--data '' \
https://meshconfig.googleapis.com/v1alpha1/projects/${PROJECT_ID}:initialize
Refer to Anthos Service Mesh documentation for details.
If you encounter a
Workload Identity Pool does not exist
error, refer to the following issue:- kubeflow/website #2121 describes that creating and then removing a temporary Kubernetes cluster may be needed for projects that haven’t had a cluster set up beforehand.
You do not need a running GKE cluster. The deployment process creates a cluster for you.
Next steps
- Set up an OAuth credential to use Cloud Identity-Aware Proxy (Cloud IAP). Cloud IAP is recommended for production deployments or deployments with access to sensitive data.
- Set up Management Cluster to deploy and manage Kubeflow clusters.
- Deploy Kubeflow using kubectl, kustomize and kpt.
Last modified 17.05.2021: (gke) Full Kubeflow Deployment guide on GKE (revision for Kubeflow 1.3) (#2686) (f50693a3)