Seldon Serving

Model serving using Seldon

This Kubeflow component has stable status. See theKubeflow versioning policies.

Seldon comes installed with Kubeflow. The Seldon documentation site provides full documentation for running Seldon inference.

If you have a saved model in a PersistentVolume (PV), Google Cloud Storage bucket or Amazon S3 Storage you can use one of the prepackaged model servers provided by Seldon.

Seldon also provides language specific model wrappers to wrap your inference code for it to run in Seldon.

Kubeflow specifics

You need to ensure the namespace where your models will be served has:

  • An Istio gateway named kubeflow-gateway
  • A label set as serving.kubeflow.org/inferenceservice=enabled

The following example applies the label my-namespace to the namespace for serving:

  1. kubectl label namespace my-namespace serving.kubeflow.org/inferenceservice=enabled

Create a gateway called kubeflow-gateway in namespace my-namespace:

  1. apiVersion: networking.istio.io/v1alpha3
  2. kind: Gateway
  3. metadata:
  4.   name: kubeflow-gateway
  5.   namespace: my-namespace
  6. spec:
  7.   selector:
  8.     istio: ingressgateway
  9.   servers:
  10.   - hosts:
  11.     - '*'
  12.     port:
  13.       name: http
  14.       number: 80
  15.       protocol: HTTP

Save the above resource and apply it with kubectl.

Examples

The Kubeflow Seldon E2E Pipeline shows how to build re-usable components for an ML pipeline.

Seldon provides a large set of example notebooks showing how to run inference code for a wide range of machine learning toolkits.

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.

Last modified 04.03.2020: fix typo in seldon serving gateway spec (#1767) (ac112966)