Deploy Lightgbm model with InferenceService

Creating your own model and testing the LightGBM server.

To test the LightGBM Server, first we need to generate a simple LightGBM model using Python.

  1. import lightgbm as lgb
  2. from sklearn.datasets import load_iris
  3. import os
  5. model_dir = "."
  6. BST_FILE = "model.bst"
  8. iris = load_iris()
  9. y = iris['target']
  10. X = iris['data']
  11. dtrain = lgb.Dataset(X, label=y)
  13. params = {
  14.     'objective':'multiclass', 
  15.     'metric':'softmax',
  16.     'num_class': 3
  17. }
  18. lgb_model = lgb.train(params=params, train_set=dtrain)
  19. model_file = os.path.join(model_dir, BST_FILE)
  20. lgb_model.save_model(model_file)

Then, we can install and run the LightGBM Server using the generated model and test for prediction. Models can be on local filesystem, S3 compatible object storage, Azure Blob Storage, or Google Cloud Storage.

  1. python -m lgbserver --model_dir /path/to/model_dir --model_name lgb

We can also do some simple predictions

  1. import requests
  3. request = {'sepal_width_(cm)': {0: 3.5}, 'petal_length_(cm)': {0: 1.4}, 'petal_width_(cm)': {0: 0.2},'sepal_length_(cm)': {0: 5.1} }
  4. formData = {
  5.     'inputs': [request]
  6. }
  7. res = requests.post('http://localhost:8080/v1/models/lgb:predict', json=formData)
  8. print(res)
  9. print(res.text)

Create the InferenceService

  1. apiVersion: "serving.kserve.io/v1beta1"
  2. kind: "InferenceService"
  3. metadata:
  4.   name: "lightgbm-iris"
  5. spec:
  6.   predictor:
  7.     lightgbm:
  8.       storageUri: "gs://kfserving-examples/models/lightgbm/iris"

Apply the above yaml to create the InferenceService

  1. kubectl apply -f lightgbm.yaml

Expected Output

  1. $ inferenceservice.serving.kserve.io/lightgbm-iris created

Run a prediction

The first step is to determine the ingress IP and ports and set INGRESS_HOST and INGRESS_PORT

  1. MODEL_NAME=lightgbm-iris
  2. INPUT_PATH=@./iris-input.json
  3. SERVICE_HOSTNAME=$(kubectl get inferenceservice lightgbm-iris -o jsonpath='{.status.url}' | cut -d "/" -f 3)
  4. curl -v -H "Host: ${SERVICE_HOSTNAME}" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/$MODEL_NAME:predict -d $INPUT_PATH

Expected Output

  1. *   Trying 169.63.251.68...
  2. * TCP_NODELAY set
  3. * Connected to 169.63.251.68 (169.63.251.68) port 80 (#0)
  4. > POST /models/lightgbm-iris:predict HTTP/1.1
  5. > Host: lightgbm-iris.default.svc.cluster.local
  6. > User-Agent: curl/7.60.0
  7. > Accept: */*
  8. > Content-Length: 76
  9. > Content-Type: application/x-www-form-urlencoded
  10. >
  11. * upload completely sent off: 76 out of 76 bytes
  12. < HTTP/1.1 200 OK
  13. < content-length: 27
  14. < content-type: application/json; charset=UTF-8
  15. < date: Tue, 21 May 2019 22:40:09 GMT
  16. < server: istio-envoy
  17. < x-envoy-upstream-service-time: 13032
  18. <
  19. * Connection #0 to host 169.63.251.68 left intact
  20. {"predictions": [[0.9, 0.05, 0.05]]}

Run LightGBM InferenceService with your own image

Since the KServe LightGBM image is built from a specific version of lightgbm pip package, sometimes it might not be compatible with the pickled model you saved from your training environment, however you can build your own lgbserver image following this instruction.

To use your lgbserver image: - Add the image to the KServe configmap

  1.         "lightgbm": {
  2.             "image": "<your-dockerhub-id>/kserve/lgbserver",
  3.         },

- Specify the runtimeVersion on InferenceService spec

  1. apiVersion: "serving.kserve.io/v1beta1"
  2. kind: "InferenceService"
  3. metadata:
  4.   name: "lightgbm-iris"
  5. spec:
  6.   predictor:
  7.     lightgbm:
  8.       storageUri: "gs://kfserving-examples/models/lightgbm/iris"
  9.       runtimeVersion: X.X.X