Description

Gradient Boosting(often abbreviated to GBDT or GBM) is a popular supervised learning model. It is the best off-the-shelf supervised learning model for a wide range of problems, especially problems with medium to large data size.

This implementation use histogram-based algorithm. See: “Mcrank: Learning to rank using multiple classification and gradient boosting”, Ping Li et al., NIPS 2007, for detail and experiments on histogram-based algorithm.

This implementation use layer-wise tree growing strategy, rather than leaf-wise tree growing strategy (like the one in “Lightgbm: A highly efficient gradient boosting decision tree”, Guolin Ke et al., NIPS 2017), because we found the former being faster in flink-based distributed computing environment.

This implementation use data-parallel algorithm. See: “A communication-efficient parallel algorithm for decision tree”, Qi Meng et al., NIPS 2016 for an introduction on data-parallel, feature-parallel, etc., algorithms to construct decision forests.

Parameters

Name Description Type Required？ Default Value
predictionCol Column name of prediction. String ✓
predictionDetailCol Column name of prediction result, it will include detailed info. String
reservedCols Names of the columns to be retained in the output table String[] null

Script Example

Script

  1. import numpy as np
  2. import pandas as pd
  3. from pyalink.alink import *
  6. def exampleData():
  7.     return np.array([
  8.         [1.0, "A", 0, 0, 0],
  9.         [2.0, "B", 1, 1, 0],
  10.         [3.0, "C", 2, 2, 1],
  11.         [4.0, "D", 3, 3, 1]
  12.     ])
  15. def sourceFrame():
  16.     data = exampleData()
  17.     return pd.DataFrame({
  18.         "f0": data[:, 0],
  19.         "f1": data[:, 1],
  20.         "f2": data[:, 2],
  21.         "f3": data[:, 3],
  22.         "label": data[:, 4]
  23.     })
  26. def batchSource():
  27.     return dataframeToOperator(
  28.         sourceFrame(),
  29.         schemaStr='''
  30.     f0 double, 
  31.     f1 string, 
  32.     f2 int, 
  33.     f3 int, 
  34.     label int
  35.     ''',
  36.         op_type='batch'
  37.     )
  40. def streamSource():
  41.     return dataframeToOperator(
  42.         sourceFrame(),
  43.         schemaStr='''
  44.     f0 double, 
  45.     f1 string, 
  46.     f2 int, 
  47.     f3 int, 
  48.     label int
  49.     ''',
  50.         op_type='stream'
  51.     )
  54. trainOp = (
  55.     GbdtRegTrainBatchOp()
  56.     .setLearningRate(1.0)
  57.     .setNumTrees(3)
  58.     .setMinSamplesPerLeaf(1)
  59.     .setLabelCol('label')
  60.     .setFeatureCols(['f0', 'f1', 'f2', 'f3'])
  61. )
  63. predictBatchOp = (
  64.     GbdtRegPredictBatchOp()
  65.     .setPredictionCol('pred')
  66. )
  68. (
  69.     predictBatchOp
  70.     .linkFrom(
  71.         batchSource().link(trainOp),
  72.         batchSource()
  73.     )
  74.     .print()
  75. )
  77. predictStreamOp = (
  78.     GbdtRegPredictStreamOp(
  79.         batchSource().link(trainOp)
  80.     )
  81.     .setPredictionCol('pred')
  82. )
  84. (
  85.     predictStreamOp
  86.     .linkFrom(
  87.         streamSource()
  88.     )
  89.     .print()
  90. )
  92. StreamOperator.execute()

Result

Batch prediction

  1.     f0 f1  f2  f3  label  pred
  2. 0  1.0  A   0   0      0   0.0
  3. 1  2.0  B   1   1      0   0.0
  4. 2  3.0  C   2   2      1   1.0
  5. 3  4.0  D   3   3      1   1.0