Description

Gradient Boosting(often abbreviated to GBDT or GBM) is a popular supervised learning model. It is the best off-the-shelf supervised learning model for a wide range of problems, especially problems with medium to large data size.

This implementation use histogram-based algorithm. See: “Mcrank: Learning to rank using multiple classification and gradient boosting”, Ping Li et al., NIPS 2007, for detail and experiments on histogram-based algorithm.

This implementation use layer-wise tree growing strategy, rather than leaf-wise tree growing strategy (like the one in “Lightgbm: A highly efficient gradient boosting decision tree”, Guolin Ke et al., NIPS 2017), because we found the former being faster in flink-based distributed computing environment.

This implementation use data-parallel algorithm. See: “A communication-efficient parallel algorithm for decision tree”, Qi Meng et al., NIPS 2016 for an introduction on data-parallel, feature-parallel, etc., algorithms to construct decision forests.

Parameters

Name Description Type Required？ Default Value
algoType null Integer
learningRate learning rate for gbdt training(default 0.3) Double 0.3
minSumHessianPerLeaf minimum sum hessian for each leaf Double 0.0
numTrees Number of decision trees. Integer 100
minSamplesPerLeaf Minimal number of sample in one leaf. Integer 100
maxDepth depth of the tree Integer 6
subsamplingRatio Ratio of the training samples used for learning each decision tree. Double 1.0
featureSubsamplingRatio Ratio of the features used in each tree, in range (0, 1]. Double 1.0
groupCol Name of a grouping column String null
maxBins MAX number of bins for continuous feature Integer 128
featureCols Names of the feature columns used for training in the input table String[] ✓
labelCol Name of the label column in the input table String ✓
categoricalCols Names of the categorical columns used for training in the input table String[]
weightCol Name of the column indicating weight String null
maxLeaves max leaves of tree Integer 2147483647
minSampleRatioPerChild Minimal value of: (num of samples in child)/(num of samples in its parent). Double 0.0
minInfoGain minimum info gain when performing split Double 0.0

Script Example

  1. import numpy as np
  2. import pandas as pd
  3. from pyalink.alink import *
  6. def exampleData():
  7.     return np.array([
  8.         [1.0, "A", 0, 0, 0],
  9.         [2.0, "B", 1, 1, 0],
  10.         [3.0, "C", 2, 2, 1],
  11.         [4.0, "D", 3, 3, 1]
  12.     ])
  15. def sourceFrame():
  16.     data = exampleData()
  17.     return pd.DataFrame({
  18.         "f0": data[:, 0],
  19.         "f1": data[:, 1],
  20.         "f2": data[:, 2],
  21.         "f3": data[:, 3],
  22.         "label": data[:, 4]
  23.     })
  26. def batchSource():
  27.     return dataframeToOperator(
  28.         sourceFrame(),
  29.         schemaStr='''
  30.     f0 double, 
  31.     f1 string, 
  32.     f2 int, 
  33.     f3 int, 
  34.     label int
  35.     ''',
  36.         op_type='batch'
  37.     )
  40. def streamSource():
  41.     return dataframeToOperator(
  42.         sourceFrame(),
  43.         schemaStr='''
  44.     f0 double, 
  45.     f1 string, 
  46.     f2 int, 
  47.     f3 int, 
  48.     label int
  49.     ''',
  50.         op_type='stream'
  51.     )
  54. trainOp = (
  55.     GbdtRegTrainBatchOp()
  56.     .setLearningRate(1.0)
  57.     .setNumTrees(3)
  58.     .setMinSamplesPerLeaf(1)
  59.     .setLabelCol('label')
  60.     .setFeatureCols(['f0', 'f1', 'f2', 'f3'])
  61. )
  63. predictBatchOp = (
  64.     GbdtRegPredictBatchOp()
  65.     .setPredictionCol('pred')
  66. )
  68. (
  69.     predictBatchOp
  70.     .linkFrom(
  71.         batchSource().link(trainOp),
  72.         batchSource()
  73.     )
  74.     .print()
  75. )
  77. predictStreamOp = (
  78.     GbdtRegPredictStreamOp(
  79.         batchSource().link(trainOp)
  80.     )
  81.     .setPredictionCol('pred')
  82. )
  84. (
  85.     predictStreamOp
  86.     .linkFrom(
  87.         streamSource()
  88.     )
  89.     .print()
  90. )
  92. StreamOperator.execute()

Result

Batch prediction

  1.     f0 f1  f2  f3  label  pred
  2. 0  1.0  A   0   0      0   0.0
  3. 1  2.0  B   1   1      0   0.0
  4. 2  3.0  C   2   2      1   1.0
  5. 3  4.0  D   3   3      1   1.0

Stream Prediction

  1.     f0    f1    f2    f3    label    pred
  2. 0    1.0    A    0    0    0    0.0
  3. 1    3.0    C    2    2    1    1.0
  4. 2    2.0    B    1    1    0    0.0
  5. 3    4.0    D    3    3    1    1.0