Description

Fit a quantile discretizer model.

Parameters

Name Description Type Required？ Default Value
selectedCols Names of the columns used for processing String[] ✓
numBuckets number of buckets Integer 2
numBucketsArray Array of num bucket Integer[] null
leftOpen left open Boolean true

Script Example

Code

  1. import numpy as np
  2. import pandas as pd
  3. from pyalink.alink import *
  6. def exampleData():
  7.     return np.array([
  8.         ["a", 1, 1, 2.0, True],
  9.         ["c", 1, 2, -3.0, True],
  10.         ["a", 2, 2, 2.0, False],
  11.         ["c", 0, 0, 0.0, False]
  12.     ])
  15. def sourceFrame():
  16.     data = exampleData()
  17.     return pd.DataFrame({
  18.         "f_string": data[:, 0],
  19.         "f_long": data[:, 1],
  20.         "f_int": data[:, 2],
  21.         "f_double": data[:, 3],
  22.         "f_boolean": data[:, 4]
  23.     })
  26. def batchSource():
  27.     return dataframeToOperator(
  28.         sourceFrame(),
  29.         schemaStr='''
  30.     f_string string, 
  31.     f_long long, 
  32.     f_int int, 
  33.     f_double double, 
  34.     f_boolean boolean
  35.     ''',
  36.         op_type='batch'
  37.     )
  40. def streamSource():
  41.     return dataframeToOperator(
  42.         sourceFrame(),
  43.         schemaStr='''
  44.     f_string string, 
  45.     f_long long, 
  46.     f_int int, 
  47.     f_double double, 
  48.     f_boolean boolean
  49.     ''',
  50.         op_type='stream'
  51.     )
  54. trainOp = (
  55.     QuantileDiscretizerTrainBatchOp()
  56.     .setSelectedCols(['f_double'])
  57.     .setNumBuckets(8)
  58. )
  60. predictBatchOp = (
  61.     QuantileDiscretizerPredictBatchOp()
  62.     .setSelectedCols(['f_double'])
  63. )
  65. (
  66.     predictBatchOp
  67.     .linkFrom(
  68.         batchSource().link(trainOp),
  69.         batchSource()
  70.     )
  71.     .print()
  72. )
  74. predictStreamOp = (
  75.     QuantileDiscretizerPredictStreamOp(
  76.         batchSource().link(trainOp)
  77.     )
  78.     .setSelectedCols(['f_double'])
  79. )
  81. (
  82.     predictStreamOp
  83.     .linkFrom(
  84.         streamSource()
  85.     )
  86.     .print()
  87. )
  89. StreamOperator.execute()

Result

Batch prediction

  1. f_string  f_long  f_int  f_double  f_boolean
  2. 0        a       1      1         2       True
  3. 1        c       1      2         0       True
  4. 2        a       2      2         2      False
  5. 3        c       0      0         1      False

Stream Prediction

  1. f_string    f_long    f_int    f_double    f_boolean
  2. 0    a    1    1    2    True
  3. 1    a    2    2    2    False
  4. 2    c    1    2    0    True
  5. 3    c    0    0    1    False