Description

Generalized Linear Model stream predict. https://en.wikipedia.org/wiki/Generalized_linear_model.

Parameters

Name Description Type Required？ Default Value
linkPredResultCol link predict col name of output String null
predictionCol Column name of prediction. String ✓

Script Example

Code

  2. # data
  3. data = np.array([
  4.     [1.6094,118.0000,69.0000,1.0000,2.0000],
  5.     [2.3026,58.0000,35.0000,1.0000,2.0000],
  6.     [2.7081,42.0000,26.0000,1.0000,2.0000],
  7.     [2.9957,35.0000,21.0000,1.0000,2.0000],
  8.     [3.4012,27.0000,18.0000,1.0000,2.0000],
  9.     [3.6889,25.0000,16.0000,1.0000,2.0000],
  10.     [4.0943,21.0000,13.0000,1.0000,2.0000],
  11.     [4.3820,19.0000,12.0000,1.0000,2.0000],
  12.     [4.6052,18.0000,12.0000,1.0000,2.0000]
  13. ])
  16. df = pd.DataFrame({"u": data[:, 0], "lot1": data[:, 1], "lot2": data[:, 2], "offset": data[:, 3], "weights": data[:, 4]})
  17. source = dataframeToOperator(df, schemaStr='u double, lot1 double, lot2 double, offset double, weights double', op_type='batch')
  20. featureColNames = ["lot1", "lot2"]
  21. labelColName = "u"
  23. # train
  24. train = GlmTrainBatchOp()\
  25.                 .setFamily("gamma")\
  26.                 .setLink("Log")\
  27.                 .setRegParam(0.3)\
  28.                 .setMaxIter(5)\
  29.                 .setFeatureCols(featureColNames)\
  30.                 .setLabelCol(labelColName)
  32. source.link(train)
  34. #  batch predict
  35. predict =  GlmPredictBatchOp()\
  36.                 .setPredictionCol("pred")
  38. predict.linkFrom(train, source)
  39. predict.print()
  41. # eval
  42. eval =  GlmEvaluationBatchOp()\
  43.                 .setFamily("gamma")\
  44.                 .setLink("Log")\
  45.                 .setRegParam(0.3)\
  46.                 .setMaxIter(5)\
  47.                 .setFeatureCols(featureColNames)\
  48.                 .setLabelCol(labelColName)
  50. eval.linkFrom(train, source)
  51. eval.print()
  54. # stream predict
  55. source_stream = dataframeToOperator(df, schemaStr='u double, lot1 double, lot2 double, offset double, weights double', op_type='stream')
  57. predict_stream =  GlmPredictStreamOp(train)\
  58.                 .setPredictionCol("pred")
  60. predict_stream.linkFrom(source_stream)
  61. predict_stream.print()

Results

u lot1 lot2 offset weights pred
0 1.6094 118.0 69.0 1.0 2.0 0.378525
1 2.3026 58.0 35.0 1.0 2.0 0.970639
2 2.7081 42.0 26.0 1.0 2.0 1.126458
3 2.9957 35.0 21.0 1.0 2.0 1.227753
4 3.4012 27.0 18.0 1.0 2.0 1.258898
5 3.6889 25.0 16.0 1.0 2.0 1.305654
6 4.0943 21.0 13.0 1.0 2.0 1.367991
7 4.3820 19.0 12.0 1.0 2.0 1.383571
8 4.6052 18.0 12.0 1.0 2.0 1.375774