Description

Calculate the evaluation data for multi classifiction.

You can either give label column and predResult column or give label column and predDetail column. Once predDetail column is given, the predResult column is ignored.

The labels are sorted in descending order in the output label array and confusion matrix..

Parameters

Name Description Type Required？ Default Value
labelCol Name of the label column in the input table String ✓
predictionCol Column name of prediction. String ✓
predictionDetailCol Column name of prediction result, it will include detailed info. String

Script Example

Code

  1. import numpy as np
  2. import pandas as pd
  3. data = np.array([
  4.     ["prefix1", "{\"prefix1\": 0.9, \"prefix0\": 0.1}"],
  5.     ["prefix1", "{\"prefix1\": 0.8, \"prefix0\": 0.2}"],
  6.     ["prefix1", "{\"prefix1\": 0.7, \"prefix0\": 0.3}"],
  7.     ["prefix0", "{\"prefix1\": 0.75, \"prefix0\": 0.25}"],
  8.     ["prefix0", "{\"prefix1\": 0.6, \"prefix0\": 0.4}"]])
  10. df = pd.DataFrame({"label": data[:, 0], "detailInput": data[:, 1]})
  11. inOp = BatchOperator.fromDataframe(df, schemaStr='label string, detailInput string')
  13. metrics = EvalMultiClassBatchOp().setLabelCol("label").setPredictionDetailCol("detailInput").linkFrom(inOp).collectMetrics()
  14. print("Prefix0 accuracy:", metrics.getAccuracy("prefix0"))
  15. print("Prefix1 recall:", metrics.getRecall("prefix1"))
  16. print("Macro Precision:", metrics.getMacroPrecision())
  17. print("Micro Recall:", metrics.getMicroRecall())
  18. print("Weighted Sensitivity:", metrics.getWeightedSensitivity())
  21. inOp = StreamOperator.fromDataframe(df, schemaStr='label string, detailInput string')
  22. EvalMultiClassStreamOp().setLabelCol("label").setPredictionDetailCol("detailInput").setTimeInterval(1).linkFrom(inOp).print()
  23. StreamOperator.execute()

Results

  1. Prefix0 accuracy: 0.6
  2. Prefix1 recall: 1.0
  3. Macro Precision: 0.3
  4. Micro Recall: 0.6
  5. Weighted Sensitivity: 0.6