Labels

pipeline pipeline

The Labels pipeline uses a text classification model to apply labels to input text. This pipeline can classify text using either a zero shot model (dynamic labeling) or a standard text classification model (fixed labeling).

Example

The following shows a simple example using this pipeline.

  1. from txtai.pipeline import Labels
  3. # Create and run pipeline
  4. labels = Labels()
  5. labels(
  6.     ["Great news", "That's rough"],
  7.     ["positive", "negative"]
  8. )

See the link below for a more detailed example.

NotebookDescription
Apply labels with zero shot classificationUse zero shot learning for labeling, classification and topic modelingOpen In Colab

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml

  1. # Create pipeline using lower case class name
  2. labels:
  4. # Run pipeline with workflow
  5. workflow:
  6.   labels:
  7.     tasks:
  8.       - action: labels
  9.         args: [["positive", "negative"]]

Run with Workflows

  1. from txtai import Application
  3. # Create and run pipeline with workflow
  4. app = Application("config.yml")
  5. list(app.workflow("labels", ["Great news", "That's rough"]))

Run with API

  1. CONFIG=config.yml uvicorn "txtai.api:app" &
  3. curl \
  4.   -X POST "http://localhost:8000/workflow" \
  5.   -H "Content-Type: application/json" \
  6.   -d '{"name":"labels", "elements": ["Great news", "Thats rough"]}'

Methods

Python documentation for the pipeline.

__init__(path=None, quantize=False, gpu=True, model=None, dynamic=True, **kwargs)

Source code in txtai/pipeline/text/labels.py

  1. 13
  2. 14
  3. 15
  4. 16
  5. 17
  1. def init(self, path=None, quantize=False, gpu=True, model=None, dynamic=True, kwargs):
  2.     super().init(“zero-shot-classification” if dynamic else “text-classification”, path, quantize, gpu, model, kwargs)
  4.     # Set if labels are dynamic (zero shot) or fixed (standard text classification)
  5.     self.dynamic = dynamic

__call__(text, labels=None, multilabel=False, flatten=None, workers=0)

Applies a text classifier to text. Returns a list of (id, score) sorted by highest score, where id is the index in labels. For zero shot classification, a list of labels is required. For text classification models, a list of labels is optional, otherwise all trained labels are returned.

This method supports text as a string or a list. If the input is a string, the return type is a 1D list of (id, score). If text is a list, a 2D list of (id, score) is returned with a row per string.

Parameters:

NameTypeDescriptionDefault
text

text|list

required
labels

list of labels

None
multilabel

labels are independent if True, scores are normalized to sum to 1 per text item if False, raw scores returned if None

False
flatten

flatten output to a list of labels if present. Accepts a boolean or float value to only keep scores greater than that number.

None
workers

number of concurrent workers to use for processing data, defaults to None

0

Returns:

TypeDescription

list of (id, score) or list of labels depending on flatten parameter

Source code in txtai/pipeline/text/labels.py

  1. 19
  2. 20
  3. 21
  4. 22
  5. 23
  6. 24
  7. 25
  8. 26
  9. 27
  10. 28
  11. 29
  12. 30
  13. 31
  14. 32
  15. 33
  16. 34
  17. 35
  18. 36
  19. 37
  20. 38
  21. 39
  22. 40
  23. 41
  24. 42
  25. 43
  26. 44
  27. 45
  28. 46
  29. 47
  30. 48
  31. 49
  32. 50
  33. 51
  34. 52
  35. 53
  36. 54
  37. 55
  38. 56
  1. def call(self, text, labels=None, multilabel=False, flatten=None, workers=0):
  2.     “””
  3.     Applies a text classifier to text. Returns a list of (id, score) sorted by highest score,
  4.     where id is the index in labels. For zero shot classification, a list of labels is required.
  5.     For text classification models, a list of labels is optional, otherwise all trained labels are returned.
  7.     This method supports text as a string or a list. If the input is a string, the return
  8.     type is a 1D list of (id, score). If text is a list, a 2D list of (id, score) is
  9.     returned with a row per string.
  11.     Args:
  12.         text: text|list
  13.         labels: list of labels
  14.         multilabel: labels are independent if True, scores are normalized to sum to 1 per text item if False, raw scores returned if None
  15.         flatten: flatten output to a list of labels if present. Accepts a boolean or float value to only keep scores greater than that number.
  16.         workers: number of concurrent workers to use for processing data, defaults to None
  18.     Returns:
  19.         list of (id, score) or list of labels depending on flatten parameter
  20.     “””
  22.     if self.dynamic:
  23.         # Run zero shot classification pipeline
  24.         results = self.pipeline(text, labels, multi_label=multilabel, truncation=True, num_workers=workers)
  25.     else:
  26.         # Set classification function based on inputs
  27.         function = “none” if multilabel is None else “sigmoid” if multilabel or len(self.labels()) == 1 else “softmax”
  29.         # Run text classification pipeline
  30.         results = self.pipeline(text, top_k=None, function_to_apply=function, num_workers=workers)
  32.     # Convert results to a list if necessary
  33.     if isinstance(text, str):
  34.         results = [results]
  36.     # Build list of outputs and return
  37.     outputs = self.outputs(results, labels, flatten)
  38.     return outputs[0] if isinstance(text, str) else outputs