Objects

pipeline pipeline

The Objects pipeline reads a list of images and returns a list of detected objects.

Example

The following shows a simple example using this pipeline.

  1. from txtai.pipeline import Objects
  3. # Create and run pipeline
  4. objects = Objects()
  5. objects("path to image file")

See the link below for a more detailed example.

NotebookDescription
Generate image captions and detect objectsCaptions and object detection for imagesOpen In Colab

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml

  1. # Create pipeline using lower case class name
  2. objects:
  4. # Run pipeline with workflow
  5. workflow:
  6.   objects:
  7.     tasks:
  8.       - action: objects

Run with Workflows

  1. from txtai import Application
  3. # Create and run pipeline with workflow
  4. app = Application("config.yml")
  5. list(app.workflow("objects", ["path to image file"]))

Run with API

  1. CONFIG=config.yml uvicorn "txtai.api:app" &
  3. curl \
  4.   -X POST "http://localhost:8000/workflow" \
  5.   -H "Content-Type: application/json" \
  6.   -d '{"name":"objects", "elements":["path to image file"]}'

Methods

Python documentation for the pipeline.

__init__(path=None, quantize=False, gpu=True, model=None, classification=False, threshold=0.9, **kwargs)

Source code in txtai/pipeline/image/objects.py

  1. 21
  2. 22
  3. 23
  4. 24
  5. 25
  6. 26
  7. 27
  8. 28
  1. def init(self, path=None, quantize=False, gpu=True, model=None, classification=False, threshold=0.9, kwargs):
  2.     if not PIL:
  3.         raise ImportError(‘Objects pipeline is not available - install “pipeline” extra to enable’)
  5.     super().init(“image-classification” if classification else “object-detection”, path, quantize, gpu, model, kwargs)
  7.     self.classification = classification
  8.     self.threshold = threshold

__call__(images, flatten=False, workers=0)

Applies object detection/image classification models to images. Returns a list of (label, score).

This method supports a single image or a list of images. If the input is an image, the return type is a 1D list of (label, score). If text is a list, a 2D list of (label, score) is returned with a row per image.

Parameters:

NameTypeDescriptionDefault
images

image|list

required
flatten

flatten output to a list of objects

False
workers

number of concurrent workers to use for processing data, defaults to None

0

Returns:

TypeDescription

list of (label, score)

Source code in txtai/pipeline/image/objects.py

  1. 30
  2. 31
  3. 32
  4. 33
  5. 34
  6. 35
  7. 36
  8. 37
  9. 38
  10. 39
  11. 40
  12. 41
  13. 42
  14. 43
  15. 44
  16. 45
  17. 46
  18. 47
  19. 48
  20. 49
  21. 50
  22. 51
  23. 52
  24. 53
  25. 54
  26. 55
  27. 56
  28. 57
  29. 58
  30. 59
  31. 60
  32. 61
  33. 62
  34. 63
  35. 64
  36. 65
  37. 66
  38. 67
  39. 68
  40. 69
  41. 70
  42. 71
  43. 72
  44. 73
  45. 74
  46. 75
  47. 76
  48. 77
  49. 78
  50. 79
  51. 80
  1. def call(self, images, flatten=False, workers=0):
  2.     “””
  3.     Applies object detection/image classification models to images. Returns a list of (label, score).
  5.     This method supports a single image or a list of images. If the input is an image, the return
  6.     type is a 1D list of (label, score). If text is a list, a 2D list of (label, score) is
  7.     returned with a row per image.
  9.     Args:
  10.         images: image|list
  11.         flatten: flatten output to a list of objects
  12.         workers: number of concurrent workers to use for processing data, defaults to None
  14.     Returns:
  15.         list of (label, score)
  16.     “””
  18.     # Convert single element to list
  19.     values = [images] if not isinstance(images, list) else images
  21.     # Open images if file strings
  22.     values = [Image.open(image) if isinstance(image, str) else image for image in values]
  24.     # Run pipeline
  25.     results = (
  26.         self.pipeline(values, num_workers=workers)
  27.         if self.classification
  28.         else self.pipeline(values, threshold=self.threshold, num_workers=workers)
  29.     )
  31.     # Build list of (id, score)
  32.     outputs = []
  33.     for result in results:
  34.         # Convert to (label, score) tuples
  35.         result = [(x[“label”], x[“score”]) for x in result if x[“score”] > self.threshold]
  37.         # Sort by score descending
  38.         result = sorted(result, key=lambda x: x[1], reverse=True)
  40.         # Deduplicate labels
  41.         unique = set()
  42.         elements = []
  43.         for label, score in result:
  44.             if label not in unique:
  45.                 elements.append(label if flatten else (label, score))
  46.                 unique.add(label)
  48.         outputs.append(elements)
  50.     # Return single element if single element passed in
  51.     return outputs[0] if not isinstance(images, list) else outputs