Extractor

pipeline pipeline

The Extractor pipeline is a combination of a similarity instance (embeddings or similarity pipeline) to build a question context and a model that answers questions. The model can be a prompt-driven large language model (LLM), an extractive question-answering model or a custom pipeline.

Example

The following shows a simple example using this pipeline.

  1. from txtai.embeddings import Embeddings
  2. from txtai.pipeline import Extractor
  3. # Embeddings model ranks candidates before passing to QA pipeline
  4. embeddings = Embeddings({"path": "sentence-transformers/nli-mpnet-base-v2"})
  5. # Create and run pipeline
  6. extractor = Extractor(embeddings, "distilbert-base-cased-distilled-squad")
  7. extractor([["What was won"] * 3 + [False]],
  8. ["Maine man wins $1M from $25 lottery ticket"])

See the links below for more detailed examples.

NotebookDescription
Extractive QA with txtaiIntroduction to extractive question-answering with txtaiOpen In Colab
Extractive QA with ElasticsearchRun extractive question-answering queries with ElasticsearchOpen In Colab
Extractive QA to build structured dataBuild structured datasets using extractive question-answeringOpen In Colab
Prompt-driven search with LLMsEmbeddings-guided and Prompt-driven search with Large Language Models (LLMs)Open In Colab

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml

  1. # Create pipeline using lower case class name
  2. extractor:

Run with Workflows

  1. from txtai.app import Application
  2. # Create and run pipeline with workflow
  3. app = Application("config.yml")
  4. list(app.extract([{"name": "What was won", "query": "What was won",
  5. "question", "What was won", "snippet": False}],
  6. ["Maine man wins $1M from $25 lottery ticket"]))

Run with API

  1. CONFIG=config.yml uvicorn "txtai.api:app" &
  2. curl \
  3. -X POST "http://localhost:8000/extract" \
  4. -H "Content-Type: application/json" \
  5. -d '{"queue": [{"name":"What was won", "query": "What was won", "question": "What was won", "snippet": false}], "texts": ["Maine man wins $1M from $25 lottery ticket"]}'

Methods

Python documentation for the pipeline.

__init__(self, similarity, path, quantize=False, gpu=True, model=None, tokenizer=None, minscore=None, mintokens=None, context=None, task=None, output='default') special

Builds a new extractor.

Parameters:

NameTypeDescriptionDefault
similarity

similarity instance (embeddings or similarity pipeline)

required
path

path to model, supports Questions, Generator, Sequences or custom pipeline

required
quantize

True if model should be quantized before inference, False otherwise.

False
gpu

if gpu inference should be used (only works if GPUs are available)

True
model

optional existing pipeline model to wrap

None
tokenizer

Tokenizer class

None
minscore

minimum score to include context match, defaults to None

None
mintokens

minimum number of tokens to include context match, defaults to None

None
context

topn context matches to include, defaults to 3

None
task

model task (language-generation, sequence-sequence or question-answering), defaults to auto-detect

None
output

output format, ‘default’ returns (name, answer), ‘flatten’ returns answers and ‘reference’ returns (name, answer, reference)

‘default’

Source code in txtai/pipeline/text/extractor.py

  1. def __init__(
  2. self,
  3. similarity,
  4. path,
  5. quantize=False,
  6. gpu=True,
  7. model=None,
  8. tokenizer=None,
  9. minscore=None,
  10. mintokens=None,
  11. context=None,
  12. task=None,
  13. output="default",
  14. ):
  15. """
  16. Builds a new extractor.
  17. Args:
  18. similarity: similarity instance (embeddings or similarity pipeline)
  19. path: path to model, supports Questions, Generator, Sequences or custom pipeline
  20. quantize: True if model should be quantized before inference, False otherwise.
  21. gpu: if gpu inference should be used (only works if GPUs are available)
  22. model: optional existing pipeline model to wrap
  23. tokenizer: Tokenizer class
  24. minscore: minimum score to include context match, defaults to None
  25. mintokens: minimum number of tokens to include context match, defaults to None
  26. context: topn context matches to include, defaults to 3
  27. task: model task (language-generation, sequence-sequence or question-answering), defaults to auto-detect
  28. output: output format, 'default' returns (name, answer), 'flatten' returns answers and 'reference' returns (name, answer, reference)
  29. """
  30. # Similarity instance
  31. self.similarity = similarity
  32. # Question-Answer model. Can be prompt-driven LLM or extractive qa
  33. self.model = self.load(path, quantize, gpu, model, task)
  34. # Tokenizer class use default method if not set
  35. self.tokenizer = tokenizer if tokenizer else Tokenizer() if hasattr(self.similarity, "scoring") and self.similarity.scoring else None
  36. # Minimum score to include context match
  37. self.minscore = minscore if minscore is not None else 0.0
  38. # Minimum number of tokens to include context match
  39. self.mintokens = mintokens if mintokens is not None else 0.0
  40. # Top n context matches to include for context
  41. self.context = context if context else 3
  42. # Output format
  43. self.output = output

__call__(self, queue, texts=None) special

Finds answers to input questions. This method runs queries to find the top n best matches and uses that as the context. A model is then run against the context for each input question, with the answer returned.

Parameters:

NameTypeDescriptionDefault
queue

input question queue (name, query, question, snippet), can be list of tuples or dicts

required
texts

optional list of text for context, otherwise runs embeddings search

None

Returns:

TypeDescription

list of answers matching input format (tuple or dict) containing fields as specified by output format

Source code in txtai/pipeline/text/extractor.py

  1. def __call__(self, queue, texts=None):
  2. """
  3. Finds answers to input questions. This method runs queries to find the top n best matches and uses that as the context.
  4. A model is then run against the context for each input question, with the answer returned.
  5. Args:
  6. queue: input question queue (name, query, question, snippet), can be list of tuples or dicts
  7. texts: optional list of text for context, otherwise runs embeddings search
  8. Returns:
  9. list of answers matching input format (tuple or dict) containing fields as specified by output format
  10. """
  11. # Save original queue format
  12. inputs = queue
  13. # Convert dictionary inputs to tuples
  14. if isinstance(queue[0], dict):
  15. # Convert dict to tuple
  16. queue = [tuple(row.get(x) for x in ["name", "query", "question", "snippet"]) for row in queue]
  17. # Rank texts by similarity for each query
  18. results = self.query([query for _, query, _, _ in queue], texts)
  19. # Build question-context pairs
  20. names, queries, questions, contexts, topns, snippets = [], [], [], [], [], []
  21. for x, (name, query, question, snippet) in enumerate(queue):
  22. # Get top n best matching segments
  23. topn = sorted(results[x], key=lambda y: y[2], reverse=True)[: self.context]
  24. # Generate context using ordering from texts, if available, otherwise order by score
  25. context = " ".join(text for _, text, _ in (sorted(topn, key=lambda y: y[0]) if texts else topn))
  26. names.append(name)
  27. queries.append(query)
  28. questions.append(question)
  29. contexts.append(context)
  30. topns.append(topn)
  31. snippets.append(snippet)
  32. # Run pipeline and return answers
  33. answers = self.answers(names, questions, contexts, [[text for _, text, _ in topn] for topn in topns], snippets)
  34. # Apply output formatting to answers and return
  35. return self.apply(inputs, queries, answers, topns)