similarity instance (embeddings or similarity pipeline)

path to model, supports a LLM, Questions or custom pipeline

True if model should be quantized before inference, False otherwise.

if gpu inference should be used (only works if GPUs are available)

optional existing pipeline model to wrap

Tokenizer class

minimum score to include context match, defaults to None

minimum number of tokens to include context match, defaults to None

topn context matches to include, defaults to 3

model task (language-generation, sequence-sequence or question-answering), defaults to auto-detect

output format, ‘default’ returns (name, answer), ‘flatten’ returns answers and ‘reference’ returns (name, answer, reference)

prompt template, it must have a parameter for {question} and {context}, defaults to “{question} {context}”

context separator

additional keyword arguments to pass to pipeline model

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

def init(
    self,
    similarity,
    path,
    quantize=False,
    gpu=True,
    model=None,
    tokenizer=None,
    minscore=None,
    mintokens=None,
    context=None,
    task=None,
    output=”default”,
    template=None,
    separator=” “,
    kwargs,
):
    “””
    Builds a new extractor.
    Args:
        similarity: similarity instance (embeddings or similarity pipeline)
        path: path to model, supports a LLM, Questions or custom pipeline
        quantize: True if model should be quantized before inference, False otherwise.
        gpu: if gpu inference should be used (only works if GPUs are available)
        model: optional existing pipeline model to wrap
        tokenizer: Tokenizer class
        minscore: minimum score to include context match, defaults to None
        mintokens: minimum number of tokens to include context match, defaults to None
        context: topn context matches to include, defaults to 3
        task: model task (language-generation, sequence-sequence or question-answering), defaults to auto-detect
        output: output format, ‘default’ returns (name, answer), ‘flatten’ returns answers and ‘reference’ returns (name, answer, reference)
        template: prompt template, it must have a parameter for {question} and {context}, defaults to “{question} {context}”
        separator: context separator
        kwargs: additional keyword arguments to pass to pipeline model
    “””
    # Similarity instance
    self.similarity = similarity
    # Model can be a LLM, Questions or custom pipeline
    self.model = self.load(path, quantize, gpu, model, task, kwargs)
    # Tokenizer class use default method if not set
    self.tokenizer = tokenizer if tokenizer else Tokenizer() if hasattr(self.similarity, “scoring”) and self.similarity.isweighted() else None
    # Minimum score to include context match
    self.minscore = minscore if minscore is not None else 0.0
    # Minimum number of tokens to include context match
    self.mintokens = mintokens if mintokens is not None else 0.0
    # Top n context matches to include for context
    self.context = context if context else 3
    # Output format
    self.output = output
    # Prompt template
    self.template = template if template else “{question} {context}”
    # Context separator
    self.separator = separator

input question queue (name, query, question, snippet), can be list of tuples/dicts/strings or a single input element

optional list of text for context, otherwise runs embeddings search

additional keyword arguments to pass to pipeline model

list of answers matching input format (tuple or dict) containing fields as specified by output format

 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139

def call(self, queue, texts=None, kwargs):
    “””
    Finds answers to input questions. This method runs queries to find the top n best matches and uses that as the context.
    A model is then run against the context for each input question, with the answer returned.
    Args:
        queue: input question queue (name, query, question, snippet), can be list of tuples/dicts/strings or a single input element
        texts: optional list of text for context, otherwise runs embeddings search
        kwargs: additional keyword arguments to pass to pipeline model
    Returns:
        list of answers matching input format (tuple or dict) containing fields as specified by output format
    “””
    # Save original queue format
    inputs = queue
    # Convert queue to list, if necessary
    queue = queue if isinstance(queue, list) else [queue]
    # Convert dictionary inputs to tuples
    if queue and isinstance(queue[0], dict):
        # Convert dict to tuple
        queue = [tuple(row.get(x) for x in [“name”, “query”, “question”, “snippet”]) for row in queue]
    if queue and isinstance(queue[0], str):
        # Convert string questions to tuple
        queue = [(None, row, row, None) for row in queue]
    # Rank texts by similarity for each query
    results = self.query([query for , query, ,  in queue], texts)
    # Build question-context pairs
    names, queries, questions, contexts, topns, snippets = [], [], [], [], [], []
    for x, (name, query, question, snippet) in enumerate(queue):
        # Get top n best matching segments
        topn = sorted(results[x], key=lambda y: y[2], reverse=True)[: self.context]
        # Generate context using ordering from texts, if available, otherwise order by score
        context = self.separator.join(text for , text,  in (sorted(topn, key=lambda y: y[0]) if texts else topn))
        names.append(name)
        queries.append(query)
        questions.append(question)
        contexts.append(context)
        topns.append(topn)
        snippets.append(snippet)
    # Run pipeline and return answers
    answers = self.answers(names, questions, contexts, [[text for , text, _ in topn] for topn in topns], snippets, kwargs)
    # Apply output formatting to answers and return
    return self.apply(inputs, queries, answers, topns)