Text To Audio

The Text To Audio pipeline generates audio from text.

Example

The following shows a simple example using this pipeline.

from txtai.pipeline import TextToAudio
# Create and run pipeline
tta = TextToAudio()
tta("Describe the audio to generate here")

See the link below for a more detailed example.

Notebook	Description
Generative Audio	Storytelling with generative audio workflows

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml

# Create pipeline using lower case class name
texttoaudio:
# Run pipeline with workflow
workflow:
  tta:
    tasks:
      - action: texttoaudio

Run with Workflows

from txtai import Application
# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("tta", ["Describe the audio to generate here"]))

Run with API

CONFIG=config.yml uvicorn "txtai.api:app" &
curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"tta", "elements":["Describe the audio to generate here"]}'

Methods

Python documentation for the pipeline.

__init__(path=None, quantize=False, gpu=True, model=None, rate=None, **kwargs)

Source code in txtai/pipeline/audio/texttoaudio.py

14
15
16
17
18
19
20
21
22

def init(self, path=None, quantize=False, gpu=True, model=None, rate=None, kwargs):
    if not SCIPY:
        raise ImportError(‘TextToAudio pipeline is not available - install “pipeline” extra to enable.’)
    # Call parent constructor
    super().init(“text-to-audio”, path, quantize, gpu, model, kwargs)
    # Target sample rate, defaults to model sample rate
    self.rate = rate

__call__(text, maxlength=512)

Generates audio from text.

This method supports text as a string or a list. If the input is a string, the return type is a single audio output. If text is a list, the return type is a list.

Parameters:

Returns:

Source code in txtai/pipeline/audio/texttoaudio.py

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

def call(self, text, maxlength=512):
    “””
    Generates audio from text.
    This method supports text as a string or a list. If the input is a string,
    the return type is a single audio output. If text is a list, the return type is a list.
    Args:
        text: text|list
        maxlength: maximum audio length to generate
    Returns:
        list of (audio, sample rate)
    “””
    # Format inputs
    texts = [text] if isinstance(text, str) else text
    # Run pipeline
    results = [self.convert(x) for x in self.pipeline(texts, forward_params={“max_new_tokens”: maxlength})]
    # Extract results
    return results[0] if isinstance(text, str) else results