txtai - RAG (original) (raw)

The Retrieval Augmented Generation (RAG) pipeline joins a prompt, context data store and generative model together to extract knowledge.

The data store can be an embeddings database or a similarity instance with associated input text. The generative model can be a prompt-driven large language model (LLM), an extractive question-answering model or a custom pipeline.

Example

The following shows a simple example using this pipeline.

`` from txtai import Embeddings, RAG

Input data

data = [ "US tops 5 million confirmed virus cases", "Canada's last fully intact ice shelf has suddenly collapsed, " + "forming a Manhattan-sized iceberg", "Beijing mobilises invasion craft along coast as Taiwan tensions escalate", "The National Park Service warns against sacrificing slower friends " + "in a bear attack", "Maine man wins 1Mfrom1M from 1Mfrom25 lottery ticket", "Make huge profits without work, earn up to $100,000 a day" ]

Build embeddings index

embeddings = Embeddings(content=True) embeddings.index(data)

Create the RAG pipeline

rag = RAG(embeddings, "Qwen/Qwen3-0.6B", template=""" Answer the following question using the provided context.

Question: {question}

Context: {context} """)

Run RAG pipeline

rag("What was won?")

Prompts with chat templating can be directly passed

The template format varies by model

rag = RAG(embeddings, "Qwen/Qwen3-0.6B", template=""" <|im_start|>system You are a friendly assistant.<|im_end|> <|im_start|>user Answer the following question using the provided context.

Question: {question}

Context: {context} <|im_start|>assistant """ ) rag("What was won?")

Inputs are automatically converted to chat messages when a

system prompt is provided

rag = RAG( embeddings, "openai/gpt-oss-20b", system="You are a friendly assistant", template=""" Answer the following question using the provided context.

Question: {question}

Context: {context} """) rag("What was won?")

LLM options can be passed as additional arguments

- Streaming RAG response with stream=True

- String inputs are always converted to user messages with defaultrole="user"

- Thinking text is removed with stripthink=True

rag("What was won?", stream=True, defaultrole="user", stripThink=True) ``

See the Embeddings and LLM pages for additional configuration options.

Check out this RAG Quickstart Example. Additional examples are listed below.

Notebook	Description
Prompt-driven search with LLMs	Embeddings-guided and Prompt-driven search with Large Language Models (LLMs)
Build RAG pipelines with txtai ▶️	Guide on retrieval augmented generation including how to create citations
Integrate LLM frameworks	Integrate llama.cpp, LiteLLM and custom generation frameworks
Generate knowledge with Semantic Graphs and RAG	Knowledge exploration and discovery with Semantic Graphs and RAG
Advanced RAG with graph path traversal	Graph path traversal to collect complex sets of data for advanced RAG
Advanced RAG with guided generation	Retrieval Augmented and Guided Generation
RAG with llama.cpp and external API services	RAG with additional vector and LLM frameworks
How RAG with txtai works	Create RAG processes, API services and Docker instances
Speech to Speech RAG ▶️	Full cycle speech to speech workflow with RAG
Parsing the stars with txtai	Explore an astronomical knowledge graph of known stars, planets, galaxies
Chunking your data for RAG	Extract, chunk and index content for effective retrieval
Medical RAG Research with txtai	Analyze PubMed article metadata with RAG
GraphRAG with Wikipedia and GPT OSS	Deep graph search powered RAG
RAG is more than Vector Search	Context retrieval via Web, SQL and other sources

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml

`# Allow documents to be indexed writable: True

Content is required for extractor pipeline

embeddings: content: True

rag: path: Qwen/Qwen3-0.6B template: | Answer the following question using the provided context.

Question:
{question}

Context:
{context}

workflow: search: tasks: - action: rag `

Run with Workflows

Built in tasks make using the extractor pipeline easier.

`from txtai import Application

Create and run pipeline with workflow

app = Application("config.yml") app.add([ "US tops 5 million confirmed virus cases", "Canada's last fully intact ice shelf has suddenly collapsed, " + "forming a Manhattan-sized iceberg", "Beijing mobilises invasion craft along coast as Taiwan tensions escalate", "The National Park Service warns against sacrificing slower friends " + "in a bear attack", "Maine man wins 1Mfrom1M from 1Mfrom25 lottery ticket", "Make huge profits without work, earn up to $100,000 a day" ]) app.index()

list(app.workflow("search", ["What was won?"])) `

Run with API

`CONFIG=config.yml uvicorn "txtai.api:app" &

curl
-X POST "http://localhost:8000/workflow"
-H "Content-Type: application/json"
-d '{"name": "search", "elements": ["What was won"]}' `

Methods

Python documentation for the pipeline.

__init__(similarity, path, quantize=False, gpu=True, model=None, tokenizer=None, minscore=None, mintokens=None, context=None, task=None, output='default', template=None, separator=' ', system=None, **kwargs)

Builds a new RAG pipeline.

Parameters:

Source code in txtai/pipeline/llm/rag.py

__call__(queue, texts=None, **kwargs)

Finds answers to input questions. This method runs queries to find the top n best matches and uses that as the context. A model is then run against the context for each input question, with the answer returned.

Parameters:

Returns:

Source code in txtai/pipeline/llm/rag.py