Note: rag.py is adapted from Underfitted's demo posted here: https://github.com/svpino/gentle-intro-to-rag/blob/main/rag.ipynb

RAG (Retrieval-Augmented Generation) Demo

This project demonstrates a simple RAG application that answers questions based on content from a plain text file ("grimmstales.txt") using a local embedding API and the Llama 3.1 model.

Setup

1. Activate the virtual environment:

   source .venv/bin/activate  # On Linux/macOS
   # or
   .\.venv\Scripts\activate  # On Windows

2. Install dependencies:

   pip install -r requirements.txt

3. Make sure you have Ollama or LM Studio installed and running with the Llama 3.1 model:

   ollama pull llama3.1
   # or use LM Studio with the appropriate model loaded

4. You'll need a text file named grimmstales.txt in the same directory (this demo uses the public domain "Grimms' Fairy Tales").

   • You can use any large text file by changing the TEXT_FILE variable in rag.py.

Running the Script

Make sure your virtual environment is activated, then run:

python rag.py

How it Works

This script:

1. Loads a plain text document ("grimmstales.txt")
2. Splits it into manageable chunks
3. Creates vector embeddings for each chunk using a local embedding API (compatible with LM Studio or similar)
4. Stores the embeddings in a FAISS vector store
5. Sets up a retrieval system to find relevant chunks based on questions
6. Configures an LLM (Llama 3.1 via Ollama or LM Studio) to generate answers
7. Combines everything into a RAG pipeline that answers questions based on the text file content

Example Questions

Here are some example questions you can use to demo the RAG system with "Grimms' Fairy Tales":

1. What lesson does the story of "Hansel and Gretel" teach about resourcefulness?
2. Which tale features a character who can spin straw into gold?
3. In "The Twelve Dancing Princesses," how do the princesses manage to escape each night?

Original Source

This script was converted from a Jupyter notebook created by Santiago Valdarrama (svpino): https://github.com/svpino/gentle-intro-to-rag/blob/main/rag.ipynb