Self-RAG: Self-Reflective Retrieval-Augmented Generation

Intelligent question-answering with built-in self-critique and reasoning transparency

A sophisticated Self-RAG system that doesn't just answer questions - it thinks about its own thinking! Built with modular architecture, real-time reasoning traces, and beautiful Streamlit interface.

Features

Intelligent Self-Reflection

• Three Specialized Critics: Retrieval, Support, and Completeness critics evaluate every answer
• Reasoning Transparency: See the AI's step-by-step decision process
• Confidence Scoring: Understand how sure the system is about its answers

Dynamic Knowledge Management

• Live Web Scraping: Add documentation from any URL
• Smart Chunking: Intelligent text segmentation with semantic overlap
• Vector Database: ChromaDB for efficient semantic search

Beautiful Interface

• Streamlit Frontend: Clean, responsive UI with real-time updates
• Conversation History: Multiple Q&A sessions in one interface
• System Monitoring: Live statistics and health checks

Modular Architecture

• Pluggable Components: Easy to extend critics, models, and retrievers
• Domain Agnostic: Works with any documentation, not just Metta
• Production Ready: Robust error handling and caching

System Architecture

self_rag/
├── src/core/ # Brain of the system
│ ├── orchestrator.py # Main pipeline coordinator
│ └── knowledge_manager.py # URL → Vector DB pipeline
├── src/critics/ # Self-reflection modules
│ ├── retrieval_critic.py # "Should I retrieve docs?"
│ ├── support_critic.py # "Is this answer supported?"
│ └── completeness_critic.py # "Is this answer complete?"
├── src/retrieval/ # Search system
│ ├── vector_retriever.py # Semantic search
│ └── database_builder.py # Vector DB management
├── src/models/ # AI Models
│ ├── generator_model.py # Gemini 2.5 Flash (answers)
│ └── critic_model.py # Mistral-7B (critiques)
├── src/memory/ # Caching system
│ └── cache_manager.py # Performance optimization
├── app/ # Streamlit frontend
│ ├── main.py # Web interface
│ └── components/ # UI components
└── tests/ # Comprehensive test suite

Quick Start

git clone https://github.com/meron-23/self-rag
# Navigate to the project folder
cd self-rag

# Create virtual environment
python -m venv venv

# Activate it (Windows)
venv\Scripts\activate

# Activate it (Mac/Linux)
source venv/bin/activate

pip install -r requirements.txt

Environment Setup

Create a .env file:

GOOGLE_API_KEY=your_gemini_api_key_here
HF_TOKEN=your_huggingface_token_here

Launch the Application

streamlit run app/main.py

Launches a beautiful streamlit interface with real-time reasoning traces.

Usage

Adding Knowledge

1. Open the sidebar in the Streamlit app
2. Paste a documentation URL (e.g., https://metta-stdlib.readthedocs.io)
3. Click "Add URL" - system automatically scrapes, chunks, and indexes

Asking Questions

1. Type your question in the main input
2. View the answer with confidence scores
3. Inspect reasoning by expanding the reasoning trace
4. Monitor system health in the sidebar stats

Example Questions

• Factual: "What are metta expressions?"
• Technical: "How do I create lists in Metta?"
• Creative: "Write a poem about programming"
• Comparative: "Compare Metta with Python"

Testing

Run the comprehensive test suite:

# Test the orchestrator
python tests/test_orchestrator.py

# Test knowledge management  
python tests/test_knowledge_manager.py

# Test individual components
python tests/test_critics.py
python tests/test_retriever.py