Aegis

A modern cross platform GUI for Garak, NVIDIA's LLM vulnerability scanner. Aegis provides an intuitive interface to scan language models for security vulnerabilities including jailbreaks, prompt injection, toxicity, and more.

Project Structure

aegis/
├── backend/          # FastAPI backend service
│   ├── api/          # API routes
│   ├── models/       # Data models
│   ├── services/     # Business logic
│   ├── config/       # Configuration
│   └── main.py       # Entry point
├── frontend/         # Flutter desktop application
│   ├── lib/          # Dart source code
│   ├── assets/       # Images, fonts, etc.
│   └── pubspec.yaml  # Flutter dependencies
└── README.md         # This file

Features

• Quick Scan: Fast vulnerability scanning with preset configurations
• Full Scan: Comprehensive testing with all available probes
• Browse Probes: Explore all available vulnerability probes
• Scan History: View and analyze past scan results
• Real-time Progress: WebSocket-based live scan monitoring
• Rich Results: Interactive HTML reports with detailed breakdowns

Prerequisites

• Python 3.8+ (for backend)
• Flutter 3.0+ (for frontend)
• Garak (LLM vulnerability scanner)
• Ollama/OpenAI/Anthropic (or other supported LLM providers)

Installation

1. Install Garak

pip install garak

2. Backend Setup

cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

3. Frontend Setup

cd frontend
flutter pub get

Running the Application

Start the Backend

cd backend
python main.py

The backend will start on http://localhost:8888

Start the Frontend

cd frontend
flutter run -d macos  # Or: -d windows, -d linux

Configuration

Backend Configuration

Create a .env file in the backend directory:

# Server settings
HOST=0.0.0.0
PORT=8888
LOG_LEVEL=INFO

# Optional: Custom garak path
GARAK_PATH=/path/to/garak

# Optional: API keys for LLM providers
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

Frontend Configuration

The frontend automatically connects to http://localhost:8888. To change this, edit:

// frontend/lib/config/constants.dart
class ApiConstants {
  static const String baseUrl = 'http://localhost:8888';
}

Development

Backend Development

cd backend
# Run with auto-reload
python main.py

# Run tests
pytest tests/

# Format code
black .

Frontend Development

cd frontend
# Hot reload is enabled by default
flutter run -d macos

# Generate code (if using code generation)
flutter pub run build_runner build --delete-conflicting-outputs

# Run tests
flutter test

API Documentation

Once the backend is running, visit:

• Swagger UI: http://localhost:8888/docs
• ReDoc: http://localhost:8888/redoc

Scan Workflow

1. Select Model: Choose your LLM provider and model
2. Configure Probes: Select vulnerability tests to run
3. Start Scan: Initiate the scan with real-time progress
4. View Results: Analyze detailed HTML reports
5. History: Access past scans anytime

Supported LLM Providers

• OpenAI (GPT-3.5, GPT-4, etc.)
• Anthropic (Claude)
• Ollama (Local models)
• HuggingFace
• Cohere
• Replicate
• LiteLLM
• NVIDIA NIM

Architecture

Backend (FastAPI)

• RESTful API: Standard HTTP endpoints for all operations
• WebSocket: Real-time scan progress updates
• Async Processing: Non-blocking scan execution
• File System: Reads historical scans from garak's output directory

Frontend (Flutter)

• Riverpod: State management
• Dio: HTTP client with interceptors
• WebSocket: Real-time progress monitoring
• Material 3: Modern dark theme UI

Troubleshooting

Backend Issues

Port already in use:

lsof -ti:8888 | xargs kill -9

Garak not found:

which garak
# Or set GARAK_PATH in .env

Frontend Issues

Build errors:

flutter clean
flutter pub get
flutter pub run build_runner build --delete-conflicting-outputs

Screenshots

Home Screen

The main dashboard provides quick access to all key features:

Model Configuration

Select your target model from multiple LLM providers with built-in presets:

Probe Selection

Browse and select from hundreds of vulnerability probes organized by category:

Scan Execution

Real-time progress tracking with detailed status information:

Scan History

View all past scans with pass/fail metrics at a glance:

Detailed Results

Comprehensive results with visualizations and metrics:

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

License

This project is a GUI wrapper for Garak. Please refer to Garak's license for the underlying scanner.

Acknowledgments

• Garak by NVIDIA for the core vulnerability scanner
• Flutter and FastAPI communities for excellent frameworks