A modular chatbot powered by a local Large Language Model (LLM), served via Docker, and designed for privacy, portability, and edge deployment.
Ideal for Raspberry Pi or cloud hosting (e.g., AWS), this project runs a Mistral-based LLM backend with a lightweight HTML/JS frontend, all behind an Apache2 reverse proxy.
- 🧠 Local LLM Inference: Runs Mistral or similar models via Ollama, no external APIs or internet access required.
- 🐳 Dockerized Backend: Isolated Python-based LLM backend, easily portable across environments.
- 🌐 Simple Web UI: Minimalist HTML/CSS/JS interface for chatting in any browser.
- 🔄 Reverse Proxy Ready: Integrated with Apache2 for secure access via HTTPS.
- 🍓 Raspberry Pi Optimized: Designed for ARM devices with low memory footprint.
- ☁️ Cloud Deployable: Tested on AWS EC2 and scalable to VPS or local networks.
- 🔐 Privacy First: All processing is done locally—your data never leaves your machine.
- Python 3 + FastAPI
- Ollama for local LLMs (e.g.,
mistral,llama2) - Docker & Docker Compose
- Apache2 reverse proxy with SSL
- HTML/CSS/JS frontend
chatBot/
├── backend/ # Python API with LLM interaction
│ ├── app.py
│ └── Dockerfile
├── frontend/ # HTML/JS chat interface
│ └── index.html
├── notebooks/ # Documentation
├── docker-compose.yml # Service orchestration
└── README.md| Section | Link |
|---|---|
| 🦙 Useful ollama commands | ollama_commands_cheat_sheet.md |
| ⚙️ Deployment Guide on MacBook | ollama_setup_on_MacBook.md |
| ⚙️ Deployment Guide on Raspberry Pi | tinyllama_setup_on_raspberry_pi.md |
| 🧪 Testing & Benchmarks | (coming soon) |
| 🗂️ Useful git commands | git_cheat_sheet.md |
git clone https://github.com/Janos11/Local-LLM-Backend-Container.git
cd chatBot
docker compose up- Personal assistant without giving away your data
- Offline chatbot for travel, remote sites, or IoT
- Embedded interface for smart devices or terminals
- Private family or team chat interface
Automated Apache IP Update — dynamic IP reverse proxy update script Add more links here as your project grows
Modern AI projects often depend on external APIs, raising privacy, latency, and cost concerns. This project is built for local-first, edge-compatible deployment—a skillset highly relevant in DevOps, MLOps, and systems engineering roles, including quant firms, infrastructure teams, or R&D environments.
| János Rostás |
👨💻 Electronic & Computer Engineer (Final Year Student) 🧠 Passionate about AI, LLMs, and RAG systems 🐳 Docker & Linux Power User 🔧 Raspberry Pi Builder | Automation Fanatic 💻 Git & GitHub DevOps Explorer 📦 Loves tinkering with Ollama, containerized models, and APIs 🌐 janosrostas.co.uk 🐙 GitHub | 🐋 Docker Hub |
| ChatGPT |
🤖 AI Pair Programmer by OpenAI 💡 Collaborates on brainstorming, prototyping, and debugging 📚 Built on a foundation of global programming knowledge 🔍 Assists with everything from low-level scripting to high-level LLM orchestration |