...
Predictive VRAM Virtualization Engine for Large-Scale AI Inference
Break the VRAM wall. Run massive AI models on consumer hardware.
SynapSwap is a low-level memory virtualization engine designed to push beyond the physical limits of GPU VRAM. It enables the execution of massive AI models (LLMs, Vision, Diffusion) on consumer-grade hardware by transforming system RAM into an intelligent extension of VRAM, using predictive prefetching and fully asynchronous, non-blocking memory swapping.
Unlike traditional paging-based solutions, SynapSwap anticipates memory needs instead of reacting to memory pressure.
Today, the primary bottleneck in AI systems is no longer compute — it is video memory.
- Consumer GPUs typically ship with 8–24 GB of VRAM
- Modern AI models routinely exceed 40, 80, or even 100+ GB
- The result: OOM (Out Of Memory) errors, crashes, or the need for prohibitively expensive hardware
Existing mechanisms (Unified Memory, driver-level paging) are reactive, expensive, and unpredictable.
SynapSwap introduces proactive VRAM virtualization driven by awareness of the model execution graph.
VRAM becomes an intelligent cache, not a hard limit.
Hides up to 90% of PCIe latency by loading layer N+1 while layer N is executing.
Leverages execution graph dependencies to predict future memory requirements.
Dynamically tunes prefetch aggressiveness using Exponential Moving Averages.
Dedicated engine for fully non-blocking memory transfers (async memcpy).
Intelligent VRAM cleanup to prevent fragmentation and execution stalls.
Runs on Linux and Windows (MinGW).
SynapSwap is built around three core components:
Analyzes declared dependencies and decides what to load, when to load it, and why.
Dedicated thread responsible for asynchronous memory transfers without blocking the inference pipeline.
Advanced LRU-based algorithm that keeps VRAM clean, coherent, and performant.
- Compiler: GCC ≥ 4.8 or MinGW-w64
- OS: Linux / Windows
- Libraries: pthreads (included by default)
git clone https://github.com/your-username/synapswap.git
cd synapswap
make clean
make -j$(nproc)The API is designed to be hook-ready and easy to inject into existing inference engines.
#include "synapswap.h"
// 1. Initialization (physical VRAM limit: 2 GB)
synapswap_init(2048ULL * 1024 * 1024, true);
// 2. Allocate a virtualized memory block
void* layer_1 = synapswap_malloc(
512 * 1024 * 1024,
10,
SS_POLICY_AUTO,
"Transformer_Block_1"
);
// 3. Declare execution graph dependencies
synapswap_register_dependency(0, layer_1, 1);
// 4. Inference loop
synapswap_precompute_hint(0);
synapswap_wait_for_data(layer_1);
// GPU kernel invocation (CUDA / OpenCL / Vulkan)
synapswap_shutdown();SynapSwap includes a real-time ANSI dashboard:
[SynapSwap Dashboard]
├─ VRAM Usage : [|||||||||| ] 50.0% (1024 / 2048 MB)
├─ Hit Rate : 98.2% ← prediction accuracy
├─ Efficiency : OPTIMAL
└─ Stall Time : 1.25 ms
Planned features:
- Native CUDA backend (
cudaMemcpyAsync) - PyTorch wrapper (
ctypes/pybind11) - Multi-GPU support
- Weight compression in system RAM
- Vulkan / ROCm integration
Contributions are highly encouraged.
- Fork the project
- Create a feature branch:
git checkout -b feature/AmazingFeature- Commit your changes:
git commit -m "Add AmazingFeature"- Push and open a Pull Request
Distributed under the MIT License.
See the LICENSE file for more information.
Academic citation is welcome if used in research.
Developed with by DamienOS
Optimizing AI inference, one byte at a time.
⭐ If this project taught you something, consider leaving a star ⭐.
It helps SynapSwap reach more developers and researchers.