vLLM Recipes

This repo intends to host community maintained common recipes to run vLLM answering the question: How do I run model X on hardware Y for task Z?

Guides

DeepSeek

• DeepSeek-OCR
• DeepSeek-V3.1
• DeepSeek-V3, DeepSeek-R1
• DeepSeek-V3.2-Exp
• DeepSeek-V3.2

Ernie

• Ernie4.5
• Ernie4.5-VL

GLM

• Glyph
• GLM-5
• GLM-4.7-Flash, GLM-4.7, GLM-4.6, GLM-4.5, GLM-4.5-Air
• GLM-4.5V, GLM-4.6V

Google

• TranslateGemma

inclusionAI

• Ring-1T-FP8

InternVL

• InternVL3.5

InternLM

• Intern-S1

Jina AI

• Jina-reranker-m0

Llama

• Llama4-Scout
• Llama3.3-70B
• Llama3.1

MiniMax

• MiniMax-M2.5, MiniMax-M2.1, MiniMax-M2

Xiaomi MiMo

• MiMo-V2-Flash

Mistral AI

• Ministral-3-Instruct
• Ministral-3-Reasoning
• Mistral-Large-3-Instruct

Moonshotai

• Kimi-K2.5
• Kimi-K2
• Kimi-K2-Think
• Kimi-Linear

NVIDIA

• Nemotron-3-Nano-30B-A3B
• Nemotron-Nano-12B-v2-VL

OpenAI

• gpt-oss

PaddlePaddle

• PaddleOCR-VL

Qwen

• Qwen3
• Qwen3-ASR
• Qwen3-VL
• Qwen3-Next
• Qwen3.5
• Qwen3-Coder-480B-A35B
• Qwen2.5-VL
• Qwen3Guard-Gen

Seed

• Seed-OSS-36B

StepFun

• Step-3.5-Flash

Tencent-Hunyuan

• HunyuanOCR

Contributing

Please feel free to contribute by adding a new recipe or improving an existing one, just send us a PR!

While the repo is designed to be directly viewable in GitHub (Markdown files as first citizen), you can build the docs as web pages locally.

uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
uv run mkdocs serve

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.