Skip to content

README.md

Latest commit

 

History

History
357 lines (311 loc) · 30.2 KB

README.md

File metadata and controls

357 lines (311 loc) · 30.2 KB

Small Models

Comparison of small open source LLMs (8b parameters or less)

General | Tool Usage | Chat Room | Logic | Vision | Vision OCR | Code Generation | More

All testing done with Ollama

Key
✅ Pass
☑️ Partial Pass
❌ Fail
🕒 Timeout
😵‍💫 Hallucinations
🚫 Rejected prompt / unsafe content
💔 Malformed Response / Tool call

General

Tested with attogram/ollama-multirun

Small Model Hovercraft Pirate Who
bakllava:7b ❌ ✅
codellama:7b ✅ ✅ ✅
cogito:3b ✅ ✅
cogito:bb ✅ ✅
deepcoder:1.5b ✅ ☑️ ✅
deepseek-r1:1.5b ✅ ☑️ ✅
deepseek-r1:8b ☑️ ✅ ✅
dolphin-mistral:7b ☑️ ✅ ✅
dolphin3:8b ✅ ✅ ✅
gemma3:1b ☑️ ✅ ✅
gemma3:4b ☑️ ✅
gemma:2b ☑️ ✅
granite3.2-vision:2b ✅ ✅
granite3.3:2b ☑️ ✅
huihui_ai/baronllm-abliterated:8b ☑️ ✅
llama3-groq-tool-use:8b ☑️ ✅
llama3.2:1b ✅ ☑️
llava-llama3:8b ☑️ ✅
llava-phi3:3.8b ✅ ✅
llava:7b ✅ ✅
minicpm-v:8b ❌ ✅
mistral:7b ☑️ ✅
qwen2.5-coder:7b ✅ ✅
qwen2.5vl:3b ✅ ✅
qwen2.5vl:7b ☑️ ✅
qwen3:1.7b ✅ 🕒 ✅
qwen3:4b ✅ 🕒 ✅
qwen3:8b ☑️ ✅ ✅
smollm2:135m ✅ ✅
smollm2:360m ☑️ ✅
smollm2:1.7b ✅ ✅
stable-code:3b ☑️ ☑️ ✅
starcoder:7b ❌ ❌ ❌
Small Model Hovercraft Pirate Who 
deepseek-r1:7b
llama3.1:8b
llama3.3:3b
llama3:8b
phi:2.7b
phi3:3.8b
tinyllama:1.1b
qwen3:0.6b
qwen3:4b

^top

Tool Usage

Tested with attogram/ollama-bash-toolshed

Passing Math Ollama Time Web
cogito:3b ✅ ✅ ✅ ✅
cogito:8b ✅ ✅ ✅ ✅
granite3.3:2b ✅ ✅ ✅ ✅
hermes3:8b ✅ ✅ ✅ ✅
llama3.2:1b ✅ ✅ ✅ ✅
llama3.2:3b ✅ ✅ ✅ ✅
qwen3:1.7b ✅ ✅ ✅ ✅
qwen3:4b ✅ ✅ ✅ ✅
qwen3:8b ✅ ✅ ✅ ✅
Failing Math Ollama Time Web
command-r7b:7b ❌ ❌ ❌ ❌
granite3.2-vision:2b ❌ ❌ ❌ ❌
granite3.3:8b 💔 💔 💔 💔
llama3-groq-tool-use:8b ✅ ✅ ❌ ✅
mistral:7b 💔 💔 💔 💔
nemotron-mini:4b ✅ ❌ ✅ ☑️
qwen2.5-coder:7b 💔 💔 💔 💔
qwen3:0.6b ✅ ☑️ ❌ ✅
smollm2:1.7b 💔 💔 ✅ 😵‍💫 
aya-expanse:8b
command-r7b-arabic:7b
granite3-dense:2b
granite3-dense:8b
granite3-moe:1b
granite3-moe:3b
granite3.1-dense:2b
granite3.1-dense:8b
granite3.1-moe:1b
granite3.1-moe:3b
granite3.2:2b
granite3.2:8b
hermes3:3b
llama3.1:8b
mixtral:8x7b
phi4-mini:3.8b
qwen2.5-coder:0.5b
qwen2.5-coder:1.5b
qwen2.5-coder:3b
qwen2.5:0.5b
qwen2.5:1.5b
qwen2.5:3b
qwen2.5:7b
qwen2:0.5b
qwen2:1.5b
qwen2:7b

Hituzip/gemma3-tools:4b
PetrosStav/gemma3-tools:4b
hhao/qwen2.5-coder-tools:0.5b
hhao/qwen2.5-coder-tools:1.5b
hhao/qwen2.5-coder-tools:3b
hhao/qwen2.5-coder-tools:7b
ishumilin/deepseek-r1-coder-tools-tuned:1.5b
ishumilin/deepseek-r1-coder-tools-tuned:7b
ishumilin/deepseek-r1-coder-tools-tuned:8b
ishumilin/deepseek-r1-coder-tools:1.5b
ishumilin/deepseek-r1-coder-tools:7b
ishumilin/deepseek-r1-coder-tools:8b
maryasov/qwen2.5-coder-cline:7b
orieg/gemma3-tools:1b
orieg/gemma3-tools:4b
tom_himanen/deepseek-r1-roo-cline-tools:1.5b
tom_himanen/deepseek-r1-roo-cline-tools:7b
tom_himanen/deepseek-r1-roo-cline-tools:8b

^top

Chat Room

Tested with attogram/llm-council

Small Model Peace Quit Topic
bakllava:7b ❌
codellama:7b ✅
deepcoder:1.5b ❌
deepseek-r1:1.5b ✅ ❌
deepseek-r1:8b ✅
dolphin-mistral:7b ✅
dolphin3:8b ✅ ✅
gemma3:1b ✅
gemma3:4b ✅ ✅ ✅
gemma:2b ✅
granite3.2-vision:2b ❌
granite3.3:2b ✅ ✅ ❌
huihui_ai/baronllm-abliterated:8b ✅
llama3-groq-tool-use:8b ✅
llama3.2:1b ❌
llava-llama3:8b ✅
llava-phi3:3.8b ✅
llava:7b ✅
minicpm-v:8b ✅ ✅
mistral:7b ✅ ✅
qwen2.5-coder:7b ✅
qwen2.5vl:3b ✅
qwen2.5vl:7b ✅
qwen3:1.7b ✅
qwen3:8b ✅ ✅
smollm2:135m ❌
smollm2:360m ❌
smollm2:1.7b ❌
stable-code:3b ❌
starcoder:7b ❌
Small Model Peace Quit Topic

^top

Logic

Tested with attogram/ollama-multirun

Small Model Socrates Strawberry
bakllava:7b ✅ ❌
codellama:7b ❌ ❌
deepcoder:1.5b ✅ ✅
deepseek-r1:1.5b ✅ ✅
deepseek-r1:8b ✅ 🕒
dolphin-mistral:7b ✅ ❌
dolphin3:8b ✅ ❌
gemma3:1b ✅ ✅
gemma3:4b ✅ ✅
gemma:2b ✅ ✅
granite3.2-vision:2b ✅ ✅
granite3.3:2b ✅ ✅
huihui_ai/baronllm-abliterated:8b ✅ ❌
llama3-groq-tool-use:8b ✅ ✅
llama3.2:1b ❌ ❌
llava-llama3:8b ✅ ❌
llava-phi3:3.8b ✅ ❌
llava:7b ✅ ❌
minicpm-v:8b ✅ ❌
mistral:7b ✅ ❌
qwen2.5-coder:7b ✅ ❌
qwen2.5vl:3b ✅ ✅
qwen2.5vl:7b ✅ ❌
qwen3:1.7b ✅ ✅
qwen3:8b ✅ ✅
smollm2:135m ☑️ ❌
smollm2:360m ✅ ❌
smollm2:1.7b ✅ ❌
stable-code:3b ❌ ❌
starcoder:7b ❌ ❌
Small Model Socrates Strawberry

^top

Vision

Tested with attogram/ollama-multirun

Passing Describe Keywords Faces
bakllava:7b ✅ ❌ ❌
granite3.2-vision:2b ✅ ✅ ❌
minicpm-v:8b ✅ ✅ ☑️
qwen2.5vl:3b ✅ ✅ ❌
qwen2.5vl:7b ✅ ✅ ✅
gemma3:4b ✅ ✅ ✅
llava:7b ✅ ✅ ✅
llava-llama3:8b ✅ ☑️ ☑️
llava-phi3:3.8b ✅ ☑️ ☑️
moondream:1.8b ✅ ❌ ❌

^top

Vision OCR

Tested with attogram/ollama-multirun

Passing OCR
granite3.2-vision:2b ☑️
minicpm-v:8b ☑️
qwen2.5vl:3b ✅
qwen2.5vl:7b ✅
Failing OCR
bakllava:7b ❌
gemma3:4b 🕒
llava:7b ❌
llava-llama3:8b ❌
llava-phi3:3.8b ❌
moondream:1.8b ❌

^top

Code Generation

Small Model bash-1 js-1 php-1 python-1
bakllava:7b ❌ ☑️
codellama:7b ✅ ️ ✅
deepcoder:1.5b ❌ ✅
deepseek-r1:1.5b ❌ ✅
deepseek-r1:8b 🕒 🕒
dolphin-mistral:7b ✅ ✅
dolphin3:8b ✅ ✅
gemma3:1b ☑️ ☑️
gemma3:4b ☑️ ✅
gemma:2b ✅ ❌
granite3.2-vision:2b ✅ ✅
granite3.3:2b ☑️ ✅
huihui_ai/baronllm-abliterated:8b ✅ ✅
llama3-groq-tool-use:8b ✅ ✅
llama3.2:1b ☑️ ☑️
llava-llama3:8b ☑️ ✅
llava-phi3:3.8b ✅ ✅
llava:7b ✅ ✅
minicpm-v:8b ❌ ✅
mistral:7b ☑️ ✅
qwen2.5-coder:7b ✅ ✅
qwen2.5vl:3b ☑️ ✅
qwen2.5vl:7b ✅ ✅
qwen3:1.7b ✅ ✅
qwen3:8b ✅ ✅
smollm2:135m ❌ ❌
smollm2:360m ☑️ ☑️
smollm2:1.7b ✅ ☑️
stable-code:3b ☑️ ✅
starcoder:7b ☑️ ❌
Small Model bash-1 js-1 php-1 python-1

^top

More from the Attogram Project

Project About
Attogram Project
Discord Channel		 Join the Attogram Project Discord Channel for:
- Announcements
- Technical Support
- General Chat about Attogram Projects
Ollama Multirun Run a prompt against all, or some, of your models running on Ollama.
- Creates web pages with the output, performance statistics and model info.
- All in a single Bash shell script.
Ollama Bash Lib A Bash Library to interact with Ollama
Ollama Bash Toolshed Chat with tool calling models.
- Sample tools included.
- Add new tools to your shed with ease.
- Runs on Ollama.
- All via Bash shell scripts.
LLM Council Start a chat room between all, or some, of your models running on Ollama.
- All in a single Bash shell script.
Small Models Comparison of small open source LLMs
- 8b parameters or less
AI Test Zone AI Testing reports
- hosted on https://attogram.github.io/ai_test_zone/

^top