bug: Model gemma-3n-E2B lead to Conversation roles must alternate user/assistant/user/assistant #464
Description
Describe the Bug
When using gemma-3n-E2B the error "Conversation roles must alternate user/assistant/user/assistant/..." occurs because Jan's backend (llm-api) automatically injects a system message at the very beginning of every chat (providing details like the current date and "You are Jan...").
The Gemma-3 chat template in llama.cpp is extremely strict and expects the conversation to start with a user role, crashing when it encounters Jan's injected system message.
got exception: {"code":500,"message":"Conversation roles must alternate user/assistant/user/assistant/... at row 19, column 27:\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}\n {{ raise_exception(\"Conversation roles must alternate user/assistant/user/assistant/...\") }}\n ^\n {%- endif -%}\n at row 19, column 9:\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}\n {{ raise_exception(\"Conversation roles must alternate user/assistant/user/assistant/...\") }}\n ^\n {%- endif -%}\n at row 18, column 69:\n{%- for message in loop_messages -%}\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}\n ^\n {{ raise_exception(\"Conversation roles must alternate user/assistant/user/assistant/...\") }}\n at row 18, column 5:\n{%- for message in loop_messages -%}\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}\n ^\n {{ raise_exception(\"Conversation roles must alternate user/assistant/user/assistant/...\") }}\n at row 17, column 37:\n{%- endif -%}\n{%- for message in loop_messages -%}\n ^\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}\n at row 17, column 1:\n{%- endif -%}\n{%- for message in loop_messages -%}\n^\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}\n at row 1, column 1:\n{{ bos_token }}\n^\n{%- if messages[0]['role'] == 'system' -%}\n","type":"server_error"}
srv log_server_r: request: POST /v1/chat/completions 192.168.1.47 500
Steps to Reproduce
- Use e.g. llama.cpp server and load it with the default jinja template https://huggingface.co/unsloth/gemma-3n-E2B-it-GGUF
- Configure a provider
- Chat with the model
Screenshots / Logs
No response
Operating System
all
Version
latest
Additional Context
No response