Image tokens not always included when running Qwen3-VL

[vincentamato]

I was interacting with a user in Discord who was trying to run Qwen3-VL and they kept getting this error:

Error: Error in iterating prediction stream: ValueError: Image features and image tokens do not match: tokens: 0, features 240

I'm pretty sure this means that their prompt was not including the image tokens for some reason. I confirmed that they had the latest runtime (v0.30.0) and their application was up to date. Interestingly, the user had no issues with Qwen2.5-VL. After some debugging, I asked the user to move their ~/.cache/lm-studio folder out of their home directory and then reinstall the LM Studio application. This solved the problem and they were able chat with Qwen3-VL.

I'm not sure what was happening or why regenerating the cache folder seemed to work, so I wanted to put this on your radar. Our messages are in the model-discussion channel in the Discord server if you want to see more context about what we tried while debugging. Throughout our conversation I was unable to reproduce the error on my end.

[vincentamato]

Update: I was able to reproduce this error by trying to run the thinking version in the LM Studio application (regenerating the cache folder did not help). However, running it with python demo.py --model mlx-community/Qwen3-VL-30B-A3B-Thinking-4bit --prompt "What do you see?" --images demo-data/toucan.jpeg works fine.

The error in question:

Error in iterating prediction stream: ValueError: Image features and image tokens do not match: tokens: 0, features 132

[egomarker]

Yeah, thinking qwen3 VL 30B gives this error 100% of the time, while -Instruct works fine.

[egomarker]

Paste prompt from -Insturct into -Thinking and it will work too. Idk if it's a 100% correct fix, but it is what it is for now.

{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {%- if messages[0].content is string %}
            {{- messages[0].content }}
        {%- else %}
            {%- for content in messages[0].content %}
                {%- if 'text' in content %}
                    {{- content.text }}
                {%- endif %}
            {%- endfor %}
        {%- endif %}
        {{- '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' }}
        {%- if messages[0].content is string %}
            {{- messages[0].content }}
        {%- else %}
            {%- for content in messages[0].content %}
                {%- if 'text' in content %}
                    {{- content.text }}
                {%- endif %}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- set image_count = namespace(value=0) %}
{%- set video_count = namespace(value=0) %}
{%- for message in messages %}
    {%- if message.role == "user" %}
        {{- '<|im_start|>' + message.role + '\n' }}
        {%- if message.content is string %}
            {{- message.content }}
        {%- else %}
            {%- for content in message.content %}
                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
                    {%- set image_count.value = image_count.value + 1 %}
                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
                    <|vision_start|><|image_pad|><|vision_end|>
                {%- elif content.type == 'video' or 'video' in content %}
                    {%- set video_count.value = video_count.value + 1 %}
                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
                    <|vision_start|><|video_pad|><|vision_end|>
                {%- elif 'text' in content %}
                    {{- content.text }}
                {%- endif %}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "assistant" %}
        {{- '<|im_start|>' + message.role + '\n' }}
        {%- if message.content is string %}
            {{- message.content }}
        {%- else %}
            {%- for content_item in message.content %}
                {%- if 'text' in content_item %}
                    {{- content_item.text }}
                {%- endif %}
            {%- endfor %}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {%- if (loop.first and message.content) or (not loop.first) %}
                    {{- '\n' }}
                {%- endif %}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' }}
                {{- tool_call.name }}
                {{- '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {%- if message.content is string %}
            {{- message.content }}
        {%- else %}
            {%- for content in message.content %}
                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
                    {%- set image_count.value = image_count.value + 1 %}
                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
                    <|vision_start|><|image_pad|><|vision_end|>
                {%- elif content.type == 'video' or 'video' in content %}
                    {%- set video_count.value = video_count.value + 1 %}
                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
                    <|vision_start|><|video_pad|><|vision_end|>
                {%- elif 'text' in content %}
                    {{- content.text }}
                {%- endif %}
            {%- endfor %}
        {%- endif %}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
{%- endif %}

[vincentamato]

@egomarker Hmm that's interesting

[neilmehta24]

Thanks for the report. In the app, we had some heuristic fail for the thinking prompt template. This issue will be fixed in the next app release

[chrisqianz]

8B also same issue.

[kochjo-dev]

Hi Vincent, it seems like others have been faster. But per your request from the lmstudio board:
30B indeed has the same issue: "Error in iterating prediction stream: ValueError: Image features and image tokens do not match: tokens: 0, features 1xx", for all pics and all quants I tried.

[vincentamato]

@kochjo-dev This wasn't the thinking variant right?

[kochjo-dev]

@kochjo-dev This wasn't the thinking variant right?

It says it's 'instruct'. It was this one: https://huggingface.co/lmstudio-community/Qwen3-VL-30B-A3B-Instruct-MLX-8bit