Poor Summarization When Using Thinking Models With Ollama

[JoshuaKimsey]

Variant

Desktop app (Linux)

Affected area

AI Insights / World Brief

Bug description

It appears as though the "Summarize This Panel," when used with a local back-end like Ollama, does a very poor job of summarizing what is being shown in the panel, generating wildly wrong summaries of what is going on, or outputting nonsensical jibberish. I am using a competent model (Qwen3-vl:8b, also tried with the new Qwen3.5:35b-A3B. Both are Thinking Models.), so I don't believe this to be a model issue. I wasn't sure it was even using Ollama until I checked in my terminal for the Ollama instance, which does show it's loading the model when called.

Steps to reproduce

Be using Ollama as a back-end for the LLMs
Summarize some of the new panels
Observe the ones that were poorly summarized or which outputted jibberish.

Expected behavior

Summarization should be more accurate in what it says when scanning over the news panels. It should also not output jibberish.

Screenshots / Console errors

Very poorly summarized:

Jibberish:

Browser & OS

Bazzite 43 (Fedora-based Linux) with Ollama 0.17.2 via Docker

[JoshuaKimsey]

One thought is perhaps both Qwen models being <Thinking></Thinking> models messed with this? But I feel like if that was the case all panels and everything using Ollama with World Monitor would be affected, and it isn't. Some panels do come out half-alright. It's just not a frequent occurrence it seems?

[JoshuaKimsey]

Ok, I switched to Gemma3:12b, which is a non-thinking model, and it appears to have solved the issue of the Summarization being extremely poor or jibberish (so you'll need to account for handling Thinking models with Ollama, which should be fairly easy I think?).

However, with this I discovered what may be a new bug possibly related to this? Clicking on the Summarize buttons seems to give summary info for past panels that were summarized? It's a bit hard to explain, so here's a screenshot:

Basically, for each panels I asked to summarize, it seemed to show the summary info from where I'd asked for the previous panel. Apologies if this isn't necessarily related to the bug I opened this issue for, I can make a new report if you want/need me to.

[FayezBast]

I did some additional testing, including running qwen3:1.7b to rule out context window size as a factor. It exhibited the same behavior, but I'm still not fully certain whether this is a parsing issue with thinking model output or a model capability limitation — I need to test further before drawing a conclusion.

That said, I've made some defensive changes on the parsing side — extended the response cleanup to strip <thinking> blocks including unterminated ones, which should help regardless.

I also took a look at what I believe is your second bug and added an explicit state reset before each new summary request, along with a refresh gate to prevent stale results bleeding into the next panel.

Still waiting on a better model to finish downloading(bad wifi) to do proper validation, and make changes

[koala73]

Both issues fixed, will deploy later in the day