feat: add image upload search via local Ollama vision

[tsgdgszjj-sketch]

Summary

• add image upload interception in the attachment flow
• add route to turn an uploaded image into a search query
• document how to wire the feature to a local Ollama vision model such as

Notes

• this keeps Vane's existing text search pipeline intact
• recommended local setup uses an OpenAI-compatible Ollama endpoint
• tested with local Ollama vision flow before deployment

---

Summary by cubic

Adds image upload search powered by a local Ollama vision model via the new /api/vision route. When users attach an image, we generate a concise search query and run it through the existing text search flow.

• New Features

  • Intercepts image uploads in Attach.tsx, calls /api/vision, then sends a search message with the returned query.
  • Adds POST /api/vision that reads an image and uses openai with an Ollama-compatible baseURL to produce a query.
  • File picker now accepts image/*.
  • README updated with a Docker example and setup notes.

• Migration

  • Set OLLAMA_BASE_URL to your OpenAI-compatible Ollama endpoint.
  • Set OLLAMA_VISION_MODEL to a vision model (e.g., qwen3-vl:latest).
  • Defaults are provided for local Docker; adjust host/IP as needed.

^{Written for commit d31ba3d. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/components/MessageInputActions/Attach.tsx">

<violation number="1" location="src/components/MessageInputActions/Attach.tsx:42">
P2: Early return on any selected image prevents uploading/attaching other selected files, causing mixed selections to be dropped.</violation>
</file>

<file name="src/app/api/vision/route.ts">

<violation number="1" location="src/app/api/vision/route.ts:13">
P2: Uploaded image is read into memory and base64-encoded without any size/type validation, allowing oversized or invalid uploads to consume memory and be forwarded to the model.</violation>
</file>

---

Since this is your first cubic review, here's how it works:

• cubic automatically reviews your code and comments on bugs and improvements
• Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
• Add one-off context when rerunning by tagging @cubic-dev-ai with guidance or docs links (including llms.txt)
• Ask questions if you need clarification on any suggestion

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Early return on any selected image prevents uploading/attaching other selected files, causing mixed selections to be dropped.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/components/MessageInputActions/Attach.tsx, line 42:

<comment>Early return on any selected image prevents uploading/attaching other selected files, causing mixed selections to be dropped.</comment>

<file context>
@@ -29,10 +29,30 @@ const Attach = () => {
       data.append('files', e.target.files![i]);
     }
 
+    if (hasImage && imageFile) {
+      const visionData = new FormData();
+      visionData.append('image', imageFile);
</file context>

Suggested change

	if (hasImage && imageFile) {
	if (hasImage && imageFile && e.target.files!.length === 1) {

[cubic-dev-ai]

P2: Uploaded image is read into memory and base64-encoded without any size/type validation, allowing oversized or invalid uploads to consume memory and be forwarded to the model.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/vision/route.ts, line 13:

<comment>Uploaded image is read into memory and base64-encoded without any size/type validation, allowing oversized or invalid uploads to consume memory and be forwarded to the model.</comment>

<file context>
@@ -0,0 +1,50 @@
+      return NextResponse.json({ message: 'Missing image' }, { status: 400 });
+    }
+
+    const buffer = Buffer.from(await imageFile.arrayBuffer());
+    const base64Image = buffer.toString('base64');
+    const mimeType = imageFile.type;
</file context>