Skip to content

feat: Media support — attachments in messages, image/video rendering, file input #6

@chubes4

Description

@chubes4

Context

@extrachill/chat is entirely text-based. No component in the package can send, receive, or render media (images, videos, files). Data Machine's backend can generate images and handle media via abilities and tools, but the chat UI has no way to display them.

Changes needed

1. Types — add Attachment interface and message fields

interface Attachment {
    id: string;
    type: 'image' | 'video' | 'file';
    url: string;
    thumbnailUrl?: string;
    filename?: string;
    mimeType?: string;
    size?: number;
}

// Add to ChatMessage:
attachments?: Attachment[];

// Add to RawMessage:
attachments?: Array<{
    id: string;
    type: string;
    url: string;
    thumbnail_url?: string;
    filename?: string;
    mime_type?: string;
    size?: number;
}>;

// Update SendRequest:
// Support FormData for file uploads alongside text

2. ChatInput — file attachment support

Currently a textarea + send button. Needs:

  • Attachment button (opens file picker)
  • Drag-and-drop support on the input area
  • Paste image from clipboard
  • Attachment preview strip (thumbnails of pending files before send)
  • Remove attachment before sending
  • Update onSend signature: (content: string, files?: File[]) => void

3. ChatMessage — media rendering

Currently renders message.content as markdown/text only. Needs:

  • Render message.attachments as inline images (with lightbox/expand)
  • Render video attachments with native <video> player
  • Render file attachments as download links with icon + filename + size
  • Handle AI-generated images from tool responses (parse structured tool results)

4. ToolMessage — media-aware tool results

Currently dumps tool result as text/JSON. When a tool returns media:

  • Detect media in tool results (look for type: 'image' or URL patterns)
  • Render images inline instead of showing raw JSON
  • Support a standard tool result media format: { type: 'image', url, alt }

5. API client — multipart upload support

Currently sends JSON only via FetchOptions.data.

  • Add FormData support to the API client
  • sendMessage() accepts optional File[] parameter
  • When files are present, send as multipart/form-data instead of JSON
  • Normalize attachment data in responses via the message normalizer

6. useChat hook — file handling

  • sendMessage(content: string, files?: File[]) — pass files to API client
  • Track upload progress state
  • Optimistic attachment preview (show thumbnails immediately, replace with server URLs on response)

7. CSS — media styles

  • Image message styles (max-width, border-radius, click-to-expand)
  • Video player styles
  • File attachment card styles
  • Attachment preview strip in input area
  • Upload progress indicator

Design principles

  • Attachments are part of the message, not separate entities
  • Images render inline in the conversation flow
  • File input should feel native (drag-drop, paste, button — all three)
  • Backward compatible — messages without attachments render exactly as before
  • The package stays provider-agnostic — it sends files to whatever backend the consumer configures

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions