-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Context
@extrachill/chat is entirely text-based. No component in the package can send, receive, or render media (images, videos, files). Data Machine's backend can generate images and handle media via abilities and tools, but the chat UI has no way to display them.
Changes needed
1. Types — add Attachment interface and message fields
interface Attachment {
id: string;
type: 'image' | 'video' | 'file';
url: string;
thumbnailUrl?: string;
filename?: string;
mimeType?: string;
size?: number;
}
// Add to ChatMessage:
attachments?: Attachment[];
// Add to RawMessage:
attachments?: Array<{
id: string;
type: string;
url: string;
thumbnail_url?: string;
filename?: string;
mime_type?: string;
size?: number;
}>;
// Update SendRequest:
// Support FormData for file uploads alongside text2. ChatInput — file attachment support
Currently a textarea + send button. Needs:
- Attachment button (opens file picker)
- Drag-and-drop support on the input area
- Paste image from clipboard
- Attachment preview strip (thumbnails of pending files before send)
- Remove attachment before sending
- Update
onSendsignature:(content: string, files?: File[]) => void
3. ChatMessage — media rendering
Currently renders message.content as markdown/text only. Needs:
- Render
message.attachmentsas inline images (with lightbox/expand) - Render video attachments with native
<video>player - Render file attachments as download links with icon + filename + size
- Handle AI-generated images from tool responses (parse structured tool results)
4. ToolMessage — media-aware tool results
Currently dumps tool result as text/JSON. When a tool returns media:
- Detect media in tool results (look for
type: 'image'or URL patterns) - Render images inline instead of showing raw JSON
- Support a standard tool result media format:
{ type: 'image', url, alt }
5. API client — multipart upload support
Currently sends JSON only via FetchOptions.data.
- Add
FormDatasupport to the API client -
sendMessage()accepts optionalFile[]parameter - When files are present, send as
multipart/form-datainstead of JSON - Normalize attachment data in responses via the message normalizer
6. useChat hook — file handling
-
sendMessage(content: string, files?: File[])— pass files to API client - Track upload progress state
- Optimistic attachment preview (show thumbnails immediately, replace with server URLs on response)
7. CSS — media styles
- Image message styles (max-width, border-radius, click-to-expand)
- Video player styles
- File attachment card styles
- Attachment preview strip in input area
- Upload progress indicator
Design principles
- Attachments are part of the message, not separate entities
- Images render inline in the conversation flow
- File input should feel native (drag-drop, paste, button — all three)
- Backward compatible — messages without attachments render exactly as before
- The package stays provider-agnostic — it sends files to whatever backend the consumer configures
Related
- Extra-Chill/data-machine#922 — Backend multi-modal message support
- Extra-Chill/extrachill-chat#3 — WordPress chat plugin integration
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request