Agent Utils

A self-hosted receipt management system with an MCP (Model Context Protocol) server for AI agents. Built on Cloudflare Workers with R2 storage.

Features

• Multiple Intake Methods: Accept receipts via SMS (Twilio), Email (SendGrid), or direct API upload
• MCP Server: Expose receipts to AI agents (like Claude Desktop) via Model Context Protocol
• Web UI: Simple password-protected interface to view and manage receipts
• AI Summaries: Optional OpenRouter integration to auto-generate receipt summaries
• Status Management: Organize receipts as "not processed" or "processed"
• Automatic Confirmations: Send SMS/email replies when receipts are received

Architecture

• Runtime: Cloudflare Workers (serverless, globally distributed)
• Storage: Cloudflare R2 (S3-compatible object storage)
• Framework: Hono (lightweight web framework)
• AI Integration: OpenRouter API (optional, for vision and text analysis)

Quick Start

Prerequisites

• Node.js 18+ and yarn
• A Cloudflare account with Workers and R2 enabled
• (Optional) Twilio account for SMS intake
• (Optional) SendGrid account for email intake
• (Optional) OpenRouter API key for AI summaries

Installation

1. Clone the repository:

git clone https://github.com/yourusername/agent-utils.git
cd agent-utils
yarn install

2. Create local environment file:

cp .env.example .dev.vars

Edit .dev.vars with your configuration:

TWILIO_AUTH_TOKEN=your_token
MCP_BEARER_TOKEN=your_secure_token
ALLOWED_PHONE_NUMBERS=+15551234567
ALLOWED_EMAILS=user@example.com
UPLOAD_API_KEY=your_key
UI_PASSWORD=your_password
# OPENROUTER_API_KEY=sk-or-v1-... (optional)
# SENDGRID_API_KEY=SG... (optional)

3. Create R2 buckets:

npx wrangler r2 bucket create receipts
npx wrangler r2 bucket create receipts-preview  # for local dev

4. Configure Wrangler: Copy the example configuration and add your Cloudflare account details:

cp wrangler.toml.example wrangler.toml

Edit wrangler.toml and set your Cloudflare account ID:

account_id = "your-cloudflare-account-id"

# Optional: Configure a custom domain or route
# routes = [
#   { pattern = "your-domain.com/*", custom_domain = true }
# ]

Note: wrangler.toml is gitignored, so your account details stay local.

5. Run locally:

yarn dev

Visit http://localhost:8787/ui to access the web interface.

Deployment

Production Setup

1. Set production secrets:

npx wrangler secret put TWILIO_AUTH_TOKEN
npx wrangler secret put MCP_BEARER_TOKEN
npx wrangler secret put ALLOWED_PHONE_NUMBERS
npx wrangler secret put ALLOWED_EMAILS
npx wrangler secret put UPLOAD_API_KEY
npx wrangler secret put UI_PASSWORD

# Optional
npx wrangler secret put OPENROUTER_API_KEY
npx wrangler secret put SENDGRID_API_KEY

2. Deploy:

yarn deploy

3. Configure webhooks:

Twilio (SMS/MMS):

• Go to your Twilio phone number settings
• Set the MMS webhook URL to: https://your-domain.com/webhooks/twilio/mms
• Method: POST

SendGrid (Email):

• Go to Settings > Inbound Parse
• Add your domain and set the destination URL to: https://your-domain.com/webhooks/sendgrid/inbound
• Enable "POST the raw, full MIME message"

Optional: Custom Domain

In wrangler.toml, uncomment and configure:

routes = [
  { pattern = "your-domain.com", custom_domain = true }
]

Usage

Web UI

Visit https://your-domain.com/ui to:

• View receipts organized by processing status
• See AI-generated summaries (merchant, amount, items)
• Open full-size images/files
• Mark receipts as processed or not processed
• Generate summaries on-demand for receipts

MCP Server (AI Agents)

Configure Claude Desktop or other MCP clients to access your receipts:

Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "receipts": {
      "type": "http",
      "url": "https://your-domain.com/mcps/receipts",
      "headers": {
        "Authorization": "Bearer YOUR_MCP_BEARER_TOKEN"
      }
    }
  }
}

Available MCP Tools:

• searchNotProcessed - Find all unprocessed receipts
• searchProcessed - Find all processed receipts
• download - View a receipt image (auto-fetches for agents)
• getPresignedUrl - Get a presigned URL for a receipt (1 hour expiry)
• setAsProcessed - Mark a receipt as processed
• setAsNotProcessed - Mark a receipt as not processed

SMS/MMS (Twilio)

Simply text a photo of your receipt to your configured Twilio number. The system will:

1. Verify the sender is in the allowlist
2. Store the image in R2 as "not_processed"
3. Generate an AI summary (if OpenRouter is configured)
4. Reply with "Thanks, your receipt is saved!"

Email (SendGrid)

Forward receipts to your configured email address. The system will:

1. Verify the sender is in the allowlist
2. Store the full email as .eml and attachments separately
3. Generate AI summaries for text and images
4. Reply with "Thanks, your receipt is saved!"

API Upload

Upload receipts programmatically:

curl -X POST https://your-domain.com/upload \
  -H "Authorization: Bearer YOUR_UPLOAD_API_KEY" \
  -F "file=@receipt.jpg"

Response:

{
  "success": true,
  "id": "01JGABCD1234567890ABCDEFGH",
  "url": "https://your-domain.com/receipts/01JGABCD1234567890ABCDEFGH/view?token=..."
}

Configuration

Environment Variables

Variable	Required	Description
TWILIO_AUTH_TOKEN	Yes*	Twilio auth token for webhook verification
MCP_BEARER_TOKEN	Yes	Secret token for MCP server authentication
ALLOWED_PHONE_NUMBERS	Yes*	Comma-separated list of allowed phone numbers
ALLOWED_EMAILS	Yes*	Comma-separated list of allowed email addresses
UPLOAD_API_KEY	Yes	API key for direct uploads
UI_PASSWORD	Yes	Password for web UI access
OPENROUTER_API_KEY	No	OpenRouter API key for AI summaries
SENDGRID_API_KEY	No	SendGrid API key for email replies

*Required only if using that intake method

Security

• All intake methods use allowlists (phone numbers and emails)
• Twilio webhooks are verified using signature validation
• Web UI uses password authentication with HMAC session tokens
• MCP server requires bearer token authentication
• Presigned URLs expire after 1 hour

Development

Project Structure

src/
├── handlers/          # Webhook handlers (Twilio, SendGrid, Upload)
├── mcp/              # MCP server implementation
│   ├── handlers/     # MCP tool handlers
│   ├── server.ts     # JSON-RPC server
│   └── tools.ts      # Tool definitions
├── middleware/       # Auth and error handling
├── routes/          # HTTP routes (UI, webhooks, MCP)
├── types/           # TypeScript type definitions
└── utils/           # Utilities (R2, auth, signing, AI)

Type Checking

yarn typecheck

Testing

yarn test

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License - see LICENSE file for details

Acknowledgments

• Built with Hono
• Uses Cloudflare Workers
• MCP implementation based on Model Context Protocol