Skip to content

feat(whatsapp): propagate sender identity to agent context#598

Open
f-liva wants to merge 44 commits intoRightNow-AI:mainfrom
f-liva:feat/whatsapp-sender-identity
Open

feat(whatsapp): propagate sender identity to agent context#598
f-liva wants to merge 44 commits intoRightNow-AI:mainfrom
f-liva:feat/whatsapp-sender-identity

Conversation

@f-liva
Copy link

@f-liva f-liva commented Mar 14, 2026

Summary

Fixes #597 — WhatsApp sender identity metadata was silently discarded in the message pipeline, causing agents to treat all users as their owner.

  • Add metadata field to MessageRequest — accepts sender info from channel gateways (WhatsApp, etc.)
  • Create SenderContext struct — typed representation of channel + sender_id + sender_name
  • Propagate through kernelsend_message_with_handle_and_blocks() now accepts optional SenderContext
  • Inject into system prompt — agents see sender name and platform ID in their "Channel Awareness" section
  • Activate allowed_users filtering — open-by-default (empty list = accept all), configurable per-channel
  • Make is_allowed() public — remove #[allow(dead_code)], available for external callers

Files changed

File Change
openfang-api/src/types.rs Add metadata: Option<HashMap<String, Value>> to MessageRequest
openfang-api/src/routes.rs Parse metadata → SenderContext, add allowed_users check
openfang-types/src/message.rs New SenderContext struct
openfang-kernel/src/kernel.rs Thread SenderContext through execute pipeline
openfang-runtime/src/prompt_builder.rs Render sender identity in system prompt
openfang-channels/src/whatsapp.rs Make is_allowed() public, open-by-default

Behavior

Before: Agent receives WhatsApp message with no sender context → treats everyone as owner
After: Agent sees sender identity in system prompt → can apply privacy rules and distinguish users

# config.toml — optional, open by default
[channels.whatsapp]
allowed_users = ["+391234567890"]  # empty or omitted = accept all

Test plan

  • Send WhatsApp message from owner's number → agent receives sender context, identifies as owner
  • Send WhatsApp message from unknown number (no allowed_users set) → message accepted, agent sees sender as non-owner
  • Set allowed_users in config → messages from unlisted numbers return 403
  • Telegram and other channels unaffected (no metadata = no sender context injected)
  • cargo build --lib compiles cleanly

🤖 Generated with Claude Code

Federico Liva and others added 30 commits March 12, 2026 00:06
- Add toolchain: Node.js 22, Claude Code CLI, Python 3, uv, Go, gh, ffmpeg
- Add gosu + non-root user (openfang) with passwordless sudo
- Entrypoint drops root privileges via gosu for Claude Code compatibility
- Add GitHub Actions workflow to auto-sync upstream releases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace opaque dark-background logo with transparent PNG and add
CSS invert filter for light theme so the snake is visible in both
dark and light modes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend the CSS invert filter to also cover .message-avatar img,
so the agent logo in chat messages is visible in light mode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, Claude Code had key_required=false so detect_auth() always
set NotRequired ("No Key Needed"), regardless of whether the CLI was
installed or authenticated. Now it checks:
- CLI installed + authenticated → Configured
- CLI installed, not authenticated → Missing (Not Set)
- CLI not installed → NotRequired (No Key Needed)

This also fixes the Runtime/Overview page showing Claude Code as "not
configured" when it is actually ready.

Fixes RightNow-AI#376

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GITHUB_TOKEN cannot push changes to .github/workflows/ files.
Use a PAT with workflows permission instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Install pip3, playwright, and Chromium with system dependencies
for browser automation and PDF generation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The rebase creates new commit hashes, and --force-with-lease fails
when the remote was updated between fetches. Since this is an
intentional rebase sync, --force is appropriate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
OpenFang's Browser Hand checks for 'python' not 'python3'.
Debian only installs the python3 binary, so add a symlink.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ResourceQuota default set max_cost_per_hour_usd to 1.0 while daily
and monthly were 0.0 (unlimited). This caused agents without explicit
quota configuration to hit a hidden $1/hour cap.

Also fixes apply_budget_defaults() which compared against the old
hardcoded default value of 1.0.

Fixes RightNow-AI#416

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously the workflow only rebuilt when a new upstream tag was
detected, so custom commits on main were silently ignored.
Now pushes to main trigger a full build+push to Docker Hub.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When upstream changes conflict with custom fork commits, the rebase
now automatically skips the conflicting commits instead of failing.
Skipped commits are typically duplicates already applied upstream.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Browser Hand no longer requires Playwright — it now needs Chromium
directly. Replace pip3 playwright install with apt chromium package.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Upstream v0.3.46 removed the flag, causing all agents using Claude Code
to be completely paralyzed — every command requires interactive terminal
approval that cannot be given via dashboard or Telegram.

Without this flag, Claude Code as a provider is unusable in any
non-interactive context (web UI, Telegram, API, scheduled tasks).

Refs: RightNow-AI#515, RightNow-AI#325

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…h exponential backoff

The gateway only reconnected on `restartRequired` and `timedOut` disconnect
reasons. All other Baileys disconnect codes (connectionClosed/428,
connectionLost/408, connectionReplaced/440, multideviceMismatch, etc.) fell
through to the else branch which treated them as "QR expired" and stopped
trying. This caused the gateway to silently die after any transient network
issue or WhatsApp server-side session refresh.

Changes:
- Treat every disconnect reason except `loggedOut` as recoverable
- Add exponential backoff (2s base, 1.5x multiplier, 60s cap) instead of
  a fixed 2s delay
- Reset backoff counter on successful connection
- Auto-connect on startup when auth credentials exist from a previous
  session, eliminating the need for a manual POST /login/start after
  process restarts

Closes RightNow-AI#529

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tability

Add qwen-code as a new subprocess-based LLM provider, mirroring the
claude-code driver pattern. Qwen Code CLI (qwen) uses --yolo for
non-interactive mode and supports json/stream-json output formats.

New files:
- drivers/qwen_code.rs: full driver with complete/stream, env filtering

Changes:
- drivers/mod.rs: register qwen-code provider (defaults, create_driver)
- model_catalog.rs: add provider info, 3 models (qwen3-coder,
  qwen-coder-plus, qwq-32b), aliases, auth detection
- claude_code.rs: extract build_args() for testability, fix duplicate
  --dangerously-skip-permissions flag that was always added regardless
  of skip_permissions setting

Tests: 31 new/updated (18 qwen-code + 13 claude-code) covering
build_args with/without permission flags, streaming, model selection,
prompt building, JSON parsing, and catalog integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Federico Liva and others added 14 commits March 12, 2026 11:09
Install @qwen-code/qwen-code alongside Claude Code so the qwen-code
provider is available in the container.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…outing and ClaudeCode driver

Root cause 1: Vision model swap (kernel.rs) changed the model to qwen-vl-plus
but left the provider as claude-code, routing images to the wrong driver.

Root cause 2: ClaudeCodeDriver.build_prompt() called text_content() which
silently dropped all ContentBlock::Image and ContentBlock::ImageUrl blocks.

Fix:
- kernel.rs: vision model swap now also updates the provider
- claude_code.rs: full image support via temp files passed with --files flag
  (handles base64, data URIs, and HTTP URLs)
- All other drivers: ensure ImageUrl content blocks are handled
- compactor.rs: handle ImageUrl in conversation compaction
- bridge.rs: improved image dispatch reliability

Closes RightNow-AI#528

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Notify on build success/failure via Telegram Bot API using
TELEGRAM_BOT_TOKEN and TELEGRAM_CHAT_ID secrets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Check model catalog's supports_vision flag before swapping. Models like
claude-opus-4-6 handle images natively — no need to swap to a separate
vision model (which may use a different, unconfigured provider).

Also warn when images arrive but no vision fallback is available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When vision_model is explicitly set in config, always use it (forced
override). Only fall back to the current agent model when no
vision_model is configured and the model supports vision natively.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The CLI option is --file (singular), not --files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The --file flag requires a session token for file downloads (Files API).
Instead, embed @/tmp/image.jpg directly in the prompt text, which tells
Claude Code CLI to read the local file natively.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a `lifecycle_reactions` boolean to ChannelOverrides (default: true).
When set to false, suppresses the ⏳→🤔→✅/❌ emoji reactions on
incoming messages. Applies to both text and multimodal dispatch paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract reply_to_message metadata from Telegram updates (text, caption,
sender, message_id) and prepend it as contextual prefix when dispatching
to agents. Supports both text and image message paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WhatsApp gateway sends sender metadata (phone number, display name) but
the API was silently discarding it because MessageRequest had no metadata
field. This caused agents to treat all WhatsApp users as their owner.

Changes:
- Add optional metadata field to MessageRequest (types.rs)
- Parse metadata into SenderContext and propagate through kernel (routes.rs, kernel.rs)
- Inject sender identity into agent system prompt (prompt_builder.rs)
- Add SenderContext struct to message types (message.rs)
- Activate is_allowed() filter with open-by-default behavior (whatsapp.rs)
- Add allowed_users check in API routes (open mode when list is empty)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WhatsApp sender identity metadata lost in message pipeline

2 participants