Skip to content

feat: support Synapse as alternative Matrix server provider#512

Draft
Jing-ze wants to merge 5 commits intoagentscope-ai:mainfrom
Jing-ze:worktree-feat-matrix-synapse
Draft

feat: support Synapse as alternative Matrix server provider#512
Jing-ze wants to merge 5 commits intoagentscope-ai:mainfrom
Jing-ze:worktree-feat-matrix-synapse

Conversation

@Jing-ze
Copy link
Copy Markdown
Contributor

@Jing-ze Jing-ze commented Mar 31, 2026 •
edited
Loading

Summary

  • Add support for Synapse as an alternative Matrix server provider alongside Tuwunel
  • Synapse is embedded inside the Manager container, listening on the same port 6167 as Tuwunel — only one runs at a time, controlled by HICLAW_MATRIX_PROVIDER env var
  • PostgreSQL runs as a sidecar container (auto-provisioned or user-provided via HICLAW_PG_HOST)
  • Provider-aware user registration: Tuwunel uses registration_token, Synapse uses HMAC shared-secret
  • Improved error diagnostics in worker/human creation scripts
  • CI auto-detects when to test Synapse based on changed files

Key Changes

  • manager/Dockerfile: install Synapse via pip into base Python 3.10
  • manager/scripts/init/start-synapse.sh: new startup script (symmetric with start-tuwunel.sh)
  • manager/supervisord.conf: add [program:synapse] entry
  • shared/lib/hiclaw-env.sh: centralized matrix_register_user / matrix_register_user_raw functions
  • install/hiclaw-install.sh / .ps1: step_matrix_provider + PG sidecar management
  • create-worker.sh: error logging, invite list fix (exclude room creator), room_id recovery
  • CI workflow: matrix strategy with path-based Synapse detection

Design Decisions

  • Same port (6167): eliminates all routing/URL/config complexity — Higress, Workers, Element Web all work unchanged
  • Embedded, not sidecar: Synapse runs inside Manager container to preserve the "everything on 127.0.0.1" assumption
  • pip install, not Docker COPY: avoids GLIBC version mismatch between Synapse's Debian Trixie and base Ubuntu 22.04
  • PG still sidecar: PostgreSQL cannot be embedded, but managing one sidecar is simple

Test Plan

  • Default install (tuwunel) — verify no regression
  • HICLAW_MATRIX_PROVIDER=synapse install — verify Synapse starts on port 6167
  • Element Web login works with Synapse backend
  • Worker creation (openclaw + copaw) works with Synapse
  • make uninstall cleans up PG container and volume
  • CI workflow triggers Synapse tests when related files change

🤖 Generated with Claude Code

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 31, 2026 •
edited
Loading

📊 CI Metrics Report

Summary

Metric Current Baseline Change
LLM Calls 189 178 +11 ↑ +6.2%
Input Tokens 6526221 4375537 +2150684 ↑ +49.2%
Output Tokens 38919 43659 -4740 ↓ -10.9%
Total Tokens 6565140 4419196 +2145944 ↑ +48.6%

By Role

Role Metric Current Baseline Change
🧠 Manager LLM Calls 145 139 +6 ↑ +4.3%
Input Tokens 5690118 3612053 +2078065 ↑ +57.5%
Output Tokens 25929 27064 -1135 ↓ -4.2%
Total Tokens 5716047 3639117 +2076930 ↑ +57.1%
🔧 Workers LLM Calls 44 39 +5 ↑ +12.8%
Input Tokens 836103 763484 +72619 ↑ +9.5%
Output Tokens 12990 16595 -3605 ↓ -21.7%
Total Tokens 849093 780079 +69014 ↑ +8.8%

Per-Test Breakdown

Test Mgr Calls Wkr Calls Δ Calls Mgr In Wkr In Mgr Out Wkr Out Δ Tokens Trend
02-create-worker 11 0 -1 ↓ -8.3% 257272 0 1107 0 +15388 ↑ +6.3% ✅ improved
03-assign-task 18 6 -3 ↓ -11.1% 456764 110624 2624 1044 +24347 ↑ +4.5% ✅ improved
04-human-intervene 17 0 -29 ↓ -63.0% 452774 0 3027 0 -463656 ↓ -50.4% ✅ improved
05-heartbeat 8 0 -4 ↓ -33.3% 262237 0 1543 0 -61267 ↓ -18.8% ✅ improved
06-multi-worker 55 28 +53 ↑ +176.7% 1903031 544381 9881 9986 +1626410 ↑ +193.4% ⚠️ regressed
14-git-collab 36 10 -5 ↓ -9.8% 2358040 181098 7747 1960 +1004722 ↑ +65.1% ✅ improved

Trends

✅ 5 test(s) improved (fewer LLM calls)
⚠️ 1 test(s) regressed (more LLM calls)

Generated by HiClaw CI on 2026-04-02 07:39:22 UTC

📦 Download debug logs & test artifacts

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 31, 2026 •
edited
Loading

❌ Integration Tests Failed

Commit: 4f64b37
Workflow run: #567

Test Results 
No test output captured.
Debug Log (tail) 
No debug logs available.

📦 Download full debug logs & test artifacts

@Jing-ze Jing-ze force-pushed the worktree-feat-matrix-synapse branch from 711b839 to 1a1a6b8 Compare April 1, 2026 01:48
Jing-ze and others added 2 commits April 1, 2026 13:58
@Jing-ze
feat: support Synapse as alternative Matrix server provider
16f1b2a 
Add Synapse as an alternative to Tuwunel for the Matrix homeserver.
Key changes:
- Provider-aware user registration (Tuwunel: registration_token, Synapse: shared-secret HMAC)
- Centralized Matrix provider abstraction in hiclaw-env.sh
- Synapse runs as sidecar inside Manager container (same port 6167)
- PostgreSQL sidecar for Synapse storage
- Install wizard step for Matrix provider selection (sh + ps1)
- CI auto-detection of Synapse-related file changes for dual-provider testing
- Higress route configuration for correct provider backend
- Improved error diagnostics in worker/human creation scripts
- Synapse rate limit relaxation for internal agent API calls
- Cleanup of Synapse containers and volumes on uninstall
@Jing-ze @claude
fix(install): add PG connectivity check and ensure network for Synaps…
62fd19c 
…e sidecar

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Jing-ze Jing-ze force-pushed the worktree-feat-matrix-synapse branch from 1a1a6b8 to 62fd19c Compare April 1, 2026 06:47
@Jing-ze Jing-ze marked this pull request as ready for review April 1, 2026 07:58
johnlanni
johnlanni requested changes Apr 1, 2026
View reviewed changes
@Jing-ze Jing-ze marked this pull request as draft April 2, 2026 02:46
Jing-ze and others added 2 commits April 2, 2026 14:00
@Jing-ze @claude
fix(install): address PR review — lock provider on upgrade, remove PG…
d13aab8 
… sidecar, fix injection

- Lock matrix provider selection during upgrade to prevent data loss
  (Tuwunel uses RocksDB, Synapse uses PostgreSQL — no migration path)
- Remove auto-managed PostgreSQL sidecar; Synapse now requires user-provided PG
- Fix shell injection in PG connectivity check (replace /dev/tcp with nc -z + host validation)
- Improve registration error handling: only silence M_USER_IN_USE, surface real errors
- Revert CI changes (no Synapse test matrix until PG setup is added)
- Clean up "sidecar" references in descriptions and comments

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Jing-ze @claude
refactor(install): remove interactive matrix provider selection
72847fe 
Synapse provider is now configured purely via environment variables
(HICLAW_MATRIX_PROVIDER=synapse), no interactive prompts in install scripts.
This keeps the install flow simple while still supporting Synapse for
advanced users who manage their own PostgreSQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Jing-ze Jing-ze marked this pull request as ready for review April 2, 2026 06:56
@Jing-ze Jing-ze marked this pull request as draft April 2, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@johnlanni johnlanni johnlanni requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Jing-ze @johnlanni