Messages to an SMS number failing to be sent when `disable_sms_portals`; BlueBubble logs report "Chat does not exist!"

[ELLIOTTCABLE]

First off, actual symptoms on my end:

• I can receive iMessage, RCS, and SMS messages just fine via the bridge (I'm using Beeper Desktop)
• I can receive (and send) iMessage, RCS, and SMS via Messages / BlueBubbles directly on the bridge machine
• I can send iMessage and RCS via the bridge;
• but I cannot send SMS, specifically, via the bridge. I get the following errors when I do:

sh-imessage: 2025-02-02T15:33:35-06:00 DBG Starting handling Matrix message $HCSD5IHeErWBRxDuAflRzdqx1QYvl4YfXbZxVMtZeL4:beeper.local portal_guid=iMessage;-;21525
bluebubbles: [2025-02-02 15:33:35.217][debug] Request to /api/v1/message/text (URL Params: {})
bluebubbles: [2025-02-02 15:33:35.218][debug] Sending message "test 3" to iMessage;-;21525
bluebubbles: [2025-02-02 15:33:35.241][info] Adding await for chat: "iMessage;-;21525"; text: test3; tempGuid: temp-mblMGzjC
bluebubbles: [2025-02-02 15:33:35.448][debug] Raw Error: Ch: [500] Message Send Error
bluebubbles: [2025-02-02 15:33:35.457][debug] API Error: [500] Message Send Error -> Chat does not exist!
bluebubbles: [2025-02-02 15:33:35.478][debug] Ch: [500] Message Send Error
bluebubbles:     at sendText (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:267547)

(Full log)

At first I thought it might be a database corruption in my local installation; perhaps the database was initialized and backfilled with one value for disable_sms_portals before the new value was picked up, and thus necessary data wasn't duplicated?

... but then the logs are a little more illuminating when a brand-new SMS is recieved:

bluebubbles: [2025-02-03 14:10:51.346][info] [BlueBubblesServer] New Message from +*******4444, "hi"; Date: 2/3/2025, 2:10:50 PM
bluebubbles: [2025-02-03 14:10:51.360][info] [BlueBubblesServer] Chat read [SMS;-;+12223334444]
sh-imessage: 2025-02-03T14:10:51-06:00 DBG Received incoming message D85CBA71-F388-D622-E5B6-24DEACD0F9BF in SMS;-;+12223334444 () module=iMessage
sh-imessage: 2025-02-03T14:10:51-06:00 INF Creating Matrix room to handle message portal_guid=iMessage;-;+12223334444
sh-imessage: 2025-02-03T14:10:51-06:00 DBG Getting chat info to create Matrix room portal_guid=iMessage;-;+12223334444
bluebubbles: [2025-02-03 14:10:51.415][debug] Request to /api/v1/chat/iMessage;-;+12223334444 (URL Params: {"with":"participants,lastMessage"})
bluebubbles: [2025-02-03 14:10:51.416][debug] Raw Error: vh: [404] The requested resource was not found
bluebubbles: [2025-02-03 14:10:51.417][debug] API Error: [404] The requested resource was not found -> Chat does not exist!

Even for brand-new numbers which BlueBubbles/mautrix-imessage have never seen, SMS;-;+12223334444 is getting rewritten into iMessage;-;+12223334444, which from BlueBubbles' perspective, does not exist. Thus, the message-sending fails.

---

My config diff:

bridge:
    # Whether or not the bridge should send error notices via m.notice events
    # when a message fails to bridge.
    send_error_notices: true

    # Should SMS chats always be in the same room as iMessage chats with the same phone number?
    disable_sms_portals: true

    backfill:
        # Maximum age of chats to sync in days.
        initial_sync_max_age: 60

    media_viewer:
        imessage_min_size: 1024288000

Versions and full configuration details:

• macOS "Sequoia" 15.1.1 (24B91)
• BlueBubbles Server v1.9.8
• bbctl v0.13.0 (built Dec 15 2024; static binary from GitHub releases)
• mautrix-imessage 0.1.0+dev.892f056 (built Nov 27 2024; automatically installed by bbctl)

All of the above are running on the same host:

  Model Name:	Mac Pro
  Model Identifier:	MacPro7,1
  Enclosure:	Rack
  Processor Name:	12-Core Intel Xeon W
  Processor Speed:	3.3 GHz
  Number of Processors:	1
  Total Number of Cores:	12
  L2 Cache (per Core):	1 MB
  L3 Cache:	19.3 MB
  Hyper-Threading Technology:	Enabled
  Memory:	48 GB
  System Firmware Version:	2069.40.2.0.0 (iBridge: 22.16.11072.0.0,0)
  OS Loader Version:	582~2132

  System Version:	macOS 15.1.1 (24B91)
  Kernel Version:	Darwin 24.1.0
  Boot Mode:	Normal
  Secure Virtual Memory:	Enabled
  System Integrity Protection:	Disabled

BlueBubbles configuration:

• Zrok proxy
• "Messages" Private API helper ✅ enabled
• "FaceTime" Private API helper 🚫 disabled
• Accessibility permissions ✅ given
• Full Disk Access ✅ given
• Notifications 🚫 disabled

Full config.yaml:

homeserver:
    address: https://matrix.beeper.com/_hungryserv/elliottcable
    websocket_proxy: wss://matrix.beeper.com/_hungryserv/elliottcable
    ping_interval_seconds: 180
    domain: beeper.local

    software: hungry
    async_media: true

appservice:
    hostname: null
    port: null
    tls_key: null
    tls_cert: null
    database:
        type: sqlite3-fk-wal
        uri: file:mautrix-imessage.db?_txlock=immediate
    
    id: REDACTED
    bot:
        username: sh-imessagebot
        displayname: iMessage bridge bot
        avatar: mxc://maunium.net/tManJEpANASZvDVzvRvhILdX
    ephemeral_events: true
    
    as_token: REDACTED
    hs_token: REDACTED

imessage:
    platform: bluebubbles
    imessage_rest_path: "darwin-barcelona-mautrix"
    imessage_rest_args: []
    contacts_mode: mac
    log_ipc_payloads: false
    hacky_set_locale: null
    environment: []
    unix_socket: mautrix-imessage.sock
    ping_interval_seconds: 15
    delete_media_after_upload: false
    bluebubbles_url: REDACTED
    bluebubbles_password: REDACTED
segment:
    key: null
    user_id: null
hacky_startup_test:
    identifier: null
    message: null
    response_message: null
    key: null
    echo_mode: false
    send_on_startup: false
    periodic_resolve: -1

bridge:
    user: "@elliottcable:beeper.com"
    
    username_template: sh-imessage_{{.}}
    displayname_template: "{{.}}"
    personal_filtering_spaces: true
    
    delivery_receipts: false
    message_status_events: true
    send_error_notices: true
    max_handle_seconds: 60
    device_id: null
    sync_direct_chat_list: false
    login_shared_secret: appservice
    double_puppet_server_url: null
    backfill:
        enable: true
        initial_limit: 100
        initial_sync_max_age: 60
        unread_hours_threshold: 720
        # ... snip ...
    periodic_sync: true
    find_portals_if_db_empty: false
    media_viewer:
        url: https://media.beeper.com
        homeserver: beeper.com
        sms_min_size: 409600
        imessage_min_size: 1024288000
        template: "Full size attachment: %s"
    convert_heif: false
    convert_tiff: true
    convert_video:
        enabled: false
        ffmpeg_args: ["-c:v", "libx264", "-preset", "faster", "-crf", "22", "-c:a", "copy"]
        extension: "mp4"
        mime_type: "video/mp4"
    command_prefix: "!im"
    force_uniform_dm_senders: true
    disable_sms_portals: true
    reroute_mms_group_replies: false
    federate_rooms: false
    caption_in_message: true
    private_chat_portal_meta: default
    
    encryption:
        allow: true
        default: true
        appservice: true
        require: true
        allow_key_sharing: true
        delete_keys:
            delete_outbound_on_ack: true
            dont_store_outbound: false
            ratchet_on_decrypt: true
            delete_fully_used_on_decrypt: true
            delete_prev_on_new_session: true
            delete_on_device_delete: true
            periodically_delete_expired: true
        verification_levels:
            receive: cross-signed-tofu
            send: cross-signed-tofu
            share: cross-signed-tofu
        rotation:
            enable_custom: true
            milliseconds: 2592000000
            messages: 10000
            disable_device_change_key_rotation: true
    
    relay:
        enabled: false
        whitelist: []
        message_formats:
            m.text: "{{ .Sender.Displayname }}: {{ .Message }}"
            m.notice: "{{ .Sender.Displayname }}: {{ .Message }}"
            m.emote: "* {{ .Sender.Displayname }} {{ .Message }}"
            m.file: "{{ .Sender.Displayname }} sent a file: {{ .FileName }}"
            m.image: "{{ .Sender.Displayname }} sent an image: {{ .FileName }}"
            m.audio: "{{ .Sender.Displayname }} sent an audio file: {{ .FileName }}"
            m.video: "{{ .Sender.Displayname }} sent a video: {{ .FileName }}"

logging:
    min_level: debug
    writers:
        - type: stdout
          format: pretty-colored
        - type: file
          format: json
          filename: ./logs/mautrix-imessage.log
          max_size: 100
          max_backups: 10
          compress: false

[ELLIOTTCABLE]

Writing down my progress before I disappear off to dinner:

Spelunking into the code a bit, at least part of this is clearly intended behaviour - maybeGetPortalByGUID() explicitly rewrites the macOS chat id into an iMessage form when disable_sms_portals is enabled:

imessage/portal.go

Lines 79 to 85 in 892f056

	if br.Config.Bridge.DisableSMSPortals && strings.HasPrefix(guid, "SMS;-;") {
	parsed := imessage.ParseIdentifier(guid)
	if !parsed.IsGroup && parsed.Service == "SMS" {
	parsed.Service = "iMessage"
	guid = parsed.String()
	}
	}

... but I haven't yet figured out how that's intended to function, or why it works for other users. My theories:

1. something missing in the initial 'portal setup' to create the paired, corresponding iMessage portal to map SMS messages into, or
2. something missing in the message-send functionality to remap the iMessage-handle back into an SMS-handle that BlueBubbles will understand before making the REST request.

[ELLIOTTCABLE]

More notes: I haven't tracked down the bug yet, but I did verify that it's replicable. I can't¹ seem to create a new config w/ disable_sms_portals active from the start; so I can only activate that mode after backfill has already started:

1. Clone the repo (as of now, HEAD is 892f056);

2. build/run once to generate config.dbg.yaml;

   $ brew install libolm
   $ export CGO_CFLAGS="-I$(brew --prefix libolm)/include" CGO_LDFLAGS="-L$(brew --prefix libolm)/lib"
   $ bbctl run --local-dev --no-override-config --config-file config.dbg.yaml \
      --param 'imessage_platform=bluebubbles' --param 'bluebubbles_url=http://localhost:1234' \
      -t imessage "sh-imessagedev"

3. kill the initial process, then enable SMS portal-merging (I also disabled backfilling to speed things up);

   ⌃C
   $ sed -i.bak \
      -e 's|disable_sms_portals: false|disable_sms_portals: true|' \
      -e '/backfilling/{n ;s/enable: true/enable: false/;}' \
      config.dbg.yaml
   $ bbctl run --local-dev --no-override-config --config-file config.dbg.yaml \
      -t imessage "sh-imessagedev"

4. find one of the generated portals that's w/ an SMS user, and try and send a message:

   

   [2025-02-03 21:28:39.680][debug] Request to /api/v1/message/text (URL Params: {})
   [2025-02-03 21:28:39.681][debug] Sending message "test 5-2" to iMessage;-;+12223334444
   [2025-02-03 21:28:39.681][info] Adding await for chat: "iMessage;-;+12223334444"; text: test52; tempGuid: temp-TVptdzqb
   [2025-02-03 21:28:39.683][debug] Raw Error: Ch: [500] Message Send Error
   [2025-02-03 21:28:39.683][debug] API Error: [500] Message Send Error -> Chat does not exist!
   [2025-02-03 21:28:39.684][debug] Ch: [500] Message Send Error
       at sendText (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:267547)
       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
       at async validateText (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:299038)
       at async /Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:311188
       at async Mh (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:256888)
       at async Jh (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:291459)
       at async Qh (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:291838)
       at async Hh (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:290965)
       at async Qh (/Applications/BlueBubbles.app/Contents/Resources/app.asar/dist/main.js:1:291838)
   [2025-02-03 21:28:39.684][debug] Request to /api/v1/message/text took 4 ms
   2025-02-03T21:28:39-06:00 ERR Failure when sending message to BlueBubbles component=bluebubbles response={"data":{"handle":{}},"error":{"message":"Chat does not exist!","type":"iMessage Error"},"message":"Message Send Error","status":500} statusCode=500
   2025-02-03T21:28:39-06:00 ERR Error sending to iMessage: could not send message module=Portal/iMessage;-;+12223334444
   

---

Footnotes

1. Notably, there doesn't seem to be any way to get --local-dev to actually use the --config-file provided when starting from scratch? Even when explicitly passing --no-override-config, it says "Existing bridge type not found, falling back to generating new config" and overwrites that given file with the template; the initial run will always backfill with disable_sms_portals set to false ...
   
   trying to use bbctl config directly to register and create a config-file without backfilling was just giving me encryption errors. ↩