Skip to content

Feature/computer use/mac support#108

Open
claude-code-best wants to merge 4 commits intomainfrom
feature/computer-use/mac-support
Open

Feature/computer use/mac support#108
claude-code-best wants to merge 4 commits intomainfrom
feature/computer-use/mac-support

Conversation

@claude-code-best
Copy link
Copy Markdown
Owner

@claude-code-best claude-code-best commented Apr 4, 2026

Summary by CodeRabbit

  • New Features

    • Computer Use available on macOS and Windows
    • Voice Mode restored
    • Claude-in-Chrome user workflow and controls
    • OpenAI API compatibility layer enabled
  • Bug Fixes

    • Improved macOS app detection and permission handling
    • Fuzzy app-name resolution for better matching
  • Documentation

    • Updated Computer Use user guide and macOS setup
    • Added Computer Use test report
    • Revised Chrome & feature enablement docs

@mintlify
Copy link
Copy Markdown

mintlify bot commented Apr 4, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
ccb-863780bf 🟢 Ready View Preview Apr 4, 2026, 3:47 AM

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 4, 2026

📝 Walkthrough

Walkthrough

Updated docs and defaults to re-enable Computer Use, Voice Mode, and an OpenAI compatibility layer; replaced macOS AppleScript app enumeration with Spotlight-based discovery; added fuzzy app-name resolution, a hotkey fallback, and safer macOS permission checks; added user-facing Computer Use and Chrome/Voice docs and an MCP test report.

Changes

Cohort / File(s) Summary
Top-level docs & feature flags
CLAUDE.md, README.md
Expanded default feature flags (added CHICAGO_MCP, VOICE_MODE) and added links/docs for Chrome, Computer Use, Voice, and OpenAI compatibility.
User-facing docs
docs/features/claude-in-chrome-mcp.md, docs/features/computer-use.md, docs/features/computer-use-mcp-test-report.md, DEV-LOG.md
Replaced engineering recovery notes with user guides (Chrome/Computer Use/Voice), added MCP test report, and logged macOS Computer Use fixes and test outcomes.
macOS app enumeration
packages/@ant/computer-use-swift/src/backends/darwin.ts
Replaced AppleScript traversal with mdfind/mdls pipeline to list .app bundles and obtain real kMDItemCFBundleIdentifier + displayName/path.
App resolution logic
packages/@ant/computer-use-mcp/src/toolCalls.ts
Added case-insensitive substring (fuzzy) fallback when exact bundle-id or name match fails.
ComputerUse API / hotkey fallback
packages/@ant/computer-use-swift/src/index.ts
Added ComputerUseAPI.hotkey that delegates to backend.hotkey when present; otherwise exposes a safe stub with registerEscape, unregister, notifyExpectedEscape.
Permission / host adapter
src/utils/computerUse/hostAdapter.ts
Guarded macOS permission flow: return granted if native Swift module lacks tcc, otherwise call cu.tcc.checkAccessibility() and cu.tcc.checkScreenRecording() and return structured result.
Misc edits
packages/@ant/...-napi (docs/CLAUDE.md references)
Documentation describing restoration status of various *-napi packages and Voice/OpenAI compatibility notes (gating/OAuth requirements).

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant CLI as Runtime/CLI
  participant MCP as Computer-Use MCP
  participant Swift as macOS Swift backend
  participant macOS as Spotlight/TCC

  User->>CLI: start with Computer Use enabled
  CLI->>MCP: request installed apps
  MCP->>Swift: call apps.listInstalled()
  Swift->>macOS: run `mdfind` / `mdls` to enumerate apps
  macOS-->>Swift: return bundleId/displayName/path
  Swift-->>MCP: return InstalledApp[]
  MCP->>MCP: resolveRequestedApps(requestedName)
  alt exact match fails
    MCP->>MCP: fuzzy substring match against displayName
  end
  MCP->>CLI: return resolved app info
  CLI->>MCP: ensureOsPermissions()
  MCP->>Swift: access cu.tcc if present
  alt tcc missing
    Swift-->>MCP: {granted: true}
  else tcc present
    Swift->>macOS: checkAccessibility(), checkScreenRecording()
    macOS-->>Swift: accessibility/screenRecording booleans
    Swift-->>MCP: permission result
  end
  MCP-->>CLI: permission status
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

Poem

🐇 Spotlight finds the bundles, clear and bright,
Fuzzy names catch Chrome in the night,
Hotkeys stand ready when backends shy away,
Permissions checked so tools can play,
A little rabbit hops—Computer Use is back today!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Feature/computer use/mac support' accurately summarizes the main change: adding macOS support for Computer Use functionality across multiple modules and documentation.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/computer-use/mac-support

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Nitpick comments (3)
docs/features/computer-use-mcp-test-report.md (1)

261-276: Add language specifier to fenced code block.

The ASCII diagram lacks a language identifier, which triggers markdownlint MD040. Adding a language specifier (even text or plaintext) improves tooling compatibility.

-```
+```text
 ┌─────────────────────────────────────────┐
 │  Tier: full                             │
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/features/computer-use-mcp-test-report.md` around lines 261 - 276, The
fenced ASCII diagram block is missing a language specifier causing markdownlint
MD040; update the opening fence for the diagram (the block that begins with the
line containing "┌─────────────────────────────────────────┐" and the line with
"│  Tier: full                             │") to include a language tag such as
text or plaintext (e.g., change ``` to ```text) so tooling recognizes it as a
plain-text code block.
packages/@ant/computer-use-mcp/src/toolCalls.ts (1)

799-809: Fuzzy substring matching may resolve ambiguously.

The substring match breaks on the first app whose displayName contains the requested string. When multiple apps match (e.g., "Code" matches both "Visual Studio Code" and "Xcode"), the selected app depends on mdfind iteration order, which is non-deterministic.

Consider either:

  1. Preferring an exact word-boundary match before falling back to substring (e.g., "Chrome" matches "Google Chrome" but not "Chromium" first).
  2. Collecting all candidates and selecting the shortest displayName (heuristic for "most specific match").
  3. Logging when multiple candidates exist so the model receives feedback about ambiguity.
Example: prefer shorter displayName as tie-breaker
     // Fuzzy fallback: match requested name as substring of display name
     // e.g. "Chrome" matches "Google Chrome", "Code" matches "Visual Studio Code"
     if (!resolved) {
       const lower = requested.toLowerCase();
+      let bestMatch: InstalledApp | undefined;
       for (const app of installed) {
         if (app.displayName.toLowerCase().includes(lower)) {
-          resolved = app;
-          break;
+          // Prefer shorter displayName as heuristic for most specific match
+          if (!bestMatch || app.displayName.length < bestMatch.displayName.length) {
+            bestMatch = app;
+          }
         }
       }
+      resolved = bestMatch;
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/`@ant/computer-use-mcp/src/toolCalls.ts around lines 799 - 809, The
fuzzy substring fallback in the block that sets resolved (using variables
requested, installed, and app.displayName) can pick a nondeterministic match;
change it to first try a word-boundary (whole-word) match of requested against
app.displayName (case-insensitive), and if no word-boundary match exists,
collect all substring candidates and pick the candidate with the shortest
displayName as a heuristic tie-breaker; if more than one candidate remains, emit
a debug/log message noting the ambiguity (include requested and the candidate
displayNames) so callers/models can observe it.
docs/features/computer-use.md (1)

40-42: Optional: Minor Chinese grammar improvements.

Static analysis suggests minor grammar refinements:

  • Line 40: "被操控" → Consider "可被……操控"
  • Line 42: "被隐藏" → Consider "会被……隐藏"

These are optional stylistic improvements for native fluency.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/features/computer-use.md` around lines 40 - 42, Update the Chinese
phrasing for naturalness in the listed steps: in item 3 (the phrase "被操控")
change to a passiveable form like "可被 … 操控" so it reads "你需要确认允许哪些 App 可被操控(…)",
and in item 5 (the phrase "被隐藏") change to a passive construction like "会被 … 隐藏"
so the sentence reads "被隐藏的窗口会被自动恢复" or "会被隐藏的窗口会自动恢复" — adjust whichever yields
the clearest, natural-sounding sentence.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/features/computer-use.md`:
- Line 14: Update the conflicting platform support statements: edit the sentence
at Line 14 that currently says "已支持 darwin / win32 / linux 三平台" to accurately
reflect current status (e.g., "已支持 darwin / win32 二平台,linux 架构已就绪") or, if Linux
is truly ready, update the Phase 3 completion status at Line 140 from "❌ 未完成" to
a completed marker; ensure the two lines (Line 14 and Line 140) consistently
state the same platform support status.
- Around line 77-79: The docs and gating code are inconsistent: the features
section claims `enabled` defaults to true and `hasRequiredSubscription()`
returns true (bypassing subscription checks) which contradicts the user-facing
requirement; either restore real subscription enforcement or explicitly mark the
bypass as a dev/testing flag and update docs. Locate the gating logic around
`enabled` and the `hasRequiredSubscription()` function and change one of two
things: (A) implement the real subscription check in `hasRequiredSubscription()`
so it validates the user's plan before enabling features, or (B) keep the
current bypass but add a clear developer-only flag/name and update the
documentation to state this is a dev/test override (and ensure sub-switches
sourced from GrowthBook are still gated appropriately). Ensure the chosen
approach is documented and consistent across the `enabled` flag,
`hasRequiredSubscription()` implementation, and any GrowthBook-driven
sub-switches.
- Around line 53-63: The "Linux 依赖工具" section currently reads like finished user
documentation but Phase 3 (Linux backend) is incomplete; update the docs by
either relocating this table under a "Planned Features" or "Future Support"
subsection or add a clear disclaimer at the start of the "Linux 依赖工具" section
(mentioning experimental/incomplete support and linking to Phase 3 status), and
if realistic, update the Phase 3 completion status; reference the section title
"Linux 依赖工具" and the listed tools (`xdotool`, `scrot`/`gnome-screenshot`,
`xrandr`, `xclip`, `wmctrl`, Wayland alternatives
`ydotool`/`grim`/`wl-clipboard`) when making the change.
- Line 20: The documentation and implementation disagree: the docs say Computer
Use requires Claude Pro/Max/Team but the implementation's
hasRequiredSubscription() currently returns true unconditionally; either enforce
subscription checks or update the docs. Fix by implementing actual validation
inside hasRequiredSubscription() (check the user's subscription flags/plan from
the auth/session object and return true only for Pro/Max/Team) and update any
callers of hasRequiredSubscription() to handle false correctly, or alternatively
update the docs text line that references the subscription requirement to state
that the feature is currently available to all users if you intend to keep the
unconditional true; reference hasRequiredSubscription() and the Computer Use
documentation string when making the change.

In `@packages/`@ant/computer-use-swift/src/backends/darwin.ts:
- Around line 179-180: The current parser uses '|' as delimiter which can break
when displayName contains '|' — update both the producer and consumer to use a
safer delimiter (e.g., tab or NUL). Change the shell-side echo that builds
"bundleId|displayName|appPath" to use "\t" (or '\0') and then update the parser
in the map callback that destructures [bundleId, displayName, path] to split on
the same delimiter (e.g., split('\t', 3) or use a NUL-aware split) so
displayName is not truncated; ensure filtering and mapping still handle empty
lines and maintain the existing variable names (bundleId, displayName, path).
- Around line 162-186: Persisted AppGrant bundleIds changed from synthetic
"com.app.*" to real Spotlight bundle IDs, causing
resolveRequestedApps/alreadyGranted lookups to miss existing grants; fix by
adding a migration/compatibility layer that maps legacy synthetic IDs to real
bundle IDs when loading grants (or when building the alreadyGranted Set in
resolveRequestedApps), e.g. detect bundleId starting with "com.app." and resolve
it to the real bundleId by looking up the app by displayName or path (or store
both legacy and real ids in the grant) so prior approvals are recognized; also
harden darwin.ts parsing by emitting a safer delimiter (NUL or JSON) from the
shell output instead of '|' and parse using that (or include an escaped/quoted
displayName) so app names containing '|' won't break the split.

In `@src/utils/computerUse/hostAdapter.ts`:
- Around line 49-56: The current hostAdapter.ts path returns { granted: true }
when requireComputerUseSwift() yields no cu.tcc, which incorrectly bypasses the
TCC gate used by ensureOsPermissions and toolCalls; change the branch in the
requireComputerUseSwift() handling so that when cu.tcc is absent it returns a
denied result (e.g., { granted: false, accessibility: false, screenRecording:
false }) instead of granted:true, updating the block that reads cu.tcc and the
return value to explicitly deny access so callers like
ensureOsPermissions/toolCalls treat permissions as not granted.

---

Nitpick comments:
In `@docs/features/computer-use-mcp-test-report.md`:
- Around line 261-276: The fenced ASCII diagram block is missing a language
specifier causing markdownlint MD040; update the opening fence for the diagram
(the block that begins with the line containing
"┌─────────────────────────────────────────┐" and the line with "│  Tier: full  
│") to include a language tag such as text or plaintext (e.g., change ``` to
```text) so tooling recognizes it as a plain-text code block.

In `@docs/features/computer-use.md`:
- Around line 40-42: Update the Chinese phrasing for naturalness in the listed
steps: in item 3 (the phrase "被操控") change to a passiveable form like "可被 … 操控"
so it reads "你需要确认允许哪些 App 可被操控(…)", and in item 5 (the phrase "被隐藏") change to
a passive construction like "会被 … 隐藏" so the sentence reads "被隐藏的窗口会被自动恢复" or
"会被隐藏的窗口会自动恢复" — adjust whichever yields the clearest, natural-sounding
sentence.

In `@packages/`@ant/computer-use-mcp/src/toolCalls.ts:
- Around line 799-809: The fuzzy substring fallback in the block that sets
resolved (using variables requested, installed, and app.displayName) can pick a
nondeterministic match; change it to first try a word-boundary (whole-word)
match of requested against app.displayName (case-insensitive), and if no
word-boundary match exists, collect all substring candidates and pick the
candidate with the shortest displayName as a heuristic tie-breaker; if more than
one candidate remains, emit a debug/log message noting the ambiguity (include
requested and the candidate displayNames) so callers/models can observe it.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 31485805-b950-42e0-a57b-a03892608e63

📥 Commits

Reviewing files that changed from the base of the PR and between 6fefff2 and ad1f90a.

📒 Files selected for processing (9)
  • CLAUDE.md
  • DEV-LOG.md
  • docs/features/claude-in-chrome-mcp.md
  • docs/features/computer-use-mcp-test-report.md
  • docs/features/computer-use.md
  • packages/@ant/computer-use-mcp/src/toolCalls.ts
  • packages/@ant/computer-use-swift/src/backends/darwin.ts
  • packages/@ant/computer-use-swift/src/index.ts
  • src/utils/computerUse/hostAdapter.ts

Comment on lines +77 to +79
- `enabled` 默认为 `true`,无需订阅检查
- `hasRequiredSubscription()` 直接返回 `true`
- 子开关(鼠标动画、隐藏窗口、剪贴板守卫等)均通过 GrowthBook 远程配置或使用默认值
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Gates configuration contradicts user documentation.

Lines 77-78 state:

  • enabled defaults to true without subscription check
  • hasRequiredSubscription() directly returns true

This directly contradicts the user-facing requirement documented at Line 20. The implementation suggests no actual subscription enforcement, making the user documentation misleading.

This appears to be intentional gating bypass for development/testing, but should be clarified in both sections.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/features/computer-use.md` around lines 77 - 79, The docs and gating code
are inconsistent: the features section claims `enabled` defaults to true and
`hasRequiredSubscription()` returns true (bypassing subscription checks) which
contradicts the user-facing requirement; either restore real subscription
enforcement or explicitly mark the bypass as a dev/testing flag and update docs.
Locate the gating logic around `enabled` and the `hasRequiredSubscription()`
function and change one of two things: (A) implement the real subscription check
in `hasRequiredSubscription()` so it validates the user's plan before enabling
features, or (B) keep the current bypass but add a clear developer-only
flag/name and update the documentation to state this is a dev/test override (and
ensure sub-switches sourced from GrowthBook are still gated appropriately).
Ensure the chosen approach is documented and consistent across the `enabled`
flag, `hasRequiredSubscription()` implementation, and any GrowthBook-driven
sub-switches.

Comment on lines +162 to 186
// Use Spotlight (mdfind) to enumerate .app bundles and mdls to get real bundle IDs.
// Searches /Applications, /System/Applications, and /System/Applications/Utilities
// so that system apps (Terminal, Chess, etc.) and core services (Finder) are found.
const proc = Bun.spawn([
'bash', '-c',
`for dir in /Applications /System/Applications /System/Applications/Utilities /System/Library/CoreServices; do
mdfind 'kMDItemContentType == "com.apple.application-bundle"' -onlyin "$dir" 2>/dev/null
done | sort -u | while read -r appPath; do
bundleId=$(mdls -raw -name kMDItemCFBundleIdentifier "$appPath" 2>/dev/null)
if [ -n "$bundleId" ] && [ "$bundleId" != "(null)" ]; then
displayName=$(basename "$appPath" .app)
echo "$bundleId|$displayName|$appPath"
fi
done`,
], { stdout: 'pipe', stderr: 'pipe' })
const text = await new Response(proc.stdout).text()
await proc.exited
return text.split('\n').filter(Boolean).map(line => {
const [bundleId, displayName, path] = line.split('|', 3)
return {
bundleId: `com.app.${displayName.toLowerCase().replace(/\s+/g, '-')}`,
displayName,
bundleId: bundleId ?? '',
displayName: displayName ?? '',
path: path ?? '',
}
})
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check if any grant persistence mechanism stores bundleIds that might mismatch
rg -n "bundleId" --type ts -C 2 src/utils/computerUse/ | head -50

# Check existing grant storage format
rg -n "allowedApps|AppGrant" --type ts src/ packages/@ant/computer-use-mcp/src/ | head -30

Repository: claude-code-best/claude-code

Length of output: 6486


🏁 Script executed:

# Find old listInstalled implementation or related code
git log --oneline -n 20 packages/@ant/computer-use-swift/src/backends/darwin.ts

# Check if there's any old implementation or TODO comments about bundle IDs
rg -n "listInstalled|bundleId|synthesize" packages/@ant/computer-use-swift/src/backends/darwin.ts

# Search for migration utilities or comments about format changes
rg -n "migration|format.*change|bundle.*id" packages/@ant/computer-use-mcp/src/ --type ts

Repository: claude-code-best/claude-code

Length of output: 2393


🏁 Script executed:

# Get the full context of darwin.ts to see all implementations
wc -l packages/@ant/computer-use-swift/src/backends/darwin.ts

# Read the file to see structure and all listInstalled implementations
cat -n packages/@ant/computer-use-swift/src/backends/darwin.ts | head -230

Repository: claude-code-best/claude-code

Length of output: 9301


🏁 Script executed:

# Check git history to see if there was a previous different implementation
git log --all --oneline -- packages/@ant/computer-use-swift/src/backends/darwin.ts | head -10

# Search for any other backends or swift implementations
find packages/@ant/computer-use-swift/src/backends -type f -name "*.ts" 2>/dev/null

# Search for any bundleId synthesis patterns in the codebase
rg -n "com\..*\." packages/@ant/computer-use-mcp/src/ --type ts | grep -i bundleid | head -20

# Check if there's any migration code or version handling for bundleIds
rg -n "migration|legacy|old.*bundle" packages/@ant/computer-use-mcp/src/ --type ts

Repository: claude-code-best/claude-code

Length of output: 1338


🏁 Script executed:

# Check what was added in the initial Computer Use commit
git show --stat e3264a1 | head -50

# Check if listInstalled existed in the previous version
git show e3264a1:packages/@ant/computer-use-swift/src/backends/darwin.ts | grep -A 30 "listInstalled" | head -40

# Also check the restore commit
git show --stat ad1f90a | head -30

Repository: claude-code-best/claude-code

Length of output: 5269


🏁 Script executed:

# Verify impact by checking if any migration code exists
rg -n "com\.app\.|migration|bundleId.*compat" packages/@ant/computer-use-mcp/src/ --type ts

# Confirm this is the only listInstalled implementation now
rg -n "listInstalled" packages/@ant/computer-use-swift/src/backends/ --type ts

Repository: claude-code-best/claude-code

Length of output: 352


🏁 Script executed:

# Check if there's any bundleId mapping or compatibility layer
rg -n "com\.app\." packages/@ant/computer-use-mcp/src/ --type ts

# Check if AppGrant persistence has any versioning or migration handling
rg -n "AppGrant|allowedApps" packages/@ant/computer-use-mcp/src/types.ts -A 3 | head -40

# Look at the actual grant storage and restoration
rg -n "localStorage|sessionStorage|store|persist|allowedApps" src/utils/computerUse/ --type tsx

Repository: claude-code-best/claude-code

Length of output: 1354


🏁 Script executed:

# Check grant storage in wrapper without tsx restriction
rg -n "allowedApps|AppGrant" src/utils/computerUse/wrapper.tsx -A 2 | head -50

# Look for any migration or version handling
rg -n "version|migrate|compat.*bundle" packages/@ant/computer-use-mcp/src/ --type ts

# Check if there's any bundleId normalization or mapping
rg -n "bundleId.*=>|map.*bundleId" packages/@ant/computer-use-mcp/src/toolCalls.ts | head -20

Repository: claude-code-best/claude-code

Length of output: 2543


Bundle ID format change breaks existing persisted grants without migration.

This implementation switches from synthesized bundle IDs (com.app.slack, com.app.terminal) in commit e3264a1 to real Spotlight bundle IDs (com.tinyspeck.slackmacgap, com.apple.Terminal). Existing AppGrant records persisted in session state store the old format. When resolveRequestedApps() at toolCalls.ts:1109 builds alreadyGranted Set from persisted grants, the bundleId lookup fails—previously-granted apps appear as new requests, forcing users to re-grant each approved application.

Consider adding a migration layer to map old com.app.* patterns to real bundleIds on load, or document this as an expected one-time re-grant on upgrade.

Secondary: Line 180 splits on | with limit of 3; if displayName contains a pipe character, parsing breaks. While unlikely (pipes in app bundle names are rare), split('|') without limit or escaping the delimiter in the shell output would be safer.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/`@ant/computer-use-swift/src/backends/darwin.ts around lines 162 -
186, Persisted AppGrant bundleIds changed from synthetic "com.app.*" to real
Spotlight bundle IDs, causing resolveRequestedApps/alreadyGranted lookups to
miss existing grants; fix by adding a migration/compatibility layer that maps
legacy synthetic IDs to real bundle IDs when loading grants (or when building
the alreadyGranted Set in resolveRequestedApps), e.g. detect bundleId starting
with "com.app." and resolve it to the real bundleId by looking up the app by
displayName or path (or store both legacy and real ids in the grant) so prior
approvals are recognized; also harden darwin.ts parsing by emitting a safer
delimiter (NUL or JSON) from the shell output instead of '|' and parse using
that (or include an escaped/quoted displayName) so app names containing '|'
won't break the split.

Comment on lines +179 to +180
return text.split('\n').filter(Boolean).map(line => {
const [bundleId, displayName, path] = line.split('|', 3)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Pipe-delimited parsing may break if displayName contains |.

The shell script uses | as delimiter (echo "$bundleId|$displayName|$appPath"), but split('|', 3) will truncate displayName if the app name contains a literal | character. While rare, consider using a delimiter less likely to appear in app names (e.g., \t or null byte).

Suggested fix using tab delimiter
-  echo "$bundleId|$displayName|$appPath"
+  printf '%s\t%s\t%s\n' "$bundleId" "$displayName" "$appPath"
...
-        const [bundleId, displayName, path] = line.split('|', 3)
+        const [bundleId, displayName, path] = line.split('\t', 3)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/`@ant/computer-use-swift/src/backends/darwin.ts around lines 179 -
180, The current parser uses '|' as delimiter which can break when displayName
contains '|' — update both the producer and consumer to use a safer delimiter
(e.g., tab or NUL). Change the shell-side echo that builds
"bundleId|displayName|appPath" to use "\t" (or '\0') and then update the parser
in the map callback that destructures [bundleId, displayName, path] to split on
the same delimiter (e.g., split('\t', 3) or use a NUL-aware split) so
displayName is not truncated; ensure filtering and mapping still handle empty
lines and maintain the existing variable names (bundleId, displayName, path).

Comment on lines +49 to 56
const cu = requireComputerUseSwift() as any
// Native .node module exposes tcc; cross-platform JS backend does not.
if (!cu.tcc) return { granted: true }
const accessibility = cu.tcc.checkAccessibility()
const screenRecording = cu.tcc.checkScreenRecording()
return accessibility && screenRecording
? { granted: true }
: { granted: false, accessibility, screenRecording }
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Returning { granted: true } when cu.tcc is absent bypasses the TCC security gate.

When the native module doesn't expose tcc (line 51), this returns { granted: true }, which falsely certifies that macOS Accessibility and Screen Recording permissions are verified. Per the ensureOsPermissions() contract in types.ts:228-234, { granted: true } means both permissions have been checked and confirmed.

From toolCalls.ts:3519-3540, the TCC gate uses this result to decide whether to proceed with ALL action tools (left_click, type_text, scroll, drag_mouse, etc.). A false granted: true bypasses this critical security check, allowing tools to execute without actual OS permissions — they'll fail at the OS level with opaque errors instead of showing the TCC permission panel.

The safe default when permission checking is unavailable should deny access:

Proposed fix
       const cu = requireComputerUseSwift() as any
       // Native .node module exposes tcc; cross-platform JS backend does not.
-      if (!cu.tcc) return { granted: true }
+      // When TCC checking is unavailable, require the user to go through
+      // request_access to surface the permission panel, rather than silently
+      // assuming permissions are granted.
+      if (!cu.tcc) return { granted: false, accessibility: false, screenRecording: false }
       const accessibility = cu.tcc.checkAccessibility()
       const screenRecording = cu.tcc.checkScreenRecording()
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const cu = requireComputerUseSwift() as any
// Native .node module exposes tcc; cross-platform JS backend does not.
if (!cu.tcc) return { granted: true }
const accessibility = cu.tcc.checkAccessibility()
const screenRecording = cu.tcc.checkScreenRecording()
return accessibility && screenRecording
? { granted: true }
: { granted: false, accessibility, screenRecording }
const cu = requireComputerUseSwift() as any
// Native .node module exposes tcc; cross-platform JS backend does not.
// When TCC checking is unavailable, require the user to go through
// request_access to surface the permission panel, rather than silently
// assuming permissions are granted.
if (!cu.tcc) return { granted: false, accessibility: false, screenRecording: false }
const accessibility = cu.tcc.checkAccessibility()
const screenRecording = cu.tcc.checkScreenRecording()
return accessibility && screenRecording
? { granted: true }
: { granted: false, accessibility, screenRecording }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/utils/computerUse/hostAdapter.ts` around lines 49 - 56, The current
hostAdapter.ts path returns { granted: true } when requireComputerUseSwift()
yields no cu.tcc, which incorrectly bypasses the TCC gate used by
ensureOsPermissions and toolCalls; change the branch in the
requireComputerUseSwift() handling so that when cu.tcc is absent it returns a
denied result (e.g., { granted: false, accessibility: false, screenRecording:
false }) instead of granted:true, updating the block that reads cu.tcc and the
return value to explicitly deny access so callers like
ensureOsPermissions/toolCalls treat permissions as not granted.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
src/utils/computerUse/hostAdapter.ts (1)

49-56: ⚠️ Potential issue | 🟠 Major

Avoid reporting OS permissions as granted when TCC cannot be checked.

When cu.tcc is missing, this returns { granted: true } even though no permission verification happened. That can bypass the permission gate in an unverified state.

Suggested change
-      if (!cu.tcc) return { granted: true }
+      if (!cu.tcc) {
+        return {
+          granted: false,
+          accessibility: false,
+          screenRecording: false,
+        }
+      }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/utils/computerUse/hostAdapter.ts` around lines 49 - 56, The early return
that sets permissions to granted when TCC cannot be checked is unsafe; change
the behavior in the requireComputerUseSwift branch where cu.tcc is tested (the
early-return inside hostAdapter.ts) to avoid returning granted:true. Instead
return a non-granted, explicit “unknown/not-checked” result (for example {
granted: false, checkNotSupported: true } or similar) so callers can treat the
permission state as unresolved and trigger a user-facing flow or fallback.
Update any callers that expect a boolean granted property to handle this
explicit unknown state.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@src/utils/computerUse/hostAdapter.ts`:
- Around line 49-56: The early return that sets permissions to granted when TCC
cannot be checked is unsafe; change the behavior in the requireComputerUseSwift
branch where cu.tcc is tested (the early-return inside hostAdapter.ts) to avoid
returning granted:true. Instead return a non-granted, explicit
“unknown/not-checked” result (for example { granted: false, checkNotSupported:
true } or similar) so callers can treat the permission state as unresolved and
trigger a user-facing flow or fallback. Update any callers that expect a boolean
granted property to handle this explicit unknown state.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1bafb9a6-6214-4b8c-826e-f5d36fad3a20

📥 Commits

Reviewing files that changed from the base of the PR and between ad1f90a and 765569b.

📒 Files selected for processing (3)
  • README.md
  • docs/features/computer-use.md
  • src/utils/computerUse/hostAdapter.ts
✅ Files skipped from review due to trivial changes (1)
  • README.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant