OpenClaw CloudPhone is a plugin that gives AI agents device management and UI automation capabilities for cloud phones.
With natural language instructions, an agent can list devices, power them on or off, capture screenshots, tap, swipe, type text, and perform other UI actions without writing manual scripts.
Starting from v2026.3.27, the package ships with built-in skills (including basic-skill) that help agents combine these tools in a more reliable way.
openclaw plugins install @whateverai/cloudphoneTo update the plugin later, run:
openclaw plugins update @whateverai/cloudphoneFor most setups, you only need to set apikey in plugins.entries.cloudphone.config. The plugin applies built-in defaults for other optional settings. Advanced users can still add optional keys such as baseUrl or timeout when self-hosting or tuning behavior; see openclaw.plugin.json in this package for the full schema.
You can configure the plugin in either of the following ways.
Add the following configuration to openclaw.json:
- apikey: Obtain your API Key by logging in or signing up at https://whateverai.ai, then add it in your account/settings and paste it into the
apikeyfield below.
{
"plugins": {
"entries": {
"cloudphone": {
"enabled": true,
"config": {
"apikey": "the apikey you can get from the user center of this website"
}
}
}
}
}You can also configure the CloudPhone plugin in the OpenClaw console UI:
- Open the OpenClaw console in your browser.
- Go to the Plugins section, find CloudPhone and enable it.
- Set apikey (from https://whateverai.ai after login or sign-up, in your account/settings).
Screenshots:
openclaw gateway restartOnce the plugin is loaded successfully, the agent can use all CloudPhone tools. If the plugin is enabled correctly, the bundled basic-skill skill will also become available.
This repository is first and foremost an OpenClaw plugin. Its job is to expose the CloudPhone OpenAPI as tools that an agent can call.
Starting from v2026.3.27, the package includes OpenClaw skills:
- Plugin: defines what the agent can do by providing
cloudphone_*tools - Skill: defines how the agent should do it reliably by teaching call order, recovery steps, and safer workflows
Together they form a complete automation loop:
- The plugin provides the low-level capabilities such as device management, UI interaction, and screenshot capture
- The skill helps the agent chain those capabilities into a stable multi-step workflow
The package includes the basic-skill skill under:
skills/basic-skill/
It contains:
SKILL.md: the main guide that defines scenarios, standard workflows, recovery strategies, and capability boundariesreference.md: a quick reference for the 14 available tools and their parameters
The skill does not add new API capabilities and does not require an extra install step. It only helps the agent use the existing tools more effectively.
basic-skill mainly improves the following areas:
- Installation and troubleshooting: checking
openclaw.jsonandapikey - Standard workflow: select device -> confirm online -> observe -> act -> verify
- UI automation stability: using short loops such as observe -> act -> verify -> observe again
- Recovery strategy: prefer
BACK,HOME, and fresh screenshots; restart the device only when needed
The current skill is built on top of the existing plugin toolset, so it does not automatically provide these higher-level capabilities:
- OCR
- Find UI controls by text
- Click controls directly by selector
- Launch an app by package name
- Complex macro recording and playback
If you need those capabilities, extend the plugin itself instead of changing only the skill.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
apikey |
string | Yes | - | Authorization credential (ApiKey) |
Obtain your API Key by logging in or signing up at https://whateverai.ai, then find it in your account/settings.
Optional fields such as baseUrl and timeout are documented in openclaw.plugin.json and use built-in defaults when omitted; set them only for custom deployments or advanced tuning.
After the plugin is installed, the agent automatically gets the following capabilities.
| Tool | Description |
|---|---|
cloudphone_get_user_profile |
Get the current user's basic information |
cloudphone_list_devices |
List cloud phone devices with pagination, keyword search, and status filters |
cloudphone_get_device_info |
Get detailed information for a specific device |
cloudphone_device_power |
Control device power: start, stop, or restart |
cloudphone_get_adb_connection |
Get ADB/SSH connection information for a device |
| Tool | Description |
|---|---|
cloudphone_tap |
Tap a specific screen coordinate |
cloudphone_long_press |
Long press a coordinate with an optional duration |
cloudphone_swipe |
Swipe from a start point to an end point |
cloudphone_input_text |
Type text into the current input field |
cloudphone_clear_text |
Clear the current input field |
cloudphone_keyevent |
Send system keys such as back, home, enter, recent apps, or power |
| Tool | Description |
|---|---|
cloudphone_wait |
Wait for a condition such as element appear/disappear or page stability |
cloudphone_snapshot |
Capture a device screenshot |
cloudphone_render_image |
Render a screenshot URL as an image directly in chat |
planActionTool maps to cloudphone_plan_action. It lets the agent call an AutoGLM model to analyze the current screenshot and goal, then return a structured next-action plan for CloudPhone UI automation.
Typical scenarios:
- uncertain next step on a dynamic UI
- deciding tap/swipe/input intent before execution
- recovering when repeated direct actions fail
Configure these plugin fields before using cloudphone_plan_action:
- required:
autoglmBaseUrl,autoglmApiKey,autoglmModel - optional:
autoglmMaxTokens(default3000),autoglmLang(defaultcn)
Example (plugins.entries.cloudphone.config):
{
"autoglmBaseUrl": "https://open.bigmodel.cn/api/paas/v4",
"autoglmApiKey": "your-api-key",
"autoglmModel": "autoglm-phone",
"autoglmMaxTokens": 3000,
"autoglmLang": "cn"
}Core input:
device_id: target cloud phone device IDgoal: natural language task goal
Minimal example:
device_id: "your-device-id"
goal: "Open WeChat and enter the search page"
Expected output:
- model reasoning summary for the current screen
- a suggested next action that can be executed with
cloudphone_*tools
- If required
autoglm*fields are missing, the tool returns a config error. - Recommended flow:
cloudphone_snapshot->cloudphone_plan_action-> execute withcloudphone_tap/cloudphone_swipe/cloudphone_input_text-> verify with new snapshot. - Keep each goal focused to one immediate UI objective for better planning quality.
After installation and configuration, you can control cloud phones through natural language prompts.
Show me my cloud phone devices
The agent will call cloudphone_list_devices and return the matching devices.
Power on my cloud phone and show me the current screen
The agent will typically call cloudphone_device_power -> cloudphone_snapshot -> cloudphone_render_image.
Open WeChat on the cloud phone, search for the "OpenClaw" public account, and follow it
The agent can combine the plugin tools with the bundled skill to plan the task and execute it using the safer pattern of observe first, then act, then verify.
Give me the ADB connection info for this cloud phone
The agent will call cloudphone_get_adb_connection and return the host and port.
keyword : string - Search keyword (device name or device ID)
status : string - Status filter: "online" | "offline"
page : integer - Page number, default 1
size : integer - Items per page, default 20
user_device_id : number - User device ID (required)
device_id : string - Device ID (required)
action : string - Action: "start" | "stop" | "restart" (required)
device_id : string - Device ID (required)
x : integer - X coordinate in pixels (required)
y : integer - Y coordinate in pixels (required)
device_id : string - Device ID (required)
x : integer - X coordinate in pixels (required)
y : integer - Y coordinate in pixels (required)
duration : integer - Press duration in milliseconds, default 1000
device_id : string - Device ID (required)
start_x : integer - Start X coordinate (required)
start_y : integer - Start Y coordinate (required)
end_x : integer - End X coordinate (required)
end_y : integer - End Y coordinate (required)
duration : integer - Swipe duration in milliseconds, default 300
device_id : string - Device ID (required)
text : string - Text to input (required)
device_id : string - Device ID (required)
key_code : string - Key code: "BACK" | "HOME" | "ENTER" | "RECENT" | "POWER" (required)
device_id : string - Device ID (required)
condition : string - Wait condition: "element_appear" | "element_disappear" | "page_stable" (required)
timeout : integer - Timeout in milliseconds, default 5000
selector : string - Element selector used with appear/disappear conditions
device_id : string - Device ID (required)
format : string - Snapshot format: "screenshot" (currently only screenshot is supported)
image_url : string - HTTPS image URL (required)
Q: The agent cannot find the CloudPhone tools after installation.
Make sure plugins.entries.cloudphone.enabled is set to true in openclaw.json, then restart the Gateway.
Q: The tools work, but the agent is not very stable when operating a cloud phone UI.
Starting from v2026.3.27, the package ships with built-in skills such as basic-skill. They teach the agent to use the tools in a short loop: observe -> act -> verify -> observe again. Make sure you installed a recent version and restarted the Gateway so the latest skills were loaded.
Q: A tool call fails with a request error or timeout.
- Check whether
apikeyis valid and that you restarted the Gateway after changing config - Check network connectivity and whether the CloudPhone service is reachable
- If you use a custom deployment or endpoint, verify routing and availability on your side
Q: How do I get an apikey?
Log in or sign up at https://whateverai.ai and get your API Key from your account/settings.
Q: cloudphone_snapshot returned a URL, but I cannot see the image in chat.
The agent should call cloudphone_render_image automatically to turn that URL into a displayable image. The current version returns an MCP image content block first and also keeps a fallback MEDIA:<filePath> text item for older hosts. If the image still does not appear, ask the agent to show the screenshot explicitly; if that still fails, the current host likely does not consume type: "image" content items yet.
Current version: v2026.3.27
- Summarized and aligned release notes based on target commit
1da1031 - Synced package/plugin/doc version references to
v2026.3.27 - Kept English and Chinese changelog/version labels consistent
- Fixed leftover version wording in README sections that still referenced
v1.1.0 - Synced release-related version identifiers to
v2026.3.26.1 - Updated English and Chinese changelog/version labels consistently
- Added verbose step-by-step logs for cloudphone_plan_action to improve debugging and failure tracing
- Expanded planActionTool documentation with prerequisites, usage flow, and safety notes in both English and Chinese README
- Synced built-in skills wording and release docs to align with the current v1.1.0+ behavior
- Enhanced screenshot handling in
cloudphone_render_imagefor improved compatibility and display reliability across hosts - Added the
cloudphone-snapshot-urlskill and alignedbasic-skillguides/reference for screenshot URL workflows - Synced screenshot-related tool docs and skill guidance in both English and Chinese content
- Simplified plugin configuration documentation: typical users only need
apikey; optionalbaseUrlandtimeoutremain inopenclaw.plugin.jsonwith built-in defaults - Updated
basic-skillskill preconditions and troubleshooting to match the streamlined config guidance - Synced English and Chinese README and changelog wording
- Revised
cloudphone_snapshotdocs to clarify that it captures screenshots only - Updated the
formatparameter description to indicate onlyscreenshotis supported - Synced related descriptions in English and Chinese README and tool reference docs
- Aligned tool overview table rows for
cloudphone_snapshotwith the parameter documentation
- Added the built-in
basic-skillskill distributed with the plugin - Added
reference.mdas a tool parameter quick reference - Expanded the documentation for plugin vs. skill responsibilities, standard workflows, and capability boundaries
This plugin follows the license terms of the repository it belongs to.