agent-mobile

AI agent skill for automating mobile apps on iOS simulators and Android emulators via Maestro.

Works with any framework: native iOS/Android, React Native, Flutter, etc.

Installation

Install the skill with the Vercel Labs skills CLI:

npx skills@latest add jeanpfs/agent-mobile --all -g

Install the CLI from npm:

npm install -g @jeanpfs/agent-mobile

Or install the latest unreleased version directly from GitHub (requires a local build step, since the published dist is not committed):

git clone https://github.com/jeanpfs/agent-mobile.git
cd agent-mobile
npm install && npm run build
npm install -g .

Requirements

• Node.js >= 18
• Maestro CLI — mobile UI automation framework
• iOS Simulator or Android Emulator with a running app

Install Maestro

Maestro requires Java (JDK 8+).

curl -Ls install.maestro.dev | bash

The installer adds Maestro to ~/.maestro/bin. You need to add it to your PATH:

Fish:

fish_add_path $HOME/.maestro/bin

Add to ~/.config/fish/config.fish to persist.

Bash:

export PATH="$PATH:$HOME/.maestro/bin"

Add to ~/.bashrc or ~/.bash_profile to persist.

Zsh:

export PATH="$PATH:$HOME/.maestro/bin"

Add to ~/.zshrc to persist.

After configuring, verify:

maestro --version

Commands

Command	Description
agent-mobile setup	Check environment (Maestro, devices)
agent-mobile snapshot	Capture accessibility tree with refs
agent-mobile tap <ref>	Tap element by ref (e.g., m3)
agent-mobile type <ref> "<text>"	Type text into field by ref
agent-mobile scroll <direction>	Scroll: up, down, left, right
agent-mobile screenshot	Save screenshot as PNG
agent-mobile assert "<text>"	Verify text exists on screen
agent-mobile logs start	Start capturing device logs in background
agent-mobile logs stop	Stop capture and display captured logs

Quick Start

1. Start a simulator with your app running

iOS:

open -a Simulator

Android:

emulator -avd <your_avd_name>

2. Verify environment

agent-mobile setup

Expected output:

Maestro: x.x.x
Platform: ios
Devices: iPhone 16
Status: READY

3. Take a snapshot

agent-mobile snapshot

Output:

Screen: (7 elements)

- header "Welcome" [ref=m1]
- textbox "Email" [ref=m2]
- textbox "Password" [ref=m3]
- button "Sign In" [ref=m4]
- link "Forgot password?" [ref=m5]

4. Interact with elements

agent-mobile type m2 "user@example.com"
agent-mobile type m3 "password123"
agent-mobile tap m4

5. Validate result

agent-mobile snapshot
agent-mobile assert "Welcome"

How It Works

1. snapshot runs maestro hierarchy to get the accessibility tree as JSON
2. Parses the tree, normalizes roles (button, textbox, link, etc.)
3. Assigns sequential refs (m1, m2, m3...) to interactive elements
4. tap/type resolve the ref back to the element, generate a temporary Maestro YAML flow, and execute it
5. assert scans all visible text in the tree for a match
6. logs start/stop runs adb logcat (Android) or xcrun simctl spawn booted log stream (iOS) in background, captures output to a temp file, and displays it on stop

Refs are ephemeral — valid only for the snapshot that generated them. Always re-snapshot after any action.

Error Codes

Code	Meaning	Solution
NO_MAESTRO	Maestro not installed	curl -Ls install.maestro.dev | bash
NO_DEVICE	No simulator/emulator running	Start one with your app
INVALID_REF	Ref not in current UI	Run snapshot again
TIMEOUT	Action timed out	Retry or check app state
TAP_FAILED	Tap failed	Element may need scroll
TYPE_FAILED	Type failed	Check ref is a text input
LOGS_ALREADY_RUNNING	Log capture already active	Run logs stop first
LOGS_NOT_RUNNING	No active log capture	Run logs start first

Install from skills.sh

npx skills@latest add jeanpfs/agent-mobile --all -g

License

MIT