Mobile app testing with AI agents

Mobile app testing with AI agents works when the agent drives a real simulator or emulator, not just source code. The usual setup is Maestro MCP for device access, Maestro flows or Detox tests written against stable accessibility IDs, and a rule that a test is accepted only after it fails on a deliberately broken build; a small device matrix gates every pull request.

Your React Native app works on the iPhone on your desk. Then the crash reports arrive: an older Android phone from a manufacturer with a custom UI layer, a small-screen iPhone where the checkout button sits under the keyboard, and a tablet layout where labels overlap. You asked an agent for “E2E tests” last month and got 40 green tests. None of them caught any of this. This page is for the developer who owns a React Native, Flutter or native app and the tech lead who decides what runs on each pull request.

What this guide gives you

• A decision table for which framework the agent should write: Maestro, Detox, Flutter integration_test or Patrol, or native XCUITest and Espresso
• Maestro MCP and MobileBuildMCP set up in Claude Code, Codex or Cursor, so the agent checks its work on a device
• Five copy-paste prompts: a Maestro smoke flow, a Detox permissions suite, a negative-control audit, a device matrix, and offline tests
• A verification routine that proves a generated test catches bugs, without reading every line of it
• A tiered device matrix and the CI commands that run it

Which mobile test framework should the agent write?

Pick the framework before you prompt. An agent asked for “mobile tests” picks for you, and it tends to default to the best-known framework, not the one that fits your app.

Framework	Best for	What the agent writes	Limits to know (checked 2026-09-26)
Maestro 2.11.0	Smoke and regression flows on any stack: React Native, Flutter, native, hybrid	Flat YAML (tapOn, assertVisible), readable by QA and reviewers	Physical iOS devices are not supported; setAirplaneMode is Android-only
Detox 20.51.4 (npm)	React Native apps that need code-level control and app-idle synchronization	TypeScript on Jest	React Native only; launchApp permissions are iOS-only
Flutter integration_test / Patrol 4.10.0 (pub.dev)	Flutter apps; Patrol adds native dialogs and system UI	Dart	integration_test alone cannot tap native permission dialogs
XCUITest / Espresso	Native apps, and real-device runs on device farms	Swift / Kotlin	Two suites to maintain; slowest for the agent to iterate on

For most cross-platform teams the answer is Maestro for the smoke suite and one code-level framework for the critical paths. The YAML flows are short enough that a reviewer can check what they assert in a minute, and that matters more than any framework feature when an agent writes the tests. Stack-specific build setup lives in the cookbook: React Native, Flutter, Expo and native iOS and Android.

Give the agent a device, not only the source code

An agent that only reads your navigation files guesses at selectors, timing and what is on screen. An agent with a device can look. Two MCP servers do most of the work:

• Maestro MCP ships inside the Maestro CLI (maestro mcp, nothing else to install). The agent inspects the screen, taps, scrolls and asserts on an emulator, simulator or physical Android device, then writes the steps down as a YAML flow. Maestro 2.5.0 dropped eight redundant tools and folded the run tools into one, so the server’s tool list stays short (checked against Maestro 2.11.0).
• MobileBuildMCP (formerly XcodeBuildMCP, owned by Sentry, npm mobilebuildmcp 2.7.1) builds iOS and macOS apps, boots simulators and runs XCTest. It needs macOS 14.5+ and Xcode 16+.

Install Maestro first (it needs Java 17 or later), then register the servers:

curl -fsSL "https://get.maestro.mobile.dev" | bash

Claude Code

claude mcp add maestro -- maestro mcp
claude mcp add mobilebuild -- npx -y mobilebuildmcp@latest mcp   # macOS only

Add -s project to write the entry to .mcp.json so the whole team gets it.

Codex

codex mcp add maestro -- maestro mcp
codex mcp add mobilebuild -- npx -y mobilebuildmcp@latest mcp   # macOS only

Cursor

Add both servers to .cursor/mcp.json in the project (or ~/.cursor/mcp.json for every project). MobileBuildMCP runs on macOS only.

{
  "mcpServers": {
    "maestro": { "command": "maestro", "args": ["mcp"] },
    "mobilebuild": { "command": "npx", "args": ["-y", "mobilebuildmcp@latest", "mcp"] }
  }
}

Both commands use each server’s documented launch command (maestro mcp, npx -y mobilebuildmcp@latest mcp). After adding them, run /mcp in Claude Code or codex mcp list to confirm the server connects. For the full server catalogue, see dev-server MCPs.

The workflow: explore on a device, write the test, prove it can fail

1. Add stable selectors first. Ask the agent to add testID (React Native), Key or Semantics identifiers (Flutter), or accessibilityIdentifier / testTag (native) to every element a flow touches. Tests that match on visible text break on the first copy change or translation.

2. Let the agent walk the flow on a booted device. With Maestro MCP connected, the agent taps through the real app and records what it sees. This replaces the guesswork of reading src/navigation/.

3. Have it write the test and run it until green. The flow or spec lands in the repository, so it becomes a deterministic test in CI rather than a one-off agent session.

4. Break the app and watch the test fail. This is the negative control. A test that stays green when the button it asserts on is removed protects nothing. See how strong is your oracle for the full method.

5. Wire the suite into the pull-request gate on one iOS simulator and one Android emulator, with failure artifacts saved.

The first prompt covers steps 1 to 3 for a smoke flow.

Copy-paste prompt: Maestro smoke flow explored on a live device

Use the Maestro MCP server. A simulator or emulator is already booted with our
debug build installed (app id com.yourcompany.app).

1. Walk the checkout happy path on the device: sign in with qa@example.com /
   the password in the E2E_PASSWORD environment variable, open the first product
   on the home feed, tap "Add to cart", open the cart, tap "Checkout", pay with
   test card 4242 4242 4242 4242 (any future expiry, any CVC).
2. Every element you touch must have a stable id. Where one is missing, add a
   testID in the React Native component and tell me which files you changed.
3. Write the path as .maestro/checkout-smoke.yaml: launchApp with clearState,
   select by id (fall back to text only for system UI), tag the flow "smoke",
   end with assertVisible on the order-confirmation id and takeScreenshot.
4. Keep it under 30 commands. No fixed sleeps; use extendedWaitUntil if a
   screen needs time.
5. Run it with `maestro test .maestro/checkout-smoke.yaml` until it passes
   twice in a row, then show me the final YAML and both run results.

For React Native flows that need permission handling and precise synchronization, Detox is the better fit. Its device.launchApp({ permissions }) grants or denies runtime permissions before launch, but only on iOS, so the prompt spells out the Android path.

Copy-paste prompt: Detox onboarding suite with permission paths

Write a Detox test suite for our React Native onboarding flow in
e2e/onboarding.e2e.ts. Flow: Splash -> Welcome carousel (3 pages) ->
Notification permission -> Profile setup -> Home.

Cases:
1. Permission granted: iOS uses
   device.launchApp({ newInstance: true, permissions: { notifications: 'YES' } }).
2. Permission denied: iOS uses notifications: 'NO'. The Home screen must show
   the "Turn on notifications" banner (testID notif-banner).
3. Android: permissions cannot be preset through launchApp, so grant or revoke
   with adb shell pm grant / pm revoke com.yourcompany.app
   android.permission.POST_NOTIFICATIONS in a beforeAll, and document why.
4. Back navigation from every step returns to the previous screen.

Rules: match by testID only, no setTimeout or sleep (rely on Detox
synchronization; if a screen never goes idle, find the looping animation or
timer and tell me instead of disabling synchronization). Put shared login and
permission helpers in e2e/utils/. Run
`detox test -c ios.sim.debug e2e/onboarding.e2e.ts` and fix failures in the
tests, not in app logic, unless you show me the app bug first.

Where each tool runs the device loop

The device loop needs a machine with simulators: iOS simulators run only on macOS with Xcode, and Android emulators need hardware acceleration. That decides where each agent does the work.

Claude Code

Iterate interactively on your Mac with Maestro MCP connected. In CI, keep the test run deterministic and call the agent only to diagnose a failure, with a narrow tool allowlist. claude -p starts in Manual permission mode (auto mode on third-party providers or with telemetry off, from v2.1.285, where a classifier decides) and cannot ask for approval, so add --permission-mode dontAsk to deny any tool call that would prompt on every provider (file reads in the working directories and read-only Bash commands still run):

claude -p "Read build/maestro/report.xml and the screenshots in build/maestro/. \
Explain why each failed flow failed and propose the smallest fix. Do not edit files." \
  --permission-mode dontAsk \
  --allowedTools "Read" "Glob" "Bash(maestro test *)" --output-format json

Codex

Iterate in the CLI or IDE extension on a machine with a booted device. For a non-interactive fix attempt, run codex exec with the workspace-write sandbox. Workspace-write blocks network access by default, which can cut the agent off from adb or the simulator. Allow it for this run with -c sandbox_workspace_write.network_access=true, or run the device step outside the agent and pass it the JUnit report instead:

codex exec --sandbox workspace-write \
  "Read build/maestro/report.xml, fix the failing Maestro flows under .maestro/, and list what you changed."

Codex cloud tasks suit writing selectors and flows from the code, but check that your cloud environment actually provides a simulator before you ask a cloud task to run the suite.

Cursor

Run the agent locally in the editor with Maestro MCP connected and a simulator open beside it, so you watch each tap as it happens. Cloud Agents run in remote VMs; use them for selector and flow writing, and leave device runs to your macOS machine or CI runner unless the Cloud Agent environment is set up with an emulator.

The CI job itself runs Maestro directly, tagged and reported:

# CI: the pull-request smoke gate
maestro test --include-tags smoke --format junit --output build/maestro/report.xml \
  --test-output-dir build/maestro .maestro/

For Detox, detox test -c android.emu.release --take-screenshots failing --record-videos failing saves evidence only for the failures. Leave --retries off in the gate: a retry hides flakiness that you want to measure.

How do you know the agent’s mobile tests catch bugs?

A green run proves only that the test and the app agree. It does not prove the test would notice a regression. Check four things instead of reading every assertion:

Check	How	Passes when
Negative control	Break the behaviour the test claims to cover (hide the button, return an error from the API mock) and re-run	Every test in the change goes red for the right reason
Selector hygiene	grep the new flows and specs for text selectors, fixed sleeps and disableSynchronization	None without a comment explaining why
Flake rate	Run the new suite 10 times on the same device with retries off	10 of 10 pass; anything less goes to quarantine, not to the gate
Evidence	Screenshots and videos for failures, the JUnit report in the PR	A reviewer can see what happened on the device without re-running it

Copy-paste prompt: negative-control audit for new mobile tests

For every Maestro flow and Detox test added or changed on this branch
(git diff main --name-only), do the following on the booted simulator:

1. Name the one user-visible behaviour the test protects.
2. Make the smallest app change that breaks that behaviour (remove the
   element, return a 500 from the mock server, skip the navigation call).
3. Run only that test. Record whether it failed, and whether the failure
   message points at the broken behaviour.
4. Revert the app change with git checkout -- <file> and confirm the test
   is green again.

Output a table: test, behaviour, mutation, failed (yes/no), message useful
(yes/no). Do not modify any test. Any test that stayed green is the finding.

Who signs off. The developer attaches the negative-control table and the JUnit report to the pull request. The tech lead owns two lists that the agent must not edit on its own: the smoke tags that gate a merge and the device matrix. Put .maestro/, e2e/ and the CI workflow under CODEOWNERS so a changed or deleted test needs a human approval; protecting the oracle covers the hooks and rules that enforce this.

How many devices do you need to test on?

You cannot test on every device, and each extra device adds run time faster than it adds coverage, so tier the matrix by how often it runs:

Tier	Runs	Devices	Purpose
PR gate	Every pull request	1 iOS simulator (current iOS), 1 Android emulator (your lowest supported API level)	Smoke flows only; minutes, not hours
Nightly	Once a day	3 to 6 simulators and emulators: smallest and largest screen, a tablet, oldest supported OS	Full regression, layouts, orientation
Pre-release	Before each store submission	Real devices on a device farm (BrowserStack, AWS Device Farm), top manufacturers from your analytics	Custom OEM dialogs, performance, real iPhones

Real iPhones matter in the last tier because Maestro 2.11.0 does not run on physical iOS devices; use XCUITest or Appium (npm appium 3.8.0) on the device farm for those.

Copy-paste prompt: device matrix from your own analytics

Design our mobile test device matrix from the attached analytics export
(analytics/devices-last-90-days.csv: platform, model, os_version, sessions,
crash_count).

Produce three tiers:
1. PR gate: exactly one iOS simulator and one Android emulator.
2. Nightly: at most six simulator/emulator configurations.
3. Pre-release: at most eight real devices for a device farm.

Selection rules: cover the lowest OS version we still support on each platform,
the smallest and largest screen, at least one tablet if tablets exceed 3% of
sessions, and the manufacturers with the highest crash_count per session, not
only the most sessions. For each device, give one sentence on which risk it
covers. Output the tiers as a table, then as the CI matrix YAML for
.github/workflows/mobile-e2e.yml (matrix section only).

How do you test offline and slow-network behaviour?

Neither framework has a single network-conditioning switch that works on both platforms, so drive connectivity from outside the test:

• Android airplane mode: Maestro’s setAirplaneMode and toggleAirplaneMode commands work on Android only, because of iOS simulator restrictions. adb shell svc wifi disable and adb shell svc data disable do the same from a script.
• Blocked or failing requests: point the app at a mock server and inject latency and 5xx responses there. Detox’s device.setURLBlacklist([...]) stops it waiting on analytics or polling URLs that never go idle.
• Simulated offline on iOS: pass a launch argument (launchArgs: { offlineMode: true } in Detox, or an env var in the Maestro flow) that the app reads to fake a disconnected network state in debug builds only.

Copy-paste prompt: offline and reconnect tests

Write Maestro flows under .maestro/offline/ for our React Native app, tagged
"nightly", running on the Android emulator:

1. Offline banner: load the feed online, setAirplaneMode: enabled,
   assertVisible the offline banner (id offline-banner), setAirplaneMode:
   disabled, assert the banner disappears within 10 seconds.
2. Queued action: go offline, add an item to the cart, go online, and
   assert the cart badge shows exactly 1 item (no duplicate submission).
3. Cached data: after one online launch, go offline, relaunch the app
   without clearState, assert the last-loaded feed items are visible.

For iOS, where airplane mode cannot be toggled, write one Detox test that
launches with launchArgs { offlineMode: true } and asserts the same banner.
Tell me which app code reads offlineMode and confirm it is compiled out of
release builds.

Gesture, orientation and layout checks follow the same pattern: Maestro has swipe and setOrientation, Detox has device.setOrientation(), element-level swipe() and an iOS-only pinch(). For layout regressions that assertions cannot describe well, compare screenshots and use a model-graded check with a rubric rather than pixel equality. Accessibility labels, VoiceOver and TalkBack are covered in accessibility testing.

When agent-written mobile tests break down

The tests pass on the simulator and fail on real devices. Simulators do not reproduce OEM permission dialogs, real keyboards or low-memory kills. Recovery: move the affected flows into the pre-release tier on a device farm and add the OEM dialog as an optional step (optional: true in Maestro) rather than a hard assertion.

The agent “fixed” a failing test by weakening it. Typical signs are a removed assertVisible, a text selector swapped for a coordinate tap, or disableSynchronization() added to Detox. Recovery: reject the change, re-run the negative-control prompt on that test, and add the test directory to CODEOWNERS so the next weakening needs approval.

Flows are flaky because of animations. Disable animations in the app process, not in the test: Maestro’s workspace config.yaml supports disableAnimations for Android and iOS, and on Android emulators you can set the three animation scales to 0. Recovery for a flaky test that survives this: quarantine it with an exclude tag, and ask the agent to find the looping animation or timer that keeps the app from going idle.

The agent cannot see the device. maestro mcp fails to start without Java 17, and MobileBuildMCP needs macOS 14.5+ with Xcode 16+. Recovery: run maestro test on one flow by hand to confirm the device connection, then check /mcp in Claude Code or codex mcp list for the server state.

The PR gate is too slow. Mobile suites run slower than web suites, and a slow gate gets skipped. Recovery: keep the gate to the flows tagged smoke (aim for three to five critical paths), move everything else to nightly, and use --shard-split in Maestro or a device farm to parallelise the nightly run.

Where to go next with mobile testing

How strong is your oracle? Mutation score, negative controls and flake rate: the bar a suite must clear before you trust it.

Protecting the oracle Stop agents from editing or deleting the tests that gate their own work.

End-to-end test automation The web counterpart: Playwright suites for the same user journeys.

Test data strategies Seeded accounts and fixtures that keep mobile flows repeatable.

Performance testing Startup time, frame drops and memory, measured rather than guessed.

React Native development patterns Project setup and build loops that the tests on this page run against.