MCP Server based on MaaFramework Providing Android device and Windows desktop automation capabilities for AI assistants
English | 中文
MaaMCP is a Model Context Protocol server that exposes MaaFramework's powerful automation capabilities through standardized MCP interfaces to AI assistants (like Claude). With this server, AI assistants can:
- 🤖 Android Automation - Connect and control Android devices/emulators via ADB
- 🖥️ Windows Automation - Control Windows desktop applications
- 🎯 Background Operation - Screenshots and controls on Windows run in the background without occupying your mouse or keyboard, allowing you to continue using your computer for other tasks
- 🔗 Multi-Device Coordination - Control multiple devices/windows simultaneously for cross-device automation
- ⚡ Dual-Mode Operation - Serial mode (synchronous execution) and Pipeline mode (background continuous screenshots), adapting to different scenarios
- 👁️ Smart Recognition - Use OCR to recognize on-screen text
- 🎯 Precise Operations - Execute clicks, swipes, text input, key presses, and more
- 📸 Screenshots - Capture real-time screenshots for visual analysis
Talk is cheap, see: 🎞️ Bilibili Video Demo
find_adb_device_list- Scan available ADB devicesfind_window_list- Scan available Windows windowsconnect_adb_device- Connect to Android deviceconnect_window- Connect to Windows window
ocr- Optical Character Recognition (efficient, recommended for priority use, OCR model auto-downloads on first use)screencap- Screenshot capture (use as needed, high token cost)
Win32 and ADB connections default to target_short_side=720, preserving aspect ratio.
OCR regions and boxes, clicks, swipes, and Pipelines all use the controller's full screenshot
coordinates, which may differ from physical device coordinates. screencap now defaults to
resolution=None, preserving that image size so full-image coordinates can be used directly.
Pass a positive target_short_side, such as 1080, when connecting, or None for native size.
This changes the coordinate basis of the entire controller: templates and Pipeline ROIs must
use the corresponding scale. Keep the default for existing 720p resources. Callers that rely
on the previous Win32 1080p coordinates can explicitly select 1080.
screencap still returns a path string by default. When locating targets in cropped or explicitly
resized images, pass include_metadata=True to receive:
| Field | Meaning |
|---|---|
path |
Absolute image path, also accepted by save_captured_image |
controller_id / coordinate_system |
Owning controller and coordinate space (controller) |
coordinate_size |
Full controller screenshot [width, height] |
image_size |
Saved image [width, height] |
image_to_controller.scale |
Image-to-controller scale [sx, sy] |
image_to_controller.offset |
Actual crop origin in controller coordinates [ox, oy] |
Map an image point (u, v) to (round(u*sx+ox), round(v*sy+oy)) before clicking.
The transform includes actual resized dimensions, rounding, and crop boundary clamping. It only
describes this capture; take another screenshot after the window size or layout changes.
Explicit resolution only affects the saved image, not controller coordinates, and cannot
restore detail lost in an earlier downsampling step.
click- Click at coordinates (supports multi-touch/mouse button selection, long press)- On Windows, supports mouse button selection: left, right, middle
double_click- Double click at coordinatesswipe- Swipe gesture (preferred for scrolling/paging on Android)input_text- Input textclick_key- Key press (supports long press)- On Android, simulates system keys: Back(4), Home(3), Menu(82), Volume keys, etc.
- On Windows, supports virtual key codes: Enter(13), ESC(27), Arrow keys, etc.
keyboard_shortcut- Keyboard shortcuts- Supports key combinations: Ctrl+C, Ctrl+V, Alt+Tab, etc.
scroll- Mouse wheel (Windows only)
start_pipeline- Start background monitoring pipeline, continuously captures screenshots and caches image pathsstop_pipeline- Stop pipelineget_new_messages- Get new screenshot paths cached by pipelineget_pipeline_status- Get pipeline running status
get_pipeline_protocol- Get Pipeline protocol documentationsave_pipeline- Save Pipeline JSON to file (supports creating and updating)load_pipeline- Load an existing Pipeline filerun_pipeline- Run Pipeline and return execution results (single/multi-file, auto-start Custom action agent, per-runpipeline_override, tool-sidetimeout_seconds)
stop_pipeline- Atomically stop Tasker / Agent subprocess / OCR loop to prevent orphan processes
pip install maa-mcp-
Clone the repository
git clone https://github.com/MistEO/MaaMCP.git cd MaaMCP -
Install Python dependencies
pip install -e .
Add an MCP server to software such as Cursor:
{
"mcpServers": {
"MaaMCP": {
"command": "maa-mcp"
}
}
}OR
Add an MCP command to software such as Cherry Studio:
maa-mcpAfter configuration, you can use it:
Android Automation Example:
Please use the MaaMCP tools to connect to my Android device, open Meituan, and help me order a Chinese meal (one portion) around 20 RMB.
Windows Automation Example:
Please use the MaaMCP tools to show me how to add a rotation animation effect to the current PPT slide, and demonstrate the steps.
Pipeline Generation Example:
Please use MaaMCP tools to connect to my device, help me open Settings, go to Display settings, and adjust brightness to 50%.
After completing the operations, generate a Pipeline JSON for this workflow so it can be run directly later.
MaaMCP will automatically:
- Scan available devices/windows
- Establish connection
- Auto-download and load OCR resources (on first use)
- Execute recognition and operation tasks
MaaMCP follows a streamlined operational workflow with multi-device/window coordination support and two operation modes:
graph LR
A[Scan Devices] --> B[Establish Connection]
B --> C1[Serial Mode]
B --> C2[Pipeline Mode]
C1 --> D[Execute Automation]
C2 --> D
- Scan - Use
find_adb_device_listorfind_window_list - Connect - Use
connect_adb_deviceorconnect_window(can connect multiple devices/windows, each gets a unique controller ID) - Operate - Execute OCR, click, swipe, etc. on multiple devices/windows by specifying different controller IDs (OCR resources auto-download on first use)
MaaMCP supports two operation modes, allowing flexible selection based on task requirements:
Traditional synchronous execution, completing one instruction before executing the next:
OCR Recognition → Analyze Results → Execute Operation → OCR Recognition → ...
Suitable for: Simple tasks, scenarios with low real-time requirements
Multi-threaded asynchronous execution, background thread continuously captures screen information while main thread focuses on decision-making and operations:
graph LR
subgraph Background Thread
S1[Continuous Screenshots] --> S2[Cache Image Paths]
S2 --> S3[Push to Message Queue]
S3 --> S1
end
subgraph Main Thread
M1[Get Screenshot Path] --> M2[Visual Analysis]
M2 --> M3[Decide if OCR Needed]
M3 --> M4[Execute Operation]
M4 --> M1
end
Workflow:
- Start Pipeline - Call
start_pipeline(controller_id)to start background monitoring - Get Screenshots - Call
get_pipeline_status()to check status,get_new_messages()to get screenshot paths - Analyze & Execute - Read images for visual analysis, call OCR as needed, execute clicks and other operations
- Stop Pipeline - Call
stop_pipeline()to release resources when task is complete
Advantages:
- Continuous background screenshots, AI can directly view complete screen for decision-making
- AI can decide whether OCR is needed and which specific regions to OCR based on image content
- Supports high-frequency screen monitoring, never misses interface changes
- Suitable for real-time automation tasks requiring fast response
- Message queue mechanism supports asynchronous processing
Usage Example:
Please use MaaMCP tools to connect to my device, use pipeline mode to monitor the screen, and automatically click confirm when a specific popup appears.
MaaMCP supports AI converting executed operations into MaaFramework Pipeline JSON format, enabling operate once, reuse infinitely.
graph LR
A[AI Executes Operations] --> B[Operations Completed]
B --> C[AI Reads Pipeline Docs]
C --> D[AI Intelligently Generates Pipeline]
D --> E[Save JSON File]
E --> F[Run Validation]
F --> G{Success?}
G -->|Yes| H[Done]
G -->|No| I[Analyze Failure]
I --> J[Modify Pipeline]
J --> F
- Execute Operations - AI performs OCR, clicks, swipes, and other automation operations normally
- Get Documentation - Call
get_pipeline_protocolto get the Pipeline protocol specification - Intelligent Generation - AI converts valid operations into Pipeline JSON based on the documentation
- Save File - Call
save_pipelineto save the generated Pipeline - Run Validation - Call
run_pipelineto verify the Pipeline works correctly - Iterative Optimization - Analyze failures and modify Pipeline until successful
Unlike mechanical recording, AI intelligent generation offers these advantages:
- Only Keeps Successful Paths: If multiple paths were tried during operation (e.g., first entering Menu A without finding the target, then returning and entering Menu B to find it), AI will only keep the final successful path, removing failed attempts
- Understands Operation Intent: AI can understand the purpose of each operation and generate semantically clear node names
- Optimizes Recognition Conditions: Intelligently sets recognition regions and matching conditions based on OCR results
- Validation & Iteration: Discovers issues through run validation, automatically fixes and enhances robustness
After Pipeline generation, AI automatically validates and optimizes:
- Run Validation - Execute Pipeline to check if it succeeds
- Failure Analysis - If failed, analyze which node failed and why
- Intelligent Fixes - Common optimization techniques:
- Add alternative recognition nodes (add multiple candidates in next list)
- Relax OCR matching conditions (use regex or partial matching)
- Adjust roi recognition regions
- Add wait time (post_delay)
- Add intermediate state detection nodes
- Re-validate - Run again after modifications until it consistently succeeds
If the Pipeline logic itself needs adjustment, AI can re-execute automation operations and combine old and new experiences to generate a more robust Pipeline.
When debugging a single node, there is no need to repeatedly edit the pipeline file, suffer slow full-screen recognition, or let the call block forever:
run_pipeline(
controller_id,
"main.json",
entry="ClickSettings", # enter from the node under test
start_agent=False, # recognition only, no CustomAction chain
pipeline_override={ # per-run only: no file edits, Resource untouched
"ClickSettings": {"roi": [520, 20, 200, 80], "timeout": 3000}
},
timeout_seconds=10, # tool-side cap: auto post_stop after 10s
)pipeline_overrideuses the same mechanism as interface.json's field: field-level merge into loaded nodes; any field (expected/threshold/enabled/ ...) can be overridden — and the tighter the ROI, the faster each recognition- On timeout the result carries
status="timeout"plus the nodes executed before the stop, showing exactly where it hung benchmark_nodeaccepts the samepipeline_override, making tuning loops file-edit-free
{
"StartTask": {
"recognition": "DirectHit",
"action": "DoNothing",
"next": ["ClickSettings"]
},
"ClickSettings": {
"recognition": "OCR",
"expected": "Settings",
"action": "Click",
"next": ["EnterDisplay"]
},
"EnterDisplay": {
"recognition": "OCR",
"expected": "Display",
"action": "Click",
"next": ["AdjustBrightness"]
},
"AdjustBrightness": {
"recognition": "OCR",
"expected": "Brightness",
"action": "Swipe",
"begin": [200, 500],
"end": [400, 500],
"duration": 200
}
}📌 Windows Automation Limitations:
- Some games or applications with anti-cheat mechanisms may block background control operations
- If the target application runs with administrator privileges, MaaMCP must also be launched with administrator privileges
- Minimized windows are not supported; please keep the target window in a non-minimized state
- If the default background screenshot/input methods are unavailable (e.g., empty screenshots, unresponsive operations), the AI assistant may attempt to switch to foreground methods, which will occupy the mouse and keyboard
| Feature | Windows | macOS | Linux |
|---|---|---|---|
MCP server (maa-mcp) |
Yes | Yes | Yes |
Pipeline mode (maa-mcp-server) |
Yes | Yes | Yes |
find_adb_device_list / connect_adb_device |
Yes | Yes | Yes |
find_window_list / connect_window |
Yes | No | No |
| Background screenshot / mouse / keyboard | Yes | No | No |
Linux notes:
- ADB-based Android control works on Linux.
- Win32 desktop window control is not supported on Linux/macOS —
find_window_list()andconnect_window()will returnNone/ raise on non-Windows platforms. - If you see ANSI color escape sequences (e.g.
[31m...) leaking into MCP stdio, that is a known interaction with some MCP clients on Linux. Log files are written to~/.local/share/MaaMCP/logs/and are unaffected.
On first use, OCR model files will be automatically downloaded. However, download failures may occur. Please check the data directory:
- Windows:
C:\Users\<username>\AppData\Local\MaaMCP\resource\model\ocr\ - macOS:
~/Library/Application Support/MaaMCP/resource/model/ocr/ - Linux:
~/.local/share/MaaMCP/resource/model/ocr/
- Check if model files exist in the above directory (
det.onnx,rec.onnx,keys.txt) - Check for resource download errors in
model/download.log - Manually run
python -c "from maa_mcp.download import download_and_extract_ocr; download_and_extract_ocr()"to retry downloading
When you configure the MCP server in LM Studio, it needs to find the maa-mcp executable on PATH. On Windows, pip install puts scripts into <Python install dir>\Scripts\ (e.g. C:\Users\<user>\AppData\Local\Programs\Python\Python310\Scripts\), which is often not on PATH by default. Fix:
# Add Scripts dir to your user PATH (permanent)
setx PATH "%PATH%;%LOCALAPPDATA%\Programs\Python\Python310\Scripts"Then restart LM Studio. To verify: where maa-mcp in a new terminal should return the script path. See #6.
When filing an issue, please provide log files. The log file path is as follows:
- Windows:
C:\Users\<username>\AppData\Local\MaaMCP\debug\maa.log - macOS:
~/Library/Application Support/MaaMCP/debug/maa.log - Linux:
~/.local/share/MaaMCP/debug/maa.log
This project is licensed under GNU AGPL v3.
- MaaFramework - Provides powerful automation framework
- FastMCP - Simplifies MCP server development
- Model Context Protocol - Defines AI tool integration standards
