Skip to content

technical-design.md

Latest commit

 

History

History
110 lines (86 loc) · 5.49 KB

technical-design.md

File metadata and controls

110 lines (86 loc) · 5.49 KB
Raw

Android Mirroring technical design

Purpose

Android Mirroring is a native macOS application that lets a person use the apps, audio, notifications, clipboard, and shared storage of a real Android device over an explicitly authorised USB or local-network ADB connection. It is not an APK runtime or Android emulator.

Product boundary

The product supports macOS 15+ and Android 12+ for the core session. Fusion mode, which places apps on Android virtual displays and shows them in separate Mac windows, is offered only where the device capability probe verifies it; the baseline fallback is a mirror of the primary device display.

It does not bypass Android debugging authorisation, DRM/secure surfaces, enterprise device policy, biometric prompts, or vendor restrictions on injected input. It has no cloud relay, account, or telemetry service in the core path.

Architecture

SwiftUI/AppKit macOS application
  ├─ Device session manager ────── ADB transport ────── Android adbd
  ├─ Video/audio renderer ─────── session sockets ───── device server
  ├─ Clipboard/notification bridge ─ control socket ─── device server
  └─ File Provider extension ───── ADB file operations ─ Android storage

macOS application

The application owns device discovery, session lifecycle, native AppKit windows, VideoToolbox decoding, AudioToolbox playback, and consent UI. SwiftUI is used for the settings and device list; AppKit owns every independently resizable app window. ADBClient is the only component permitted to spawn adb.

ADB transport and device server

The app invokes the bundled, version-pinned Android Platform Tools binary. It first uses adb devices -l, then reads SDK/device capabilities, and only starts a session after the state is device. For each session it pushes a signed, versioned server JAR to a private path under /data/local/tmp, starts it through app_process, and forwards three local sockets: video, audio, and control. The server is removed when the session ends unless cleanup fails; stale artifacts are removed before the next launch.

The server design is adapted from scrcpy's proven host/server split: H.264 is the baseline video stream, Opus is the baseline audio stream, and control events use a bounded binary protocol. It begins with display mirroring; no user-facing feature depends on Fusion mode. scrcpy is Apache-2.0, so any copied code retains its notices and license text in ThirdPartyNotices.md.

Input and coordinate correctness

InputMapper maps a macOS event to a (displayID, x, y, action, timestamp) event after removing letterboxing and applying video rotation, pixel dimensions, and device density. The server injects it through the ADB-authorised shell context. It selects UHID keyboard/mouse simulation when the device accepts it, then key/text injection as a compatible fallback. The UI reports an actionable capability error when a vendor ROM rejects injected events.

Video and audio

The decoder drops obsolete frames rather than accumulating latency. It requests H.264 first, negotiates H.265 only if both endpoints expose support, and selects resolution/fps/bitrate from measured decode time and transport backlog. The audio stream timestamps against the video clock; a reconnect creates a new clock epoch rather than trying to conceal drift.

Fusion mode

When the capability probe passes on Android 14+, the device server creates a virtual display, launches a selected package to that display, and exposes a separate stream and display ID to the Mac app. Closing a Fusion window returns the Activity to the primary display when supported; otherwise the user is warned before close. Any failure disables Fusion for that device build and starts a normal mirror session. This contains the risk from hidden/system APIs and OEM changes.

Finder integration

The macOS File Provider target uses NSFileProviderReplicatedExtension. It enumerates Android storage on demand, materializes a file with adb pull, and uploads edits with upload-to-temporary-name, SHA-256 verification, and atomic rename. Concurrent writes preserve both versions: the Mac edit gets a conflict suffix instead of overwriting phone data. The cache is encrypted only through standard File Provider/macOS data protection; no application-level encryption claim is made.

Security and privacy decisions

  • ADB RSA authorisation remains Android's source of trust; this app cannot auto- approve a computer on the phone.
  • Wireless sessions require a user-approved pairing and the same local network.
  • No port listens off-host. No content is sent to a product-operated server.
  • Debug logs redact clipboard data, filenames by default, and control payloads.
  • Device server compatibility is pinned to the desktop release; mismatches refuse to start instead of attempting undefined protocol negotiation.

Dependencies and references