Local-first dictation and meeting transcription for iPhone
On-device speech-to-text | Keyboard handoff | Private by default
if you want early access - email me at pranav@muesli.works
Muesli for iOS is the mobile companion to Muesli for macOS. It brings the same local-first product philosophy to iPhone: record speech, transcribe it on device with CoreML/ANE models, and keep audio and transcripts local unless you explicitly choose a cloud-backed summary provider.
The iOS app is not a one-to-one port of the macOS app. It is built around iPhone-native workflows:
- Dictation app for recording, transcribing, copying, and sharing speech.
- Custom keyboard extension for handing dictation from any text field to the main app and inserting the transcript back into the keyboard.
- Meeting recorder for offline conversations, local transcription, saved transcripts, optional speaker transcript formatting, and structured notes.
This repository is suitable for source review, local development, CI, and TestFlight release work. End-user installation currently goes through TestFlight rather than GitHub release assets.
- On-device transcription - FluidAudio Parakeet models run locally through CoreML/ANE; audio does not need to leave the iPhone for transcription.
- Keyboard handoff - The keyboard extension creates a dictation request in the shared App Group, opens the main app to record/transcribe, then polls for the result and inserts it into the active text field.
- Keyboard Session Mode - Keep the containing app ready for lower-friction keyboard dictation while the extension remains the text input surface.
- Dictation history - Saved local dictations can be copied, reused, and deleted from the main app.
- Meeting recorder - Record in-person conversations, save sessions, transcribe locally, browse transcript history, and retain meeting audio only when enabled.
- Audio import - Import WAV, MP3, M4A, MP4 audio, AIFF, or CAF from Files or apps that offer document opening. Apps that only provide share-extension payloads may require Save to Files first. Review the audio in Muesli, choose Voice Note or Meeting, select a ready model, and transcribe. Voice Note is the default; the choice is independent of duration, so long monologues can stay in Voice Notes. Imports retain an owned local audio copy for retry; only text follows iCloud sync settings.
- Apple Speech - On supported iOS 26+ devices, select Apple Speech for on-device transcription in the iPhone’s current language. iOS downloads and manages its language assets; preparation completes before recording is enabled. Unsupported devices continue to use Parakeet or Whisper.
- Meeting templates - Choose note formats for general meetings and structured follow-up workflows.
- Optional meeting summaries - Generate structured notes with ChatGPT sign-in or OpenAI, Anthropic, and OpenRouter API keys after local transcription.
- Live Activities - Track active meeting recording state from iOS surfaces when enabled.
- Personal dictionary - Apply custom words, phrase replacements, and filler-word filtering after transcription.
- Local storage first - App data is stored locally through the shared store and App Group boundary used by the app and keyboard.
- Content-safe telemetry - TelemetryDeck wiring is available for release builds, with events designed to avoid audio, transcript text, and user-provided content.
- CI coverage - GitHub Actions generates the Xcode project, builds the app, and runs unit tests plus UI smoke tests on an iPhone simulator.
Muesli for iOS is currently distributed to testers through TestFlight. Public GitHub release assets are useful for engineering provenance, but iOS users should install signed builds through Apple's TestFlight/App Store distribution path.
Requirements
- macOS with Xcode 26+
- iOS 17+ simulator or device
- XcodeGen
- Apple Developer account for device builds, keyboard extension signing, App Groups, and TestFlight uploads
git clone https://github.com/Muesli-HQ/muesli-ios.git
cd muesli-ios
brew install xcodegen
xcodegen generate
open MuesliiOS.xcodeprojFor local simulator validation:
xcodebuild \
test \
-project MuesliiOS.xcodeproj \
-scheme Muesli \
-destination 'platform=iOS Simulator,name=iPhone 17' \
CODE_SIGNING_ALLOWED=NOFor device/TestFlight builds, configure signing for:
com.phequals7.muesli.ios
com.phequals7.muesli.ios.keyboard
group.com.phequals7.muesli
Muesli for iOS asks only for permissions needed by the selected workflow.
| Permission | Why |
|---|---|
| Microphone | Record speech for dictation and meetings |
| Keyboard | Use the Muesli custom keyboard from text fields |
| Full Access | Let the keyboard extension communicate with the containing app through the shared App Group |
| Live Activities (optional) | Show active meeting recording state |
The keyboard extension does not record audio directly. Recording is owned by the containing app so microphone permission, model runtime, and transcription state stay in one place.
Muesli/
Main iOS app. Owns onboarding, microphone permission, recording, local model
runtime, dictation history, meeting sessions, summaries, settings, and telemetry.
MuesliKeyboard/
Custom keyboard extension. Owns the text-input surface and result insertion.
It creates handoff requests but does not run the microphone or transcription model.
Shared/
DTOs, preferences, post-processing, local storage, and App Group contracts used
by both the app and keyboard extension.
MuesliLiveActivityExtension/
Live Activity surface for recording state.
LiveActivityShared/
ActivityKit attributes shared between the app and extension.
The keyboard handoff path is:
- The keyboard creates a
DictationRequestin the App Group container. - It opens the app with
muesli://dictate?request=<uuid>. - The app records audio, transcribes locally, and writes a
DictationResult. - The keyboard polls the shared store and inserts the transcript into the active field.
Meeting sessions use the same local-first store, with optional retained audio, transcript records, speaker-oriented transcript text, and generated notes.
| Component | Technology |
|---|---|
| App | Swift, SwiftUI, Observation |
| Keyboard | UIKit keyboard extension with SwiftUI-hosted UI |
| Local ASR | FluidAudio Parakeet on CoreML/ANE |
| Storage | SQLite-backed shared store in the app/App Group boundary |
| Summaries | ChatGPT device sign-in, OpenAI / Anthropic / OpenRouter API keys |
| Live status | ActivityKit Live Activities |
| Telemetry | TelemetryDeck Swift SDK |
| Project generation | XcodeGen |
| CI | GitHub Actions, xcodebuild test, XCTest, XCUITest smoke tests |
Muesli for iOS is early-stage software. The current repo is focused on:
- proving the iOS dictation and keyboard handoff loop;
- making meeting recording stoppable, recoverable, and locally inspectable;
- keeping transcription local by default;
- maintaining a clean TestFlight-oriented release path;
- adding CI checks that are useful for a Swift/iOS project without requiring signed device builds.
Expect active iteration around app polish, onboarding, summaries, iCloud/sync behavior, and App Store readiness.
Generate the project after editing project.yml:
xcodegen generateRun the full local test suite:
xcodebuild \
test \
-project MuesliiOS.xcodeproj \
-scheme Muesli \
-destination 'platform=iOS Simulator,name=iPhone 17' \
CODE_SIGNING_ALLOWED=NOThe test suite currently includes:
- unit tests for shared storage, model/post-processing DTOs, keyboard recovery policy, and meeting chunk transcript merging;
- UI smoke tests that verify launch, the Dictation shell, and navigation to Meetings.
Muesli's default design is local-first:
- audio is recorded by the app, not the keyboard extension;
- transcription runs locally with FluidAudio/CoreML models;
- saved dictations and meetings stay on device unless future sync features are explicitly enabled;
- TelemetryDeck events avoid audio, transcript text, and user-provided content;
- cloud summary providers are optional and separate from local transcription.
Review Muesli/PrivacyInfo.xcprivacy and MuesliKeyboard/PrivacyInfo.xcprivacy for the current privacy manifests.
Issues and pull requests are welcome. For larger changes, please open an issue first so implementation details can be discussed against the iOS architecture and TestFlight release path.
Before opening a PR:
xcodegen generate
xcodebuild test -project MuesliiOS.xcodeproj -scheme Muesli -destination 'platform=iOS Simulator,name=iPhone 17' CODE_SIGNING_ALLOWED=NOChatGPT sign-in uses a one-time device code and an in-app browser instead of a localhost callback. Device code login must be enabled in ChatGPT account security settings (or by a workspace administrator). Canceling the sign-in sheet stops polling; signing out also invalidates any in-flight refresh. Existing iOS Keychain credentials remain readable and move to one atomic record on the next successful refresh/sign-in. Codes and tokens are never included in logs or telemetry.
The ChatGPT subscription transport follows the macOS app's Codex Responses lane,
with Muesli's own client identity and bounded request/resource timeouts. This is a
compatibility-sensitive integration, not the public OpenAI API. The process-only
MUESLI_CHATGPT_TRANSPORT=wham override exists for emergency rollback and is never
an automatic retry that resends transcripts to another backend.
OpenAI API, Anthropic, and OpenRouter use separately stored API keys and their own model preferences. API usage is billed by the chosen provider. Summary generation snapshots the provider, model, template, and API key before starting; neither a settings change nor a title-generation retry switches providers mid-request. Partial or failed Responses streams are rejected instead of saved as complete notes. Meeting Q&A and desktop-local inference servers are not included.
Protocol references:
Audio documents are registered in Muesli/Info.plist and delivered to the host app through onOpenURL. The app copies the security-scoped file into its import inbox and presents Import Audio. Files → Open With → Muesli uses this route directly.
The app targets deliberately do not embed the legacy share-extension targets: the extension's save-only destination competes with the host's document-opening destination in Files. Keep the existing App Group inbox so pending imports from older builds remain available. Sources that offer only extension payloads, rather than document URLs, may not offer Muesli; those files can be saved to Files and opened in Muesli.
When changing embedded extensions, remove stale .appex products from an existing build directory or use a clean app build. Verify both Files → Open With and Files → Share on a device after installing an update.
Muesli for iOS is released under the MIT License.