agent-ear

Open-source agentic CLI tool for voice and video comprehension.

Warning

Alpha Release: agent-ear is in active development. APIs and configuration flags may change between minor versions.

Agent ear offers open-source audio and video comprehension for your agentic IDE, including Antigravity. Despite its deep Gemini integration, the tool is CLI-agnostic; it runs wherever command-line execution is available as long as an API key is provided.

Move Beyond Simplistic Transcription

Agent-ear's elegant multimodal architecture enables prosodic contextualization¹: preserving the semantic meaning carried by emphasis, tone, and pacing. By sending media directly to a multimodal model rather than a speech-to-text intermediary, the model hears what plain transcription throws away — the stress, hesitation, and timing that change what words actually mean. It handles the inherent messiness of unscripted speech from meetings, monologues, YouTube videos, and lecture recordings. WCAG-compatible video descriptions extend this to visual content, ensuring human accessibility for visually impaired users while grounding visual information as machine-readable text that persists for any downstream model.

End-to-End Agentic Pipeline

Agent-ear exposes a full pipeline where an AI agent validates its own 'extraction-comprehension prompt', records responses from quick notes to hour-long meetings, transcribes within agent-defined constraints, and can brief human users with spoken instructions using advanced text-to-speech.

Quick Start

Install via Homebrew (macOS)

brew tap Aurelian-Shuttleworth/tools
brew install agent-ear

→ Full guide: How to Install via Homebrew

Install via Nix (macOS / Linux)

# Run without installing (from GitHub)
nix run github:Aurelian-Shuttleworth/agent-ear

# Or clone and run locally
git clone https://github.com/Aurelian-Shuttleworth/agent-ear.git
cd agent-ear && nix run .

→ Full guide: How to Add agent-ear to Your Nix Flake

Features

• 🎤 Voice Capture — Record audio via microphone; stop with Ctrl+C or the on-screen button
• 🗣️ TTS Briefing — Speak instructions to the user before recording (with Director's Notes prosody control)
• 📝 Prompt Validation — LLM-as-a-judge scoring prevents garbage-in/garbage-out
• 🎬 Video Contextualization — WCAG-compliant descriptions of local video files or YouTube URLs
• 🤝 Meeting Mode - Multi-speaker contextualization with action items and notable quotes
• 💰 Cost Tracking — Per-call token usage and estimated dollar cost reporting
• 📜 Open Source - Licensed under Apache 2.0.
• ☁️ Smart Upload — Files up to 2 GB (auto-routed), or GCS staging for larger files

Architecture

agent-ear (the Shell)
├── --non-interactive or non-TTY → exec agent-ear-core (the Engine)
└── interactive TTY   → Launch the Wizard (interactive TUI)
                             └── exec agent-ear-core --non-interactive

Two entry points, one tool:

Binary	Purpose
agent-ear	Main entry point (the Shell) — handles routing and the Wizard
agent-ear-core	The Engine — the Pipeline that agents and scripts call

Usage

Freeform Recording

agent-ear --non-interactive --output-format markdown

Meeting Transcription

agent-ear --non-interactive --prompt "Transcribe this meeting with action items" --model gemini-3.1-pro-preview

Video / YouTube

agent-ear --non-interactive --video ./presentation.mp4
agent-ear --non-interactive --video "https://youtube.com/watch?v=..."

With TTS Briefing

agent-ear --non-interactive --prompt-file ./prompt.md --briefing-file ./briefing.md

Configuration

All configuration follows a priority chain:

CLI flag → Environment variable → Auto-detected → Default

Setting	CLI Flag	Env Var	Default
Output dir	--output-dir	AGENT_EAR_OUTPUT_DIR	Current directory
GCP project	--project-id	GOOGLE_CLOUD_PROJECT	gcloud config
GCS bucket	--gcs-bucket	AGENT_EAR_GCS_BUCKET	{project}-transcribe-staging
Model	--model	—	gemini-3.5-flash

→ Full reference: CLI flags · Environment variables

Authentication

agent-ear supports two authentication backends:

Backend	Setup	Capabilities
Vertex AI	Application Default Credentials + GCP project	Full (GCS uploads, all models)
Google AI Studio	GOOGLE_API_KEY only	Most features (no GCS)

→ Setup guides: Google AI Studio · Vertex AI · Auth reference

Documentation

Full documentation follows the Diátaxis framework. Start at the docs landing page or browse the table below:

Type	Document	Description
Tutorial	Your First Transcription	Get recording in 5 minutes
Tutorial	Home Manager Setup	Add agent-ear to a Home Manager flake
How-to	How to Install via Homebrew	Install on macOS with Homebrew
How-to	How to Set Up Google AI Studio	Free API key authentication
How-to	How to Set Up Vertex AI	Full-featured GCP authentication
How-to	How to Use the Interactive Wizard	Guided setup via the terminal wizard
How-to	How to Record Meetings	Multi-speaker meetings with action items
How-to	How to Write Your Own Prompt Template	Custom templates for the wizard
How-to	How to Set Up GCS Staging	GCS staging (Vertex AI / files > 2 GB)
How-to	How to Brief Users with Spoken Instructions	TTS briefings before recording
How-to	How to Add agent-ear to Your Nix Flake	Use agent-ear in your flake
Explanation	Architecture	Shell/Engine design, pipeline flow, cost model
Explanation	GCS Staging	What staging is and when you need it
Reference	CLI Flags & Exit Codes	Complete flag and exit code reference
Reference	Interactive Wizard Screens	Screen-by-screen wizard specification
Reference	Environment Variables	All env vars
Reference	Authentication	Auth resolution & feature matrix

See the Changelog for release history.

For Nix Consumers

As a flake input

{
  inputs.agent-ear.url = "github:Aurelian-Shuttleworth/agent-ear";

  # Use the overlay
  nixpkgs.overlays = [ inputs.agent-ear.overlays.default ];

  # Or reference the package directly
  environment.systemPackages = [ inputs.agent-ear.packages.${system}.agent-ear ];
}

As a flake-parts module

{
  imports = [ inputs.agent-ear.flakeModules.agent-ear ];
  agent-ear.enable = true;
}

The flake-parts module currently only exposes the enable option; add the package via the overlay or packages output above. For a turnkey setup, use the Home Manager module below.

With Home Manager (recommended)

Installs the binary, agent skill, and voice-mode workflow automatically:

{
  # In your flake.nix: apply the overlay
  nixpkgs.overlays = [ inputs.agent-ear.overlays.default ];
}

# In your home-manager config:
{
  imports = [ inputs.agent-ear.homeManagerModules.default ];
  agent-ear.enable = true;
}

This places:

• agent-ear binary in your PATH
• ~/.gemini/config/skills/agent-ear/SKILL.md for AI agent auto-discovery
• ~/.gemini/config/workflows/voice-mode.md for voice interaction workflows

Options:

Option	Default	Description
agent-ear.enable	false	Enable agent-ear
agent-ear.skills.enable	true	Install skill for AI agent discovery
agent-ear.workflows.enable	true	Install voice-mode workflow
agent-ear.configDir	".gemini/config"	Base path for skills/workflows (change to ".agents" for Antigravity 2.0)

→ Full guide: Nix Consumer Integration

For Homebrew Users

Install agent-ear and all native dependencies with two commands:

brew tap Aurelian-Shuttleworth/tools
brew install agent-ear

This installs both agent-ear (interactive wizard) and agent-ear-core (Python engine), plus all native dependencies (gum, portaudio, libsndfile, ffmpeg, yt-dlp).

# Verify
agent-ear --help

# Update
brew upgrade agent-ear

→ Full guide: How to Install via Homebrew

Development

# Enter dev shell
nix develop

# Run checks
nix flake check

# Build the package
nix build

Community & Governance

• Contributing Guide — How to set up the dev environment and submit PRs
• Security Policy — How to report vulnerabilities safely
• Code of Conduct — Community guidelines

License

Dual-licensed under MIT and Apache 2.0. See LICENSE-MIT and LICENSE-APACHE.

Footnotes

1. prosody - the rhythmic and intonational aspect of language ↩