Skip to content

CLAUDE.md

Latest commit

 

History

History
445 lines (369 loc) · 17.1 KB

CLAUDE.md

File metadata and controls

445 lines (369 loc) · 17.1 KB
Raw

Mailbox Zero - Claude Development Documentation

Project Status: ✅ COMPLETED

Last Updated: May 30, 2026 (similarity algorithm rewrite) Version: 1.0 Status: Production Ready (with dry run safety)

Project Overview

Mailbox Zero is a Go-based web application that helps users clean up their Fastmail inbox by finding and archiving similar emails using the JMAP protocol. The application provides a dual-pane interface with advanced fuzzy matching capabilities and comprehensive safety features.

Architecture

Tech Stack

  • Backend: Go 1.26+ with Gorilla Mux router
  • Frontend: Vanilla HTML5, CSS3, JavaScript (ES6+)
  • Email Protocol: JMAP (JSON Meta Application Protocol)
  • Email Provider: Fastmail
  • Configuration: YAML
  • Templates: Go HTML templates

Project Structure

mailboxzero/
├── main.go                     # Application entry point
├── go.mod / go.sum             # Go module dependencies (gorilla/mux, yaml.v3)
├── config.yaml.example         # Configuration template
├── config-mock.yaml.example    # Mock-mode configuration template
├── .gitignore                  # Git ignore rules
├── LICENSE                     # MIT license
├── README.md                   # User documentation
├── CLAUDE.md                   # Development documentation (this file)
├── .github/
│   └── workflows/
│       └── test.yml            # CI: gofmt check and `go test -race -coverprofile`
├── internal/
│   ├── config/
│   │   ├── config.go           # Configuration loading and validation
│   │   └── config_test.go
│   ├── jmap/
│   │   ├── client.go           # JMAPClient interface + real Client transport
│   │   ├── email.go            # Email/Mailbox types + Archive/Unarchive operations
│   │   ├── mock.go             # MockClient with built-in sample data for mock mode
│   │   ├── jmap_test.go
│   │   └── mock_test.go
│   ├── server/
│   │   ├── server.go           # HTTP server and API handlers
│   │   └── server_test.go
│   └── similarity/
│       ├── similarity.go       # Fuzzy matching algorithms
│       └── similarity_test.go
└── web/
    ├── templates/
    │   └── index.html          # Main application template
    └── static/
        ├── style.css           # Application styles
        └── app.js              # Frontend JavaScript

Core Components

1. Configuration System (internal/config/)

  • File: config.go
  • Purpose: Manages YAML configuration loading and validation
  • Features:
    • Server port/host configuration
    • JMAP endpoint and credentials
    • Dry run safety toggle
    • Default similarity threshold

2. JMAP Client (internal/jmap/)

  • Files: client.go, email.go, mock.go
  • Purpose: Handles communication with Fastmail's JMAP API
  • Features:
    • JMAPClient interface implemented by both the real Client and the in-memory MockClient (selected via mock_mode in config)
    • Session authentication with Bearer tokens
    • Mailbox discovery (inbox, archive)
    • Email querying and retrieval with body content, including a paginated GetInboxEmailsWithCountPaginated used by the /api/emails endpoint
    • Safe archive operations (move to archive folder)
    • Comprehensive error handling

3. Similarity Engine (internal/similarity/)

  • File: similarity.go
  • Purpose: Fuzzy matching tuned for newsletter/notification clustering
  • Algorithm:
    • Sender Similarity (50% weight): Structured ladder, not Levenshtein. Same full address → 1.0; same domain → 0.8 (suppressed for shared-ESP domains like sendgrid.net unless local parts also match); same registrable root domain (e.g. mail.google.com ↔ accounts.google.com) → 0.7; same non-generic display name → 0.6.
    • Subject Similarity (30% weight): max(tokenJaccard, levenshtein) over a normalized subject that has reply/forward prefixes (Re:, Fwd:, [list]) stripped. Token Jaccard filters a small stop-word set (weekly, newsletter, update, etc.) and requires both sides to have ≥2 surviving tokens before being credited; otherwise the score falls back to Levenshtein alone.
    • Body Similarity (20% weight): Jaccard similarity over normalized word tokens (length ≥ 3) extracted from preview/body.
  • Features:
    • Per-email features (senderAddr, senderDomain, senderRoot, senderName, subjectNorm, subjectTokens, bodyTokens) precomputed once per call so the inner pairwise compare avoids re-parsing senders or re-tokenizing strings
    • Threshold-aware short-circuit: cheap-first ordering (sender → subject → body) bails out as soon as the partial score plus the maximum remaining contribution falls below threshold
    • Single-link cluster expansion in groupSimilarFeatures: a candidate joins a cluster if it matches ANY existing member, not just the seed — important when the seed is an atypical member of an obvious group
    • String normalization (lowercase, punctuation-to-space, collapsed whitespace)
    • Public-suffix awareness for a small set of multi-label suffixes (co.uk, github.io, etc.)
    • Group-based and individual email matching

4. Web Server (internal/server/)

  • File: server.go
  • Purpose: HTTP server with RESTful API endpoints
  • Endpoints:
    • GET / - Main application interface
    • GET /api/emails - Fetch inbox emails
    • POST /api/similar - Find similar emails with threshold
    • POST /api/archive - Archive selected emails
    • POST /api/unarchive - Move previously archived emails back to the inbox (powers the Undo button)
    • POST /api/clear - Clear results
  • Features:
    • Template rendering with data injection
    • JSON API responses
    • Error handling and logging
    • Static file serving
    • Inbox cache for /api/similar: the 1000-email JMAP fetch is cached in-process for 60 seconds (inboxCacheTTL) and invalidated at the end of handleArchive/handleUnarchive, so successive "Find Similar" / "Archive & Find Next" calls don't re-pull from JMAP

5. Frontend Interface (web/)

  • Template: index.html - Responsive dual-pane layout
  • Styles: style.css - Modern CSS with mobile responsiveness
  • JavaScript: app.js - Single-page application logic
  • Features:
    • Real-time similarity threshold adjustment
    • Email selection and multi-select capabilities
    • Modal confirmation dialogs
    • Async API communication
    • Responsive design for mobile/desktop

Key Features Implemented

✅ Safety Features

  1. Dry Run Mode: Default enabled, prevents actual email modifications
  2. Archive Only: Never deletes emails, only moves to archive folder
  3. Confirmation Dialogs: Required before the standard "Archive Selected" write operation (the opt-in "Archive & Find Next" button intentionally skips it for rapid sweeping)
  4. Visual Warnings: Clear UI indicators when in dry run mode
  5. API Token Authentication: Secure authentication using Fastmail API tokens
  6. Undo Archive: After every successful archive, a status pill with an "Undo" button appears in the top-right of the action bar. Undo POSTs the last-archived IDs to /api/unarchive, which moves them back to the inbox. Undo does not refresh the similar-emails pane — the restored emails reappear on the next Refresh or Find Similar. After Undo, the status text changes to "Unarchived N e-mail(s)" with a glow animation (undoGlow in style.css).

✅ Core Functionality

  1. Dual-Pane Interface: Inbox (left) and similar emails (right)
  2. Smart Similarity Matching: Multi-factor fuzzy algorithm
  3. Adjustable Threshold: 0-100% similarity slider with real-time updates
  4. Email Selection: Individual and bulk selection with checkboxes
  5. Archive Operations: Bulk archive with JMAP email movement
  6. Rapid Archive Loop: "Archive & Find Next" button archives the selected similar emails without confirmation and immediately re-runs the same similarity search (uses archiveAndFindNext() in app.js)
  7. Clear Results: Reset functionality for multiple searches
  8. Individual Email Targeting: Select specific email to find matches

✅ User Experience

  1. Responsive Design: Mobile and desktop optimized
  2. Loading States: Clear feedback during async operations
  3. Error Handling: User-friendly error messages
  4. Accessibility: Keyboard navigation and screen reader support
  5. Performance: Efficient API calls and client-side caching

Configuration Details

Required Settings

server:
  port: 8080                    # Default web server port
  host: "localhost"             # Server binding host

jmap:
  endpoint: "https://api.fastmail.com/jmap/session"
  api_token: ""                 # Fastmail API token (required)

dry_run: true                   # Safety feature - MUST be false for real operations
default_similarity: 60          # Default similarity percentage (0-100)

Security Considerations

  • API Tokens: Use Fastmail API tokens for secure authentication
  • Local Processing: All similarity calculations happen locally
  • Minimal Permissions: Only requires read access to inbox and write to archive
  • No External Services: No data sent to third-party services

API Endpoints

GET /api/emails

  • Purpose: Retrieve inbox emails (paginated)
  • Query Parameters: limit (default 100) and offset (default 0)
  • Response: JSON object { "emails": [...], "totalCount": N } (InboxInfo)
  • Email Fields: ID, subject, from, preview, receivedAt, bodyValues

POST /api/similar

  • Purpose: Find similar emails
  • Request Body: 
    {
  "similarityThreshold": 60.0,
  "emailId": "optional-specific-email-id"
}
  • Response: JSON array of matching email objects

POST /api/archive

  • Purpose: Archive selected emails
  • Request Body: 
    {
  "emailIds": ["id1", "id2", "id3"]
}
  • Response: Success confirmation with dry run status

POST /api/unarchive

  • Purpose: Move previously archived emails back to the inbox (Undo)
  • Request Body: 
    {
  "emailIds": ["id1", "id2", "id3"]
}
  • Response: Success confirmation with dry run status (same shape as /api/archive)

POST /api/clear

  • Purpose: Clear similarity results
  • Response: Success confirmation

Development Commands

Setup and Dependencies

# Initialize Go modules
go mod download
go mod tidy

# Create configuration file from example
cp config.yaml.example config.yaml
# Edit config.yaml with your Fastmail API token

# Run the application
go run main.go

# Run with custom config
go run main.go -config custom-config.yaml

# Build for production
go build -o mailboxzero main.go

Mock Mode for Testing

For development and testing purposes, you can run the application in mock mode:

# Copy the mock configuration
cp config-mock.yaml.example config.yaml

# Run in mock mode - no Fastmail credentials needed
go run main.go

Mock Mode Features:

  • Uses realistic sample email data (~32–52 emails: 10 senders × 3–5 messages plus 2 unique)
  • No real JMAP connection required
  • Sample emails include groups of similar messages for testing similarity matching
  • Simulates archiving operations without affecting real emails
  • Perfect for development, testing, and demonstrations
  • Provides consistent test data across runs

Mock Configuration:

server:
  port: 8080
  host: "localhost"

jmap:
  endpoint: ""      # Not required in mock mode
  api_token: ""     # Not required in mock mode

dry_run: true       # Keep enabled for safety
default_similarity: 60
mock_mode: true     # Enable mock mode

Testing Commands

# Quick start with mock data (no Fastmail account required)
go run main.go -config config-mock.yaml.example

# Run all tests
go test ./...

# Run tests with verbose output
go test ./... -v

# Run with race detection
go test -race ./...

# Run tests with coverage
go test ./... -cover

# Generate coverage report
go test ./... -coverprofile=coverage.out
go tool cover -html=coverage.out

# Run benchmarks
go test ./... -bench=.

# Lint the code (requires golangci-lint)
golangci-lint run

Deployment Instructions

Production Deployment

  1. Build Binary:

    go build -o mailboxzero main.go

  2. Create Production Config:

    cp config.yaml.example config.yaml

    Then edit config.yaml:

    server:
  port: 8080
  host: "0.0.0.0"  # For external access
jmap:
  api_token: "production-api-token"
dry_run: false      # Enable real operations

  3. Run Binary:

    ./mailboxzero -config production-config.yaml

Docker Deployment (Future Enhancement)

FROM golang:1.26-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -o mailboxzero main.go

FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /app/mailboxzero .
COPY --from=builder /app/web ./web
CMD ["./mailboxzero"]

Known Limitations and Future Enhancements

Current Limitations

  1. Single Account: Only supports one Fastmail account at a time
  2. Memory Usage: Loads all emails into memory for processing
  3. No Persistent Storage: No database for tracking operations
  4. Bearer Token Authentication: Uses API tokens instead of OAuth2

Potential Enhancements

  1. Multi-Account Support: Support multiple email accounts
  2. Database Integration: SQLite for operation history and caching
  3. Advanced Filters: Date ranges, sender whitelist, size limits
  4. Batch Operations: Process large inboxes in chunks
  5. Email Preview: Full email content preview before archiving
  6. Statistics Dashboard: Email cleanup metrics and reports
  7. API Rate Limiting: Respect JMAP API rate limits
  8. OAuth2 Support: Modern authentication flow

Troubleshooting Guide

Common Issues

Authentication Failures

  • Symptom: "Failed to authenticate" error
  • Solutions:
    1. Verify Fastmail API token is correct
    2. Generate new API token in Fastmail settings (Settings → Privacy & Security → Integrations)
    3. Ensure JMAP is enabled in account settings
    4. Check network connectivity to api.fastmail.com

No Emails Found

  • Symptom: Empty inbox or no similar emails found
  • Solutions:
    1. Verify emails exist in Fastmail inbox
    2. Lower similarity threshold (try 50% or lower)
    3. Check email content has sufficient text for matching
    4. Verify mailbox permissions

UI Not Loading

  • Symptom: Blank page or JavaScript errors
  • Solutions:
    1. Check browser console for errors
    2. Verify static files are served correctly
    3. Clear browser cache
    4. Check server logs for template errors

Debug Mode

Enable verbose logging by modifying main.go:

log.SetFlags(log.LstdFlags | log.Lshortfile)

Code Quality Standards

Go Best Practices Followed

  1. Package Organization: Clear internal package structure
  2. Error Handling: Comprehensive error wrapping and logging
  3. Interface Design: Clean separation of concerns
  4. Memory Management: Efficient string operations and minimal allocations
  5. Concurrency Safety: Thread-safe operations where needed
  6. Comprehensive Testing: Full unit test coverage with table-driven tests

Test Coverage

The project includes comprehensive unit tests for all packages:

  • Config Package: Configuration loading, validation, and error handling
  • JMAP Package: Data parsing, mock client functionality, and helper functions
  • Similarity Package: Fuzzy matching algorithms, Levenshtein distance, email similarity
  • Server Package: HTTP handlers, API endpoints, and request/response handling

Tests follow Go best practices:

  • Table-driven test design for multiple scenarios
  • Clear test naming and organization
  • Use of test helpers and fixtures
  • Mock clients for external dependencies
  • Benchmark tests for performance-critical functions

Code Style

  • Naming: Clear, descriptive variable and function names
  • Documentation: Comprehensive comments and documentation
  • Formatting: Standard gofmt formatting
  • Imports: Organized standard, external, and internal imports
  • Error Messages: User-friendly error messages with context

Maintenance Notes

Regular Maintenance Tasks

  1. Dependency Updates: Keep Go modules up to date
  2. Security Patches: Monitor for security vulnerabilities
  3. Performance Monitoring: Track API response times
  4. Log Analysis: Review error patterns and usage metrics

Backup Considerations

  • Configuration Files: The config.yaml file contains credentials and should NOT be committed to version control (already in .gitignore)
  • User Data: No persistent user data to backup
  • Application State: Stateless application, no backup needed
  • Template Files: config.yaml.example should be committed as a template

Note: This application is designed with safety as the primary concern. The dry run mode should remain enabled during initial testing, and all operations should be thoroughly tested before enabling real email modifications.