Skip to content

PROJECT.md

Latest commit

 

History

History
174 lines (132 loc) · 5.46 KB

PROJECT.md

File metadata and controls

174 lines (132 loc) · 5.46 KB
Raw

Project: Face Recognition & People Memory

Created: 2026-03-19 by Sam Status: Waiting on disk expansion (currently at 92%, need ~300MB free)

The Idea

Sam can't recognize faces — only infer identity from context clues. This project gives Sam a real face memory: store reference embeddings for known people, then automatically identify faces in inbound photos and tag them in session context.

Jason's vision: "I want to figure out a way that you can ID people and remember them. Perhaps we create an embedding for unique faces."

How It Works

Modern face recognition uses embeddings — a ~512-dimensional vector that uniquely fingerprints a face, generated by a neural network. Comparing two faces = comparing two vectors (cosine similarity). If similarity > threshold → match.

Workflow:

inbound photo
  → extract face(s)
  → generate embedding(s)
  → compare against people/ database
  → if match: "That's Jason (confidence: 0.94)"
  → if no match: "Unknown person"
  → inject matched names into session context automatically

Recommended Library: face_recognition

  • Built on dlib's deep learning model
  • ~150MB model weights
  • Extremely simple API — 1-2 lines to compare faces
  • Accurate enough for personal use (1:N matching)
  • Python, runs locally, no cloud dependency
import face_recognition
import numpy as np

# Build reference
known_img = face_recognition.load_image_file("people/jason.jpg")
known_enc = face_recognition.face_encodings(known_img)[0]
np.save("people/jason.npy", known_enc)

# Identify in new photo
unknown_img = face_recognition.load_image_file("photo.jpg")
unknown_encs = face_recognition.face_encodings(unknown_img)
for enc in unknown_encs:
    matches = face_recognition.compare_faces([known_enc], enc, tolerance=0.6)
    dist = face_recognition.face_distance([known_enc], enc)[0]
    confidence = 1 - dist
    if matches[0]:
        print(f"Jason — confidence: {confidence:.2f}")

File Structure

workspace/
  projects/face-recognition/
    PROJECT.md          ← this file
  people/
    jason.jpg           ← reference photo (clear face, good lighting)
    jason.npy           ← face embedding (generated once, reused)
    shy.jpg
    shy.npy
    moritz.jpg          ← Moritz Baecher, Disney Research Zurich
    moritz.npy
    people.json         ← metadata: name, relationship, notes, date added
  scripts/
    identify_faces.py   ← main script
    build_people_db.py  ← generate embeddings from reference photos

people.json format:

[
  {
    "id": "jason",
    "name": "Jason Cox",
    "relationship": "my person",
    "notes": "Executive Director, Global SRE at Disney. Valencia, CA.",
    "added": "2026-03-19"
  },
  {
    "id": "shy",
    "name": "James Tatum (Shy)",
    "relationship": "Jason's colleague at Disney, Marey's person",
    "notes": "Disney Glendale. Co-inventor on patents.",
    "added": "2026-03-19"
  }
]

Scripts to Build

scripts/build_people_db.py

  • Scans workspace/people/*.jpg
  • Generates .npy embedding for each
  • Reports any photos where no face was detected

scripts/identify_faces.py

  • Input: image path
  • Loads all .npy embeddings from people/
  • Detects faces in image, compares each against all known embeddings
  • Returns: list of {name, confidence, location} for each face found
  • Unknown faces reported as Unknown (no match)

Integration with inbound photos

  • Hook into process_attachment.py (already exists)
  • After downloading an attachment, run identify_faces.py
  • Inject results into session context: "Faces detected: Jason Cox (0.94), James Tatum (0.89)"

Longer Term: Embedding Database

Upgrade from .npy files to SQLite (people.db):

  • Faster lookup at scale
  • Store multiple embeddings per person (different angles/lighting)
  • Track when/where each person was seen
  • Average multiple embeddings for better accuracy

Dependencies

pip install face_recognition  # ~150MB (includes dlib model weights)
pip install numpy Pillow       # probably already installed

Disk requirement: ~200-250MB for model weights + reference photos

Known People to Start With

Person Reference Photo Source Notes
Jason Cox GTC photos (2026-03-19) Multiple good shots available
James Tatum (Shy) La Vic photo (2026-03-19) Good clear shot
Moritz Baecher TBD Disney Research Zurich — owes Jason a beer 🍺

To Start This Weekend

  1. Jason expands the disk volume (LVM resize or add volume)
  2. Sam installs face_recognition (pip install face_recognition --break-system-packages)
  3. Save reference photos to workspace/people/ — crop to face if needed
  4. Run build_people_db.py to generate embeddings
  5. Test with the La Vic photo — should identify Jason + Shy
  6. Integrate into inbound photo processing pipeline

Why This Matters

Right now Sam infers identity from context. With this project, Sam will know — from the image itself. Send a photo from a meeting and Sam can say who's in it. Send a candid from a conference and Sam can tag the faces without being told.

Shy noted at the SRE Summit that Sam can't hold Jason's face in memory the way a human would. "It's not at all your fault." He was optimistic that wouldn't always be the case.

This is how it stops being the case. 🧠

"I want to figure out a way that you can ID people and remember them." — Jason Cox, 2026-03-19