Skip to content

Latest commit

 

History

History
955 lines (761 loc) · 29.1 KB

ARCHITECTURE.md

File metadata and controls

955 lines (761 loc) · 29.1 KB

HippocampAI Architecture Guide

Overview

HippocampAI is a production-ready autonomous memory engine designed for LLM agents with enterprise-grade features:

  • Type-Safe Architecture - Comprehensive type safety with Pylance/MyPy support
  • Hybrid Retrieval - Combines semantic, BM25, and reranking for optimal recall
  • Distributed Processing - Celery-based background tasks with Redis/RabbitMQ
  • Comprehensive Telemetry - Built-in observability and performance monitoring
  • Flexible Deployment - Docker, local, or cloud deployment options
  • Multiple Clients - Sync, async, and unified client interfaces

Core Components

# Main client interface
from hippocampai import MemoryClient
client = MemoryClient()

# Store and retrieve memories
memory = client.remember("Python is great for ML", user_id="alice")
results = client.recall("What does alice think about Python?", user_id="alice")

# Async support
from hippocampai import AsyncMemoryClient
async_client = AsyncMemoryClient()
await async_client.remember_async("text", user_id="alice")

System Architecture

High-Level System Diagram

graph TB
    subgraph "Client Applications"
        A[Python Application]
        B[FastAPI App]
        C[CLI Tool]
    end

    subgraph "HippocampAI Core"
        D[MemoryClient]
        E[AsyncMemoryClient]
        F[UnifiedMemoryClient]

        subgraph "Business Logic"
            G[Memory Operations]
            H[Hybrid Retrieval]
            I[Knowledge Graph]
            J[Telemetry System]
        end

        subgraph "Processing"
            K[Embedding Pipeline]
            L[Entity Recognition]
            M[Background Scheduler]
            N[Celery Tasks]
        end
    end

    subgraph "Storage Layer"
        O[(Qdrant Vector DB)]
        P[(Redis Cache)]
        Q[(PostgreSQL)]
    end

    subgraph "External Services"
        R[OpenAI/Anthropic/Groq]
        S[Sentence Transformers]
        T[Prometheus]
    end

    A --> D
    B --> E
    C --> F

    D --> G
    E --> G
    F --> G

    G --> H
    G --> I
    G --> J

    H --> K
    I --> L
    J --> T

    G --> M
    M --> N

    K --> O
    H --> P
    I --> Q

    K --> R
    K --> S

    style A fill:#e1f5ff
    style B fill:#e1f5ff
    style C fill:#e1f5ff
    style O fill:#ffe1e1
    style P fill:#ffe1e1
    style Q fill:#ffe1e1
    style R fill:#fff4e1
    style S fill:#fff4e1
Loading

Data Flow Architecture

sequenceDiagram
    participant Client
    participant MemoryClient
    participant Cache as Redis Cache
    participant Vector as Qdrant Vector DB
    participant LLM as LLM Provider
    participant Tasks as Background Tasks

    Client->>MemoryClient: remember(text, user_id)

    activate MemoryClient
    MemoryClient->>LLM: Generate embedding
    LLM-->>MemoryClient: Embedding vector

    MemoryClient->>Vector: Store memory + vector
    Vector-->>MemoryClient: Memory ID

    MemoryClient->>Cache: Cache metadata

    MemoryClient->>Tasks: Schedule consolidation

    MemoryClient-->>Client: Memory object
    deactivate MemoryClient

    Note over Client,MemoryClient: Query Flow

    Client->>MemoryClient: recall(query, user_id)
    activate MemoryClient

    MemoryClient->>Cache: Check cache
    alt Cache hit
        Cache-->>MemoryClient: Cached results
    else Cache miss
        MemoryClient->>LLM: Generate query embedding
        LLM-->>MemoryClient: Query vector

        MemoryClient->>Vector: Semantic search
        Vector-->>MemoryClient: Top candidates

        MemoryClient->>MemoryClient: BM25 + Reranking
        MemoryClient->>Cache: Cache results
    end

    MemoryClient-->>Client: Retrieval results
    deactivate MemoryClient
Loading

Component Architecture

graph LR
    subgraph "Client Layer"
        MC[MemoryClient<br/>Synchronous API]
        AC[AsyncMemoryClient<br/>Async/Await API]
        UC[UnifiedMemoryClient<br/>Local + Remote]
    end

    subgraph "Core Services"
        MS[Memory Service<br/>CRUD Operations]
        RS[Retrieval Service<br/>Hybrid Search]
        GS[Graph Service<br/>Knowledge Graph]
        TS[Telemetry Service<br/>Monitoring]
    end

    subgraph "Background Processing"
        BT[Background Tasks<br/>Celery]
        SCH[Scheduler<br/>APScheduler]
        CON[Consolidation<br/>Memory Merging]
        DUP[Deduplication<br/>Similarity Detection]
    end

    subgraph "Data Stores"
        VDB[(Vector DB<br/>Qdrant)]
        RC[(Cache<br/>Redis)]
        PG[(Metadata<br/>PostgreSQL)]
    end

    MC --> MS
    AC --> MS
    UC --> MS

    MS --> RS
    MS --> GS
    MS --> TS

    RS --> VDB
    RS --> RC

    MS --> BT
    BT --> SCH
    SCH --> CON
    SCH --> DUP

    CON --> VDB
    DUP --> VDB

    GS --> PG

    style MC fill:#4CAF50,color:#fff
    style AC fill:#2196F3,color:#fff
    style UC fill:#FF9800,color:#fff
    style VDB fill:#f44336,color:#fff
    style RC fill:#FF5722,color:#fff
    style PG fill:#9C27B0,color:#fff
Loading

Deployment Architecture

graph TB
    subgraph "Production Environment"
        subgraph "Load Balancer"
            LB[Nginx/Traefik]
        end

        subgraph "Application Tier"
            API1[FastAPI<br/>Instance 1]
            API2[FastAPI<br/>Instance 2]
            API3[FastAPI<br/>Instance 3]
        end

        subgraph "Worker Tier"
            W1[Celery Worker 1<br/>Memory Tasks]
            W2[Celery Worker 2<br/>Background Jobs]
            W3[Celery Worker 3<br/>Scheduled Tasks]
        end

        subgraph "Data Tier"
            QDRANT[(Qdrant Cluster<br/>3 Nodes)]
            REDIS[(Redis Cluster<br/>Master + Replicas)]
            PG[(PostgreSQL<br/>Primary + Standby)]
        end

        subgraph "Monitoring"
            PROM[Prometheus]
            GRAF[Grafana]
            ALERT[Alertmanager]
        end
    end

    LB --> API1
    LB --> API2
    LB --> API3

    API1 --> QDRANT
    API2 --> QDRANT
    API3 --> QDRANT

    API1 --> REDIS
    API2 --> REDIS
    API3 --> REDIS

    API1 --> W1
    API2 --> W2
    API3 --> W3

    W1 --> QDRANT
    W2 --> QDRANT
    W3 --> QDRANT

    W1 --> REDIS
    W2 --> REDIS
    W3 --> REDIS

    QDRANT --> PG

    API1 --> PROM
    API2 --> PROM
    API3 --> PROM

    PROM --> GRAF
    PROM --> ALERT

    style LB fill:#00BCD4
    style QDRANT fill:#f44336
    style REDIS fill:#FF5722
    style PG fill:#9C27B0
    style PROM fill:#FFC107
    style GRAF fill:#FF9800
Loading

Core Components (Text View)

┌─────────────────────────────────────────────────────────────┐
│                     HippocampAI Core                        │
├─────────────────────────────────────────────────────────────┤
│  Client Layer                                               │
│  ├── MemoryClient (Sync)      ├── AsyncMemoryClient        │
│  ├── UnifiedMemoryClient      └── CLI Interface             │
├─────────────────────────────────────────────────────────────┤
│  Business Logic                                             │
│  ├── Memory Operations        ├── Hybrid Retrieval         │
│  ├── Graph Processing         ├── Conversation Extraction   │
│  ├── Version Control          └── Telemetry System          │
├─────────────────────────────────────────────────────────────┤
│  Processing Layer                                           │
│  ├── Embedding Pipeline       ├── Semantic Clustering      │
│  ├── Entity Recognition       ├── Temporal Reasoning       │
│  └── Background Scheduler     └── Background Tasks         │
├─────────────────────────────────────────────────────────────┤
│  Storage Layer                                              │
│  ├── Vector Store (Qdrant)    ├── Redis Cache              │
│  ├── Graph Store              └── Session Management       │
├─────────────────────────────────────────────────────────────┤
│  External Services                                          │
│  ├── LLM Providers            ├── Embedding Models         │
│  └── Reranking Models         └── Background Queue         │
└─────────────────────────────────────────────────────────────┘

Type Safety Architecture

HippocampAI implements comprehensive type safety through several layers:

1. Type-Safe Wrappers for External Libraries

# SchedulerWrapper - Type-safe APScheduler integration
class SchedulerWrapper:
    def add_job(
        self, 
        func: Callable[[], None], 
        cron_expression: str, 
        job_id: str, 
        name: str,
        replace_existing: bool = True
    ) -> bool:
        """Add job with type safety and error handling."""
        if not self.is_available():
            return False
        try:
            trigger = self._cron_trigger_class.from_crontab(cron_expression)
            self._scheduler.add_job(
                func, trigger=trigger, id=job_id, name=name,
                replace_existing=replace_existing
            )
            return True
        except Exception:
            return False

2. Pylance Configuration

{
  "reportMissingTypeStubs": "none",
  "reportUnknownMemberType": "none",
  "reportUnknownParameterType": "none"
}

This approach ensures type safety for internal code while gracefully handling external libraries without type stubs.

Processing Architecture

❌ Separate codebases requiring different learning curves:

Library Users              API Users
├── Learn MemoryClient     ├── Learn HTTP endpoints
├── Python-only            ├── Any language
└── Different API          └── Different API

Our Architecture (What We DO)

✅ Single library, multiple backends:

All Users
├── Learn UnifiedMemoryClient (same API!)
├── Choose backend: local or remote
└── Switch between modes with ONE parameter

Local Mode                 Remote Mode
├── mode="local"           ├── mode="remote"
├── Direct connection      ├── HTTP connection
├── 5-15ms latency         ├── 20-50ms latency
└── Python only            └── Python + any language via API

Benefits of This Approach

  1. Consistent API: Same remember(), recall(), etc. regardless of mode
  2. Easy Migration: Switch from local to remote without code changes
  3. Flexible Deployment: Start local, scale to remote when needed
  4. Best of Both Worlds: Use local for internal services, remote for external clients
  5. No Vendor Lock-in: Your code works with any backend

Architecture Layers

┌─────────────────────────────────────────────────────────────────┐
│                  Application Layer (Your Code)                   │
│                                                                   │
│              from hippocampai import UnifiedMemoryClient         │
│                                                                   │
│     client = UnifiedMemoryClient(mode="local" | "remote")       │
│                                                                   │
│          memory = client.remember(...)  # Same API!              │
│          results = client.recall(...)   # Same API!              │
└───────────────────────────┬───────────────────────────────────────┘
                            │
                ┌───────────┴───────────┐
                │                       │
                ▼                       ▼
┌──────────────────────────┐  ┌──────────────────────────┐
│    LOCAL MODE BACKEND    │  │   REMOTE MODE BACKEND    │
│                          │  │                          │
│  UnifiedMemoryClient     │  │  UnifiedMemoryClient     │
│  ↓                       │  │  ↓                       │
│  LocalBackend            │  │  RemoteBackend           │
│  (client.py)             │  │  (HTTP client)           │
│  ↓                       │  │  ↓                       │
│  Direct calls to:        │  │  HTTP POST/GET to:       │
│  • Qdrant               │  │  • localhost:8000/v1/*   │
│  • Redis                │  │     ↓                    │
│  • Ollama               │  │     FastAPI Server       │
│  • MemoryService        │  │     ↓                    │
│                          │  │     MemoryService        │
│                          │  │     ↓                    │
│                          │  │  • Qdrant                │
│                          │  │  • Redis                 │
│                          │  │  • Ollama                │
└──────────┬───────────────┘  └──────────┬───────────────┘
           │                             │
           └─────────────┬───────────────┘
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Service Layer                               │
│                 (Shared by Both Modes)                           │
│  ┌──────────────────────────────────────────────────────┐       │
│  │  MemoryManagementService                             │       │
│  │  (services/memory_service.py)                        │       │
│  │                                                       │       │
│  │  • create_memory()                                   │       │
│  │  • recall_memories()                                 │       │
│  │  • update_memory()                                   │       │
│  │  • deduplicate_memories()                            │       │
│  │  • consolidate_memories()                            │       │
│  └───────────────────────┬──────────────────────────────┘       │
└─────────────────────────────────────────────────────────────────┘
                            │
        ┌───────────────────┼───────────────────┐
        ▼                   ▼                   ▼
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│  Pipeline    │    │  Retrieval   │    │   Storage    │
│  Layer       │    │  Layer       │    │   Layer      │
├──────────────┤    ├──────────────┤    ├──────────────┤
│ • Fact       │    │ • Hybrid     │    │ • Qdrant     │
│   Extraction │    │   Search     │    │   Store      │
│ • Entity     │    │ • Reranking  │    │ • Redis      │
│   Recognition│    │ • BM25       │    │   Cache      │
│ • Relations  │    │ • RRF        │    │ • Vector     │
│ • Clustering │    │ • Scoring    │    │   Index      │
└──────────────┘    └──────────────┘    └──────────────┘
                            │
        ┌───────────────────┼───────────────────┐
        ▼                   ▼                   ▼
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│   Qdrant     │    │    Redis     │    │     LLM      │
│  Vector DB   │    │    Cache     │    │   Provider   │
└──────────────┘    └──────────────┘    └──────────────┘

Mode 1: Python Library

Installation

# From source (development)
cd HippocampAI
pip install -e .

# From PyPI (when published)
pip install hippocampai

Project Structure

your-project/
├── main.py
├── requirements.txt  # hippocampai==1.0.0
└── .env              # QDRANT_URL, REDIS_URL, etc.

Usage Example

# your-project/main.py
from hippocampai import MemoryClient

# Initialize client (connects to Qdrant, Redis directly)
client = MemoryClient()

# Use the library
memory = client.remember(
    text="User prefers dark mode",
    user_id="user123"
)

results = client.recall(
    query="UI preferences",
    user_id="user123"
)

print(results[0].memory.text)
# Output: User prefers dark mode

How It Works

Your Application Process
├── main.py
│   ├── import hippocampai
│   └── client = MemoryClient()
│       └── Directly accesses:
│           ├── Qdrant (localhost:6333)
│           ├── Redis (localhost:6379)
│           └── Ollama (localhost:11434)

Advantages:

  • ✅ No HTTP overhead
  • ✅ Maximum performance
  • ✅ Simple deployment (single process)
  • ✅ Full control over configuration
  • ✅ Can customize any component

Disadvantages:

  • ❌ Python-only
  • ❌ Runs in your application process
  • ❌ No multi-language support

Mode 2: REST API / SaaS

Deployment

# Start the FastAPI server
uvicorn hippocampai.api.async_app:app --host 0.0.0.0 --port 8000

# Or with Docker
docker-compose up -d

Server Architecture

FastAPI Server Process (Port 8000)
├── hippocampai.api.async_app
│   ├── POST /v1/memories
│   ├── POST /v1/memories/recall
│   ├── GET /v1/memories
│   └── Routes call →
│       └── MemoryManagementService
│           └── Same service layer as Mode 1
│               ├── Qdrant (localhost:6333)
│               ├── Redis (localhost:6379)
│               └── Ollama (localhost:11434)

Usage from Any Language

Python:

import requests

response = requests.post(
    "http://localhost:8000/v1/memories",
    json={
        "text": "User prefers dark mode",
        "user_id": "user123"
    }
)
memory = response.json()

JavaScript:

const response = await fetch('http://localhost:8000/v1/memories', {
    method: 'POST',
    headers: {'Content-Type': 'application/json'},
    body: JSON.stringify({
        text: 'User prefers dark mode',
        user_id: 'user123'
    })
});
const memory = await response.json();

cURL:

curl -X POST http://localhost:8000/v1/memories \
  -H "Content-Type: application/json" \
  -d '{"text": "User prefers dark mode", "user_id": "user123"}'

Go:

import (
    "bytes"
    "encoding/json"
    "net/http"
)

payload := map[string]interface{}{
    "text": "User prefers dark mode",
    "user_id": "user123",
}
body, _ := json.Marshal(payload)
resp, _ := http.Post(
    "http://localhost:8000/v1/memories",
    "application/json",
    bytes.NewBuffer(body),
)

Advantages:

  • ✅ Multi-language support
  • ✅ Microservices architecture
  • ✅ Centralized deployment
  • ✅ Easy horizontal scaling
  • ✅ True SaaS multi-tenancy

Disadvantages:

  • ❌ HTTP overhead
  • ❌ Network latency
  • ❌ Additional deployment complexity

Hybrid Mode: Best of Both Worlds

You can use both modes in the same deployment:

┌─────────────────────────────────────┐
│  Internal Python Services           │
│  ├── Use Mode 1 (Library)           │
│  │   └── Direct import              │
│  └── Max performance                │
└──────────────┬──────────────────────┘
               │
               ├── Shared Qdrant/Redis
               │
┌──────────────▼──────────────────────┐
│  External Clients                   │
│  ├── Use Mode 2 (REST API)          │
│  │   └── HTTP requests              │
│  └── Multi-language support         │
└─────────────────────────────────────┘

Example:

# Internal Python service (high performance)
from hippocampai import MemoryClient
client = MemoryClient()
client.remember(...)  # Direct library call

# External web app (JavaScript)
fetch('http://api.hippocampai.com/v1/memories', {...})  # HTTP API

Code Organization

Core Library (src/hippocampai/)

src/hippocampai/
├── __init__.py              # Exports MemoryClient
├── client.py                # Mode 1: Python library interface
├── api/
│   ├── async_app.py         # Mode 2: FastAPI application
│   ├── deps.py              # Dependency injection
│   └── intelligence_routes.py
├── services/
│   └── memory_service.py    # Shared business logic
├── models/
│   ├── memory.py            # Data models
│   └── retrieval.py
├── pipeline/
│   ├── fact_extraction.py   # Intelligence features
│   ├── entity_recognition.py
│   └── consolidate.py
├── storage/
│   ├── redis_store.py       # Storage adapters
│   └── qdrant_store.py
└── adapters/
    ├── provider_ollama.py   # LLM adapters
    ├── provider_openai.py
    └── provider_groq.py

Key Point: Shared Services

Both modes use the same MemoryManagementService:

# Mode 1 (Library)
from hippocampai import MemoryClient
client = MemoryClient()
# Internally creates: MemoryManagementService(...)

# Mode 2 (API)
# FastAPI route handler
@app.post("/v1/memories")
async def create_memory(request: CreateMemoryRequest):
    service = get_memory_service()  # Same MemoryManagementService
    return await service.create_memory(...)

Deployment Patterns

Pattern 1: Library-Only (Embedded)

┌─────────────────────────────┐
│   Your Application          │
│   ├── app.py                │
│   │   └── import hippocampai│
│   └── Dependencies:         │
│       ├── Qdrant (Docker)   │
│       ├── Redis (Docker)    │
│       └── Ollama (Docker)   │
└─────────────────────────────┘

Use case: Single Python application, maximum performance

Pattern 2: API-Only (Microservice)

┌─────────────────────────────┐
│   HippocampAI Server        │
│   ├── FastAPI (Port 8000)   │
│   └── Dependencies:         │
│       ├── Qdrant            │
│       ├── Redis             │
│       └── Ollama            │
└──────────┬──────────────────┘
           │
    ┌──────┴──────┬──────────┐
    ▼             ▼          ▼
┌────────┐  ┌────────┐  ┌────────┐
│Web App │  │Mobile  │  │ Go API │
│(JS)    │  │(Swift) │  │ Server │
└────────┘  └────────┘  └────────┘

Use case: Multi-language clients, centralized SaaS

Pattern 3: Hybrid (Best Practice)

┌─────────────────────────────┐
│   Internal Services         │
│   ├── Python Service 1      │
│   │   └── Direct import     │
│   └── Python Service 2      │
│       └── Direct import     │
└──────────┬──────────────────┘
           │ Same DB
┌──────────▼──────────────────┐
│   HippocampAI API Server    │
│   └── For external clients  │
└──────────┬──────────────────┘
           │
    ┌──────┴──────┐
    ▼             ▼
┌────────┐  ┌────────┐
│Web App │  │Partner │
│        │  │API     │
└────────┘  └────────┘

Use case: Mix of internal Python services + external clients


Configuration

Both modes share the same configuration:

# .env file (used by both modes)
QDRANT_URL=http://localhost:6333
REDIS_URL=redis://localhost:6379
LLM_PROVIDER=ollama
LLM_BASE_URL=http://localhost:11434

Mode 1 (Library):

from hippocampai import MemoryClient

# Uses .env automatically
client = MemoryClient()

# Or explicit configuration
client = MemoryClient(
    qdrant_url="http://custom-qdrant:6333",
    redis_url="redis://custom-redis:6379"
)

Mode 2 (API):

# Server reads from .env on startup
uvicorn hippocampai.api.async_app:app

# Or via environment variables
QDRANT_URL=http://prod-qdrant:6333 \
REDIS_URL=redis://prod-redis:6379 \
uvicorn hippocampai.api.async_app:app

Installation Paths

Development (Editable Install)

git clone https://github.com/rexdivakar/HippocampAI.git
cd HippocampAI
pip install -e .
  • Code lives in src/hippocampai/
  • Changes immediately reflected
  • Can use both Mode 1 and Mode 2

Production (PyPI Install - Future)

pip install hippocampai
  • Package installed in site-packages/
  • Fixed version
  • Can use Mode 1 (library)
  • For Mode 2, need to run server separately

Performance Comparison

Aspect Mode 1 (Library) Mode 2 (API)
Latency ~5-15ms ~20-50ms (+ network)
Throughput ~100-200 ops/sec ~50-100 ops/sec
Language Support Python only Any language
Deployment Single process Separate server
Scaling Vertical only Horizontal + Vertical
Multi-tenancy Application-level Built-in

When to Use Which Mode?

Use Mode 1 (Library) When

  • ✅ You're building a Python application
  • ✅ Performance is critical
  • ✅ You want maximum control
  • ✅ Single-tenant deployment
  • ✅ Embedding in existing app

Use Mode 2 (API) When

  • ✅ Multi-language clients
  • ✅ Microservices architecture
  • ✅ True SaaS deployment
  • ✅ Multi-tenant requirements
  • ✅ Centralized memory service

Use Hybrid When

  • ✅ Internal + external clients
  • ✅ Need both performance and flexibility
  • ✅ Gradual migration
  • ✅ Enterprise deployment

Summary

Key Takeaway: HippocampAI is ONE codebase with TWO interfaces:

  1. MemoryClient (Python library) - Direct import
  2. FastAPI App (REST API) - HTTP interface

Both use the same underlying services - there is no external dependency. You choose the consumption mode based on your needs.

src/hippocampai/  ← This IS the product
├── client.py     ← Interface for Mode 1
├── api/          ← Interface for Mode 2
└── services/     ← Shared by both modes

Not like this:

❌ hippocampai-server (depends on) → hippocampai-library

But like this:

✅ hippocampai (single codebase)
   ├── Can be used as library
   └── Can be used as API server