Skip to content

PROJECT_DAG.md

Latest commit

 

History

History
527 lines (456 loc) · 19.1 KB

PROJECT_DAG.md

File metadata and controls

527 lines (456 loc) · 19.1 KB

Synap Project DAG (Directed Acyclic Graph)

Overview

This document provides a visual representation of component dependencies and implementation order for the Synap project. Each node represents a component or feature, and edges represent dependencies.

Dependency Graph

Legend

  • [✓] Completed (documentation)
  • [ ] Not started
  • -> Depends on
  • => Strong dependency (blocking)
  • ~~> Weak dependency (optional)

Phase 1: Foundation

                    ┌──────────────────┐
                    │  Project Setup   │ [✓]
                    │  - Repo          │
                    │  - CI/CD         │
                    │  - Standards     │
                    └────────┬─────────┘
                             │
                ┌────────────┴────────────┐
                │                         │
                ▼                         ▼
      ┌──────────────────┐    ┌──────────────────┐
      │  Core Types      │    │  Error Handling  │ [ ]
      │  - Enums         │    │  - Result<T>     │
      │  - Structs       │    │  - Error types   │
      └────────┬─────────┘    └────────┬─────────┘
               │                       │
               └───────────┬───────────┘
                           │
                           ▼
                ┌──────────────────────┐
                │  Radix Tree          │ [ ]
                │  - radix_trie crate  │
                │  - CRUD operations   │
                │  - Prefix search     │
                └──────────┬───────────┘
                           │
                           ▼
                ┌──────────────────────┐
                │  Memory Manager      │ [ ]
                │  - Allocation        │
                │  - Eviction (LRU)    │
                │  - TTL tracking      │
                └──────────┬───────────┘
                           │
                           ▼
                ┌──────────────────────┐
                │  Key-Value Store     │ [ ]
                │  - GET/SET/DELETE    │
                │  - Atomic ops        │
                │  - Batch ops         │
                └──────────┬───────────┘
                           │
                  ┌────────┴────────┐
                  │                 │
                  ▼                 ▼
        ┌──────────────────┐  ┌──────────────────┐
        │  HTTP Server     │  │  Router          │ [ ]
        │  - Axum setup    │  │  - Routes        │
        │  - Middleware    │  │  - Handlers      │
        └────────┬─────────┘  └────────┬─────────┘
                 │                     │
                 └──────────┬──────────┘
                            │
                            ▼
                 ┌──────────────────────┐
                 │  StreamableHTTP      │ [ ]
                 │  - Protocol impl     │
                 │  - Message envelope  │
                 └──────────────────────┘

Phase 2: Core Features

     ┌──────────────────────┐
     │  Key-Value Store     │ [from Phase 1]
     │  (Foundation)        │
     └──────────┬───────────┘
                │
      ┌─────────┴─────────────────────────┐
      │                                   │
      ▼                                   ▼
┌──────────────────┐            ┌──────────────────┐
│  Queue System    │            │  Event Streams   │ [ ]
│  - FIFO queue    │            │  - Ring buffer   │
│  - Priorities    │            │  - Rooms         │
│  - ACK/NACK      │            │  - History       │
│  - Retry logic   │            │  - Offsets       │
│  - DLQ           │            │  - Compaction    │
└────────┬─────────┘            └────────┬─────────┘
         │                               │
         │        ┌──────────────────┐   │
         └───────>│  Pub/Sub Router  │<──┘ [ ]
                  │  - Topics        │
                  │  - Wildcards     │
                  │  - Fan-out       │
                  │  - Hierarchies   │
                  └────────┬─────────┘
                           │
                  ┌────────┴─────────┐
                  │                  │
                  ▼                  ▼
        ┌──────────────────┐  ┌──────────────────┐
        │  Persistence     │  │  WebSocket       │ [ ]
        │  - WAL           │  │  - Upgrade path  │
        │  - Snapshots     │  │  - Streaming     │
        │  - Recovery      │  │  - Pub/Sub       │
        └──────────────────┘  └──────────────────┘

Phase 3: Advanced Features

     ┌──────────────────────┐
     │  All Core Features   │ [from Phase 2]
     │  (KV, Queue, Stream) │
     └──────────┬───────────┘
                │
      ┌─────────┴──────────────────────────┐
      │                                    │
      ▼                                    ▼
┌──────────────────┐            ┌──────────────────┐
│  Replication     │            │  Compression     │ [ ]
│  - Master node   │            │  - LZ4 (fast)    │
│  - Replica nodes │            │  - Zstd (ratio)  │
│  - Append log    │            │  - Auto-detect   │
│  - Sync protocol │            │  - Content-aware │
│  - Lag monitor   │            └────────┬─────────┘
└────────┬─────────┘                     │
         │                               │
         │         ┌──────────────────┐  │
         └────────>│  Cache System    │<─┘ [ ]
                   │  - L1 hot cache  │
                   │  - L2 warm cache │
                   │  - Adaptive TTL  │
                   │  - Promotion     │
                   └────────┬─────────┘
                            │
                   ┌────────┴────────┐
                   │                 │
                   ▼                 ▼
         ┌──────────────────┐  ┌──────────────────┐
         │  MCP Protocol    │  │  UMICP Protocol  │ [ ]
         │  - Resources     │  │  - Matrix ops    │
         │  - Tools         │  │  - Vector ops    │
         │  - Prompts       │  │  - Envelopes     │
         │  - AI integration│  │  - Similarity    │
         └──────────────────┘  └────────┬─────────┘
                                        │
                              ┌─────────┴─────────┐
                              │                   │
                              ▼                   ▼
                    ┌──────────────────┐  ┌──────────────────┐
                    │  Monitoring      │  │  Metrics         │ [ ]
                    │  - Prometheus    │  │  - Stats         │
                    │  - Health checks │  │  - Counters      │
                    │  - Tracing       │  │  - Histograms    │
                    └──────────────────┘  └──────────────────┘

Phase 4: Production Ready

     ┌──────────────────────┐
     │  All Advanced        │ [from Phase 3]
     │  Features            │
     └──────────┬───────────┘
                │
      ┌─────────┴──────────────────────────┐
      │                                    │
      ▼                                    ▼
┌──────────────────┐            ┌──────────────────┐
│  Security        │            │  Packaging       │ [ ]
│  - Auth          │            │  - MSI (Windows) │
│  - API keys      │            │  - DEB (Linux)   │
│  - RBAC          │            │  - Brew (macOS)  │
│  - TLS/SSL       │            │  - Docker        │
│  - Rate limiting │            │  - Helm charts   │
└────────┬─────────┘            └────────┬─────────┘
         │                               │
         │         ┌──────────────────┐  │
         └────────>│  GUI Dashboard   │<─┘ [ ]
                   │  - Electron      │
                   │  - Vue.js 3      │
                   │  - Charts        │
                   │  - Config editor │
                   │  - Log viewer    │
                   └────────┬─────────┘
                            │
                   ┌────────┴────────┐
                   │                 │
                   ▼                 ▼
         ┌──────────────────┐  ┌──────────────────┐
         │  Documentation   │  │  Testing Suite   │ [ ]
         │  - User guide    │  │  - Load tests    │
         │  - Admin guide   │  │  - Stress tests  │
         │  - API docs      │  │  - Chaos tests   │
         │  - Tutorials     │  │  - Security scan │
         └──────────────────┘  └──────────────────┘

Phase 5: Scale & Optimize

     ┌──────────────────────┐
     │  Production System   │ [from Phase 4]
     │  (v1.0.0)            │
     └──────────┬───────────┘
                │
      ┌─────────┴──────────────────────────┐
      │                                    │
      ▼                                    ▼
┌──────────────────┐            ┌──────────────────┐
│  Raft Consensus  │            │  Sharding        │ [ ]
│  - Leader elect  │            │  - Hash-based    │
│  - Log replication│           │  - Range-based   │
│  - Quorum        │            │  - Rebalancing   │
│  - Auto failover │            │  - Cross-shard   │
└────────┬─────────┘            └────────┬─────────┘
         │                               │
         │         ┌──────────────────┐  │
         └────────>│  Clustering      │<─┘ [ ]
                   │  - Multi-master  │
                   │  - Split-brain   │
                   │  - Health check  │
                   │  - Discovery     │
                   └────────┬─────────┘
                            │
                   ┌────────┴────────┐
                   │                 │
                   ▼                 ▼
         ┌──────────────────┐  ┌──────────────────┐
         │  Geo-Replication │  │  Advanced        │ [ ]
         │  - Cross-DC sync │  │  Analytics       │
         │  - Conflict res. │  │  - Query engine  │
         │  - Regional fail.│  │  - Time-series   │
         └──────────────────┘  └──────────────────┘

Component Dependency Matrix

Component Depends On Required By Phase
Core Types - All components 1
Error Handling Core Types All components 1
Radix Tree Core Types KV Store 1
Memory Manager Radix Tree KV Store 1
KV Store Memory Manager Queue, Streams, Cache 1
HTTP Server KV Store All protocols 1
StreamableHTTP HTTP Server - 1
Queue System KV Store Pub/Sub 2
Event Streams KV Store Pub/Sub 2
Pub/Sub Queue, Streams WebSocket 2
Persistence All core Replication 2
WebSocket Pub/Sub GUI 2
Replication Persistence Clustering 3
Compression All core Cache 3
Cache Compression - 3
MCP HTTP Server GUI 3
UMICP HTTP Server - 3
Monitoring All GUI 3
Security HTTP Server Production 4
Packaging All Distribution 4
GUI WebSocket, MCP - 4
Documentation All Release 4
Raft Replication Clustering 5
Sharding Clustering Geo-replication 5
Clustering Raft, Sharding - 5

Critical Path Analysis

Longest Dependencies Chain

Project Setup
    ↓
Core Types
    ↓
Error Handling
    ↓
Radix Tree
    ↓
Memory Manager
    ↓
Key-Value Store
    ↓
HTTP Server
    ↓
Queue System
    ↓
Persistence
    ↓
Replication
    ↓
Compression
    ↓
Cache System
    ↓
Security
    ↓
Packaging
    ↓
v1.0.0 Release

Critical Path Duration: ~40 weeks (10 months)

Parallel Development Opportunities

Can Be Developed in Parallel

  1. Phase 1:

    • HTTP Server + Radix Tree (after Core Types)
    • Router + Error Handling

  2. Phase 2:

    • Queue System + Event Streams (after KV Store)
    • Persistence + WebSocket

  3. Phase 3:

    • Replication + Compression (after Persistence)
    • MCP + UMICP (after HTTP Server)
    • Monitoring (independent)

  4. Phase 4:

    • Security + Packaging (parallel)
    • GUI + Documentation (parallel)
    • Testing (continuous)

Risk Dependencies

High-Risk Dependencies

These components have many dependents. Delays here impact multiple features:

  1. Radix Tree → Blocks: KV Store, Queue, Streams
  2. HTTP Server → Blocks: All protocols, GUI
  3. Persistence → Blocks: Replication, Production readiness
  4. Replication → Blocks: Clustering, Geo-replication

Mitigation Strategies

  • Start high-risk components early
  • Allocate extra time for critical path
  • Create abstract interfaces first
  • Implement mocks for dependent teams
  • Regular integration testing

Development Workflow

Sequential Dependencies

Week 1-2:   Project Setup
Week 3-4:   Core Types + Error Handling
Week 5-6:   Radix Tree
Week 7-8:   Memory Manager
Week 9-10:  Key-Value Store
Week 11-12: HTTP Server + Router

Parallel Work (Example - Phase 2)

Team A: Queue System (Weeks 13-16)
Team B: Event Streams (Weeks 13-16)
Team C: Persistence (Weeks 17-20)
Team D: WebSocket (Weeks 17-20)

Testing Dependencies

Unit Tests
    ↓
Integration Tests
    ↓
System Tests
    ↓
Performance Tests
    ↓
Load Tests
    ↓
Chaos Tests
    ↓
Security Tests
    ↓
Production Release

Documentation Dependencies

Code Implementation
    ↓
API Documentation
    ↓
User Guide
    ↓
Admin Guide
    ↓
Tutorials
    ↓
Release Notes

Deployment Dependencies

Core Features
    ↓
Packaging (MSI, DEB, Brew)
    ↓
Docker Images
    ↓
Helm Charts
    ↓
Cloud Marketplace
    ↓
Production Deployment

Version Dependencies

v0.1.0 (Alpha)

  • Core Types ✓
  • Radix Tree
  • KV Store
  • HTTP Server
  • StreamableHTTP

v0.2.0 (Beta)

  • v0.1.0 +
  • Queue System
  • Event Streams
  • Pub/Sub
  • Persistence

v0.3.0 (RC)

  • v0.2.0 +
  • Replication
  • Compression
  • Cache
  • MCP
  • UMICP

v1.0.0 (Production)

  • v0.3.0 +
  • Security
  • Packaging
  • GUI
  • Documentation

v1.5.0 (Scale)

  • v1.0.0 +
  • Raft
  • Clustering
  • Sharding
  • Geo-replication

Quick Reference

Immediate Next Steps

  1. ✓ Complete documentation
  2. Setup CI/CD pipeline
  3. Implement Core Types
  4. Build Radix Tree
  5. Create Memory Manager

Blocking Issues

  • None currently (documentation phase)

Ready to Implement

  • Core Types (no dependencies)
  • Error Handling (minimal dependencies)
  • Project infrastructure

References

Last Updated: October 16, 2025
Phase: Documentation Complete
Next Milestone: Start Phase 1 Implementation