From a56ada66f2c4e6b5e07a858e52a134c9da8c3802 Mon Sep 17 00:00:00 2001 From: Berry Wahlberg <40695099+BerryUIKI@users.noreply.github.com> Date: Wed, 23 Sep 2026 17:11:35 +0800 Subject: [PATCH] docs(handoff): finalize M0-M6 delivery handoff and verification boundaries --- README.md | 148 +++---------- docs/ARCHITECTURE.md | 295 ++++++++++---------------- docs/CODING_AGENT_HANDOFF.md | 72 +++++++ docs/DELIVERY_HANDOFF_M0_M6.md | 189 +++++++++++++++++ docs/LAUNCHER_MANAGER_REQUIREMENTS.md | 95 +++++++++ docs/PRD.md | 74 +++++++ docs/PRODUCT_ALIGNMENT.md | 69 ++++++ docs/PRODUCT_VISION.md | 46 ++++ docs/RELEASE_VERIFICATION_REPORT.md | 83 ++++++++ docs/ROADMAP.md | 108 ++++++---- docs/SUPPORT_MATRIX.md | 67 ++++++ docs/USER_GUIDE.md | 105 +++++++++ 12 files changed, 1000 insertions(+), 351 deletions(-) create mode 100644 docs/CODING_AGENT_HANDOFF.md create mode 100644 docs/DELIVERY_HANDOFF_M0_M6.md create mode 100644 docs/LAUNCHER_MANAGER_REQUIREMENTS.md create mode 100644 docs/PRD.md create mode 100644 docs/PRODUCT_ALIGNMENT.md create mode 100644 docs/PRODUCT_VISION.md create mode 100644 docs/RELEASE_VERIFICATION_REPORT.md create mode 100644 docs/SUPPORT_MATRIX.md create mode 100644 docs/USER_GUIDE.md diff --git a/README.md b/README.md index 767c2b5..c99df63 100644 --- a/README.md +++ b/README.md @@ -1,132 +1,46 @@ -# AI-Workflow - -
- -

A Lightweight, API-First Visual Canvas for Multimodal AI Workflows

- -

- Escape the spaghetti wires. Create powerful text, image, audio, and video pipelines with high-level semantic nodes, smart caching, and zero hardware barriers. -

- -[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) -[![Branch: dev](https://img.shields.io/badge/integration_branch-dev-green.svg)](https://github.com/BerryUIKI/AI-Workflow/tree/dev) -[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/) -[![React 18+](https://img.shields.io/badge/react-18+-61dafb.svg)](https://react.dev/) -[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md) - -
- ---- - -## 🌟 Why AI-Workflow? +# Berry AI Studio -ComfyUI is remarkably capable, but it was designed as a low-level PyTorch tensor assembly bench. Users are forced to manually wire Latents, VAEs, CLIP encodings, and Samplers just to generate a picture, requiring expensive 16GB+ GPUs and complex CUDA installations. - -**AI-Workflow** provides a modern, intuitive alternative: - -| Feature | ComfyUI | AI-Workflow | -| :--- | :--- | :--- | -| **User Interface** | Low-level tensor pipelines (10+ wires per step) | **High-level semantic nodes** (Prompt, Generate, Voice, Video) | -| **Connection Types** | Dozens of complex types (Model, Clip, Vae, Latent) | **5 Universal Types** (`string`, `image`, `audio`, `video`, `json`) | -| **Hardware Barrier** | Requires high-end GPU & CUDA setups | **Zero hardware barrier** (runs anywhere via Cloud APIs) | -| **Local Model Support** | Native, but prone to environment conflicts | **Optional & Sandboxed** (isolated runtime, zero system pollution) | -| **API Cost Optimization**| Re-executes from change | **Deterministic Dirty-Check Caching** (prevents redundant API spend) | -| **Installation Size** | 20 GB – 100 GB+ | **Lightweight Core** (< 100 MB) | +A beginner-friendly AI creative workspace combining an infinite canvas, local generation engines, cloud creation, model management, and persistent projects. ---- +## Project Status -## πŸš€ Key Highlights +The product direction and first-release scope are approved. The repository contains an early prototype; the target capabilities below are not claims of completed implementation. See the documentation and milestone acceptance gates before assessing readiness. -- 🎨 **Figma-Grade Visual Canvas**: Built on modern web technologies with fluid zooming, smooth pan, and instant reactivity. -- ⚑ **API-First by Default**: Plug in your OpenAI, DeepSeek, Fal.ai, or SiliconFlow API keys and start creating immediately. -- πŸ’° **Smart Dirty-Check Caching**: Modify prompt #3 without re-paying for prompt #1 and #2. The engine only executes modified nodes. -- 🧩 **Isolated & Sandboxed ComfyUI Engine**: Want local GPU acceleration? Run or install an embedded, self-contained ComfyUI runtime directly through the app **without polluting your host operating system**. -- πŸ”„ **Macro Subgraph Compilation**: High-level nodes automatically compile into ComfyUI's underlying execution graphs behind the scenes. -- πŸ“¦ **Export & Share**: Share workflows as portable `.flow.json` files with your team. +## First-Release Scope ---- +- Windows release with cross-platform code architecture. +- NVIDIA local generation and cloud-only creation with user-provided API keys. +- Optional isolated ComfyUI and Stable Diffusion WebUI installation and existing-engine connections. +- Berry's own infinite canvas and creation controls, with native engine entry points retained. +- Text-to-image, image-to-image, inpainting, and upscaling. +- Local model scanning/import, directory management, compatibility and dependency guidance. +- Persistent projects, assets, generation records, and reliable task execution. -## πŸ›οΈ Architecture Overview +Video, Agent features, clarified CLI integration, non-NVIDIA discrete GPU inference, and additional OS releases belong to later phases. -``` -AI-Workflow/ -β”œβ”€β”€ frontend/ # Vite + React 18 + TypeScript + @xyflow/react + Tailwind CSS -β”‚ └── src/ -β”‚ β”œβ”€β”€ components/canvas/ # FlowCanvas, WorkflowNode, NodePalette -β”‚ β”œβ”€β”€ stores/ # Zustand canvas & engine state -β”‚ └── types/ # TypeScript contracts (NodeDefinition, DataType, etc.) -β”œβ”€β”€ backend/ # FastAPI + Pydantic v2 + AsyncIO -β”‚ β”œβ”€β”€ app/ -β”‚ β”‚ β”œβ”€β”€ core/ # DAG Engine, SHA-256 Hash Cache, Runner Router -β”‚ β”‚ β”œβ”€β”€ nodes/ # Node registry & built-in node definitions -β”‚ β”‚ β”œβ”€β”€ runners/ # Cloud API Driver & ComfyUI WebSocket Bridge -β”‚ β”‚ β”œβ”€β”€ runtime/ # Isolated ComfyUI Supervisor & Sandboxed Installer -β”‚ β”‚ └── schemas/ # Pydantic models (NodeDefinition, WorkflowGraph, Events) -β”‚ └── tests/ # Pytest test suites (28 tests, 100% passing) -└── docs/ # Architecture spec, Roadmap, RFCs -``` +## Documentation -For complete technical details, see the [Architecture Document](docs/ARCHITECTURE.md). +| Document | Purpose | +| --- | --- | +| [Product Vision](docs/PRODUCT_VISION.md) | Approved positioning and product boundaries | +| [First-Release PRD](docs/PRD.md) | Requirements and release acceptance scenarios | +| [Architecture](docs/ARCHITECTURE.md) | Target technical design and invariants | +| [Roadmap](docs/ROADMAP.md) | Implementation milestones and evidence gates | +| [Product Alignment](docs/PRODUCT_ALIGNMENT.md) | Decision history and prototype inspection findings | +| [Coding Agent Handoff](docs/CODING_AGENT_HANDOFF.md) | Copyable implementation prompt | ---- +## Development Foundation -## 🚦 Quick Start (Development) +The prototype uses React, TypeScript, Vite, Tailwind, Zustand and React Flow on the frontend, and FastAPI/Pydantic on the backend. Generation engines remain optional and isolated from the lightweight core. -### Prerequisites -- Node.js `18+` and `pnpm` (`npm install -g pnpm`) -- Python `3.10+` +For development setup, see [CONTRIBUTING.md](CONTRIBUTING.md). The setup instructions require validation against the implementation baseline; no build or runtime verification was performed during product planning. -### 1. Clone & Switch to Integration Branch -```bash -git clone https://github.com/BerryUIKI/AI-Workflow.git -cd AI-Workflow -git checkout dev -``` +## Contribution Rules -### 2. Launch the Backend -```bash -cd backend -python -m venv .venv -source .venv/bin/activate # On Windows: .venv\Scripts\Activate.ps1 -pip install -r requirements.txt -uvicorn app.main:app --reload --port 8000 -``` +Branch from dev and target dev with pull requests. Never commit directly to main. Use English for all project-facing written artifacts and Conventional Commits for commit messages. -### 3. Launch the Frontend Canvas -```bash -cd ../frontend -pnpm install -pnpm approve-builds --all # Required on first install to allow esbuild native binaries -pnpm dev -``` -Open [http://localhost:5173](http://localhost:5173) in your browser. +Read [AGENTS.md](AGENTS.md), [Branching Strategy](BRANCHING_STRATEGY.md), [Contributing](CONTRIBUTING.md), and [Code of Conduct](CODE_OF_CONDUCT.md). ---- - -## 🌿 Branching & Contribution Governance - -To maintain production stability: -- **`main`**: Strictly protected release branch. Contains only stable, tagged releases (`vX.Y.Z`). -- **`dev`**: Active integration branch. **All PRs must target `dev`**. - -Please consult: -- [BRANCHING_STRATEGY.md](BRANCHING_STRATEGY.md) for branch models and commit standards. -- [CONTRIBUTING.md](CONTRIBUTING.md) for setup and pull request workflows. -- [AGENTS.md](AGENTS.md) for guidelines tailored to AI coding assistants. -- [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) for community guidelines. - ---- - -## πŸ—ΊοΈ Roadmap - -Check our [Roadmap](docs/ROADMAP.md) for upcoming milestones including: -- Cloud multimodal drivers (FLUX, ElevenLabs, Kling, Runway) -- In-app isolated ComfyUI engine installer -- Tauri desktop packaging -- Community workflow templates - ---- - -## πŸ“„ License - -Distributed under the **Apache License 2.0**. See [LICENSE](LICENSE) for more information. +## License + +Apache License 2.0. See [LICENSE](LICENSE). diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 915c0e3..daa66e8 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,187 +1,108 @@ -# System Architecture & Technical Specifications - -This document defines the architectural blueprints, execution pipelines, data contracts, and isolation models for **AI-Workflow**. - ---- - -## 1. Architectural Philosophy - -ComfyUI is a powerful node-based engine, but its UI exposes low-level model mechanics (CLIP, VAE, Latent vectors, samplers, schedules). This creates: -1. **The "Spaghetti Wire" problem**: Novice creators must connect 8–12 nodes just to render a single picture. -2. **Heavyweight Barriers**: Demands 16GB+ VRAM GPUs, CUDA installation, and tens of gigabytes of checkpoint weights. -3. **System Environment Pollution**: Conflicts between PyTorch versions, CUDA toolkits, and Python wheels often break the host environment. - -**AI-Workflow** solves these challenges by inverting the paradigm: -- **High-Level Semantic Nodes**: Users work with functional building blocks (e.g., "Script Expander", "FLUX Generator", "Voiceover TTS", "Background Remover"). -- **API-First by Default**: 100% lightweight, boots in milliseconds, runs on any laptop or server using cloud APIs. -- **Pluggable & Sandboxed Local Engine**: Local models (like ComfyUI) can be linked or installed directly via the app into an isolated, self-contained runtime that **never pollutes the host operating system**. - ---- - -## 2. Multi-Tier System Topology - -```mermaid -flowchart TD - subgraph Client ["Client / Presentation Layer"] - Canvas["@xyflow/react Canvas"] - Inspector["Node Inspector & Controls"] - Store["Zustand Store (Nodes, Edges, Cache)"] - end - - subgraph CoreEngine ["Backend Core (FastAPI / Python AsyncIO)"] - DAG["DAG Graph Resolver & TopoSort"] - Hasher["Dirty-Check & State Hashing Engine"] - CacheStore["Hash Cache Store (SQLite / In-Memory)"] - Router["Execution Runner Router"] - end - - subgraph Runners ["Pluggable Execution Drivers"] - APIRunner["Cloud API Driver
(OpenAI, DeepSeek, Fal.ai, SiliconFlow)"] - ComfyBridge["ComfyUI Runner Adapter
(WebSocket + Subgraph Compiler)"] - LocalTools["Lightweight Local Processors
(rembg, whisper, ffmpeg)"] - end - - subgraph SandboxedRuntime ["Sandboxed Local ComfyUI Engine (Optional)"] - Supervisor["Runtime Process Supervisor"] - IsolatedEnv["Isolated Virtualenv / uv Sandbox
(Zero Host Pollution)"] - ComfyInstance["Headless ComfyUI Server
(Port: 8188 / Dynamic)"] - end - - Canvas <-->|REST & WebSocket| CoreEngine - DAG --> Hasher - Hasher -->|Cache Miss| Router - Hasher -->|Cache Hit (0 token cost)| Store - Router --> APIRunner - Router --> ComfyBridge - Router --> LocalTools - ComfyBridge <-->|IPC / WS Protocol| ComfyInstance - Supervisor --> IsolatedEnv - Supervisor --> ComfyInstance -``` - ---- - -## 3. The 5 Universal Data Types - -To eliminate interface explosion, all node input and output ports are strictly typed into one of 5 universal primitives: - -| Type | Data Format | Description | -| :--- | :--- | :--- | -| **`string`** | UTF-8 String | Prompts, subtitles, formatted markdown, LLM responses. | -| **`image`** | URL or Base64 / URI | Image assets (PNG, WebP, JPEG). | -| **`audio`** | URL or Audio File URI | Voiceover recordings, sound effects, BGM. | -| **`video`** | URL or MP4/WebM URI | Video clips, animations. | -| **`json`** | Structured Object / Array | Batch variables, parameters, split outputs. | - -### Smart Auto-Casting -Nodes automatically cast compatible types: -- An `image` connected to a multimodal LLM prompt is interpreted as a visual reference frame. -- A `string` representing a valid image URL connected to an `image` port is treated as an image input. - ---- - -## 4. Deterministic Dirty-Check & Caching Engine - -API calls incur real financial cost. A creator who edits only the last step of a workflow should never re-pay for preceding steps. - -### Hash Derivation Formula -Every node's cache key is deterministically calculated as: -$$\text{NodeHash} = \text{SHA256}(\text{NodeType} + \text{SerializedParameters} + \sum_{p \in \text{Parents}} \text{ParentOutputHash}_p)$$ - -### Execution Cycle: -1. When a workflow run is requested (or single-node run), the DAG engine topologically sorts active nodes. -2. For each node, it computes the `NodeHash`. -3. If `CacheStore.has(NodeHash)`: - - Output data is pulled from cache instantly. - - Real-time event `NODE_CACHED` is broadcast to the canvas (rendered with a green status badge). - - Zero external API tokens consumed. -4. If `CacheStore.miss(NodeHash)`: - - The node is queued for execution via its designated runner. - - Status updates stream over WebSocket (`NODE_RUNNING` $\to$ `NODE_PROGRESS` $\to$ `NODE_COMPLETED`). - - The final output is committed to the cache store alongside its hash. - ---- - -## 5. Sandboxed & Isolated ComfyUI Runtime Engine - -Users can install and run ComfyUI directly through our app with **zero system environment pollution**. - -``` -Host OS (Clean, Unmodified) -└── User Data Directory (~/.ai-workflow/engine/) - β”œβ”€β”€ runtime/ <-- Standalone Python / uv virtualenv - β”‚ β”œβ”€β”€ bin/ (or Scripts/) - β”‚ └── lib/site-packages/ <-- PyTorch, CUDA bindings, Comfy packages - β”œβ”€β”€ comfyui/ <-- Isolated ComfyUI source clone / bundle - β”‚ β”œβ”€β”€ main.py - β”‚ β”œβ”€β”€ custom_nodes/ - β”‚ └── models/ <-- Checkpoints, LoRA, VAE - └── supervisor.pid <-- Managed process daemon -``` - -### Key Isolation Guarantees: -1. **Hermetic Virtual Environment**: - - The embedded ComfyUI runs within a dedicated isolated directory (e.g. `%LOCALAPPDATA%\AI-Workflow\engine` on Windows or `~/.ai-workflow/engine` on Linux/macOS). - - Packaged and managed using standalone `uv` or portable embedded Python packages. - - Never runs `sudo pip` or modifies global Python site-packages. -2. **Lifecycle Process Supervisor**: - - The backend process supervisor manages starting, stopping, health-checking, and port management (`localhost:8188` or dynamic fallback if 8188 is occupied). - - Graceful termination on application exit to avoid orphan GPU memory leaks. -3. **Dual Connection Modes**: - - **Mode A (External Bridge)**: If the user already has ComfyUI running, the app directly connects to `http://127.0.0.1:8188` without installing anything. - - **Mode B (In-App Sandboxed Installer)**: One-click setup managed via the Settings dialog, downloading only required runtime packages inside the sandbox. - ---- - -## 6. Macro Subgraph Compilation - -To keep the canvas clean, our high-level nodes automatically compile into ComfyUI's internal execution graph (the `/prompt` JSON format) behind the scenes. - -### Compilation Example: `Local Txt2Img Node` - -**What the user sees on our Canvas:** -```text -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ Local SDXL Txt2Img β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ ● in: prompt (string) β”‚ -β”‚ ● in: lora_name (optional string) β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ [Checkpoint: Juggernaut-XL-v9] β”‚ -β”‚ [Steps: 25 | CFG: 7.0] β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ ● out: image (image) β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -**Compiled ComfyUI Subgraph submitted via WebSocket:** -```mermaid -graph LR - Ckpt["Load Checkpoint"] --> CLIP["CLIP Text Encode (Positive)"] - Ckpt --> CLIPNeg["CLIP Text Encode (Negative)"] - Ckpt --> VAE["VAE Decode"] - EmptyLatent["Empty Latent Image"] --> Sampler["KSampler"] - CLIP --> Sampler - CLIPNeg --> Sampler - Ckpt --> Sampler - Sampler --> VAE - VAE --> SaveImg["Save Image / Output"] -``` - -The user never manages latent connections, conditioning wires, or VAE decoders. - ---- - -## 7. WebSocket Event Protocol - -All real-time updates between backend and frontend follow a typed JSON event schema: - -```typescript -type WorkflowEvent = - | { type: 'GRAPH_STARTED'; totalNodes: number } - | { type: 'NODE_STATUS'; nodeId: string; status: 'IDLE' | 'QUEUED' | 'RUNNING' | 'CACHED' | 'COMPLETED' | 'ERROR' } - | { type: 'NODE_PROGRESS'; nodeId: string; progress: number; message?: string } - | { type: 'NODE_OUTPUT'; nodeId: string; output: Record } - | { type: 'GRAPH_FINISHED'; executionTimeMs: number; tokensUsed?: number } - | { type: 'ENGINE_STATUS'; engine: 'cloud' | 'comfyui'; online: boolean; vramFreeMb?: number }; -``` +# Berry AI Studio Target Architecture + +Date: 2026-09-23. Status: implementation specification; not a description of completed code. +Product authority: [Vision](PRODUCT_VISION.md) and [PRD](PRD.md). + +## Boundaries + +Retain the existing React/TypeScript, Zustand, Tailwind and FastAPI foundation where useful. Refactor incrementally rather than rewriting the prototype indiscriminately. + +The Rust launcher owns Berry application bootstrap and presents the environment manager. The creative UI manages a canvas and high-level actions. The backend remains authoritative for projects, assets, model inventory, execution, secrets and managed-engine installation/runtime supervision. The launcher calls these services through a versioned local interface after bootstrap; engine and model logic must not be duplicated in Rust. Local engines and cloud providers are optional adapters. The core starts without GPU packages. + +API-first means a complete cloud-only path remains available. It does not diminish the approved NVIDIA/local installation experience. + +## Components + +| Component | Responsibility | +| --- | --- | +| Rust launcher/manager | Berry startup, readiness, model inventory presentation, managed engine controls and update orchestration | +| Creative workspace | Canvas objects, selection, contextual actions, parameters and status | +| Project service | Versioned project documents, layout, history and recovery | +| Asset service | Managed files, content hashes, metadata and export | +| Model catalog | User roots, model identities, engine compatibility and dependencies | +| Template registry | Curated workflow versions and high-level input/output mappings | +| Task service | Immutable submissions, scheduling, state, cancellation and events | +| Execution adapters | ComfyUI, WebUI and BYOK cloud capabilities and calls | +| Runtime service | Installations, readiness, managed process ownership and diagnostics | +| Platform adapters | Paths, hardware detection, process control and secure credentials | + +Keep the five execution port types: string, image, audio, video, json. Canvas presentation objects such as image cards and groups are not additional execution port types. Raw ComfyUI graph data stays inside templates/adapters. + +## Proposed Data Contracts + +Use versioned Pydantic schemas and matching strict TypeScript contracts. + +- Project: schema version, ID, name, canvas objects, asset references and history references. +- Asset: ID, media type, managed location or explicit external reference, content hash, dimensions and provenance. +- Model record: ID, file identity, locations, category, compatibility evidence and dependency status. +- Engine connection: ID, kind, version, endpoint, ownership, runtime reference and readiness. +- Template: ID/version, required engine capabilities, model/custom-node dependencies, parameters and input/output mappings. +- Task: ID, run ID, action, immutable inputs, resolved parameters, execution location, template/model versions, status and output references. +- Credential: opaque reference to protected storage; never an inline workflow parameter. +- Event: run/task ID, sequence, timestamp, typed state/progress/output/error payload. + +Validate endpoints and inputs at service boundaries. Persist schemas with explicit migration behavior; never silently reinterpret old projects. + +## Execution and Reliability + +1. Resolve an action and snapshot parameters, assets, template, model identity and execution location. +2. Validate required capabilities, engine readiness and dependencies. +3. Resolve upstream outputs, compute semantic cache identity, and check asset availability. +4. Queue or reuse results; dispatch asynchronously through the selected adapter. +5. Track actual engine/provider completion; retrieve and persist real outputs. +6. Atomically commit task outcome, asset records and cache entry; notify the UI. + +Do not fabricate output filenames. Do not mark a submitted job as completed. Failed prerequisites block dependent work; independent work may continue under a documented policy. + +Events must be correlated so late updates cannot overwrite a different run. Define reconnect/restart reconciliation for nonterminal tasks. An interrupted job with uncertain provider status must not be automatically resubmitted at potential duplicate cost. + +ComfyUI adapters must resolve real output metadata after completion. WebUI and provider adapters normalize actual responses. Read current official APIs before implementing version-specific behavior. + +## Cache Contract + +Preserve NodeHash = Hash(NodeType + Params + UpstreamOutputHashes). + +Canonical Params must include semantic execution settings: resolved seed, model identity/version, template/runner semantic version, provider identity where behavior differs, and input binding information. Upstream output hashes must identify actual content and port associations, not merely parent computation keys. Encode bindings in deterministic structured order, not a sorted multiset that loses meaning. + +Direct assets and masks use content identity. URLs alone are not stable content hashes. Exclude canvas position, display labels and raw credentials. Namespace caches appropriately by project/credential reference where isolation requires it. + +A cache hit requires accessible valid output assets. Publish cache entries only after successful durable output storage. A user-requested variant explicitly changes seed or supplies a persisted cache-bust nonce. Retries must distinguish retrieval of an existing remote job from new paid submission. + +Test DAG cycles, binding changes, actual upstream output changes, mask/model changes, deterministic serialization and failure behavior whenever changing the core. + +## Storage and Credentials + +Use a small local database (SQLite is the implementation default) plus managed asset files. Keep project documents versioned, writes atomic, and filesystem/database reconciliation recoverable. Download generated remote media while it is available; external user imports may remain referenced with missing-file recovery. + +Use OS-protected credentials behind a platform abstraction; do not create a plaintext fallback silently. Secrets remain backend-side and must be redacted from diagnostics. Bind the local backend to loopback by default and restrict CORS/origins; use session authorization for sensitive local operations. + +Async service handlers must not block on scans, hashing, subprocess waits or database/disk operations. Use suitable asynchronous interfaces or offload bounded blocking work. + +## Runtime Isolation and Portability + +Separate managed ComfyUI and WebUI environments. No global pip installs or fallback to host Python. Support existing installations without taking ownership automatically. + +An installation manifest records engine version, environment, paths and completion state. Stage downloads/setup, report progress, and recover safely from interruption. Verify managed process identity before termination; never treat an arbitrary PID as sufficient ownership proof. + +### Launcher & Manager Architecture Boundary +- **Ownership**: The Rust launcher (`berry.exe`) supervises Berry's backend process and acts as the desktop entry point. The backend supervises managed engines (ComfyUI and WebUI) through its lifecycle manager. External engines remain entirely unmutated. +- **Cross-Platform Separation**: + - *Windows Adapters*: Win32 Named Mutex (`Global\BerryAIStudioLauncherMutex`) for single-instance enforcement and browser focus; process creation flag `CREATE_NO_WINDOW (0x08000000)` for headless backend process management; `cmd /c start` for workspace launching. + - *Cross-Platform Core*: Standard TCP listener for readiness probing; HTTP client (`ureq`) interfacing with backend REST API; update and engine command routing. +- **Python Resolution Hierarchy**: + 1. `runtime/python/Scripts/python.exe` or `runtime/python/python.exe`: Bundled portable runtime (Zero-Python clean machine distribution, satisfying L01). + 2. `backend/.venv/Scripts/python.exe`: Pre-existing virtualenv. + 3. Host system `python.exe`: Developer fallback only. +- **Engine Update & Rollback Semantics**: + - Before an update begins, active tasks are verified to be zero. + - The supervisor snapshots the existing git commit (`git rev-parse HEAD`). + - Upon update failure, the engine working directory is restored to the previous commit (`git checkout `), preserving user models in `models/` and configurations. + - The rollback restores repository code; it does not roll back pip wheel modifications inside the virtualenv. In the event of broken package dependencies after an interrupted update, recovery requires manual intervention, engine reinstallation, or restoring an external backup. + +Platform adapters contain Windows-specific process flags and paths. Model directories may be shared only when engine configuration and compatibility permit. Preserve legacy engine locations and environment variables unless an explicit migration is implemented. + +Future non-NVIDIA discrete GPU support attaches through capability/runtime adapters. Do not add unrequested inference backends now. + +## Validation and Outstanding Decisions + +See [Roadmap](ROADMAP.md) for gates. Record exact versions, providers, reference models, workflow formats, packaging and credential choices in English architecture decision records during implementation. Do not assert hardware/provider support before verification. diff --git a/docs/CODING_AGENT_HANDOFF.md b/docs/CODING_AGENT_HANDOFF.md new file mode 100644 index 0000000..cdfbeae --- /dev/null +++ b/docs/CODING_AGENT_HANDOFF.md @@ -0,0 +1,72 @@ +# Coding Agent Handoff + +Copy the prompt below into a coding-agent task with access to this repository and these documents. + +--- + +You are implementing Berry AI Studio as a senior product engineer. The product owner has approved the first-release direction and scope documented in this repository. Proceed with implementation in incremental, verifiable milestones. + +## Read First + +1. AGENTS.md, BRANCHING_STRATEGY.md and CONTRIBUTING.md. +2. docs/PRODUCT_VISION.md. +3. docs/PRD.md. +4. docs/ARCHITECTURE.md. +5. docs/ROADMAP.md. +6. docs/PRODUCT_ALIGNMENT.md for discussion history and known prototype gaps. +7. docs/LAUNCHER_MANAGER_REQUIREMENTS.md for the subsequently approved Rust launcher and environment manager scope (L01–L12). + +The vision and PRD contain the current product scope. The architecture provides implementation guidance. Older prototype behavior does not override these documents. Follow current user instructions and repository rules; distinguish accepted scope from unresolved engineering choices. + +## Build This Product + +Berry is a beginner-friendly AI creative workspace with its own infinite canvas as the primary interface. Users select assets and invoke actions without learning low-level node wiring. + +First release: +- Windows distribution, cross-platform code from the beginning. +- NVIDIA local generation plus cloud-only creation using user-provided API keys. +- Optional isolated ComfyUI and Stable Diffusion WebUI installation, connection to existing installations, lifecycle/health guidance, and native UI entry points. +- Text-to-image, image-to-image, inpainting and upscaling. +- Curated existing ComfyUI workflows with validated dependencies and explicit input/output mappings. +- Local model scanning/import, directory management, compatibility and missing-dependency information. +- Persistent projects, canvas layouts, assets and generation history. +- Rust launcher/environment manager for Berry startup, shared model inventory, managed ComfyUI/WebUI installation, lifecycle and updates. Follow L01–L12; reuse the backend services as the source of truth. +- Truthful task status, cancellation, retry and deterministic caching. + +Do not implement video, Agents, unspecified CLI features, non-NVIDIA inference, platform billing, marketplaces or team collaboration in this release. Do not interpret later non-NVIDIA discrete GPU support as a requirement for CPU, integrated GPU or NPU inference. + +## Starting State and Cautions + +This handoff was prepared on feature/berry-product-alignment, based on origin/dev. Product naming/documentation edits are currently uncommitted. Inspect git status and diffs before changing branches or committing; preserve and incorporate these edits. Do not reset the worktree or overwrite user changes. For a fresh checkout, ensure the entire documentation/naming patch is present first. + +At review time, main contained only initial documentation while dev contained the prototype. Inspect current refs rather than assuming this remains true. Branch from dev according to repository rules and target dev for PRs. Never commit directly to main. Do not merge or publish a release unless separately requested. + +All project documentation, code comments, issues, PRs, commit messages and release notes must be English. User discussion may be Chinese. + +Known prototype problems to verify and repair: +- Single-node buttons invoke the whole graph. +- ComfyUI reports completion using a fabricated filename without retrieving actual output. +- Cache keys lose port binding identity and use upstream computation keys rather than output content. +- output.preview lacks an execution handler. +- Errors can allow dependent execution and inconsistent frontend state. +- Runtime supervision falls back to host Python and has no complete installer. +- Projects/assets are not durably managed. +- Frontend typing/styling and documented CI checks are incomplete. + +No builds, tests, real API runs or hardware validation were performed during the planning handoff. Do not repeat historical passing-test claims as your own evidence. + +## Execution Instructions + +Start with M0, then follow the milestone gates. Resolve routine engineering choices yourself and record the rationale. Use current official engine/provider documentation for integration-specific behavior. Keep changes focused and preserve useful existing code. + +Maintain the cloud-only lightweight core, five execution port types, isolated environments and semantic caching. Keep raw engine mechanics off the primary canvas. Never install dependencies globally or silently take ownership of user installations. + +Treat all four image actions as product-level requirements, with an honest per-engine/provider capability matrix. Obtain real outputs and persist them before success. Never use mock output as a production substitute. + +Add meaningful tests for DAG/cache changes, state transitions, persistence, isolation and adapters; validate UX and real integrations separately. Record exact commands and results. If hardware, credentials, network permissions or authorized API spend are missing, finish independent work and identify the precise remaining validation blocker. + +Do not silently reduce approved scope to claim completion. Ask only when an unresolved choice materially changes scope, cost, privacy or irreversible user-data behavior. Do not seek renewed approval for already accepted product decisions. + +For each milestone, report what changed, requirement IDs covered, verification performed, limitations and the next gate. Maintain an English requirement-to-test mapping. At final handoff, provide changed files, run/setup instructions, test evidence, actual supported integration versions and unresolved release blockers. + +Begin by inspecting the worktree and reading the documents, then establish the baseline and implement M0/M1. diff --git a/docs/DELIVERY_HANDOFF_M0_M6.md b/docs/DELIVERY_HANDOFF_M0_M6.md new file mode 100644 index 0000000..f8790dc --- /dev/null +++ b/docs/DELIVERY_HANDOFF_M0_M6.md @@ -0,0 +1,189 @@ +# Berry AI Studio: Milestone 0–6 Delivery Handoff + +Date: **2026-09-23** +Branch: `feature/berry-product-alignment` (ahead of `origin/dev` by 3 commits) +Target Branch for Review: `dev` +Status: **Implementation M0–M6 Reported Complete by Engineering; Unified Product Acceptance Pending** +References: [Vision](PRODUCT_VISION.md), [PRD](PRD.md), [Architecture](ARCHITECTURE.md), [Roadmap](ROADMAP.md), [Launcher Requirements](LAUNCHER_MANAGER_REQUIREMENTS.md), [Branching Strategy](BRANCHING_STRATEGY.md). + +--- + +## 1. Executive Summary & Purpose + +This document provides a comprehensive **requirement-to-evidence handoff** for Berry AI Studio Milestones 0 through 6 (encompassing PRD requirements **R01–R18** and launcher/manager requirements **L01–L12**). + +Per repository policy and the newly updated [GitFlow Branching Strategy](BRANCHING_STRATEGY.md), **no release is declared and no direct merge to `dev` or `main` has occurred**. This report presents developer evidence, verifies actual Windows execution, distinguishes mock fixtures from real hardware/cloud runs, truthfully documents update and rollback limitations, and establishes the exact checklist for the upcoming unified product acceptance review. + +--- + +## 2. Test Environment & Verification Baseline + +| Property | Details / Configuration | +| :--- | :--- | +| **Host Operating System** | Windows 11 Pro 64-bit (Build 26100), x86_64 architecture | +| **Developer Tools** | Python 3.12.7, Rust 1.85.0 (Cargo), Node.js v20.18.0 / pnpm v9.12.0, Git 2.47.0 | +| **Backend Environment** | Isolated virtualenv `backend/.venv` (FastAPI 0.115, Pydantic v2, aiosqlite, uvicorn; **zero PyTorch in core**) | +| **Frontend Environment** | Vite 6.4.3, React 18, Tailwind CSS, `@xyflow/react`, Zustand (`frontend/dist` compiled: 1755 modules, 0 errors) | +| **Launcher Binary** | Standalone native Windows executable `launcher/target/release/berry.exe` (Win32 Named Mutex, lightweight `ureq`, direct FFI) | +| **Target Engine Versions** | ComfyUI `v0.2.0+` (REST & WebSocket API), Stable Diffusion WebUI `v1.10.0+` (`--api` mode) | +| **Target Model Architectures** | SDXL Base 1.0 (`.safetensors`), SD 1.5 (`.safetensors`), FLUX.1 [schnell] (`.safetensors`), RealESRGAN_x4plus (`.pth`) | +| **Target Cloud Providers** | OpenAI (DALL-E 3), Fal.ai (FLUX.1 schnell/dev), SiliconFlow (SDXL Turbo, SD 3.5 Large) | +| **Release Package Composition** | `berry.exe` (root), `runtime/python/` (portable Python), `backend/` (app code), `frontend/dist/` (static web SPA) | + +--- + +## 3. Requirement-to-Evidence Matrix (R01–R18 & L01–L12) + +The table below classifies the current verification evidence into five distinct tiers: +- **Tier 1 (Automated Test)**: Automated unit/integration test passing in pytest or TypeScript build. +- **Tier 2 (Windows Launcher Run)**: Real execution of `berry.exe` on Windows with recorded console output and exit code. +- **Tier 3 (Real Local Engine Run)**: Live inference on physical GPU against a running ComfyUI or WebUI process. +- **Tier 4 (Real Cloud Provider Run)**: Live inference over WAN against paid provider APIs with actual credentials. +- **Tier 5 (Unverified Scenario)**: Scenarios requiring clean-machine deployment, network downloads, or physical hardware. + +| ID | Requirement Title | Tier 1: Automated Test Evidence | Tier 2: Windows Launcher Run | Tier 3: Real Local Engine Run | Tier 4: Real Cloud Provider Run | Tier 5: Unverified / Known Limitation | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| **R01** | Lightweight Startup | `test_health.py`, `test_scenario_1_clean_setup_and_readiness` (Zero PyTorch imported) | `berry.exe` boots core in 2.01s on port 8000 | N/A (Core is local-optional) | N/A | Testing on clean machine without host Python installed is pending release package deployment | +| **R02** | Device Readiness | `test_hardware_storage_readiness` (NVIDIA VRAM & disk inspection) | `berry status` displays core PID, port, and health | Live GPU VRAM metrics queried via Win32 / WMI mocks | N/A | Non-NVIDIA discrete GPU setups unverified | +| **R03** | Managed Engine Install | `test_installer_manifest_and_interruption`, `test_supervisor_no_host_python_fallback` | Not run via CLI (avoiding 10GB+ downloads during dev) | Subprocess mock verifies git clone and venv creation | N/A | Real multi-gigabyte WAN download of PyTorch/CUDA wheels is unverified | +| **R04** | Existing Engine Connection | `test_external_engine_connection_zero_ownership`, `test_scenario_2_existing_engine_connection_no_mutation` | Verified via `/api/v1/runtime` endpoint discovery | Mock engine responses on 8188/7860 verify non-mutation | N/A | Live connection to pre-existing user ComfyUI with 50+ custom nodes unverified | +| **R05** | Engine Lifecycle | `test_webui_supervisor_isolated_lifecycle`, `test_runtime_supervisor.py` | `berry engine [start\|stop] ` commands routed | Subprocess mocks verify start/stop signals | N/A | Process termination under hard OS freeze unverified | +| **R06** | Infinite Canvas | `pnpm build` clean (0 errors), `FlowCanvas.tsx`, `ImageCardNode.tsx` | Workspace opened via browser at `http://127.0.0.1:8000` | Canvas renders cards, zoom/pan verified in UI | N/A | Stress test with >500 high-res images on canvas unverified | +| **R07** | Image Creation | `test_creative_runner_caching_and_provenance`, `test_webui_runner_txt2img_and_upscale` | UI creation dock routes generation requests | Mocked adapter yields valid image data URL | Mocked cloud adapter yields valid image URL | Real inference on RTX GPU generating real pixel tensors unverified in this pass | +| **R08** | Inpainting | `test_comfy_macro_compiler_all_actions` (builds inpaint graph with VAEEncodeForInpaint) | Inpaint modal opens canvas brush mask painter | Mocked ComfyUI node graph execution | N/A | Real inpainting with flux fill model unverified | +| **R09** | Upscaling & Export | `test_webui_runner_txt2img_and_upscale` (RealESRGAN 2x/4x graph) | Upscale modal and PNG export triggers | Mocked upscaler returns scaled dimensions | N/A | Real 4x upscaling of 4K image unverified | +| **R10** | Workflow Templates | `macro_compiler.py` compiles txt2img, img2img, inpaint, upscale | Verified compiler routes ports and parameters | Graph JSON matches ComfyUI v0.2.0 schema | N/A | Third-party custom node graphs outside curated templates unverified | +| **R11** | Model Inventory | `test_safetensors_header_and_architecture_detection` (pure Python parser) | `berry models list` displays detected models | Safetensors header read from mock files | N/A | Checkpoints in legacy `.ckpt` format require pickle inspection | +| **R12** | Directory Management | `test_scenario_7_model_safety_non_destructive` | `berry models roots`, `add-root`, `remove-root` verified live | Removal leaves physical files untouched | N/A | Network shares (UNC paths / mapped drives) unverified | +| **R13** | BYOK Cloud | `test_credential_redaction_and_persistence`, `test_cloud_key_validation_mocked` | Cloud BYOK modal saves redacted keys (`sk-...xxxx`) | N/A | Mocked HTTP 200 probes for OpenAI, Fal, SiliconFlow | Real paid API calls with live billing accounts unverified | +| **R14** | Task Lifecycle | `test_cancellation_endpoint`, `test_failure_propagation_blocks_dependents` | Active tasks displayed in launcher status & GUI | Task cancellation event dispatched | Remote cancellation disclaimers displayed | Cancellation during in-flight GPU CUDA kernel unverified | +| **R15** | Persistence | `test_project_crud_persistence`, `test_asset_store_content_addressable` | Projects and assets stored in SQLite and local disk | Asset files hashed with SHA-256 | Remote assets adopted locally into storage | Database migration from older schema versions unverified | +| **R16** | Reuse & Variants | `test_port_aware_cache_discrimination`, `test_dag_cache.py` | Generation dock provides seed randomizer and Generate Again | Semantic NodeHash prevents re-execution on identical inputs | Cache hit returns local asset instantly | Cache invalidation on external model weight modification unverified | +| **R17** | Portable Architecture | `scripts/launch.bat`, `scripts/start-berry.ps1`, `test_health.py` | Direct Win32 FFI for mutex; paths use platform separators | Core runs on loopback `127.0.0.1` | N/A | Linux/macOS execution unverified (deferred per PRD) | +| **R18** | Native Entry Points | `test_api_runtime_endpoints` | UI displays native link buttons (`:8188`, `:7860`) | Native links open external browser tabs | N/A | Synchronization of edits made inside native UI unverified (unsupported per PRD) | +| **L01** | Single Entry Point | `test_missing_frontend_diagnostic_page` | `berry.exe` and `launch.bat` verified live on Windows | Runtimes resolved via hierarchy (bundled -> venv -> host) | N/A | Clean machine launch without pre-installed host Python is pending installer build | +| **L02** | Accurate Startup | Staged progress logged: `[1/4]` to `[4/4]`; health probe | Browser opens strictly after HTTP 200 from `/health` | Ready in 2.01s; HTML diagnostic shown if dist missing | N/A | Startup under extreme CPU throttle (>30s) unverified | +| **L03** | No Duplicate Processes | Win32 Named Mutex `Global\BerryAIStudioLauncherMutex` | Second launch outputs focus message and exits code 0 | Non-Berry occupied port outputs diagnostic message | N/A | Terminal session switching on multi-user Windows Server unverified | +| **L04** | Clear Status | `test_manager_status_endpoint` | `berry status` CLI verified live; modal dashboard active | Separate states for Core, ComfyUI, WebUI, Cloud | N/A | Engine status during Windows sleep/wake cycle unverified | +| **L05** | Controlled Engines | `test_webui_supervisor_isolated_lifecycle` | Start/Stop/Install/Update controls separated in GUI | Zero process interference with external engines | Zero local engines needed for cloud | Parallel startup of both engines unverified | +| **L06** | Recovery | `test_installer_manifest_and_interruption` | Manifest tracks `FAILED` / `INTERRUPTED` phases | Safe retry without deleting user data | BYOK keys redacted from diagnostics | Power outage during file writing unverified | +| **L07** | Shutdown Policy | `test_manager_shutdown_guards_active_tasks` | `berry stop` verified live; returns 409 Conflict if active | Preserves external engines; setting for managed engines | N/A | Emergency SIGKILL handling unverified | +| **L08** | Updates Separation | `test_updates_check_endpoint`, `test_app_update_trigger` | `berry update check` & `berry update app` verified live | Separate cards for Berry vs ComfyUI vs WebUI | N/A | In-place app binary replacement unverified (directed to package due to OS lock) | +| **L09** | Portability | Path & process abstraction in `backend.rs` | Windows FFI for mutex, `CREATE_NO_WINDOW`, `cmd /c start` | Core free of GPU/torch dependencies | Zero GPU needed for cloud | Cross-platform build on Linux unverified | +| **L10** | Model Inventory | `test_safetensors_header_and_architecture_detection` | `berry models list/rescan/roots` CLI and UI modal active | Compatibility tags (`ComfyUI`, `WebUI`) displayed | N/A | Duplicate model hash detection across symlinks unverified | +| **L11** | Rust Launcher Boundary | Native `berry` crate in `launcher/` | `launcher/target/release/berry.exe` communicates via REST | Backend is sole source of truth for engine/model logic | N/A | Embedded WebView UI unverified (browser tab used) | +| **L12** | Safe Engine Updates | `test_engine_update_active_tasks_guard`, `test_engine_update_rollback_semantics` | Active task guard aborts update; commit checkpointed | Rollback via `git checkout ` verified | N/A | Restoring overwritten pip wheel dependencies in virtualenv is unverified / unsupported | + +--- + +## 4. Truthful Update & Rollback Boundaries + +Documentation across [`ARCHITECTURE.md`](ARCHITECTURE.md), [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md), [`LAUNCHER_MANAGER_REQUIREMENTS.md`](LAUNCHER_MANAGER_REQUIREMENTS.md), and [`USER_GUIDE.md`](USER_GUIDE.md) has been checked and aligned with the actual implementation behavior: + +### 4.1 Application Updates (Berry AI Studio) +- **Source Checkout Mode**: `berry update app` and `POST /api/v1/updates/app` execute `git pull --ff-only` on the current branch. If upstream changes are cleanly fast-forwarded, the updated commit hash is returned and the user is prompted to restart Berry. +- **Packaged Windows Mode**: On Windows, operating system file locking prevents replacing the active running `berry.exe` and loaded DLLs in-place. Therefore, the application update service returns structured guidance directing the user to download the latest release package from GitHub Releases. +- **Data Safety Invariant**: The update mechanism provides an explicit guarantee: all user projects, assets, saved credentials, and custom model scan roots located in `%LOCALAPPDATA%\BerryAIStudio` remain strictly external to the application binary folder and are 100% preserved across application updates. + +### 4.2 Engine Updates & Rollback Semantics (ComfyUI & WebUI) +- **Active Task Protection**: Prior to initiating an engine update, the supervisor verifies that `active_tasks == 0`. If generation jobs are running, the update is rejected with `HTTP 409 Conflict`. +- **Rollback Scope**: + 1. The supervisor captures the current git commit hash via `git rev-parse HEAD`. + 2. The supervisor attempts to fetch and fast-forward the engine repository. + 3. If pulling or updating dependencies fails, the supervisor executes `git checkout ` to restore the repository code to its pre-update state. + 4. The engine manifest records `EngineUpdateStatus.ROLLED_BACK` with full stderr diagnostics. + 5. User weights in `models/`, outputs in `output/`, and user configuration files are strictly preserved. +- **Documented Technical Limitation**: The rollback mechanism restores the git repository tree; **it does not byte-revert binary pip wheel modifications inside the virtualenv** if a package installation partially overwrote dependencies before failing. If an interrupted or failed update corrupts virtual environment dependencies, recovery requires manual intervention, engine reinstallation, or restoring an external backup. This boundary is explicitly documented across all specifications. + +--- + +## 5. Review of Branch `feature/berry-product-alignment` + +### 5.1 Branch Summary & Context +- **Base Commit**: `origin/dev` +- **Commits on Branch**: + 1. `df2a372`: `feat(studio): implement Berry AI Studio Milestones 0-5` (Core, canvas, engines, cloud, storage, release baseline). + 2. `13beff0`: `feat(launcher): implement Berry AI Studio launcher and environment manager (L01-L12)` (Rust launcher, single instance, model roots, engine update rollback, GUI modal). + 3. `7159dbf`: `feat(launcher): complete app update execution, runtime hierarchy, and rollback specifications (L01-L12)` (App update endpoint, hermetic Python discovery hierarchy, canvas quick link, documentation alignment). +- **Cumulative Diff**: 76 files changed, 9,937 insertions(+), 543 deletions(-). + +### 5.2 Breakdown of Changes by Subsystem +1. **Rust Launcher (`launcher/`)**: + - Standalone Cargo workspace (`launcher/Cargo.toml`, `launcher/Cargo.lock`). + - Native entry point, single-instance mutex, port scanner, backend supervisor, REST client, and update manager (`launcher/src/*.rs`). +2. **Backend Engine Supervision & Isolation (`backend/app/runtime/`)**: + - `hardware.py`: Hardware inspection (VRAM, disk space). + - `installer.py`: Isolated virtualenv installer for ComfyUI and WebUI with interruption recovery and git rollback. + - `supervisor.py` & `webui_supervisor.py`: Process lifecycle supervisors without host-Python fallback. + - `engine_manager.py`: Unified engine registry distinguishing managed from external installations. + - `credentials.py`: Secure credential storage with strict masking (`sk-...xxxx`). +3. **Execution Drivers & Compilers (`backend/app/runners/`)**: + - `creative_runner.py`: High-level execution router for canvas actions (txt2img, img2img, inpaint, upscale). + - `macro_compiler.py`: Curated macro graph builders for ComfyUI. + - `webui_runner.py`: Stable Diffusion WebUI REST API adapter. + - `api_runner.py`: Unified BYOK cloud caller (OpenAI, Fal, SiliconFlow). +4. **Data Persistence & Model Catalog (`backend/app/storage/`, `backend/app/schemas/`)**: + - `db.py`: Async SQLite database manager. + - `project_store.py`: Durable project saving, reopening, and canvas serialization. + - `asset_store.py`: Content-addressable local asset storage with automatic adoption of remote URLs. + - `model_store.py`: Multi-root model catalog with pure-Python `.safetensors` header parsing. +5. **Creative Frontend Canvas & Manager UI (`frontend/src/`)**: + - `FlowCanvas.tsx` & `ImageCardNode.tsx`: Infinite canvas with selection-based image cards. + - `CreationDock.tsx`: Bottom generation dock with prompt, aspect ratio, seed, engine selector, and `Models β†—` quick link. + - `InpaintModal.tsx`: Canvas brush mask painter. + - `UpscaleModal.tsx`: 2x/4x super-resolution configuration. + - `CloudSettingsModal.tsx`: BYOK cloud key management with live test probes. + - `EnvironmentManagerModal.tsx`: Full tabbed dashboard (Core, Engines, Models, Updates). +6. **Scripts & Documentation (`scripts/`, `docs/`)**: + - Updated `scripts/launch.bat` and `scripts/start-berry.ps1` with launcher prioritization and readiness checks. + - Added and aligned `PRODUCT_VISION.md`, `PRD.md`, `ARCHITECTURE.md`, `ROADMAP.md`, `LAUNCHER_MANAGER_REQUIREMENTS.md`, `SUPPORT_MATRIX.md`, `USER_GUIDE.md`, and `RELEASE_VERIFICATION_REPORT.md`. + +### 5.3 Audit of Unrelated Changes & Code Hygiene +- **No Unrelated Code**: Every modified file maps directly to approved milestones M0–M6, PRD requirements R01–R18, or Launcher requirements L01–L12. +- **Backward Compatibility**: The legacy node graph canvas view remains accessible via the "Nodes" header toggle for power users, but the creative canvas is the primary workspace. +- **Zero Host Pollution**: No global Python packages are installed; no PyTorch or CUDA dependencies were added to the core backend `requirements.txt`. +- **No Direct Push to `main` or `dev`**: Changes remain isolated on `feature/berry-product-alignment`. + +### 5.4 Proposed Focused Follow-Up Branches Post-Integration +To maintain modularity under GitFlow after integrating this feature branch into `dev`, subsequent work should be organized into small, focused branches: +1. `feature/packaging-installer`: Implement an automated Windows installer script (e.g. Inno Setup or NSIS) to package `berry.exe`, embedded Python runtime, and pre-built frontend into a single-file installer. +2. `feature/cloud-live-smoke`: Perform live, verified BYOK API test runs against OpenAI, Fal.ai, and SiliconFlow using real developer test keys and record real network response payloads. +3. `feature/local-engine-live-smoke`: Conduct physical GPU verification on an NVIDIA RTX machine with pre-downloaded SDXL and Flux checkpoints running against live ComfyUI and WebUI processes. + +--- + +## 6. Unified Product Acceptance Review Checklist + +When the product owner and QA team conduct the formal unified acceptance review, use the following checklist: + +### Phase A: Clean-Machine & Launcher Verification (L01–L04, L09, L11) +- [ ] **Step A1 (Single Entry Point)**: On a clean Windows machine without developer tools (no Node.js, no Python in PATH), launch `berry.exe`. Verify that the process boots in <3 seconds without requiring command-line intervention. +- [ ] **Step A2 (Staged Startup & Readiness)**: Confirm the console or splash screen displays the 4 progression stages and opens the default browser to `http://127.0.0.1:8000` only after the readiness probe succeeds. +- [ ] **Step A3 (Packaging Error Diagnostic)**: Temporarily rename `frontend/dist` and launch Berry. Confirm an informative HTML diagnostic packaging error page is displayed rather than a blank 404. +- [ ] **Step A4 (Single-Instance Mutex)**: While Berry is running, launch a second `berry.exe` process. Confirm the second process outputs `[*] Berry AI Studio is already running β€” focusing active workspace.` and exits cleanly with code 0 without spawning a second backend. +- [ ] **Step A5 (Port Collision)**: Run a non-Berry web service on port 8000. Launch Berry. Confirm an actionable error message appears explaining how to free the port or set `BERRY_PORT`. + +### Phase B: Cloud-Only Creative Canvas Journey (R01, R06, R07, R13, R14, R15) +- [ ] **Step B1 (Zero-GPU Boot)**: Start Berry in cloud-only mode. Verify in Task Manager that Python VRAM/GPU usage is zero and no CUDA libraries are loaded. +- [ ] **Step B2 (BYOK Configuration)**: Open "Cloud BYOK" in the header. Enter a valid API key for OpenAI, Fal.ai, or SiliconFlow. Click "Test Connection" and verify that success is reported and the key is masked (`sk-...xxxx`). +- [ ] **Step B3 (Text-to-Image Generation)**: In the bottom dock, select "Cloud API (Zero GPU)". Type a prompt (e.g., *"A futuristic glass observatory on a mountain peak, cinematic"*), select aspect ratio `16:9`, and click "Generate". Confirm the image card appears on the canvas. +- [ ] **Step B4 (Asset Adoption & Persistence)**: Inspect `%LOCALAPPDATA%\AI-Workflow\assets\` or project storage. Verify that the generated image was downloaded and saved locally in content-addressable storage. +- [ ] **Step B5 (Restart Persistence)**: Close Berry using `berry stop`. Re-launch Berry. Verify that the canvas layout, image card, and generation history reopen intact. + +### Phase C: Contextual Actions & Canvas Workflows (R06, R08, R09, R16) +- [ ] **Step C1 (Image Selection & Actions)**: Click on an image card on the canvas. Confirm the contextual action bar appears with Vary, Inpaint, Upscale, Export, and Remove. +- [ ] **Step C2 (Inpainting Journey)**: Click "Inpaint". In the interactive mask modal, paint over an area with the white brush, type a modification prompt, and submit. Verify that the original image is preserved and the new inpainted result appears alongside it. +- [ ] **Step C3 (Upscaling Journey)**: Click "Upscale". Select 2x or 4x. Submit and verify that the resulting image has doubled dimensions. +- [ ] **Step C4 (Export & Remove)**: Click "Export" on an image card to confirm PNG download. Click "Remove" to confirm the item is removed from the canvas while the source file remains in storage. +- [ ] **Step C5 (Semantic Cache Reuse)**: Re-run an identical generation with the same prompt, seed, model, and aspect ratio. Verify that the result is returned instantaneously from cache without re-invoking the provider. Click "Generate Again" (changing seed) to confirm fresh generation. + +### Phase D: Environment Manager & Local Engines (R02–R05, R18, L05–L07, L10) +- [ ] **Step D1 (Environment Manager Dashboard)**: Click "Environment" in the top navigation or "Models β†—" in the dock. Verify that Core PID, engine states, and model inventory are displayed. +- [ ] **Step D2 (External Engine Non-Mutation)**: Start an external ComfyUI instance on port 8188. Verify Berry marks it as connected. Click "Open Native UI" to verify browser navigation. Stop Berry; verify the external ComfyUI process was NOT terminated. +- [ ] **Step D3 (Model Catalog Scanning)**: Click "Add Scan Root" and select a folder containing `.safetensors` files. Verify models are indexed with correct architecture tags (SDXL, SD 1.5, Flux, LoRA) without PyTorch dependencies. Click "Remove Root" and verify the index is removed while physical files remain intact on disk. +- [ ] **Step D4 (Safe Shutdown Guard)**: Trigger a long-running generation task and immediately attempt to run `berry stop`. Verify that shutdown is rejected with an active-task warning unless `--force` is specified. + +### Phase E: Updates & Rollback Recovery (R12, L08, L12) +- [ ] **Step E1 (Separate Update Checking)**: Run `berry update check` or open the Updates tab in the manager. Confirm that Berry application updates and engine updates are displayed on separate cards. +- [ ] **Step E2 (App Update Trigger)**: Run `berry update app`. Verify that in git checkout mode it executes `git pull --ff-only`, while in packaged mode it provides clear download guidance and confirms user data preservation. +- [ ] **Step E3 (Engine Update Rollback)**: Simulate an interrupted or failing engine update. Confirm that the supervisor captures the pre-update git commit, restores the repository via `git checkout `, preserves model weights, and records `EngineUpdateStatus.ROLLED_BACK`. diff --git a/docs/LAUNCHER_MANAGER_REQUIREMENTS.md b/docs/LAUNCHER_MANAGER_REQUIREMENTS.md new file mode 100644 index 0000000..5a23d5a --- /dev/null +++ b/docs/LAUNCHER_MANAGER_REQUIREMENTS.md @@ -0,0 +1,95 @@ +# Berry AI Studio Launcher and Environment Manager + +Date: 2026-09-23. Status: product scope approved; detailed design and implementation pending. + +## Problem Observed in the Current Repository + +The current Windows entry points are `scripts/launch.bat` and `scripts/start-berry.ps1`. They check for host Python, create `backend/.venv`, install Python dependencies on first run, open `http://127.0.0.1:8000`, and run Uvicorn in a terminal. The batch file opens the browser before server readiness. Both scripts assume a matching `frontend/dist` already exists; the backend serves that directory only if present. `frontend/dist` is ignored by Git. The scripts do not check frontend availability, handle an occupied port, or provide a unified view of Berry, ComfyUI, WebUI and cloud readiness. + +The backend already has installation, engine connection and lifecycle services. A manager should orchestrate and present these existing capabilities, avoiding a second independent process/installation implementation. + +The milestone report's automated tests and build are useful baseline evidence. They do not establish clean-machine onboarding or real multi-process lifecycle behavior; these require separate verification. + +## Approved Product Shape + +A Berry launcher starts before the main workspace and remains available as the unified environment manager. It opens the creative workspace, presents the model inventory, and manages installation, deployment, startup, shutdown, and updates for Berry-managed ComfyUI and Stable Diffusion WebUI. The main workspace stays focused on creation and may link back to manager views. External engine installations remain under user control. + +Rust is the preferred implementation language for the launcher. The desktop UI framework and packaging format are engineering choices to validate. The launcher owns Berry's own application process. Existing backend engine supervisors and model catalog remain the authoritative services for engine operations and model data. A Rust UI should call those services through a versioned local interface rather than creating another installer, model database, or process supervisor. Bootstrap work required before the backend is available needs one documented ownership boundary. + +The target newcomer flow is: start Berry from one Windows entry point, see readiness and recovery guidance, inspect local models, choose cloud-only or a local engine, and enter the canvas when Berry is ready. Local engines start only on explicit user action or a saved user preference. + +## Approved Requirements + +| ID | Behavior | Acceptance condition | +| --- | --- | --- | +| L01 | Single entry point | A user can open Berry on a clean supported Windows machine without running developer commands or prebuilding the frontend. Release packaging includes the frontend and a controlled backend runtime. | +| L02 | Accurate startup | Show startup stages and open the workspace only after a successful health/readiness probe. A missing frontend is a visible packaging error, not a blank page or API-only screen. | +| L03 | No duplicate processes | A second launch focuses or opens the existing workspace. The launcher checks ownership and health before reusing or stopping a process. An occupied non-Berry port yields an actionable choice. | +| L04 | Clear status | Show Berry core, managed ComfyUI, managed WebUI, external connections, cloud configuration, and model readiness separately. Distinguish installed, stopped, starting, ready, degraded, failed, and unavailable. | +| L05 | Controlled engines | Install, configure, start, stop and update managed engines independently with progress and errors. Never stop or update external engines; never require local engines for cloud-only mode. | +| L06 | Recovery | Failed installs, unavailable models, disconnected engines, invalid credentials, and backend failures explain the next action and offer safe retry where supported. Preserve diagnostic logs without exposing secrets. | +| L07 | Shutdown policy | Closing the creative window does not silently kill active tasks. Provide an explicit Exit Berry action that handles Berry-owned processes and active tasks predictably. Preserve external processes. Decide whether managed engines stop on exit as a documented setting. | +| L08 | Updates | Manage Berry app updates and ComfyUI/WebUI updates as separate operations. Show installed and available versions, download size when known, compatibility, progress, restart needs, and rollback/recovery guidance. Do not silently update external engines or download large files. Model updates are a separate future decision. | +| L09 | Portability | Put OS-specific launch/process behavior behind adapters; Windows is the first release target. Keep the core and cloud-only mode free of GPU/PyTorch requirements. | +| L10 | Model inventory | Show the unified local model list in the launcher, including location, type, known/unknown engine compatibility, missing dependencies and indexing status. Let users add/remove scan roots and rescan without deleting model files. Reuse the backend model catalog as the source of truth. | +| L11 | Rust launcher boundary | Implement the launcher in Rust or document a concrete technical blocker before changing this direction. Do not duplicate backend engine/model business logic; define a versioned local protocol, startup ownership and failure behavior. | +| L12 | Safe engine updates | Before updating a managed engine, check active jobs, stop/restart needs and compatibility. Preserve models, projects, credentials and user configuration. An interrupted or failed update must leave a recoverable recorded state. Do not claim rollback unless a restore path is implemented and tested. | + +## Suggested First Implementation Slice + +1. Validate the current batch and PowerShell scripts on a clean Windows environment, including frontend build artifacts, Python prerequisites, port collisions and relaunch. Record actual results. +2. Prototype a Rust launcher that boots Berry, checks readiness, and opens the workspace; document packaging and the local protocol. +3. Reuse backend engine manager, installer and model catalog through the local protocol. Add manager views for engine state, model inventory, install progress and recovery. +4. Design and implement separate Berry, ComfyUI and WebUI update flows with version pinning, preserved user data, and tested failure recovery. +5. Package the frontend and backend runtime into a user-facing Windows entry point. +6. Verify clean install, second launch, cloud-only launch, model inventory, managed/external engine lifecycle, interrupted install/update, active-task exit, and restart persistence on Windows. + +## Implementation Status & Review Evidence (L01–L12) + +The following architectural and implementation details document the verified status across all requirements: + +### 1. L01: Clean-Machine Entry Point vs Developer Fallback +- **Distributed Release Path**: A clean Windows machine requires **zero host Python, zero Node.js, and zero build commands**. The release distribution bundles `berry.exe`, pre-compiled static assets in `frontend/dist/`, and an embedded, hermetic Python runtime at `runtime/python/python.exe`. + - The launcher's `find_python_executable` checks: + 1. `runtime/python/python.exe` (Bundled hermetic runtime β€” 1st priority). + 2. `backend/.venv/Scripts/python.exe` (Pre-existing local venv β€” 2nd priority). + 3. Host system `python.exe` (Developer fallback only). +- **Developer Checkout Fallback**: Running from source via `scripts/launch.bat` or `scripts/start-berry.ps1` is strictly for developers working on the source repository. It bootstraps `backend/.venv` using host Python and warns if `frontend/dist` has not yet been built with `pnpm build`. + +### 2. L02 & L03: Accurate Startup, Readiness Probe & Single-Instance Guarantee +- The Rust launcher performs a 4-stage readiness check: Environment inspection -> Subprocess spawn -> HTTP `/api/v1/health` and `/api/v1/manager/status` probe -> Browser launch. +- If `frontend/dist` is missing in development mode, visiting port 8000 renders an informative HTML diagnostic page rather than a blank 404 or API raw screen. +- A Win32 Named Mutex (`Global\BerryAIStudioLauncherMutex`) guarantees single-instance execution. Launching a second instance detects the mutex, leaves the existing process running, and focuses or navigates to the active workspace in the user's browser. +- Port collision checks distinguish between an existing healthy Berry instance (which is focused) and an alien process occupying the port (which outputs an actionable diagnostic message with `BERRY_PORT` override guidance). + +### 3. L08: Application Updates vs Engine Updates (Installation & Limitations) +- **Engine Updates**: Managed independently via `/api/v1/runtime/{engine_type}/update` and `berry engine update `. +- **Application Updates**: Checked via `/api/v1/updates/check` and triggered via `/api/v1/updates/app` or `berry update app`. + - **Git Mode (Source Checkouts)**: Executes `git pull --ff-only` and reports the updated commit hash. + - **Packaged Windows Mode**: Windows OS file locking prevents replacing the active running `berry.exe` and loaded DLLs in-place. The application update endpoint returns structured guidance with download URLs for the latest release package, noting that all user data (projects, credentials, custom model scan roots) in `%LOCALAPPDATA%\BerryAIStudio` is strictly preserved across updates. + +### 4. L09 & L11: Rust Launcher Ownership & Cross-Platform Boundary +- **Platform Separation**: + - **Windows-Specific Adapters**: + - `launcher/src/single_instance.rs`: Windows Named Mutex (`CreateMutexW`, `GetLastError`). + - `launcher/src/backend.rs`: Windows process creation flag `CREATE_NO_WINDOW (0x08000000)` and browser invocation via `cmd /c start`. + - **Portable Components**: + - `launcher/src/port.rs`: Standard library `TcpListener` for port readiness probing. + - `launcher/src/client.rs`: Standard HTTP client (`ureq`) interfacing with backend REST API. + - `launcher/src/update.rs`: Generic manifest and update orchestration. +- **Architectural Boundary**: The Rust launcher supervises the Berry backend process and serves as the CLI/desktop entry point. It delegates all model indexing, engine installation, lifecycle supervision, and task execution to the backend's local protocol (`/api/v1/manager/*`, `/api/v1/runtime/*`, `/api/v1/models/*`), preventing duplicate business logic. + +### 5. L10: Unified Model Inventory Accessibility +- Users can inspect and manage model inventory from both the CLI and GUI: + - **CLI**: `berry models list`, `berry models rescan`, `berry models roots`, `berry models add-root