Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
5bd38e3
docs: add internal web app design spec
shenge010101 Mar 26, 2026
958d191
chore: ignore local worktrees
shenge010101 Mar 26, 2026
20ba3ee
docs: add internal web app implementation plan
shenge010101 Mar 26, 2026
4360300
feat: bootstrap internal web app package
shenge010101 Mar 26, 2026
69d8fd7
feat: add persistent job repository
shenge010101 Mar 26, 2026
84c06c7
feat: add queue coordination and worker recovery
shenge010101 Mar 26, 2026
35e6e86
feat: add upload draft and video validation service
shenge010101 Mar 26, 2026
1a02900
feat: extract masking refinement service
shenge010101 Mar 26, 2026
99f4811
feat: wrap matanyone2 inference for queued jobs
shenge010101 Mar 26, 2026
20b8037
feat: add transparent export packaging
shenge010101 Mar 26, 2026
12fa4e4
feat: add internal web upload and job flow
shenge010101 Mar 26, 2026
b4efec4
test: cover internal web app flow and docs
shenge010101 Mar 26, 2026
b85f152
feat: implement real masking, worker, and artifacts
shenge010101 Mar 26, 2026
24f3fc2
feat: implement prores 4444 export
shenge010101 Mar 26, 2026
e6a7d45
feat: add draft template frame and annotate page
shenge010101 Mar 26, 2026
0e03181
feat: wire internal web pages to browser flow
shenge010101 Mar 26, 2026
a027aff
feat: expose job warnings in status api
shenge010101 Mar 26, 2026
173dbe4
fix: validate venv runtime and python310 support
shenge010101 Mar 26, 2026
4c30811
feat: add internal webapp smoke runner
shenge010101 Mar 26, 2026
0833703
feat: add powershell entrypoints for internal webapp
shenge010101 Mar 26, 2026
66fc7d8
docs: add internal webapp ui redesign spec
shenge010101 Mar 26, 2026
af3edea
fix: support m4v inference inputs
shenge010101 Mar 26, 2026
f451072
docs: add ui redesign implementation plan
shenge010101 Mar 26, 2026
fbd12d8
fix: stabilize webapp review blockers
shenge010101 Mar 26, 2026
850a5c2
feat: add workbench annotation state api
shenge010101 Mar 26, 2026
47637ec
feat: redesign internal webapp upload shell
shenge010101 Mar 26, 2026
55fe510
feat: rebuild annotation workbench ui
shenge010101 Mar 26, 2026
2791ebe
fix: declare matplotlib for sam runtime
shenge010101 Mar 26, 2026
792ba18
feat: redesign result review page
shenge010101 Mar 26, 2026
6040856
docs: update webapp redesign workflow
shenge010101 Mar 26, 2026
5b8857a
feat: tighten workbench stage flow
shenge010101 Mar 26, 2026
b1c595b
feat: add canvas mask review modes
shenge010101 Mar 26, 2026
a451c1d
feat: add undo and reset controls
shenge010101 Mar 26, 2026
45f200a
feat: add workbench keyboard shortcuts
shenge010101 Mar 26, 2026
87f49d5
feat: add browser overlay review
shenge010101 Mar 26, 2026
6b6aa7f
feat: enrich review panel and target layer controls
shenge010101 Mar 27, 2026
c24b788
fix: scope stop script orphan detection
shenge010101 Mar 27, 2026
bdf0493
feat: switch internal webapp annotation to sam2
shenge010101 Mar 27, 2026
3fc820e
feat: rebuild annotation workbench workflow
shenge010101 Mar 27, 2026
aa9ffa8
feat: apply refine presets to saved masks
shenge010101 Mar 27, 2026
f15904a
feat: refresh preset previews in the workbench
shenge010101 Mar 27, 2026
71678b9
feat: add per-target review to results page
shenge010101 Mar 27, 2026
1501b55
docs: finalize ui rebuild handoff
shenge010101 Mar 27, 2026
26dc283
feat: switch workbench to sam3 keyframe flow
shenge010101 Mar 27, 2026
e5e69e5
fix: add keyframe scrubber and live preview feedback
shenge010101 Mar 27, 2026
be773ec
fix: stabilize browser previews across the webapp
shenge010101 Mar 27, 2026
e34ff79
refactor: merge keyframe scrubbing into the main canvas
shenge010101 Mar 27, 2026
e1a1ebf
feat: add processing range and anchor workflow
shenge010101 Mar 27, 2026
79af00a
feat: simplify annotation monitor workflow
shenge010101 Mar 27, 2026
5300ba6
fix: keep source monitor visible while scrubbing
shenge010101 Mar 27, 2026
797c680
feat: unify the internal webapp workspace
shenge010101 Mar 27, 2026
a96a429
feat: tighten workspace timeline and matte controls
shenge010101 Mar 27, 2026
8bf5166
refactor: remove compare drawer from workspace
shenge010101 Mar 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,7 @@ hugging_face/assets/
results/
test_sample/
pretrained_models/
data/
data/
.worktrees/
runtime/
.venv/
119 changes: 119 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,21 @@
pip3 install -r hugging_face/requirements.txt
```

### Windows `.venv` Setup For Internal Web App
If you are running the internal web app on a Windows workstation, a local `.venv` is the most reliable setup path:

```powershell
py -3.10 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 --index-url https://download.pytorch.org/whl/cu128
python -m pip install -e .
python -m pip install decord
python -m pip install -e D:\my_app\lens_hunter2\models\sam3\sam3_repo --no-deps
```

The gradio demo keeps its extra dependencies in [`hugging_face/requirements.txt`](./hugging_face/requirements.txt), so install those separately only when you need the demo UI.

## 🔥 Inference

### Download Model
Expand Down Expand Up @@ -138,6 +153,110 @@ By launching, an interactive interface will appear as follow.

![overall_teaser](assets/teaser_demo.gif)

## 🧪 Internal Web App
The repository now also includes a desktop-first internal web app for the queue-based review workflow.

Current flow:

1. Upload a short source clip from the `New Session` page.
2. Enter the `Annotation Workbench` and build one or more target layers from the template frame.
3. Submit the queued job and inspect `source / overlay / alpha / foreground` from the `Result Review` page before downloading artifacts.

Current outputs:

- `foreground.mp4`
- `alpha.mp4`
- `rgba_png.zip`
- `output_prores4444.mov`

### Annotation Workbench

The current workbench is a single-page desktop UI built around `SAM3 -> key-frame mask -> MatAnyone2 bidirectional video matting`.

Workbench capabilities:

- multi-target layer management
- arbitrary template-frame selection before annotation
- `Positive / Negative` point placement
- `Balanced / Hair / Edge / Motion` refine presets
- `Add / Remove / Feather` brush cleanup
- numeric `Preset strength / Motion softness / Temporal stability` controls
- `Source / Overlay / Mask` review modes
- export-mask selection before queue submission

The internal web app now uses local `SAM3` as the default interactive segmentation backend for the annotation workbench, while `MatAnyone2` remains the video matting backend after submission. When the selected template frame is not frame `0`, the worker now splits inference into forward and backward passes from that frame and stitches the result before export. You can still override the segmentation backend or checkpoint at launch time:

```powershell
$env:MATANYONE2_WEBAPP_SAM_BACKEND = "sam3"
$env:MATANYONE2_WEBAPP_SAM3_CHECKPOINT_PATH = "D:\my_app\lens_hunter2\models\sam3\checkpoints\sam3.pt"
# fallback: switch back to SAM2 if needed
$env:MATANYONE2_WEBAPP_SAM_BACKEND = "sam2"
$env:MATANYONE2_WEBAPP_SAM2_VARIANT = "sam2.1_hiera_large"
```

### Start For Testing

Recommended on Windows:

```powershell
.\scripts\start_internal_webapp.ps1
```

Open:

```text
http://127.0.0.1:8010
```

Check or stop the service:

```powershell
.\scripts\check_internal_webapp.ps1
.\scripts\stop_internal_webapp.ps1
```

Manual split-process launch is still available:

```shell
python scripts/run_internal_webapp.py
```

Run the worker in a separate process:

```shell
python scripts/run_internal_worker.py
```

### Quick Test Path

1. Upload a short clip.
2. Choose the best template frame for segmentation.
3. Use positive and negative clicks to isolate the person.
4. Switch the preset to `Hair`, `Edge`, `Motion`, or `Balanced` based on the edge you care about.
5. Tune `Preset strength`, `Motion softness`, and `Temporal stability`.
6. Use `Add / Remove / Feather` to manually correct the key-frame mask.
7. Save one or more targets, choose which saved masks to export, and submit the job.
6. Review `Overlay / Alpha / Foreground` on the result page before downloading artifacts.

### Smoke Test

Run an end-to-end smoke test that launches the web app, launches the worker, submits two back-to-back jobs, and verifies queueing plus artifact export:

```shell
python scripts/smoke_internal_webapp.py --copies 2
```

Windows one-click helpers are also available:

```powershell
.\scripts\start_internal_webapp.ps1
.\scripts\check_internal_webapp.ps1
.\scripts\stop_internal_webapp.ps1
.\scripts\smoke_internal_webapp.ps1 --copies 2
```

The smoke flow submits jobs through the same HTTP workflow used by the UI, carries the active SAM backend settings into the temporary service environment, and verifies that the queue reaches a completed result with generated artifacts.

## 📊 Evaluation
Please refer to the [evaluation documentation](docs/EVAL.md) for details.

Expand Down
Loading