Skip to content

fix(runners): normalize painted inpainting masks for each provider (fixes #106) - #159

Merged
BerryUIKI merged 4 commits into
devfrom
bugfix/normalize-inpaint-masks-106
Oct 6, 2026
Merged

BerryUIKI merged 4 commits into
devfrom
bugfix/normalize-inpaint-masks-106

Conversation

@BerryUIKI

Copy link
Copy Markdown
Owner

Summary

Fixes #106 by normalizing Berry-painted masks to each provider's specific mask semantics. The canonical Berry convention (painted white = edit, transparent = protect) is now correctly converted for ComfyUI, WebUI, OpenAI, and Fal.ai.

Problem

Different providers interpret mask alpha channels differently:

  • ComfyUI: LoadImage MASK output = 1 - alpha, so painted regions (alpha=255) became 0 (protect) instead of 1 (edit)
  • WebUI: Expects grayscale where white=edit, black=protect
  • OpenAI: Expects transparent=edit, opaque=protect (inverted from Berry)
  • Fal.ai: Expects opaque=edit, transparent=protect (matches Berry)

The original implementation passed Berry masks directly without conversion, causing painted regions to be protected instead of edited in ComfyUI and OpenAI.

Implementation

Canonical Berry Convention

Documented in mask_converter.py:

  • Painted regions (opaque white, alpha=255) = areas TO BE EDITED
  • Unpainted regions (transparent, alpha=0) = areas TO BE PROTECTED

Provider-Specific Conversions

  1. ComfyUI (normalize_mask_for_comfyui):

    • Inverts alpha channel before upload
    • Creates temporary converted mask
    • Cleans up after upload completes
  2. WebUI (normalize_mask_for_webui):

    • Converts RGBA alpha to grayscale L mode
    • White (255) = edit, Black (0) = protect
  3. OpenAI (normalize_mask_for_openai):

    • Inverts alpha channel
    • Transparent (0) = edit for API
  4. Fal.ai (normalize_mask_for_fal_ai):

    • Validates format only
    • Already matches Berry convention

Integration Points

  • creative_runner._run_comfy(): Converts and uploads mask, validates dimensions
  • webui_runner._run_inpaint(): Converts mask before base64 encoding
  • creative_runner._run_cloud(): Converts based on provider_id
  • All paths validate source/mask dimension alignment

Cache Invalidation

Runner version bumped from 0.1.0 to 0.2.0 in compute_creative_cache_hash() to prevent reuse of results generated with incorrect mask semantics.

Tests

New Test Suite (test_mask_converter.py - 13 tests, all passing)

  • ✅ Berry mask convention documentation
  • ✅ ComfyUI alpha inversion
  • ✅ WebUI grayscale conversion
  • ✅ OpenAI alpha inversion
  • ✅ Fal.ai passthrough validation
  • ✅ Dimension validation (success, mismatch, file not found)
  • ✅ Dimension preservation during conversion
  • ✅ Partial alpha (anti-aliased edges)
  • ✅ Fully painted mask
  • ✅ Fully transparent mask

Existing Tests

  • ✅ 22 ComfyUI bridge and upload tests passing
  • ✅ No regressions in existing functionality

Verification

What Was Tested

  • Unit tests verify correct alpha inversion for each provider
  • Dimension validation prevents misaligned masks
  • File handle management (Windows lock issues resolved)
  • Edge cases: partial transparency, fully painted, fully transparent

What Remains Unverified

⚠️ Real GPU inference with actual ComfyUI/WebUI/cloud providers

Contract tests verify payload construction and mask conversion logic. Live inference testing requires:

  • Running ComfyUI instance with real models
  • WebUI instance with real models
  • Paid cloud provider API access
  • Manual visual verification of edited regions

Architecture Preservation

  • ✅ Original Berry assets remain unchanged
  • ✅ Source/mask dimensions strictly validated
  • ✅ Deterministic caching preserved (version bump)
  • ✅ API-first, local-optional architecture maintained
  • ✅ No GUI or engine lifecycle changes

Documentation

Added MASK_NORMALIZATION.md documenting:

  • Canonical Berry mask convention
  • Provider-specific contracts
  • Integration points
  • Cache invalidation strategy

Related Issues

Fixes #106

🤖 Generated with Claude Code

…ixes #106)

- Document canonical Berry mask convention: painted white = edit, transparent = protect
- Add mask_converter module with provider-specific normalization functions
- ComfyUI: invert alpha (LoadImage MASK = 1 - alpha)
- WebUI: convert alpha to grayscale (white=edit, black=protect)
- OpenAI: invert alpha (transparent=edit for API)
- Fal.ai: validate only (matches Berry convention)
- Validate source/mask dimension alignment before conversion
- Update creative_runner to apply conversions before execution
- Bump cache version 0.1.0 -> 0.2.0 to invalidate old results
- Add comprehensive test suite: 13 mask converter tests
- Test edge cases: partial alpha, fully painted, fully transparent
- Preserve original Berry assets unchanged
- Clean up temporary converted masks after ComfyUI upload

Issue #106 reported that ComfyUI LoadImage MASK output inverts alpha,
causing painted regions to be protected instead of edited. This fix
ensures consistent mask semantics across all supported providers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@BerryUIKI
BerryUIKI enabled auto-merge (squash) October 6, 2026 15:59
Pillow (PIL) is required by mask_converter.py for image manipulation
during mask normalization. CI was failing due to missing dependency.
…ests

- Replace 1x1 PNG_BYTES with proper 512x512 images matching test dimensions
- Mock normalize_mask_for_comfyui in inpaint test to avoid file system issues
- Update test_comfy_asset_transfer_contract to detect converted mask uploads
- All creative_runner_upload tests now passing (5/5)

The mask conversion creates temporary files with different names, so tests
need to account for the conversion step and the temporary file cleanup.
The test was using a 1x1 pixel PNG which caused PIL.UnidentifiedImageError
when the mask converter tried to validate dimensions. Now creates a proper
512x512 RGBA PNG for testing.
@BerryUIKI
BerryUIKI merged commit 5a9c318 into dev Oct 6, 2026
3 checks passed
@BerryUIKI
BerryUIKI deleted the bugfix/normalize-inpaint-masks-106 branch October 6, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant