diff --git a/diffusion/README.md b/diffusion/README.md index e6d0833..1305c4f 100644 --- a/diffusion/README.md +++ b/diffusion/README.md @@ -6,7 +6,7 @@ This directory contains ready-to-use Diffusion application notebooks built with | No. | Model | Description | | :-- | :---- | :------------------------------ | -| 1 | [ddpm](./ddpm/train_ddpm_generation.ipynb) | ddpm training and inference application based on MindSpore NLP. | +| 1 | [ddpm](./ddpm/train_ddpm_generation.ipynb) | ddpm training and inference application based on MindSpore NLP. | 2 | [cartoonify](./cartoonify/cartoonify_demo.ipynb) | Photo-to-style portrait generation demo based on MindSpore and MindSpore NLP, supporting cartoonify, ghibli and guohua styles. | ## Contributing New Diffusion Applications diff --git a/diffusion/cartoonify/cartoonify_demo.ipynb b/diffusion/cartoonify/cartoonify_demo.ipynb new file mode 100644 index 0000000..3d012f6 --- /dev/null +++ b/diffusion/cartoonify/cartoonify_demo.ipynb @@ -0,0 +1,860 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a15a9187", + "metadata": {}, + "source": [ + "# 基于MindSpore NLP实现真人照片到特定风格图像生成案例开发\n", + "\n", + "## 项目概览\n", + "\n", + "本案例以开源项目 `cartoonify-main` 为迁移参考，将原始基于 Diffusers 的卡通风格能力整理为一个更适合 **MindSpore 课程应用案例** 的中文 notebook 与交互 DEMO。\n", + "\n", + "案例目标是围绕“**真人照片到风格化人像**”这一应用任务，构建一套可直接运行的推理方案。当前提供三类风格模板：\n", + "\n", + "- **吉卜力动画感**\n", + "- **Cartoonify 卡通角色插画感**\n", + "- **古风国画感**\n", + "\n", + "整体流程采用“先整图转风格、再局部修人物”的双阶段思路：前者负责风格表达，后者负责身份保持与脸部结构回融。\n" + ] + }, + { + "cell_type": "markdown", + "id": "e0205568", + "metadata": {}, + "source": [ + "## 案例介绍\n", + "\n", + "- **从单一卡通模型到多风格应用**：保留 `cartoonify` 的卡通迁移能力，并扩展为吉卜力与古风国画双附加风格。\n", + "- **从脚本调用到课程案例**：将原始模型调用方式整理为 notebook 结构，补齐中文说明、参数建议与 DEMO 入口。\n", + "- **从单阶段生成到身份保持增强**：增加脸部结构回融、细节恢复与风格后处理，提升人物一致性。\n", + "- **从实验界面到展示页**：重新设计交互界面，强调风格卡片、结果展示和参数引导。\n" + ] + }, + { + "cell_type": "markdown", + "id": "c8cff418", + "metadata": {}, + "source": [ + "## 推荐运行环境\n", + "\n", + "建议环境如下：\n", + "\n", + "| 组件 | 推荐版本 |\n", + "| :--- | :--- |\n", + "| Python | 3.10.x |\n", + "| MindSpore | 2.7.0 |\n", + "| MindSpore NLP | 0.5.1 |\n", + "| Gradio | 6.2.0 |\n", + "| 运行设备 | Ascend |\n" + ] + }, + { + "cell_type": "markdown", + "id": "3168770e", + "metadata": {}, + "source": [ + "## 初始化依赖\n", + "\n", + "本单元负责完成：补充 `mindtorch` 兼容占位；导入推理依赖；\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1272158f", + "metadata": {}, + "outputs": [], + "source": [ + "# -*- coding: utf-8 -*-\n", + "\n", + "try:\n", + " import mindtorch.autograd.function as _mt_function\n", + " if not hasattr(_mt_function, \"FunctionCtx\"):\n", + " class FunctionCtx:\n", + " pass\n", + " _mt_function.FunctionCtx = FunctionCtx\n", + "except Exception as _shim_error:\n", + " print(f\"[WARN] mindtorch compatibility shim skipped: {_shim_error}\")\n", + "\n", + "import os\n", + "import sys\n", + "import traceback\n", + "from dataclasses import dataclass\n", + "from functools import lru_cache\n", + "from typing import Dict, Tuple\n", + "\n", + "os.environ.setdefault(\"HF_HOME\", \"/root/autodl-tmp/hf_cache\")\n", + "os.environ.setdefault(\"HUGGINGFACE_HUB_CACHE\", \"/root/autodl-tmp/hf_cache/hub\")\n", + "os.environ.setdefault(\"TRANSFORMERS_CACHE\", \"/root/autodl-tmp/hf_cache/hub\")\n", + "\n", + "import numpy as np\n", + "from PIL import Image, ImageChops, ImageDraw, ImageEnhance, ImageFilter, ImageOps\n", + "\n", + "import mindspore as ms\n", + "import mindnlp\n", + "from diffusers import DDIMScheduler, StableDiffusionImg2ImgPipeline\n", + "import gradio as gr\n", + "\n", + "INFER_DTYPE = ms.float16\n", + "PIPELINE_DEVICE_MAP = \"cuda\"\n" + ] + }, + { + "cell_type": "markdown", + "id": "b6b86149", + "metadata": {}, + "source": [ + "## 推理运行参数\n", + "\n", + "为了减少单元执行顺序对 notebook 的影响，推理相关的常量统一放在这一单元中管理，同时打印当前版本信息与缓存路径，便于快速确认环境状态。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b563950c", + "metadata": {}, + "outputs": [], + "source": [ + "EXPECTED_MS = \"2.7.0\"\n", + "EXPECTED_MNLP = \"0.5.1\"\n", + "EXPECTED_PY_MIN = (3, 10)\n", + "EXPECTED_PY_MAX = (3, 12)\n", + "\n", + "print(\"MindSpore version:\", getattr(ms, \"__version__\", \"unknown\"))\n", + "print(\"MindSpore NLP version:\", getattr(mindnlp, \"__version__\", \"unknown\"))\n", + "print(\"Python version:\", sys.version.split()[0])\n", + "print(\"device_target:\", ms.get_context(\"device_target\"))\n", + "print(\"HF cache:\", os.environ.get(\"HUGGINGFACE_HUB_CACHE\"))\n" + ] + }, + { + "cell_type": "markdown", + "id": "2fe07475", + "metadata": {}, + "source": [ + "## 风格配置中心\n", + "\n", + "这里使用数据类统一描述风格配置，包括：\n", + "\n", + "- 模型地址\n", + "- 正向与反向提示词\n", + "- 推荐参数\n", + "- 界面说明\n", + "- 后处理类型\n", + "\n", + "这种写法便于后续继续扩展更多风格，而无需修改主推理流程。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3df22efc", + "metadata": {}, + "outputs": [], + "source": [ + "@dataclass(frozen=True)\n", + "class StyleProfile:\n", + " label: str\n", + " model_id: str\n", + " prompt: str\n", + " negative_prompt: str\n", + " recommended_strength: float\n", + " strength_cap_when_preserve: float\n", + " style_face_blend: float\n", + " line_keep: float\n", + " detail_keep: float\n", + " default_steps: int\n", + " default_guidance: float\n", + " intro: str\n", + " recommendation: str\n", + " finish_mode: str\n", + " cartoon_global_boost: float = 0.0\n", + " cartoon_face_boost: float = 0.0\n", + "\n", + "\n", + "STYLE_LIBRARY: Dict[str, StyleProfile] = {\n", + " \"吉卜力(Ghibli)\": StyleProfile(\n", + " label=\"吉卜力(Ghibli)\",\n", + " model_id=\"nitrosocke/Ghibli-Diffusion\",\n", + " prompt=(\n", + " \"portrait of the same exact person, same identity, same facial proportions, same jawline, same hairstyle, \"\n", + " \"studio ghibli anime film still, hand-painted anime illustration, clean lineart, soft cel shading, \"\n", + " \"natural expression, upper body, masterpiece\"\n", + " ),\n", + " negative_prompt=(\n", + " \"different person, changed face, aged face, child face, huge anime eyes, lowres, blurry, \"\n", + " \"bad face, deformed face, disfigured, mutated, extra eyes, bad anatomy, watermark, text, logo\"\n", + " ),\n", + " recommended_strength=0.40,\n", + " strength_cap_when_preserve=0.48,\n", + " style_face_blend=0.18,\n", + " line_keep=0.22,\n", + " detail_keep=0.28,\n", + " default_steps=25,\n", + " default_guidance=7.5,\n", + " intro=\"强调手绘动画电影质感，颜色柔和，线稿干净，适合半身人像与轻故事感照片。\",\n", + " recommendation=\"建议从 0.34 ~ 0.44 起步，若想更像本人，优先降低 strength。\",\n", + " finish_mode=\"ghibli\",\n", + " ),\n", + " \"卡通插画(Cartoon)\": StyleProfile(\n", + " label=\"卡通插画(Cartoon)\",\n", + " model_id=\"lavaman131/cartoonify\",\n", + " prompt=(\n", + " \"portrait of the same exact person, same identity, same facial proportions, same jawline, same hairstyle, \"\n", + " \"disney pixar style, polished cartoon illustration, animated feature film character portrait, clean cartoon lineart, \"\n", + " \"simplified facial planes, soft cel shading, stylized but recognizable face, upper body, masterpiece\"\n", + " ),\n", + " negative_prompt=(\n", + " \"different person, changed face, exaggerated face, huge eyes, tiny chin, malformed mouth, waxy skin, lowres, blurry, \"\n", + " \"deformed, bad anatomy, watermark, text, logo\"\n", + " ),\n", + " recommended_strength=0.52,\n", + " strength_cap_when_preserve=0.56,\n", + " style_face_blend=0.32,\n", + " line_keep=0.24,\n", + " detail_keep=0.28,\n", + " default_steps=28,\n", + " default_guidance=7.5,\n", + " intro=\"强调块面与轮廓，卡通化更明显，整体更接近动画角色插画效果。\",\n", + " recommendation=\"建议从 0.46 ~ 0.56 起步，卡通感强但更容易带来五官漂移。\",\n", + " finish_mode=\"cartoon\",\n", + " cartoon_global_boost=0.34,\n", + " cartoon_face_boost=0.26,\n", + " ),\n", + " \"古风国画(Guohua)\": StyleProfile(\n", + " label=\"古风国画(Guohua)\",\n", + " model_id=\"Langboat/Guohua-Diffusion\",\n", + " prompt=(\n", + " \"portrait of the same exact person, same identity, same facial proportions, same jawline, same hairstyle, \"\n", + " \"traditional Chinese guohua painting, elegant ancient Chinese portrait, ink wash painting, refined brush strokes, \"\n", + " \"soft rice paper texture, graceful costume portrait, artistic composition, masterpiece\"\n", + " ),\n", + " negative_prompt=(\n", + " \"different person, changed face, modern cartoon, photorealistic, 3d render, over-sharpened, lowres, blurry, \"\n", + " \"deformed face, bad anatomy, watermark, text, logo\"\n", + " ),\n", + " recommended_strength=0.38,\n", + " strength_cap_when_preserve=0.44,\n", + " style_face_blend=0.16,\n", + " line_keep=0.26,\n", + " detail_keep=0.30,\n", + " default_steps=30,\n", + " default_guidance=7.0,\n", + " intro=\"强调古风人像、笔墨层次与宣纸感，整体更柔和，适合营造东方绘画审美。\",\n", + " recommendation=\"建议从 0.32 ~ 0.42 起步，优先保留面部结构，再逐步增强笔触风格。\",\n", + " finish_mode=\"guohua\",\n", + " ),\n", + "}\n", + "\n", + "DEFAULT_STYLE = \"吉卜力(Ghibli)\"\n" + ] + }, + { + "cell_type": "markdown", + "id": "b45e114d", + "metadata": {}, + "source": [ + "## Pipeline 加载与缓存\n", + "\n", + "为减少重复加载模型造成的时间消耗，这里使用缓存方式管理不同风格对应的 pipeline。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "44058d36", + "metadata": {}, + "outputs": [], + "source": [ + "@lru_cache(maxsize=3)\n", + "def get_style_pipeline(model_id: str) -> StableDiffusionImg2ImgPipeline:\n", + " pipe = StableDiffusionImg2ImgPipeline.from_pretrained(\n", + " model_id,\n", + " ms_dtype=INFER_DTYPE,\n", + " device_map=PIPELINE_DEVICE_MAP,\n", + " )\n", + " pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)\n", + "\n", + " try:\n", + " pipe.enable_attention_slicing()\n", + " except Exception:\n", + " pass\n", + " try:\n", + " pipe.set_progress_bar_config(disable=True)\n", + " except Exception:\n", + " pass\n", + " try:\n", + " pipe.safety_checker = None\n", + " pipe.requires_safety_checker = False\n", + " except Exception:\n", + " pass\n", + "\n", + " print(f\"[OK] pipeline loaded: {model_id}\", flush=True)\n", + " return pipe\n" + ] + }, + { + "cell_type": "markdown", + "id": "80fc7e72", + "metadata": {}, + "source": [ + "## 图像处理与身份保持模块\n", + "\n", + "这一部分是本案例的关键增强点。整体策略为：\n", + "\n", + "1. 先做统一人像预处理；\n", + "2. 对整图做一次风格化生成；\n", + "3. 对面部区域进行“结构回注”，保留人物身份特征；\n", + "4. 根据不同风格再叠加差异化后处理。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e416e21e", + "metadata": {}, + "outputs": [], + "source": [ + "def ensure_rgb_image(image_obj) -> Image.Image:\n", + " if isinstance(image_obj, Image.Image):\n", + " return image_obj.convert(\"RGB\")\n", + " if isinstance(image_obj, ms.Tensor):\n", + " array = np.clip(image_obj.asnumpy(), 0, 255).astype(np.uint8)\n", + " return Image.fromarray(array).convert(\"RGB\")\n", + " if isinstance(image_obj, np.ndarray):\n", + " array = np.clip(image_obj, 0, 255).astype(np.uint8)\n", + " return Image.fromarray(array).convert(\"RGB\")\n", + " raise TypeError(f\"Unsupported image type: {type(image_obj)}\")\n", + "\n", + "\n", + "def prepare_portrait(image_obj: Image.Image, size: int) -> Image.Image:\n", + " image_obj = ImageOps.exif_transpose(image_obj).convert(\"RGB\")\n", + " image_obj = ImageEnhance.Sharpness(image_obj).enhance(1.12)\n", + " image_obj = ImageEnhance.Contrast(image_obj).enhance(1.05)\n", + " image_obj = ImageEnhance.Color(image_obj).enhance(1.02)\n", + " return ImageOps.fit(\n", + " image_obj,\n", + " (size, size),\n", + " method=Image.Resampling.LANCZOS,\n", + " centering=(0.5, 0.20),\n", + " )\n", + "\n", + "\n", + "def estimate_main_face_box(size: int) -> Tuple[int, int, int, int]:\n", + " left = int(size * 0.32)\n", + " top = int(size * 0.11)\n", + " right = int(size * 0.68)\n", + " bottom = int(size * 0.49)\n", + " return left, top, right, bottom\n", + "\n", + "\n", + "def create_soft_patch_mask(width: int, height: int, blur_radius: int) -> Image.Image:\n", + " mask = Image.new(\"L\", (width, height), 0)\n", + " drawer = ImageDraw.Draw(mask)\n", + " outer = (\n", + " int(width * 0.08),\n", + " int(height * 0.08),\n", + " int(width * 0.92),\n", + " int(height * 0.92),\n", + " )\n", + " inner = (\n", + " int(width * 0.18),\n", + " int(height * 0.14),\n", + " int(width * 0.82),\n", + " int(height * 0.86),\n", + " )\n", + " drawer.rounded_rectangle(outer, radius=max(8, min(width, height) // 9), fill=208)\n", + " drawer.ellipse(inner, fill=255)\n", + " return mask.filter(ImageFilter.GaussianBlur(radius=blur_radius))\n", + "\n", + "\n", + "def transfer_style_statistics(source_face: Image.Image, target_face: Image.Image) -> Image.Image:\n", + " source = np.asarray(source_face.convert(\"RGB\")).astype(np.float32)\n", + " target = np.asarray(target_face.convert(\"RGB\")).astype(np.float32)\n", + " merged = np.empty_like(source)\n", + " for channel in range(3):\n", + " src = source[..., channel]\n", + " tgt = target[..., channel]\n", + " src_mean, src_std = float(src.mean()), float(src.std()) + 1e-6\n", + " tgt_mean, tgt_std = float(tgt.mean()), float(tgt.std()) + 1e-6\n", + " merged[..., channel] = (src - src_mean) * (tgt_std / src_std) + tgt_mean\n", + " merged = np.clip(merged, 0, 255).astype(np.uint8)\n", + " return Image.fromarray(merged, mode=\"RGB\")\n", + "\n", + "\n", + "def match_luma_distribution(base_face: Image.Image, style_face: Image.Image) -> Image.Image:\n", + " style_y = style_face.convert(\"YCbCr\").split()[0]\n", + " channels = list(base_face.convert(\"YCbCr\").split())\n", + " channels[0] = Image.blend(channels[0], style_y, 0.35)\n", + " return Image.merge(\"YCbCr\", tuple(channels)).convert(\"RGB\")\n", + "\n", + "\n", + "def restore_facial_details(base_face: Image.Image, original_face: Image.Image, amount: float) -> Image.Image:\n", + " if amount <= 0:\n", + " return base_face\n", + " sharpened = original_face.filter(ImageFilter.UnsharpMask(radius=1.2, percent=135, threshold=2))\n", + " high_freq = ImageChops.subtract(sharpened, sharpened.filter(ImageFilter.GaussianBlur(radius=1.6)))\n", + " high_freq = ImageOps.autocontrast(high_freq)\n", + " high_freq = ImageEnhance.Contrast(high_freq).enhance(0.82)\n", + " restored = ImageChops.overlay(base_face, high_freq)\n", + " return Image.blend(base_face, restored, float(amount))\n", + "\n", + "\n", + "def keep_soft_lines(base_face: Image.Image, original_face: Image.Image, amount: float) -> Image.Image:\n", + " if amount <= 0:\n", + " return base_face\n", + " edges = original_face.convert(\"L\").filter(ImageFilter.FIND_EDGES).filter(ImageFilter.GaussianBlur(radius=1.0))\n", + " edges = ImageOps.autocontrast(edges)\n", + " edges = edges.point(lambda pixel: int(255 - pixel * 0.42))\n", + " edge_rgb = Image.merge(\"RGB\", (edges, edges, edges))\n", + " with_lines = ImageChops.multiply(base_face, edge_rgb)\n", + " return Image.blend(base_face, with_lines, float(amount))\n", + "\n", + "\n", + "def stylize_face_seed(face_image: Image.Image, finish_mode: str) -> Image.Image:\n", + " face_image = face_image.convert(\"RGB\")\n", + " if finish_mode == \"cartoon\":\n", + " face_image = face_image.filter(ImageFilter.MedianFilter(size=3))\n", + " face_image = face_image.filter(ImageFilter.SMOOTH_MORE)\n", + " face_image = ImageOps.posterize(face_image, 5)\n", + " face_image = ImageEnhance.Color(face_image).enhance(1.10)\n", + " face_image = ImageEnhance.Contrast(face_image).enhance(1.10)\n", + " face_image = ImageEnhance.Sharpness(face_image).enhance(1.18)\n", + " return face_image\n", + " if finish_mode == \"guohua\":\n", + " face_image = face_image.filter(ImageFilter.SMOOTH_MORE)\n", + " face_image = ImageEnhance.Color(face_image).enhance(0.88)\n", + " face_image = ImageEnhance.Contrast(face_image).enhance(0.94)\n", + " face_image = ImageEnhance.Sharpness(face_image).enhance(0.96)\n", + " return face_image\n", + " face_image = face_image.filter(ImageFilter.SMOOTH)\n", + " face_image = ImageOps.posterize(face_image, 6)\n", + " face_image = ImageEnhance.Color(face_image).enhance(1.04)\n", + " face_image = ImageEnhance.Contrast(face_image).enhance(1.02)\n", + " face_image = ImageEnhance.Sharpness(face_image).enhance(1.06)\n", + " return face_image\n", + "\n", + "\n", + "def apply_cartoon_finish(image_obj: Image.Image, amount: float) -> Image.Image:\n", + " if amount <= 0:\n", + " return image_obj.convert(\"RGB\")\n", + " base = image_obj.convert(\"RGB\")\n", + " smooth = base.filter(ImageFilter.MedianFilter(size=3)).filter(ImageFilter.SMOOTH_MORE)\n", + " flat = ImageOps.posterize(smooth, 5)\n", + " flat = ImageEnhance.Color(flat).enhance(1.08)\n", + " flat = ImageEnhance.Contrast(flat).enhance(1.10)\n", + " edges = base.convert(\"L\").filter(ImageFilter.FIND_EDGES).filter(ImageFilter.GaussianBlur(radius=0.7))\n", + " edges = ImageOps.autocontrast(edges)\n", + " edges = edges.point(lambda pixel: max(36, 255 - int(pixel * 1.55)))\n", + " edge_rgb = Image.merge(\"RGB\", (edges, edges, edges))\n", + " merged = ImageChops.multiply(flat, edge_rgb)\n", + " merged = ImageEnhance.Sharpness(merged).enhance(1.10)\n", + " return Image.blend(base, merged, float(amount))\n", + "\n", + "\n", + "def apply_guohua_finish(image_obj: Image.Image) -> Image.Image:\n", + " base = image_obj.convert(\"RGB\")\n", + " softened = base.filter(ImageFilter.GaussianBlur(radius=0.6))\n", + " softened = Image.blend(base, softened, 0.35)\n", + " softened = ImageEnhance.Color(softened).enhance(0.84)\n", + " softened = ImageEnhance.Contrast(softened).enhance(0.93)\n", + " paper = Image.new(\"RGB\", softened.size, (246, 240, 227))\n", + " return Image.blend(paper, softened, 0.82)\n", + "\n", + "\n", + "def build_identity_patch(original_face: Image.Image, stylized_face: Image.Image, profile: StyleProfile) -> Image.Image:\n", + " remapped = transfer_style_statistics(original_face, stylized_face)\n", + " remapped = match_luma_distribution(remapped, stylized_face)\n", + " seeded = stylize_face_seed(remapped, profile.finish_mode)\n", + " fused = Image.blend(seeded, stylized_face, float(profile.style_face_blend))\n", + " fused = restore_facial_details(fused, original_face, float(profile.detail_keep))\n", + " fused = keep_soft_lines(fused, original_face, float(profile.line_keep))\n", + " if profile.finish_mode == \"cartoon\":\n", + " fused = apply_cartoon_finish(fused, amount=float(profile.cartoon_face_boost))\n", + " fused = ImageEnhance.Color(fused).enhance(1.05)\n", + " fused = ImageEnhance.Contrast(fused).enhance(1.08)\n", + " fused = ImageEnhance.Sharpness(fused).enhance(1.20)\n", + " elif profile.finish_mode == \"guohua\":\n", + " fused = apply_guohua_finish(fused)\n", + " fused = ImageEnhance.Sharpness(fused).enhance(0.96)\n", + " else:\n", + " fused = ImageEnhance.Color(fused).enhance(1.02)\n", + " fused = ImageEnhance.Sharpness(fused).enhance(1.10)\n", + " return fused\n" + ] + }, + { + "cell_type": "markdown", + "id": "c63f9cfe", + "metadata": {}, + "source": [ + "## 主推理流程\n", + "\n", + "主流程包含两个阶段：\n", + "\n", + "1. **整图风格化**：使用目标风格模型完成全图 `img2img` 推理；\n", + "2. **局部身份回注**：在面部区域回注原始结构，减少“重新捏脸”的问题。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "af437ac0", + "metadata": {}, + "outputs": [], + "source": [ + "def run_img2img(\n", + " pipe: StableDiffusionImg2ImgPipeline,\n", + " prompt: str,\n", + " negative_prompt: str,\n", + " image_obj: Image.Image,\n", + " strength: float,\n", + " steps: int,\n", + " guidance_scale: float,\n", + "):\n", + " result = pipe(\n", + " prompt=prompt,\n", + " negative_prompt=negative_prompt,\n", + " image=image_obj,\n", + " strength=float(strength),\n", + " num_inference_steps=int(steps),\n", + " guidance_scale=float(guidance_scale),\n", + " )\n", + " if hasattr(result, \"images\") and result.images:\n", + " return result.images[0]\n", + " if isinstance(result, (list, tuple)) and result:\n", + " return result[0]\n", + " return result\n", + "\n", + "\n", + "def stylize_portrait(\n", + " image: Image.Image,\n", + " style_name: str,\n", + " strength: float,\n", + " steps: int,\n", + " guidance_scale: float,\n", + " seed: int,\n", + " size: int,\n", + " preserve_identity: bool,\n", + ") -> Image.Image:\n", + " if image is None:\n", + " raise ValueError(\"请先上传一张真人照片。\")\n", + " if style_name not in STYLE_LIBRARY:\n", + " raise ValueError(f\"暂不支持风格：{style_name}\")\n", + "\n", + " profile = STYLE_LIBRARY[style_name]\n", + " pipe = get_style_pipeline(profile.model_id)\n", + " prepared = prepare_portrait(image, size=size)\n", + "\n", + " if int(seed) > 0:\n", + " ms.set_seed(int(seed))\n", + " np.random.seed(int(seed))\n", + "\n", + " global_strength = min(float(strength), float(profile.strength_cap_when_preserve)) if preserve_identity else float(strength)\n", + "\n", + " global_result = ensure_rgb_image(\n", + " run_img2img(\n", + " pipe=pipe,\n", + " prompt=profile.prompt,\n", + " negative_prompt=profile.negative_prompt,\n", + " image_obj=prepared,\n", + " strength=global_strength,\n", + " steps=int(steps),\n", + " guidance_scale=float(guidance_scale),\n", + " )\n", + " )\n", + "\n", + " if profile.finish_mode == \"cartoon\":\n", + " global_result = apply_cartoon_finish(global_result, amount=float(profile.cartoon_global_boost))\n", + " elif profile.finish_mode == \"guohua\":\n", + " global_result = apply_guohua_finish(global_result)\n", + "\n", + " if not preserve_identity:\n", + " return global_result\n", + "\n", + " face_box = estimate_main_face_box(size)\n", + " origin_face = prepared.crop(face_box)\n", + " stylized_face = global_result.crop(face_box)\n", + " patched_face = build_identity_patch(origin_face, stylized_face, profile)\n", + "\n", + " patch_width = face_box[2] - face_box[0]\n", + " patch_height = face_box[3] - face_box[1]\n", + " patched_face = patched_face.resize((patch_width, patch_height), Image.Resampling.LANCZOS)\n", + " face_mask = create_soft_patch_mask(patch_width, patch_height, blur_radius=max(3, size // 128))\n", + "\n", + " merged = global_result.copy()\n", + " merged.paste(patched_face, (face_box[0], face_box[1]), mask=face_mask)\n", + " return merged\n" + ] + }, + { + "cell_type": "markdown", + "id": "c76aaa8d", + "metadata": {}, + "source": [ + "## 风格联动说明\n", + "\n", + "为了减少用户试错，界面在切换风格时会自动刷新：\n", + "\n", + "- 风格介绍\n", + "- 推荐调参建议\n", + "- 默认 `strength`\n", + "- 默认 `steps`\n", + "- 默认 `guidance_scale`\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dd663d27", + "metadata": {}, + "outputs": [], + "source": [ + "def get_style_panel(style_name: str):\n", + " profile = STYLE_LIBRARY[style_name]\n", + " info_text = (\n", + " f\"### {profile.label}\\n\"\n", + " f\"**风格特点：** {profile.intro}\\n\\n\"\n", + " f\"**调参建议：** {profile.recommendation}\"\n", + " )\n", + " return (\n", + " info_text,\n", + " profile.recommended_strength,\n", + " profile.default_steps,\n", + " profile.default_guidance,\n", + " )\n", + "\n", + "\n", + "def generate_for_ui(img, style, strength, steps, guidance, seed, size, preserve_identity):\n", + " try:\n", + " result = stylize_portrait(\n", + " image=img,\n", + " style_name=style,\n", + " strength=float(strength),\n", + " steps=int(steps),\n", + " guidance_scale=float(guidance),\n", + " seed=int(seed),\n", + " size=int(size),\n", + " preserve_identity=bool(preserve_identity),\n", + " )\n", + " return result\n", + " except Exception as error:\n", + " traceback.print_exc()\n", + " raise gr.Error(str(error))\n", + "\n", + "\n", + "APP_DESCRIPTION = '''\n", + "# 真人照片到特定风格图像生成DEMO\n", + "支持吉卜力、Cartoonify 卡通插画、古风国画三类效果。\n", + "'''\n" + ] + }, + { + "cell_type": "markdown", + "id": "b8489c63", + "metadata": {}, + "source": [ + "## 交互 DEMO 布局\n", + "\n", + "界面层重新组织为三部分：\n", + "\n", + "- 顶部展示项目说明与使用建议；\n", + "- 中间采用“左参数、右结果”的双栏布局；\n", + "- 风格卡片、参数区、结果区彼此分组，界面更接近成品演示页。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "02c48a43", + "metadata": {}, + "outputs": [], + "source": [ + "CUSTOM_CSS = '''\n", + ".demo-shell {max-width: 1180px; margin: 0 auto;}\n", + ".hero-card {\n", + " padding: 4px 0 6px 0;\n", + " border-radius: 0;\n", + " background: transparent;\n", + " border: none;\n", + " box-shadow: none;\n", + "}\n", + ".panel-note {\n", + " border-radius: 14px;\n", + " padding: 10px 14px;\n", + " background: #f7f1e7;\n", + " border: 1px solid #eadfce;\n", + "}\n", + ".compact-gap {gap: 8px;}\n", + ".generate-btn button {\n", + " background: linear-gradient(135deg, #c96a3d 0%, #a84a2a 100%) !important;\n", + " border: none !important;\n", + " color: white !important;\n", + " box-shadow: 0 10px 24px rgba(169, 74, 42, 0.22) !important;\n", + "}\n", + ".generate-btn button:hover {\n", + " filter: brightness(1.04);\n", + "}\n", + ".gradio-container h1, .gradio-container h2, .gradio-container h3 {\n", + " letter-spacing: 0.02em;\n", + "}\n", + "'''\n", + "\n", + "with gr.Blocks(title=\"Photo2Style DEMO\", css=CUSTOM_CSS) as demo:\n", + " with gr.Column(elem_classes=[\"demo-shell\"]):\n", + " with gr.Group(elem_classes=[\"hero-card\"]):\n", + " gr.Markdown(APP_DESCRIPTION)\n", + "\n", + " with gr.Row(equal_height=True, elem_classes=[\"compact-gap\"]):\n", + " with gr.Column(scale=4):\n", + " gr.Markdown(\"## 控制台\")\n", + " style_name = gr.Dropdown(\n", + " choices=list(STYLE_LIBRARY.keys()),\n", + " value=DEFAULT_STYLE,\n", + " label=\"选择风格模板\",\n", + " )\n", + " style_card = gr.Markdown()\n", + "\n", + " with gr.Group():\n", + " input_image = gr.Image(type=\"pil\", label=\"上传原始照片\")\n", + " preserve_identity = gr.Checkbox(\n", + " value=True,\n", + " label=\"人物特征保留增强\",\n", + " )\n", + " size = gr.Dropdown(\n", + " choices=[512, 640, 768],\n", + " value=512,\n", + " label=\"输出尺寸\",\n", + " )\n", + " strength = gr.Slider(\n", + " minimum=0.20,\n", + " maximum=0.75,\n", + " value=STYLE_LIBRARY[DEFAULT_STYLE].recommended_strength,\n", + " step=0.01,\n", + " label=\"风格强度（strength）\",\n", + " )\n", + " steps = gr.Slider(\n", + " minimum=10,\n", + " maximum=50,\n", + " value=STYLE_LIBRARY[DEFAULT_STYLE].default_steps,\n", + " step=1,\n", + " label=\"推理步数（steps）\",\n", + " )\n", + " guidance = gr.Slider(\n", + " minimum=1.0,\n", + " maximum=12.0,\n", + " value=STYLE_LIBRARY[DEFAULT_STYLE].default_guidance,\n", + " step=0.5,\n", + " label=\"文本引导强度（guidance_scale）\",\n", + " )\n", + " seed = gr.Number(value=0, precision=0, label=\"随机种子（0 表示随机）\")\n", + " run_button = gr.Button(\n", + " \"生成风格化结果\",\n", + " variant=\"primary\",\n", + " size=\"lg\",\n", + " elem_classes=[\"generate-btn\"],\n", + " )\n", + "\n", + " with gr.Column(scale=5):\n", + " gr.Markdown(\"## 结果展示\")\n", + " with gr.Row(elem_classes=[\"compact-gap\"]):\n", + " output_image = gr.Image(type=\"pil\", label=\"生成结果\", height=560)\n", + "\n", + " gr.Markdown(\n", + " \"
\"\n", + " \"结果说明： 若出现风格很强但不像本人，可降低 strength；\"\n", + " \"若风格表达不明显，可适当提高 steps 或 strength。\"\n", + " \"
\"\n", + " )\n", + "\n", + " with gr.Row():\n", + " gr.Markdown(\n", + " \"### 使用提示\\n\"\n", + " \"- Cartoonify 风格更适合中高强度参数\\n\"\n", + " \"- 吉卜力更适合较柔和的强度范围\\n\"\n", + " \"- 古风国画建议从低强度开始，逐步增强笔触感\"\n", + " )\n", + "\n", + " demo.load(\n", + " fn=lambda: get_style_panel(DEFAULT_STYLE),\n", + " outputs=[style_card, strength, steps, guidance],\n", + " )\n", + " style_name.change(\n", + " fn=get_style_panel,\n", + " inputs=[style_name],\n", + " outputs=[style_card, strength, steps, guidance],\n", + " )\n", + " run_button.click(\n", + " fn=generate_for_ui,\n", + " inputs=[input_image, style_name, strength, steps, guidance, seed, size, preserve_identity],\n", + " outputs=[output_image],\n", + " )\n", + "\n", + "demo\n" + ] + }, + { + "cell_type": "markdown", + "id": "841f288b", + "metadata": {}, + "source": [ + "## 启动 DEMO\n", + "\n", + "执行下方单元即可启动 Gradio 服务。如需自定义端口，可修改环境变量 `PORT` 或直接修改 `server_port`。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fc558e4f", + "metadata": {}, + "outputs": [], + "source": [ + "demo.queue(max_size=20).launch(\n", + " server_name=\"0.0.0.0\",\n", + " server_port=int(os.getenv(\"PORT\", \"7861\")),\n", + " show_error=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "a8b83293", + "metadata": {}, + "source": [ + "## 后续可扩展方向\n", + "\n", + "- 接入更多风格模型，例如钢笔画、工笔画、迪士尼角色风。\n", + "- 引入更准确的人脸检测与多人脸选择策略。\n", + "- 将 DEMO 部署到魔乐社区，并在此处补充正式访问链接。\n", + "\n", + "> DEMO 链接：待部署后补充。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.10" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}