Skip to content

add 跨系统 DiT 推理对比评测框架 + 组件级 marker/offload 修复 - #39

Merged
shoyofight merged 1 commit into
mainfrom
test
Jul 22, 2026
Merged

shoyofight merged 1 commit into
mainfrom
test

Conversation

@shoyofight

Copy link
Copy Markdown
Collaborator
  • 新增统一评测编排器 eval_all.py:按任务路由 T2I/edit/video 指标, 模型只加载一次,从 config.resolved.yaml 权威读真实 prompt,质量分回写
  • 新增 edit/video 专属指标:directional CLIP、相邻帧时序一致性
  • 新增全指标出表 make_matrix_tables.py(DiT 纯采样独立列 + 端到端标口径)
  • cal_ir.py 内联 transformers>=5 自足兼容层(干净安装可跑)
  • src 组件级 phase marker(ED_BENCHMARK_MARKERS gate,默认零开销)
    • model_runtime offload 无 max-vram 时自动降级修复
  • 补 benchmark/requirements.txt 与 README 评测章节

Summary

Describe the change and the affected model/backend/API surface.

Validation

  • CPU build or tests
  • CUDA build or tests, if relevant
  • Python tests, if relevant
  • Correctness or image/video quality check, if generation behavior changed
  • Benchmark numbers, if performance changed

Notes

Include hardware, model paths in generic form, and any known limitations. Do not
include secrets, private paths, model weights, or generated large artifacts.

- 新增统一评测编排器 eval_all.py:按任务路由 T2I/edit/video 指标,
  模型只加载一次,从 config.resolved.yaml 权威读真实 prompt,质量分回写
- 新增 edit/video 专属指标:directional CLIP、相邻帧时序一致性
- 新增全指标出表 make_matrix_tables.py(DiT 纯采样独立列 + 端到端标口径)
- cal_ir.py 内联 transformers>=5 自足兼容层(干净安装可跑)
- src 组件级 phase marker(ED_BENCHMARK_MARKERS gate,默认零开销)
  + model_runtime offload 无 max-vram 时自动降级修复
- 补 benchmark/requirements.txt 与 README 评测章节

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 22, 2026 08:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shoyofight
shoyofight merged commit caeeed8 into main Jul 22, 2026
2 checks passed
@shoyofight
shoyofight deleted the test branch July 22, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants