Add Qwen3.5 architecture support

[ZhangYiqun018]

Summary

• add architecture support for Qwen3.5 dense and MoE models
• cover Qwen3.5 multimodal wrapper weights, text decoder, vision tower, mixed linear/full attention, shared experts, and MTP weights
• add Qwen3.5 architecture tests for dense passthrough and MoE linear merges

Testing

• pytest tests/test_qwen35_architecture.py -q
• pytest -q
• verified official full-precision Qwen3.5 dense/MoE model index coverage with zero missing tensors

---

Note

Medium Risk
Adds a new architecture mapping that drives weight discovery for Qwen3.5 models (dense, MoE, multimodal), so mistakes could cause missing/extra tensors or incorrect merges for these checkpoints, but the change is largely additive and gated by architecture name checks.

Overview
Adds first-class Qwen3.5 architecture support by routing arch_info_for_config to a new qwen35_architecture_for_config resolver when the config architectures[0] matches known Qwen3.5 dense/MoE names.

Introduces mergekit/architecture/qwen35.py, defining module weight layouts for the Qwen3.5 text decoder (including mixed linear/full attention and optional attention biases), MoE/shared-expert variants, optional MTP blocks, and the multimodal vision tower; it also sets multimodal tagalong files and correct vocab-size config keys.

Adds tests/test_qwen35_architecture.py to assert full coverage of transformers state_dict keys for dense and MoE configs and to smoke-test passthrough (dense) and linear (MoE) merges end-to-end.

^{Reviewed by Cursor Bugbot for commit a812d01. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[ZhangYiqun018]

I have read the CLA Document and I hereby sign the CLA

[ZhangYiqun018]

recheck

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1296196de9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Include the full-attention output bias

When attention_bias is enabled, Qwen3.5 full-attention layers also instantiate self_attn.o_proj.bias (o_proj is constructed with bias=config.attention_bias in the Transformers Qwen3.5 implementation), but this architecture only enumerates q/k/v biases. For any Qwen3.5 dense or MoE checkpoint with attention_bias=True, mergekit will never plan or write the existing output-projection bias, so merged outputs silently drop that tensor.

Useful? React with 👍 / 👎.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 69f901f. Configure here.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a812d01a66

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Include attention biases for MTP layers

When attention_bias=True and mtp_num_hidden_layers is enabled, the MTP full-attention block uses the same q/k/v/o projections as Qwen3.5 decoder attention, so checkpoints contain mtp.layers.N.self_attn.{q,k,v,o}_proj.bias tensors. The main decoder path now enumerates those biases, but the MTP architecture still only lists the weights and norms here, so merges will silently omit the MTP attention biases from the output checkpoint.

Useful? React with 👍 / 👎.