Skip to content

Add missing norm.weight parameters to cohere.json#674

Merged
cg123 merged 2 commits into
arcee-ai:mainfrom
jukofyork:Fix-cohere-schema-missing-norms
Jun 17, 2026
Merged

Add missing norm.weight parameters to cohere.json#674
cg123 merged 2 commits into
arcee-ai:mainfrom
jukofyork:Fix-cohere-schema-missing-norms

Conversation

@jukofyork

@jukofyork jukofyork commented Mar 24, 2026

Copy link
Copy Markdown
Contributor

This fixes the missing the self_attn.q_norm.weight and self_attn.k_norm.weight parameters, see:

https://huggingface.co/CohereLabs/c4ai-command-r-plus/blob/main/model.safetensors.index.json

{
  "weight_map": {
    "model.embed_tokens.weight": "model-00001-of-00044.safetensors",
    "model.layers.0.input_layernorm.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.mlp.down_proj.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.mlp.gate_proj.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.mlp.up_proj.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.self_attn.k_norm.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.self_attn.k_proj.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.self_attn.o_proj.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.self_attn.q_norm.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.self_attn.q_proj.weight": "model-00002-of-00044.safetensors",
    "model.layers.0.self_attn.v_proj.weight": "model-00002-of-00044.safetensors",
    "...",
}

Note

Low Risk
Single architecture JSON metadata change with no runtime logic; only affects which Cohere tensors mergekit recognizes during merges.

Overview
Updates the Cohere mergekit architecture template so per-layer weight lists match real checkpoints (e.g. Command R+).

Adds model.layers.${layer_index}.self_attn.q_norm.weight and model.layers.${layer_index}.self_attn.k_norm.weight to layer_templates, placed with the other self-attention tensors so merges no longer omit these norms.

Reviewed by Cursor Bugbot for commit bb241b6. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Mar 24, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@jukofyork

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@cg123 cg123 merged commit a6e4028 into arcee-ai:main Jun 17, 2026
6 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants