feat(models): register gemma4_unified_text text-only GPTQ definition#2925
Merged
Qubitium merged 1 commit intoJun 19, 2026
Merged
Conversation
The gemma4_unified family also has a standalone text stack (model_type "gemma4_unified_text", Gemma4UnifiedTextModel) registered in transformers, mirroring how gemma4_text parallels gemma4. ModelCloud#2921 added the composite gemma4_unified but not the text-only variant, so a text-only checkpoint still fails with "Unsupport model_type gemma4_unified_text". This extracts the shared decoder layout into a constant and adds Gemma4UnifiedTextQModel rooted at model.layers (model.norm / model.rotary_emb), reusing the same per-layer rope replay helper.
2 tasks
Collaborator
|
@Anai-Guo LGTM. Thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #2921 (which added the composite
gemma4_unified). Thegemma4_unifiedfamily also exposes a standalone text stack —model_type = "gemma4_unified_text"(Gemma4UnifiedTextModel), registered intransformers(src/transformers/models/auto/auto_mappings.py/modeling_auto.py) — exactly mirroring howgemma4_textparallelsgemma4.MODEL_MAPcurrently hasgemma4_text,gemma4, andgemma4_unifiedbut notgemma4_unified_text, so a text-only unified checkpoint still fails with:What this does
_GEMMA4_UNIFIED_DECODER(no behavior change for the composite definition).Gemma4UnifiedTextQModel, rooted atmodel.layerswithmodel.norm/model.rotary_emb(the standalone text-model layout, nolanguage_modelnesting), reusing the same per-layer-type rope replay helper as the composite variant."gemma4_unified_text"inMODEL_MAP.The quantizable layout is identical to the composite (
q/k/v/o_proj+gate/up/down_proj, withq/k/v_normand the four sandwich norms non-quantized); only the module-tree root and norm/rotary paths differ, just asGemma4TextQModeldiffers fromGemma4ForConditionalGenerationGPTQ.Test plan
auto.check_and_get_model_definition(<gemma4_unified_text ckpt>)returnsGemma4UnifiedTextQModel.gemma4_unified_textcheckpoint.I don't have local access to a
gemma4_unified_textcheckpoint to run the full quantization, so maintainer validation on real hardware is appreciated. The definition deliberately mirrors the merged (tested) compositegemma4_unifiedpattern.cc @Qubitium — this is the "additional patching" for the text-only variant you asked about in #2922.
🤖 Generated with Claude Code