Skip to content

feat(catalog): Gemma 4 E4B/E2B uncensored GGUF models (#1836) #1837

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jaylfc merged 1 commit into from
Jul 15, 2026
+108 −0
Merged

feat(catalog): Gemma 4 E4B/E2B uncensored GGUF models (#1836) #1837

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
id: gemma-4-e2b-uncensored-gguf
name: Gemma 4 E2B Uncensored (GGUF)
type: model
version: 4.0.0
description: "Google Gemma 4 E2B (~5B raw / 2.3B effective via PLE), abliterated/uncensored GGUF (no refusals, original Google behaviour). Natively multimodal: text, image, video and audio in one model (mmproj needed for vision/audio). CPU/llama.cpp + Ollama path only; does NOT run on the Hailo NPU."

@kilo-code-bot kilo-code-bot Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: context_window: 131072 differs from the sibling Gemma 4 GGUF manifests (which use 32768). Gemma 4 E2B/E4B base context is 32K (with 128K window available via rope scaling in some builds). Confirm the 128K value is valid for these specific mradermacher GGUF uploads; if wrong, this can cause oversized KV-cache allocation/oom. The non-uncensored gemma-4-e2b-gguf uses 32768, so this inconsistency should be reconciled.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

homepage: https://huggingface.co/mradermacher/gemma-4-E2B-it-uncensored-GGUF
license: "Gemma Terms of Use"
capabilities: [chat, tool-calling, vision, audio]

@kilo-code-bot kilo-code-bot Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: capabilities: [chat, tool-calling, vision, audio] advertises multimodal vision/audio, but the manifest provides no mmproj (multimodal projector) download in either variant. The description itself notes "mmproj needed for vision/audio". Without the projector file, these capabilities cannot be fulfilled and UIs may offer vision/audio that silently fails. Either add the mmproj GGUF as a separate asset/variant, or drop vision/audio from capabilities until it's provided.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@coderabbitai coderabbitai Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Remove multimodal capabilities until the mmproj projector is included.

The manifests claim vision and audio capabilities, and the description correctly notes that an mmproj file is needed for them. However, the variants only download the base text .gguf file. Without the required projector file, backends will fail to process image or audio inputs.

  • app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L8-L8: Remove vision and audio from the capabilities list.
  • app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L8-L8: Remove vision and audio from the capabilities list.
🛠️ Proposed fix (apply to both files) 
-capabilities: [chat, tool-calling, vision, audio]
+capabilities: [chat, tool-calling]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
capabilities: [chat, tool-calling, vision, audio]
capabilities: [chat, tool-calling]
📍 Affects 2 files
  • app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L8-L8 (this comment)
  • app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L8-L8
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml` at line 8,
Remove vision and audio from the capabilities lists in both
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml:8-8 and
app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml:8-8, leaving only
capabilities supported by the downloaded base text GGUF files.

context_window: 131072

variants:
- id: q4_k_m
name: "Q4_K_M GGUF (3.4GB)"
format: gguf
size_mb: 3512
download_url: https://huggingface.co/mradermacher/gemma-4-E2B-it-uncensored-GGUF/resolve/main/gemma-4-E2B-it-uncensored.Q4_K_M.gguf
# TODO: sha256 not yet resolved (xet-backed repo, no LFS pointer). Fill in before release.
sha256: ''
requires:
backends:

@kilo-code-bot kilo-code-bot Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: sha256: '' (empty) leaves both variants without integrity verification. The PR documents this as a deliberate TODO for xet-backed repos, but shipping a release manifest with no checksum means downloads are unverified and silently corrupted/partial files are accepted. Add the real sha256 before release, or gate the manifest behind a draft/experimental flag if your schema supports it.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

- id: ollama
targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 4096
- id: llama-cpp
targets: [x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 4096
Comment on lines +24 to +26

@coderabbitai coderabbitai Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Restore missing apple-silicon and x86-cuda targets for llama-cpp.

The llama-cpp backend configuration omits apple-silicon and x86-cuda from its targets, despite these being natively supported by llama.cpp and explicitly enabled for the ollama backend on the same models. This omission will incorrectly prevent Mac and Nvidia users from utilizing the llama-cpp backend for these variants.

  • app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L24-L26: Add apple-silicon and x86-cuda to the Q4_K_M llama-cpp targets list.
  • app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L40-L42: Add apple-silicon and x86-cuda to the Q3_K_M llama-cpp targets list.
  • app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L24-L26: Add apple-silicon and x86-cuda to the Q4_K_M llama-cpp targets list.
  • app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L40-L42: Add apple-silicon and x86-cuda to the Q3_K_M llama-cpp targets list.
🛠️ Proposed fix (apply to all 4 locations) 
         - id: llama-cpp
-          targets: [x86-vulkan, arm-vulkan, cpu]
+          targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- id: llama-cpp
targets: [x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 4096
- id: llama-cpp
targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 4096
📍 Affects 2 files
  • app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L24-L26 (this comment)
  • app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L40-L42
  • app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L24-L26
  • app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L40-L42
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml` around lines 24
- 26, Add apple-silicon and x86-cuda to the targets list for every llama-cpp
entry: Q4_K_M and Q3_K_M in
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml at lines 24-26 and
40-42, and Q4_K_M and Q3_K_M in
app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml at lines 24-26 and
40-42; retain the existing x86-vulkan, arm-vulkan, and cpu targets.


- id: q3_k_m
name: "Q3_K_M GGUF (3.2GB)"
format: gguf
size_mb: 3277
download_url: https://huggingface.co/mradermacher/gemma-4-E2B-it-uncensored-GGUF/resolve/main/gemma-4-E2B-it-uncensored.Q3_K_M.gguf
# TODO: sha256 not yet resolved (xet-backed repo, no LFS pointer). Fill in before release.
sha256: ''
requires:
backends:
- id: ollama
targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 4096
- id: llama-cpp
targets: [x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 4096

hardware_tiers:
arm-npu-16gb: {recommended: q4_k_m}
arm-npu-32gb: {recommended: q4_k_m}
apple-silicon: {recommended: q4_k_m}
x86-cuda-12gb: {recommended: q4_k_m}

@kilo-code-bot kilo-code-bot Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The E2B manifest's hardware_tiers omits the 6GB tiers (arm-npu-6gb, x86-cuda-6gb, x86-vulkan-6gb) that the E4B manifest includes. Users on 6GB-class hardware get no recommendation for the smaller E2B model. Add the 6GB tier entries (recommending q3_k_m / q4_k_m as appropriate) for parity, since E2B's min_ram is only 4096MB.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

x86-vulkan-8gb: {recommended: q4_k_m}
x86-vulkan-16gb: {recommended: q4_k_m}
cpu-only: {recommended: q3_k_m}
arm-npu-4gb: {recommended: q3_k_m}
x86-cuda-4gb: {recommended: q3_k_m}
x86-vulkan-4gb: {recommended: q3_k_m}
54 changes: 54 additions & 0 deletions app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
id: gemma-4-e4b-uncensored-gguf
name: Gemma 4 E4B Uncensored (GGUF)
type: model
version: 4.0.0
description: "Google Gemma 4 E4B (~7.5B raw / 4B effective via PLE), abliterated/uncensored GGUF (no refusals, original Google behaviour). Natively multimodal: text, image, video and audio in one model (mmproj needed for vision/audio). CPU/llama.cpp + Ollama path only; does NOT run on the Hailo NPU."

@kilo-code-bot kilo-code-bot Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: context_window: 131072 differs from the sibling gemma-4-e4b-gguf manifest (32768). Confirm the 128K context is valid for these mradermacher GGUF uploads; a wrong value can cause oversized KV-cache allocation/oom. Reconcile with the non-uncensored E4B entry.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

homepage: https://huggingface.co/mradermacher/gemma-4-E4B-it-uncensored-GGUF
license: "Gemma Terms of Use"
capabilities: [chat, tool-calling, vision, audio]

@kilo-code-bot kilo-code-bot Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: capabilities: [chat, tool-calling, vision, audio] advertises multimodal vision/audio, but no mmproj projector GGUF is provided in either variant. The description notes "mmproj needed for vision/audio". Without the projector file these capabilities cannot be fulfilled. Add the mmproj asset or drop vision/audio until provided.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

context_window: 131072

variants:
- id: q4_k_m
name: "Q4_K_M GGUF (5.3GB)"
format: gguf
size_mb: 5470
download_url: https://huggingface.co/mradermacher/gemma-4-E4B-it-uncensored-GGUF/resolve/main/gemma-4-E4B-it-uncensored.Q4_K_M.gguf
# TODO: sha256 not yet resolved (xet-backed repo, no LFS pointer). Fill in before release.
sha256: ''
requires:
backends:

@kilo-code-bot kilo-code-bot Bot Jul 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: sha256: '' (empty) leaves both variants without integrity verification. Documented as a deliberate TODO, but a release manifest with no checksum accepts silently corrupted/partial downloads. Add the real sha256 before release.

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

- id: ollama
targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 6144
- id: llama-cpp
targets: [x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 6144

- id: q3_k_m
name: "Q3_K_M GGUF (4.9GB)"
format: gguf
size_mb: 4966
download_url: https://huggingface.co/mradermacher/gemma-4-E4B-it-uncensored-GGUF/resolve/main/gemma-4-E4B-it-uncensored.Q3_K_M.gguf
# TODO: sha256 not yet resolved (xet-backed repo, no LFS pointer). Fill in before release.
sha256: ''
requires:
backends:
- id: ollama
targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 6144
- id: llama-cpp
targets: [x86-vulkan, arm-vulkan, cpu]
min_ram_mb: 6144

hardware_tiers:
arm-npu-16gb: {recommended: q4_k_m}
arm-npu-32gb: {recommended: q4_k_m}
apple-silicon: {recommended: q4_k_m}
x86-cuda-12gb: {recommended: q4_k_m}
x86-vulkan-8gb: {recommended: q4_k_m}
x86-vulkan-16gb: {recommended: q4_k_m}
cpu-only: {recommended: q3_k_m}
arm-npu-6gb: {recommended: q3_k_m}
x86-cuda-6gb: {recommended: q3_k_m}
x86-vulkan-6gb: {recommended: q3_k_m}
Loading