Skip to content

fix: multimodal-LM (Qwen3.5_5) support for ex-LRP #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mann1x wants to merge 13 commits into
base: feature/ex-lrp
Choose a base branch
from
Open

fix: multimodal-LM (Qwen3.5_5) support for ex-LRP #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
90a9d86
docs: Update README with eX-LRP documentation and features
Tusm11 Apr 25, 2026
0125b2e
Address PR feedback: wire lrp_scores, migrate to lxt for true AttnLRP…
Tusm11 Apr 26, 2026
3e92daf
Fix LRP bugs: add safetensors support, implement global cache, suppor…
Tusm11 Apr 26, 2026
4ac9c1d
Fix pipeline argument parsing, enforce strict LRP scores without fall…
Tusm11 Apr 26, 2026
a899412
Fix test fallback logic, use dynamic mergekit-yaml path, and cleanly …
Tusm11 Apr 26, 2026
d6e8dd1
Guarantee gradient checkpointing restoration via try/finally and add …
Tusm11 Apr 26, 2026
ae32700
Fix build_mask import, correctly trace embedding gradients in LRP, is…
Tusm11 Apr 26, 2026
26571b8
Fix tied embeddings in LRP, remove incompatible 4-bit loading, fix le…
Tusm11 Apr 26, 2026
2022d37
fix: Address Cursor Bugbot issues - CUDA fallback, Qwen matching, sub…
Tusm11 Apr 26, 2026
9c039e6
fix: Address 4 Cursor Bugbot issues - weight normalization, cache mut…
Tusm11 Apr 26, 2026
a928e72
fix: Remove misleading dead code - unused total_weight validation
Tusm11 Apr 26, 2026
cfcfc82
fix: Address 2 pipeline issues - fail fast on missing paths, resolve …
Tusm11 Apr 26, 2026
ef76d4b
fix: multimodal-LM (Qwen3.5_5) support for ex-LRP
mann1x Apr 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9,193 changes: 3,931 additions & 5,262 deletions LRP_Merge.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

828 changes: 246 additions & 582 deletions README.md
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion examples/lrp.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
merge_method: lrp
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
base_model: meta-llama/Llama-2-13b-hf
parameters:
density: 0.7
models:
- model: psmathur/orca_mini_v3_13b
parameters:
weight: 1.0
lrp_scores: "orca_scores/lrp_scores.safetensors"
- model: garage-bAInd/Platypus2-13B
parameters:
weight: 1.0
lrp_scores: "platypus_scores/lrp_scores.safetensors"
dtype: float16
Loading