QuIP results on Llama-2 models not reproducible.

[seannz]

Hello Jerry,

Thank you for sharing your QuIP code with us! I'm running into a bit of trouble with correctly running QuIP on the Llama-2 (7b/14b/70b) models:

1. The latest llama.py seems slightly broken, would you be able to verify for us whether it is working as intended for you?
2. So I put together my own llama.py using GPTQ's version of the code + your opt.py and got it to run without errors for Llama-2-7b-hf. However, the evaluation PPLs seem a bit high (wikitext2: 15.611, ptr-new: 353.37), suggesting that I must have done something wrong...

Any help/suggestions would be appreciated!

Sean,

[mostafaelhoushi]

I am facing a similar issue as well.