Can't replicate result of Commonsense Reasoning task of LLaMa-7B with r = 16

[TonAnh]

Hi, I'm trying to replicate the DoRA result in your paper by following the hyperparameter config in Table 8. All the results seem fine except the case LLaMa-7B with rank = 16.

Here are the fine-tuning results when I tried to replicate:

Model	r	lr	BoolQ	PIQA	SIQA	HellaSwag	WinoGrande	ARC-e	ARC-c	OBQA	Average
LLaMA-7B-DoRA (first time)	16	2e-4	68.7	56.9	62.8	76	11.5	72.4	58.7	74	60.1
LLaMA-7B-DoRA (second time)	16	2e-4	67.2	65.3	54.8	46.3	55	43.1	35.4	61.8	53.6

Here are the evaluation results I ran on my machine using your checkpoint:

Model	r	lr	BoolQ	PIQA	SIQA	HellaSwag	WinoGrande	ARC-e	ARC-c	OBQA	Average
LLaMA-7B-DoRA (checkpoint)	16	2e-4	70	82.6	79.7	83.2	80.6	80.6	65.4	77.6	77.5

Any ideas about what could cause the differences?

[OCEANOUXIN]

Hi, I'm trying to replicate the DoRA result in your paper by following the hyperparameter config in Table 8. All the results seem fine except the case LLaMa-7B with rank = 16.你好，我正在尝试通过遵循 Table 8 中的超参数配置来复现你论文中的 DoRA 结果。所有结果似乎都很好，除了秩为 16 的 LLaMa-7B 情况。

Here are the fine-tuning results when I tried to replicate:这是我尝试复现时的微调结果：

Model r lr  学习率 BoolQ PIQA SIQA HellaSwag WinoGrande ARC-e ARC-c OBQA Average  平均
LLaMA-7B-DoRA (first time)
LLaMA-7B-DoRA (首次) 16 2e-4 68.7 56.9 62.8 76 11.5 72.4 58.7 74 60.1
LLaMA-7B-DoRA (second time)
LLaMA-7B-DoRA (第二次) 16 2e-4 67.2 65.3 54.8 46.3 55 43.1 35.4 61.8 53.6
Here are the evaluation results I ran on my machine using your checkpoint:这是我在使用您的检查点在我的机器上运行的评价结果：

Model r lr  学习率 BoolQ PIQA SIQA HellaSwag WinoGrande ARC-e ARC-c OBQA Average  平均
LLaMA-7B-DoRA (checkpoint) 16 2e-4 70 82.6 79.7 83.2 80.6 80.6 65.4 77.6 77.5
Any ideas about what could cause the differences?有什么想法可能导致这些差异？

Hi, do you compare the results to Lora? Which one is better dora or lora?