Test the fine tune model

[ZaraNaSha]

Hello,
At first, thank you for interesting work and code.
I fine tune the model with the script and my data. Now I want to evaluate the model on my test data with "generate" function in "modeling_neo_chat.py" I got error. I do not know how to evaluate the model.
Best regards,
Zahra.

[Lucenova]

@ZaraNaSha If you load the model directly, please use its chat() method, which prepares the prompt and image tokens before calling generate().

If the error persists, could you please provide:

• The complete traceback;
• The code used to load and call the model;
• The checkpoint type/model size;
• Your PyTorch and Transformers versions.

With these details, we can identify the exact cause.
Best regards.

[ZaraNaSha]

@ZaraNaSha If you load the model directly, please use its chat() method, which prepares the prompt and image tokens before calling generate().

If the error persists, could you please provide:

* The complete traceback;

* The code used to load and call the model;

* The checkpoint type/model size;

* Your PyTorch and Transformers versions.

With these details, we can identify the exact cause. Best regards.

Thanks for your reply.
When I want to use the "chat" function, I got this error "AttributeError: 'NEOChatModel' object has no attribute 'template'", When I check the function I found that it uses "self.template" that is not defined.
Best regards.

[Lucenova]

@ZaraNaSha
Thanks for pointing this out.

self.template is initialized in NEOChatModel.__init__() from the model configuration:

self.template = config.template

Therefore, the error is likely caused by the loaded checkpoint/config not containing the template field, rather than self.template being completely undefined in the current implementation.

Could you please check whether your checkpoint directory contains a config.json, and whether it includes a field similar to:

"template": "..."

You can also verify it after loading the configuration:

config = NEOChatConfig.from_pretrained(model_path)
print(config.template)

If your fine-tuned checkpoint only saves adapter weights or contains an older config.json, please load the original base-model configuration or copy the corresponding template value from the base model into the fine-tuned checkpoint configuration.

As a temporary workaround, you may also set it explicitly after loading:

model.template = config.template

or provide the correct template name manually, but it should match the template used by the original checkpoint and training pipeline.

Please also share the model-loading code and the config.json from your fine-tuned checkpoint if the issue persists.

[ZaraNaSha]

template

I used the following two file to fine-tune the model. I checked the template in the config file. the default value is None and in the modeling_neo_chat.py the "NEOChatConfig" is invoke without any input. The checkpoint was saved every 200 steps and the files in its folder are:

zero3.json
2B_3SFTsi.sh