Training with older v0 versus new v1

[AsmoKoskinen]

"F5-TTS v1 base model with better training and inference performance."

I trained a Finnish model at home (4070 Ti Super (16GB)). I used about four days of Finnish speeches as my data (...total duration of all wav files: 106:10:05.17 (HH:MM:SS.ss)). The training took approximately four days.

What are the benefits of retraining the model with v1? I might buy a 5090 (32G) one day, but I have no plans to train before that.

[SWivid]

What are the benefits of retraining the model with v1?

"F5-TTS v1 base model with better training and inference performance." is in comparison with v0.
If you use same setting (learning rate, max updates, total batch size), v1 converges faster and gives better WER/SIM result (lower word error rate, higher speaker similarity).

[AsmoKoskinen]

Thank you, when I got some day faster GPU I do that again.

[AsmoKoskinen]

I started a new training with the same Finnish dataset as before. Looks good for the first day. And sounds, too.

RTX 4070 Ti Super.

{
"exp_name": "F5TTS_v1_Base",
"learning_rate": 0.0001,
"batch_size_per_gpu": 2000,
"batch_size_type": "frame",
"max_samples": 96,
"grad_accumulation_steps": 16,
"max_grad_norm": 0.3,
"epochs": 200,
"num_warmup_updates": 3000,
"save_per_updates": 10000,
"keep_last_n_checkpoints": -1,
"last_per_updates": 5000,
"finetune": true,
"file_checkpoint_train": "",
"tokenizer_type": "pinyin",
"tokenizer_file": "",
"mixed_precision": "bf16",
"logger": "tensorboard",
"bnb_optimizer": true
}

---

---

---

---

[AsmoKoskinen]

Done. Three days, 153 epochs and 195 000 steps. Dataset was 106 hours.

[AsmoKoskinen]

Claude says:

Summary of the Training Process

The training has progressed excellently and achieved its goals! Here's a summary of the final screenshots:

Training Status

Progress: 153/200 epochs (76.5% complete)
Total time: 3 days 0 hours 17 minutes
Step count: 195.1k
Current loss: ~0.627 (smoothed), individual values vary

Loss Curve Analysis

Loss has decreased steadily throughout the entire training from the initial ~0.67 to the current ~0.627
During the last day, the loss curve shows a particularly clear downward trend
The curve shows no signs of plateauing, suggesting the model could benefit from additional training

Individual Loss Values

The terminal view shows excellent individual values:

0.441, 0.466, 0.496, 0.497

These are significantly better than at the beginning of training

Conclusions

1. The training has been highly successful

2. The model's performance has improved consistently throughout the training

3. Reducing the learning rate was a good decision as it helped the model find lower loss values

4. The model might benefit from additional training as the loss curve continues to decline

I recommend testing different checkpoints (e.g., latest vs. epoch 100) to compare audio quality. This will show how much the audio quality has improved and which checkpoint produces the best results.

[AsmoKoskinen]

First two days:

{
"exp_name": "F5TTS_v1_Base",
"learning_rate": 0.0001,
"batch_size_per_gpu": 2000,
"batch_size_type": "frame",
"max_samples": 96,
"grad_accumulation_steps": 16,
"max_grad_norm": 0.3,
"epochs": 200,
"num_warmup_updates": 3000,
"save_per_updates": 10000,
"keep_last_n_checkpoints": -1,
"last_per_updates": 5000,
"finetune": true,
"file_checkpoint_train": "",
"tokenizer_type": "pinyin",
"tokenizer_file": "",
"mixed_precision": "bf16",
"logger": "tensorboard",
"bnb_optimizer": true
}

Last day:

{
"exp_name": "F5TTS_v1_Base",
"learning_rate": 0.00005,
"batch_size_per_gpu": 2000,
"batch_size_type": "frame",
"max_samples": 96,
"grad_accumulation_steps": 16,
"max_grad_norm": 0.3,
"epochs": 200,
"num_warmup_updates": 500,
"save_per_updates": 5000,
"keep_last_n_checkpoints": -1,
"last_per_updates": 5000,
"finetune": true,
"file_checkpoint_train": "",
"tokenizer_type": "pinyin",
"tokenizer_file": "",
"mixed_precision": "bf16",
"logger": "tensorboard",
"bnb_optimizer": true
}

[AsmoKoskinen]

I will upload files to the HF, and write a pull request.

[AsmoKoskinen]

I have added new files to the HF. But I don't think I'll make a new pull request. There is already a link to the Finnish model in HF.