Combine with dreambooth

[thanhhung0112]

I wonder whether this method can be used to improve the result of dreambooth finetuning? such as face fidelity or modify the plastic skin of personalized character in dreambooth result to make it more realistic something like that?

[yk7333]

It should be possible. By importing the model from the diffusion framework and generating a series of images, one can then use our loss function for fine-tuning after sorting or scoring them. This process could potentially generate images preferred by humans. However, my main research direction is in reinforcement learning, and my understanding of Dreambooth may not be extensive enough to guarantee a significant improvement in results. I suggest you start by annotating a small amount of data to observe the effects. Additionally, if there is a face fidelity calculation model available, you can use our provided code with a reward model for training and observe whether the metrics show significant improvement.