Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About model training #117

Open
xinyuz8270 opened this issue Oct 31, 2023 · 4 comments
Open

About model training #117

xinyuz8270 opened this issue Oct 31, 2023 · 4 comments

Comments

@xinyuz8270
Copy link

Do I need to take any other steps during the training process? After training for a few days, the loss did not continue to decrease and the sampling results obtained were also very blurry. Do anyone have any training suggestions?
Here are some of my sampling results. I trained it for 250K iterations.
real_data

@cuonghoang29
Copy link

Hi there. Did you solve your problem? I had the same issue. How many images do u have for the training?

@1zeryu
Copy link

1zeryu commented Mar 10, 2024

I had the same issue. How many images do u have for the training?

@cuonghoang29
Copy link

cuonghoang29 commented Mar 13, 2024

My dataset is only 1000 images, but after checking the code, I found that we should use random crop instead of using center crop (because only 1k images). In my case, If I train from scratch (4 GPUs A100), after 100k steps the generated images will be clearer. If I use a pre-trained model to train, after about 50k steps the generated would be better.

@xinyuz8270
Copy link
Author

Hi there. Did you solve your problem? I had the same issue. How many images do u have for the training?

Sorry for not replying in time. The ImageNet dataset I used for training has about 1.2 million images. After checking my code, I found that my problem was caused by a coding error in the sampling process. So I'm sorry I can't answer your question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants