Questions about GPU and VRAM usage for Baseline finetune.

[piqiuni]

In the Table in Finetune, it says when Batch size = 4, the required VRAM is 34GB.
We are using RTX4090*2 for finetune, but we can only run with Batch size = 1, and the VRAM usage is about 20GB per card. Why it take so much VRAM?

Do you have any suggestions on the GPU usage? Should we try to get more GPUs for training? Maybe give me some detailed suggestions on GPU types and numbers?

Thanks for your suggestions! Forgive me for being a beginner~

[ChonghaoSima]

moving to issue section for better visibility to the general audience