CUDA Error: out of memory

[jessiffmm]

Hi!!

I have a question. I'm doing a project where I can use keras, tensorflow and darknet like in dl-detectionSuite. But I have a problem. When I use first keras or tensorflow, and then I use darknet. I get CUDA Error: out of memory.

I have trying to use clear_session for free memory but I get the same error.
Do you have some idea about this problem?

Thanks!

[vinay0410]

Hi @jessiffmm ,
As far as I know, this error occurs when your GPU runs out of Video Memory or VRAM. I have faced this error before, and I reduced my batch size to solve it, so that training takes less memory at a time.

Also, have look here, clear_session() cannot forcefully clean GPU's memory since it can cause fragmentation, try killing the process.

See if quitting and restarting DetectionSuite helps, if it helps than we can probably alter the code so that this error doesn't occur again.

[jessiffmm]

Hi @vinay0410

I get this problem in my project. I have a trained network in keras, tensorflow and Darknet. I load the network in keras and I used it. Then I try to load the Darknet network but I get that problem.

[vinay0410]

HI @jessiffmm ,
Sorry for late reply, could you just verify if restarting detectionsuite fixes this problem.
So, after you have run your keras network, restart it and then run your darknet network.
This would better help in debugging. If restarting in between helps, then probably we can fix it in code also.