Reference article to
solve RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED
The pytorch torch.backends.cudnn setting works.
At first, I thought it was a problem of too much data, but I also reported an error when running with 4 graphics cards. I looked at the issue and said that the batch size was reduced, but it was useless for me. The reference article, added the following code, found useful.
Add the following code
torch.backends.cudnn.enabled = False