This blog post outlines techniques for improving the training performance of your PyTorch model without compromising its accuracy. To do so, we will wrap a PyTorch model in a LightningModule and use the Trainer class to enable various training optimizations. By changing only a few lines of code, we can reduce the training time on a single GPU from 22.53 minutes to 2.75 minutes while maintaining the model’s prediction accuracy.
Yes, that’s a 8x performance boost!
(This blog post was updated on 03/17/2023, now using PyTorch 2.0 and Lightning 2.0!)
Some Techniques To Make Your PyTorch Models Train (Much) Faster (sebastianraschka.com)