YOLO-NAS detailed tutorial on how to start on multiple GPUs (DDP)

training mode

SuperGradients allows users to train models in different modes: 1. CPU 2. Single GPU - (CUDA) 3. Multiple GPUs - Data Parallelism (DP) 4. Multiple GPUs - Distributed Data Parallelism (DDP)

1、CPU

Requirements : None.

How to use it : If you don't have any CUDA devices available, your training will automatically run on the CPU. Otherwise, the default device will be CUDA, but you can still setup_deviceeasily set it to CPU using:

from super_gradients import Trainer
from super_gradients.training.utils.distributed_training_utils import setup_device

setup_device(device='cpu')

# Unchanged
trainer = Trainer(...)
trainer.train(...)

2. CUDA

Requirement : At least one available CUDA device

How to use it : If you have at least one CUDA device, nothing! Otherwise, you will have to use the CPU...