nan stepped pit remember

1, inspection data
if the input data is a picture, first open the check image / picture clearly abnormal size;
if the input model data check function used to provide the required loss interface consistent.

2. Check all except where the denominator
especially in the normalization functions own implementation, in particular, need attention.

3, check whether there were log_softmax.

4, can be used in the following code PyTorch to check whether nan occurred, and erroneous input position location may occur
with torch.autograd.detect_anomaly ():
loss.backward ()