pytorch mine collection

Error 1: Multiple Loss

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
原因：一次feed，多次backward()了。

Method:
Add the weighted sum of each LOSS, and then backward(). The
original problematic code:

           final_loss = (1-alpha ) * loss_func(predict_label,labels) 
           single_loss = alpha * loss_func(single_label,labels)
           optimizer.zero_grad()
           final_loss.backward()
           single_loss.backward()
           optimizer.step()

Correct code:

            final_loss = (1-alpha ) * loss_func(predict_label,labels) + alpha * loss_func(single_label,labels)
            optimizer.zero_grad()
            final_loss.backward()
            optimizer.step()

错误2：Expected object of scalar type Long but got scalar type Float for argument

The label of cross entropy must be converted to Long type.
Solution:

labels.to(dtype=tensor.long)

错误3：RuntimeError: 1D target tensor expected, multi-target not supporte

Solution: The shape of the cross-entropy Label must be(样本个数,)

Mistake 4: For the same model as keras, the accuracy of pytorch's model will be many percentage points lower than that of keras

First check whether the two models implemented in Keras and pytorch are consistent, such as whether the number of parameters in each layer is consistent, and whether the default initialization methods and activation functions are consistent.
Secondly, check whether the pytorch model is in the training process and whether the data set has been randomized to disrupt the order. The model implemented by Keras will shuffle the training data and validation data by default when it is fit. If the training data input to pytorch is not shuffled, then the model may only remember the characteristics of the data of the last batch, which will seriously bias the generalization of the model.