[Runtimeerror] Solve the tensor dimension mismatch error message

Problem Description

When doing undergraduate graduation design, reproduce the code:

# ...
# We split the whole train dataset into 100 segments.
        for i in range(20):
            t1 = time.time()
            total_loss = 0
            train_dataset = QD.QDloadStrokeData(no=i,val = False,transforms = trans)          
            train_loader = DataLoader(dataset=train_dataset, batch_size=256,shuffle=False)
            for t, (x,stroke, y) in enumerate(train_loader):
                model.train()  
                x = x.to(device=device, dtype=dtype)  
                y = y.to(device=device, dtype=torch.long)

                #add the center feature returned from resnet 
                scores,cf_pred = model(x)
                
                #Caculate entropy loss
                entropy_loss = F.cross_entropy(scores, y)
                
                #Caculate the center loss 
                center_loss = F.mse_loss(cf_pred,cf_class[y])
                
                loss = entropy_loss + alpha * center_loss
                
                total_loss += loss
# ...

Report the following error:

RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_14660/27735771.py in <module>
----> 1 train(cnn_model,optimizer, epochs=5,args=args)

/tmp/ipykernel_14660/4015440672.py in train(model, optimizer, epochs, args)
    100 
    101                 #Caculate the center loss
--> 102                 center_loss = F.mse_loss(cf_pred,cf_class[y])
    103                 print(center_loss.shape)
    104 
RuntimeError: The size of tensor a (2048) must match the size of tensor b (2088) at non-singleton dimension 1

Error message: Tensor a (2048) and tensor b (2088) must match in the dimension with index 1.

solution

This alarm message does not say which two tensor dimensions do not match, but it prompts an error when calculating mse_loss.

So there is nothing to say, the best way is to output all the dimensions of all the tensors in the middle to see:

Start another piece of code, input a random tensor with the same dimensions as the original input for error checking:

import torch
import torch.nn.functional as F
import numpy as np

cf_class = torch.from_numpy(np.load("center_feature_ssn.npy"))
scores = torch.randn(256,40)
scores = torch.tensor(scores, dtype=torch.float)
cf_pred = torch.randn(256,2048)
print('scores:',scores.shape)
y = torch.zeros(256)
y = y.long()
print('y:',y.shape)
entropy_loss = F.cross_entropy(scores, y)
print('entropy_loss:',entropy_loss.shape)
print('cf_pred:',cf_pred.shape)
print('cf_class[y]:',cf_class[y].shape)
center_loss = F.mse_loss(cf_pred,cf_class[y])

output:

scores: torch.Size([256, 40])
y: torch.Size([256])
entropy_loss: torch.Size([])
cf_pred: torch.Size([256, 2048])
cf_class[y]: torch.Size([256, 2088])

RuntimeError: The size of tensor a (2048) must match the size of tensor b (2088) at non-singleton dimension 1

So far we have successfully reproduced the error (doge). That is, dimension 1 of cf_pred and cf_class[y] does not match, which is the same as the number in the error message, one is 2048 and the other is 2088.

Find out what went wrong, and then find out how the dimensions of these two variables change, and then keep them consistent.

Attached:

I also stepped on a pit when executing cross_entropy():

RuntimeError: expected scalar type Long but found Float

This is because the cross-entropy loss function requires that the target type should be long type, and the input type does not require it. So add a conversion type statement y = y.long() after y.