Solve the problem of [TypeError: forward() got an unexpected keyword argument 'reduction']

First of all, the project background is to use BiLSTM+CRF to do NER tasks. When defining the loss function, an error was reported in the title. The following is the source code:

# model.py
class Model(nn.Module):
    def __init__(self):
        ......
        self.crf = CRF(TARGET_SIZE)

    def forward(self, input, mask):
        ......
        return self.crf.decode(out, mask)

    def loss_fn(self, input, target, mask):
        y_pred = self._get_lstm_feature(input)
        return -self.crf.forward(y_pred, target, mask, reduction='mean')

The focus is on loss_fn, so the initialization and feed-forward layers will not be shown for the time being. This way of writing will cause the error shown in the title when training the model, which means:

"TypeError: forward() got an unexpected keyword argument 'reduction'"

Indicates that the forward method of the CRF class does not accept a reduction parameter"

Try to remove the reduction parameter, train the model again, and report the following error again:

RuntimeError: grad can be implicitly created only for scalar outputs

"RuntimeError: grad can only be created implicitly for scalar output"

Meaning the loss function returns a tensor with multiple elements, which is not allowed when computing gradients. Therefore, the reduction parameter cannot be removed, and loss_fn should be kept to return a scalar value.

Finally, another way of writing is changed, which can run normally and model training:

    def loss_fn(self, input, target, mask):
        y_pred = self._get_lstm_feature(input)
        loss = -self.crf.forward(y_pred, target, mask)
        return loss.mean()

 First use a loss variable to receive the loss value, and then take mean() when returning

hope it is of help to you