Based on the loss function PyTorch

Foreword

This blog provides a reference for some common loss function, you can easily import into the code.
Loss function defines how the neural network model calculated from the residual overall error per round, which in turn affects the adjustment coefficient in performing the back-propagation mode thereof, so select function directly affects the performance loss model.
Segmentation and classification for other tasks, the default choice of loss function is a binary cross-entropy (BCE). When a particular metric, such as dice or IOU coefficients, is used to model performance is determined, competitors sometimes derived from these test measures the loss function - usually in the form. 1 - f (x), where f (x) is problematic measure. These functions can not simply write NumPy, as they are implemented in the GPU, the rear end of the function is required from the respective model library, which also comprises a gradient for the back-propagation algorithm. This did not imagine complex.
In many types of segmentation, each class to calculate the average loss is calculated from the entire loss prediction tensor generally used loss function instead. This blog will serve as a basic reference template code, but for many types of average and modify it should be very simple. For example, if the tensor comprises a continuous flattened classes, you can divide them into four classes of equal length, the loss of their respective calculated and averaged.
Hopefully this blog for your help, you are welcome any proposed changes.

dice loss

Dice coefficient or Sørensen-Dice coefficient, is a common standard binary classification task, such as pixel division, it may also be modified as a loss function:

#PyTorch
class DiceLoss(nn.Module):
    def __init__(self, weight=None, size_average=True):
        super(DiceLoss, self).__init__()

    def forward(self, inputs, targets, smooth=1):
        
        #comment out if your model contains a sigmoid or equivalent activation layer
        inputs = F.sigmoid(inputs)       
        
        #flatten label and prediction tensors
        inputs = inputs.view(-1)
        targets = targets.view(-1)
        
        intersection = (inputs * targets).sum()                            
        dice = (2.*intersection + smooth)/(inputs.sum() + targets.sum() + smooth)  
        
        return 1 - dice

BCE-Dice Loss

This loss of binding and loss dice binary cross entropy criteria (BCE) loss, which is usually the default segmentation model. Combine these two methods can reduce the loss to a certain extent, while benefiting from the stability of BCE. Any person who learned logistic regression equations are familiar with many types of BCE:

#PyTorch
class DiceBCELoss(nn.Module):
    def __init__(self, weight=None, size_average=True):
        super(DiceBCELoss, self).__init__()

    def forward(self, inputs, targets, smooth=1):
        
        #comment out if your model contains a sigmoid or equivalent activation layer
        inputs = F.sigmoid(inputs)       
        
        #flatten label and prediction tensors
        inputs = inputs.view(-1)
        targets = targets.view(-1)
        
        intersection = (inputs * targets).sum()                            
        dice_loss = 1 - (2.*intersection + smooth)/(inputs.sum() + targets.sum() + smooth)  
        BCE = F.binary_cross_entropy(inputs, targets, reduction='mean')
        Dice_BCE = BCE + dice_loss
        
        return Dice_BCE

Jaccard/Intersection over Union (IoU) Loss

IoU indicators, or Jaccard index, similar to dice index, calculated as positive examples of overlap between the two sets of the ratio between the value thereof in combination with each other:

the same as dice metric, it is also a common method of pixel division model performance evaluation.

#PyTorch
class IoULoss(nn.Module):
    def __init__(self, weight=None, size_average=True):
        super(IoULoss, self).__init__()

    def forward(self, inputs, targets, smooth=1):
        
        #comment out if your model contains a sigmoid or equivalent activation layer
        inputs = F.sigmoid(inputs)       
        
        #flatten label and prediction tensors
        inputs = inputs.view(-1)
        targets = targets.view(-1)
        
        #intersection is equivalent to True Positive count
        #union is the mutually inclusive area of all labels & predictions 
        intersection = (inputs * targets).sum()
        total = (inputs + targets).sum()
        union = total - intersection 
        
        IoU = (intersection + smooth)/(union + smooth)
                
        return 1 - IoU

Focal Loss

Artificial Intelligence Research Facebook Lin, who in 2017 introduced a focal loss, as a hedge against extreme imbalance data sets, relatively few of these data set positive examples. Their paper "Focal Loss for Dense Object Detection" can be found here: https://arxiv.org/abs/1708.02002. In practice, the researchers used a modified version of the alpha function, so I'll include it in the implementation.

#PyTorch
ALPHA = 0.8
GAMMA = 2

class FocalLoss(nn.Module):
    def __init__(self, weight=None, size_average=True):
        super(FocalLoss, self).__init__()

    def forward(self, inputs, targets, alpha=ALPHA, gamma=GAMMA, smooth=1):
        
        #comment out if your model contains a sigmoid or equivalent activation layer
        inputs = F.sigmoid(inputs)       
        
        #flatten label and prediction tensors
        inputs = inputs.view(-1)
        targets = targets.view(-1)
        
        #first compute binary cross-entropy 
        BCE = F.binary_cross_entropy(inputs, targets, reduction='mean')
        BCE_EXP = torch.exp(-BCE)
        focal_loss = alpha * (1-BCE_EXP)**gamma * BCE
                       
        return focal_loss