标签平滑和样本类别不均衡处理方式

1. 论文

1.1 Label Smoothing

1. https://arxiv.org/abs/1906.02629 When Does Label Smoothing Help?
2. https://arxiv.org/abs/1701.06548 Regularizing Neural Networks by Penalizing Confident Output Distributions
3. https://arxiv.org/abs/1706.04599 On Calibration of Modern Neural Networks

1.2 Focal Loss

https://arxiv.org/abs/1708.02002 Focal Loss for Dense Object Detection
标签平滑: 提高模型的泛化能力，对于未知域任务，分类任务，可以提高精度。

2. 代码实现

2.1 Label Smoothing loss function

class LabelSmoothingCrossEntropy(nn.Module):
    def __init__(self, eps=0.1, reduction='mean'):
        super(LabelSmoothingCrossEntropy, self).__init__()
        self.eps = eps
        self.reduction = reduction

    def forward(self, output, target):
        c = output.size()[-1]
        log_preds = F.log_softmax(output, dim=-1)
        if self.reduction=='sum':
            loss = -log_preds.sum()
        else:
            loss = -log_preds.sum(dim=-1)
            if self.reduction=='mean':
                loss = loss.mean()
        return loss*self.eps/c + (1-self.eps) * F.nll_loss(log_preds, target, reduction=self.reduction)

2.2 labels mooth+类别不均衡loss