pytorch dropout用法

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/jacke121/article/details/85928924

链接：https://www.zhihu.com/question/67209417/answer/302434279
 

刚踩的坑, 差点就哭出来了TT. --- 我明明加了一百个dropout, 为什么结果一点都没变

使用F.dropout ( nn.functional.dropout )的时候需要设置它的training这个状态参数与模型整体的一致.

比如:

1. Class DropoutFC(nn.Module):

2. def __init__(self):

3. super(DropoutFC, self).__init__()

4. self.fc = nn.Linear(100,20)

5.  

6. def forward(self, input):

7. out = self.fc(input)

8. out = F.dropout(out, p=0.5)

9. return out

10.  

11. Net = DropoutFC()

12. Net.train()

13.  

14. # train the Net

这段代码中的F.dropout实际上是没有任何用的, 因为它的training状态一直是默认值False. 由于F.dropout只是相当于引用的一个外部函数, 模型整体的training状态变化也不会引起F.dropout这个函数的training状态发生变化. 所以, 此处的out = F.dropout(out) 就是 out = out. Ref: https://github.com/pytorch/pytorch/blob/master/torch/nn/functional.py#L535

正确的使用方法如下, 将模型整体的training状态参数传入dropout函数

1. Class DropoutFC(nn.Module):

2. def __init__(self):

3. super(DropoutFC, self).__init__()

4. self.fc = nn.Linear(100,20)

5.  

6. def forward(self, input):

7. out = self.fc(input)

8. out = F.dropout(out, p=0.5, training=self.training)

9. return out

10.  

11. Net = DropoutFC()

12. Net.train()

13.  

14. # train the Net

或者直接使用nn.Dropout() (nn.Dropout()实际上是对F.dropout的一个包装, 也将self.training传入了) Ref: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/dropout.py#L46

1. Class DropoutFC(nn.Module):

2. def __init__(self):

3. super(DropoutFC, self).__init__()

4. self.fc = nn.Linear(100,20)

5. self.dropout = nn.Dropout(p=0.5)

6.  

7. def forward(self, input):

8. out = self.fc(input)

9. out = self.dropout(out)

10. return out

11. Net = DropoutFC()

12. Net.train()

13.