Preface
Workers at the bottom, adjust bugs online
About the model in pytorch
model.train()
Symbol description:
- Encoder means embedding model (perhaps bert or glove...)
- model represents the main model, and the output of the encoder is used as the model input
Problem description:
Case 1: Encoder and model are separate models. At this time, when you train the model, you must add model.train(); Case 2: If the encoder is defined in the model, that is, model. init () In, you wrote self.encoder=encoder, and now you add model.train().
In these two cases, the calculation of the model part is the same, but the encoder is different. One is in eval() mode and the other is in train() mode, which will lead to different results.
seed
Symbol description:
- model represents the model you defined
- a represents the first linear layer
- b represents the second linear layer
Problem description:
Code 1: Only the linear layer b is defined in the model; Code 2: For fun, the two linear layers a and b are defined in the model at the same time, but the actual use is still only the linear layer b.
In the two pieces of code, even if your seed is fixed, the parameter initialization of the linear layer of your b is different.