torch.tensor和torch.nn.Parameter

reference:

• In-depth understanding of Pytorch torch.nn.Parameter()https://blog.csdn.net/qq_43391414/article/details/120484239
• After pytorch Variable and Tensor are merged, requires_grad() default and modificationhttps://blog.csdn.net/weixin_43635550/article/details/100192797

---

torch.tensor

Use torch.tensorfor creating tensors.

torch.nn.Parameter

Using is torch.nn.Parameterused to create parameter matrices/vectors, which torch.tensoris quite different from using directly to create parameter matrices. The main differences are as follows:

1. Gradient tracking and updating: When parameters are created using the nn.Parameter() function, PyTorch automatically adds these parameters to the model's parameter list and tracks and updates their gradients during training. This means that parameters created via nn.Parameter() can be automatically optimized by gradient descent. Tensors created directly using torch.randn() will not be automatically added to the parameter list, nor will their gradients be tracked and updated automatically.
2. Access to model parameters: The parameters created by nn.Parameter() can be accessed by the model.parameters() method. This is very convenient for operations such as initialization, saving and loading of model parameters. The tensors created directly using torch.randn() will not be included in model.parameters() and need to be handled manually.

An example of not automatically updating in 1 is as follows: using torch.tensor or similar torch.zeros, torch.normal, torch.randn functions are all created as a tensor, and the tensor itself does not have a gradient and Do not track gradient changes, i.e. required_grad==falsedefault to false.

a = torch.tensor([1,2])	# just like :: torch.tensor([1,2],dtype=torch.int64,requires_grad=False)
w = torch.tensor([1,2],dtype=torch.float32,requires_grad=True)

For the access to the model parameters in 2, that is, the direct use of similar methods model.parameter()cannot access the parameters and similar methods can only access subclasses inherited from nn, examples are as follows:

import torch
import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.W1 = nn.Parameter(torch.randn(3, 4))  # 使用 nn.Parameter() 创建可训练参数
        self.W2 = torch.randn(3, 4)  # 直接使用 torch.randn() 创建张量

model = MyModel()

# 访问模型的参数列表
parameters = list(model.parameters())
print(len(parameters))  # 输出：1

# 访问模型的成员变量
variables = list(model.__dict__.keys())
print(len(variables))  # 输出：2

In the above example, the parameter self.W1 created using nn.Parameter() is automatically added to the model's parameter list, while the tensor self.W2 created directly using torch.randn() is not added to the parameter list . Therefore, accessing the parameter list of the model through model.parameters() can only get the parameter self.W1.

To summarize, use the nn.Parameter() function to mark tensors as trainable parameters of a model, have them automatically gradient tracked and updated, and provide easy access to the model's parameter list. The tensor created directly using torch.randn() does not have these functions, and the gradient and access method of parameters need to be manually handled.