Deep Learning (23): Using one tensor/numpy to initialize another tensor/numpy using assignment has a problem

Article directory

1. Consequences of using one `tensor`to initialize `tensor`another

1 Introduction

When debugging a program recently, I found that for convenience, I used direct assignment to obtain a tensor with the same shape as the previously defined tensor, which is not advisable.

2 Initialize another tensor with a tensor assignment – something goes wrong

Next, use tensor a to assign tensor b to initialize tensor b

import torch
a=torch.ones((3,4))
print("a:",a)
print("================")
b=a
print("a:",a)
print("b:",b)
print("================")
b[1][2]=5
print("a:",a)
print("b:",b)
print("================")

Run pythonthe script and the output is as follows. It can be clearly seen that modifying tensor b causes tensor a to also be modified

a: tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
================
a: tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
b: tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
================
a: tensor([[1., 1., 1., 1.],
        [1., 1., 5., 1.],
        [1., 1., 1., 1.]])
b: tensor([[1., 1., 1., 1.],
        [1., 1., 5., 1.],
        [1., 1., 1., 1.]])
================

3 tensor's memory sharing mechanism

In order to achieve efficient calculations, PyTorch provides some in-place operations, that is, in-place operations, which perform calculations directly on the original memory without copying. For memory sharing, there are mainly the following three situations

Initialize Tensor through Tensor: Initialize another Tensor directly through Tensor, or initialize another Tensor through combination, block, index, and deformation operations of Tensor, then the two Tensors share memory.
In-place operator: PyTorch implements in-place operations by adding a suffix "_" to some operations, such as add_() and resize_(). As long as this operation is executed, the Tensor itself will be changed.
Tensor and NumPy conversion: Tensor and NumPy can be converted efficiently, and variables before and after conversion share memory. When performing operations that PyTorch does not support, you can even save the country with curves, convert Tensor to NumPy type, and then convert to Tensor after the operation
Reference: PyTorch Basics: Tensor Memory Sharing

4 Define the same tensor as the previous shape tensor

import torch
a=torch.ones((3,4))
print("a:",a)
print("================")
"""  错误示范
b=a
print("a:",a)
print("b:",b)
print("================")
# 对 b 进行操作 
b[1][2]=5
print("a:",a)
print("b:",b)
print("================")
"""
b=torch.ones(a.shape)  # 或  b=torch.zeros(a.shape)
print("a:",a)
print("b:",b)
print("================")
# 对 b 进行操作
b[1][2]=5
print("a:",a)
print("b:",b)
print("================")

Run the output. In this case, changing b does not affect a

a: tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
================
a: tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
b: tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
================
a: tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
b: tensor([[1., 1., 1., 1.],
        [1., 1., 5., 1.],
        [1., 1., 1., 1.]])
================

5. Use torch.clone()

You can use the torch.clone() function to create a new tensor with the same shape and data as the original tensor, and then operate on the new tensor so that the original tensor will not be affected.

import torch

# 原有 tensor
x = torch.tensor([[1, 2], [3, 4]])
print('x before update: ', x)

# 创建新 tensor，并对其进行操作
y = x.clone()
y[0, 0] = 5
print('y: ', y)

# 原有 tensor 不受影响
print('x after update: ', x)

The output is:

x before update:  tensor([[1, 2],
        [3, 4]])
y:  tensor([[5, 2],
        [3, 4]])
x after update:  tensor([[1, 2],
        [3, 4]])

2. Consequences of using one `numpy`to initialize `numpy`another

1. Both numpy and tensor also have a memory sharing mechanism

>>> import numpy as np
>>> a=np.ones((3,3))
>>> a
array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]])
>>> b=a
>>> b[1][1]=3
>>> b
array([[1., 1., 1.],
       [1., 3., 1.],
       [1., 1., 1.]])
>>> a
array([[1., 1., 1.],
       [1., 3., 1.],
       [1., 1., 1.]])

2. Use numpy.copy()

To copy a NumPy array without memory sharing, use the copy() method. This will return a new array whose data does not share memory with the original array.
Examples are as follows:

import numpy as np

# 创建一个 NumPy 数组
a = np.array([1, 2, 3, 4, 5])

# 复制该数组并返回新的数组
b = a.copy()

# 修改新的数组
b[0] = 10

# 查看两个数组的值
print(a)  # 输出 [1 2 3 4 5]
print(b)  # 输出 [10  2  3  4  5]

In the above example, we first created a NumPy array a with 5 elements and then copied it into a new array b using the copy() method. We then modified the first element of the new array b and looked at the values of both arrays. Since a and b do not share memory, modifying b does not affect a.