Tensor operation
The main operations of Tensor are usually divided into four categories:
- Reshaping operations
- Element-wise operations
- Reduction operations
- Access operations
element manipulation
The position of an element in the tensor is determined by the index that locates each element, and the element operation is an operation on the tensor elements. These tensor elements correspond to or have the same index position in the tensor. What needs to be emphasized here is that the two tensors Quantities must have the same shape to perform an element-wise operation, and having the same shape means that the tensors have the same number of corresponding axes, and they have the same length, which ensures that the required correspondence for element-wise operations is possible!
arithmetic operation
Here is an example to illustrate the definition of the above element operation:
t1 = torch.tensor([
[1, 2],
[3, 4]
], dtype=torch.float32)
t2 = torch.tensor([
[5, 6],
[7, 8]
], dtype=torch.float32)
print(t1 + t2)
Show results:
tensor([[ 6., 8.],
[10., 12.]])
We can see that each pair of elements at corresponding positions are added together, resulting in a new tensor with the same shape, but only if the two tensors have the same shape! In fact, this is true for all element-wise operations (subtraction, multiplication, division).
However, operations that we commonly see with tensors are operations that use scalar values, as shown in the following diagram:
Seeing this picture, you may have such a question: all scalar values are zero-order tensors, so this situation does not satisfy the so-called premise that two tensors have the same shape to perform element operations?
At this time we need to introduce the concept of tensor broadcasting!
tensor broadcast
The broadcast mechanism defines how to handle tensors of different shapes during the element operation process. In fact, in the above-mentioned t1 + 2 operation, the scalar tensor is expanded to the shape of t1 before the operation, and then performed Addition operation, which essentially satisfies the premise of the element operation we formulated!
Next, we study the broadcast form on scalar values by using the numpy broadcast function:
np.broadcast_to(2, t1.shape)
Show results:
array([[2, 2],
[2, 2]])
t1 + torch.tensor(
np.broadcast_to(2, t1.shape)
)
Show results:
tensor([[3., 4.],
[5., 6.]])
It can be seen that the scalar value at this time becomes a second-order tensor with the same shape as t1, and then the addition operation is performed.
Therefore, the element operations of scalar-valued tensors that we usually see are actually in the underlying implementation, first use the broadcast mechanism to copy one or more axes of the tensor, convert it into a tensor of the same shape, and then perform The corresponding element operation!
Next, a more complex example will be shown to deepen the understanding:
t1 = torch.tensor([
[1, 1],
[1, 1]
], dtype=torch.float32)
t2 = torch.tensor([2, 4], dtype=torch.float32)
What would the result look like if we added the two tensors above?
We can also use numpy's broadcast function to get the result:
np.broadcast_to(t2.numpy(), t1.shape)
Show results:
array([[2., 4.],
[2., 4.]], dtype=float32)
So the result of adding two tensors should look like this:
tensor([[3., 5.],
[3., 5.]])
comparison operation
After understanding arithmetic operations, we can then do the same for comparison operations. Because the object of the comparison operation is also an element in the tensor, the result returned by the comparison operation of two tensors is a new tensor of the same shape containing 0 or 1 (false or true) (0 means false, 1 means true )
# 以张量t为例
t = torch.tensor([
[0, 1, 2],
[3, -1, 5],
[6, 7, -1]
], dtype=torch.float32)
Element-wise operations of functions
For the operation of elements, we can also use functions. For functions, just assume that this function is applied to each element of the tensor, such as the following example:
