In python, both Numpy and Pytorch support broadcasting.
The principle of broadcasting: Two arrays are considered broadcast compatible if the axis lengths of the trailing dimension (that is, the dimension counting from the end) match, or if one of them has a length of 1. Broadcasting is performed on missing and/or length-1 dimensions.
It's a bit complicated, so here are a few examples:
If the two arrays a and b have the same shape, that is, a.shape == b.shape, then the result of a*b is the multiplication of the corresponding bits of a and b arrays. This requires the same number of dimensions and the same length for each dimension.
import numpy as np
a = np.array([1, 2, 3, 4])
b = np.array([10, 20, 30, 40])
c = a * b
print(c)
>>
[ 10 40 90 160]
The shape of arr1 is (4,3), and the shape of arr2 is (3,). The shape of arr2 is similar to (1, 3), so their trailing edge dimensions are equal. The length of the second dimension of arr1 is 3, which is the same as the dimension of arr2. The shapes of arr1 and arr2 are not the same, but they can perform addition operations, which is done by broadcasting, in this case expanding arr2 along the 0 axis.
import numpy as np
arr1 = np.array([[0, 0, 0],[1, 1, 1],[2, 2, 2], [3, 3, 3]])#(4,3)
arr2 = np.array([1, 2, 3]) #相当于(1,3)
arr_sum = arr1 + arr2
print(arr_sum)
>>
[[1 2 3]
[2 3 4]
[3 4 5]
[4 5 6]]
Legend:
Another set of legends:
Obviously, it is to align the two arrays according to the last dimension, and then determine whether the lengths are equal or at least one is 1, and if any dimension has no length, it is regarded as 1. From the back to the front, each dimension does this, and if there are exceptions mentioned above, it is judged that it cannot be broadcast.
For example:
(3,4,2)+(4,2), from back to front, 2 is equal to 2, 4 is equal to 4, and then the length that is not there is regarded as 1, 3 and 1 have at least one 1, so it can be broadcasted, (4,2) The way of broadcasting is 3*(4,2)
(4,3)+(4,1), from back to front, there is at least one 1 between 1 and 3, and 4 is equal to 4, so it can be broadcast. The broadcast method of (4,1) is (4,1)*3
(4,3)+(4,), (4,) can be regarded as (1, 4). From the back to the front, 4 is not equal to 3, so it cannot be broadcast, and an error will be reported after writing such code and executed.
Let's look at an example of pytorch:
import torch
x = torch.tensor([[[[1,2],[1,2],[1,2]]]])#1 1 3 2
y=torch.tensor([[-2],[4],[-8]])#3 1
print(x.shape)
print(y.shape)
print(x+y)
>>
torch.Size([1, 1, 3, 2])
torch.Size([3, 1])
tensor([[[[-1, 0],
[ 5, 6],
[-7, -6]]]])
(1,1,3,2)+(3,1), from back to front, 1 and 2 have at least one 1, 3 is equal to 3, then the length without is regarded as 1, 1 and 1 have at least one 1, and then there is no length as 1, 1 and 1 have at least one 1, so it can be broadcasted.
[If you have any questions, please speak up]