Let's briefly talk about Hadamard product:
(refer to Wikipedia: https://en.wikipedia.org/wiki/Hadamard_product_(matrices) )
and matrix product is:
(refer to Wikipedia: matrix product )
The two are completely different , but why python is represented by ∗ ∗? By consulting the official documentation ( PEP465 ), we can get the following explanation:
For numpy.ndarray objects, * performs elementwise multiplication, and matrix multiplication must use a function call (numpy.dot). For numpy.matrix objects, * performs matrix multiplication, and elementwise multiplication requires function syntax.
That is, when the variable type is numpy.ndarray, ∗ represents the Hadamard product; when the variable type is numpy.matrix, ∗ represents the matrix product. The variable type in the LSTM source code is numpy.ndarray, so use ∗The operation naturally represents the Hadamard product, and the problem is solved.
Here's a simple example to distinguish between these two operations:
## Hadamard product import numpy as np a = np.array([[1,2],[3,4]]) b = np.array([[5,6],[7,8]]) a * b Out[5]: array([[ 5, 12], [21, 32]]) ## matrix product c = np.matrix(a) d = np.matrix(b) c * d Out[8]: matrix([[19, 22], [43, 50]])
It can be seen that the results obtained by the same * operation are different because the variable types are different.
Of course, a variable whose variable type is numpy.array can also perform matrix multiplication through the numpy.dot() function:
np.dot(a,b) Out[9]: array([[19, 22], [43, 50]])
参考:https://blog.csdn.net/flying_sfeng/article/details/79330336