Linear Algebra review - Matrix multiplication properties

摘要: 本文是吴恩达 (Andrew Ng)老师《机器学习》课程，第三章《线性代数回顾》中第18课时《矩阵乘法特征》的视频原文字幕。为本人在视频学习过程中记录下来并加以修正，使其更加简洁，方便阅读，以便日后查阅使用。现分享给大家。如有错误，欢迎大家批评指正，在此表示诚挚地感谢！同时希望对大家的学习能有所帮助。

Matrix multiplication is really useful since you can pack a lot of computation into just one matrix multiplication operation. But you should be careful of how you use them. In this video (article), I want to tell you about a few properties of matrix multiplication.

Matrix multiplication is not commutative

When working with just real numbers, or when working with scalars, multiplication is commutative. And what I mean by that is if you take $3\times 5$ , that is equal to $5\times 3$ , and the ordering of this multiplication doesn't matter. And this is called the commutative property of multiplication of real numbers. It turns out this property is not true for matrix multiplication. Concretely, if $A$ and $B$ are matrices, $A\times B$ is not equal to $B\times A$ . Just be careful of that. It is not okay to arbitrarily reverse the order in which you were multiplying matrices. So, we say the matrix multiplication is not commutative, it's a fancy way of saying it.

As a concrete example, $\begin{bmatrix} 1 & 1\\ 0 & 0 \end{bmatrix} \times \begin{bmatrix} 0 & 0\\ 2 & 0 \end{bmatrix}=\begin{bmatrix} 2 & 0\\ 0 & 0 \end{bmatrix}$ . Now, let's swap around the order of these two matrices. It turns out if you multiply these two matrices, you get $\begin{bmatrix} 0 & 0\\ 2 & 0 \end{bmatrix} \times \begin{bmatrix} 1 & 1\\ 0 & 0 \end{bmatrix} =\begin{bmatrix} 0 & 0\\ 2 & 2 \end{bmatrix}$ . Clearly, these two results are not equal to each other.

So, in fact, in general, if you have a matrix operation, like $A\times B$ . If $A$ is $m\times n$ matrix, and $B$ is $n\times m$ matrix. Then it turns out that $A\times B$ is going to be an $m\times m$ matrix, whereas $B\times A$ is going to be $n\times n$ matrix, so the dimension even doesn't match.

Matrix multiplication is associative

When talking about real numbers, or scalars, let's say, I have $3\times 5\times 2$ . I can either compute this as $3\times (5\times 2)=30$ . Or I can compute $(3\times 5)\times 2=30$ . And both of these give you the same answer. This is called the associative property of real number multiplication. It turns out that matrix multiplication is associative. So concretely, let's say I have a product of three matrices, $A\times B\times C$ . Then I can compute this either as $A\times (B\times C)$ , or I can compute this as $(A \times B)\times C$ , and these will actually give me the same answer.

So just be clear what I mean by these two cases. Let's look at the first case. What I mean by that is if you actually want to compute $A\times B\times C$ , you can first compute $B\times C$ . So that $D=B\times C$ , then compute $A\times D$ . Or for this second case, you can set $E=A\times B$ , then compute $E\times C$ . And it turns out that both of these two options will give you, is guaranteed to give you the same answer. And so, we say that matrix multiplication is does enjoy the associative property.

Identity matrix

Finally, I want to tell you about the identity matrix which is special matrix. When dealing with real numbers or scalar numbers, the number $1$ , is you can think of it as the identity of multiplication, and what I mean by that is that for any number $z$ , $1\times z=z\times 1=z$ . So $1$ is the identity operation and so it satisfies this equation. So, it turns out that in the space of matrices, there's an identity matrix as well. And it's usually denoted $I$ , or sometimes we write it as $I_{n\times n}$ . We want to make explicit the dimensions.

And so, there's a different identity matrix for each dimension $n$ , and here are a few examples. Here's the $2\times 2$ identity matrix. Here's the $3\times 3$ identity matrix. Here's the $4\times 4$ identity matrix. So, the identity matrix has the property that it has ones along the diagonals, and is zero everywhere else. And by the way, the $1\times 1$ identity matrix is just a number one ( $[1]$ ).

And informally when I or others are being sloppy very often, we'll write the identity matrix using fine notation. Just write 1111, dot, dot, dot, 1 and then we'll, maybe, somewhat sloppily write a bunch of zeros there.

It turns out that identity matrix has this property that for any matrix $A$ , $A.I=I.A=A$ . So that's a lot like this equation ( $1\times z=z\times 1=z$ ) that we have up here. Just make sure we have the dimensions right. So, for $A.I$ , if $A$ is $m\times n$ matrix, then this identity matrix is an $n\times n$ identity matrix. And for $I.A$ , if $A$ is $m\times n$ , this identity matrix is an $m\times m$ matrix. In either case, the outcome of this process is you get back to matrix $A$ which is $m\times n$ . So, whenever you write the identity matrix $I$ , the dimension will be implicit in the context. So these two $Is$ , they're actually different dimension matrices, one is $n\times n$ , the other is $m\times m$ . But when we want to make the dimension of the matrix explicit, then sometimes we'll write to this $I_{n\times n}$ . But very often the dimension will be implicit.

Finally, just want to point out that, earlier I said that, in general, $A. B\neq B. A$ , right? That for most matrices $A$ and $B$ , this is not true. But when $B$ is the identity matrix, this does hold true. That $A. I=I. A$ . This is not true for other matrices $B$ in general.

So that's it for the properties of matrix multiplication. And the special matrices, like the identity matrix. In the next and final video (article) in our linear algebra review, I'm going to quickly tell you about a couple of special matrix operations, and after that you know everything you need to know about linear algebra for this course.

<end>

王彩旗

发布了41 篇原创文章 · 获赞 12 · 访问量 1306

私信关注