Derivation of Normal Equation

Normal Equation Official:

$$\theta= (X^TX)^{-1}X^TY$$

---

The derivation process:

Start:

$$X·\theta=Y$$
Step 1: Multiply Left X ^T
$$X^TX·\theta=X^TY$$
Step 2: Left multiply (X ^T X) ^-1
$$(X^TX)^{-1}X^TX·\theta= (X^TX)^{-1}X^TY$$
among them:
$$(X^TX)^{-1}(X^TX)=I$$
and so:
$$\theta= (X^TX)^{-1}X^TY$$

Gradient Descent method:

• Need to choose a learning rate $\alpha$
• Need multiple iterations
• When the number of features is large, the efficiency is still high

Normal Equation method:

• No need to choose a learning rate $\alpha$
• No need to iterate
• Not suitable for situations with a large number of features, because matrix multiplication is required