[Numerical calculation methods (Huang Mingyou)] Calculation of matrix eigenvalues and eigenvectors (5): Householder method [theory to procedure]

  eigenvalues ​​of the matrix (eigenvalue) and eigenvectors (eigenvector) has important mathematical and physical significance in many applications. Householder matrices and transformations provide an efficient way to map a vector to a standard direction through reflection transformation, which is of great significance for some numerical computing problems.

  This article will introduce the basic principles and steps of the Householder method in detail, and give its Python implementation.

1. Jacobi rotation method

  In each iteration of the Jacobi rotation method, you need to select a position where the non-diagonal element is the largest, and then construct the corresponding rotation matrix, Perform a similarity transformation to gradually diagonalize the matrix.

2. Jacobi clearance method

  Jacobi’s threshold method is an improved version of Jacobi rotation method. Its main purpose is to reduce calculation work and increase running speed. This method dynamically adjusts the threshold and selectively rotates the non-diagonal elements according to the threshold to gradually diagonalize the symmetric matrix.

3. Householder method

  If for any vector $z$, we can decompose it into $u$ parallel quantity $au$ Wayo $u$ orthogonal quantity $bv$，即 $With=au+bv$，nan么 Householder 变换会将 $z$ 变换为 $-au+bv$. This transformation can be understood as specular reflection, which does not change the vector between $u$ Projection on orthogonal planes, but with vectors along $Reflection\; in\; the\; direction\; of\; u$. The mathematical expression is:

$$Hz=au+bv\to -au+bv$$

  This property makes the Householder transform very useful in some numerical calculation applications, such as QR decomposition.

1. Rotation transformation

  In the Householder method, through a series of orthogonal similarity transformations, the real symmetric matrix (A) can be transformed intotridiagonal matrix$a_{ij}=a_{ji}=0$), the rotation matrix of the Householder method selects an angle such that$c_{ik}=c_{kj}=0$。

a. Selection of rotation transformation

  For a real symmetric matrix A A $a_{ik}$ , choose the appropriate rotation angle $\theta$ ，Can be used$a_{ik}$ becomes zero. Specifically, choose $\theta$ 使得：
$$c_{ik}=c_{kj}=a_{ik}\cos (\theta )+a_{jk}\sin (\theta )=0$$

  Through this choice, we can construct a rotation matrix $P_{i,j,k}$, the orthogonal similarity transformation corresponding to this matrix can convert $c_{ik}$becomes zero. This process realizes the orthogonal similarity transformation of the real symmetric matrix, causing some elements to become zero, and gradually realizes the transformation of the matrix into a tridiagonal form.

b. The order of rotation transformation

  When performing Householder transformation, the order of rotation is important. Typically, you can choose to transform by column. For example, for $i=2,\dots ,n-1$, $j=i+1,i+2,\dots ,n$，Naturally $(i,j)$ perform orthogonal similarity transformation, transform $a_{ik}$ becomes zero. The iteration of the entire process will gradually transform the real symmetric matrix $A$ is transformed into a tridiagonal matrix $C$。

2. Householder Matrix

a. Definition of H matrix

  设 $u$amount of direction, immediately $\Vert u\Vert =1$. set义 Householder 矩阵 $H=I-2u{u}^T$, inside $I$ is the identity matrix. This matrix has the following properties:

• 对称性： $H^T=H$, that is, the Householder matrix is ​​symmetric.
• 正交性： $H^{T}H=I$, that is, the Householder matrix is ​​an orthogonal matrix.
• Conservation: for any non-zero vector $x$ sum $y$, result $\Vert x{\Vert }^2=\Vert y{\Vert }^2$，则existing Householder 矩阵 $H$, use $Hx=y$。
  • Consider Householder matrix versus vector $u$ action:$Hu=(I-2u{u}^T)u=-u$. Description Householder squareforward direction amount $u$ Reflection on the direction of the loss.
  • What do you think? $u$ Orthogonal direction $v$，Yes $Hv=(I-2u{u}^T)v=v$，Immediately Householder $u$ Orthogonal direction/quantity change.
  • So for any vector $z$，设 $With=au+bv$，nan么 Householder 变换会将 $z$ 变换为 $-au+bv$, Mathematical expression:
    $$Hz=a(Hu)+b(Hv)\to -au+bv$$

b. Geometric interpretation of H transformation

  The Householder transformation can be thought of as specular reflection . Consider $u$ is the unit normal vector on the reflecting surface. For any $z$，Householder 变换将$z$ projection reflection reaches $-u$ direction while maintaining the projection on the reflective surface.

c. Application scenarios of H transformation

1. Matrix tridiagonalization: In computational linear algebra, Householder transformation is often used to convert matrices into tridiagonal form, making it easier to perform operations such as eigenvalue calculations.
2. QR decomposition: The Householder transform is a basic tool for computing QR decomposition, which is used to decompose a matrix into the product of an orthogonal matrix and an upper triangular matrix.

3. Detailed explanation of H transformation process

a. Process introduction

  对于矩阵$A$ A certain one-column direction quantity $\mathbf{a}=(a_1,a_2,\dots ,a_n{)}^T$, if we want to convert the back end of the vector $n-(r+1)$Partialization is zero, immediate general $\mathbf{a}$ 变为 $\mathbf{c}=(a_1,a_2,\dots ,a_r,-\text{sign}(a_{r+1})s,0,\dots ,0{)}^T$, inside $s=\Vert a{\Vert }_2$ （从$r+1$calculation reached$n$) except $a_{r+1}\ne 0$, then the Householder matrix $H$，Use$Ha=c$. The Householder matrix is ​​calculated as follows:
$$In=a-\text{sign}(a_{r+1})s{\mathbf{e}}_{r+1}$$
  Among them, ${It\; is}_{r+1}$ is a unit vector$(0,0,\dots ,0,1,0,\dots ,0{)}^T$, concrete location $r+1$ 个.

b. Detailed analysis

• Construction of Householder matrix:
  • Construct the Householder matrix through Householder transformation $H$，Sho certain line $a_j$的 $r+1$ to $n$ components are reduced to zero.

• Calculation process stability:

  • General $a$ 的 $r+1$ to $The\; signs\; of\; the\; n$ components are set to$-\text{sign}\left(a_{r+1}\right)$ to enhance the stability of calculations.

• Compute similar tridiagonal matrices:

  • General $A$ Perform orthogonal similarity transformation column by column to obtain$A_1,A_2,\dots ,A_{n-1}$。
  • Finally get a similar tridiagonal matrix$G=A_n=H_{n-2}\cdot \dots \cdot H_2\cdot H_1\cdot A$。

• Optimization in actual calculations:

  • In actual calculations, there is no need to form all Householder matrices or perform matrix multiplication operations, and calculations can be performed directly on the original matrices.
    

4. Analysis of H transformation examples

  integer:
$$A=\left[\begin{array}{cccc}1& 2& 1& 2\\ 2& 2& -1& 1\\ 1& -1& 1& 1\\ 2& 1& 1& 1\end{array}\right]$$

Final tridiagonal matrix $A_2$：

• $A_2$Integers
  $$A_2=\left[\begin{array}{cccc}1& -3& 0& 0\\ -3& 2.3333& -0.4714& 0\\ 0& -0.4714& 1.1667& -1.5003\\ 0& 0& -1.5003& 0.5002\end{array}\right]$$

  Thus, by selecting $r=1$ sum $r=2$ Perform two Householder transformations, matrix $A$ Successful place conversion 为三对angled form $A_2$。

4. Python implementation

import numpy as np


def householder_matrix(v):
    """
    给定向量 v，返回 Householder 变换矩阵 H
    """
    v = np.array(v, dtype=float)
    if np.linalg.norm(v) == 0:
        raise ValueError("无效的输入向量，它应该是非零的。")

    v = v / np.linalg.norm(v)
    H = np.eye(len(v)) - 2 * np.outer(v, v)
    return H


def householder_reduction(A):
    """
    对矩阵 A 执行 Householder 变换，将其化为三对角形式。
    """
    m, n = A.shape
    if m != n:
        raise ValueError("输入矩阵 A 必须是方阵。")

    Q = np.eye(m)  # 初始化正交矩阵 Q

    for k in range(n - 2):
        x = A[k + 1:, k]
        e1 = np.zeros_like(x)
        e1[0] = 1.0
        v = np.sign(x[0]) * np.linalg.norm(x) * e1 + x
        v = v / np.linalg.norm(v)

        H = np.eye(m)
        H[k + 1:, k + 1:] -= 2.0 * np.outer(v, v)
        Q = np.dot(Q, H)
        A = np.dot(H, np.dot(A, H))

    return Q, A


# 示例矩阵
A = np.array([[1, 2, 1, 2],
              [2, 2, -1, 1],
              [1, -1, 1, 1],
              [2, 1, 1, 1]], dtype=float)

# Householder 变换
Q, tridiagonal_A = householder_reduction(A)
np.set_printoptions(precision=4, suppress=True)
print("正交矩阵 Q:")
print(Q)
print("\n三对角矩阵:")
print(tridiagonal_A)

Debugging process

• first:
  

• the second time:
  

• final：