Solving systems of linear equations , methods of least squares , and eigenvalue/eigenvector solutions are the main research contents of (numerical) linear algebra.

In mechanics, meteorology, electromagnetism, finance and other disciplines, many problems ultimately boil down to the solution of eigenvalues and eigenvectors.

ARPACK uses IRAM ( Implicit Restarted Arnoldi Method ) to solve some eigenvalues and eigenvectors of large-scale coefficient matrices. Understanding or being familiar with the IRAM algorithm will definitely help you to better use the related eigenvalue solving functions in ARPACK.

This article intends to analyze the IRAM algorithm for solving eigenvalues in ARPACK , hoping to help friends who are engaged in related research.

Note 1: Limited to the research level, the analysis is inevitably inappropriate, criticism and correction are welcome.

zero, advance repair

For n-order complex matrices $\boldsymbol{A}\in \mathbb{C}^{n\times n}$ , if there are n-order vectors $\boldsymbol{x}\in \mathbb{C}^{n}$ and scalars $\lambda \in \mathbb{C}^{1}$ that satisfy $\boldsymbol{\mathbf{A}}\boldsymbol{\mathbf{x}}=\lambda \boldsymbol{x}$ , it is called the eigenvalue of $\lambda$ the matrix and is the corresponding eigenvector. $\boldsymbol{A}$ $\boldsymbol{x}$

More generally, if for an n-order complex matrix $\boldsymbol{A}\in \mathbb{C}^{n\times n}$ , $\boldsymbol{B}\in \mathbb{C}^{n\times n}$ , if there are n-order vectors $\boldsymbol{x}\in \mathbb{C}^{n}$ and scalars $\lambda \in \mathbb{C}^{1}$ , satisfying $\boldsymbol{\mathbf{A}}\boldsymbol{\mathbf{x}}=\lambda\boldsymbol{B} \boldsymbol{x}$ , then it is called the generalized eigenvalue of $\lambda$ the matrix , and is the corresponding generalized eigenvector. $\boldsymbol{A}$ $\boldsymbol{x}$

If $\boldsymbol{Q}\in \mathbb{C}^{n\times n}$ , is satisfied $\boldsymbol{Q}^{H}\boldsymbol{Q}=\boldsymbol{Q}\boldsymbol{Q}^{H}=\boldsymbol{I}$ , it $\boldsymbol{Q}$ is called a unitary matrix.

Define the Hessenberg matrix $\boldsymbol{H}$ ,

$\boldsymbol{H}=\begin{pmatrix} h_{1,1} & h_{1,2} & \cdots & h_{1,k}\\ h _{2,1} & h_{2,2} & \cdots &h_{2,k} \\ 0& \ddots& \ddots &\vdots \\ 0& 0 & h_{k,k-1}& h _{k,k} \end{pmatrix}$

Schur's decomposition theorem : Suppose $\boldsymbol{A}\in \mathbb{C}^{n\times n}$ , then there exists a unitary matrix $\boldsymbol{U}\in \mathbb{C}^{n\times n}$ such that

$\boldsymbol{U}^{H}\boldsymbol{A}\boldsymbol{U}=\boldsymbol{T}$

Among them, $\boldsymbol{T}$ is the upper triangular matrix.

Real Schur decomposition : Suppose $\boldsymbol{A}\in \mathbb{R}^{n\times n}$ , then there is an orthogonal matrix $\boldsymbol{Q}\in \mathbb{R}^{n\times n}$ such that

$\boldsymbol{Q}^{T}\boldsymbol{A}\boldsymbol{Q}=\begin{pmatrix} \boldsymbol{R}_{11} & \boldsymbol{R}_{12} & \cdots & \boldsymbol{R}_{1m}\\ & \boldsymbol{R}_{22}& \cdots & \boldsymbol{R}_{2m}\\ & & \ddots & \vdots \\ & & & \boldsymbol{R}_{mm} \end{pmatrix}$

where $\boldsymbol{R}_{ii}$ is a real number, or a 2nd order simulation with a pair of complex conjugate eigenvalues.

For the convenience of description, this paper discusses the standard eigenvalue problem instead of the generalized eigenvalue problem.

1. Large sparse matrix eigenvalue algorithm

1.1 Lancos algorithm

1.2 Arnoldi Algorithm

In an article published in 1951, Arnoldi proposed a method to obtain some eigenvalues and eigenvectors of large sparse matrices. In this method, the matrix is reduced to a low-order Hessenberg matrix by the Ritz \ Galerkin residual method , and then the eigenvalues and eigenvectors of the Hessenberg matrix are solved to approximate part of the eigenvalues and eigenvectors of the source matrix, namely $\boldsymbol{u}$ $\boldsymbol{u}$

$\left [ \kappa _{i} \right ]\left ( \lambda \boldsymbol{I}-\boldsymbol{u} \right )\left [ k_{i} \right ]\boldsymbol{c}=0$

The above formula can also be written

$\boldsymbol{A}\boldsymbol{V}=\boldsymbol{V}\boldsymbol{H}+\boldsymbol{f}\boldsymbol{e}_{k}^{T}$

Among them, $\boldsymbol{V}=\left ( \boldsymbol{v}_{1}, \boldsymbol{v}_{2},\cdots ,\boldsymbol{v}_{k} \right )\in \mathbb{C}^{n\times k }$ , $\boldsymbol{V}^{H}\boldsymbol{V}=\mathbf{I}$ , $\boldsymbol{H}=h_{i,j}\in \mathbb{C}^{k\times k }$ , $\boldsymbol{f}\in \mathbb{C}^{n }$ , $\boldsymbol{V}^{H} \mathbf{f}=\mathbf{0}$ , $e_{k}^{T}=\left ( 0,0,\cdots,0,1 \right )\in \mathbb{C}^{k}$ .

For $k+1$ , set

$\boldsymbol{A}\boldsymbol{\tilde{V}}=\boldsymbol{\tilde{V}}\boldsymbol{\tilde{H}}+\boldsymbol{\tilde{f}}\boldsymbol{e}_{k+1}^{T}$

Among them $\boldsymbol{\tilde{V}}=\left ( \boldsymbol{V},\boldsymbol{v}_{k+1} \right )\in \mathbb{C}^{n\times \left (k+1 \right ) }$ , $\mathbf{\tilde{H}}=\begin{pmatrix} \boldsymbol{H} & \mathbf{h}_{2}\\ \mathbf{h}_{1}& h_{3} \end{pmatrix}\in \mathbb{C}^{\left (k+1 \right )\times \left (k+1 \right ) }$ , $\boldsymbol{\tilde{f}}\in \mathbb{C}^{n }$ , $\boldsymbol{\tilde{V}}^{H}\boldsymbol{\tilde{f}}=0$ , $e_{k+1}^{T}=\left ( 0,0,\cdots,0,1 \right )\in \mathbb{C}^{k+1}$ , then there are

$\left\{\begin{matrix} \boldsymbol{A}\boldsymbol{\tilde{V}}=\left ( \boldsymbol{A}\boldsymbol{V},\boldsymbol{A}\boldsymbol{v}_{k+1} \right )\\ \boldsymbol{\tilde{V}}\boldsymbol{\tilde{H}}=\left ( \boldsymbol{V},\boldsymbol{v}_{k+1} \right ) \begin{pmatrix} \boldsymbol{H} & \boldsymbol{h}_{2}\\ \boldsymbol{h}_{1}& h_{3} \end{pmatrix} \end{matrix}\right.\Rightarrow \left\{\begin{matrix} \boldsymbol{A}\boldsymbol{V}=\boldsymbol{V}\boldsymbol{H}+\boldsymbol{v}_{k+1}\boldsymbol{h}_{1}\\ \boldsymbol{A}\boldsymbol{v}_{k+1}=\boldsymbol{V}\boldsymbol{h}_{2}+\boldsymbol{v}_{k+1}h_{3}+\boldsymbol{\tilde{f}} \end{matrix}\right.$

If taken $\boldsymbol{v}_{k+1}=\frac{\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|}$ , there is

$\left\{\begin{matrix} \boldsymbol{h}_{1}=\left \| \mathbf{f} \right \|\boldsymbol{e}_{k}^{T} \\ \boldsymbol{h}_{2}=\frac{\boldsymbol{V}^{H}\boldsymbol{A}\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|}\\ h_{3}=\frac{\boldsymbol{f}^{H}\boldsymbol{A}\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|^{2}}\end{matrix}\right.$ ， $\left\{\begin{matrix} \boldsymbol{\tilde{V}}=\left ( \boldsymbol{V}, \frac{\boldsymbol{f}}{ \left \| \boldsymbol{f} \right \| }\right )\\ \boldsymbol{\tilde{H}}=\begin{pmatrix} \boldsymbol{H} &\frac{\boldsymbol{V}^{H}\boldsymbol{A}\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|} \\ \left \| \mathbf{f} \right \|\boldsymbol{e}_{k}^{T} &\frac{\boldsymbol{f}^{H}\boldsymbol{A}\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|^{2}} \end{pmatrix}\\ \boldsymbol{\tilde{f}}= \frac{\boldsymbol{A}\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|}-\boldsymbol{V}\frac{\boldsymbol{V}^{H}\boldsymbol{A}\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|}-\frac{\boldsymbol{f}^{H}\boldsymbol{A}\boldsymbol{f}}{\left \| \boldsymbol{f} \right \|^{3}}\boldsymbol{f}\end{matrix}\right.$

In addition, it can be seen $\boldsymbol{v}_{k+1}=\frac{\boldsymbol{A}\boldsymbol{v}_{k}-\boldsymbol{V}\boldsymbol{h}\left ( :,k \right )}{\left \| \boldsymbol{f} \right \|}$ that $\boldsymbol{V}=\left ( \boldsymbol{v}_{1}, \boldsymbol{v}_{2},\cdots ,\boldsymbol{v}_{k} \right )$ is a set of orthonormal basis of the Krylov subspace . $\boldsymbol{K}\left ( \boldsymbol{A},\mathbf{v}_{1},k \right )=\left ( \mathbf{v}_{1} ,\boldsymbol{A}\mathbf{v}_{1},\boldsymbol{A}^{2}\mathbf{v}_{1},\cdots ,\boldsymbol{A}^{k-1}\mathbf{v}_{1}\right )$

In the actual numerical calculation process, due to calculation accuracy and other issues, it is usually difficult to guarantee , that is to say , there is a problem of loss of orthogonality in the process of $\boldsymbol{V}^{H}\boldsymbol{V}=\boldsymbol{I}$ matrix column-by-column calculation. $\boldsymbol{V}=\left ( \boldsymbol{v}_{1}, \boldsymbol{v}_{2},\cdots ,\boldsymbol{v}_{k} \right )$

In response to this problem, Sorensen proposed an algorithm called IRAM ( Implicit Restarted Arnoldi Method ) in his 1992 paper. Sorensen believes that the error $\boldsymbol{f}$ is actually a function of the initial vector $\boldsymbol{v}_{1}$ , which can be forced by continuously selecting new initial vectors $\boldsymbol{v}_{i}^{new}=\boldsymbol{A}\boldsymbol{v}_{k+1}$ and iterating the Arnoldi process $p$ times $\boldsymbol{f}\rightarrow \mathbf{0}$ , and gives a proof of convergence.

$\boldsymbol{A}\boldsymbol{V}_{k+p}=\boldsymbol{V}_{k+p}\boldsymbol{H}_{k+p}+\boldsymbol{f}_{k+p}\boldsymbol{e}_{k+p}^{T}$

For the obtained $\boldsymbol{H}_{k+p}$ , a QR $p$ decomposition with sub-band offset needs to be applied to remove unwanted eigenvalues/eigenvectors. This is actually the IRAM ( Implicit Restarted Arnoldi Method )) algorithm used by ARPACK .

reference books

Gene H. Golub. Matrix Computations.

Xu Shufang. Numerical Linear Algebra. Peking University Press, 2013.

references

Arnoldi W E .The principle of minimized iterations in the solution of the matrix eigenvalue problem[J].Quarterly of Applied Mathematics, 1951, 9(1).DOI:10.1093/qjmam/4.4.466.

D,C,Sorensen.Implicit Application of Polynomial Filters in a k-Step Arnoldi Method[J].Siam Journal on Matrix Analysis & Applications, 1992.DOI:10.1137/0613025.

Burrus C S , Cox S J , Radke R J .A Matlab Implementation of the Implicitly Restarted Arnoldi Method for Solving Large-Scale Eigenvalue Problems[J]. 2000.

R.B. Lehoucq, Analysis and Implementation of an Implicitly Restarted Arnoldi Iteration", Rice University Technical Report TR95-13, Department of Computational and Applied Mathematics.

network information

ARPACK https://github.com/opencollab/arpack-ng

Numerical Linear Algebra: Arnoldi solving eigenvalues/eigenvectors