迭代法
迭代思想
将方程组转换成 A x ⃗ = b ⃗ A \vec x=\vec b Ax=b 的形式，其中 A A A 是矩阵。例如
{ 8 x 1 − 3 x 2 + 2 x 3 = 20 4 x 1 + 11 x 2 − x 3 = 33 6 x 1 + 3 x 2 + 12 x 3 = 36 \begin{cases} 8 x_1-3x_2+2x_3=20 \\ 4x_1+11x_2-x_3=33\\ 6x_1+3x_2+12x_3=36\\ \end{cases} ⎩⎪⎨⎪⎧8x1−3x2+2x3=204x1+11x2−x3=336x1+3x2+12x3=36
可得出， A = ( 8 − 3 2 4 11 − 1 6 3 12 ) x ⃗ = ( x 1 x 2 x 3 ) b ⃗ = ( 20 33 36 ) A= \begin{gathered} \begin{pmatrix} 8 & -3&2 \\ 4 & 11&-1 \\ 6&3&12 \end{pmatrix} \end{gathered} \qquad \vec x= \begin{gathered} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} \end{gathered} \qquad \vec b= \begin{gathered} \begin{pmatrix} 20 \\ 33 \\ 36 \end{pmatrix} \end{gathered} A=⎝⎛846−31132−112⎠⎞x=⎝⎛x1x2x3⎠⎞b=⎝⎛203336⎠⎞
首先使用一般解线性方程组的方法解出其精确解 x ∗ = ( 3 , 2 , 1 ) T x^*=(3,2,1)^T x∗=(3,2,1)T
而迭代法的做法是将原方程组改写成 x ⃗ = B x ⃗ + f ⃗ \vec x=B \vec x+\vec f x=Bx+f 的形式，即 { x 1 = 1 8 ( 3 x 2 − 2 x 3 + 20 ) x 2 = 1 11 ( − 4 x 1 + x 3 + 33 ) x 3 = 1 12 ( − 6 x 1 − 3 x 2 + 36 ) \begin{cases} x_1=\frac{1}{8}(3x_2-2x_3+20) \\ x_2=\frac{1}{11}(-4x_1+x_3+33) \\ x_3=\frac{1}{12}(-6x_1-3x_2+36) \\ \end{cases} ⎩⎪⎨⎪⎧x1=81(3x2−2x3+20)x2=111(−4x1+x3+33)x3=121(−6x1−3x2+36) 可得出 B = ( 0 3 8 − 2 8 − 4 11 0 1 11 − 1 2 − 3 12 0 ) f ⃗ = ( 20 8 33 11 36 12 ) B= \begin{gathered} \begin{pmatrix} 0& \frac{3}{8}&-\frac{2}{8} \\ -\frac{4}{11} &0&\frac{1}{11} \\ -\frac{1}{2}&-\frac{3}{12}&0 \end{pmatrix} \end{gathered} \qquad \vec f= \begin{gathered} \begin{pmatrix} \frac{20}{8} \\ \frac{33}{11} \\ \frac{36}{12} \end{pmatrix} \end{gathered} B=⎝⎛0−114−21830−123−821110⎠⎞f=⎝⎛82011331236⎠⎞
取 x ⃗ ( 0 ) = ( 0 , 0 , 0 ) T \vec x^{(0)}=(0,0,0)^T x(0)=(0,0,0)T，由 x ⃗ = B x ⃗ + f ⃗ \vec x=B \vec x+\vec f x=Bx+f 算得 x ⃗ ( 1 ) = ( 2.5 , 3 , 3 ) T \vec x^{(1)}=(2.5,3,3)^T x(1)=(2.5,3,3)T，以此类推， x ⃗ ( 10 ) = ( 3.000032 , 1.999838 , 0.9998813 ) T \vec x^{(10)}=(3.000032,1.999838,0.9998813)^T x(10)=(3.000032,1.999838,0.9998813)T，此时已经非常接近精确解了，也即随着迭代次数的增加，算出的解与精确解的误差逐渐变小。
以上，迭代公式为 x ⃗ ( k + 1 ) = B x ⃗ ( k ) + f ⃗ \vec x^{(k+1)}=B \vec x^{(k)}+\vec f x(k+1)=Bx(k)+f.
上例是一个收敛的例子，但还有一种情况，即迭代是发散的。如果迭代发散的话，随着迭代的增加，结果不会逐渐靠近精确解。
判断迭代的收敛性：
设 e e e为迭代法求得的解与精确解之间的差值，有 e ⃗ ( k + 1 ) = x ⃗ ( k + 1 ) − x ∗ ⃗ = ( B x ⃗ ( k ) + f ⃗ ) − ( B x ⃗ ∗ + f ⃗ ) = B e ⃗ ( k ) = B 2 e ⃗ ( k − 1 ) \vec e^{(k+1)}=\vec x^{(k+1)}-\vec {x^*}=(B\vec x^{(k)}+\vec f)-(B\vec x^{*}+\vec f)=B \vec e^{(k)}=B^2 \vec e^{(k-1)} e(k+1)=x(k+1)−x∗=(Bx(k)+f)−(Bx∗+f)=Be(k)=B2e(k−1)
因此可推出， e ⃗ ( k ) = B k e ⃗ ( 0 ) \vec e^{(k)}=B^k \vec e^{(0)} e(k)=Bke(0)，因此如果 B k → 0 B^k \rightarrow 0 Bk→0，则 e ( k ) → 0 e^{(k)} \rightarrow 0 e(k)→0，即迭代是收敛的。
判断 B k B^k Bk是否趋近于0等价于 B B B的谱半径小于1。 B k → 0 ⇔ e ( B ) < 1 B^k \rightarrow 0 \Leftrightarrow e(B)<1 Bk→0⇔e(B)<1
B B B矩阵的不同会影响方程组的计算结果是否收敛以及收敛的速度，因此不同的迭代方法就是寻找不同的 B B B。
雅可比迭代
对 A x ⃗ = b ⃗ A \vec x=\vec b Ax=b其中的A，有
A = ( a 11 a 12 ⋅ ⋅ ⋅ a 1 n a 21 a 22 ⋅ ⋅ ⋅ a 2 n ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ a m 1 a m 2 ⋅ ⋅ ⋅ a m n ) = ( a 11 0 ⋅ ⋅ ⋅ 0 0 a 22 ⋅ ⋅ ⋅ 0 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ 0 0 ⋅ ⋅ ⋅ a m n ) − ( 0 0 ⋅ ⋅ ⋅ 0 − a 21 0 ⋅ ⋅ ⋅ 0 − a 31 − a 32 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ − a m 1 − a m 2 ⋅ ⋅ ⋅ 0 ) − ( 0 − a 12 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ − a 1 n 0 0 − a 23 ⋅ ⋅ ⋅ − a 2 n 0 0 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ 0 0 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ 0 ) A= \begin{pmatrix} a_{11} &a_{12}&···&a_{1n}\\ a_{21} &a_{22}&···&a_{2n}\\ ··· &···&···&···\\ a_{m1} &a_{m2}&···&a_{mn}\\ \end{pmatrix}= \begin{pmatrix} a_{11} &0&···&0\\ 0 &a_{22}&···&0\\ ··· &···&···&···\\ 0 &0&···&a_{mn}\\ \end{pmatrix}- \begin{pmatrix} 0 &0&···&0\\ -a_{21}&0&···&0\\ -a_{31}&-a_{32}&···&···\\ ···&···&···&···\\ -a_{m1} &-a_{m2}&···&0 \end{pmatrix}- \begin{pmatrix} 0 &-a_{12}&···&···&-a_{1n}\\ 0&0&-a_{23}&···&-a_{2n}\\ 0&0&······&&···\\ 0&0&···&···&0 \end{pmatrix} A=⎝⎜⎜⎛a11a21⋅⋅⋅am1a12a22⋅⋅⋅am2⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅a1na2n⋅⋅⋅amn⎠⎟⎟⎞=⎝⎜⎜⎛a110⋅⋅⋅00a22⋅⋅⋅0⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅00⋅⋅⋅amn⎠⎟⎟⎞−⎝⎜⎜⎜⎜⎛0−a21−a31⋅⋅⋅−am100−a32⋅⋅⋅−am2⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅00⋅⋅⋅⋅⋅⋅0⎠⎟⎟⎟⎟⎞−⎝⎜⎜⎛0000−a12000⋅⋅⋅−a23⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅−a1n−a2n⋅⋅⋅0⎠⎟⎟⎞
简写为 A = D + L + U A=D+L+U A=D+L+U， D D D为对角矩阵， L L L为下对角矩阵， U U U为上对角矩阵。
带入 A x ⃗ = b ⃗ A \vec x=\vec b Ax=b有 ( D − L − U ) x ⃗ = b ⃗ (D-L-U)\vec x=\vec b (D−L−U)x=b， D x ⃗ = ( L + U ) x ⃗ + b ⃗ D \vec x=(L+U)\vec x+\vec b Dx=(L+U)x+b进一步化简， x ⃗ = D − 1 ( L + U ) x ⃗ + D − 1 b ⃗ \vec x=D^{-1}(L+U)\vec x+D^{-1}\vec b x=D−1(L+U)x+D−1b，可得到雅可比迭代的迭代公式为 x ⃗ ( k + 1 ) = D − 1 ( L + U ) x ⃗ ( k ) + D − 1 b ⃗ \vec x^{(k+1)}=D^{-1}(L+U)\vec x^{(k)}+D^{-1}\vec b x(k+1)=D−1(L+U)x(k)+D−1b，对每个元素的迭代公式如下，为了方便理解，以行列式的方式显示：
a 11 x 1 + a 12 x 2 + ⋅ ⋅ ⋅ + a 1 j x j + ⋅ ⋅ ⋅ + a 1 N x N = b 1 a 21 x 1 + a 22 x 2 + ⋅ ⋅ ⋅ + a 2 j x j + ⋅ ⋅ ⋅ + a 2 N x N = b 2 ⋮ ⋮ ⋮ ⋮ ⋮ a j 1 x 1 + a j 2 x 2 + ⋅ ⋅ ⋅ + a j j x j + ⋅ ⋅ ⋅ + a j N x N = b j ⋮ ⋮ ⋮ ⋮ ⋮ a N 1 x 1 + a N 2 x 2 + ⋅ ⋅ ⋅ + a N j x j + ⋅ ⋅ ⋅ + a N N x N = b N a_{11}x_1+a_{12}x_2+···+a_{1j}x_j+···+a_{1N}x_N=b_1 \\ a_{21}x_1+a_{22}x_2+···+a_{2j}x_j+···+a_{2N}x_N=b_2 \\ \vdots \quad\quad\quad\quad \vdots \quad\quad\quad\quad\quad \vdots \quad\quad\quad\quad \vdots \quad \quad\quad\quad \vdots \\ a_{j1}x_1+a_{j2}x_2+···+a_{jj}x_j+···+a_{jN}x_N=b_j \\ \vdots \quad\quad\quad\quad \vdots \quad\quad\quad\quad\quad \vdots \quad\quad\quad\quad \vdots \quad \quad\quad\quad \vdots \\ a_{N1}x_1+a_{N2}x_2+···+a_{Nj}x_j+···+a_{NN}x_N=b_N \\ a11x1+a12x2+⋅⋅⋅+a1jxj+⋅⋅⋅+a1NxN=b1a21x1+a22x2+⋅⋅⋅+a2jxj+⋅⋅⋅+a2NxN=b2⋮⋮⋮⋮⋮aj1x1+aj2x2+⋅⋅⋅+ajjxj+⋅⋅⋅+ajNxN=bj⋮⋮⋮⋮⋮aN1x1+aN2x2+⋅⋅⋅+aNjxj+⋅⋅⋅+aNNxN=bN
可得元素的迭代公式为：
x j ( k + 1 ) = b j − a j 1 x 1 ( k ) − ⋯ − a j ( j − 1 ) x ( j − 1 ) ( k ) − a j ( j + 1 ) x ( j + 1 ) ( k ) − ⋯ − a j N x N ( k ) a j j x_j^{(k+1)}=\frac{b_j-a_{j1}x_1^{(k)}-\cdots -a_{j (j-1)}x_{(j-1)}^{(k)} - a_{j (j+1)}x_{(j+1)}^{(k)}- \cdots -a_{j N}x_N^{(k)} }{a_{jj}} xj(k+1)=ajjbj−aj1x1(k)−⋯−aj(j−1)x(j−1)(k)−aj(j+1)x(j+1)(k)−⋯−ajNxN(k)