《Numerical Recipes in C++》原文摘要：
There exists a very powerful set of techniques for dealing with sets of equations or matrices that are either singular or else numerically very close to singular. In many cases where Gaussian elimination and LU decomposition fail to give satisfac- tory results, this set of techniques, known as singular value decomposition, or SVD, will diagnose for you precisely what the problem is. In some cases, SVD will not only diagnose the problem, it will also solve it, in the sense of giving you a useful numerical answer, although, as we shall see, not necessarily “the” answer that you thought you should get.
SVD is also the method of choice for solving most linear least-squares prob- lems. We will outline the relevant theory in this section, but defer detailed discussion of the use of SVD in this application to Chapter 15, whose subject is the parametric modeling of data.
SVD methods are based on the following theorem of linear algebra, whose proof is beyond our scope: Any M N matrix A can be written as the product of an M N column-orthogonal matrix U, an N N diagonal matrix W with positive or zero elements (the singular values), and the transpose of an N N orthogonal matrix V . The various shapes of these matrices are clearer when shown as tableaus.
凑字数的狗屁不通的译文：
存在一套非常强大的技术来处理奇异或数值上非常接近奇异的方程或矩阵集。在许多情况下，高斯消元和LU分解无法给出令人满意的结果，这组技术（称为奇异值分解，或SVD）将为您准确诊断问题所在。在某些情况下，奇异值分解不仅可以诊断问题，还可以解决问题，从某种意义上说，它可以给你一个有用的数字答案，尽管，正如我们将看到的，不一定是你认为应该得到的“答案”。
奇异值分解也是求解大多数线性最小二乘问题的首选方法。我们将在本节概述相关理论，但将在此应用中使用奇异值分解的详细讨论推迟到第15章，其主题是数据的参数建模。
奇异值分解方法基于线性代数的以下定理，其证明超出了我们的范围：任何M N矩阵A都可以写成M N列正交矩阵U、具有正或零元素（奇异值）的N N对角矩阵W和N N正交矩阵V的转置的乘积。当以表格形式显示时，这些矩阵的各种形状更清晰。
《Numerical Recipes in C++》原文摘要：
The decomposition (2.6.1) or (2.6.2) can always be done, no matter how singu- lar the matrix is, and it is “almost” unique. That is to say, it is unique up to (i) making the same permutation of the columns of U, elements of W , and columns of V (or rows of V T ); or (ii) performing an orthogonal rotation on any set of columns of U and V whose corresponding elements of W happen to be exactly equal. (A special case is multiplying any column of U, and the corresponding column of V by 1.) A consequence of the permutation freedom is that for the case M < N , a numerical algorithm for the decomposition need not return zero wj ’s in the canonical positions j M; :::;N 1; the N M zero singular values can be scattered among all positions j 0; 1; :::;N 1, and one needs to perform a sort to get the canonical order. In any case, it is conventional to sort all the singular values into descending order.
A Webnote [1] gives the details of the routine that actually performs SVD on an arbitrary matrix A, yielding U, W , and V . The routine is based on a routine
by Forsythe et al. [2], which is in turn based on the original routine of Golub and Reinsch, found, in various forms, in [4-6] and elsewhere. These references include extensive discussion of the algorithm used. As much as we dislike the use of black- box routines, we need to ask you to accept this one, since it would take us too far afield to cover its necessary background material here. The algorithm is very stable, and it is very unusual for it ever to misbehave. Most of the concepts that enter the algorithm (Householder reduction to bidiagonal form, diagonalization by QR procedure with shifts) will be discussed further in Chapter 11.
凑字数的狗屁不通的译文：
分解（2.6.1）或（2.6.2）始终可以完成，无论矩阵多么单一，并且它“几乎”唯一。也就是说，它是唯一的，直到（i）对U的列、W的元素和V的列（或V T的行）进行相同的排列；或（ii）对U和V的任何列集执行正交旋转，其对应的W元素恰好相等。（一种特殊情况是将U的任意列与V的相应列相乘1。）置换自由度的一个结果是，对于M<N的情况，分解的数值算法不需要在正则位置j M处返回零wj；：；N 1；N M零奇异值可以分散在所有位置j 0之间；1; :::;N 1，需要执行排序以获得规范序。在任何情况下，通常将所有奇异值按降序排序。
Webnote[1]给出了在任意矩阵A上实际执行奇异值分解的例程的详细信息，产生U、W和V。例程基于例程
由Forsythe等人[2]以Golub和Reinsch的原始例程为基础，在[4-6]和其他地方以各种形式发现。这些参考文献包括对所用算法的广泛讨论。虽然我们不喜欢使用黑盒例程，但我们需要请您接受这一点，因为这将使我们走得太远，无法在这里涵盖其必要的背景材料。该算法非常稳定，而且其行为异常是非常罕见的。进入算法的大多数概念（Householder约化为双对角形式，通过带移位的QR过程对角化）将在第11章中进一步讨论。
源程序（POWER BY 315SOFT.COM）
using System;
namespace Legalsoft.Truffer
{
/// <summary>
/// Object for singular value decomposition of a matrix A. - PTC
///
/// A = u * w * vT
/// </summary>
public class SVD
{
private int m { get; set; }
private int n { get; set; }
/// <summary>
/// The matrices U and V
/// </summary>
private double[,] u { get; set; }
public double[,] v { get; set; }
/// <summary>
/// The diagonal matrix W
/// </summary>
public double[] w { get; set; }
private double eps { get; set; }
public double tsh { get; set; }
/// <summary>
/// The single argument is A.The SVD computation is done by decompose, and the
/// results are sorted by reorder.
/// </summary>
/// <param name="a"></param>
public SVD(double[,] a)
{
this.m = a.GetLength(0);
this.n = a.GetLength(1);
this.u = a;
this.v = new double[n, n];
this.w = new double[n];
eps = float.Epsilon;
decompose();
reorder();
tsh = 0.5 * Math.Sqrt(m + n + 1.0) * w[0] * eps;
}
/// <summary>
/// Solve A*x = b for a vector x using the pseudoinverse of A as obtained by
/// SVD.If positive, thresh is the threshold value below which singular values
/// are considered as zero.If thresh is negative, a default based on expected
/// roundoff error is used.
/// </summary>
/// <param name="b"></param>
/// <param name="x"></param>
/// <param name="thresh"></param>
/// <exception cref="Exception"></exception>
public void solve(double[] b, double[] x, double thresh = -1.0)
{
if (b.Length != m || x.Length != n)
{
throw new Exception("SVD::solve bad sizes");
}
double[] tmp = new double[n];
tsh = (thresh >= 0.0 ? thresh : 0.5 * Math.Sqrt(m + n + 1.0) * w[0] * eps);
for (int j = 0; j < n; j++)
{
double s = 0.0;
if (w[j] > tsh)
{
for (int i = 0; i < m; i++)
{
s += u[i, j] * b[i];
}
s /= w[j];
}
tmp[j] = s;
}
for (int j = 0; j < n; j++)
{
double s = 0.0;
for (int jj = 0; jj < n; jj++)
{
s += v[j, jj] * tmp[jj];
}
x[j] = s;
}
}
/// <summary>
/// Return reciprocal of the condition number of A
/// </summary>
/// <returns></returns>
public double inv_condition()
{
return (w[0] <= 0.0 || w[n - 1] <= 0.0) ? 0.0 : w[n - 1] / w[0];
}
/// <summary>
/// Solves m sets of n equations A* X = B using the pseudoinverse of A.The
/// right-hand sides are input as b[0..n - 1][0..m - 1], while x[0..n - 1][0..m - 1]
/// returns the solutions.thresh as above.
/// </summary>
/// <param name="b"></param>
/// <param name="x"></param>
/// <param name="thresh"></param>
/// <exception cref="Exception"></exception>
public void solve(double[,] b, double[,] x, double thresh = -1.0)
{
int p = b.GetLength(1);
if (b.GetLength(0) != m || x.GetLength(0) != n || x.GetLength(1) != p)
{
throw new Exception("SVD::solve bad sizes");
}
double[] xx = new double[n];
double[] bcol = new double[m];
for (int j = 0; j < p; j++)
{
for (int i = 0; i < m; i++)
{
bcol[i] = b[i, j];
}
solve(bcol, xx, thresh);
for (int i = 0; i < n; i++)
{
x[i, j] = xx[i];
}
}
}
/// <summary>
/// Return the rank of A, after zeroing any singular values smaller than
/// thresh.If thresh is negative, a default value based on estimated roundoff
/// is used.
/// </summary>
/// <param name="thresh"></param>
/// <returns></returns>
public int rank(double thresh = -1.0)
{
int nr = 0;
tsh = (thresh >= 0.0 ? thresh : 0.5 * Math.Sqrt(m + n + 1.0) * w[0] * eps);
for (int j = 0; j < n; j++)
{
if (w[j] > tsh)
{
nr++;
}
}
return nr;
}
public int nullity(double thresh = -1.0)
{
int nn = 0;
tsh = (thresh >= 0.0 ? thresh : 0.5 * Math.Sqrt(m + n + 1.0) * w[0] * eps);
for (int j = 0; j < n; j++)
{
if (w[j] <= tsh)
{
nn++;
}
}
return nn;
}
/// <summary>
/// Give an orthonormal basis for the range of A as the columns of a returned
/// matrix.thresh as above.
/// </summary>
/// <param name="thresh"></param>
/// <returns></returns>
public double[,] range(double thresh = -1.0)
{
int nr = 0;
double[,] rnge = new double[m, rank(thresh)];
for (int j = 0; j < n; j++)
{
if (w[j] > tsh)
{
for (int i = 0; i < m; i++)
{
rnge[i, nr] = u[i, j];
}
nr++;
}
}
return rnge;
}
/// <summary>
/// Return the nullity of A, after zeroing any singular values smaller than
/// thresh.Default value as above.
/// </summary>
/// <param name="thresh"></param>
/// <returns></returns>
public double[,] nullspace(double thresh = -1.0)
{
int nn = 0;
double[,] nullsp = new double[n, nullity(thresh)];
for (int j = 0; j < n; j++)
{
if (w[j] <= tsh)
{
for (int jj = 0; jj < n; jj++)
{
nullsp[jj, nn] = v[jj, j];
}
nn++;
}
}
return nullsp;
}
public void decompose()
{
int l = 0;
int nm = 0;
double[] rv1 = new double[n];
double s;
double g = 0.0;
double scale = 0.0;
double anorm = 0.0;
for (int i = 0; i < n; i++)
{
l = i + 2;
rv1[i] = scale * g;
g = s = scale = 0.0;
if (i < m)
{
for (int k = i; k < m; k++)
{
scale += Math.Abs(u[k, i]);
}
if (scale != 0.0)
{
for (int k = i; k < m; k++)
{
u[k, i] /= scale;
s += u[k, i] * u[k, i];
}
double f = u[i, i];
g = -Globals.SIGN(Math.Sqrt(s), f);
double h = f * g - s;
u[i, i] = f - g;
for (int j = l - 1; j < n; j++)
{
s = 0.0;
for (int k = i; k < m; k++)
{
s += u[k, i] * u[k, j];
}
f = s / h;
for (int k = i; k < m; k++)
{
u[k, j] += f * u[k, i];
}
}
for (int k = i; k < m; k++)
{
u[k, i] *= scale;
}
}
}
w[i] = scale * g;
g = 0.0;
s = 0.0;
scale = 0.0;
if (i + 1 <= m && i + 1 != n)
{
for (int k = l - 1; k < n; k++)
{
scale += Math.Abs(u[i, k]);
}
if (scale != 0.0)
{
for (int k = l - 1; k < n; k++)
{
u[i, k] /= scale;
s += u[i, k] * u[i, k];
}
double f = u[i, l - 1];
g = -Globals.SIGN(Math.Sqrt(s), f);
double h = f * g - s;
u[i, l - 1] = f - g;
for (int k = l - 1; k < n; k++)
{
rv1[k] = u[i, k] / h;
}
for (int j = l - 1; j < m; j++)
{
s = 0.0;
for (int k = l - 1; k < n; k++)
{
s += u[j, k] * u[i, k];
}
for (int k = l - 1; k < n; k++)
{
u[j, k] += s * rv1[k];
}
}
for (int k = l - 1; k < n; k++)
{
u[i, k] *= scale;
}
}
}
anorm = Math.Max(anorm, (Math.Abs(w[i]) + Math.Abs(rv1[i])));
}
for (int i = n - 1; i >= 0; i--)
{
if (i < n - 1)
{
if (g != 0.0)
{
for (int j = l; j < n; j++)
{
v[j, i] = (u[i, j] / u[i, l]) / g;
}
for (int j = l; j < n; j++)
{
s = 0.0;
for (int k = l; k < n; k++)
{
s += u[i, k] * v[k, j];
}
for (int k = l; k < n; k++)
{
v[k, j] += s * v[k, i];
}
}
}
for (int j = l; j < n; j++)
{
v[i, j] = v[j, i] = 0.0;
}
}
v[i, i] = 1.0;
g = rv1[i];
l = i;
}
for (int i = Math.Min(m, n) - 1; i >= 0; i--)
{
l = i + 1;
g = w[i];
for (int j = l; j < n; j++)
{
u[i, j] = 0.0;
}
if (g != 0.0)
{
g = 1.0 / g;
for (int j = l; j < n; j++)
{
s = 0.0;
for (int k = l; k < m; k++)
{
s += u[k, i] * u[k, j];
}
double f = (s / u[i, i]) * g;
for (int k = i; k < m; k++)
{
u[k, j] += f * u[k, i];
}
}
for (int j = i; j < m; j++)
{
u[j, i] *= g;
}
}
else
{
for (int j = i; j < m; j++)
{
u[j, i] = 0.0;
}
}
++u[i, i];
}
for (int k = n - 1; k >= 0; k--)
{
for (int its = 0; its < 30; its++)
{
bool flag = true;
for (l = k; l >= 0; l--)
{
nm = l - 1;
if (l == 0 || Math.Abs(rv1[l]) <= eps * anorm)
{
flag = false;
break;
}
if (Math.Abs(w[nm]) <= eps * anorm)
{
break;
}
}
double c;
if (flag)
{
c = 0.0;
s = 1.0;
for (int i = l; i < k + 1; i++)
{
double ff = s * rv1[i];
rv1[i] = c * rv1[i];
if (Math.Abs(ff) <= eps * anorm)
{
break;
}
g = w[i];
double hh = pythag(ff, g);
w[i] = hh;
hh = 1.0 / hh;
c = g * hh;
s = -ff * hh;
for (int j = 0; j < m; j++)
{
double yy = u[j, nm];
double zz = u[j, i];
u[j, nm] = yy * c + zz * s;
u[j, i] = zz * c - yy * s;
}
}
}
double z = w[k];
if (l == k)
{
if (z < 0.0)
{
w[k] = -z;
for (int j = 0; j < n; j++)
{
v[j, k] = -v[j, k];
}
}
break;
}
if (its == 29)
{
throw new Exception("no convergence in 30 svdcmp iterations");
}
double x = w[l];
nm = k - 1;
double y = w[nm];
g = rv1[nm];
double h = rv1[k];
double f = ((y - z) * (y + z) + (g - h) * (g + h)) / (2.0 * h * y);
g = pythag(f, 1.0);
f = ((x - z) * (x + z) + h * ((y / (f + Globals.SIGN(g, f))) - h)) / x;
c = 1.0;
s = 1.0;
for (int j = l; j <= nm; j++)
{
int i = j + 1;
g = rv1[i];
y = w[i];
h = s * g;
g = c * g;
z = pythag(f, h);
rv1[j] = z;
c = f / z;
s = h / z;
f = x * c + g * s;
g = g * c - x * s;
h = y * s;
y *= c;
for (int jj = 0; jj < n; jj++)
{
x = v[jj, j];
z = v[jj, i];
v[jj, j] = x * c + z * s;
v[jj, i] = z * c - x * s;
}
z = pythag(f, h);
w[j] = z;
if (z > 0.0)
{
z = 1.0 / z;
c = f * z;
s = h * z;
}
f = c * g + s * y;
x = c * y - s * g;
for (int jj = 0; jj < m; jj++)
{
y = u[jj, j];
z = u[jj, i];
u[jj, j] = y * c + z * s;
u[jj, i] = z * c - y * s;
}
}
rv1[l] = 0.0;
rv1[k] = f;
w[k] = x;
}
}
}
public void reorder()
{
int inc = 1;
double[] su = new double[m];
double[] sv = new double[n];
do
{
inc *= 3;
inc++;
} while (inc <= n);
do
{
inc /= 3;
for (int i = inc; i < n; i++)
{
double sw = w[i];
for (int k = 0; k < m; k++)
{
su[k] = u[k, i];
}
for (int k = 0; k < n; k++)
{
sv[k] = v[k, i];
}
int j = i;
while (w[j - inc] < sw)
{
w[j] = w[j - inc];
for (int k = 0; k < m; k++)
{
u[k, j] = u[k, j - inc];
}
for (int k = 0; k < n; k++)
{
v[k, j] = v[k, j - inc];
}
j -= inc;
if (j < inc)
{
break;
}
}
w[j] = sw;
for (int k = 0; k < m; k++)
{
u[k, j] = su[k];
}
for (int k = 0; k < n; k++)
{
v[k, j] = sv[k];
}
}
} while (inc > 1);
for (int k = 0; k < n; k++)
{
int s = 0;
for (int i = 0; i < m; i++)
{
if (u[i, k] < 0.0)
{
s++;
}
}
for (int j = 0; j < n; j++)
{
if (v[j, k] < 0.0)
{
s++;
}
}
if (s > (m + n) / 2)
{
for (int i = 0; i < m; i++)
{
u[i, k] = -u[i, k];
}
for (int j = 0; j < n; j++)
{
v[j, k] = -v[j, k];
}
}
}
}
public double pythag(double a, double b)
{
double absa = Math.Abs(a);
double absb = Math.Abs(b);
return (absa > absb ? absa * Math.Sqrt(1.0 + Globals.SQR(absb / absa)) : (absb == 0.0 ? 0.0 : absb * Math.Sqrt(1.0 + Globals.SQR(absa / absb))));
}
}
}