Tensor method of model reduction method

Tensor method of model reduction method

Introduction

The introduction of the POD method is as follows:
https://blog.csdn.net/lusongno1/article/details/125944587

The classic POD method cannot deal with nonlinear terms, so many methods have been developed, such as DEIM:
https://blog.csdn.net/lusongno1/article/details/125955245

If there is a set of parameters in the equation we want to consider, and different parameters can get a Matrix, then how should we put the data obtained under multiple sets of parameters together as a priori to model the new set of parameters? What about downgrading? This is the problem to be considered in this article, that is, high-order tensor decomposition and interpolation in parameter dimensions. Simply put, if there is already a snapshot under a set of parameters, then a new parameter is added, and the SVD decomposition under it does not need to find a new snapshot, but only needs to be interpolated from the original parameters.

This blog is a bit difficult to understand, you can take a look at it, and then read some articles. We omit the explanation of the meaning of some conventional symbols, and you can refer to the references for the meaning of symbols that you do not understand. We only present the method, proof analysis and experiments and can read the paper.

basic knowledge

Let's start with some basic definitions. A tensor in an algebraic tensor space can be written as a linear combination of elementary tensors:
$$v=\sum _{i=1}^m{v}_i^{(1)}\otimes \cdots \otimes v_i^{(d)}$$

Note that there is actually no combination coefficient here.

The smallest subspace of a known tensor, which is a set of spaces. It is $v\in {⨂}_{v=1}^d{U}_v$中，$U_{}$minimization. That is to say, it can wrap vvThe smallest spatial component of$v\; .$

The rank of a second-rank tensor, which is the smallest rrUse$r$
$$u=\sum _{i=1}^r{s}_i\otimes v_i$$

For second-order tensors, the singular value decomposition is generally written as follows:
$$u=\sum _{i=1}^n{p}_i{s}_i\otimes v_i$$

For high-order tensors, its Canonical rank is the extension of the second-order tensor rank, that is, the smallest $r$ 使得，
$$v=\sum _{i=1}^r{v}_i^{(1)}\otimes \cdots \otimes v_i^{(d)}$$
The so-called $\alpha$ is worth the dimensionality of the smallest subspace,$\alpha interface$ ,
$${\mathrm{rank}}_{}(v)=\dim \left(U_a^{\min }(v)\right).$$

Tucker Rank is actually a vector, which is defined by the dimension of the smallest subspace. That is, one with Tucker Rank $r={\left(r_{}\right)}_{v\in D}$tensor set, which satisfies that in all index sets, the dimension of the corresponding minimum subspace is limited, namely:
$${\mathcal{T}}_r=\left\{v\in X:{\mathrm{rank}}_v(v)=\dim \left(U_v{}^{min}\left(v\right)\right)\le r_{},v\in D\right\},$$
then an element in the Tucker tensor can be written as:
$$v=\sum _{i_1=1}^{r_1}\dots \sum _{i_d=1}^{r_d}{C}_{i_1,\dots ,i_d}{v}_{i_1}^{(1)}\otimes \cdots \otimes v_{i_d}^{(d)}$$

Tree-Based Rank means that we do not limit the dimensionality of the smallest component subspace corresponding to a single indicator, but we limit the dimensionality of the tensor decomposition space corresponding to a given indicator set, that is, BT
$$\mathcal{B}{\mathcal{T}}_r=\left\{v\in X:{\mathrm{rank}}_{}(v)=\dim \left(U_a^{\min }(v)\right)\le r_{},a\in T_D\right\}.$$

Note here that a set of indicators is given a limit, and there are as many sets of indicators as there are limits. Tucker Rank is a special case when it aggregates a single indicator for each indicator.

TT (Tensor-train) Rank is also a special case of Tree-Based Rank. It obtains index sets that are ordered, a set of index sets that grow from the tail, that is,
TT r = { v ∈ X : rank ⁡ { k + 1 , … , d } ( v ) ≤ rk } \mathscr{T} \mathscr{T}_{r}=\left\{v \in X: \operatorname{rank}_{\{k+1, \ldots, d\}}(v) \leq r_{k}\right \}TTr​={ v∈X:rank{ k+1,…,d}​(v)≤rk​}

The following is a brief introduction to Higher Order Singular Value Decomposition (HOSVD). Given an indicator set $\alpha$ , the vector space can be divided into two parts. A general truncated SVD can be written as:
$u_{a,r_{}}={\sum }_{i=1}^{r_{}}{p}_i^{(a}{u}_i^{(a}\otimes u_i^{\left(a^c\right)}$

Both the Tucker format and the tree-based Tucker format of HOSVD are implemented based on projection, so I won’t go into details.

Parametric problems and traditional POD methods

Suppose the problem we are solving is as follows,
$$u_t=F(t,\mathbf{u},a),t\in (0,T),\text{ and }{\mathbf{u}|}_{t=0}={\mathbf{u}}_0$$
We arrange the solution of this problem into a matrix, called snapshots,
$${Phi}_{\text{pod }}\left(a\right)=\left[p_1\left(a\right),\dots ,{\varphi }_N\left(a\right)\right]\in {\mathbb{R}}^{M\times N}$$
performs singular value decomposition on the snapshots matrix,
$${Phi}_{\mathrm{pod}}\left(a\right)={\mathrm{U\Sigma V}}^T$$
takes the first few left singular vectors as a set of bases of the subspace, which we call POD bases.

Higher-order tensor processing for parametric dynamical systems

Now let's deal with it, assuming that the above-mentioned parameter is not one, but many, how should we do SVD decomposition, and how should we get the POD basis?

symbol definition

We might as well assume that the above parameters are located in a high-dimensional box, and the dimension is $D$，
$$\mathcal{A}=\underset{i=1}{\overset{D}{⨂}}\left[a_i^{\min },a_i^{\max }\right]$$
We can cut this high-dimensional box several times in all directions to get some node sets,
$$\hat{\mathcal{A}}=\left\{\hat{a}={\left({\hat{a}}_1,\dots ,{\hat{a}}_D\right)}^T:{\hat{a}}_i\in {\left\{{\hat{a}}_i^j\right\}}_{j=1,\dots ,n_i},i=1,\dots ,D\right\}$$
The total number of nodes is,
$$K=\prod _{i=1}^D{n}_i$$
Each point is a parameter, and each parameter can be taken as snapshots to obtain a matrix. All these matrices are organized together to form a high-order tensor, ( Φ ) : , i
$$(F{)}_{:,i_1,\dots ,i_D,k}={\varphi }_k\left({\hat{a}}_1^{i_1},\dots ,{\hat{a}}_D^{i_D}\right)$$
The definition of the Frobenius norm of the tensor is similar to that of the matrix,
$$\Vert \mathbf{\Phi }{\Vert }_F:={\left(\sum _{j=1}^M\sum _{i_1=1}^{n_1}\dots \sum _{i_D=1}^{n_D}\sum _{k=1}^N{Phi}_{j,i_1,\dots ,i_D,k}^2\right)}^{1/2}$$
Under this definition, we have to assume a low-order approximation of the tensor$\widetilde{Phi}$Define,
$$\Vert \widetilde{Phi}-\mathbf{\Phi }{\Vert }_F\le \tilde{e}\Vert \mathbf{\Phi }{\Vert }_F$$
For a tensor, we can define the matrix multiplication vector operation in a certain direction as follows:
$${\left(Ps{\times }_k\mathbf{a}\right)}_{j_1,\dots ,j_{k-1},j_{k+1},\dots ,j_m}=\sum _{j_k=1}^{N_k}{Ps}_{j_1,\dots ,j_m}{a}_{j_k}$$

$ \times_{k}$ represents the kkthPerform linear combination compression in$k\; directions.$

We can define the operator ${\mathbf{e}}^i(\hat{a})={\left(e_1^i(\hat{a}),\dots ,e_{n_i}^i(\hat{a})\right)}^T\in {\mathbb{R}}^{n_i},i=1,\dots ,D$ 为，
$$e_j^i(\hat{a})=\{\begin{array}{ll}1& \text{ if }{\hat{a}}_i={\hat{a}}_i^j\\ 0& \text{ otherwise }\end{array},j=1,\dots ,n_i$$
In fact, $e^i$ means that for$\hat{a}={\left({\hat{a}}_1,\dots ,{\hat{a}}_D\right)}^T$ 's$i$ dimension, what is its value, then$e^{The\; first\; few\; positions\; of\; i}$ are 1, and the others are zero.
With the above symbol definition, we can extract the snapshot on each parameter node as follows:
$${Phi}_e(\hat{a})=Phi{\times }_2{\mathbf{e}}^1(\hat{a}){\times }_3{e}^2(\hat{a})\cdots {\times }_{D+1}{e}^D(\hat{a})\in {\mathbb{R}}^{M\times N}$$
Similarly, for the approximation tensor, we can also extract in the same way,
$${\widetilde{Phi}}_e(\hat{a})=\widetilde{Phi}{\times }_2{\mathbf{e}}^1(\hat{a}){\times }_3{e}^2(\hat{a})\cdots {\times }_{D+1}{e}^D(\hat{a})\in {\mathbb{R}}^{M\times N}$$
若 $\tilde{N}=\mathrm{rank}\left({\widetilde{Phi}}_e\left(\hat{a}\right)\right)$ ,then in other words,
$$\sum _{i=1}^N{\Vert {\varphi }_i-\sum _{j=1}^{\tilde{N}}⟨{\varphi }_i,{\mathbf{z}}_j⟩z_j\Vert }_{{\ell }^2}^2\le {\tilde{e}}^2\Vert \mathbf{\Phi }{\Vert }_F^2$$

Similar to ei \mathbf{e}^{i} above${\mathbf{e}}^{The\; way\; i}$ is defined, we can also define it as parameter box$Lagrangian\; interpolation\; node\; basis\; functions\; in\; i$ dimensions,
$${\mathbf{e}}^i:a\to {\mathbb{R}}^{n_i},i=1,\dots ,D$$
e j i ( α ) = { ∏ m = 1 , m ≠ k p ( α ^ i i m − α i ) / ∏ m = 1 , m ≠ k p ( α ^ i i m − α ^ i j ) ,  if  j = i k ∈ { i 1 , … , i p } , 0 ,  otherwise  , e_{j}^{i}(\boldsymbol{\alpha})= \begin{cases}\prod_{\substack{m=1, m \neq k}}^{p}\left(\widehat{\alpha}_{i}^{i_{m}}-\alpha_{i}\right) / \prod_{\substack{m=1, m \neq k}}^{p}\left(\widehat{\alpha}_{i}^{i_{m}}-\widehat{\alpha}_{i}^{j}\right), & \text { if } j=i_{k} \in\left\{i_{1}, \ldots, i_{p}\right\}, \\ 0, & \text { otherwise },\end{cases} eji​( a )={ ∏m=1,m=k​p​(a iim​​−ai​)/∏m=1,m=k​p​(a iim​​−a ij​),0,​ if j=ik​∈{ i1​,…,ip​}, otherwise ,​
According to the definition of polynomial interpolation, we can say that a function is interpolated on an interpolation node as follows,
$$g\left(a_i\right)\approx \sum _{j=1}^{n_i}{e}_j^i\left(a\right)g\left({\hat{a}}_i^j\right),i=1,\dots ,D$$
Under these definitions, for high-dimensional tensors, we can interpolate the dimensions corresponding to the parameters into a quantity. This quantity is a function, Φ ~ e ( α ) = Φ ~ × 2 e
$${\widetilde{Phi}}_e\left(a\right)=\widetilde{Phi}{\times }_2{\mathbf{e}}^1\left(a\right){\times }_3{e}^2\left(a\right)\cdots {\times }_{D+1}{e}^D\left(a\right)\in {\mathbb{R}}^{M\times N}$$
Of course, if we only partially step on each dimension for the parameter box, for example,
A ^ p : = { α ^ = ( α ^ 1 , … , α ^ D ) T : α ^ i ∈ { α ^ ij } j ∈ { i 1 , … , ip } , i = 1 , … , D } ⊂ A ^ \widehat{\mathcal{A}}_{p}:=\left\{\widehat{\ boldsymbol{\alpha}}=\left(\widehat{\alpha}_{1}, \ldots, \widehat{\alpha}_{D}\right)^{T}: \widehat{\alpha}_{ i} \in\left\{\widehat{\alpha}_{i}^{j}\right\}_{j \in\left\{i_{1}, \ldots, i_{p}\right\ }}, i=1, \ldots, D\right\} \subset \widehat{\mathcal{A}}A p​:={ a =(a 1​,…,a D​)T:a i​∈{ a ij​}j∈{ i1​,…,ip​}​,i=1,…,D}⊂A
There can also be similar results. After these preparations, let me introduce the selection method of the POD base under the three formats. No more details, just try to make the algorithm clear, with the goal of being able to program. In fact, the core ideas of these three methods are the same, that is, we first decompose the tensor snapshot in some ways, then interpolate the meta-tensor in the parameter part to obtain the center matrix, and finally calculate the center Do a singular value decomposition of the matrix. Select the first nn of the left singular vector after decomposition$n$ number.

CP-TROM

Canonical polyadic (CP) decomposition is the most basic tensor decomposition method. The tensor approximation under this decomposition formulation is simply the following process.

Offline stage:
First of all, we must solve the problem through some traditional numerical methods, and get a tensor snapshot of the solution (for the sake of Sinochem, it may be called a snapshot below). For this snapshot tensor, given the target canonical rank $R$ , we use the Alternating Least Squares algorithm (ALS) to get its low-rank CP decomposition,
$$Phi\approx \widetilde{Phi}=\sum _{r=1}^R{\mathbf{u}}^r\circ p_1^r\circ \cdots \circ p_D^r\circ {\mathbf{v}}^r$$

ALS 算法参考：Harshman R A. Foundations of the PARAFAC procedure: Models and conditions for an" explanatory" multimodal factor analysis[J]. 1970.

Secondly, we get the decomposed ${\mathbf{u}}^r$和${\mathbf{V}}^{If\; r}$ is a continuous variable,
$$\hat{\mathrm{U}}=\left[{\mathbf{u}}^1,\dots ,u^R\right]\in {\mathbb{R}}^{M\times R},\hat{V}=\left[{\mathbf{v}}^1,\dots ,{\mathbf{v}}^R\right]\in R^{N\times R}$$
Finally, we decompose the two matrices obtained by QR respectively,
$$\hat{\mathrm{U}}={\mathrm{UR}}_U,\hat{\mathrm{V}}={\mathrm{VR}}_V$$

Online stage:
Specify a reduced space dimension $n\le R$ , and given a set of parameter vectors$a\in A.$ _
First, we can put the tensor in each dimension at the point$a\in Do\; interpolation\; at\; A$ , for CP decomposition, in fact, the node basis functions on each dimension are in$a\in A$ takes a value, and makes an inner product with the elementary tensor, turns into some numbers and makes a product, that is,
$${\widetilde{Phi}}_e\left(a\right)=\sum _{r=1}^R{s}_r{\mathbf{u}}^r\circ {\mathbf{v}}^r\in {\mathbb{R}}^{M\times N},\text{ with }{s}_r=\prod _{i=1}^D⟨p_i^r,{\mathbf{e}}^i\left(\alpha \right)⟩\in R$$
Next, put these$s_r$Spanning a diagonal matrix $S\left(a\right)=\mathrm{diag}\left(s_1,\dots ,s_R\right)$ , and construct the core Matrix as follows,
$$C\left(a\right)={\mathrm{R}}_{U}S\left(a\right)R_V^T$$
Next, perform singular value decomposition on the core Matrix,
$$C\left(a\right)={\mathrm{U}}_c{S}_c{\mathrm{V}}_c^T$$
Finally, we get the singular value decomposition under this online parameter,
$${\widetilde{Phi}}_e\left(a\right)=UC\left(a\right)V^T=\left({UU}_c\right)S_c{\left({\mathrm{VV}}_c\right)}^T$$
那么，${\left\{b_i\left(a\right)\right\}}_{i=1}^n\subset {\mathbb{R}}^R$ can be directly taken as${\mathrm{U}}_c$ex $n$ columns.

HOSVD-TROM

Compared with CP decomposition, higher order singular value decomposition (higher order SVD, HOSVD) can have a guaranteed minimum property. That is,
$$\Vert \mathbf{\Phi }-\widetilde{Phi}\Vert \le \sqrt{D+2}\Vert Phi-{Phi}^{\mathrm{opt}}\Vert \text{ and }\Vert \Phi -\widetilde{Phi}\Vert \le {\left(\sum _{i=1}^{D+1}{D}_i^2\right)}^{\frac{1}{2}}$$
Offline phase:
we are given a snapshot of the tensor and the target precision $\tilde{e}$.
The offline stage is to calculate the HOSVD decomposition to obtain the decomposition matrix and coefficient (core tensor). There are many algorithms that can be used, such as:
De Lathauwer L, De Moor B, Vandewalle J. A multilinear singular value decomposition[J]. SIAM journal on Matrix Analysis and Applications, 2000, 21(4): 1253-1278.

把得到的分量都拼一拼，
$$\begin{array}{rl}\mathrm{U}& =\left[{\mathbf{u}}^1,\dots ,{\mathbf{u}}^{\tilde{M}}\right]\in {\mathbb{R}}^{M\times \tilde{M}},\mathrm{V}=\left[{\mathbf{v}}^1,\dots ,{\mathbf{v}}^{\tilde{N}}\right]\in R^{N\times \tilde{N}},\\ {\mathrm{S}}_i& ={\left[p_i^1,\dots ,p_i^{{\tilde{n}}_i}\right]}^T\in {\mathbb{R}}^{{\tilde{n}}_i\times n_i},i=1,\dots ,D.\end{array}$$

Note that here ${\mathrm{S}}_i$The expression has a transpose.

Online stage:
given the reduced space dimension n ≤ min ⁡ { M ~ , N ~ } n \leq \min \{\widetilde{M}, \widetilde{N}\}n≤min{ M ,N } , and the online parameter$a\in A.$ _
First, like CP, some interpolation expression is still performed on the parameter components to obtain the core Matrix (hereinafter referred to as the central matrix),
$${\mathrm{C}}_e\left(a\right)=C{\times }_2\left({\mathrm{S}}_1{\mathrm{e}}^1\left(a\right)\right){\times }_3\left(S{}_2{e}^2\left(a\right)\right)\cdots {\times }_{D+1}\left({\mathrm{S}}_D{\mathbf{e}}^D\left(a\right)\right)\in {\mathbb{R}}^{\tilde{M}\times \tilde{N}}$$
Then in fact,
$${\widetilde{Phi}}_e\left(a\right)={\mathrm{UC}}_e\left(a\right)V^T$$
Secondly, do SVD decomposition on the central matrix,
$${\mathrm{C}}_e\left(a\right)={\mathrm{U}}_c{S}_c{\mathrm{V}}_c^T$$
Finally, the orthogonal decomposition of the matrix after parameter dimension interpolation approximation is obtained ( $U$和$V$ is orthogonal at the beginning, never mind),
$${\widetilde{Phi}}_e\left(a\right)=\left({UU}_c\right)S_c{\left({\mathrm{VV}}_c\right)}^T$$
这时，令 ${\left\{b_i\left(a\right)\right\}}_{i=1}^n\subset {\mathbb{R}}^{\tilde{M}}$ 是 ${\mathrm{U}}_c$in front of $n$ columns will do.

TT DRUM

Tensor Train decomposition is a special tree-based decomposition.

Offline stage:
Given a snapshot tensor and target precision.
Use the algorithm mentioned in the paper
Oseledets I V. Tensor-train decomposition[J]. SIAM Journal on Scientific Computing, 2011, 33(5): 2295-2317.
For a given target accuracy, do the TT decomposition of the tensor snapshot ,
$$Phi\approx \widetilde{Phi}=\sum _{j_1=1}^{{\tilde{r}}_1}\cdots \sum _{j_{D+1}=1}^{{\tilde{r}}_{D+1}}{\mathbf{u}}^{j_1}\circ p_1^{j_1,j_2}\circ \cdots \circ p_D^{j_D,j_{D+1}}\circ {\mathbf{v}}^{j_{D+1}}$$
Still do some splicing of the obtained quantities,
$$\mathrm{U}=\left[{\mathbf{u}}^1,\dots ,{\mathbf{u}}^{{\tilde{r}}_1}\right]\in {\mathbb{R}}^{M\times {\tilde{r}}_1},\mathrm{V}=\left[{\mathbf{v}}^1,\dots ,{\mathbf{v}}^{{\tilde{r}}_{D+1}}\right]\in R^{N\times {\tilde{r}}_{D+1}}$$
$${\left({\mathbf{S}}_i\right)}_{jkq}={\left(p_i^{jq}\right)}_k,j=1,\dots ,{\tilde{r}}_i,k=1,\dots ,n_i,q=1,\dots ,{\tilde{r}}_{i+1}$$
Note that each parameter component here can be spliced ​​into a three-dimensional tensor.

Also, for V \mathrm{V}Put the modulo of$V$
$${\mathrm{W}}_c=\mathrm{diag}\left(\Vert {\mathbf{v}}^1\Vert ,\dots ,\Vert v^{{\tilde{r}}_{D+1}}\Vert \right)\in {\mathbb{R}}^{{\tilde{r}}_{D+1}\times {\tilde{r}}_{D+1}}$$

On-line phase:
given the dimensionality of the reduced space and a set of parameters on-line.
The first step is to interpolate the parameter tensor and do multiplication to get the center matrix:
$${\mathrm{C}}_e\left(a\right)=\prod _{i=1}^D\left({\mathbf{S}}_i{\times }_2{\mathbf{e}}^i\left(\alpha \right)\right)$$
则
$${\widetilde{Phi}}_e\left(a\right)={\mathrm{UC}}_e\left(a\right)V^{The}$$
second step is to perform singular value decomposition on the scaled center matrix:
$${\mathrm{C}}_e\left(a\right)W_c={\mathrm{U}}_c{S}_c{\mathrm{V}}_c^T$$
The third step is to obtain the singular value decomposition of the matrix after interpolation:
$${\widetilde{Phi}}_e\left(a\right)={\mathrm{UC}}_e\left(a\right)W_c{\mathrm{W}}_c^{-1}{\mathrm{V}}^T=\left({UU}_c\right)S_c{\left({\mathrm{VW}}_c^{-1}{V}_c\right)}^T.The$$
fourth step, even if POD depends on${\left\{b_i\left(a\right)\right\}}_{i=1}^n\subset {\mathbb{R}}^{{\tilde{r}}_1}$, which is ${\mathrm{U}}_c$ex $n$ columns.

General parameter sampling

The above methods are given a snapshot of the grid-based parameter space, which is not very economical, and will cause trouble when the parameter area is not a box. In this case, we can express the online parameters as a linear combination of existing parameters,
$$a=\sum _{j=1}^K{a}_j{\hat{a}}_j,e\left(a\right)={\left(a_1,\dots ,a_K\right)}^T$$
$e$ is the extracted coefficient,
$$\mathbf{e}:a\to {\mathbb{R}}^{KWe}$$
assume that the smooth function satisfies some linearity,
$$g\left(a\right)\approx \sum _{j=1}^K{a}_{j}g\left({\hat{a}}_j\right)$$
Then, for each offline parameter, we can construct a third-order tensor,
$$(F{)}_{}=u_i\left(t_k,{\hat{a}}_j\right),i=1,\dots ,M,j=1,\dots ,K,k=1,\dots ,N$$
Then, a new parameter is introduced, which can be expressed as a linear combination of existing parameters. The corresponding snapshot is just a linear combination of existing snapshots:
$${\widetilde{Phi}}_e\left(a\right)=\widetilde{Phi}{\times }_{2}e\left(a\right)$$

Is this reasonable?

Simply put, $e\left(\alpha \right)$ has changed from the original interpolation function to a linear combination function, and other CP, HOSVD, and TT are done in the same way.

references

1、Model reduction and approximation: theory and algorithms[M]. Society for Industrial and Applied Mathematics, 2017.
2、Mamonov A V, Olshanskii M A. Interpolatory tensorial reduced order models for parametric dynamical systems[J]. Computer Methods in Applied Mechanics and Engineering, 2022, 397: 115122.