Feb.27~image super-resolution reconstruction， paper reading

Multi-frame image super-resolution reconstruction via low-rank fusion combined with sparse coding

这篇文章看起来和2月初读的一篇关于图像去噪是自相矛盾的，这篇文章可以方法成立的条件是，HR和LR图片用字典表示时候的协方差系数矩阵是类似的，也就是那个 $\alpha$ 在两种情况下是一样的。在求解中也应用到了从三个方向特征提取的技巧，还有K-SVD方法。

（思考：那么在这里的特征提取是否可以接入神经网络，模型自己训练，自己提取特征呢？）这里面提取特征是在第二部，是为了提取出特征之后这些特征向量组成训练集，再用K-SVD方法提取出字典。

之前的文章中，两种情况下的 $\alpha$ 是有偏差的，之后在模型优化中，专门加入了一项，表示两个协方差的距离，接着进行一系列的估计，迭代，然后求解。

According to the number of input LR images, image SR can be divided into single -frame and multi - frame.

Single-frame SR refers to restore a HR image from a LR image, and multi-frame SR means to recover a HR image from LR image sequence.

2. Low rank fusion and sparse coding

2.1 Low rank fusion

A data matrix $M\in \Bbb{R}^{n\times m}$ contains structural information as well as noise. Then we can decompose M as
M = L + S
where L is low-rank matrix(contains the internal structure information which are linearly related.)
S is sparse (noise is sparse)
RPCA(robust principle component analysis) method model used to optimize above problem:
$\underset{L,S}{min}~rank(L)+\lambda||S||_0, s.t. M = L +S~~~(1)$
$\lambda$ is a balance parameter.
(1) is a non-convex problem and it can be replaced by (2)
$\underset{L,S}{min}~||L||_*+\lambda||S||_1, s.t. M = L +S~~~(2)$
Augmented Lagrange Multiplier (ALM) algorithm is usually used to solve (2)

Supposing there are N frame registered images, then they are converted into a column vector and stored in a matrix by column. Now the matrix M={ $M_1,M_2,⋯M_N$} has low-rank characteristics.
Next we decompose the matrix by low-rank and sparse decomposition.
Finally the decomposed low-rank part L = { $L_1, L_2, ⋯L_N$} is average fused to get the final fusion LR image L∗ =( $L_1 +L_2 +⋯+L_N$)/N,

2.2 Sparse coding

After down sampling S and fuzzy B, HR image X is degenerated into LR image Y:
$Y = SBX ~~~~~~~~~~(3)$
Where SB = L, Y = LX.
If y is an image patch taken from Y ,x is an image patch taken from X which is in the same location with y.
The sparse representation model can be represented as following:
$min ||\alpha||_1, s.t. x= D_h\alpha~~~~~~(4)$
$D_h\in\Bbb{R}^{n\times K}$(K>n) is a HR over-complete dictionary.
According to (3)(4), so y can be expressed as following:
$y = LD_h\alpha = D_l \alpha~~~~~~~~~~~~~~(5)$
Where $D_l = LD_h, D_l\in \Bbb{R}^{m\times K}$(K>m) is a LR over-complete dictionary.
So it can be clarified that HR and LR image patches have the same sparse representation coefficient. ( $\alpha_x = \alpha_y$)
Based on a pair of HR and LR dictionaries { $D_h, D_l$}, we are able to rebuild the correspond- ing HR image patch as long as we acquire sparse representation coefficient of the LR image patch.

3. Multi-frame image SR via low-rank fusion combines with sparse coding

3.1 method

The Multi-frame SR using low-rank fusion combines with sparse coding includes three steps: image registration and low-rank fusion, { $D_h, D_l$} dictionary training and SR reconstruction, as shown in Fig. 3.
In image registration and low-rank fusion phase, SURF and RANSAC algorithm are used to image registration. Then registered images are decomposed into the low-rank images and the sparse images, and the low-rank images are fused into a LR image.
In { $D_h, D_l$} dictionary training phase, patch haar wavelet transform is used to get image patches characteristic vectors, and characteristic vectors constitute the joint training set. K-SVD algorithm is applied to train joint training set to obtain the { $D_h, D_l$}.
In SR reconstruction phase, after computing sparse representation coefficient $\alpha$ of LR patch, the HR patch can be obtained from the coefficient $\alpha$ multiplied by the HR dictionary $D_h$.

扫描二维码关注公众号，回复： 5496279 查看本文章

The END of Method of this paper.