Image quality evaluation (3): FSIM

Feature similarity index mersure (FSIM) uses feature similarity for quality evaluation. Because the human visual system (HVS) perceives images based on some low-level features, and the phase congruency (PC) feature can well describe the local structure. At the same time, because PC is relatively invariant to image changes, it is helpful to extract stable features in images, but sometimes image changes do affect the look and feel, so gradient features (gradient magnitude, GM) are needed to compensate. Two features of PC and GM are used in FSIM to complement each other.

Phase congruency feature extraction (phase congruency, PC)

The PC model assumes that the feature at the point where the Fourier component has the largest phase will be perceived. There are different methods for calculating the PC map, here is the method proposed by Kovesi. For a one-dimensional signal g(x), its PC is calculated as follows:

For the above orthogonal filter pair, a log-Gabor filter is used.

The PC calculation of two-dimensional grayscale images can first calculate one-dimensional PC in multiple directions, and then integrate the results through certain rules. The above one-dimensional log-Gabor filter can be transformed into two-dimensional by using a propagation function in a direction perpendicular to its direction, and the propagation function can use a Gaussian function.

The value of PC above is between 0 and 1.

Gradient feature extraction (GM)

The gradient can be calculated by many operators:

Calculate the gradient in the horizontal and vertical directions, and then calculate the final gradient;

FSIM calculation

For two images f1 and f2, the FSIM calculation can first obtain PC1, GM1, PC2, GM2 through the above steps.

First, the similarity of PC is calculated as follows;

The GM similarity is calculated as follows:

The similarity of the fusion of PC and GM is:

FSIM is calculated as follows:

Color image FSIM calculation

First, perform color space conversion on the color image:

The chromaticity similarity is calculated as follows:

The FSIM calculation for color images is as follows:

Experimental results

Database	Nonlinear fitting code	Results	FSIM	FSIMC
SROCC	KROCC	PLCC	RMSE	SROCC	KROCC	PLCC	RMSE
TID2013	NonlinearFittingTID2013	FSIMOnTID2013	0.8015	0.6289	0.8589	0.6349	0.8510	0.6665	0.8769	0.5959
TID2008	NonlinearFittingTID	FSIMOnTID2008	0.8805	0.6946	0.8738	0.6525	0.8840	0.6991	0.8762	0.6468
CSIQ	NonlinearFittingCSIQ	FSIMOnCSIQ	0.9242	0.7567	0.9120	0.1077	0.9310	0.7690	0.9192	0.1034
LIVE	NonlinearFittingLIVE	FSIMOnLIVE	0.9634	0.8337	0.9597	7.6780	0.9645	0.8363	0.9613	7.5296
IVC	NonlinearFittingIVC	FSIMOnIVC	0.9262	0.7564	0.9376	0.4236	0.9293	0.7636	0.9392	0.4183
Toyama-MICT	NonlinearFittingMICT	FSIMOnMICT	0.9059	0.7302	0.9078	0.5248	0.9067	0.7303	0.9075	0.5257
A57	NonlinearFittingA57	FSIMOnA57	0.9181	0.7639	0.9393	0.0844	0.9181	0.7639	0.9393	0.0844
WIQ	NonlinearFittingWIQ	FSIMOnWIQ	0.8006	0.6215	0.8546	11.8949	0.8006	0.6215	0.8546	11.8949
Weighted-Average

reference

https://sse.tongji.edu.cn/linzhang/IQA/FSIM/FSIM.htm

《FSIM: A Feature SIMilarity Index for Image Quality Assessment》

If you are interested, please follow the WeChat public account Video Coding