[CV] NSCT: Nonsubsampled Contourlet transformation algorithm and matlab implementation

###Date: 2018.4.30

===================================================

1. Introduction to the Nonsubsampled Contourlet transform algorithm:

    The sparse representation of signals is the basis of many signal processing and applications. In 2004, Minh N Do and Martin Vetterli proposed a mathematical tool that can better represent two-dimensional signals--Contourlet transform. Contourlet uses the Pyramid Directional Filter Bank (PDFB) to decompose the image into directional subbands at different scales. According to the structure of PDFB, PDFB is a superposition of a Laplacian Pyramid (LP) filter and a directional filter bank. Experiments show that the performance of Contourlet transform in image noise reduction, texture and shape feature extraction is significantly improved than 2-D discrete wavelet transform.
To obtain translation invariance, the Nonsubsampled Contourlet Transform (NSCT) used in this chapter is a transform based on the Nonsubsampled Pyramid (NSP) and the Nonsubsampled Directional Filter (NSDFB). First, the input image is decomposed into a tower shape by NSP, which is decomposed into two parts: high-pass and low-pass. Then, the high-frequency sub-band is decomposed into multiple directional sub-bands by NSDFB, and the low-frequency part continues to be decomposed as above. NSCT is a novel translation-invariant, multi-scale, multi-directional fast transformation.
1. Nonsubsampled Pyramid (NSP):
Nonsubsampled Pyramid (NSP) and Contourlet's Laplacian Pyramid (LP) have different multi-scale analysis characteristics. The image is multi-scale decomposed by Nonsubsampled Pyramid (NSP), NSP removes upsampling and downsampling, reduces the distortion of sampling in the filter, and obtains translation invariance. NSP is a multi-scale analysis of NSCT with a translation-invariant filtering structure, which can obtain the same multi-scale analysis characteristics as LP decomposition. Figure 2.4(a) is divided into three scales.
2. Nonsubsampled Directional Filter (NSDFB)
The Nonsubsampled Directional Filter (NSDFB) is a two-channel filter that combines singular points distributed in the same direction into the coefficients of the NSCT. The directional filter (DFB) was proposed by Bamberger and Smith. Through the decomposition of an l-layer tree structure, the signal is effectively divided into sub-bands, and the frequency band is divided into a wedge shape. Nonsubsampled DFB (NSDFB) is non-sampling, which reduces the distortion of sampling in the filter and obtains translation invariance. And the size of the direction sub-map at each scale is the same as the original image, and the Contourlet transforms the sum of all sub-bands to be equal to the original image. NSCT has more details preserved and transform coefficients are redundant. Figure 2.4(b) is a segmentation diagram of the image frequency domain at three scales, in which the number of directional subbands at each scale is increased by 2 times, so that the number of directional subbands at scales 1, 2, and 3 is 2, respectively. 4, 8.

 

NSCT internal variables:

The following lists the main variables and structure definitions in the program:

Image information structure :

typedef struct        

{ int high; //-----------------------------------------image height

   int width; //---------------------------------------image width

  double max; //--------------------------------------Maximum pixel value

  double min; //---------------------------------------minimum pixel value

  double *data; //--------------------------------------pixel data area

} ImgData;  

 

NSCT transform space structure: (used to store basic information such as NSCT transform filter, scale, self-contained space, etc.)

typedef struct

{ unsigned int *nlevels;//The number of subband directions   must be an even number   sublevels[0] is the number of the highest level direction

sublevels[4] is invalid when the lowest level direction number    is negative

  unsigned int clevels; //-----------------scale levels

  ImgData ***high;//------------------------high-pass subband area

  ImgData low;//---------------------------- Low pass subband area

  ImgData H1;//------------------------------high-pass filter bank

  ImgData H2; //------------------------------high pass filter bank

  ImgData G1; //------------------------------low pass filter bank

  ImgData G2; //----------------------------- low pass filter bank

  ImgData ***filters_dec; //-----------------Decomposition direction filter bank

  ImgData ***filters_rec; //-----------------Reconstruction direction filter bank

  BOOL sign;

}nsct_t;

Scale and decomposition layer filter:

const int dlevels[3]={3,3,3};//---------------The scale is three, and the number of scale directions is

const int Qunx[4]={0,0,0,0};//--------------------------------------Q plum sampling matrix

BMGImageStruct SourceBmp;//-------------------------------------- Bitmap operation structure

int pfilter_type,dfilter_type; //Filter type selection , both default to ,'maxflat'&'dmaxflat7'filter

int pralevels; //------------The number of image pyramid layers (cannot be greater than 4, the scale is within 4 enough to meet general image processing needs)

nsct_t nsct; //nsct-------Data calculation area

int shift[2]; //-------------------- delay component

2.3.6  NSCT internal functions:

The following lists the main function definitions in the program:

void ImgDataM_Var_3m(ImgData *x,float *mean,float *var,float *mom);

//---------------------------------Calculate the mean and third-order central moment of the nsct subband coefficients

void ImgDataIFFT_2D(complex<double> * pCFData, complex<double> * pCTData, int nWidth, int nHeight);

//---------------------------------2D-FFT inverse transform

void ImgDataFFT_2D(complex<double> * pCTData, int nWidth, int nHeight, complex<double> * pCFData);

//---------------------------------2D-FFT inverse transform

void ImgDataFConv2(ImgData *out,ImgData *img,ImgData *mask);

//---------------------------------Frequency domain convolution function

void Nsct_Extend2(ImgData *in,int ru,int rd,int cl,int cr,ImgData *out,const int flag=1);

//---------------------------------nsct mirror extension function

void Nsct_Zconv2(ImgData *in,ImgData *out,ImgData *filter,const int *mup);

//------------------------------------------------ Time-domain filter convolution function (fast algorithm)

void Nsct_Efilter2(ImgData *in,ImgData *out,ImgData *filter);

//--------------------------------- 2D filtering for edge extension

void Nsct_Nssfbdec(ImgData * input,ImgData * out1,ImgData * out2,ImgData * filter1,ImgData * filter2,const int * Qx=Qunx);

//--------------------------------- 2-channel unsampled subband decomposition

void Nsct_Mirror2(ImgData *tomirror);

//---------------------------------Mirror transformation

void ImgDataConv2(ImgData *out,ImgData *img,ImgData *mask,ImgData *imgunext);

//------------------------------------------------ Correlation convolution

void Nsct_Upsample2df(ImgData *h0,int lev,ImgData *ho);

//----------------------------------频域2D上采样

void Nsct_Symext(ImgData *imglow,ImgData *upedHi,ImgData *pp );

//----------------------------------2D平衡矩阵生成

void Nsct_Atrousc(ImgData * out,ImgData * signal,ImgData *filter,int (*matrix)[2]);

//----------------------------------包含抽取的滤波器卷积

void Nsct_Nsdfbdec(ImgData *imghigh,int currentlevel);

//----------------------------------非抽取方向子带分解

void Nsct_iTtransform();

//----------------------------------反变换主函数

void Nsct_New(HBITMAP hBitmap,const int *Nlevels=dlevels,const unsigned int Level=3,const int dfilter=1,const int pfilter=1);

//-----------------------------用来初始化原始目标图像，申请nsct计算空间和存储空间

void Nsct_Transform();

//-----------------------------变换主函数

void Nsct_NsfbDec(ImgData *imglow,ImgData *imghigh,int i);

//-----------------------------nsct图像金字塔分解

void NsctDelete();

//-----------------------------nsct析构

void ImgDataFreeMemery(ImgData  *image);

//-----------------------------内存空间释放

BOOL ImgDataAllocMemery(ImgData * image);

//-----------------------------内存空间申请

一个示例程序，和显示的样图：

2、NSCT用于图像融合的应用（matlab实现）

%% 实现红外与可见光图像融合NSTC
%计时开始%
tic;%计时  
path(path,'nsct_toolbox');%加载matlab自带的nsct工具箱
path(path,'transform');%加载RGB和IHS转换用的函数的文件夹

%% 1、输入图像和初始化参数
kk=1.5;%参数
[input_image_TV_int_GIF, input_image_TV_int_MAP] = imread('test_kejianguang.gif' );%输入gif格式的可见光图像%
input_image_TV_int_RGB = uint8(256*ind2rgb(input_image_TV_int_GIF, input_image_TV_int_MAP));%将gif格式数据转换为RGB格式数据

[input_image_IR_int_GIF, input_image_IR_int_MAP] = imread('test_hongwai.gif' ); %输入gif格式的红外图像%
input_image_IR_int_RGB = uint8(256*ind2rgb(input_image_IR_int_GIF, input_image_IR_int_MAP));%将gif格式数据转换为RGB格式数据%

[Ny,Nx] = size(input_image_TV_int_RGB);%求取输入图像的宽和高

%将可见光图像的格式转换为IHS格式
input_image_TV_int_IHS = rgb2ihs( input_image_TV_int_RGB );%IHS
input_image_TV_IHS = double(input_image_TV_int_IHS);  %将输入图像的数据类型转换为双精度数据类型，便于处理
input_image_TV_I =  256*input_image_TV_IHS(:,:,3);  %将输入图像的I分量提取出来以进行NSCT分解

%将红外图像的格式转换为gray格式，灰度格式
input_image_IR_int = rgb2gray( input_image_IR_int_RGB );%灰度化
input_image_IR= double(input_image_IR_int); %将输入图像的数据类型转换为双精度数据类型

input_image_TV_S =  input_image_TV_IHS(:,:,2);
input_image_TV_H =  input_image_TV_IHS(:,:,1);

figure;
subplot(1,2,1);
imshow(uint8(input_image_TV_I));title('可见光图像亮度分量I') %显示输入可见光图像的I分量
subplot(1,2,2);
imshow(uint8(input_image_IR));title('红外图像灰度化') %显示输入红外图像的灰度图像

%金字塔表示参数%
Nsc = ceil(log2(min(Ny,Nx)) - 7);    %分解尺度的数量 (自适应于图像尺寸)%
Nor = 8;				             %每级分解的方向数%

%% 2、NSCT子带分解
%初始化NSCT子带分解参数%
pfilter = 'maxflat' ;  %金字塔滤波器
dfilter = 'dmaxflat7' ; %方向滤波器

nlevels=zeros(1,Nsc+1);
for i=1:Nsc+1
    nlevels(i)=log2(Nor); %初始化分解尺度%
end

%NSCT分解
coeffs_TV_int = nsctdec( input_image_TV_I, nlevels, dfilter, pfilter ); %分解可见光图像
coeffs_IR_int = nsctdec( input_image_IR, nlevels, dfilter, pfilter ); %分解红外图像

%% 3、图像融合
%3.1 低频分量采用自适应模糊逻辑算法
u=mean2(coeffs_IR_int{1});%均值
s=std2(coeffs_IR_int{1});%方差
[lx,ly]=size(coeffs_IR_int{1});
for x=1:lx
    for y=1:ly
        Nb=exp(-(coeffs_IR_int{1}(x,y)-u)^2/(kk*(2*s)^2));
        Nt=1-Nb;
        coeffs_rec{1}(x,y)=Nt*coeffs_IR_int{1}(x,y)+Nb*coeffs_TV_int{1}(x,y);
    end
end
%3.2 高频分量采用模值取大的方法
for i=2:Nsc+2
    for j=1:Nor
        band=abs(coeffs_TV_int{i}{j})-abs(coeffs_IR_int{i}{j});
        band_coffes1=double(im2bw(band,0));
        band_coffes2=ones(size(band))-band_coffes1;
        coeffs_rec{i}{j} = coeffs_TV_int{i}{j}.*band_coffes1+coeffs_IR_int{i}{j}.*band_coffes2;  
    end
end

%% 4. NSCT subband reconstruction

out_image_end_I = nsctrec( coeffs_rec, dfilter, pfilter ) ;% reconstructed grayscale image%  
figure;
imshow(uint8(out_image_end_I));title('Fused image luminance component');
out_image_end=zeros(Ny,Nx,3); %Initialize the color image output matrix
out_image_end=cat(3,input_image_TV_H,input_image_TV_S,out_image_end_I/256);                      
out_image_end_RGB=256*ihs2rgb(double(out_image_end)); % Convert the output color image to RGB format              

figure;
imshow(uint8(out_image_end_RGB));% Display the fused color image%
title('Fused color image')
imwrite(uint8(out_image_end_RGB),'fusion_image.bmp','bmp');
toc; % timer expired

Experimental result graph:

Reference: https://www.cnblogs.com/molakejin/p/5918976.html