Matlab realizes CCA multi-mode feature fusion

Currently we have a task to search in a target database with 130 IDs. The data used for retrieval belongs to one of 130 categories, with 1 to 3 in each category, and a total of 224 retrieved data. (That is, choose one from 224 data, and calculate the similarity with 130 data), and the accuracy rate is calculated based on the top-1 of 130.

We have feature extraction methods F ( X ), G ( X ) for this type of data X , and the extracted features are denoted as f and g.

Only the feature  f  can achieve 91% accuracy, and only the feature g can achieve 84% accuracy.

We hope to combine these two features to make the accuracy rate exceed 91%. Simply stringing together the eigenvalues ​​is only 85%.

 

Canonical Correlation Analysis (CCA) can be used to do this.

Code  https://github.com/mhaghighat/ccaFuse

 

The experimental data storage method

Index_Image
    |---------id(1)
                |--------xxx.jpg
    .
    .
    .
    |---------id(130)
Index_Feature
    |---------id(1)
                |--------xxx.f
                |--------xxx.g
    .
    .
    .
    |---------id(130)
Database_Image
    |---------id(1).jpg
    .
    .
    |---------id(130).jpg
Database_Feature
    |---------id(1).f
    |---------id(1).g
    .
    .
    
                

Both .f and .g files are feature value files in json format. To use json in matlab, you need to download a jsonlab library, refer to  https://blog.csdn.net/sophia_xw/article/details/70141208

The feature fusion code is as follows, the goal is to generate a fusion feature .cca file under the same level path of the corresponding feature file (such as xxx.f and xxx.g) of each data (such as xxx.jpg)

clear 
clc

addpath('D:\\Program Files\\MATLAB\\R2016a\\bin\\my_workshop\\jsonlab-1.5');

trainX = [];
trainY = [];
testX = [];
testY = [];

train_json_save_path = {};
test_json_save_path = {};

for i = 1:130
    f_path = sprintf('D:\\Database_Feature\\id(%d).f', i);
    g_path = sprintf('D:\\Database_Feature\\id(%d).g', i);
    trainX = [trainX;loadjson(f_path)];
    trainY = [trainY;loadjson(g_path)];
    savepath = sprintf('D:\\Database_Feature\\id(%d).cca', i);
    train_json_save_path = [train_json_save_path, {savepath}];
end

for i = 1:130
    path = sprintf('D:\\Index_Image\\id(%d)',i);
    subdir  = dir(path);
    for j = 1:length(subdir)
        if(isequal(subdir(j).name, '.')||isequal(subdir(j).name, '..'))
            continue;
        end
        len = length(subdir(j).name);
        title_name = subdir(j).name(1:len-4)
        f_path = sprintf('D:\\Index_Feature\\id(%d)\\%s.f',i,title_name);
        g_path = sprintf('D:\\Index_Feature\\id(%d)\\%s.g',i,title_name);
        testX = [testX;loadjson(f_path)];
        testY = [testY;loadjson(g_path)];
        savepath = sprintf('D:\\Index_Feature\\id(%d)\\%s.cca',i,title_name);
        test_json_save_path = [test_json_save_path, {savepath}];
    end
end

[trainZ,testZ] = ccaFuse(trainX,trainY,testX,testY,'fusion');

for i = 1:130
    savepath = char(train_json_save_path(i));
    raw = trainZ(i,:);
    savejson('', raw, savepath);
end

for i = 1:224
    savepath = char(test_json_save_path(i));
    raw = testZ(i,:);
    savejson('', raw, savepath);
end

Finally, the accuracy of features using .cca can reach 96%.