PCA (principal components analysis) is principal component analysis technology, also known as principal component analysis , which aims to use the idea of dimensionality reduction to convert multiple indicators into a few comprehensive indicators.
In statistics, Principal Component Analysis (PCA) is a technique for simplifying a data set. It is a linear transformation . This transformation transforms the data into a new coordinate system such that the first largest variance of any data projection is on the first coordinate (called the first principal component) and the second largest variance is on the second coordinate (second principal component ingredients), and so on. Principal component analysis is often used to reduce the dimensionality of a dataset while maintaining the features of the dataset that contribute the most to the variance. This is done by retaining low-order principal components and ignoring high-order principal components. Such low-level components tend to preserve the most important aspects of the data. However, this is not certain, it depends on the specific application.
% Assuming that PCA is now used to reduce two-dimensional data (that is, only x-axis and y-axis) to 1-dimensional (only 1 coordinate axis),
% After decentralization, we need to find a new coordinate axis that passes through the origin, so that the data fits the new axis with the highest degree
% That is, the variance of the projection of these data on this axis needs to be the largest
% so that when only this axis is kept and other coordinate axes are removed, the information of the data is kept the most
% and this new axis is called the principal component
% Of course, some data is more than 2-dimensional, such as pictures have multiple dimensions, which means that multiple principal components must be retained
%
% So how do you find these new axes (principal components)?
% Decentralize the data first
% Find the eigenvalues and eigenvectors of the covariance matrix of the data
% The eigenvalues of the covariance matrix represent the variance of the projection of the data on the new coordinate axis (principal component)
% The eigenvector of the covariance matrix represents the direction of the new coordinate axis (principal component) relative to the original coordinate axis
% The covariance matrix has several eigenvalues, and how many principal components the data has
% We arrange these eigenvalues in descending order, add them one by one from the largest eigenvalue, and calculate the ratio of the cumulative value to the sum of all eigenvalues
% This ratio is called the cumulative contribution. When the cumulative contribution reaches the threshold we set, stop accumulating, keep these accumulated eigenvalues and discard the rest
% to achieve the dimensionality reduction effect of PCA
%
% Detailed case can refer to the video:
https://www.bilibili.com/video/BV1E5411E71z/?spm_id_from=333.337.search-card.all.click
%%--------------------------The application of PCA technology in image recognition --------------- ------------------
% The pictures in this experiment come from the ORL standard face database.
%The database contains 40 faces, each face has 10 pictures of different angles or light, a total of 40*100=400 pictures, the picture size is 112*92=10304 pixels
%Select the first 7 pictures of each face, a total of 280 pictures, as a training set, and the last 3 pictures of each face, a total of 120 pictures as a test set
% Note: The size of the training picture and the test picture must be the same. If they are inconsistent, they must be processed before testing.
clc ; clear ; close all ;
%--------------------------- Matrix vectorization ------------------- ----------
In %matlab, images are stored in matrix form
%Rearrange each image 112*92 into a column vector of 10304*1, form a matrix X of 10304*280 by column, 280 is the number of training images
X = zeros(10304, 280); % Create an empty matrix with 10304*280 data all 0
for i = 1 : 40
for j = 1 : 7
image = imread( [ 'ORL_Faces\s' , num2str(i) , '\', num2str(j), '.pgm'] ) ; % read image
% Note that [ ] is added to ensure that this is a complete sentence. num2str(i) converts i from a number to a character form
%In order to save storage space, Matlab provides a special data type uint8 (8-bit unsigned integer) for images, and images stored in this way are called 8-bit images.
%When the function imread() is used, it will store the grayscale image into an 8-bit matrix, and when it is an RGB image, it will be stored in an 8-bit RGB matrix.
% Therefore, the data that Matlab reads into the image is uint8, and the value in Matlab is generally stored and operated in double type (64 bits).
% Therefore, the image must first be converted to double format before operation, that is, uint8 (0-255 range type) is converted to double (0-1 range type)
image = double(image); %double is used to convert data to double-precision floating-point numbers
for n = 1 : 92 % Each image matrix has 92 columns, note that the subscript of the matlab matrix starts from 1
for m = 1 : 112 % each column has 112 elements
X( m+(n-1)*112, j+(i-1)*7 ) = image( m, n ); % matrix rearrangement
end
end
end
end
%----------------------------Decentralization------------------ ------------
% Decentralization refers to moving the coordinate origin to the center of all image data
%Add the corresponding elements of each column of X and divide by the number of columns to get the mean u, then subtract u from each column of X to get the centered matrix Y
u = zeros( 10304, 1); % Create a column vector u with all 0s
for i = 1 : 280 %X has a total of 280 columns
u = u+X( : , i ) ; % add the corresponding elements of each column
end
u = u/280; % average value
Y = zeros(10304, 280);
for j = 1 : 280
Y( : , j ) = X( : , j ) - u; % Subtract u from each column of X
end
%-------------------- Find the eigenvalues and eigenvectors of the covariance matrix ------------------
% covariance matrix C = (Y * Y') / (n-1); where Y' represents the transposition of Y, n is the number of images
%You can calculate C first and then find its eigenvalues and eigenvectors, but the size of C is (10304×280)×(280×10304)=10304*10304, the dimension and calculation amount are too large
%So the singular value decomposition (SVD) method is introduced to solve the eigenvalues and eigenvectors of C by solving the eigenvalues and eigenvectors of Y'*Y (280*280)
%What is SVD? In simple terms,
%Matrix M can be decomposed into U ∑ V' multiplication of these three matrices (V' represents the transposition of V), that is, M=U * ∑ * V'
%The eigenvector of M*M' is U, and the eigenvector of M'*M is V
%The eigenvalue of M'*M=the eigenvalue of M*M'=the square of the elements on the diagonal of ∑
% covariance matrix C = (Y * Y') / (n-1); the eigenvalue is 1/ (n-1) of the eigenvalue of Y * Y', and the eigenvector is the same as the eigenvector of Y * Y'
%So you can first find the eigenvalue and eigenvector of Y'*Y, and then find the eigenvector of Y*Y'
C1 = Y'*Y;
[ V1, D1 ] = eig( C1 );
% returns the diagonal matrix D1 of the eigenvalues of C1 and the matrix V1 of the eigenvectors
The main diagonal element of %D1 is the eigenvalue of C1, and the column vector of V is the eigenvector of C1
D2 = diag( D1 ); % returns a column vector consisting of all main diagonal elements of D1
[D1_sort, D1_index] = sort( D2, 'descend');
% Return the elements in D2 to D1_sort in descending order, and D1_index is the row number of the sorted elements in the original matrix D2
V1_sort = V1( : , D1_index ); %Sort the eigenvectors so that the eigenvectors correspond to the sorted eigenvalues
D_sum = 0; % The initial cumulative value of the eigenvalue is 0
k = 0 ; % the number of accumulated eigenvalues
All_sum = sum(D1_sort); % Find the sum of all eigenvalues, the sum function is the sum
while( D_sum / All_sum < 0.9 ) %Set the threshold of cumulative contribution to 90%
k = k+1;
D_sum = D_sum + D1_sort( k, 1);
end
k = k-1;
% Find the eigenvector U corresponding to the first k eigenvalues of the covariance matrix C
U = zeros( 10304, k );
for i = 1 : k
U( : , i ) = (Y * V1_sort( : , i) ) / sqrt( D1_sort(i) ); %sqrt() find square root
end
% For convenience, we can also directly use the following functions to find the eigenvalues and eigenvectors of the covariance matrix C, but the accuracy may be insufficient
%[ V , D ] = eigs( C , k );
% returns the matrix V and the diagonal matrix D,
%The main diagonal elements of D are the eigenvalues of C, and the column vectors of V are the corresponding eigenvectors
%k means to return the k largest eigenvalues of matrix C to D from large to small
%-------------------Construct feature space ------------------
% Use the remaining k eigenvalues and their corresponding eigenvectors to construct the feature space
W = U' * Y ; % Construct feature space W, W size is k*280
%----------------------------test-------------------- -------
correct_num = 0;
for i = 1 : 40
for j = 8 : 10
image = imread( [ 'ORL_Faces\s' , num2str(i) , '\', num2str(j), '.pgm'] ) ; % read image
image1 = double(image);
test_image = zeros(10304,1) ;
for n = 1 : 92
for m = 1 : 112
test_image( m+(n-1)*112, 1 ) = image1( m, n ); % matrix vectorization
end
end
test_W = U' * (test_image - u ); %Project the image to be detected into the feature space
% find the closest face
distance = ones( 1, 280 );
%Create an array to store the Euclidean distance between the picture to be detected and all known pictures (280 pictures)
for x = 1 : 280
distance( 1, x ) = norm( test_W - W( : , x ) ); %norm(ab); calculates the Euclidean distance between two vectors
end
%[ M, I ] = min(A) means to return the minimum value M in A and the corresponding index I, ~ means to ignore the returned parameters
[ ~, min_index ] = min( distance ); % returns the serial number of the known picture with the smallest distance from the picture to be detected
%Judge whether the picture to be detected and the picture with the smallest distance to it are in the same folder (whether they belong to the same face)
local1 = ceil( min_index / 7 ); %ceil() round up
if local1 == i
correct_num = correct_num + 1;
end
end
end
fprintf('\n\nA total of 120 face pictures were tested\n');
fprintf('%d face pictures were correctly identified\n', correct_num);
fprintf('Recognition accuracy: %0.2f%%\n\n', (correct_num/120)*100);