RBM network feature selection can be done, remove redundant and irrelevant features, reduce computation and improve accuracy
1, the principles outlined
Has completed training for the RBM network, it can be considered relevant connection has been established between features. For any feature i, if can be calculated through other network features via rbm, then we can think of this feature i is redundant. Specifically how to calculate it? First define v 'is the reconstructed v (see above a conceptual understanding section), then for any feature i, defined EI reconstruction error (i.e. vi and v'i much difference, of course, not to do simple addition and subtraction, specifically see below), then the training data network RBM tested twice, the first time using the original data, the second feature value i becomes zero retest, if the result of the error after the two runs, becomes 0 or less the same, then that, i this feature is redundant.
2, the specific process
Feature selection is performed by three steps: 1 ) initial training: the training data including all features will be rbm training; 2 ) elimination wherein: the initial training rbm remove additional feature; 3 ) Main Training: Training selected features comprising with initial training on data rbm training dbm .
1) RBM network training, see above article detailed, easy to read here, first of all the relevant formula treated as follows
That is: the gradient formula expressed directly from the desired form. Wherein Edata derived from training data represents the desired, Emodel represents the desired model in all possible cases, wherein the second term V (upper slip ~) represents approximate data obtained using Gibbs sampling
2) feature eliminates
The algorithm starts from input set of all features using the trained rbm to evaluate the effect of each characteristic.
Reconstruction error is defined
Formally, ei is the i reconstruction characteristics with respect to the variance of the original input value, i.e. the degree of deviation, is referred to the original reconstruction error
Then, we want to judge a feature of i, the corresponding setting unit vi = 0, then the data input rbm network, and calculates
Note that the V (superscript -) is set after the reconstruction of vi = 0 vi obtained. The EI (superscript -) after setting is the reconstruction error. Referred to Comparative reconstruction error
The following comparison of the two reconstruction error, if the comparison after the reconstruction error <original reconstruction error, it shows characteristics vi of i = 0, worth than his results with the original excellent or better, i.e., i honor this feature, give up. On the contrary adopted.
Note : understanding personal mark, the first use of this process is already well-established RBM. Next, the calculated reconstruction error, to use the features of the training set for all elements i, i.e. ei desires is input to each of the training element i RBM its own characteristics with respect to the reconstruction of the original input and the offset value is added
3) using the selected feature and the corresponding weight RBM network initialization and training the like DBM
3, pseudocode (two kinds)
References:
[1] Aboozar Taherkhani, Deep-FS: A feature selection algorithm for Deep Boltzmann Machines, 2018