基于模因框架的包装过滤特征选择算法

引用

LaTex

@ARTICLE{4067093,
author={Z. Zhu and Y. S. Ong and M. Dash},
journal={IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)},
title={Wrapper ndash;Filter Feature Selection Algorithm Using a Memetic Framework},
year={2007},
volume={37},
number={1},
pages={70-76},
keywords={biology computing;genetic algorithms;learning (artificial intelligence);pattern classification;search problems;classification problem;feature selection algorithm;genetic algorithm;local search;memetic framework;microarray data set;wrapper filter;Acceleration;Classification algorithms;Computational efficiency;Filters;Genetic algorithms;Machine learning;Machine learning algorithms;Pattern recognition;Pervasive computing;Spatial databases;Chi-square;feature selection;filter;gain ratio;genetic algorithm (GA);hybrid GA (HGA);memetic algorithm (MA);relief;wrapper;Algorithms;Artificial Intelligence;Biomimetics;Computer Simulation;Models, Theoretical;Pattern Recognition, Automated;Software;Systems Theory},
doi={10.1109/TSMCB.2006.883267},
ISSN={1083-4419},
month={Feb},}

Normal

Z. Zhu, Y. S. Ong and M. Dash, “Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework,” in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 37, no. 1, pp. 70-76, Feb. 2007.
doi: 10.1109/TSMCB.2006.883267
keywords: {biology computing;genetic algorithms;learning (artificial intelligence);pattern classification;search problems;classification problem;feature selection algorithm;genetic algorithm;local search;memetic framework;microarray data set;wrapper filter;Acceleration;Classification algorithms;Computational efficiency;Filters;Genetic algorithms;Machine learning;Machine learning algorithms;Pattern recognition;Pervasive computing;Spatial databases;Chi-square;feature selection;filter;gain ratio;genetic algorithm (GA);hybrid GA (HGA);memetic algorithm (MA);relief;wrapper;Algorithms;Artificial Intelligence;Biomimetics;Computer Simulation;Models, Theoretical;Pattern Recognition, Automated;Software;Systems Theory},
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4067093&isnumber=4067063

---

摘要

a novel hybrid wrapper and filter feature selection algorithm for a classification problem using a memetic framework

a filter ranking method
genetic algorithm
univariate feature ranking information

the University of California, Irvine repository and microarray data sets

classification accuracy, number of selected features, and computational efficiency.

memetic algorithm (MA) — balance between local search and genetic search
to maximize search quality and efficiency

---

主要内容

1) filter methods
2) wrapper methods

---

wrapper–filter feature selection algorithm (WFFSA) using a memetic framework

WFFSA

Lamarckian learning

local improvement
Genetic operators

---

A 编码表示与初始化

a chromosome is a binary string of length equal to the total number of features

randomly initialized

---

B 目标函数

the classification accuracy

$S_c$ — the corresponding selected feature subset encoded in chromosome $c$
$J \left( S_c \right)$ — criterion function

---

C LS改进过程

domain knowledge and heuristics

filter ranking methods as memes or LS heuristics

three different filter ranking methods, namely:
1) ReliefF;
2) gain ratio;
3) chi-square.

based on different criteria:
1) Euclidean distance,
2) information entropy,
3) chi-square statistics

basic LS operators:
1) “Add”: select a feature from Y using the linear ranking selection and move it to X.
2) “Del”: select a feature from X using the linear ranking selection and move it to Y .

The intensity of LS — the LS length $l$ and interval $w$
LS length $l$ — the maximum number of Del and Add operations in each LS — $l^2$ possible combinations of Add and Del operations
interval $w$ — the $w$ elite chromosomes in the population

until a local optimum or an improvement is reached

1. Improvement First Strategy: a random choice from the $l^2$ combinations. stops once an improvement is obtained either in terms of classification accuracy or a reduction in the number of selected features without deterioration in accuracy greater than $ε$.
   
2. Greedy Strategy: carries out all possible $l^2$ combinations — the best improved solution
   
3. Sequential Strategy: the Add operation searches for the most significant feature $y$ in $Y$ in a sequential manner; the Del operation searches for the least significant feature x from X in a sequential manner
4. Evolutionary Operators:
   
   
5. Computational Complexity:
   The ranking of features based on the filter methods — linear time complexity — a one-time offline cost — negligible
   the computational cost of a single fitness evaluation — the basic unit of computational cost
   GA — $O(pg)$: $p$ — the size of population, $g$ — the number of search generations
   +improvement first strategy — $O (pg + l^2wg/2)$
   +the greedy strategy ($l^2w/p$) — $O (pg + l^2wg)$
   +the sequential strategy ($(2|Y | − l)l/2$ and ($2|X| + l)l/2$ — Add and Del operations — $nlw$) — $O(pg + nlwg)$
   $n \gg  l$ — $nlw \gg l^2w > l^2w/2$ — sequential LS strategy requires significantly more computations

---

试验

UCI data sets
ALL/AML, Colon, NCI60, and SRBCT

population size — $30$ and $50$ or $100$ (microarray data sets)
fitness function calls — $6000$ and $10000$ or $20 000$ (microarray data sets)

the one nearest neighbor (1NN) classifier
the leave-one-out cross validation (LOOCV)