Feature Selection - Filter, Wrapper, Embedded

Filter methods:

• information gain
• chi-square test
• fisher score
• correlation coefficient
• variance threshold

Wrapper methods:

• recursive feature elimination
• sequential feature selection algorithms
• genetic algorithms

Embedded methods:

• L1 (LASSO) regularization
  
  • Add a penalty term (regular term) to control overfitting
  • regularized_cost = cost + regularization_penalty
  • LASSO way:$\lambda \sum_i|w_i|$
• decision tree