【Paper Reading】A Review on Feature Selection and Ensemble Techniques for Intrusion Detection System

Original Title: A Review on Feature Selection and Ensemble Techniques for Intrusion Detection System
Original Author: Majid Torabi1*, Nur Izura Udzir 2*, Mohd Taufik Abdullah3, Razali Yaakob4
Published: IJACSA 2021
Original Link: https://pdfs.semanticscholar.org /06b7/2a8cc22b4f9fecfd8fcf396a693429dd6326.pdf
Chinese title: A Survey of Feature Selection and Integration Techniques for Intrusion Detection Systems

1 Motivation

This paper focuses on the application of machine learning algorithms for feature selection in anomaly-based IDS and detection based on ensemble learning methods. It also discusses and compares the previously proposed models in this field, and points out the characteristics of each model to promote more research in this field.

2 main tasks

• Introduce the classification of IDS detection methods
• Introduces classification methods based on machine learning techniques used in anomaly-based IDS
• Introducing feature selection techniques for anomaly-based IDS
• Introducing Ensemble Learning Techniques for Anomaly-Based IDS
• Discuss future directions of state-of-the-art anomaly-based feature selection and ensemble classification methods

At the same time, this article will answer the following questions one by one as the context of writing:

• RQ1. What are the detection methods used by IDS?
• RQ2. What assessment tools are used to assess the effectiveness of the IDS?
• RQ3. What datasets are available for anomaly-based IDS?
• RQ4. What feature selection methods are used for anomaly-based IDS?
• RQ5. In anomaly-based detection, what are the machine learning algorithms used to detect intrusions?
• RQ6. Which integration techniques are used for exception-based IDS?

3 IDS detection method

The detection methods used in intrusion detection are mainly divided into four types: signature-based, anomaly-based, template-based and hybrid .

• In signature-based detection, when system or network actions match threat patterns (called signatures) stored in IDS's local database, IDS identifies the threat and activates an alert. Signature-based IDS are effective and efficient in identifying existing attacks. However, this technique is not effective in identifying 0-day attacks and new variants of previously identified attacks.
• Anomaly-based detection aims to predict "normal" patterns of a system to overwrite and to warn of anomalies when the difference between impending behavior and normal behavior reaches a predetermined threshold. The main benefit of anomaly-based detection methods is their ability to identify previously undetected attack events. However, in anomaly-based systems, the false positive rate (FP), or the rate at which an attack is misdefined as an attack, is usually higher than in signature-based approaches, given possible inaccuracies in formal signature specifications.
• For template-based detection methods, human experts manually construct the required template, which consists of a series of rules (norms) designed to evaluate the valid behavior of the device. If the parameters are accurate enough, the template can identify patterns of illegal behavior. Furthermore, the false positive rate is reduced, mainly because previously unobserved benign behaviors are not flagged as intrusions in such systems.
• Hybrid detection methods aim to exploit the strengths of each intrusion detection method, minimize their weaknesses and build powerful patterns to detect intrusions. A common example of hybrid detection is the combination of a key signature-based detection system with an additional anomaly-based model. This integration of both forms of detection strategies in a "hybrid NIDS" aims to improve the ultimate accuracy of signature-based intrusion detection models while eliminating the usual high level of false positives of network-based IDSs (NIDSs), so most existing The platforms all take a hybrid approach.

The table below shows the types of detection methods used by IDS researchers. From the table, it is evident that feature-based and template-based detection methods do not utilize feature selection and ensemble classifiers to detect intrusions, while anomaly-based detection utilizes both of them. The NSL-KDD dataset is the most commonly used dataset based on this review article.

4 Machine Learning Approaches in Anomaly-Based IDS

In this section, the author expands each method in detail, and introduces the specific applications of ANN, MLP, KNN and other methods in previous work, which will not be expanded here. The overall classification diagram is as follows:

5 Feature Selection Techniques

Feature selection (FS) is a method to remove unnecessary and redundant features and select the most suitable subset of features, which will lead to better classification of patterns belonging to various attack categories. FS can not only significantly improve detection accuracy, but also significantly improve computational efficiency, where: a) irrelevant or redundant features lead to poor detection rate and overfitting, so reducing them can improve detection accuracy; b) More features per data point lead to higher computational cost and complexity - reducing irrelevant features will increase computational efficiency. FS methods are usually divided into Filter, Wrapper and Optimization-based methods.

• Filter : Filter methods use different information theory and mathematical formulas for feature selection. Due to their simplicity, ranking methods are used and have good performance in practical applications. Variables are rated based on accepted ranking criteria, and thresholds are used to eliminate variables just below the threshold.
• Wrapper : Wrapper feature selection uses machine learning as a fitness function and determines the best feature subset of all feature subsets. This problem formulation allows general optimization techniques to be used with machine learning to rank subsets of features based on their predictions. Therefore, the Wrapper method is usually better than the Filter method in terms of the final prediction accuracy of machine learning.
• Optimization-based : The classic Filter and Wrapper strategies evaluate and select subsets independently, but some features are not independent, but they are truly successful when they work together. So the classic strategy in this area is not very successful. Meta-heuristic-based methods have been used to select and classify selected features due to their ability to greatly improve detection. Examples of optimization-based methods are Particle Swarm Optimization (PSO) Network Feature Entropy, Genetic Algorithms, Ant Colony Optimization, and Kernel Principal Component Analysis (KPCA), among others.

Table II illustrates the advantages and disadvantages of the above feature selection methods, and Table III summarizes the feature selection methods for anomaly-based IDS, respectively, as follows.



According to the articles in Table III and the results in the figure below, it shows that optimization-based methods are mainly used for feature selection in recent years. This approach provides a large improvement in the number of features. According to the survey, researchers mainly use the NSL-KDD dataset to prove their models. Furthermore, some studies utilize different datasets to highlight the generality of their solutions, such as Kyoto2006+, ISCX 2012, UNSW-NB15, and CIC-IDS2017.

The author believes that the current feature selection method has some limitations:

• Optimal detection methods or strategies for various datasets have not been established
• Lack of an appropriate feature subset for faster training with minimal computation and best performance in detecting intrusions with high accuracy and fewer false positives

6 Integrated Learning Technology

Three widely used ensemble models are bagging, boosting and stacking. In this section, the author reviews the application of these three techniques respectively, and this article does not expand in detail.
The author summarizes the detection methods proposed by previous work and their corresponding advantages and disadvantages, as shown in the following table:

The frequency of common data sets used in past papers is as follows:

7 summary

This paper reviews the research on feature selection and integration methods for anomaly-based intrusion detection systems, discusses the main challenges of IDS, namely feature dimensionality reduction for anomaly-based IDS, which reduces irrelevant attributes in the dataset; and how to construct appropriate Feature subset selection for better intrusion detection by improving performance metrics. Therefore, this study categorizes and discusses feature selection methods and demonstrates their performance in terms of detection accuracy. Another important challenge of anomaly-based IDS lies in using suitable machine learning algorithms in the detection process. To illustrate their effectiveness in improving IDS performance, this paper reviews and categorizes various machine learning modalities and discusses their application in IDS, emphasizing ensemble approaches as an emerging trend in anomaly-based IDS.

This article summarizes two points on how to improve the performance of exception-based IDS:

• Optimization-based feature selection with excellent combination and well-tuned parameters will select a suitable subset of features for IDS. Through this study, it is clear that optimization-based optimization has remarkable performance in designing optimal feature sets. Furthermore, feature selection can be significantly enhanced if their parameters are well tuned.
• The integrated detection of different types of classification can enhance the capability of the detection stage and reduce the false positive rate.

This article also has some limitations:

• This paper focuses on feature selection and integrated detection for anomaly-based IDS
• This paper does not focus on the performance parameters used by IDS
• This paper does not delve into the characteristics of IDS datasets, attack types, etc.

8 Possible Future Jobs

• Anomaly-based IDS datasets have a crucial impact on performance evaluation. For now, it is necessary to take advantage of newer datasets to show that the proposed solution is applicable to new types of attacks. Even though KDD Cup 99 is the dataset most researchers use as a benchmark for comparison, the attack packets and even the signatures are 20 years old. Furthermore, researchers can deploy their models on different anomaly-based IDS datasets to demonstrate the generalizability of their models to detect different attacks.
• Finding a suitable feature selection pattern plays an important role in anomaly-based IDS. Optimization-based feature selection aims to obtain an optimal subset of features from all features in different domains. New optimization-based feature selection methods can be considered and applied in anomaly-based IDS.
• Ensembling methods allow multiple combinations of models or algorithms. In the implementation, after constructing various classification models using partial data sets, the results of various classifiers are combined to form the final conclusion. Different schemes can be provided for the combination of classifier generation and integration.