This article was written by Alexander Polyakov, CTO and co-founder of ERPScan, President of EAS-SEC, and evangelist for SAP cybersecurity.
There have been quite a few articles now covering machine learning and its ability to protect us from cyberattacks. Nonetheless, we need to clearly separate the ideal from the reality and see what exactly machine learning (ML), deep learning (DL) and artificial intelligence (AI) algorithms can do in cybersecurity.
First of all, I have to disappoint you because we have to admit that despite the good results of machine learning in the two areas of image recognition or natural language processing, machine learning is by no means a silver bullet for cybersecurity (silver bullet). : a metaphor for a new technology, a new technology that people have high hopes for) . There will always be people trying to find problems in our systems and try to bypass them. To make matters worse, these advanced technologies are also being used by hackers, such as hackers who can also use machine learning to achieve their intentions.
Machine learning can not only help us with typical ML tasks, including regression (prediction), classification, clustering, recommendation. ML can also solve problems with varying efficiencies for various needs, depending on the algorithm you choose. Now, we will use machine learning to solve typical cybersecurity tasks.
According to Gartner's PPDR model, all security tasks can be divided into five categories: prediction, prevention detection, response and monitoring. More precisely, they can be used at technical layers like network (network traffic analysis and intrusion detection), endpoint (anti-malware), application (WAF or database firewall) or user (UBAs, anti-fraud).
Now, let's look at an example of how current machine learning methods can be applied to cybersecurity tasks.
1. Return
Regression is a simple task, in other words prediction is a simple task. We want to use our knowledge of existing data to make predictions about new data, the simplest example being house price predictions. In cybersecurity, it can be used for tasks such as user behavior analysis as well as fraud detection. Network traffic analysis is another good option for using machine learning. As for the technical aspects of regression, various types of recurrent neural networks work best.
2. Classification
The classification problem is also simple. If you have two stacks of photos, say a dog and a cat, you can easily put new photos on top of the corresponding photos, this is often called supervised learning. We know exactly what we are looking for and put them in the right place. So how can the classification algorithm of machine learning be applied to network security? Suppose we want to detect malicious activity on different layers. For the network layer, we can apply it to Intrusion Detection System (IDS) and identify different classes of network attacks like scanning, spoofing, etc. At the application layer, we can apply it to WAF and detect OWASP top 10 attacks. At the endpoint layer, we can classify software into categories such as malware, spyware, and ransomware. Finally, at the user level, it can be applied to anti-phishing solutions to tell us whether a particular email is legitimate or not. Technically, algorithms like SVM or random forests and better simple artificial neural networks or convolutional neural networks can solve these tasks.
3. Clustering
The idea of clustering and classification to solve network security problems is basically the same, there is only one major difference: we do not know any information about the data class. Furthermore, we do not know whether the data can be classified. This is called unsupervised learning. We do not participate in the data labeling process and leave all the tasks to the machine, which sounds like a very interesting attempt.
I feel one of the best tasks for clustering is forensic analysis - when we don't know what's going on and classify all activity to find outliers, a malware analysis solution (i.e. Malware Protection) can implement it to Separate legitimate files from outliers. Another interesting area where clustering can be applied is user behavior analysis. In this case, application users are clustered together and can see if they belong to a specific group. According to their group, provide corresponding effective network security solutions.
4. Recommendation
The recommendation system is a very famous system in the Internet age. For example, when we all use Netflix and SoundCloud, they recommend you movies or songs they think you like based on your movie or music preferences. This idea can also be applied to cybersecurity, where it can be used primarily for incident response. If a company faces a series of events and proposes various types of responses, the system can learn which type of response should be recommended for a particular event. Risk management solutions can also benefit as they can automatically assign risk values to new vulnerabilities or build misconfigurations based on their descriptions. There are now many algorithms for recommendation tasks, the latest ones are based on restricted Boltzmann machines and their newer versions, such as deep belief networks.
V. Conclusion
In addition to these security areas I mentioned, there are many more security areas where machine learning can be applied. Machine learning is by no means a perfect solution if you want to protect your system, but at the same time, it will become standard in cybersecurity in the near future as hackers are already using machine learning as a means of attack.
Original address: http://mp.weixin.qq.com/s/CpCvIxNChRVDhkgzETVIQQ
So how does NetEase Yunyidun use artificial intelligence? Share two more dry goods articles:
2. Tracking Predators: How Does Underground Ash Production Kill Startups?