Day by day, more and more machine learning (ML) models are being developed. Machine learning models are used to find patterns in training data and can yield impressive detection and classification capabilities. Machine learning already powers many areas of artificial intelligence, including sentiment analysis, image classification, face detection, threat intelligence, and more.
Billions of dollars are being poured into machine learning research production. There is clearly a strong interest in machine learning projects, what are the security risks to keep in mind when training these models?
The OWASP Machine Learning Security Top 10 analyzes the most common vulnerabilities related to machine learning. The following summarizes each risk in the top ten list and considers how to protect the integrity and security of the model from creation to deployment.
OWASP Machine Learning Security Top 10
ML01:2023 Adversarial Attacks
This type of attack involves malicious actors intentionally changing the input data of a model. For example, consider an image classification model. An attacker can create an adversarial image with slight changes that lead to misclassification. In a cybersecurity context, adversarial changes can help attackers avoid detection by machine learning-driven intrusion detection systems.
ML02:2023 Data Poisoning Attacks
Another risk to consider is an attacker manipulating the data the model was trained on. If a data storage system is compromised, an attacker could insert mislabeled data. For example, this can cause a spam detection model to misidentify spam as legitimate communication. Incorrect classification and wrong decisions can lead to potentially unsafe outcomes.
ML03:2023 Model Inversion Attack
A model inversion attack occurs when an actor reverse-engineers a model to obtain hidden information. Inverting a model can be done by training one model and using it to invert the predictions of another model. This vulnerability could allow attacks to go undetected, or hackers to obtain sensitive or personal information based on the model's predictions.
ML04:2023 Membership Inference Attacks
Membership inference is another type of attack where an attacker is able to infer sensitive data from a model. Hackers can do this by taking the training data and then using that model to query whether a particular individual's record is contained in the dataset. Membership inference risks are challenging to both exploit and detect.
ML05:2023 Model Stealing
This attack involves bad actors or competitors stealing or replicating the model itself. Deployed models can be insecure and thus easily stolen. Alternatively, the model can be reverse engineered. Once stolen, the model could be used for competing business purposes, causing financial loss to the original model owner.
ML06:2023 corrupt package
Most modern software depends on a large number of open source or third-party dependencies, and the same is true for machine learning. One risk is that hackers could insert malicious code to corrupt the public libraries that the models depend on. Once an ML project downloads an updated version, the project is compromised.
ML07:2023 Transfer Learning Attacks
Transfer learning is when engineers take a pre-trained model and fine-tune it using additional data. Attackers can use this strategy to retrain existing models on malicious datasets. If they can successfully change the model used by the end application, they can successfully bypass things like intrusion detection systems.
ML08:2023 Model Tilt
Another risk involves attackers exploiting the MLOps feedback process to distort the training data. For example, a hacker could feed in feedback data and retrain the entire model to prioritize specific results. Model skew attacks can introduce bias and compromise the accuracy and fairness of the system.
ML09:2023 Output Integrity Attacks
An output integrity attack is when an attacker takes the output of a machine learning model and manipulates that output to provide falsified information. For example, if an interface displaying ML output is compromised, a hacker could use a man-in-the-middle (MitM) attack to change the behavior of its appearance or edit the results.
ML10:2023 Neural Network Reprogramming
Finally, this attack refers to the attacker manipulating the parameters of the model to change its expected behavior. For example, this can be achieved by changing the images in the training set or modifying parameters. Neural network reprogramming attacks can cause models to make wrong judgments, which can be exploited by bad actors for financial gain.
Mitigate the Top 10 Machine Learning Risks
All of these strategies are similar in that they can cause the model to make bad decisions or behave unsafely. So, how can you reduce risk when developing and deploying machine learning models? Here are some advanced prevention tips recommended by OWASP:
■ Adversarial changes to the training model and include defense mechanisms.
■ Ensure training data is carefully validated and secured.
■ Restrict access to model predictions and encrypt the model's code.
■ Implement strict access controls to protect machine learning models.
■ Verify package signatures and use trusted repositories of third-party software.
■ Regularly monitor and update your data sets.
■ Verify the authenticity of feedback data.
■ Protect model interfaces through secure communication and encryption.
It's also worth noting that, as in other areas of cybersecurity, many risks are not technical in nature, but arise from social engineering tactics, such as bribery or threats. Therefore, consider insider threats seriously and adopt a zero-trust model for all development environments, including machine learning projects.
As we continue to ramp up our investments in artificial intelligence, securing machine learning projects will continue to be a top priority. The above summarizes the key issues to keep in mind when developing machine learning models. Before proceeding to develop and run ML in production, cybersecurity professionals should consider reviewing each OWASP risk and detailed mitigation recommendations for each risk.