The development and application of any new technology has two aspects of mutual promotion and mutual restriction: on the one hand, the development of technology can bring about social progress and change; Constraints on security mechanisms.
artificial intelligence safety
Artificial intelligence security is divided into three sub-directions:
- AI for Security
- Artificial Intelligence Endogenous Security (AI Security)
- AI-derived safety (AI Safety)
Among them, boosting security reflects the empowering effect of artificial intelligence technology; endogenous security and derived security reflect the accompanying effect of artificial intelligence technology. Artificial intelligence systems are not built purely on technology, but also need to work together with multiple external constraints to form a complete and compliant system.
The architecture and external associations of AI security are shown in Figure 1.
AI Boosts Security
It is mainly manifested in two aspects: assisting defense and assisting attack .
- Defenders are leveraging AI technologies to enhance and expand their existing defense methods.
AI machine learning models bring new avenues for proactive cyber defense. Intelligent models take a proactive approach, rather than the traditional reactive approach; at the same time, leveraging the predictive capabilities of artificial intelligence and the evolutionary capabilities of machine learning can provide us with the means to defend against complex cyber threats. Essentially, the most important change is to provide early warning and prevent cyberattacks before they occur .
AI2, an artificial intelligence-based network security platform developed by the Massachusetts Institute of Technology, uses artificial intelligence methods to analyze network attacks and help network security analysts do tasks similar to "finding a needle in a haystack". The AI2 system first autonomously scans data and activity using machine learning techniques, and feeds findings back to cybersecurity analysts. Network security analysts will mark which are real network attack activities, and incorporate feedback from engineers into the AI2 system for automatic analysis of new logs. In tests, the team found that AI2 was about three times more accurate than automated analysis tools in use today, greatly reducing the chance of false positives. In addition, AI2 can continuously generate new models during the analysis process, which means that it can quickly improve its prediction rate. The more attacks the system detects, the more feedback it receives from analysts, which in turn improves the accuracy of future predictions. According to reports, AI2 has trained more than 360 million lines of log files, enabling it to analyze 85% of attack behaviors in order to alert suspicious behaviors.
- In terms of assisting attacks, attackers are using artificial intelligence technology to break through the boundaries of their original capabilities.
Artificial intelligence can empower cyber attacks, which the industry calls automated or intelligent cyber attacks. Computer attacks are carried out automatically through robots without human intervention at all. Major hacking incidents that have occurred continuously in recent years, including core database leaks, hundreds of millions of accounts being hacked, WannaCry ransomware, etc., all have the characteristics of automated attacks. With the help of automated tools, attackers can scan and detect vulnerabilities on a large number of different websites in a more efficient and covert manner in a short period of time, especially for the whole network detection of 0day/Nday vulnerabilities, which will be more frequent and efficient. The powerful data mining and analysis capabilities of artificial intelligence, as well as the resulting intelligent services, are often used by hacker organizations. With the help of artificial intelligence technology, a more anthropomorphic and sophisticated automated attack trend is formed. This type of robot simulation The behavior of real people will be smarter, bolder, and harder to track and trace. At present, automated and intelligent network attacks are constantly causing network security defenses to fall frequently, and this obviously needs to attract enough attention from the network security industry. It is necessary to start with understanding the characteristics of automated network attack behaviors and take timely measures.
Artificial Intelligence Endogenous Security
Artificial intelligence endogenous security refers to the vulnerability of the artificial intelligence system itself . The causes of vulnerability include many factors. Any link such as artificial intelligence framework/components, data, algorithms, and models may introduce vulnerabilities to the system.
In terms of frameworks/components, it is difficult to ensure the correctness and transparency of the implementation of frameworks and components, which is an inherent security problem of artificial intelligence. Frameworks (such as TensorFlow, Caffe) are the basic environment for developing artificial intelligence systems, which are equivalent to the familiar Visual C++ SDK library or Python's basic dependency library, and their importance is self-evident.
In terms of data, the lack of ability to discriminate the correctness of data is an inherent security problem of artificial intelligence. For example, the loss and deformation of data and the input of noisy data will cause serious interference to the artificial intelligence system.
In terms of algorithms, it is difficult to ensure the correctness of algorithms is an endogenous security problem of artificial intelligence. The security flaws in intelligent algorithms have always been a serious problem in artificial intelligence security. For example, adversarial examples are a technique that exploits algorithm flaws to carry out attacks, and many safety accidents in autonomous vehicles can also be attributed to immature algorithms.
In terms of models, it is difficult to ensure that the models will not be stolen or polluted, which is an inherent security problem of artificial intelligence. The model is a copyable and modifiable physical file, so there is a security risk of being stolen or implanted with a backdoor. This is the issue that needs to be studied in the security of artificial intelligence models.
Artificial intelligence itself has vulnerabilities. For example, adversarial examples are an endogenous security problem of artificial intelligence. Adversarial examples are an interesting phenomenon of machine learning models that reflect weaknesses in AI algorithms. Attackers can make machine learning models accept and make incorrect classification decisions by adding subtle changes to the source data that are difficult for humans to perceive through the senses . A typical scenario is the adversarial example of the image classification model. By superimposing carefully constructed changes on the image, it is difficult for the naked eye to make the classification model misjudgment. Adversarial examples exist not only in the field of image recognition, but also in other fields, such as speech and text . From the perspective of network security, there are also attacks similar to adversarial samples. Attackers may deceive artificial intelligence models by inserting perturbation operations into malicious codes. For example, someone designed a malicious sample to allow the classifier to identify a software with malicious behavior as a benign variant, so that an attack method that can automatically escape the PDF malware classifier can be constructed to combat machine learning. application in security. All of the above security issues may lead to the same consequences, which is to cause wrong decisions, judgments, and system control in artificial intelligence systems .
AI-derived security
Artificial intelligence-derived security refers to the fact that artificial intelligence systems endanger the security of other fields due to their own vulnerabilities . Derivative security issues
mainly include four categories:
- Artificial intelligence systems can be attacked due to vulnerabilities
- Artificial intelligence system caused safety accident due to its own mistakes
- Artificial intelligence weapons development could spark an international arms race AIA
- Once out of control, it will endanger human safety
The mistakes of artificial intelligence may bring disasters to human beings, which will form derivative security problems. On May 7, 2016, a Tesla Model S in "autopilot" mode on a Florida highway crashed into a large white trailer truck at a speed of 74 miles per hour. The Model S passed under the van and the roof was completely blown off, killing the driver, 40-year-old Joshua Brown. The speed limit on the road where the crash occurred was 65 mph. Since the high-definition camera in front of the car in "autopilot" mode is a telephoto lens, when the white trailer truck enters the visual area, the camera can only see the middle of the truck suspended on the ground, but cannot see the entire vehicle; in addition, the sunlight at that time Strong (blue sky and white clouds), the automatic driving system cannot recognize that the obstacle is a truck, but more like a cloud floating in the sky, causing the automatic braking to not take effect. The accident sparked controversy over the safety of self-driving cars. This kind of autopilot defect that leads to human casualties is a typical case of artificial intelligence-derived safety.
"Artificial Intelligence Security Discussion" Fang Binxing1,2,3 Cui Xiang2,3 Gu Zhaoquan2,3 Academician Fang Binxing: My opinion on artificial intelligence security Fang Binxing
on artificial intelligence security
Artificial intelligence security risk analysis and connotation
1. New attack threats:
Attack method: attack against samples, data poisoning, model stealing, artificial intelligence system attack
Attack impact: Models may be attacked during training, testing, and inference; compromise the confidentiality, integrity, and availability of data and models.
2. Artificial intelligence security risks
① Potential safety hazards of algorithm models: Algorithms are written by humans, and models are also written by humans, both of which may have flaws, discrimination, and the possibility of black-box operations.
② Hidden dangers of data security and privacy protection: Collected data, used data, and stored data are abused and leaked to varying degrees.
③ Hidden dangers of infrastructure security: Simply understand, artificial intelligence also depends on databases, operating systems, and codes. These are infrastructures, and once these foundations are controlled by hackers, data is leaked.
④ Hidden dangers of application security: automatic driving (remote control by hackers leads to crashes), biometric identification (primary school students successfully fooled face recognition with photos), smart speakers, etc.
⑤ Artificial intelligence abuse: Using speech synthesis technology to pretend to be the victim's relatives to commit fraud, artificial intelligence technology is getting better and better at cracking login verification codes, and it is difficult to prevent, using artificial intelligence technology to imitate humans, such as face changing, handwriting forgery, human voice Fake, chatbots.
3. Security impact:
National security impact: Artificial intelligence can be used to build a new type of military strike force, which poses a threat to national defense security.
Social ethical challenges: Intelligent artificial robots replace humans, causing massive unemployment; people stop falling in love, and fall in love with robots.
Personal Safety Risk: Abstract
Artificial Intelligence Security Standardization White Paper (2019 Edition)
cyberspace security
A computing-based discipline involving technology, people, information, and processes that enable assured operations in the context of an adversary. It deals with the creation, operation, analysis and testing of secure computer systems. It is an interdisciplinary course of study that includes aspects of law, policy, human factors, ethics and risk management.
Cyberspace security not only focuses on the confidentiality, integrity and availability of information studied in traditional information security, but also focuses on the security and credibility of the infrastructure that constitutes cyberspace, as well as the impact of the network on real social security.
Professional analysis: The three attributes of confidentiality, integrity and availability (CIA for short) are commonly used internationally as the three elements of security. All content related to one of the three elements of the CIA in cyberspace is included in the scope of cyberspace security. Including: preventing information from being leaked, preventing unauthorized access and tampering, and preventing system unavailability.
Cyberspace
Cyberspace is an overall domain in the information environment, which consists of independent and interdependent information infrastructures and networks. Includes the Internet, telecommunication networks, computer systems, embedded processor and controller systems.
Professional analysis: Professionally, devices that follow the ISO/OSI 7-layer protocol framework (sometimes TCP/IP protocol framework) are collectively referred to as IT (Information Technology) equipment or systems, such as routers, servers, PCs, and various application software. If the entire scope is expanded to all non-IT equipment systems that can be connected to the network: including industrial equipment systems (Operation Technology, referred to as OT equipment) such as nuclear power plants; Internet of Things equipment systems (Internet of Things, referred to as IoT equipment) such as Bluetooth speakers, Self-driving cars. Such is the extent of cyberspace. Features are: massive + all things.
Network SecurityNetwork Security
Consists of policies, procedures, and practices adopted to prevent, detect, and monitor computer networks and network-accessible resources for unauthorized access, misuse, modification, or denial. Including network equipment security, network information security, and network software security.
Professional analysis: Network security usually refers to how to ensure confidentiality, integrity and availability between IT devices that follow the ISO 7-layer protocol framework (or TCP/IP). For example: the system is attacked, and the device is sniffed by hackers to obtain the password during communication. Features: IT equipment.
information security
Rigorous definition: ISO27001 definition: the organization, policy and process established to protect the confidentiality, integrity and availability of valuable information assets of the organization.
Professional analysis: Valuable information assets within an enterprise include hardware, software, services, personnel, data, intangible assets, etc. How to protect the confidentiality, integrity and availability of these assets. For example: prevent the company's important database server from being destroyed. It could be an external hacker, or it could be sabotage by an insider.
Data Security
Rigorous definition: Wikipedia: Protecting digital data from destructive forces and harmful actions by unauthorized users, such as cyberattacks or data breaches.
Professional analysis: protection of confidentiality, integrity and availability of structured data, semi-structured data and unstructured data throughout their life cycle.
importance
In the 21st century, with the rapid development of information technology and IT technology, the application of various network technologies has become more extensive and in-depth. At the same time, many network security problems have emerged, which has made the importance of network security technology more prominent, and network security has become the concern of all countries. Focus, not only related to the information resources and asset risks of institutions and individual users, but also related to national security and social stability, has become a new field of hot research and talent demand. Practical and effective measures must be taken in all aspects of law, management, technology, and ethics in order to ensure the "good and fast" stable development of network construction and application.
Cyberspace has gradually developed into the fifth largest strategic space after land, sea, air, and space. It is the core, key, and foundation that affects national security, social stability, economic development, and cultural communication. Cyberspace has the characteristics of openness, heterogeneity, mobility, dynamics, and security. New network forms such as the next-generation Internet, 5G mobile communication network, mobile Internet, and the Internet of Things are constantly evolving, as well as cloud computing, big data, Social networking and many other new service models.
Network security has become one of the world's hot research topics and has attracted widespread attention from the society. Network security is a systematic project, which has become the primary task of informatization construction and application. Network security technology involves laws and regulations, policies, strategies, norms, standards, mechanisms, measures, management and technology, and is an important guarantee for network security.
Information, materials, and energy have become the three pillars and important guarantees for the survival and development of human society. The rapid development of information technology has brought profound changes to human society. With the rapid development of computer network technology, my country has made remarkable achievements in network construction. The wide application of e-banking, e-commerce and e-government has made the computer network penetrate into the country's politics, economy, culture and national defense. In all fields of construction, in every aspect of work and life in the modern information society, the integration of "digital economy" and global electronic transactions is taking shape. Network security is not only related to the national economy and people's livelihood, but also closely related to national security. It not only involves all aspects of national politics, military affairs and economy, but also affects national security and sovereignty. With the wide application of information technology and network technology, the importance of network security is particularly prominent. Therefore, the most critical and easily overlooked security issue in network technology is jeopardizing the healthy development and application of the network, and network security technology and application are attracting more and more attention from the world.
jtj2008 know partner education experts
https://blog.csdn.net/agiogo/article/details/115861564