Shared todayAI seriesIn-depth research report: "AI large model report: 2023 large model Trustworthy Research Report》.
(Report produced by: China Academy of Information and Communications Technology)
Total report: 48 pages
Development status of large models
Large models drive a new round of technological revolution
In the past ten years or so, the generalization ability, innovation ability and application efficiency of artificial intelligence technology have been continuously improved, and it has become an important engine for promoting economic and social development. Around 2015, the face recognition algorithm reached recognition capabilities close to those of the human eye, which was regarded as a representative event for the industrial application level of artificial intelligence technology. In 2022, large models represented by ChatGPT will bring a new interactive experience to users. Through its efficient and easy-to-operate performance in tasks such as content generation, text transformation, and logical reasoning, large models are gradually becoming an important part of current mainstream applications.
With continuous breakthroughs in data, algorithms and computing power, large models will continue to optimize and evolve. In terms of data, massive, multi-modal data will continue to be used in large model pre-training to improve the knowledge, understanding and reasoning capabilities of large models. In terms of algorithms, we will turn to the processing of complex tasks such as mining and execution of massive knowledge across knowledge fields, languages, and multi-modal features. In terms of computing power, the construction of infrastructure such as intelligent computing centers and computing power networks has been accelerated to provide sufficient performance support for the development and services of large models. By 2026, Gartner predicts that more than 80% of enterprises will use APIs or models for generative AI or deploy large model-enabled applications in production. Large model applications represented by general intelligence, embodied intelligence, and brain-like intelligence may bring about a new round of technological revolution and industrial change.
Large models accelerate and empower industrial applications
The "big model +" model accelerates application empowerment and promotes the upgrading of the artificial intelligence industry. Currently, artificial intelligence has become the core competitiveness in the global emerging technology field. Governments of various countries are accelerating the research and development and deployment of artificial intelligence technology to promote rapid industrial development. According to statistics1, the scale of my country's core artificial intelligence industry has reached US$500 billion, and the number of companies exceeds 4,300. At the beginning of 2023, my country's large model market is booming. Baidu, SenseTime, iFlytek, Alibaba and other units have successively released self-developed large models and will gradually provide services to users in the second half of 2023. Large models are widely used in energy, finance, education, medical care, transportation, government affairs and other fields. The main application scenarios focus on data analysis, customer service, marketing, office, etc. Among them, the two major industries, led by energy and finance, have combined industry data construction foundations to actively deploy large-scale model applications and accelerate the intelligent transformation of the industry.
The large-model technology ecosystem is gradually improving, significantly lowering industry application thresholds. On the one hand, open source large models accelerate the penetration of large model applications, open up the development stages such as pre-training, fine-tuning, deployment, and evaluation, and further reduce the cost of large model R&D and application. In July 2023, the Shanghai Artificial Intelligence Laboratory officially open sourced InternLM-7B, a lightweight version of the 7 billion parameter model of the Shusheng Puyu large model, and launched the first full-chain open source system for large model development and application, and also provided free Commercial use has attracted widespread attention from academic and industrial circles. In July of the same year, OpenAI officially opened the code analysis plug-in Code Interpreter to users, allowing ChatGPT and GPT-4 to write and execute code based on user questions, thereby expanding the model's capabilities in data analysis, complex calculations, and function calls. On the other hand, large models are gradually evolving in the direction of intelligent agents, moving from understanding and generation to complex task processing capabilities. By combining the large model with the action executor, the agent can plan and make decisions through the large model after accepting user input, and call third-party plug-ins or tools, thereby achieving complex task processing capabilities and further lowering the application threshold. .
Large model risk analysis
Big model risk view
The rapid deployment and widespread application of large models also induces more hidden risks: First, framework risks. Deep learning frameworks face malicious attacks at the physical and network levels, making it difficult to maintain the stability and security of the infrastructure on which large models rely. Guarantee; the second is data risk. Collecting and processing massive, multi-modal training data may introduce more harmful data, which can easily lead to personal privacy leakage, intellectual property infringement, data bias and other issues; the third is model risk. At this stage, The anti-interference ability of large models is relatively weak, and there are problems such as malicious attacks, decision-making biases, and model operation risks. The fourth is the risk of generating content. Large models have "hallucinations", answering questions incorrectly, and generating illegal and bad information. These problems have become the most common problems of large models. Risks of concern. The efficient and convenient content generation capabilities of large models have greatly lowered the threshold for malicious behaviors such as fraud and phishing emails. However, the traceability guarantee mechanism for generated content has not yet been perfected, making the supervision of malicious content generation more difficult.
This report focuses on the trustworthy goals of large models such as reliability, robustness, security, fairness, accountability, and explainability, and analyzes from the perspective of large model risk factors such as frameworks, data, models, and generated content, and combines The whole life cycle management concept of large models, such as data collection, model pre-training, model fine-tuning, deployment and operation, optimization and update, etc., is proposed to comprehensively improve the trustworthy performance of large models.
At the framework level, software vulnerabilities are the shortcomings of existing deep learning frameworks
Infrastructure risks in the field of large models mainly include vulnerabilities at the software level such as deep learning frameworks and development kits, as well as the instability of the operating environment. Possible risks include physical attacks, network attacks, operating environment tampering, operation and maintenance failures, etc.
The operating environment of the deep learning framework has low fault tolerance, and core asset protection faces challenges. The risk of instability in the running environment of large models mainly comes from the operation and maintenance of large model services and the insufficient service level of the service level agreement (SLA) caused by poor robustness during model iterative updates, which may affect the availability of large model services. During the training and inference process, model training or inference tasks may be interrupted due to equipment, network, or communication failures. In addition, the running environment of large models also faces security risks. On the one hand, the lack of strict network isolation of the infrastructure from other systems may lead to the risk of lateral penetration from other internal systems. If an attacker successfully invades the infrastructure system and injects backdoors, Trojans and other malicious programs, the entire system will face serious security risks. On the other hand, the running environment of large models lacks security protection measures for training data, models and network communications, making core assets such as training data and model parameter files vulnerable to threats such as leakage, tampering and theft.
Large model trustworthy practices
At the framework level, the trusted framework and execution environment ensure operational security.
In view of the software vulnerability risks and unreliable operating environment faced by deep learning frameworks, on the one hand, technical measures such as vulnerability management, malicious program detection, and access control are used to reduce the possibility of malicious access and attacks on the deep learning framework; on the other hand, through Build an AI core asset protection mechanism to ensure the security and credibility of the deep learning framework operating environment.
At the data level, security detection and processing help make large models reliable
Data is used throughout the entire life cycle of large models, and security assurance and effective processing are key measures to ensure the reliability of large models. At the data level, trustworthy practices mainly involve measures such as security and compliance processing of the entire data process, data security sandbox technology, poisoning detection, and data analysis.
At the model level, full-process prevention and control enhances the credibility of large models
At the model level, trustworthy practices can be carried out in three stages: design and development, model training, and deployment and operation. The design and development stage mainly involves safety and ethical design assessment in the early stage of large model development; the model training stage mainly involves large model pre-training, trustworthy capability testing and reinforcement measures in the fine-tuning process; the deployment and operation stage mainly involves the operation of large models Operation and maintenance capabilities in the process to enhance users’ trust in model operation.
At the content generation level, filtering and identification make content controllable and accountable.
In terms of generated content, trustworthy practices mainly involve the construction of generated content evaluation, content review mechanisms and content traceability capabilities, so as to achieve content security and controllability and a certain degree of traceability capabilities. In order to alleviate the "illusion" phenomenon of large models, the generated content evaluation mainly focuses on authenticity, accuracy and safety. In order to reduce the security risk of generated content, the content review mechanism usually takes the form of a combination of machine review and manual review. In order to further improve the problem of difficult traceability of generated content caused by secondary editing, digital watermarking technology is gradually improving its robustness.
Total report: 48 pages