Digital content includes text, pictures, videos and other forms. It originated with the advent of computers and with the rapid development of the Internet and smartphones, today, digital content has become a necessary way for individuals and enterprises to build their image and spread their value.
Starting from 2022, with the popularity of ChatGPT, the powerful generation capabilities of AI large models have attracted national attention. The digital content production method has also entered the AIGC era after passing through PGC and UGC. The number of pictures, videos, texts and other forms has increased exponentially. .
The high degree of freedom in digital content production methods and the great abundance of digital content have brought security challenges to corporate content ecological operations. Companies face risks such as content non-compliance, online fraud, hacker attacks, spam registration, and proxy fraud. How to protect the platform content ecology, defend against the invasion of black and gray products, and maintain the reputation of the platform has become a significant pain point for corporate content operations.
Digital content risk control solutions can meet the needs of enterprises for digital content security operations. Digital content risk control solutions involve machine review, human review, strategic operations, public opinion monitoring, blue army services and training and other countermeasures, aiming to achieve effective management and risk prevention and control of digital content through multi-dimensional comprehensive supervision.
In the process of implementing digital content risk control solutions, what are the differences in the needs of various industries for digital content risk control? What capabilities should companies build in different digital content risk control scenarios? How to choose a digital content risk control solution provider? And how effective are the digital content risk control practices of leading companies?
Against this background, on September 7, iAnalysis and NetEase Yidun, a subsidiary of NetEase Intelligence Enterprise, officially released the "2023 Digital Content Risk Control Industry White Paper". As the first white paper in the digital content risk control industry, this white paper will systematically answer the above questions.
The white paper first introduces the origin and definition of digital content risk control, then explains the digital content risk control scenario needs that enterprises face and the capabilities they should have, and analyzes the differences in the needs of digital content risk control in key industries. Next, the white paper explains The digital content risk control solution that integrates humans and machines will be elaborated in detail, and finally the digital content risk control practice cases of leading companies in the entertainment and social networking, gaming e-sports, automobile travel and other industries will be displayed in detail to provide readers with reference.
Note: This article is a simplified version of the white paper. For the full version of the "2023 Digital Content Risk Control Industry White Paper", please pay attention to "爱analytic ifenxi" to receive it.
01 The origin and definition of digital content risk control
1.1 Analysis of the development history and driving factors of digital content risk control
1.1.1 Development History of Digital Content Risk Control
(1) From PGC, UGC to AIGC, digital content production once again ushered in an era of rapid development
Figure 1: Significant development events from PGC, UGC to AIGC
In 1958, China's first electronic computing machine, the 103, came out, laying the hardware foundation for the development of PGC. In December 1993, "Hangzhou Daily·Afternoon Edition" was transmitted through Hangzhou's online service network, marking the official beginning of the PGC era.
The establishment of Tianya Community in 1999 marked the official entry of UGC into the public eye. At this time, there are still few users. The release of Apple smartphones in 2007 marked the advent of the UGC era, and digital content production ushered in its first growth spurt. At that time, content production shifted from being dominated by professionals to a broader group of non-professional users.
AlphaGo's victory over world champion Lee Sedol in 2016 became a landmark event for AIGC to enter application. The emergence of DeepFake in 2018 made people lament the advancement of AI technology, but also created a fear of it. In 2022, the intelligence level of ChatGPT will once again detonate the AIGC market. Under the influence of AIGC, it is expected that domestic digital content production will usher in a second explosive period of growth. Unlike the first time, digital content production will no longer be done entirely by humans, but will mainly be done by virtual digital humans.
Table 1: Comparative analysis of PGC, UGC and AIGC
(2) The amount of content has exceeded growth, and the efficiency of digital content risk control is under pressure
According to the 51st "Statistical Report on the Development of China's Internet Network" released by CNNIC, as of 2022 In December, the number of Internet users in my country reached 1.067 billion, accounting for more than 90% of the number of people with Internet access in China. Although the production cycle is moving from days to hours, people's time is always limited. After the number of Internet users approaches the upper limit of stock, UGC represented by users enters a growth bottleneck period.
Figure 2: Analysis of China’s total data volume and proportion of AIGC production content
AIGC content is generated through program training and the process is automated. As long as the computing power is in place, the number of digital people can theoretically increase indefinitely. The number of "people" participating in digital content will break through physical limits and usher in a new round of growth. At the same time, AIGC content is automatically generated and is not limited by time and energy. Content can be published in seconds, and the amount of future releases is immeasurable. At this stage, AIGC content accounts for less than 1‰ of the total content, and is in a stage of rapid growth.
(3) Content quality is gradually uncontrollable, and digital content risk control is challenged
The production of user content is diversified and it is difficult to ensure continuity and quality. However, compared with AIGC, the content is relatively controllable. In the context of AIGC, content is generated by algorithms. Due to the black box problem of the process, people cannot fully grasp the results of content production, risks are becoming uncontrollable, and the requirements for digital content risk control have also increased significantly.
1.1.2 Analysis of driving factors of digital content risk control
Policies evolve with precision. Precise policies and regulations require companies to achieve accurate identification, strengthen corporate accountability, strengthen industry self-discipline requirements, and require content platforms and regulatory authorities to use more efficient and accurate technical means to accurately identify and classify bad information so that timely action can be taken. Corresponding risk prevention measures. At the same time, content platforms and related companies are also required to assume more legal responsibilities, assume more social responsibilities in content management and risk control, and protect the legitimate rights and interests of users and social public interests. Relevant companies must continue to refine and develop digital content risk control methods in order to keep up with and meet regulatory needs and achieve their long-term development.
Figure 3: Development of digital content risk control policies
Economic development goes online. The development of the online economy has led to tremendous changes in the way digital content is produced and consumed, and it has also brought more risks and challenges. A large amount of data and information are stored, transmitted and used, and cyber crimes, false propaganda and infringements have also increased accordingly. This has also provided more hotbeds for the promotion of copyright infringement, illegal live broadcasts, obscene and pornographic content, etc., which requires effective Content risk control means are used to manage and control the content.
Figure 4: Online data representation of economic development in 2022
Diversification of social content production. In terms of industry, different industry terminology and entrepreneurial content vary greatly, resulting in differences in content risk control strategies and priorities. For example, in the entertainment and social networking industry, as one of the main fields of UGC content entrepreneurship, the platform attracts participation from enthusiasts from all provinces and cities across the country. A large amount of text, pictures, videos, and audios are generated every day, and due to local differences, various dialects are involved. Identification and high requirements for digital content risk control.
The blessings and challenges of technology. The continuous improvement of technology has provided more creation tools, automated production capabilities, and more convenient and diverse distribution platforms for digital content. At the same time, more personalized needs are constantly being met. Technology may also pose certain challenges to content. For example, with the support of technology, deep forgery technology has become more and more advanced. Digital content will also face more risks in the process of dissemination and use, which may lead to content homogeneity and lack of originality. .
1.2 Definition and classification of digital content risk control
1.2.1 Definition
Digital content risk control is a process of monitoring, identifying, evaluating and managing APP downloads, registrations, content publishing and interaction, and continuous content maintenance for normal users and black and gray products through professional technical means and business rules such as AI. By identifying and preventing illegal, harmful or inappropriate behaviors in digital content, we will protect the interests and security of users and platforms, and ultimately achieve the goal of "keeping the bottom line for digital security and raising the upper limit for digital operations." Specifically, APP reinforcement is used during the APP download stage to curb black and gray application cracking, communication hijacking, server-side attacks and other behaviors; during the registration stage, intelligent risk control, registration and login protection, channel fake volume identification, and equipment risk identification are carried out to account, Multi-dimensional review of mobile phone numbers, IPs, and devices is carried out to identify and determine the identity of black and gray personnel, as well as review illegal content; in the release, interaction, and continuous maintenance stages, by identifying the content of text, pictures, audio, and video posted by users, Ensure the legality and compliance of platform content.
1.2.2 Classification
Compared with "content security" which focuses on checking text, pictures, audio, and video violations, digital content risk control has a wider scope. In addition to content security, digital content risk control also includes application security and business security. Provide users and platforms with full-process security guarantees from APP downloading and registration to content production and interaction.
Figure 5: Digital content risk control classification
02 Digital content risk control scenarios and key industry needs
2.1 Digital content risk control demand scenarios and solutions
2.1.1 PGC+UGC: Observe the bottom line of safety and compliance and create operational growth performance
Table 2: Analysis of normal users and black and gray production characteristics
Because the content production methods, purposes and violation methods are quite different, the user groups can be divided into two categories: normal users and black and gray products. Normal users are mainly professionals who report PGC and ordinary users who report UGC.
The white paper attributes the violations of normal users to observing the bottom line of safety and compliance, and attributes the rectification of black and gray products to creating operational growth performance. At the same time, creating operational growth performance also includes public opinion management and precise customer operation strategies.
(1) Observing the bottom line of safety and compliance is the basic requirement to ensure the continued operation of the platform
Figure 6: Risk control links and key points for normal user digital content
Digital content risk control solution for normal users
Aiming at the four-step compliance content review to identify pornographic, political-related, prohibited, violent, terrorist, abusive and other behaviors and content, enterprises need to have complete machine review + human review capabilities.
In order to cope with the large number of audit needs brought about by the rapid growth of data volume, machine review has become an essential capability for enterprises to conduct digital content risk control. Machine review can effectively improve review efficiency and reduce review costs. However, machine review is not a panacea. The parts that cannot be confirmed by machine review and the review of machine review results require final checks by human reviewers. At the same time, quality inspection and statistical analysis of machine review results can optimize the machine review model, improve the accuracy of machine review, and further reduce reliance on human review. Through the four processes of "preliminary", "review", "inspection" and "tracing" of the review management system, machine review, manual review and model optimization can be efficiently divided and managed, and content security monitoring can be done well.
(2) Creating operational growth performance is an advanced requirement for digital content risk control
- Eliminate black and gray products and create a safe product operation environment for users
Internet fraud, black product group control, traffic diversion, etc. are the main forms of black and gray products on the Internet, which are very harmful to the platform. For example, diverting users to specific platforms to commit evil is also one of the common methods used by black businesses. Use comments, private chats, avatars, personalized signatures and other available UGC locations to publish inducement information, use various reasons to lure customers away from the current product, and go to the communication platform designated by the black and gray product for follow-up communication, and then complete the illegal behavior.
The network risks brought by black and gray products bring harm to Internet companies such as the loss of high-quality users, damage to the content ecosystem, reduced competitiveness, legal liability, and bad money driving out good money.
Compared with ordinary users, the management of black and gray products requires two steps: APP security reinforcement and registrant and device review on the client side, which involves more complex monitoring content.
- Public opinion is known in advance, and hot content can be controlled in a targeted manner
Public opinion has various characteristics such as suddenness and large-scale nature. The emergence of public opinion will have an impact on the APP product operation ecology and pose potential risks to APP. Judging from the specific characteristics, public opinion information is highly sudden and requires timely monitoring and analysis; the amount of public opinion data is huge and needs to be processed and analyzed through big data technology; public opinion information involves multiple fields and topics, and often has different opinions, Conflicts of interest and other issues; public opinion information usually has a strong emotional color, including different emotional tendencies such as positive, negative, neutral, etc.; there is a certain degree of uncertainty in the authenticity and credibility of public opinion information, which requires further verification and Verify.
- High-precision identification enables product marketing empowerment
Figure 7: Recognition results of a female avatar
Today's computer review is already very accurate in the field of content recognition. It can analyze and judge various types of text, images, voices, etc., and output accurate results. These results contain a wealth of information that can help companies more accurately understand customer needs, preferences, interests and behavioral characteristics, thereby providing strong support for companies to achieve precision marketing.
2.1.2 AIGC: contradictions and shields in digital content risk control
(1) AIGC improves content production efficiency and is a “sharp tool” for digital content production
Starting in 2022, from AI models such as DALL-E 2 and Stable Diffusion that detonated the field of AI painting, to conversational robots that are close to human levels represented by ChatGPT, AIGC technology has gradually entered people's vision and daily life, and has gradually become a A new engine for content production, its powerful content production capabilities have brought great shock to people.
The emergence of AIGC has also created conditions for more precise prevention and control of digital content risk control, and provided more tools and means for digital content risk control, helping to achieve more precise prevention and control. AIGC can quickly generate a large amount of training data, and can analyze and judge content of different types and scenarios, which helps improve the accuracy and generalization ability of the model.
(2) AIGC has also become an important starting point for black and gray products, making it more difficult to control digital content risks.
AIGC has reshaped the content production model, providing a powerful starting point for corporate digital transformation and personal creation, and also bringing major development opportunities. But at the same time, the vigorous development of AIGC also brings huge challenges to content risk control.
Black and gray properties are able to take advantage of the better tools they provide to reduce the cost of harmful content production and increase productivity. This has prompted the generation of harmful content to become more diverse and irregular, including the use of automatically generated text, images, and videos. Because this content is automatically created through algorithms and can be generated on a large scale in a short period of time, there may be risks and issues with uncontrollable content.
(3) Quick response and full-process response to AIGC risks
In the current AIGC environment, issues such as false information, malicious content, and online fraud have become urgent problems that need to be solved. Therefore, AIGC platform manufacturers need to establish a complete closed-loop risk control system to achieve comprehensive prevention and disposal. The system should start with pre-prevention and control, and put risk prevention and control in advance through data annotation, data governance and other methods; then, in the ongoing stage, various contents should be accurately reviewed and advanced artificial intelligence algorithms should be used to quickly screen and Determine whether there are violations; finally, in the post-processing optimization stage, effective measures should be taken to deal with violations, and through continuous monitoring and feedback mechanisms, the risk control system should be continuously improved and optimized to ensure the safe and stable operation of the AIGC platform.
2.2 Analysis of similarities and differences in digital content risk control needs in key industries
2.2.1 Overview of industry needs
Figure 8: Analysis of penetration rate of content security machine review industry
Figure 9: Calculation of the overall scale of content security review
Judging from the industry penetration rate, the overall penetration rate of the entertainment and social networking industry is already high. Among them, live short videos and social platforms are key industries for digital content risk control, and security methods are at the forefront of the industry; gaming and e-sports, as a key regulatory industry, are at the forefront. The penetration rates of middle and lower-end enterprises are both high; in other industries, the penetration rates of online education, e-commerce, media, and automobile companies have declined in sequence.
From the perspective of market size, machine review and human review are the key investment directions for digital content risk control. The overall market size of the industry in 2022 for machine review alone will be approximately 2 billion yuan. In the future, with the rise of emerging key industries such as AIGC and the increased attention paid by companies in the industry, the growth rate is expected to be around 25% in the next two years.
2.2.2 Analysis of digital content risk control needs in key industries
The white paper selects key industries for analysis, including entertainment and social networking, gaming and e-sports, finance, media, car companies, and new retail.
03 Integration of humans and machines to continue building a good corporate operating ecosystem
The overall solution for digital content risk control includes comprehensive response measures such as machine review, human review, strategic operations, public opinion monitoring, blue army services and training, etc., aiming to achieve effective management and risk prevention and control of digital content through multi-dimensional comprehensive supervision. Among them, the combination of machine review and human review can improve review efficiency and accuracy; strategic operations can formulate corresponding review rules and processes based on actual conditions; public opinion monitoring can promptly discover and respond to negative information on the Internet; and blue army services can provide professional Technical support and security; training and can help users improve digital content security awareness and response capabilities.
3.1 Machine review
Technical accuracy and service response speed are the main determinants of enterprise machine review selection
Figure 10: Survey on key procurement factors of enterprise machine review
Love analysis survey, n=100
According to the survey results conducted by iAnalytics, 95% of companies are most concerned about technical accuracy when choosing machine-reviewed digital content risk control service providers (hereinafter referred to as "service providers"), followed by service responsiveness and product price. and ease of use. Compared with brand awareness, these factors are more important and can directly affect the level of digital content security of an enterprise.
Looking at different industries, in industries such as entertainment and social networking, gaming and e-sports, due to fierce market competition, prices are relatively sensitive, but product accuracy and review efficiency are the focus of more attention in these two types of industries. The characteristics of the industry determine that social entertainment, gaming and e-sports need to quickly identify risk points to ensure the stability of platform operations. In addition to these factors, companies also need to focus on improving product quality, user experience, innovation and brand image to attract more users and customers and maintain market competitiveness. In contrast, in industries such as radio and television, new retail, and automobile companies, competition among service providers is relatively weak. Therefore, companies can pay more attention to improving product quality, user experience, innovation capabilities, and brand image to meet consumers' demands for high-end products. demand for quality products and services, and build a good brand image and reputation. Of course, price is still an important consideration, and companies need to reasonably formulate price strategies based on market demand and competition to ensure product cost-effectiveness and market position.
3.2 Human trial
Figure 11: Relationship diagram between machine review and human review
In the field of digital content security, human review serves as a supplement to machine review and is mainly used to handle content that cannot be judged by machine review, randomly check content that passes or fails both machine review and human review, and handle user complaints and appeals. According to the survey results, 100% of the companies that purchase machine review services require human review services, and there are two modes: outsourced and self-built. Companies that purchase external machine review services often outsource human review services to digital content security vendors to reduce their own labor costs and management burdens, and to obtain more professional and efficient review services. Self-built enterprises can better realize personnel management, realize rapid adjustment of human review strategies, and achieve comprehensive control of the quality of human review.
Looking at procurement needs, in addition to the need for outsourced review personnel, some companies have needs for personnel systems and personnel management services. The comprehensive service of human review can help companies achieve full custody of human review.
Figure 12: Survey on key procurement factors of corporate human review
Love analysis survey, n=100
According to the survey results conducted by iAnalysis, 98% of companies pay most attention to auditor experience when choosing digital content security vendors, followed by auditor price, personnel management system, auditor academic qualifications, and personnel management services.
3.3 Other comprehensive services
Figure 13: Introduction to other comprehensive services
A comprehensive service provided by strategic operation digital content security service providers to enterprises, aiming to help enterprises reduce audit costs and risks and improve the efficiency and quality of human audits. This service usually includes three aspects: risk analysis and assessment, risk rule formulation and optimization, and technical support and solutions.
Digital content risk control and public opinion detection is an important service in the field of digital content security. It is mainly used to detect and analyze public opinion information on the Internet, and to promptly discover and deal with various risk events and reputation issues. According to whether it is predictable or not, public opinion monitoring is divided into predictability and non-predictability.
Blue Army service is a customized service that simulates risk events and attacks the company's digital content risk control system to discover system vulnerabilities, improve system security as a whole, and help companies establish and improve digital content security systems. According to capability requirements, Blue Army data can be divided into three aspects: data richness, AI technology advancement, and security technology comprehensiveness.
Digital content risk control training service is a professional training service provided by service providers to help enterprises improve their digital content risk awareness and risk control capabilities. This service mainly includes four aspects: basic knowledge training, practical operation training, risk case training, and customized program training.
04 Content risk control practices of leading companies
The white paper introduces the origin, risk control capabilities, and service areas of NetEase Yidun, and shows the product development and capability iteration process of Yidun. This chapter focuses on the digital content risk control practice cases of leading companies in the entertainment and social networking industry, gaming e-sports industry, and automobile travel industry. Starting from the industry background, challenges and visions faced by the industry, best practice cases are used to show the challenges faced by leading companies. Pain points and solutions for digital content risk control.