Tencent is both a service provider and a user of enterprise service products. The original starting point of many products was only to solve a certain demand of Tencent itself. After continuous development and improvement and tempering of business scenarios, it finally evolved into a mature enterprise service product. This series of articles tells such a group of Made in Tencent stories, and this is the second article in the series.
Due to the reputation of its offensive and defensive capabilities, Tencent's security expert service team often needs to play the role of an "old Chinese doctor" in dealing with difficult and miscellaneous diseases, and solve various difficult security problems for enterprise users.
All kinds of attacks on enterprises emerge one after another, such as ransomware, supply chain attacks, data theft, mining Trojans... Tencent’s security responsibility lies in the urgent need to save customers. There are two problems, one is that the number of attacks will only continue to increase, and the energy of experts is limited; the other is that experts' offensive and defensive experience cannot be reproduced and passed on. The combination of the two results in the bottleneck of expert service capabilities. If there is another global cybersecurity incident like WannaCry, many companies may be helpless. And with supply chain attacks prevalent today, a repeat of WannaCry is very likely.
how to solve this problem? Tencent Security's answer is MSS - Managed Security Service.
Usually, a typical security vendor develops a new product through the following process: first conduct market research and customer pain point analysis, find a number of opportunities, and then match products suitable for your company based on your judgment of its own advantages and capabilities. But the situation in Tencent is slightly different. In a sense, Tencent itself is a customer of Tencent Security. Therefore, in many cases, the birth of Tencent security products is based on solving Tencent’s own demands and gradually matures—Tencent Security MSS is a typical example. such a situation. This starts with the security construction of the Tencent Cloud platform itself.
Since its establishment in 2016, Tencent Security Yunding Lab has been responsible for the platform security of Tencent Cloud. In 2019, with the acceleration of the digitalization process, the core assets of enterprises are on the cloud, and the number of attacks on the cloud has increased significantly, and Yunding has increased again. A new responsibility: to undertake the security governance and emergency response of tenants on the cloud. "Although the international practice is that the cloud platform and cloud tenants 'share responsibility', first, there is no consensus on this concept in China, and second, many customers start relatively late in informatization and lack the ability to deal with risks on the cloud, so At that stage, the cloud platform needs to undertake some tenant risks and vulnerability containment work.” Li Bin, general manager of Tencent Cloud Security, introduced.
When I first took over, due to lack of experience and tools, Yunding Lab fell into the ocean of vulnerability wars, and everyone was operating at full capacity, even so it was still very difficult. "Tencent Cloud has 80 to 90 data centers in more than 20 countries and regions around the world, and the number of paying users exceeds 1 million. To respond to the security issues of such a large cloud platform and tenants, the number of security incidents faced every day exceeds Hundreds of millions." Li Bin said.
However, a coin has two sides. If you look at these massive attacks from another perspective, it can also be a "safety gold mine". "Based on Tencent Cloud's global nodes, we can see the global security situation. Many attacks today are organized and large-scale, and they will also be tried on a global scale, through big data analysis and long-term artificial intelligence tracking. , and eventually we will know what is the source of the attack, which malicious nodes are controlled, and what kind of attack happened to whom.”
In the experience of escorting the Tencent cloud platform, Yunding Lab found that although there are a large number of attack events on the cloud, they are actually infinite repetitions of limited types of attack events. For example, most of the security problems are caused by primary configuration errors. , Key leaks and known vulnerabilities are triggered by events such as key leaks accounted for 22%. On Tencent Cloud alone, thousands of incidents occur every year because developers write keys in code and share them on open source platforms. information leakage incidents.
The Yunding team made a rough calculation and found that only some technical means are needed - such as extracting the abstract features of each event, and fixing the corresponding expert processing process with an automated workflow method, through machine automation Batch processing - Eliminating these common problems can increase the efficiency of handling security incidents by 80%.
Based on this observation, Yunding Lab has built a series of tools to integrate the capabilities of various asset mapping, vulnerability scanning, configuration analysis, leak monitoring, and work order management within the group and at home and abroad. At the same time, it also linked Gitbub to create a key The automated mechanism for leak response, from discovery to notification of user disposal, can reduce the entire cycle to the minute level—while the traditional processing cycle may be as short as days, and the premise is that key leaks have been discovered.
For those highly repetitive attacks, Yunding adopts a method of "building blocks" to reduce:
Step 1: Several engineers extracted and encapsulated the general module capabilities required for incident handling, which included dozens of modules including common security scans, work orders, enterprise IM notifications, and open APIs;
Step 2: How to build these "building blocks"? Although there is already SOAR on the market, the SOAR engine is opaque on the one hand, which is not conducive to the debugging process, and the impact on the business is unknown. A set of process orchestration engine arranges these dozens of "building blocks" according to different event handling processes to achieve once and for all.
This work greatly improves the efficiency of the team, and the experts are freed from repetitive labor to do more important work. "Through the automated workflow, the MTTD/MTTR (average detection time/average response time) of events during the period of business launch and re-protection is greatly reduced." Liu Zhigao, senior security expert of Tencent Yunding Lab, mentioned, "With this ability, we will have more energy to focus on problem solving itself, and figure out what kind of ability and process can be used to better solve or close a certain type of event. What remains to be done is to use the workflow engine to quickly implement the solution and practical operations.”
During this time period, the network security industry is also undergoing some significant changes. Around 2020, after the peak of the last round of IT and informatization construction has passed, enterprises' demand for network security is no longer mainly about buying and buying, but how to make the purchased network security products and tools really effective, After the rules came into effect, many research institutions agreed that service-oriented network security products will be a development trend in the future. Zeng Yongjiang, director of security services at Tencent, judged that platform delivery and security operation trusteeship will be the mainstream direction for enterprises to deal with future security challenges.
This means that these tools built by Yunding can not only be used by various security operation teams within Tencent Group, but also have the opportunity to become an enterprise service product and be used by more enterprise customers.
But if it is to be used as an enterprise service-level product, it is not enough to just do this. In many exchanges of re-insurance and customer emergency response, Tencent's security expert service team found that many customers have a deep-rooted pain for security management-not only the system has been compromised, but also in many cases, the security department has no idea how the system is. It is impossible to know where the captured ones are and where the progress of the disposal is. In front of them is a black box. "One person handles the attack while a group of people watch" is the norm at the customer's emergency scene, because there is no tool to allow them to understand the progress of the disposal, and the customer can only relieve anxiety by being "on the scene".
Yunding Lab decided to further improve this tool. The first is to introduce more advanced security capabilities on the platform, such as business baseline behavior detection, vulnerability intelligence, and ASM (Asset Mapping)—many of which are unique tools developed by experts from Tencent’s security research team; in addition, BAS ( Intrusion and attack simulation), Tencent Security Joint Laboratory has assembled first-class security attack and defense experts in the industry. They are familiar with various attack methods. In order to achieve the purpose of "promoting defense with attack", the attack methods in actual combat are written as "attack scripts" , after orchestrating with automated workflow, to verify whether the customer's WAF can be bypassed, whether the firewall policy is strict, and the effectiveness of the host security agent function - to help users discover deeper security problems from the perspective of security effectiveness verification .
Tencent's security service team conducts more than 100 re-insurance sessions and conducts more than a dozen offensive and defensive drills every year. During this process, it has accumulated a complete set of procedures. "Before a re-insurance project starts, when does the security team need to enter the site to understand and analyze the entire system architecture, when to do system security testing, code testing, when to conduct red-blue confrontation drills, and emergency drills? Round, how long do we need to stay for loophole rectification, when to conduct the final retest, close the system, and start to shift to the continuous monitoring stage. In the monitoring stage, we may have a series of tables and processes, what events trigger what kind of signals , How to deal with it, which ones are linked to the intelligence center, which signals should be distinguished by the intelligence center for timely emergency response, etc.," Li Bin said.
And these valuable expert experiences are also deposited in Tencent Security MSS in the form of workflow. Enterprises only need to configure parameters according to their actual IT conditions to carry out security operations. On the MSS platform, enterprises can "see" their security status and processing progress.
The essence of network security is confrontation, and the form of confrontation is always changing. For a good MSS service, in addition to a series of automated tools, it is also necessary to have an "expert think tank" to maintain attack scripts, vulnerability databases, and intelligence in real time, which is exactly the strength of Tencent Security. With its own massive business volume, Tencent naturally needs to maintain awareness of various attack situations.
There will always be an unfair contest between offense and defense in network security. For an attacker, an attack is like throwing a handful of sand into the sesame, which is easy; while the defender needs to find the sand out of the sesame. Digitization has swept every industry in a devastating way, and all enterprises must go digital, but is every enterprise ready? For those companies without a professional security team, how can they resist such an unfair contest?
MSS, maybe an answer. For large enterprises, they can use MSS to fill in the weak points in the construction of the security system, or use the platform tool capabilities of the MSS team to improve their own operational capabilities, and get a strong support outside their own security team; for business hosting in public For small enterprises on the cloud, they can obtain the same security services as large enterprises at a very low cost.
MSS (Managed Security Service) originated in the 1990s and was first implemented by operators, cloud vendors and security vendors in North America. In recent years, it has also shown a significant growth trend in the Chinese network security market. According to IDC data, the security hosting service market will become the largest sub-market in the global network security product and service market in 2020, accounting for more than 20%; from 2021 to 2025, China's security hosting service market will maintain a compound growth rate of 39%.
Hundreds of rivers flow into the sea. For Yunding Lab, maybe it didn't expect to be a next big thing at the beginning. It just started from simply solving the pain points of itself and customers, but it also moved towards a rising blue ocean market in the end.
"Currently, Tencent Security MSS has served many companies such as Digital Guangdong, Canton Fair, China Overseas Real Estate, FAW-Volkswagen, Changyou, China Travel Group, etc., and guaranteed zero accidents at major moments such as the seventh national census and CCTV Spring Festival Gala re-insurance. , 0 risk.” Zeng Yongjiang said.