At 19:30 on the evening of July 18, 2023, the Software Supply Chain Security Technology Exchange Group (OSCS) organized the first online closed-door seminar. In this seminar, we received 71 people from various companies concerned about the software supply chain For the registration of security technical experts, according to the requirements of the seminar participation rules, we reviewed the applicants, and finally more than 35 security experts from the Internet, finance, operators, software manufacturers, and state-owned enterprises participated in the seminar that night.
"The Value of Software Supply Chain Security Construction" theme sharing
The first part of the seminar was a theme sharing by Zhang Huapeng, founder of Murphy Security & OSCS initiator. Zhang Huapeng used three formulas to share how to think about the construction value of software supply chain security.
Next, each factor in the formula is disassembled and analyzed one by one. First of all, the risk of software supply chain security must be managed. It is clear that the security assets of the software supply chain are the most critical link, and the asset ledger is the object of risk that needs to be managed and controlled. The total collection is also an important denominator for measuring the value of its construction in the future.
The collection of software supply chain asset ledgers can be collected and counted from three dimensions: office software, self-developed software, and outsourced production software.
Data breaches that may be caused by attacks on the software supply chain are still the most concerned risk point for enterprises.
The second is the satisfaction of software intellectual property compliance. Both software users and software producers have very important code intellectual property compliance management issues.
In addition, everyone is most concerned about not being compromised during the hvv period. After all, the most commonly used attack method in offensive and defensive drills is to exploit vulnerabilities from the software supply chain.
In addition to the annual large-scale offensive and defensive drills, the risk that everyone is most concerned about is actually the risk of being blackmailed due to common vulnerabilities in the software supply chain.
For middle and high-level managers within the enterprise, software supply chain security is a new technical direction. In the new technical direction, there are some cutting-edge technical explorations and practices, and output to the industry is bound to enhance their influence in the industry. A very good way to force.
Now that we have finished talking about the benefits of construction, let’s talk about the cost of construction. Most of the time, this cost is mainly operating costs. In fact, the operating costs are very controllable when integrated with mature software supply chain security capabilities.
The first is the cost of deploying and using the security capability of the software supply chain. If the cost is to use a mature commercial product, the cost is very controllable. If it is self-developed, it is very high in terms of engineering and vulnerability knowledge base operation. .
In addition, the cost of dealing with security issues in the software supply chain, how to quickly repair vulnerabilities and avoid compatibility issues caused by software upgrades, and filter out vulnerabilities that will not actually trigger will be the core challenge.
Regarding the daily security technology operation is a very important means to ensure the effect of continuous operation, it is necessary to continuously improve the security awareness of developers and clarify the security governance requirements of the enterprise software supply chain.
Share some cases of enterprises on software supply chain security governance
Cost-effective governance solution
Systematized case studies
Excerpts from the workshop that collected questions before the meeting
Q1: How to ensure the correctness of the vulnerability data
? For many CVE vulnerabilities, many data fields are missing. For some vulnerabilities collected from various public channels, some data information may not be accurate. The core of this question is how to ensure that the data in the entire vulnerability database is accurate when doing security testing of supply chain tools? Including some of its fields, fixes, scope of influence, etc.
| A boss of an Internet company: In the past, a lot of vulnerability information was received, and after the collection, some students were specially arranged to process and process the data to calibrate the data.
| A big boss of a software manufacturer: We actually don’t have much accumulation in this area, mainly using the group’s capabilities. We have some precipitation in the vulnerability, for example, after the SCA is identified, and then the vulnerability repair after colliding with the vulnerability library, our team will do some extraction of the conditions for exploiting the vulnerability, and deposit these things as much as possible for the development to see. Converting to some developed languages to help them better fix bugs is actually reducing their workload.
| A big boss of a software manufacturer: We will find that the external vulnerability library has several pitfalls, such as CVE vulnerabilities. Some time ago, we communicated with the security team of a financial company. Many of the CVE vulnerabilities identified by the component impact range are very inaccurate. In addition, the log4j vulnerability actually affects different modules. For example, there are log4j-core, log4j-api, etc. There are many different modules, and the modules affected by the vulnerability are different, but only one log4j is written in the identification information in the CVE. When you do vulnerability matching, it may cause all log4j-related vulnerabilities to be matched. But in fact, the modules affected by different vulnerabilities are actually different. This is a relatively typical pit. As mentioned earlier, this pit may require manual analysis of the actual impact of each vulnerability. This analysis can guarantee some accuracy of the subsequent matching results, which is one of the points.
Another example is the repair plan. In fact, a lot of data is not available. It is even said that the accuracy of the repair plan for each vulnerability must be calibrated in the data collected from different data sources.
More than ten years ago, Ali's vain and prodigal son focused on Java vulnerability research. He mentioned how to dig out Java's 0-day vulnerabilities. Some 0-day vulnerabilities in Java will only trigger this 0-day when a certain configuration is required. Some people will teach others to do that wrong configuration in the community. Many of the data collected outside, and the repair solutions given are actually wrong. In fact, I don’t think there are any special skills. Each vulnerability may need to sort out its real impact, including the repair plan of the vulnerability may need to do some verification. When it is pushed to R&D for repair in this way, the professionalism and authority of ensuring safety will not be challenged.
Q2: Threats and preventive measures of software supply chain attacks, supply chain transparency and traceability, supply chain security and compliance .
| A big boss of a software manufacturer: The first question is actually some capacity building of the supply chain, and the second part is actually more about the construction of the asset ledger of the software supply chain. The third part is more about compliance. One is that going to sea may be more about license compliance. In addition, some domestic self-controllable Xinchuang compliance. Next time we can do a seminar on this issue.
Q3: How to reduce false positives of supply chain vulnerabilities
| A big boss of a software manufacturer: To reduce false positives I understand that there are two things to do.
The first thing is to identify the accuracy of the supply chain components. Some of our past practices in this part are mainly to identify. For example, many projects like Java have marked the scope as test when configuring in pom.xml , some are runtime and so on, it has different tags. Different tags correspond to components that will be used in different compilation environments. This is a point that needs attention. For example, the components used in the test environment may not be affected online, so this part should be excluded when publishing vulnerabilities.
The second is to consider the accuracy of component analysis when doing supply chain component analysis. For example, Python, as a package management tool, may not identify the exact specific version when doing package management dependencies. If you go to generic matching, it may lead to false positives. This is the most basic false positive. Of course, there are also Many other situations.
The second main situation is the verification of the accessibility of vulnerabilities, which is done a lot overseas, but very little in China. According to our statistics, more than 95% of the vulnerabilities are false positives. For example, fastjson only matches this version and is vulnerable, but in fact, will the fastjson defect function method be triggered in the code? Including that the function class method triggered by it is called in the code, but whether its taint can pass in some controllable parameters, or even some loopholes, it depends on the dependent environment, such as the JDK version, then it will be triggered. These are the points that need to be considered when reducing false positive detection results. Overseas, there is a relatively mature foundation called the analysis of vulnerability accessibility. We are currently doing more function-based methods. This kind of code-level identification, and recently we are also doing this kind of matching that relies on environment identification to reduce the unreachability of vulnerabilities. Case.
It is also mentioned in the above answer: Is the development of the demonstration vulnerability really affected? Sometimes R&D understands better than security that vulnerabilities in the code will trigger this function class method, but it is not called, which will lead to inaccurate results that say that R&D will question security.
| A big boss of a software manufacturer: I agree with the two ideas just mentioned. In fact, there is another way to link vulnerability databases together. However, we don’t have much experience in tool modification because we have to use the company’s products. What we maintain is actually the exploit condition of the vulnerability. The other is to maintain a false positive vulnerability database and reversely optimize our vulnerability database for false positive vulnerabilities.
| A boss of an Internet company: I am a little confused why everyone has false positives? Except for fastjson and log4j, which have a relatively large impact range, other high-risk vulnerabilities affect at most dozens or hundreds of assets, and the repair cost is not high.
| A big boss of a software manufacturer: In fact, there are some companies that need to upgrade some components with complex dependencies, but the system is no longer maintained. The newcomer just took over to maintain the old system and upgrade it. There is a compatibility problem, so I am not willing to fix it.
In addition, there are some companies whose research and development are relatively strong, and the progress of the project is relatively tight. In many cases, the psychology of research and development is that I don’t want to fix loopholes, and I don’t agree with security asking me to fix loopholes every day. Question: Does the vulnerability really have an impact? If R&D happens to know security very well, and it is judged that the code logic will not be triggered by this vulnerability, if such doubts arise, security will ask him to fix the vulnerability, and his cooperation will definitely not be high. Because only version matching is done, it is actually just a potential hidden danger. It does not mean that this vulnerability will definitely be triggered, or that it will affect the online code. The R&D departments of many companies will question this matter a lot.
However, state-owned enterprises or companies like XX Cloud have high requirements for management and control, especially for national infrastructure. If the company has a system, it must be changed, and the degree of cooperation in research and development will be very high. But most of the pan-Internet customers are actually very difficult to solve this problem.
Q4: How to effectively build and promote? How can it fully function after it is built?
| A big boss of a software manufacturer: Let's not expand today, but first discuss the main common methods. The first operation and promotion, such as the experience of XX cloud, will basically come out with systems, norms, and requirements. XX Cloud has put forward the requirement of clearing the vulnerabilities, and the vulnerabilities of all the basic containers delivered to customers are required to be cleared. Develop systems, assessments, and rewards and punishments to promote and implement the system and provide supporting tools for research and development. To do the assessment, the promotion of XX Cloud is relatively strong. It is mentioned in the question that it will be suspended if it is not approved by the development. I personally feel that the company's control requirements are not so strict.
Q5: How to define the main contradiction and benefits
| A big boss of a software manufacturer: It used to be event-driven, but now there may not be so many events every day, so doing this is equivalent to the loss of the original driving force. If you do this routinely, how to evaluate the long-term benefits.
I used to have a very big change in my thinking and concept of security when I was in Keike and when I was in Baidu. It may also be because the boss of Keike came from CCB. The typical thinking of Internet companies is that if a big event happens, I will take the opportunity to build capabilities. However, in banking or in many key industries, its construction is more inclined to reduce the overall risk of the entire company, for example, to identify the entire asset: such as the office network, production environment, and all systems including outsourced systems. The assets are sorted out, and will be analyzed after sorting out. Especially at the level of CTO or even CEO, the concern is whether they can clearly explain the overall status of the entire company in the direction of software supply chain security? For example, how many assets are there, and how many hidden dangers do these assets have? Can hidden dangers be solved through our historical discovery of problems? For example, how many similar vulnerabilities have been discovered by all SRCs of Meituan, or how many attacks have occurred, how many attacks have occurred during the network protection period, and how many risks are associated with global assets from all dimensions. It is even said to predict which key assets from the analysis results, and the loopholes corresponding to key businesses are the company's biggest potential risk. Next this year, based on the above results, some systems with the greatest potential hidden dangers will be targeted for this year’s governance. Next year, this requirement may become even higher. When there are more resources, then the second phase of governance can be done. This is to analyze the main contradiction of the company from its own perspective.
Another dimension is horizontal comparison. The logic of horizontal comparison is that in the same industry, if my company's security construction is better, it may not attack me, and then attack other friends. For example, how is everyone in the same industry doing? To give an example of ants, ants themselves have financial attributes and Internet attributes, so ants have a strong ability to continuously ensure the leadership and depth of security capabilities.
Another aspect is to clarify the overall risks. On the premise of making it clear, the leader recognizes and determines that these important risks must be managed, and then resources will be invested. If the leader does not recognize that risks need to be governed, at least clarify the direction of responsibility. I think this is a very important logic for security. It is necessary to explain to the leader how many potential risks the company currently has, how much resources need to be invested, and to what extent it can be governed. As for the extent to which governance is required, it is left to the company to make decisions. But if it is not clear, as an employee, the responsibility is relatively great.
| A boss of an Internet company: Your premise is to first build the vision of assets. Assets must be built, but subsequent vulnerability management and control may not necessarily be considered a risk. From the perspective of cases, there are relatively few cases that actually exploit vulnerabilities, including those that actually use log4j to generate attacks. At present, there are occasional requests for vulnerability scanning, but the risk is not that high in all fairness.
| A big boss of a software manufacturer: It depends on the specific situation of each company. Meituan actually invests a lot in this area, and the basic security is relatively good. So I think this place needs to assess whether there is a global risk of stock. If not, then the point may be that when a new 0-day or a new vulnerability is discovered, it is enough to ensure good governance.
| A boss of an Internet company: To share a vision, I found that after I found no loopholes inside, there are actually many components developed by myself that require upgrades.
| A big boss of a software manufacturer: I learned that an Internet company is currently doing this. Their case is to be a source security gateway. All internal second-party components of the private source card, internally developed common components must be uploaded to the warehouse of the private source, and it must be ensured that there is no security problem when uploading all second-party components.
| A boss of an Internet company: My vision is not from a security point of view. The pain point I found is that the research and development of the second-party components is troubled by the low version. The low version may be unstable and some functions are not compatible, so it needs to be upgraded, but it is very difficult to push it by yourself. In fact, we can help the company's components to perform unified version control, which may be a benefit other than security.
Q6: How can AI help companies predict and prevent security threats in that supply chain?
| A big boss of a software manufacturer: Let me share some of our recent explorations in combination with AI. Share some of our thoughts on a small scale. Combined with AI, we are not making predictions, but mainly solving the problem of software supply chain security.
The first is doing things related to the selection of open source components. We found that the security of the software supply chain cannot be simply understood as a security issue. For example, when selecting a certain component or upgrading a certain component, not only the security issues must be considered, but also the scalability of the component; the health and maturity of community maintenance; the adaptation of scenarios (for example, in financial Can the industry scenario be adapted? Can the transaction payment scenario be adapted?). Therefore, in terms of component selection, we use AI to do intelligent analysis of open source component selection. Extract and summarize the content, and analyze the scenarios (what kind of open source components are suitable for use in which scenarios).
Another point is to generate repair code, or to do compatibility evaluation of software upgrades. We found that when upgrading a software, everyone will consider whether there will be compatibility problems after the upgrade. It is necessary to analyze a large number of differences between different versions of this component and some of its issues and release notes. In this regard, we use the ability of summarizing and refining AI large models to do some exploration.
Q7: Analyze dependency status and threat level
| A big boss of a software manufacturer: As mentioned in the previous ppt, we can analyze all the dependencies of all software in all code warehouses, and then correlate the severity of the vulnerabilities, and do the impact analysis and data statistics of the vulnerabilities. We did one thing in this regard: remodeled the CVE threat and divided the threat into three categories (strongly recommended fixes, recommended fixes, and optional fixes). It is strongly recommended to sort out the vulnerabilities that have POC, EXP, and the risk level is high and serious, and regard them as truly influential. The analysis of the threat level can also correlate how many assets are affected by a single vulnerability, or the importance of the business, so as to perform hierarchical processing and generate reports. I think this is a very important point for value analysis.
Q8: How to strengthen developers' recognition of the value of software supply chain security construction
| A big boss of a software manufacturer: I think it is very important for enterprises to do software supply chain security construction to rely heavily on developers to participate in this work, such as developers fixing security vulnerabilities, so it is necessary for developers to recognize the value of security construction. In this process, I think two points are very important.
The first point is to share real cases within the enterprise through some events, even through some events and cases within the enterprise. This is a more effective way. For example, loopholes in log4j and fastjson lead to security incidents in the company, or through the attack and defense simulation of the Blue Army to conduct offensive and defensive drills, it is found that the code can be attacked, resulting in company data leakage, or the risk of service intrusion.
The second point is that after the security risks are managed, not only do you need to report to the security boss, but you also need to do positive publicity within the company. Positive publicity allows developers to understand the true impact and risks of security. For example, internal case sharing is carried out on a regular basis: which developers and which departments are in the development process, when doing the selection and access of open source components, they have already improved the security of the project very well. Even which business departments have solved which security issues every week and every month, the timeliness of the solution, and how effective the solution is. Another approach is to share and train the results of the construction to all developers involved in this work every week or every month, such as the publicity of risk data and internal interview benchmark cases. This can also better establish developers' recognition of the security construction value of the entire software supply chain.
Q9: How to quantify and reflect the effect of supply chain security governance?
| A big boss of a software manufacturer: There are some key indicators, such as: the timeliness of vulnerability disposal, the average number of repairs for each vulnerability, the number of risk reductions related to asset ledgers, the timeliness of sudden 0-day vulnerabilities or supply chain vulnerabilities, and the timeliness of a single event. Improvement in processing efficiency, etc.
If it is a partial key industry, the completeness of the process system, the best practice in the industry, and the selection of excellent topics for paper patents can all explain the value directly and indirectly.
Brainstorming
| Security chief of a logistics company: I transferred from an older traditional company to a more Internet-based company. The main problem now is that after the SCA process is established, the boss's central idea is to reduce costs and increase efficiency, and try to do it internally to save resources. When doing SCA, our R&D team will be more cooperative in repairing vulnerabilities, but there must be a more precise repair method. For example, high-risk vulnerabilities or fastjson-type agreements must be repaired. We have also researched many products, and the Murphysec product has a function that the code is actually called, which can also be used as a condition for precise repair.
The second problem is how to invest in the case of low resources. Now that the process is up and running, high- and medium-risk vulnerabilities can basically be resolved, and some relatively good interactions and norms have been formed between R&D and security. But when the boss is unwilling to invest resources, how should we measure and respond as a security. For example, what size the business has grown to, what level of supply chain security should be achieved, or what level of personnel should be invested in order to achieve a better effect. It seems that there is no measurement ratio or standard in the industry. So let's discuss together how you do it?
| A big boss of a software manufacturer: I feel that this problem is a bit similar to the problem of Meituan. After a certain level of governance, the driving force may no longer be there. Should we invest more?
Let me throw a brick first. I have tried some more effective methods before: the
first point is the comparison of companies in the same industry and the same scale. For example, comparing Meituan with Baidu and Tencent, how many people in the same industry have invested in this direction, and to what extent. For example: Many bugs in ants can be repaired automatically, and ants require all repairs and so on.
The second point is to return to the perspective of actual internal risks of the enterprise and expose risks from the perspective of offense and defense. After detecting the vulnerabilities, find some real vulnerabilities for infiltration, or find an important vulnerability in an important business system for internal attack and defense through a method similar to the red-blue confrontation, and do a practical demonstration to find out how much data leakage will be caused by the vulnerability.
Around 2008, Baidu did not pay much attention to security and did not have many resources. At that time, the first thing Jianxin did in Baidu was to attack the core internal system and report the risk situation to the boss. This is very intuitive for the boss. An important way to see risk.
| Security chief of a logistics company: This is indeed more reasonable, and we are also applying it from the perspective of attack and defense, but there are basically no loopholes in supply chain security. Compared with the same industry, our investment is indeed not enough.
| A security boss of an operator: Just to add, in the initial stage of our software supply chain, the reason that prompted us to invest the most in the security of the software supply chain is related to the entire stage. First of all, the traditional approach is to keep doing protection and testing. For example, the log4j proposed by the big boss of Meituan, it is inevitable that there will be non-standard applications in an enterprise. I have invested in identification in this aspect, but I can't do it 100%. Then we have to simulate the attack, and then do the detection in this area, etc. to make up for the lack, and finally reach a standard.
We found that the investment in this area is very high. When a vulnerability occurs, such as log4j, when we analyze the cause and exploit the chain from the perspective of the attacker, there is no way to quickly confirm the impact area, especially when there are more than 2,500 assets affected by log4j at that time. There is no way to quickly get the business to fix and prioritize.
So what we did at that time was to dismantle the attack chain and the exploit chain, which took about six steps. The six-step exploit chain involved DNS and the specific code that was executed. In short, it is a step-by-step dismantling in these six steps. At the network layer, when the code is running, set restrictions and cards in other links layer by layer, invest some capabilities in the intranet, and prevent these reverse connections from going out, etc. Wait for these operations. From the perspective of incident response, the cost of investing in this matter is very high, and the business impact will be relatively large. In those years, many nuclear-bomb-level vulnerabilities broke out, and the business itself could not withstand this kind of torture. They would also take the initiative to ask whether there is a better way to solve the problem in advance, so we can appropriately give the business Find some trouble.
During this process, we found that we have invested some energy in shifting safety to the left, including three synchronizations, including control of non-standard applications from procurement, and testing of new product launches. In this set of logic, if the detection capabilities and means of supply chain security, or the reserve of the knowledge base are added, the cost of incident response will actually be greatly reduced and the efficiency will be greatly improved. of. From this perspective, the ROI return on investment ratio is definitely positive.
| A bank security boss: When the construction of the supply chain reaches a certain stage, including black, white, gray and SCA, what is the next construction direction? I would like to ask everyone. In addition to continuously optimizing the vulnerability discovery capabilities, vulnerability repair capabilities, and vulnerability accuracy of construction tools. What are the other directions of construction?
I have a few ideas, you can look at the feasibility. On the one hand is AI. I think AI can reflect the value of security in the future. The second aspect is the audit of application security data. I have done some construction on application data security audit this year. After investigating many top manufacturers, there is a problem that everyone is based on the traffic layer products, mirror the traffic, and then check whether there is any sensitive data transmission about the API in the traffic. However, the data encryption on our side is relatively strict. If there is no way to decrypt it, this kind of equipment will be invalid. In practice, it is found that IAST supports decrypted traffic very well. I don't know if these two aspects can be combined.
The third aspect is the verification of unreachable vulnerabilities. When we were fixing bugs, our scenes were basically head-slapping. On the one hand, it depends on which loopholes have been repaired in the industry, and on the other hand, it depends on whether the loopholes can be exploited and whether they are threatening. Therefore, the analysis of unreachable vulnerabilities is also a direction we will consider in the future. See if you have other good ideas, you can refer to them.
| Security boss of a company going overseas: We have established the process now, and recently we just set the goal for the second half of the year. Our business scenarios are more complex and include many things: IoT, cloud, APP, embedded, etc., as well as various smart home hardware, which can be built in many directions. Our next direction is to find some businesses with insufficient coverage in the past, and focus on threat modeling. For example, some of our external businesses require developers to access our platform, including things like the SDK. From the perspective of developers, we will find out whether there is any problem in the SDK. In the process of docking the cloud platform and APP embedded hardware development, we will provide some IDE plug-ins, but these things may not have been covered deeply enough in the past, and this will be our focus later.
| A security chief of an Internet company: We are also doing AI-oriented construction recently. For example, when we are doing SDL, it will involve process promotion or tool QA issues. General QA questions, or suggested fixes, such as a security SDK. We will transfer part of the work similar to customer service to AIGC.