DevSecOps and Supply Chain Security
Many enterprises have established a DevOps process, but security is basically outside the process and has not been integrated into the traditional DevOps process. As a result, security has always been the bottleneck of agile delivery. In this article, we will talk about security issues in the R&D process from the perspective of DvSecOps and software supply chain security.
01 DevSecOps - the cornerstone of supply chain security governance
In the context of many enterprises seeking DevSecOps solutions, the concept of shifting security to the left is becoming more and more important. The White Paper of the U.S. Department of Defense also mentioned that the core of DevSecOps implementation is the security left shift, but the security left shift emphasized in it is to conduct security testing in the research and development stage, such a left shift is not thorough. It is Microsoft's SDLC concept that really implements the security left shift more thoroughly, emphasizing that security should be involved in an earlier stage .
After continuous practice in recent years, DevSecOps has achieved a relatively in-depth security shift to the left. It does not just do security testing in the R&D phase, but considers security in the demand phase, and even intervenes in security in the project approval phase.
The value of safety shift left cannot be ignored, we know that the cost of finding and fixing problems at different stages grows geometrically. If the cost of finding and fixing a problem in the R&D phase is "1", the cost of fixing the problem in the testing phase is "2", the cost of fixing the problem in the release phase becomes "10", and the cost of finding and fixing the problem in the running phase is "10". The cost will be "100", and the impact on business operations is even incalculable, because some security issues are the lifeline of an enterprise.
02 Difficulties in Implementing Supply Chain Security Governance
Supply chain security governance is decentralized . Problems are prone to occur in coding, compilation, deployment and other links.
The cost of R&D and repair is high, the repair of security problems is lagging behind, and frequent regressions .
Supply chain security governance relies on multi-tool capabilities . It is difficult for a single tool to cover all the governance objects of supply chain security. Now it is mentioned that the supply chain governance tools are basically SCA, but the operating system and middleware are all governance objects that need to be considered in the supply chain governance process.
Security is silos not processes . The security tools are used independently, and security vulnerabilities only exist in the security tools, which are not connected with the R&D, testing, security, and operation and maintenance processes, and are not integrated into the process, which leads to the intervention of security work only in stages, affecting the delivery rhythm and cycle.
Lack of full-process governance experience . Based on the theory of safety barrels, making up for the shortcomings of safety, and achieving comprehensive governance of supply chain safety also depends on safety experience
There is no supply chain asset management . This piece is mainly for the SBOM solution. SBOM is just a static software bill of materials. If a 0day security problem is found, the efficiency of scanning all code warehouses is obviously too low, and whether the problematic software can be quickly displayed after scanning is also a need. To consider, from the perspective of asset management and post-mortem inspections.
03 Delivery pipeline
The CI/CD delivery pipeline, that is, the pipeline, and the current security tools are basically connected to the pipeline in the form of orchestration. Because there are many types of security tools, it is impossible to log in to make a scanning plan for each one. In addition, many core systems have realized service-oriented transformation. The workload of creating detection tasks for each service application on each tool is very heavy on the security team. It is almost impossible to complete, so the current security detection tools have risen to the dimension of orchestration.
It is worth mentioning that although multiple detection tools can be launched simultaneously in the pipeline for detection, the pipeline is not organized in this way in actual situations. If it is the assembly line of the R&D test environment, all security inspections are usually left behind, because what needs to be done at this time is to quickly build the application for deployment, and then let the R&D and testing verify the functions. After verifying the function, look at the vulnerabilities detected by the security detection tool, because after deployment, the security detection tool starts detection at the same time, and the functional test and security detection can be completed almost at the same time.
If it is for the on-line pipeline, all security detection tools will be arranged as early as possible, and at this time the security detection tools must have the ability to be stuck in safety, that is, set a safety threshold in each tool, for example, for SCA tools, define It cannot have more than one critical component, otherwise subsequent pipelines are not allowed to execute. In this way, the quality consistency guarantee channel can be realized. The so-called quality consistency guarantee means that no matter what kind of business research and development background is in, the products delivered through the assembly line meet the quality requirements every time. This can form a culture, that is, when developers submit code, they will think about whether they can pass the security check point, so the way of thinking about security issues can be changed.
In addition, many current solutions for security tools are to write a script in the pipeline, submit the code to the white box detection tool for scanning, and then get the detection results. However, many microservice applications now range from dozens to many. Hundreds of thousands. In the face of thousands of applications, the code is submitted to the pipeline for testing once a day, and any white box tool cannot support it. Therefore, when security tools are put into the assembly line, the technical requirements for normalized and automated execution of security tools have also undergone fundamental changes. It is not simply writing a script in the pipeline to call the detection interface of the detection tool and then obtain data, but to consider that the detection capabilities of each detection tool are distributed.
Therefore, in the DevSecOps pipeline, there can be many places that are different from the current implementation ideas of enterprises. We may need to evaluate whether the current pipeline can support the future business development of the core system, and whether the efficiency of the entire security implementation meets the requirements . Regardless of the state, the software quality must comply with the acceptable range of risk.
DevSecOps Security Capability System Construction
DevSecOps has two development directions, one is One by one and the other is All in one.
One by one is a solution based on Jira, Jenkins, and GitLab, which are combined to form an automated process. All in one is an integrated platform. It does not simply integrate tools, but redevelops them on the platform.
01 DevSecOps integrated platform
DevSecOps is synonymous with efficiency. The reason why many companies have been advocating it is because of the complexity of supply chain governance. In addition to supply chain governance, cloud-native security, application security, data security, and compliance security must also be considered. It is more difficult to do these security tasks well. In 2021, the White Hat organization released a security research report on the Internet applications of the public sector in the United States. The report shows that more than 60% of the enterprises in the manufacturing industry have exploitable vulnerabilities, and the exposure window period is more than 365 days. 40% of the financial industry The above companies have such problems . It can be seen that the security situation of the domestic online operation system is not optimistic. In this case, an integrated platform will be an inevitable trend of development.
According to the official websites of GitLab and Jenkins, the configuration file of GitLab is called gitlab-ci.yml, which solves the problem of continuous integration; the configuration file of Jenkins is Jenkinsfile, which is a solution for continuous deployment. CI/CD pipelines in foreign countries are relatively rigorous, and the functions of CI and CD are more clear, but simply adding GitLab and Jenkins is not equivalent to CI/CD, so when doing CI/CD, it must develop towards an integrated trend .
Under the framework of the integrated platform, a large amount of data mining can be carried out. The requirements are put forward from the mailing list, enter the demand center, go to the production and research environment, and then to the pre-release environment, and then go online. The data object of requirements flows through many tools, and the time that each tool is stagnant is determined by the work of different roles. DevOps pays attention to efficient end-to-end value stream delivery, but if indicators such as average demand delivery cycle, demand stream load, and production and research cycle cannot be counted, it cannot be regarded as real DevOps. It is just an automated process, which is also the goal of many enterprises. status quo. Therefore, integrated DevOps will also be the development trend of future R&D efficiency.
02 DevSecOps security capability system
The DevSecOps security capability system is to abstract security capabilities for the entire research and operation process. For example, threat modeling is done in the design phase, but there are not many companies doing threat modeling now, because its implementation is very dependent on the rich experience of the person in charge. In addition to understanding security, you also need to understand penetration, attack, R&D, architecture, Business, etc., can consider how to solve these risks.
A lot of testing needs to be done in the supply chain, and most of the methods are to directly refer to third-party components. We often refer to open source software in the supply chain, that is, open source components or third-party components, because the scope of open source software is too broad. For example, Redis cache, from the perspective of third-party components, is to introduce a Redis calling component into the code. This component may have problems, but the services deployed by Redis may also have problems. Therefore, the scope of open source software governance ranges from components and software From two perspectives, there are two different dimensions that need to be governed.
The Redis environment basically requires baseline scanning, and even manual security configuration checks, so security should not only focus on introducing tools and how to use them, but on what problems this tool can solve, and how to put it in place next. In the process, automation is established.
Next, if you want to do supply chain security governance, you only need to create a view, and you can find component vulnerability information under all application assets, as well as information about each deployment environment. All detection tasks or vulnerability repairs can only be completed in the process.
DevSecOps practice landing
Practice 1: IDE security self-test
Once the process is established, it is necessary to ensure that R&D has an efficient process that can be perceived in advance, because R&D will consider that the submitted code must pass the security check point, otherwise it will affect the KPI assessment. Moreover, it is time-consuming to trigger the CI process after the code is submitted online. If there is a detection plug-in at the IDE level, R&D can quickly self-test, know which problematic packages you have introduced, and upload the code after the modification and detection are correct.
Practice 2: Create a secure private server
If you use a third-party private server, or do not do any security control on the internal private server, you need to rely on a lot of inspections after the software is built. As mentioned earlier, the cost of fixing vulnerabilities found in the testing phase is twice that of the research and development phase. Therefore, in order to avoid investing more costs in the research and development phase, a security audit can be conducted when all third-party components enter the private server. Enter the private server. In fact, this is also a function of the SCA tool. It opens a network proxy. When we go to the public component warehouse to download components, the components will be checked by the proxy. After the check is passed, they will be placed in the warehouse; if there is a problem with the check, you can There are different processing strategies, such as putting it in a cache queue for evaluation or rejecting it directly. In the specific implementation, you can configure the external private server, and then dynamically perform storage detection and construction actions.
Practice 3: Establish a consistent quality internal cycle
CI/CD is a two-stage process, but not many companies can do CD now, because CD has different understandings, one is continuous delivery, and the other is continuous deployment. Continuous deployment is a technical behavior, similar to CI, but continuous delivery requires a lot of capabilities to support it. The CI link is not a process, because there are different assembly lines in different links to do security work. The entire security work has achieved a normalized automation. Incremental vulnerabilities are not only defined to repair super-critical vulnerabilities, but low-risk vulnerabilities must also be repaired together. This is a relatively good state. The establishment of the internal quality cycle will be very efficient, and the quality control of software security will be more efficient.
Practice 4: Establish a security management and control mechanism
The security management and control mechanism is the security tool stuck point mentioned above. Now almost no security tool has the stuck point capability. Generally, when doing security checkpoints, you write scripts by yourself, and then judge the current security vulnerability data situation. The current tools can be configured to add different threshold requirements for stuck points, and then choose an execution strategy, such as termination or continuation, and can quickly notify the relevant responsible person.