This article is shared from the Huawei Cloud Community "Building a Three-Layer Code Protection System in DevSecOps" by Uncle_Tom.

In the application process of DevSecOps, static analysis tools assume a very important task of guarding code quality and security during the development phase. Based on the differences in development environment, code characteristics and detection tool capabilities at different locations in the development process, this article proposes the need to deploy detection tools according to local conditions to form a progressive three-layer code security defense system, thereby improving the overall security of application software while also It can effectively implement the security shift left strategy and reduce the maintenance cost of security issues.

1. DevSecOps

In recent years, the proportion of DevSecOps introduced by large enterprises has increased year by year, from 41.3% in 2020 to over 63.5% in 2022, with a compound growth rate of more than 20%.

The term DevSecOps was first proposed by Gartner in 2012, and has gradually become a hot topic in software development in recent years. DevSecOps builds a bridge between developers, testers, security teams, and operations teams; it improves communication and collaboration between teams, with the goal of delivering faster and more efficiently. DevSecOps adds security activities on the basis of DevOps. On the basis of ensuring rapid development and rapid deployment, it improves security and embeds security into applications to respond to security threats more quickly.

The figure below shows the nine stages of the DevSecOps software life cycle defined by the U.S. Department of Defense (DOD): plan, develop, build, test, release, deliver, deploy, operate, and monitor. Security is embedded at every stage. The DevSecOps lifecycle is highly adaptable and has many feedback loops to drive continuous improvement. DevSecOps poses new challenges to traditional software development in terms of management concepts, management methods, organizational structure, development processes, software development, development platforms, tool integration, and corporate culture.

In the application process of DevSecOps, static analysis tools assume a very important task of guarding code quality and security during the development phase. This article focuses on the important role of software static analysis tools in the development domain of DevSecOps.

2. DOD DevSecOps

The U.S. Department of Defense has released a series of documents related to DevSecOps since 2021: "DoD Enterprise DevSecOps Strategic Guide", "DoD Enterprise DevSecOps Basics", "DevSecOps Reference Design", "DevSecOps Action Manual" and other related supporting documents.

The DevSecOps Essentials Guide: Activities and Tools was supplemented in May this year. In this document, the security activities and corresponding tools that need to be completed in the DevSecOps life cycle are more clearly defined. The safety activities involved are listed below:

safety activities	stage	Dependent tools
Task-based cyber risk assessment	all	Task-based cyber risk assessment tool
Threat modeling	plan	Threat modeling tools
Code submission scan	develop	Code warehouse security plug-in
Secure code development	develop	IDE
Static code scanning before submission	develop	IDE security plug-in
Dependent component vulnerability check	Construct	Dependency Checking/BOM Checking Tool
Static Application Security Testing and Scanning (SAST)	Construct	Static Application Security Testing and Scanning Tool (SAST)
Database security testing	test	Security Compliance Tools
Dynamic Application Security Testing and Scanning (DAST)	test	Dynamic Application Security Testing and Scanning Tool (DAST) or Interactive Application Security Testing Tool (IAST)
Interactive Application Security Testing (IAST)	test	Dynamic Application Security Testing and Scanning Tool (DAST) or Interactive Application Security Testing Tool (IAST)
Manual security testing (such as penetration testing)	test	Various tools and scripts (may include network security testing tools)
Service security testing	test	Security Compliance Tools
Post-deployment security scan	deploy	Security Compliance Tools
Compliance Monitoring (Resources and Services)	monitor	Compliance tools, operational dashboards
Compliance monitoring	monitor	Compliance tools, operational dashboards
Database monitoring and security auditing	monitor	Security Compliance Tools
Runtime Application Security Protection (RASP)	monitor	Security Compliance Tools
System security monitoring	monitor	Information Security Continuous Monitoring (ISCM)
SBOM software composition analysis	Late build	SBOM and Software Factory Risk Continuous Monitoring Tool
Continuous monitoring of software factory risks	Late build	SBOM and Software Factory Risk Continuous Monitoring Tool
Interface security testing	Construct	API security testing tools
Cooperative and adversarial testing	Operation and maintenance	Cooperative and adversarial testing tools
Continuous network operations testing	Operation and maintenance	Continuous Network Operations Testing Tool
engineering obfuscation	Operation and maintenance	Engineering Obfuscation Tool

From this activity table, we can see that there are three key security checkpoints during the development process, which are the parts marked in red in the table), and the information about these three checkpoints in the file is combined into the following table:

Activity	baseline	describe	enter	output	Dependent tools
Static analysis of code before submission	Require	Code is scanned and analyzed as developers write it. Notify developers of potential code weaknesses and make recommendations for remediation.	Source code, known weaknesses	Find weaknesses in code	IDE plug-in
Code commit check	Require	Before pushing changes to the repository, check the changes for sensitive information. If suspicious content is found, it notifies developers and blocks submissions.	Locally submitted code	Checked security issues and warnings	Code warehouse security plug-in
Static application security testing	Require	Perform static analysis checks on software systems	Source code, known security issues and vulnerabilities	Static inspection reports and fix recommendations	Static analysis tools

From this table we can see that during the development phase of the code, corresponding static analysis detection needs to be completed at the following detection points:

In the IDE, use the IDE security plug-in to complete security testing of the code that needs to be submitted;
In the code warehouse, code submission scanning is performed through security plug-ins, which is also what we often call "access control";
During the build, complete Static Application Security Testing and Scanning (SAST) through static analysis tools

The "DevSecOps Basic Guide: Activities and Tools" only puts forward the activity requirements and rough tool requirements for process detection points, and does not give specific process integration and how to select tools at different detection points. .

3. OWASP DevSecOps

The Open Worldwide Application Security Project (OWASP) is a non-profit foundation dedicated to improving software security. The Foundation works to improve software security through its community-led open source software projects, hundreds of chapters around the world, tens of thousands of members, and hosting local and global conferences.

The OWASP DevSecOps Guideline guides us on how to implement secure pipelines and uses best practices, and introduces tools that can be used to do so.

It guides the practice of DevSecOps and also uses a special chapter to introduce the pre-commit process. As shown below:

This picture clearly shows the pre-submission inspection process and two types of checks that need to be completed during pre-submission:

Make sure there are no password or key issues in the code;
The code follows Linter rules.

Linter here I can't find a suitable Chinese translation for this English. Linter originally means that after the clothes are washed in the washing machine, the fibers of the clothes gather together to form small balls of fluff or fibers due to rolling friction. I used to want to get rid of these extra "little balls", but then people invented an artifact called Linter, which can remove these "little balls" with just one roll.

In 1978, Stephen C. Johnson, who was working at the Bell Experiment, was debugging his own C language project and suddenly thought why not make a tool to prompt him for any problems in the code he wrote? This tool is also known as Linter. Linter is a static analysis tool mainly used to find grammatical errors, potential bugs, coding style, etc. in the code. Our common tools named lint are this type of static checking tools. Almost every language has its own language Linter tool, such as the ones we are familiar with: Java: checkstyle , Javascript: ESLint, Python: PyLint, C/C++: cpplint , Go: golint , etc.

In "OWASP DevSecOps Guide", the role of Linter is given:
- Detect errors in code and errors that could lead to security vulnerabilities;
- Detect formatting or style issues and make the code more readable, resulting in more secure code;
- Test recommended best practices;
- Can improve the overall quality of the code;
- Since everyone follows the same linting rules, maintaining the code becomes easier.
In the "OWASP DevSecOps Guide", static inspection tools are defined as Linter and advanced static inspection tools according to different inspection problems:
- Linter tools are the most basic form of static analysis. Using the lint tool can help identify common errors, such as:
  - Array index out of bounds;
  - null pointer dereference;
  - (Potentially) dangerous combinations of data types;
  - Inaccessible code (dead code);
  - Non-portable structure.
- Advanced static analysis tools. Advanced static analysis tools typically provide:
  - Pattern-based checks;
  - quality and complexity metrics;
  - Best practice recommendations for developers;
  - Supports multiple safety and security-focused coding standards;
  - For developing safety-critical applications, such as out-of-the-box authentication.

The "OWASP DevSecOps Guide" gives the differences in defect types detected by different tools. It mainly explains that different types of checking tools need to be configured at different detection points.

4. Safe left shift

Capers Jones explains from the perspective of software engineering practice in "Application Software Measurement: A Comprehensive Analysis of Productivity and Quality" that most problems are introduced during the coding phase, and as defects are discovered later in the development process, they need to be fixed. The higher the cost.

Therefore, we hope that by integrating post-development testing into the development process, we can effectively reduce the cost of repairing defects introduced during the development process.

Testing moves to the left of development phase
Testing is shifted left and defects are reduced during the development phase

After the concept of DevSecOps was proposed, people naturally proposed the concept of "security left shift" . "Security Shift Left" (as defined in the OWASP DevSecOps Guide) is an approach or solution that embeds security into the development process and considers security from the initial steps of application or system design. In other words, security is the responsibility of everyone involved in the software development and operation process. Of course, security is a profession and we need highly skilled people to play security related roles; but in this approach any designer, software architect, developer, DevOps engineer and... responsible.

From this description we can see that safe shift left includes several specific activities:

Safety starts from design and continues throughout the entire process;
All employees participate in safety activities;

According to our previous understanding of the US Department of Defense's "DevSecOps Basics Guide: Activities and Tools" and OWADSP's "OWASP DevSecOps Guidance", there are three checkpoints in the coding phase of the development process: IDE, access control, and continuous build (CI). According to the concept of "safety shift left", can we also "go all the way to the left" and put the static inspection tool in the IDE or access control to realize the left shift of the static inspection tool, thus eliminating the inspection link in continuous build (CI)? ? Is it possible to go all the way to the left?

4.1. Idiom stories “adapt to local conditions”

It’s been a while since I told a story on my blog. China has a long history. The sages integrated various principles and philosophies on how to behave and deal with things in stories. They are easy to understand and made into idiom stories that people enjoy talking about. They have a long history and allow ordinary people to remember these stories. The essence of philosophy is passed down from generation to generation. In addition to the songs themselves, people are more interested in exploring the various poignant stories contained in the songs that Daolang has become popular recently.

At the end of the Spring and Autumn Period, the King of Chu believed the slander and killed Wu Zixu's family. Wu Zixu fled to the state of Wu. In order to avenge himself with the help of the Kingdom of Wu, Wu Zixu gave the King of Wu a suggestion: if he wanted to make the country prosperous and the people stable, he must first build a high city wall, so as to strengthen the defense power and prevent other countries from invading. We also need to strengthen our military strength and enrich our reserves of weapons and supplies, so that we can act as a deterrent to other countries. At the same time, agriculture must be developed. Only with the development of agriculture can the country be prosperous and strong, the people can live and work in peace and contentment, and the soldiers have sufficient supplies, and the granaries must be enriched to prepare for wartime needs. Only in this way can the country be stable and develop. The King of Wu agreed very much with Wu Zixu's strategy. So he cleverly used the topography of Wu State to build a city surrounded by mountains and rivers. There are many gates in the city, and three of them have towers. There are two small cities in the big city, the east and west. The west city is where the palace of King Wu is located, and the east city is where troops are stationed and armaments are stored. In this way, he set up garrison in the city, accumulated food, and enriched his arsenal in preparation for dominating the princes. After a period of preparation, the state of Wu gradually became stronger and stronger. In the end, the Wu army launched a large-scale attack on Chu, winning five out of five battles, and finally captured Chu's capital: Yingdu. Wu Zixu finally avenged the murder of his father and brother. This is the story of the idiom "adapt to local conditions".

From this idiom, we also realize that in the development stage, the use of static analysis tools cannot be "all the way to the left". It is necessary to consider the differences in detection points and select appropriate detection tools in order to establish an effective defense system.

4.2. Establish a three-layer code protection system

We can clearly see the differences between the three detection points through the table below, and select different static inspection tools according to the differences, so as to achieve the effect of "adapting measures to local conditions".

comparison item	IDE	access control	CI
Scan range	module level code	Code files with minor modifications	Full engineering code
Difference in code size	Generally less than <100,000 LOC	The amount of modification is generally <500LOC, and the number of modified files is <10	Determined according to the size of the project, the amount of code can reach hundreds of millions.
Compiler Environment	Some languages have local compilation environments, and C language requires complex compiler configuration.	No compilation environment	Have compilation environment
Tool configuration	Simple, ready to use out of the box, can have a small amount of configuration, and integrates with the IDE	There is a certain configuration that needs to be integrated with pre-commit	There can be complex configurations for rule selection and rule parameter setting.
Tool usage	Some developers use	Forced. Note: IDE inspection does not guarantee that developers must have performed the inspection task. Forced means are used here to ensure that the code completes basic security checks before being put into the library.	force
Analysis accuracy	The tool is required to have low false positives, but a small amount of false positives is allowed	Tools are required to have lower false alarms and reduce unnecessary repetitions due to tool false alarms.	Tolerate a certain false alarm rate, recommended <30%
Analysis efficiency	Second-level scanning, does not affect local development, and is best to complete fact checking	Minute-level scan, <5 minutes recommended	Hourly scanning, you can get the scan results before going to work the next day
Static checking tool requirements	Compiled/non-compiled, single file analysis, based on syntax tree analysis; without affecting efficiency or obtaining dependency information, moderate cross-file analysis is performed to improve the scope and ability of inspection.	Non-compiled, single file analysis, based on syntax tree analysis. Complete full inspection through CI when the overall project is small.	Compilation/non-compilation, cross-file analysis, various checking capabilities

During the implementation process, DevSecOps not only emphasizes the cooperation of tools to realize the automation of each detection point, but more importantly, it needs to provide a complete tool integration platform to realize the automated operation and supervision of the overall process. This is why most enterprises choose to support the implementation of DevSecOps through cloud platforms when implementing DevSecOps.

5. Huawei Cloud’s three-tier inspection system

Excellent code quality assurance practices often integrate code inspection into the development workflow, conduct automated audit inspections when user code is written and submitted, and conduct continuous programming specifications and quality inspections on the code produced by the team every day.

The practical requirements of this activity have been recommended by many excellent development models such as SDL and DevSecOps in the security development process.

Huawei Cloud Code Check Service (CodeArts Check) provides a rich API interface, covering the entire inspection business process such as task creation, task setting, task scanning, task report analysis, etc., and supports users to use the interface to seamlessly integrate into self-built CI/CD or CodeArts allows job data to be flexibly connected to customer dashboards, making code quality visible and manageable.

At the same time, through flexible task management, it supports exclusion directory settings to avoid invalid scanning, and supports mixed language inspection, simplified deployment, and acquisition of the entire version code quality at one time.

For details, see: Three-layer defect protection of "code writing, code merging, and version release", taking into account both efficiency and quality

6. Reference

DevSecOps Fundamentals Guidebook: Activities & Tools
DoD Enterprise DevSecOps Strategy Guide
DoD Enterprise DevSecOps Fundamentals
DevSecOps Fundamentals Playbook
DoD Enterprise DevSecOps-Pathway to Reference Design
OWASP devsecops guidelien
Capers Jones, Applied Software Measurement: Global Analysis of Productivity and Quality.
What Is Shift-Left Testing? - Parasoft

Click to follow and learn about Huawei Cloud’s new technologies as soon as possible~

In software development, how to build a three-layer protection system for your code