In-depth interpretation of the State of DevOps DORA 2022 report

foreword

Last month, Google's DORA released the annual State of DevOps 2022 report . Bytebase, as a vendor that develops database CI/CD open source tools for DevOps teams, conducts an in-depth interpretation of DORA every year. For the interpretation of last year, see " State of DevOps DORA 2021 Report Interpretation and Commentary " .

background

DORA is not as well-known in China as Gartner. Its full name is DevOps Research and Assessments. Since 2014, it will publish a report on the entire DevOps industry every year. It will not be published in 2020 due to the epidemic, so add this year There are 8 reports in total. DORA started as an independent research organization but joined Google Cloud in late 2018. Generally speaking, DORA's report is the most professional and objective in the entire DevOps industry, which should be the reason why he was favored by Google in the first place. Even after joining Google, its report can basically maintain neutrality, so DORA's report can be used as an authoritative weather vane for the entire industry. Alright, let's officially get into the report.

different from the 2021 report

Compared with last year's report, this year's report has a lot of changes:

The first is the focus. This year is the first time in the history of DORA to highlight a theme, the Software Supply Chain.
The distribution of work experience of the respondents this year has changed a lot, with less than 5 years of experience accounting for 35% (compared to 14% in 21 years), and only 13% of respondents with more than 16 years of experience (compared to 41% in 21 years).
The grades for assessing the team's software delivery performance (Software delivery performance) have changed from 4 categories to 3 categories, and the Elite level has been cancelled.
Combined with software delivery performance and operational stability performance, a new classification is introduced, with 4 categories, Starting/Flowing/Slowing/Retiring - Start/Flowing/Slowing/Stopping.

Subject

This year's report highlights the three major performance themes, Software delivery performance (software delivery performance), Operational performance (operating performance, mainly stability), and Organizational performance (organizational performance, referring to company performance), while last year it was just a sentence .

sponsor

This year's sponsors have some changes compared to last year. This is last year's lineup, and the red ones are absent this year. This year's lineup, framed are new additions:

Old faces from last year

armory and circleci are doing CI/CD, and the former is the commercial company behind the open source project Spinnaker.
Deloitte , IT Implementation Consulting.
GitLab is a one-stop DevOps platform.
Liquibase , like Bytebase, is an open source commercial company doing Database DevOps.
sysdig is for Secure DevOps, which is now abbreviated as DevSecOps, which is also the focus of GitLab/GitHub's recent efforts.

new faces this year

Broadcom Software , a somewhat unexpected face, Broadcom is considered a relatively traditional software manufacturer, and most of its DevOps-related product lines come from the acquisition of CA in 2018. Headquartered in Silicon Valley, CA's big logo was still hanging on the only tall building next to Highway 101.
JetBrains , a well-known IDE manufacturer, also entered the field of software delivery because of the launch of JetBrains Space. The slogan of JetBrains Space is "A complete software development platform", which is in direct opposition to GitLab's "The One DevOps Platform". Headquartered in the Czech Republic.
JFrog , the leader in the Software Supply Chain space, fits well with this year's theme. His headquarters is in Silicon Valley, with a low-key appearance, a hidden champion, and he often passed by when driving.
Octopus Deploy , a manufacturer specializing in continuous delivery of CDs, is also a leader in this field. Headquartered in Australia.

It can be seen that companies in the DevOps field are distributed all over the world . After all, the tools for developers are very common and there are not many regional differences.

disappearing faces

redgate , a Database DevOps tool vendor similar to Bytebase and Liquibase. Duplicated with Liquibase, so it's reasonable not to appear this year.
PagerDuty , the leader in fault response. Didn't come this time, which made the sponsor lineup a bit eclipsed.
CD Foundation , a non-governmental organization for continuous delivery.

Generally speaking, this year's sponsor lineup covers more comprehensive fields in the entire Software Development Life Cycle (SDLC) . There are still a few areas that are absent:

Troubleshooting (representative company PagerDuty)
Monitoring alarm (representative company Datadog)
Feature Flag (representative company LaunchDarkly)
API lifecycle management (representative company Postman)
Cloud infrastructure (representative company HashiCorp)

Cloud

Some time ago DHH "Why we're leaving the cloud" ( https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47e0 ) article caused another round of up and down clouds discussion between. However, this DORA report once again shows that the growth of cloud is unstoppable, and the utilization rate of public cloud, multi-cloud, hybrid cloud, and private cloud is all accelerating, eroding the share of non-cloud. Cost factors like those mentioned in the DHH article are only part of the equation when deciding whether to use cloud services. The core advantage of the cloud is that it represents a more advanced way of working. Although organizations that adopt this way pay more for the unit cost of pure raw materials, they can get it back from other aspects. translate

Using any cloud computing platform, whether public or private, can have a positive impact on company culture and work environment (e.g., more spontaneous culture, less burnout, more stability, higher employee satisfaction). Cloud users score 16% higher on culture than non-cloud users.

The use of cloud computing services has a positive impact on the performance of the entire organization. Respondents who use the cloud are 14 percent more likely to meet organizational performance goals than those who do not use the cloud.

Calculating economic accounts depends on the impact on the overall performance indicators, and this point is also clearly pointed out in the DORA report that organizations that use Cloud have better performance in Organizational Performance than organizations that do not use Cloud . Expensive is often a bargain.

SDO Performance

The four core indicators of Software Delivery and Operational Performance have not changed: However, this year’s survey found that the difference between Elite and High is not obvious enough, so the Elite file was canceled, a hypothesis given by the report

Software development has seen reduced innovation in terms of practices, tooling, and information sharing. This could be the result of the ongoing pandemic hampering the ability to share knowledge and practices across teams and organizations

I also agree with reduced innovation, but I feel that the more likely reason is that after so many years of development of Software Delivery / DevOps, best practices have begun to spread on a large scale, so the head gap has narrowed, rather than the relationship between the epidemic. This year's report also introduces a new classification, Staring/Flowing/Slowing/Retiring - starting/flowing/easing/stopping. This is easily reminiscent of Tuckman's organizational development model ( https://en.wikipedia.org/wiki/Tuckman%27s_stages_of_group_development ). This is a segmentation based on the dimension of team development stage, or adding a dimension of time, which makes the comparison more comprehensive.

SRE and DevOps

A highlight of this year's report is the more dialectical exposition of the importance of Reliability:

Across this breadth of teams, the data reveal a nuanced relationship between reliability, software delivery, and outcomes: when reliability is poor, software delivery performance does not predict organizational success. However, with better reliability, we begin to see the positive influence of software delivery on business success.

This passage is quite exquisite

Extensive team data reveals a delicate relationship between stability, software delivery, and business outcomes: when stability is poor, fast delivery does not lead to an organization's business success. However, when the stability is improved, we will see that fast delivery speed is positively correlated with business success.

It is not enough to be fast, but to be fast and not chaotic, in order to achieve business success. The report also reiterated the J-Curve concept in the 2018 report. After the introduction of SRE practice has achieved a little early success, it will enter a period of bottleneck or even regress. It needs to be able to persist and touch the inflection point.

as teams adopt more SRE, they reach an inflection point where the use of SRE starts to strongly predict reliability, and in turn, organizational performance.

Software Supply Chain

It is the first time for DORA to highlight a topic in the report. I speculate that there are several reasons for this special talk about Supply chain security:

Compared with the traditional CI/CD, it is indeed a more novel topic, and it is also a hot spot in the industry. For example, GitLab has also changed from talking about DevOps to talking about DevSecOps.
It is a good angle to evaluate other topics.
- For example, we can see the situation of CI/CD: the Software Supply Chain is not doing well, which more or less reflects the serious problems of the CI/CD system.
- For example, the whole report has been talking about people, process and tooling, but at the end of the topic Supply Chain, these three are linked together. Of course, it is also possible that under the influence of Google Cloud and major commercial sponsors, this report is tilted in the direction of security that major customers/CSO/CIO/CTO are more concerned about.

some regrets

The biggest problem in this year's report should be that there has been a big change in the sample distribution. The distribution of junior/senior respondents among the respondents is completely reversed from last year's sampling ratio. This has led to abnormalities in many data, and because it is the first year, it is not easy to draw clear conclusions. The second point is that DORA, as an authoritative report in the field of DevOps, lacks some forward-looking observations. CI/CD, Version Control and other capabilities are indeed core, but there is no mention of the evolution of lower-level production relations like Platform Engineering. The last point is that this year's report focuses on the Software Supply Chain, relatively ignoring other technical capabilities (Technical Capability), and other points are only placed on the official website. For example, Database Change Management, which affects R&D efficiency with a higher weight, was mentioned several times in last year's report, but it was not mentioned in this year's report at all. However, the industry has had a lot of innovations in the field of Database Change Management in recent years, such as PlanetScale, Neon from the perspective of database engines, and Bytebase, Prisma from the perspective of database tools. From the background of the author of DORA, this team is good at system analysis, but relatively lacks the most front-line practical experience, which makes them unable to keep up with the latest trends in the industry, and cannot design relevant questions in the questionnaire, and thus cannot reflected in the report. The DORA report is in its 8th year. From the beginning, the systematic analysis of the DevOps field gave people a refreshing feeling, and now it gradually feels like doing research in an ivory tower. In this regard, the combination of Gartner and the industry is relatively close, such as the 2022 Gartner Hype Cycle for Emerging Technologies published in August:

epilogue

When the sample size is large enough, there will always be different sounds. And there are two types of people who can help us make better decisions, one is close to our own background, and the other is opposite to our own background. The former can tell us what we are suitable for, while the latter can tell us what we are not suitable for. In many cases, the reverse signal of what is not suitable for us is much clearer and stronger. Let’s take a look at the personal background of DHH, who advocated leaving Cloud mentioned above: the industry really needs KOLs like DHH to put forward sharp opinions from time to time to spark discussions, but industry reports like DORA, which use large-scale sampling, analyze data, and then refine opinions, For the R&D team, it is more valuable to guide the actual work. It is precisely because of his professionalism that DORA still enjoys an authoritative reputation even when various industry reports are flying all over the world. And what DORA wants to tell the R&D team is the same as his style of writing reports over the years, which has not changed - focus on fundamentals. If the R&D team wants to improve efficiency, it still needs to focus on those four indicators:

Lead Time for Changes
Deployment Frequency (Deployment Frequency)
MTTR - Medium Time to Recovery
and Change Failure Rate (change failure rate)

This is the end of the interpretation of this year's DORA report. Our product Bytebase provides a complete solution around database DevOps, through one stop for review, change, version and rollback Model capabilities help the team improve these four core indicators of R&D effectiveness, and then, as pointed out in the DORA report, ultimately help the organization achieve better business results. Bytebase is also the only database CI/CD tool included in CNCF Landscape. Interested students can directly go to our official website bytebase.com to download and try it in 5 seconds, or follow the Bytebase WeChat official account to get the way to join the user group, and we Real-time communication with product students.

see you next year.