About the author
Gene Kim is award-winning in several roles: CTO, researcher, and writer. He was the founder of Tripwire and served as CTO for 13 years. He has written two books, including The Visible Ops Handbook, and he is currently writing The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win and the DevOps Cookbook. Gene is a huge fan of IT operations and is obsessed with improving operations processes - enabling developers to deliver more working functionality to production, rather than just code completion, without impacting the current IT production environment. He has worked with several top Internet companies to improve their publishing processes and improve the integrity of their IT operations processes. In 2007, Computer World included Gene on its "40 Innovative IT People Under 40" list, and Purdue University presented him with the Distinguished Alumni Award in recognition of his professional achievements and leadership.
content
- What is DevOps
- How is DevOps different from Agile?
- How DevOps is different from ITIL and ITSM
- DevOps and Visual Operations
- Fundamentals of DevOps
- Application areas of the DevOps pattern
- The value of DevOps
- How Information Security and QA Fit into DevOps Workflows
- One of my favorite DevOps patterns
- My Favorite DevOps Pattern Two
- Three of my favorite DevOps patterns
1. What is DevOps
The term "DevOps" generally refers to the emerging specialization movement that advocates a high degree of synergy between development and IT operations to improve the reliability, stability, resiliency, and reliability of production environments while accomplishing high-frequency deployments. safety.
Why Development and IT Operations? Because the typical value flow is between the business (defining requirements) and the customer (delivering value).
The origins of the DevOps movement are usually placed around 2009, with many movements complementing and reinforcing each other - the Efficiency Workshop movement, notably the groundbreaking "10 Deployments a Day" presented by John Allspaw and Paul Hammond, Infrastructure As Code Movement (Mark Burgess and Luke Kanies), Agile Infrastructure Movement (Andrew Shafer), Agile Systems Management Movement (Patrick DeBois), Lean Startup Movement (Eric Ries), Jez Humble's Continuous Integration and The launch movement, as well as Amazon's "Platform-as-a-Service movement," grew out of the complementary and mutually reinforcing movements of these movements.
DevOps co-author John Willis wrote a very good post here
http://itrevolution.com/the-convergence-of-devops/
2. How is DevOps different from Agile?
A fundamental tenet of the agile development process is to deliver the smallest available software at a faster frequency than the waterfall development model. The most obvious goal of agile is to have deliverable functionality at the end of each Sprint iteration cycle.
The high frequency of deployments often causes deployments to pile up in front of IT operations. Clyde Logue, founder of StreamStep, summed it up: "Agile is great for development to regain business trust, but it's not intended to shut IT operations out. DevOps enables the IT organization as a whole to regain the trust of the business. business trust".
DevOps and agile software development go hand-in-hand as it extends and refines continuous integration and release processes, thus ensuring that code is production-ready and truly delivers value to customers.
DevOps not only creates a workflow for IT operation and maintenance, when the code has been developed but cannot be deployed to production, these deployments will accumulate in front of IT operation and maintenance, and customers will not enjoy any value. , and to make matters worse, deployments often result in outages and unavailability of services in the IT environment.
DevOps has an innate cultural change genetic makeup as it revolutionizes the workflow and traditional metrics between development and IT operations. John Willis and Damon Edwards (both co-authors of the DevOps Cookbook) have written extensively on this topic:
http://itrevolution.com/devops-culture-part-1/
3. How is DevOps different from ITIL and ITSM ?
Although many people see DevOps as a disruption of ITIL and ITSM, I have a different view. When it comes to the business processes that support IT operations, ITIL and ITSM processes are undoubtedly the best. In fact, they describe the set of capabilities that need to be supported by IT operations that are sufficient to support a DevOps-style workflow.
Agile and Continuous Integration and Continuous Release are outputs of development, these outputs also serve as inputs to IT Operations, in order to accommodate the rapid deployment rhythm associated with DevOps, many aspects of the ITIL process, especially around the change, configuration and release processes , requires automation.
The goal of DevOps is not only to increase the frequency of changes, but also to enable the successful deployment of features without disrupting and disrupting current services, while also enabling the rapid detection and remediation of defects. This introduces new ITIL guidelines for service design, incident and problem management.
4. How DevOps and Visual Operations Work Together
In 2004, I co-authored The Visible Ops Handbook with Kevin Behr and George Spafford, a prescriptive guide for enabling high-performance IT operations to transition from good to great , one of the key points is how to control and reduce unplanned work.
Between development and IT operations, DevOps not only focuses on creating fast and stable planned workflows, but DevOps also has a more holistic approach to systematically eliminating unplanned work, defining development resilience guidelines, and taking responsibility for managing and reducing technical debt .
5. Fundamentals of DevOps
In the books "The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win" and "DevOps Cookbook", we describe the underlying principles of DevOps - "Three Basics of DevOps", all DevOps patterns are can be derived from these 3 fundamental points.
The first basic point emphasizes the performance of the entire system, rather than limiting performance to a specific work area, which can be as large as a department (such as development and IT operations) or as small as an individual contributor (such as a developer) , system administrator, etc.).
The focus is on the business value stream driven by IT. In other words, it starts with the definition of requirements (such as defined by the business or IT department), develops and builds, and then hands it over to IT operations, and finally the value is delivered as a service. to customers.
The result of practicing the first basic point - never passing a known defect downstream, never failing a small one, always working to improve the process, and being obsessed with a deep understanding of the system requirements (according to Deming's theory)
The second fundamental point is about creating a right-to-left feedback loop, and the goal of almost all process improvement initiatives is to shorten and amplify the feedback loop so that necessary corrections can be made on an ongoing basis.
Apply the results of the second fundamental point – including understanding and responding to all internal and external customers, shortening and amplifying all feedback loops, and, when necessary, embedding the knowledge customers need very easily.
The third basic point is to create a culture that promotes two things - a spirit of continuous exploration, a spirit of taking risks, and the ability to learn from success and failure, while also remembering that repetition and practice are A prerequisite for integration.
These two things are equally important to us, the spirit of exploration and the spirit of taking risks can ensure that we continue to improve, it even means that we may reach dangerous areas that we have not been before, so it also forces us to learn, Mastering and mastering those skills allows us to get out of the danger zone without a hitch.
The result of the third fundamental point - allocating time to improve the daily routine, fostering a culture that rewards risk-taking, while proactively introducing failures into the system to increase resiliency.
6 . Application areas of the DevOps pattern
In the "DevOps Cookbook", we divided the DevOps pattern into 4 areas,
Domain 1, extending development into production – including extending continuous integration and release capabilities into production, integrating QA and information security into the entire workflow, and ensuring code and environments can be deployed directly in production.
Domain 2, adding production feedback to development – including establishing a complete timeline of development and IT operations events to help resolve incidents, making development into blame-free production reflection, making development self-service as much as possible, and creating informative instructions The controller is used to indicate how local decisions affect global goals.
Domain 3, development is embedded in IT operation and maintenance - including development investment in the entire production problem processing chain, allocating development resources for production problem management, and assisting in returning technical debt, and development provides cross-training for IT operation and maintenance, increasing IT operation and maintenance. Reduces the number of escalated issues by reducing the ability to handle issues.
Domain 4, Embedding IT Operations into Development - including embedding and liaising IT operations resources into development, helping development create reusable user stories for IT operations (deployment, management of production code, etc.), defining some Non-functional requirements that are common to all projects.
7. The value of DevOps
I believe there are 3 business benefits that enterprises can achieve by adopting DevOps: faster time-to-market (eg, shorter development cycle time and higher deployment frequency), improved quality (eg, higher availability, higher change success rates, and fewer failures) , etc.) and improve organizational effectiveness (eg, spend time on value-adding activities, reduce waste, and deliver more value to customers).
Fast time-to-market products:
In 2007, at the IT Process Association, after evaluating more than 1,500 IT organizational structures, we concluded that an efficient IT organization is 5 to 7 times more efficient than the average organization. There are 14 times more changes, 1/2 the change failure rate, 4 times the first fix rate, and 10 times less time for a level 1 incident. Repeated audit findings are 4 times less, detection of vulnerabilities through internal controls is about 5 times higher, and project due date performance is 8 times higher!
In our research, the highest observed deployment frequency was around 1000 production changes per week, with a 99.5% change success rate, which we think is really fast...
One of the hallmarks of high performance is that the best are continuing to get better. This is definitely happening in the realm of deployment frequency. Organizations that have applied DevOps practices are exhibiting faster rapid deployment and implementation, and orders of magnitude faster than the average organization.
Accenture recently did a study: What are Internet companies doing? According to Amazon's records, they maintain the current 1000 deployments per week, while still guaranteeing a 99.999% success rate!
http://www.youtube.com/watch?v=dxk8b9rSKOo
http://www.slideshare.net/slideshow/embed_code/9466635?startSlide=33)
The ability to maintain a high deployment rate (that is, a fast number of iterations) translates into business value in two main ways:
How quickly an organization can turn an idea into value and deliver it to customers (for example, Damon Edwards and John Willis say "concept to implementation"), the organization can do multiple attempts at the same time.
For me, if I'm in an organization that only deploys every 9 months, and my competitors can deploy 10 times a day, I'll undoubtedly have a clear competitive disadvantage.
High frequency deployment also enables rapid and continuous deployment. Intuit's founder, Scott Cook, has been advocating a "sharp innovation culture" at all levels of the organization, one of my favorite examples is documented at http://network.intuit.com/2011/04/20 /leadership-in-the-agile-age/.
"Every employee should be able to deliver fast, high speed...Dan Maurer was in charge of our consumer division, including the TurboTax website. When he took over, we were only doing a few deployments a year, but by creating a sharp Innovation culture, they now do 165 deployments in 3 months of tax season. Business value? Website conversion rate up to 50%. Employee value? to the market”
To me, the most shocking thing about Scott Cook's story is that they do all these deployments during busy tax season! During peak season, most organizations freeze any changes (for example, from October to January, retailers may often have change freezes). But if during peak season, if you can increase your conversion rate and your competitors can't, then that's a real competitive advantage.
The premise of achieving this effect is that many small changes can be quickly completed and deployed without affecting customers.
Reduce total IT waste:
Mike Orzen and I talked very early on about the huge waste in the IT value stream that stems from extended delivery deadlines, poor handoffs, unplanned work and rework. Based on an article by Michael Krigsman, after applying the principles of DevOps, a lot of value can be recovered rather than wasted.
We've calculated that reducing IT waste in half and then reinvesting the savings would yield a 5x return on investment, which could generate $30 trillion in value annually.
This is the tremendous value and opportunity that slips through our fingertips. It accounts for 4.7% of global GDP, and even exceeds the economic output of Germany.
I feel like this is really important, especially when I think about the world my three children will inherit, the increasing potential impact of this waste on productivity, living standards, and economic prosperity that makes me feel compelled to change.
However, there is an even bigger cost. In most IT organizational structures, the work is thankless and frustrating, and people feel like they are trapped in a repeating horror movie that cannot be changed. final outcome. Managers are supposed to manage IT well, but they give up such responsibilities, leading directly to endless departmental conflicts between development, IT operations and information security, and the presence of auditors will only make things worse .
In the long run, the inevitable result is slow progress. The lives of IT professionals are often frustrating and frustrating, often leading to a sense of powerlessness and stress that permeates every aspect of life. IT professionals face stress-related health issues, social issues, and staying at home, making them not only unhealthy, but also potentially unsustainable.
As human beings, we are meant to contribute and feel as if we are making a difference and making a difference. But often when IT professionals ask their organization for help, they get the answer: "You don't understand," or, worse, they even get the "You don't matter."
8. How information security and QA fit into DevOps workflows
The high deployment frequency of DevOps usually puts a lot of pressure on QA and information security. Consider a situation where development is deployed 10 times a day, and information security usually takes 4 months to evaluate the security of the application. Clearly, the rates of code development and code security audits don't match at all.
The 2011 Dropbox failure is a well-known example of how risky untested development code can be. As a result of this incident, authentication was disabled for four hours, allowing unauthorized users to access all stored data.
Of course, it's not all bad news for QA and information security, development will maintain high frequency deployments through continuous integration and good release practices (a culture of continuous testing). In other words, the automated tests run as soon as the code is committed, and as soon as a problem is found, it must be fixed immediately, just like a developer checking code that hasn't been compiled yet.
By integrating functional testing, integration testing and information security testing into the day-to-day routine of development, problems will be discovered and resolved faster.
Similarly, there are more and more information security tools, such as Gauntlet and Security Monkey, which can help us test security objects during development and launch to achieve information security goals.
But there is also an important point to consider, static code analysis tools usually take a long time to run, in hours or days. In this case, information security must specify specific modules with security risks, such as encryption and authentication modules. As long as these modules change, a full round of information security testing runs, otherwise deployment can continue without the need for full coverage information security testing.
In particular, we observed that DevOps workflows rely a little more on detection and recovery than standard functional unit testing. In other words, of course, when development is delivered as a software suite, it is more difficult to deploy changes and patches, and QA also relies heavily on code testing to verify functional correctness. On the other hand, when software is delivered as a service, defects can be fixed quickly. And QA can also reduce test dependencies and instead rely more on production monitoring of defects, as long as defects can be fixed quickly.
Code failure recovery can be achieved by enabling or disabling certain code functions in the form of configuration by means of "feature tags", so as to avoid rolling out a full-featured deployment, and deploy only the functions that pass the test to production.
Relying on detection and recovery for QA would be a better option when worse-case scenarios such as feature unavailability or performance degradation occur. But when faced with loss of confidentiality or data and system consistency, we cannot rely on detection and recovery methods. Instead, it must be fully tested before deployment, or it could lead to major security incidents.
9. My Favorite DevOps Pattern One
Typically, in software development projects, development uses up all of its planned time to develop features. This can lead to inability to adequately address IT operations issues, so they save time by taking shortcuts in defining, creating and testing code dependencies such as databases, operating systems, networking, virtualization, etc.
So this is the main reason for the eternal tension between development and IT operations and sub-optimal outcomes. The consequences are very serious, such as inappropriate definition and specified environment, inability to redeploy, incompatibility between code and environment, and so on.
In this mode, we put environment requirements early in the development process and enforce a strategy that code and environment must be tested together, and once agile development methods are used, we can be very concise and elegant.
At the end of each iteration as required by agile
http://www.infoq.com/cn/articles/11devops