Hi everyone. DevOps, CI/CD, Docker, Kubernetes...it seems like the whole world is talking about these technologies, so much so that you feel like you are about to reach the NoOps stage. Don’t worry, it’s normal to feel lost in the vast sea of tools and various best practices. It’s time for us to analyze what DevOps is.
1. Introduction to DevOps
1. Introduction
DevOps is a series of software development practices that emphasize communication and cooperation between developers (Dev) and operations personnel (Ops). Through automated processes and tools, software construction, testing, and release are faster, more frequent, and more reliable. DevOps emphasizes a culture that breaks down the inherent barriers between development and operation and maintenance personnel, and emphasizes communication and cooperation between development, testing, operation and maintenance, and other links. DevOps is a practical method that extends agile development practices to the operation and maintenance stage, emphasizing the process of software construction, deployment, and delivery. DevOps is also a series of tool chains, from coding, building, testing, packaging, publishing to configuration, monitoring, etc., based on a series of basic principles and practical methodologies, forming a set of instrumental and automated tool chains. It enables them to build, test and deploy applications while significantly reducing time to market. DevOps makes the software process both "fast" and "stable". Fastness and stability are reflected in the four dimensions of deployment frequency, delivery cycle, average repair time, and change failure ratio.
Quality assurance is an integral part of the waterfall development process, but it still plays an important role in DevOps. The process of DevOps is never about spending a lot of time on comprehensive test creation between coding and release. It also means teams are constantly working to specify, build, test and deploy software efficiently and quickly.
The increase in testing volume also increases the requirements for test automation. DevOps requires backup of test automation to stay agile and efficient. Test automation is essential to maintain complete quality control and maintain release velocity. Regardless, dependence on automated testing cannot be avoided in CI/CD pipelines.
2. Benefits
Test automation has many advantages that enterprises can leverage to streamline their DevOps practices:
Eliminates the possibility of human error;
No need for human intervention during test runs;
Gets faster feedback;
More device coverage;
Automation ensures consistent quality;
Automatic reconfiguration;
Although it has several advantages, automated testing can be completely replaced Manual testing? Well, the answer is not yes. However, the best approach should be to automate as much as possible while only performing manual testing on less critical application functionality. This includes updating test scripts, reviews, completing one-off tests, and testing for usability, etc.
Continuous implementation of test automation makes it easier to quantify the degree of automation based on historical data. However, someone who is new to the field of testing can use this standard formula to calculate the estimated ROI of test automation.
Test automation cost = tool cost + labor cost to create scripts + cost of automated test maintenance
If you use an automated test multiple times, the ROI will add up after each use. So, if the automation evaluation is lower than manual testing, continue with the strategy and find as many areas of automation as possible.
2. Analysis of each testing stage
1. Unit testing
The focus of the unit test is that there is no database call or Web service code. These tests are very fast due to their narrow focus and no dependencies on external services or systems.
Unit tests focus solely on ensuring that all paths pass through the code and are verified correctly. Consider a payroll algorithm designed to calculate the wages of workers for every hour they work. The algorithm will calculate the salary by taking into account the number of hours worked and the hourly rate. However, this algorithm will be required for many types of situations, including:
Standard working hours (0–40 hours)
Overtime (hours greater than 40 to the company’s maximum hours per period)
Correction of errors (negative hours, negative wages, maximum hours exceeded)
Use widely accepted tools (including NUnit, RSpec and JUnit, etc.) to verify this situation. This situation can be effectively verified through unit testing using recognized tools including NUnit, JUnit, RSpec, etc.
2. Integration testing
Integration tests verify the behavior between components. It includes checking the behavior between database calls, web services or other API interactions.
Compared to unit tests, integration tests are slower due to the large amount of "rituals" involved in establishing a connection, authenticating, and dealing with network and service delays. Integration tests should be included in more critical verification rather than granular verification.
3. Functional testing
Functional testing aims to functionally verify a part of the system. Unlike integration tests, functional tests are much slower because they span the length and breadth of the user interface. Understandably, they are much slower than unit tests.
Slow due to brittle nature; letting functional tests handle high-value cases is feasible. Letting functional tests handle too many low-level operations can drastically reduce your time to market.
Effective testing coverage can only be achieved with close collaboration between delivery teams. Ensuring effective testing also ensures that any form of duplication in test coverage is avoided. Preventing duplication of testing is critical to the business so that expensive tools can be used to solve specific problems.
So, under what circumstances is test automation most feasible?
During requirements testing of critical functionality so that users can clearly see its failures;
labor-intensive and repetitive parts;
functional testing with a history of causing problems;
testing of components that require large amounts of data;
stress/load testing;
targeting different versions, Data sets and browsers for testing;
4. Automated testing
Businesses need to automate the testing phase as much as possible to ensure that the required integration delivery phases can effectively meet timelines. To effectively implement DevOps practices, teams need to start the testing phase early in development and maintain it continuously throughout its lifecycle. The desired release frequency is easily achieved by detecting issues from the outset, when they are easier to fix and don't cost a fortune.
However, you need to use different testing methods at each stage of the development cycle. The different types of automated testing you need on different projects include regression, unit, performance, integration, load, accessibility, security and production monitoring as well as functional testing, etc.
3. Automation and DevOps
1. What is the significance?
In order to adapt to the speed of frequent delivery, testing work needs to be more efficient, which makes automated testing a must. When it comes to automated testing, we have to talk about the testing pyramid. The pyramid is roughly divided into unit testing, interface testing, and interface testing from bottom to top. The lower the bottom of the pyramid, the lower the cost of solving problems. The higher up the pyramid, the higher the cost of solving problems and the lower the efficiency. Therefore, according to the pyramid principle, we must do enough in terms of unit testing at the lowest level. Adequacy here is not only a large number of use cases and high coverage, but also requires high test quality. If a problem is discovered during interface testing, we need to go from interface to interface, module, etc., and require the cooperation of multiple parties to locate the problem. If a problem is found in the unit test, it is a problem with the module itself, and the problem will be located and solved much faster. Automated testing can improve both testing efficiency and quality. However, automated testing has certain requirements for testers' automated testing capabilities and testing tools, and requires a certain amount of investment, but the later benefits are also obvious.
2. How to advance?
In the agile development life cycle, we enrich and update the product through each iteration so that it best meets the customer's needs for the system. The focus of testing is in the development stage to ensure that the product meets the launch standards. After introducing DevOps, we not only focus on the quality of products, but also on the timely verification of product value. Therefore, we not only need to test shift left, discover problems during the development and even design stages and verify them in time, but also test shift right, verify the value of the product by monitoring the product's operating data in the production environment, and obtain feedback in a timely manner, so as to continuously improve the product. .
While we are advancing the DevOps project, we are also constantly exploring how to better complete testing work under DevOps. In DevOps, testing is not only used to detect product problems early and verify product quality, but also to verify product value and obtain feedback to achieve the purpose of continuous product improvement. Testing is no longer the exclusive job of testers. The entire team should be responsible for quality. DevOps has put forward higher requirements for automated testing. As a team member, everyone should improve their testing skills, and testers should be more Pay attention to the improvement of automated testing skills and team members work together to better utilize the value of testing.
3. What resources are needed?
DevOps tools are all open source and support everything from container building and orchestration to microservices networking, configuration management, CI/CD automation, full-stack monitoring, and more. Here are some of the most popular DevOps tools today:
Kubernetes: Instead of manually publishing microservices, Kubernetes automates the deployment, maintenance, and scaling of fleets of containers in production
Docker: Docker is a free, open-source platform for building, publishing, and running applications in the form of lightweight containers . Containers package the binaries, libraries, configuration files, and dependencies required for a program to run.
Jenkins: A big part of the DevOps philosophy is finding ways to automate and deploy new iterations more efficiently. Part of this goal is to create streamlined continuous integration and continuous delivery (CI/CD) pipelines. Jenkins is an open source automation server with hundreds of plug-ins that automate the building, deployment, and testing of software projects.
ELK Stack: ELK Stack is a combination of three open source projects: Elasticsearch, Logstash and Kibana maintained by Elastic. Using these three components, developers can acquire and log data from any source and create useful visualizations.
Prometheus: The Prometheus server collects time series metrics by scraping HTTP endpoints and generates systems that interact with that data, providing deep querying, visualization, storage, and other capabilities.
Istio: Istio is an open source service mesh built on Envoy, opening it up to plugins and extensibility options.
4. Summary
The process of DevOps relies on automated setup, configuration and deployment to ensure faster delivery of updates. All this makes automated testing a critical part of CI/CD as every code commit needs to be properly tested before deployment. Test automation helps find and fix bugs faster while they are still small. It can reduce risk while responding to customer needs within days or even hours.
The advantages of automated testing provide incredibly high efficiency in DevOps. However, the general practice of testing has still not caught up with modern technology. A study by Capgemini shows that large and medium-sized enterprises are not fully using automation. The research was conducted through interviews with 500 senior IT executives. The study also revealed that only 24% of test cases are executed through automated testing.
Test automation is crucial for DevOps to ensure timely delivery of high-quality deliverables. However, it can never completely eliminate dependence on manual testing methods. The right strategy should be to strike the best balance between automated and manual testing to get the best results.
Finally, I would like to thank everyone who reads my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, if you can use it, you can take it directly:
