Testing is an important part of the software development life cycle (SDLC). Every stage of the SDLC should include testing to get faster feedback and improve product quality.
Let’s talk about topics related to continuous testing and how to get more benefits from continuous testing.
What is continuous testing
Continuous testing helps provide faster quality feedback at all stages of the software development life cycle (SDLC). In most cases during the software development lifecycle, if unit testing and automated testing are performed less on the core functional logic, the pressure to perform manual exploratory testing at the top of the testing pyramid will be very high, and quality problems may easily occur, leading to project delays.
This can actually impact quality, as it is very expensive to catch bugs after development is complete. Below is the cost of fixing a bug at Google. You can see that when a bug is discovered during system testing, it can cost up to $5,000.
| BUG discovery stage | Cost (USD) |
|---|---|
| System test | 5000 |
| Integration Testing | 500 |
| Compile and build | 50 |
| Single test/self-test | 5 |
Continuous testing helps us evaluate the quality of this current progress on the software by providing early feedback as soon as the code is committed to the repository. The main goal of continuous testing is to test as early as possible, test as often as possible, and get feedback on the build faster through automation at all stages of the SDLC.
Using continuous testing, we can step in and fix any failures as soon as they occur before moving on to the next stage, ultimately helping to save overall time and money.
Why Continuous Testing
In one of my previous projects, we were developing a mobile app for iOS and Android platforms. Customers want everything to be automated from the start. Any bug that leaks into production means it will directly impact the business and cost several customers.
We were asked to come up with an automation plan where testing would be done at every stage of development to minimize the risk of bug leaks. Therefore, we decided to implement a testing pyramid and create a CI/CD pipeline that tests continuously at each stage.
To improve product quality, we came up with a plan to conduct testing at every stage of the pipeline and once any error signal appeared, it should be resolved before we move to another stage. So whenever a developer commits code to the remote repository, the following scan is run
- Static code analysis: This will ensure that best coding practices are followed and alert us through code bad smells in case of any errors.
- SecOps Scanning: This will scan the code and all libraries used in the code for any security vulnerabilities and alert you if there are vulnerabilities that should be noted.
Once the above scan passes, the pipeline moves forward and runs the following tests in the development environment:
- unit test
- Integration Testing
- System test
- End-to-end testing
All the above tests will ensure that the code runs perfectly as expected.
If any of the above tests fails, the pipeline will be interrupted and an alert will be issued. The developer who submitted the code is responsible for fixing those corresponding failing tests. It's not about playing the blame game, it's about finding the commit that broke the build and fixing it. The team will provide assistance to developers to resolve issues.
Once all the above tests pass successfully, the build is deployed to the QA environment where end-to-end automated tests are again run on the test build as part of regression testing. Once the end-to-end automated tests pass the test build, you need to select the build and perform manual exploratory testing to find more defects. It will eventually be deployed to the acceptance environment, where a team of testers will conduct further testing, and finally, the build will be deployed to the production environment.
Static code analysis and SecOps scans help us implement best coding practices and fix vulnerable libraries by updating to the latest version or discarding and using them and updating them frequently so that the code is less prone to vulnerabilities Security risk.
While we also found issues during exploratory testing done manually, these were less severe and most were resolved during the initial phase itself, which gave us faster feedback.
现在我也找了很多测试的朋友，做了一个分享技术的交流群，共享了很多我们收集的技术文档和视频教程。
如果你不想再体验自学时找不到资源，没人解答问题，坚持几天便放弃的感受
可以加入我们一起交流。而且还有很多在自动化，性能，安全，测试开发等等方面有一定建树的技术大牛
分享他们的经验，还会分享很多直播讲座和技术沙龙
可以免费学习！划重点！开源的！！！
qq群号：691998057【暗号：csdn999】
Continuous testing throughout the SDLC lifecycle
- Requirements change frequently: As requirements change frequently, code changes also need to be made, and every change we make involves risk. There are two risks involved here, whether the changed code will work as expected, and the other is whether this change will affect existing code. With continuous testing, we can address both risks by setting up an automated pipeline that will run unit, integration, and ultimately automated regression tests.
- Continuous Integration: Continuous Integration has gained widespread popularity with the implementation of agile development, where developers merge as much of their code as possible into the master branch to make it production-ready. Before the merge occurs, the code is run through automated tests on the build in the pipeline. If a failure occurs, the codes are not merged and an early warning is issued.
- Production-ready: With continuous testing, we are production-ready because all our checks and tests run on an automated pipeline as soon as a developer commits their code.
- Reduces human errors: In the case of regression testing, if automated tests are written, it serves as a documented proof of the functionality and helps reduce human errors in testing.
Advantages of continuous testing
- Fast feedback: In traditional software development, teams have to wait for feedback from testers, who will manually test the build after developers finish writing features. After feedback from testers, they had to rework to fix issues that were time-consuming and more expensive. Through continuous testing, we can get feedback on new code commits faster, saving time and money.
- Quality is baked into the product: By running all tests in an automated pipeline, from unit, integration, functional, security, performance and end-to-end user, we ensure that quality is baked into the product itself and there is no need to worry about releasing it to production.
- Reduce bug leakage: Continuous testing helps eliminate the chances of bugs in the build by providing us with timely updates about software glitches.
- Minimizing risks: It also helps in identifying risks, addressing them and improving product quality.
As the demand for high-quality software continues to grow and an economy with digitalization at its core booms, continuous testing is considered an important aspect. Software companies need to respond to the frequent changes that occur every day in their SDLC, and continuous testing is the answer.
Important Types of Continuous Testing
- Unit testing: This involves testing a piece of code in isolation that implements a certain functionality. Basically the test writes a use case for that functionality covering each method. The main purpose of this test is to check if the code works as expected, which means that all functions, inputs, outputs, and performance of the code are as expected.
- Integration testing: This involves testing two modules together. The purpose of this test is to check whether the integration between the two components is working properly.
- Regression testing: This is the most widely used test to check whether the existing functionality of the application is working as expected after recent additions or modifications to the code repository,
- End-to-end tests: These tests are added to check the end-to-end working of the software. The purpose of these tests is to check whether end users are able to use the application end-to-end.
The role of cloud service platform
In today's fast-paced world, software can run on multiple platforms, from browsers to mobile phones and tablets. When we release an application to production, we need to make sure it runs as expected on all required platforms and fix the things we find aren't working.
To do this however, we need to test it on the respective device/browser to make sure it works without any worries. It is possible, but it will cost money and time since we have to buy the hardware and provide the required resources to make it work. From hiring engineers to building infrastructure.
As we continue to test and perform parallel runs on different browsers and their respective versions or different mobile devices with different operating system versions, these services provide us with the required devices, browsers/operating systems and their respective versions to help us continuously test, so we catch bugs early and use early feedback to fix needed issues and stop bugs from leaking.
in conclusion
Quality is an important part of software and needs to be reflected in software. Continuous testing helps us build the right product by implementing testing at every stage of the software development life cycle.
We need to make every feature we build production-ready, and it's necessary to get fast feedback through a fail-fast strategy. There are various test types available that help us implement continuous testing using automated pipelines.
Finally, I would like to thank everyone who read my article carefully. Looking at the increase in fans and attention, there is always some courtesy. Although it is not a very valuable thing, if you can use it, you can take it directly!
Software Testing Interview Document
We must study to find a high-paying job. The following interview questions are from the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and some Byte bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
