Table of contents
1. "Developer testing" means "developers to test"
2. There is no test that cannot be "automated test"
3. Developers test "benefit in the present" and "win the future"
4. TDD is not necessary to write test code first
5. 100% UT coverage is really bad
6. Use tests to drive architecture and code quality
7. From "I want to write test-dependent code" to "I want to write test-dependent code"
1. "Developer testing" means "developers to test"
Developer testing is a very important part of modern software engineering. Advanced project management methods and processes such as agile development and backbone development are based on perfect developer testing. When a version is delivered every month or even every week, it is impossible to invest a large number of test engineers to conduct large-scale system-level tests. At this time, it is necessary to automate most of the tests in the entire test pyramid.
Let's talk about developer testing today. What is "developer testing"? Our company has a clear distinction between development and testing. Writing code belongs to the development of the siege lion, and testing belongs to the test of the siege lion. In most cases, the two sides are in a state of "confrontation between red and blue". This is very similar to the situation of my R&D team more than 10 years ago. However, in current software engineering, there are very few full-time "testing siege lions". Many companies have a development and testing ratio greater than 10:1, and even some departments do not test siege lions. The role of the test siege lion is no longer a "coolie" who manually runs test cases, but manages the product test system, plans product tests, analyzes and summarizes function test mind maps, designs test cases, and leads the R&D team to test Work more like a "testing expert/testing coach". To give a simple example, the product I made before was an online video conference collaboration product. Our daily online regular meeting is to use our own products, and we will use the new function testing site developed by ourselves to hold "stand-up meetings". In addition to spending a small amount of time on dialy update, the test experts lead the team (including PO, architect, SM, Dev) to carry out concentrated (half an hour) testing according to the plan. Therefore, "developer testing" means "developers to test", and many of our traditional test engineers face three ways out: growth, transformation, and elimination. "Testing experts" also have a high right to speak in the project. My previous company used backbone development and had a "one-in-one-out" review. This type of "testing expert" in the team has a veto power. There are even PE-level testing experts in the company (equivalent to 20-21 technical experts in our company).
- One-in: For whether a feature can enter the release branch, open the feature toggle in the release branch for release-level testing.
- One out: During the engineer release, the quality of the function is qualified, and the feature toggle is allowed to enter the production line.
2. There is no test that cannot be "automated test"
Back to the test pyramid, from the perspective of the four dimensions of "development cost", "execution cost", "test coverage", and "problem location" of the test, white-box testing based on the code level is extremely important.
- Development Cost: The cost of implementing the test case.
- Execution cost: The cost of running a test case.
- Test coverage: what we usually call line coverage and branch coverage
- Problem location: problems occur in the test, and the efficiency of locating the problem
Through the test pyramid and its four test dimension evaluations, we can conclude that:
- Do as many Low Level Tests as possible: Because their execution speed is much faster and relatively stable compared to the upper-level test types, they can be executed multiple times a day. Generally speaking, LLT gray is implemented in the continuous integration construction task, and even executed in MR to ensure the quality of the code entering the code warehouse.
- In the case of automation guarantee, execute a certain scale of IT, ST, and UI Test: because their execution speed is slow and the environment depends more, the test is unstable first. It is usually performed once at night to periodically check the code quality and give feedback on code problems.
- Do as little large-scale manual testing as possible: because their execution speed is lower than that of LLT and is not stable enough, the labor cost is high, and they cannot be executed multiple times a day, and each execution has to wait for a long time to get feedback results. However, they are closer to real user scenarios, so it is necessary to ensure that the following tests are performed within a certain period or at a critical node time to ensure software quality.
Now many companies have iterative release cycles getting shorter and shorter, even reaching 2 weeks. Manual testing obviously cannot adapt to this development mode, and automating the use cases of manual testing through various technical solutions is the only way. At the code level, from the underlying business code to the UI code, as long as the architecture design is reasonable, UT can be done. The top-level UI interaction test, test cases can also be automated (most UI frameworks can conduct UI automated testing through the accessibility interface), imagine that even our mobile phone hardware can automatically test the extreme test of "dropping the phone", the software has What can't be done? At least for some products of some of the industry's leading technology companies, from the code submission of the Merge Request to the production line of the product can be calculated in days. Testing of this product will not and cannot be done by manual testing.
3. Developers test "benefit in the present" and "win the future"
Many people think that the underlying developer tests spend a lot of time and write a lot of code to ensure the correctness of the function, but every time the code function or structure changes, the test code must be modified. It is more efficient for me to debug and verify manually. It is true that debugging the code through UT and API tests is not much different from manually running the debugging, but debugging the code through developer testing can ensure the quality of the current project iteration; but its more important role is not this.
We have such nouns in the bug category: Build Regression Bug, Release Regression Bug.
- Build Regression Bug: The same function in development has a bug in the new version, but there is no such problem in the previous version, we call it Build Regression Bug.
- Release Regression Bug: The same function on the production line has a bug in the new version, but there is no such problem in the previous version, we call it Release Regression Bug.
Every time we commit the code to the product, no one can guarantee 100% that there will be no problems. Under the rapid iteration of agile development, it is impossible to perform full-featured manual testing, so developer testing, especially the underlying The UT, API test, and integration test can easily identify and discover such problems. Therefore, it is said that developers test "benefit in the present" and "win the future".
4. TDD is not necessary to write test code first
For TDD, everyone's perception is to write the test code first, and then write the implementation code. This statement is right or wrong. The concept is correct, but if you strictly do this, the efficiency may not be the highest, which is one of the reasons why TDD is difficult to promote. We divide the coding implementation into three parts: implementing code, testing code, and debugging code. According to the concept of TDD, write test code first, then code, and finally debug. Usually, when implementing code, we are unlikely to consider it very clearly at the beginning, and define the interface completely and accurately. If we strictly follow the testing, coding, and debugging, the test code must be frequently modified along with the coding. Of course, this is not a big problem in itself. In the actual implementation process, many people are used to setting up the code framework and test framework first, and then coding and testing. Debug after the test is completed. Therefore, from the perspective of Huawei's grayscale management, as long as the unit test is performed before debugging, it can be called the TDD development mode. BTW, of course BDD is becoming popular now. What I want to say here is that if the team can't even do the TDD I mentioned, don't consider BDD.
(Behavior-Driven Development: BDD combines the general techniques and principles of TDD with the ideas of Domain-Driven Design (DDD). BDD is a design activity where you gradually build functional blocks based on expected behavior. The focus of BDD is on the Languages and interactions used. Behavior-driven developers use their native language in combination with Domain-Driven Design language to describe the purpose and benefits of their code. Teams using BDD should be able to provide extensive "functional documentation" in the form of user stories ", and add executable scenarios or examples. BDD usually helps domain experts understand the implementation rather than exposing code-level testing. It's usually defined in GWT format: GIVEN WHEN & THEN.)
5. 100% UT coverage is really bad
For unit testing, we all pay attention to an indicator "coverage". Regardless of the coverage of modules, functions, lines, and branches, there must be a certain percentage of coverage. But if you have achieved 100% in each item, then you will be given a "bad review". It's not that you can't do it well (we don't talk about whether you used the right way here), but the cost and cost performance. With the most difficult branch coverage (branch coverage), if you want to achieve 100% coverage, some branches of memory allocation or fault tolerance protection must be tested, then your test cases should be doubled, but it does not bring to the corresponding value. Even some code conditional branches have not been executed during the life cycle of the program running.
- Module coverage: The business module code passes through UT, and the architecture module code passes through IT; from the perspective of UT coverage, there is no need to test the architecture code.
- Function coverage: Don't write UT for some code without any logic. For example, some of our functions are get/set an attribute, and the internal implementation uses a variable to assign and save. This kind of function writing UT is written for coverage, without any real meaning.
- Line coverage: Generally speaking, a line coverage of 80% or so is a reasonable indicator. Some can be 0%, while some need 100%. If all codes exceed 90%, the cost is high and the efficiency is low. This is not recommended.
- Branch coverage: the more complex the business logic is, the more test cases need to be written to cover it, and some memory allocation error logic judgments do not need to be tested.
6. Use tests to drive architecture and code quality
Here we talk about test-driven architecture and code quality, mainly to make the code have perfect testability. What is the testability of the code? To put it simply, it is decoupling between classes and between modules, and programming between classes and modules through interfaces. The dependent interface is passed in through passive injection instead of active acquisition. When the program is running normally, the interface parameters passed in are real business objects, and when testing, the simulation implementation of fake can be passed in. Of course, not all dependent modules do this. Some Utility Libraries that have nothing to do with business, or some specific data object implementations, can be called directly.
Here we talked about fake and mock. Regarding Test Doubles, the basic concept is as follows. What does each type represent? You can search online by yourself.
- dummy
- stub
- spy
- mock object
- fake object
At present, everyone in our company is basically using Mock Object when doing developer testing (in fact, in the process of using it, many of them are using the Stub controlled by the return value of the input parameter). Aside from the conceptual problems, although the code can be tested through Mock, in fact, using Mock basically means that our code has strong correlation, the module display has heavy dependencies, and the module portability is poor, especially in C language. There are many such problems in programming. As a result, many modules are now unable to carry out unit tests at all, and more integration tests are being done.
Why does this happen? Our high-level architects are more concerned with the system-level architecture design, sorting out the relationship between system modules and various applications very clearly. Usually, high-level architects can make the relationship between system modules or applications The relationship design is more reasonable. However, when it comes to the design and implementation of the specific application business, it is handed over to low-level architects. In fact, the amount of code inside these modules is not small, many of which are hundreds of thousands or even millions of lines of code. At this time, the level of the architect determines the Clean Code quality of the code. Many of the problems in our current code are not system architecture problems, but rather the lack of strict requirements and reasonable architecture design in the implementation of specific businesses. If there is a set of architectural solutions to standardize at the application level, then at least the interface of the module and the interaction between modules can be as clear and reasonable as the system design. Then the uncertain part is the code part with tens of thousands of lines inside each submodule.
The reason for proposing to use test-driven architecture and code quality is that when a high standard is proposed for testing, everyone has to solve the testing problem from the architecture. When the testing problem is solved, Clean Code L3 will naturally be achieved.
7. From "I want to write test-dependent code" to "I want to write test-dependent code"
This sentence looks strange, but it is actually the fundamental way to solve unit testing fundamentally. There are dependencies between modules, whether it is through Mock or Fake, no matter how reasonable the architecture is, this kind of dependency cannot be eliminated. We do more reasonable design to decouple dependencies from modules. The first "I want to write test dependent code" means that when I implement my module, I need to write test code to test. However, what I want to test is how to write my test dependencies. And the second "I want to write test dependent code" means that when I implement my code, what I have to consider is how to solve my dependencies when I rely on my modules to test, "I want to write tests Dependency code" (that is, what I call fake objects and implementations) to help rely on my modules to solve test dependency problems.
- Thinking change, test-driven: When developing a module, don’t think about how to test yourself first, but how can I make it easier for others to test if others rely on me. The provider of the module should not only provide the module code, but also provide a reusable faked object (call verification; return value; parameter verification; parameter processing; function simulation, etc.).
- The writer of the module code implements his own Fake implementation, basically most of the code is done by the module writer, and this is a reusable Fake implementation. The module relying party adds its own code according to some special business requirements of its own. Basically follow the 80/20 principle.
- Dependency decoupling on the architecture, interface programming by injecting dependencies. Developer tests use Fake to implement dependencies.
- When writing test code, all dependent interfaces and dependent implementations are basically completed, and more attention should be paid to test cases rather than test dependencies.
Summarize:
Thanks to everyone who read my article carefully! ! !
I personally sorted out some technical materials I have compiled in my software testing career in the past few years, including: e-books, resume modules, various job templates, interview books, self-study projects, etc. Welcome everyone to click on the business card below to get it for free, don't miss it.
Python automated testing learning exchange group: a full set of automated testing interview resume learning materials to obtain Click the link to join the group chat [python automated testing exchange]: 
http://qm.qq.com/cgi-bin/qm/qr?_wv=1027&k=DhOSZDNS -qzT5QKbFQMsfJ7DsrFfKpOF&authKey=eBt%2BF%2FBK81lVLcsLKaFqnvDAVA8IdNsGC7J0YV73w8V%2FJpdbby66r7vJ1rsPIifg&noverify=0&group_code=198408628
