Author: JD Retail Zhang Qiang
I write this article in the hope of summarizing some of my experiences that I think are very valuable, which can help new colleagues who have just done testing, or old colleagues with rich testing experience to share. I hope that our lovely new colleagues, partners who are going to work in the testing field, can understand the underlying logic of testing through my articles, that is, we may not see hidden deeper points in our testing work, not just what we see every day Write use cases, raise bugs, develop automation, and build platforms; as the saying goes, laymen watch the excitement, and experts watch the way.
I think that testers should not be PRD porters, and senior test engineers should not just be developers of test tools;
Testers must master the most basic basic theory of testing, and of course R&D coding technology is also essential; if the two lack the same, they will not be able to become an excellent tester; many testers did not like to write code before, and then Tested; but in the future or now, it is difficult for a tester who does not understand code to become an excellent tester; but a person who only understands code and does not understand the theoretical basis of testing (does not understand test analysis, use case design, test strategy) people, or even if they know something, but don't use it in actual work), they must not be a qualified tester.
Let's take you to understand some of the underlying logic of testing and the way of testing.
1. The core competencies that excellent testers should possess
According to Testerhome's "2021 Testing Industry Questionnaire Survey Report" - [Technology/Abilities Excellent Testers Should Possess] analysis,
1. "Programming/scripting/automation, communication expression, and basic testing theory" are considered to be the three core competencies of excellent testers , and continue to lead other items;
2. Technical requirements for databases, performance testing, security testing, and big data algorithms will increase significantly from 2020 ;
The three core competencies are basically recognized by everyone, and they are also stable; however, there has been a large demand for new technical requirements in recent years; from the analysis data, it can be seen that the market's requirements for testers will increase with the development of new technologies. However, the three core competencies are compulsory courses for testers, and the changes will not be too great, and they will always occupy the core position.
Since the beginning of QTP more than 10 years ago, automated testing has been the goal pursued by testers; until today, various automation technologies and frameworks have been dazzling; the market has higher and higher requirements for testers, and testers must not only be able to write automation use cases , but also need to have the ability to develop and maintain the automation framework platform; pure black box testers have either completed the capability upgrade or are on the way to upgrade; Use cases or do not understand the programming language, it is estimated that it will be difficult to pass the job resume.
However, things are often reversed. The code ability of testers is getting stronger and stronger, but the basic ability of testing is ignored, and the professional ability in the field of testing is gradually weakened; just like going against the current, if you don’t advance, you will retreat; the three core capabilities should go hand in hand, and should not Take care of one thing and lose another.
I have participated in many recruitment interviews in the department over the years. My feeling is that although many testers have worked for many years, they do not have a good grasp of test case design methods and strategies; at least 60% of them do not know what to use in use case design. The use case design method does not think about how to conduct test analysis and design. Most of them are just executors of functional tests. They rarely think about test design, and few people write test plans. Test cases are just splitting of PRD; In short, testers will become PRD porters if they are not careful.
As an old tester, I still hope that the testing industry can develop healthily. With the improvement of new skills, the professionalism of testing can also keep pace with the times. After all, quality assurance is the foundation of testers.
2. The underlying logic of black box testing
What is black box testing?
It regards the program as a black box, and checks whether the program function is normally used according to the PRD regulations, whether the program can properly receive input data, and produce correct output without considering the internal structure of the program.
This is actually the definition of black-box testing and the underlying logic of black-box testing; most people don't pay attention to definitions, but definitions often tell you the truth.
Many people at work are used to one type of system testing, and then switch to a new type of business, and suddenly they don’t know how to start; maybe there is always a time to adapt to new ones; in fact, everything is the same, as long as Mastering the underlying logic of black-box testing will allow you to get started quickly without adapting and adjusting;
Most of what we do is black-box testing, so no matter what type of system, our testing plan is " check whether the program functions normally according to the PRD, whether the program can properly receive input data, and produce correct output. " .
Our test is based on PRD . First of all, we must have a good grasp of PRD, and then analyze its input and output; these are all covered, and you can basically achieve 80 points, that is, you have won this project. Not a problem.
Finally, I still want to emphasize again, I am afraid that everyone still does not understand what I said, because everyone understands what I said above. If you understand the test on the first day, you will know when the black box test is and what is the input and output; but often The truth is hidden in the ordinary, remember his definition! ! !
When you encounter a project and don’t know how to start testing, take out the definition and read it carefully three times, and you will definitely find the answer! ! !
Emphasis: In fact, there are not many pure black boxes. In addition to understanding the input and output, the intermediate processing logic must also be clear, which is more helpful for testing; in addition, the more important thing is: you must be familiar with PRD, and you must be familiar with PRD. The content analysis is thorough, and no paragraph of text or word is missed. In fact, there will be many loopholes in the PRD and design documents waiting for you to dig.
3. Detailed explanation of the underlying logic of black box testing: [input and output test model]
[Input]: The input here is not a simple interface input box; any input that can trigger the system to run is an input. According to the layering of the code structure, the input can also be classified as follows:
1. Input for interface operation :
Forward operation:
1) Single operation:
•Normal operations: specified operations of input boxes, buttons, radio check boxes, buttons, drop-down boxes, etc. Abnormal operations: abnormal values of input boxes, long input, etc., multiple clicks of buttons, rapid continuous clicks (very fast) It is easy to find various problems such as repeated submission of data, or slow system response, and the system may crash because of this)
2) Complex operations:
•Combined operation: the general system functions are a combination of various operations; the other is related to business scenarios, that is, various business scenarios are combined and operated at the same time.
• Parallel operation: multiple people operate concurrently on the same function point; or multiple people operate on the same data, such as two people modifying or deleting a unit price at the same time.
Reverse operation:
3) Reverse operation:
• Rollback operation: Rollback operation through browser or APP Cancellation operation: Normal operation is suddenly canceled, for example, the user fills in a lot of forms and suddenly cancels. Do you need to save or prompt? Delete operation: delete data through the functions provided by the system
The following inputs are the easiest to ignore:
2. Input of the service layer:
• Interface service: The interface provided externally is also a very common input for the system, and this input is also the most prone to problems;
•File upload: Some system functions trigger the system to run by obtaining excel, xml and other files on the ftp server, so the input at this time becomes a file.
•MQ message: It is also the most common input form of JD. MQ may also include file addresses, etc., which makes this input more flexible.
emphasize:
• For the input upstream of the interface, regardless of the form, each field of the upstream data must be analyzed to understand the possibility of various upstream inputs.
• For some fields, the meaning of this field, possible enumerated values, possible results, etc. must be understood from the business [source] ;
• In addition, due to historical reasons, there may be various unimaginable data in the source data;
•It is very important for the analysis of the input, at this time you can use the [equivalent class] method for analysis.
3. Input of the data layer:
• Data changes: There are many background processing tasks that monitor whether there is new data insertion or deletion, etc. Changes in data fields: background processing tasks monitor changes in data status, or changes in combined fields and other cached data changes: in addition to database Changes, some of which are changes in the time of cached data changes: timed tasks are not only the input of data, but also the time.
【Output】: The output is divided into visible output and invisible output
Visible output: It is our common system operation feedback, changes that users can directly see; such as pop-up boxes, prompts, jumps, changes after data addition, modification, and deletion, pictures, videos, etc. after operations change and so on.
Invisible output: Invisible output is the easiest to ignore and the most problematic; [Invisible output] includes: database changes, cache changes, system file changes, data sent to downstream interfaces, etc.
[Visible output] Although it can help us verify more than 90% of the functions, the data displayed through the interface can also verify the data we add or modify, whether the new addition is successful or the correct modification; but we see Only a part of it is seen; there are many fields that are not displayed, some may only be used by downstream or other systems, and some may be reserved for future use; these invisible parts often cause system exceptions, It is also the biggest pit hidden in the system;
Therefore, testing should not only test the system from the user's point of view, but also test from the designer's point of view, and should also be considered from the point of view of the entire product.
4. The underlying logic of test analysis and design
When it comes to test analysis and design, I think this is the core ability of testers; the black-box test and input-output model mentioned above are only methods for functional testing, although functional testing accounts for 80% of general system testing- About 90%, but not all. And it is only one stage and one type in the test. To do a good test, test analysis and design are essential.
You can think about a question: how do you test when you get a project? How to ensure the quality?
The answer that many people gave me during the interview was: analyze requirements, write use cases, execute tests, and send reports; Doing testing and analysis will not guarantee the quality of the system.
How do you test a project?
You can use the 5W2H method to analyze. In fact, as a test architect, you don’t need 5W or 3W. 2W+1H is enough!
Because this 2W+1H is the most important, and it is also the easiest to be replaced by experience and then ignored; the daily test work is basically fixed due to the shape of the product and the structure of the system, so the idea of the test plan is also basically fixed, which leads to getting similar I no longer have to think about the project, and basically start working quickly based on experience; once I encounter a different type of system or encounter a new business, I don’t know how to start. At this time, the 2W1H analysis method can help us solve this problem .
Test architects only need to think about three questions ( 2W+1H) :
1. Why? Why do you do this project? What is the background of the project? ——Only knowing why can you know what users want.
2. What? What do we need to test for this project? What is the scope of our testing? ——Only when the scope is clear, the test will not be missed.
3. How? How do we measure this project? What testing strategies, testing methods should we use? ——This tells us that testing should have strategies and methods
Two more questions for the test lead (and possibly the test architect) to determine:
4. When? What is the expected completion time of the project?
5. Who? What resources can be called?
Two other questions that the project manager needs to consider: (Two questions that the test leader also needs to think about)
6. Where? Does it need to be centralized and closed, and does it need to sit together for R&D and testing?
Testers can also think about what environment they need to test in? test environment? Pre-launch environment? online environment? Windows environment? Linux environment? ios environment? Android? what browser? what version etc.
7. How Much? How much does this project cost? How many man-days are needed? How much server resources are needed?
The underlying logic of test analysis and design is how to answer the three questions of 2W1H; Why and What can guide us to conduct test analysis, understand the [background] of the project, and confirm the [scope] of the test; How can guide us to carry out test design, Determine the [strategy] of the test according to the project background and test scope.
However, the current discussion is still on the methodology, and the specific operation steps have not been discussed yet; since this part has a lot of content, it can be explained in detail through an article; you can also learn about it yourself, which is the HTSM heuristic test strategy model, this model It just corresponds to 2W+1H.
5. Internal strength training of testers
As a tester, "soft skills such as communication and expression" is considered to be one of the three core competencies of an excellent tester. According to the above statistics, more than 90% of people recognize it. Let me share some of my previous conclusions:
1. Active communication
In the field of e-commerce, the characteristics are fast and changing; some needs or projects often require fast launch. Due to the short time, some logics in PRD may have loopholes or have not been considered at all. Faced with such a situation, how should we test What to do? At this time, communication is needed. Communicate requirements with products at any time, communicate design with development at any time, and communicate with other systems for joint debugging at any time. Without communication, pitfalls in the project are easily missed . Communication must also be proactive, looking for products, research and development, and other lines of testing. Treat yourself as the boss, and the quality of this project is basically guaranteed; employees who regard themselves as the boss must be the most reassuring to the boss staff.
2. Have your own standards
The most basic basis for system testing is the requirements specification; as testers, we are the last guarantee; testing must have its own analysis, and you cannot easily follow the R&D ideas , because what he tells you has already been processed by them , there may already be deviations from the original requirements; this is where the value of testing lies. Even if 99% of what they say is correct, it can only be used as a material for our analysis; we must analyze it ourselves through requirements.
3. Be skeptical about everything
Requirements are the basis for testing, but the basis is sometimes wrong; therefore, we must also have a skeptical attitude towards PRD. Testing is to doubt everything; every detail of every process; when I look at the requirements, I basically assume that he is right for the first time. After I have a certain understanding of the whole, I start to doubt whether the process is complete? Are there any loopholes? Can the function of the module meet the user's requirements? Will there be problems with abnormal operation? Will the user know how to use it? Does this feature actually solve a problem for the customer? Wait, these questions can be doubted and thought through some of our quality standards, testing strategies, and testing experience. In short, "think twice" when testing every function.
4. Think from the perspective of the company and users
The bigger the company, the more departments there are, and the more complex the system will be; the more interdependence, the greater the chance of problems; because there are more boundaries, the cost of communication will be higher; there are more points that need to be communicated, as long as there are some details If the communication is not in place, or they are not considered, or they all think that the other party is responsible, then pitfalls will appear; of course, most of such pitfalls will be found during the joint debugging test stage, but they have a great impact on the test progress and often cause reverse work , delay and other risks;
If these pits can be found in the testing phase, then there needs to be a main test; but I think the main test should not be a person, all testers should be the "main test"; as a tester, the final test of software quality Gatekeepers should not only focus on the part they are responsible for, but should not be limited to their own departments and teams. As long as there is any doubt about the entire system, we all have the responsibility to raise it and find someone to solve it. The test should not only look at the part, but also the overall situation; we must think and test from the perspective of the company and the user.
The above mainly summarizes some of my experience, some methods in the test, and I hope you can add more if there are deficiencies or not comprehensive enough; I will continue to share the HTSM model that I think is very important, and what I think is very important, etc. Valence classification, scenario testing, genetic testing, some good methods of exploratory testing, etc. In short, I want to share some of my valuable experiences for sharing.