During the ChinaTest conference, the three questions I heard the most were:
-
Prospects of the testing industry;
-
Career planning for testers;
-
KPI。
For the first question, I am confident. The complexity of modern software and hardware systems is increasing at a geometric rate, and the more complex systems need to be tested, the greater the intensity of testing, as can be seen from the rapid increase in the number of testing practitioners in recent years. Until a world like The Matrix is ruled by artificial intelligence, testing is still promising. Regarding the third question, it seems that no matter whether it is the grassroots, middle or high level, it will be a headache. Duan Nian also said in his final lightning speech:
We cannot provide a ready-made method for the time being, and we can only make KPIs a good guide. act on the principle.
I am also one of the people who is still worried about KPIs, so I won't say more.
Regarding the second question, I think whether it is a new student, a test engineer who has been in the industry for a few years, or a big cow, they all need to face it and think about it carefully.
What is the future of doing testing? How to achieve excellence?
Perhaps this issue seems a bit utilitarian and too divergent, and ChinaTest did not specifically take it as an issue. However, from the current personnel structure of the entire industry, newcomers account for the vast majority, and it will be very important to have a bright light in the distance. When thinking about this question, I thought of the answer to this question by James Whittaker, the author of the book "Exploratory Software Testing" I read in the previous paragraph. He likened the path of testing career development to climbing a mountain, with good examples of what we should do at each stage. I had a strong resonance during the reading process. Now I found this article and posted it to everyone, hoping to help. (If there is any copyright violation, please notify me and I will delete it in time). It is particularly worth mentioning that a stage of the development of Jame testers is called
"downhill"
. When communicating with their peers, many people have penetrated this concept into their bones, and they have already walked on the road of "downhill". Testing has been developed in China for many years, and there are many great people in our testing field. This is very gratifying. Maybe there will be masters like Jame Bach and quasi-masters like James Whittaker in the next ten years. Let us witness together this process.
Running a Successful Testing Career
(James A. Whittaker）
How did you start doing testing work?
In 1989, when I was a graduate student at the University of Tennessee, I completed the transition from software developer to software tester. And this transition was not of my own choosing. My fate changed one morning when my professor asked me why I was absent from so many development meetings. I explained that because the meeting was scheduled for Saturday morning, it was inconvenient.
As a newly admitted graduate student who left home for the first time in his life, this time was a bit troublesome. Interestingly enough, the punishment for waiting for me was not a dismissal notice, but a punishment of being the only tester on the team and not having any communication with the development team.
What a great decision for my career! It was this decision that led to dozens of papers on testing, more tools than I can remember, five books published, and endless hours of joy at work. Testing has always been the joyful creative and technically challenging profession I have had. However, not everyone likes this. It can be said that my earliest exposure to testing was during my graduate studies. It is undeniable that the intense study and work at that time really benefited me a lot. Also, I think there is a "testing mountain" between the beginner stage and the expert stage, where people need to go through a series of personal coaching, getting information, and receiving regular instruction to climb the mountain. Becoming a testing beginner is easy, and becoming a professional tester is not difficult either. The focus of this chapter is on how to climb that mountain between professional testers and test experts.
As a newly admitted graduate student who left home for the first time in his life, this time was a bit troublesome. Interestingly enough, the punishment for waiting for me was not a dismissal notice, but a punishment of being the only tester on the team and not having any communication with the development team.
What a great decision for my career! It was this decision that led to dozens of papers on testing, more tools than I can remember, five books published, and endless hours of joy at work. Testing has always been the joyful creative and technically challenging profession I have had. However, not everyone likes this. It can be said that my earliest exposure to testing was during my graduate studies. It is undeniable that the intense study and work at that time really benefited me a lot. Also, I think there is a "testing mountain" between the beginner stage and the expert stage, where people need to go through a series of personal coaching, getting information, and receiving regular instruction to climb the mountain. Becoming a testing beginner is easy, and becoming a professional tester is not difficult either. The focus of this chapter is on how to climb that mountain between professional testers and test experts.
back to the Future
In the field of software testing, time seems to have stood still. The way we do things in the 21st century is almost identical to the last century. Bill Hetzel's Test Knowledge series, published in 1972, is still quite valuable today. And my own How to Break Software series, first published in 2002, is still today synonymous with the primary resource for practical software testing techniques.
Indeed, if we can take the testers of the 1970s from time to time and use them today, I would guess that their skills are adequate for testing modern software. Of course, they will need to learn networking and various networking protocols, but the actual testing skills they have will be put to good use. If you get a tester from the 1990s, hardly any training is required.
Not so for developers, who have mastered the skills of the last century that are almost completely outdated. Ask someone who hasn't written code for a while to start programming again and see how it reacts. What bothers me is that we can hire people straight from the road and hire people who can test and pay off from day one. Are things really that simple? Or are our expectations only that low? What makes me even more disturbed is that we don't have any predictable way to train the right test talent from competent to test specialist. Is the test really that difficult?
This is that mountain again. The barriers to entry are low, but the road to mastery is tough.
At the entrance to the test mountain, we relied on the fact that many aspects of the test are easy to grasp. Most people can learn well. Even just applying a little common sense to the choice of input, I'm flawed. Testing at this level is like fishing in a bucket, simple enough for anyone to think they're smart. After the entrance, however, the road steepens rapidly, and the test knowledge becomes increasingly obscure. We find people who are good at it, we call these people "gifted people" and appreciate their instincts.
Do you have to rely on instinct? Is there a way to climb the mountain for people who don't seem to have the skills? Could testing skills be taught in some way to produce more experts? To think this mountain is traversable, and this chapter is just my notes on how to go that way, that you can apply in your own career. This is not a cookbook recipe, a career cookbook. There are a few things you can do to accelerate your professional growth, though. But, as you might have guessed, it's really easier said than done.
Indeed, if we can take the testers of the 1970s from time to time and use them today, I would guess that their skills are adequate for testing modern software. Of course, they will need to learn networking and various networking protocols, but the actual testing skills they have will be put to good use. If you get a tester from the 1990s, hardly any training is required.
Not so for developers, who have mastered the skills of the last century that are almost completely outdated. Ask someone who hasn't written code for a while to start programming again and see how it reacts. What bothers me is that we can hire people straight from the road and hire people who can test and pay off from day one. Are things really that simple? Or are our expectations only that low? What makes me even more disturbed is that we don't have any predictable way to train the right test talent from competent to test specialist. Is the test really that difficult?
This is that mountain again. The barriers to entry are low, but the road to mastery is tough.
At the entrance to the test mountain, we relied on the fact that many aspects of the test are easy to grasp. Most people can learn well. Even just applying a little common sense to the choice of input, I'm flawed. Testing at this level is like fishing in a bucket, simple enough for anyone to think they're smart. After the entrance, however, the road steepens rapidly, and the test knowledge becomes increasingly obscure. We find people who are good at it, we call these people "gifted people" and appreciate their instincts.
Do you have to rely on instinct? Is there a way to climb the mountain for people who don't seem to have the skills? Could testing skills be taught in some way to produce more experts? To think this mountain is traversable, and this chapter is just my notes on how to go that way, that you can apply in your own career. This is not a cookbook recipe, a career cookbook. There are a few things you can do to accelerate your professional growth, though. But, as you might have guessed, it's really easier said than done.
go up the mountain
The early stages of a testing career are primarily about preparing for the long ascent of conquering test peaks. The best advice I can give is to think about it from two perspectives. For every project you work on, there are two (not necessarily equal) tasks. The task of the first part is to ensure the success of the current test project. And the second part of the task is to learn what you should do to make the next test project easier. I call it "testing today's project, preparing tomorrow's project". If you do each project and divide it into the above two halves, you're almost guaranteed to keep making progress. This way, you can gradually grow into a better tester with each project you work on.
Now let's focus on the second part of the task - preparing for the next project. We need to pay attention to three concepts: duplication, technology and vulnerability.
Now let's focus on the second part of the task - preparing for the next project. We need to pay attention to three concepts: duplication, technology and vulnerability.
repeat
Do anything and never do it twice without realizing or questioning that it is actually a problem. I hope all young testers keep this in mind. I've seen many beginners who waste too much time on monotonous tasks like, setting up a test machine, configuring the test environment, installing the application to be tested in the lab, choosing a production version to test - these tasks The list can get very long, and in the end you'll find that there's very little time really spent testing the software.
This is a common mistake many newbies make. They fail to see the repetitive nature of the work they do day in and day out, and by the time they become aware of the repetition, hours have passed during which they have not done any actual testing work . Watch out for the duplication of effort and the resulting loss of real software testing work time. In order to climb the mountain of testing, one must do the job of a tester, not a lab manager or a tester manager.
This is a common mistake many newbies make. They fail to see the repetitive nature of the work they do day in and day out, and by the time they become aware of the repetition, hours have passed during which they have not done any actual testing work . Watch out for the duplication of effort and the resulting loss of real software testing work time. In order to climb the mountain of testing, one must do the job of a tester, not a lab manager or a tester manager.
Test automation is a solution to duplication of effort and is a topic later in this chapter.
Technology
Testers often analyze software failures. When analyzing defects, we learn from developer failures how to write reliable code. We also analyze the flaws that we ignore. After the application goes to market, customers will start reporting bugs, and we're going to be faced with dealing with a bunch of failure scenarios and looking for critical bugs in them. Every bug reported by users is a problem with our process, and our testing knowledge is not perfect.
But analyzing our success is just as important, and many new testers fail to take advantage of this readily available resource. Every bug we find in our testing shows that our testing process is working effectively, and it's a success. We need to seize this great opportunity tightly, so that the success can be repeated over and over again.
Sports teams do this all the time, watching game footage and analyzing why each move worked or didn't. I vividly remember a little story where a friend of mine took some pictures of my son playing football. One of the photos captured the moment she kicked the ball, which passed the goalkeeper and managed to score. When I showed it to my son, I saw that the leg he was standing on was perfect, he kicked the ball with his toes tight and the ball hit just the right spot between the shoelaces. He stared at that picture for a long time, and since then he has rarely played with incorrect form. He probably just happened to get it right that time, but he's used those techniques consciously since then to make it close to perfection.
Now back to the novice tester lesson. Each of us will have moments of pride. We find important bugs or high-priority bugs and get excited. But take a moment to consider the overall situation first. What technique did we use to find that flaw? Can we create a way to find more of these kinds of defects? Can we memorize...some practical testing experience and apply it continuously to help improve our productivity? What symptoms of software suggest that it is defective? Could we get more warnings from those symptoms in the future? In other words, it's not just a flaw or a success, what does this flaw teach us, does it make us better testers in the future like my son's goal, even though the first flaw was discovered by accident , but it does not mean that the rest of the success is accidental. It is important to understand why we are successful, and only by doing so can success be replicated. For testers, the reason for this guaranteed success is a range of testing techniques, advice, and tools that can improve our productivity on future projects.
But analyzing our success is just as important, and many new testers fail to take advantage of this readily available resource. Every bug we find in our testing shows that our testing process is working effectively, and it's a success. We need to seize this great opportunity tightly, so that the success can be repeated over and over again.
Sports teams do this all the time, watching game footage and analyzing why each move worked or didn't. I vividly remember a little story where a friend of mine took some pictures of my son playing football. One of the photos captured the moment she kicked the ball, which passed the goalkeeper and managed to score. When I showed it to my son, I saw that the leg he was standing on was perfect, he kicked the ball with his toes tight and the ball hit just the right spot between the shoelaces. He stared at that picture for a long time, and since then he has rarely played with incorrect form. He probably just happened to get it right that time, but he's used those techniques consciously since then to make it close to perfection.
Now back to the novice tester lesson. Each of us will have moments of pride. We find important bugs or high-priority bugs and get excited. But take a moment to consider the overall situation first. What technique did we use to find that flaw? Can we create a way to find more of these kinds of defects? Can we memorize...some practical testing experience and apply it continuously to help improve our productivity? What symptoms of software suggest that it is defective? Could we get more warnings from those symptoms in the future? In other words, it's not just a flaw or a success, what does this flaw teach us, does it make us better testers in the future like my son's goal, even though the first flaw was discovered by accident , but it does not mean that the rest of the success is accidental. It is important to understand why we are successful, and only by doing so can success be replicated. For testers, the reason for this guaranteed success is a range of testing techniques, advice, and tools that can improve our productivity on future projects.
Vulnerability
Testers will eventually become good at finding bugs, but to get past the peak of testing, we have to be faster and more efficient: high speed and low resistance. In other words, we must have a defect-finding technique that is inherently defect-free!
I like to think of it this way: testers also need to use that same flaw-finding ability to look at their work. We have to find our own testing process, the flaws in the testing process, using the same process we use to find product defects. Is there something wrong with my testing process? Are there any flaws here? Are there any barriers here that are preventing me from being productive?
You have to keep looking for a better way. Consciously identify those things that limit ability, hinder progress, and slow things down. Just as defects limit the ability of software to meet user needs, what limits the ability to test? Use the testing abilities you have to optimize your testing process, which will help you climb fast on the peaks tested and increase your chances of becoming an expert once you've climbed them.
The summit of the test peak is a wonderful place. If you managed to get there, congratulations. But this is not the final target. This means that you have become a great tester. The downhill here is to use your insight and expert knowledge to help those around you become good testers too. It's one thing to reach the summit by yourself, it's quite another to help others (those who are not as capable as you) to reach the summit.
In general, those who have successfully reached the pinnacle of testing become masters of tools. Those commercial tools, open source free tools, and self-written tools (my personal favorite) are excellent ways to increase your productivity and productivity. However, a tool is just one way to achieve that, but in many other ways it's a limitation because too many people can't see beyond what a tool can do. They are limited in what the tool can do for them, failing to see or understand that there is more need for the tool. What you really need to get to the top is "information". Because many tools process information and make it easier to obtain, testers become overly dependent on their tools. But the information itself and how to use it is the real key to success.
Mastering information means understanding what information is available and how it will affect the test, ensuring that it is used to its fullest extent. There are several types of information that test climbers must focus on. I'm talking about two of them here: information from the application and information from previous tests.
Information from an application includes requirements, architecture, code structure, source code... and even operational information about what the application does when it executes. This type of information needs to be considered when writing and executing test cases, but the amount of information depends largely on the tester's ability, which is the ability to make testing more efficient. The more this information is used in testing, the more the testing tends to be engineering rather than guesswork.
At Microsoft, we have a Games Test Organization (GTO) that tests Xbox and PC games. When it comes to utilizing the app's information, they are the best. The game is unimaginably rich and very complicated to test. A lot of the testable stuff in the game is hidden (because having those players looking for items to trade is part of the fun of the game) o If all the testers at GTO were doing was playing the game, the problems they found wouldn't be more than end users. To make it even better, they worked with the game's developers to create dashboards that exposed what could be essentially cheating to testers. In this way, testers can know in advance where monsters will be dropped, where items will be hidden, they can see the other side of the wall, and they can control certain enemy behaviors. Their cheat tool (i.e. test harness) basically makes them gods in the game, allowing them to control the information they see for faster and smarter testing. This example is a lesson for all testers.
Information from the test means you have to pay attention to everything you do while testing and use the information you gain to influence future tests. Do you know how your tests fit into the requirements and know when a particular requirement has been adequately tested? Do you use code coverage to influence future tests? Do you know which tests will be affected when code is updated or bug fixed, or do you know to rerun all tests? Understanding how far the test has progressed and adjusting the test strategy with it is a sign of test maturity.
I used to work on a team in Microsoft's Visual Studio where we heavily used code changes (code that changed due to adding new features or fixing bugs) and code coverage to affect our tests. We put a lot of effort into informing testers of code coverage and code changes, helping them understand which test cases contribute to coverage, and helping them test changed or modified components. The end result is that when the code does change, we know exactly which tests are affected and only rerun those tests. We also know how each new test case contributes to the overall interface, feature and code coverage, guiding our testers so that everyone on the team builds on all the test cases they create and writes more meaningful tests.
What information do you use to guide your testing? How do you ensure that information is available so that it is readily available during testing? How do you make the information useful so that it affects your tests in a good way? The answers to these questions will determine how fast you will go as you descend the expert test mountain.
I like to think of it this way: testers also need to use that same flaw-finding ability to look at their work. We have to find our own testing process, the flaws in the testing process, using the same process we use to find product defects. Is there something wrong with my testing process? Are there any flaws here? Are there any barriers here that are preventing me from being productive?
You have to keep looking for a better way. Consciously identify those things that limit ability, hinder progress, and slow things down. Just as defects limit the ability of software to meet user needs, what limits the ability to test? Use the testing abilities you have to optimize your testing process, which will help you climb fast on the peaks tested and increase your chances of becoming an expert once you've climbed them.
The summit of the test peak is a wonderful place. If you managed to get there, congratulations. But this is not the final target. This means that you have become a great tester. The downhill here is to use your insight and expert knowledge to help those around you become good testers too. It's one thing to reach the summit by yourself, it's quite another to help others (those who are not as capable as you) to reach the summit.
In general, those who have successfully reached the pinnacle of testing become masters of tools. Those commercial tools, open source free tools, and self-written tools (my personal favorite) are excellent ways to increase your productivity and productivity. However, a tool is just one way to achieve that, but in many other ways it's a limitation because too many people can't see beyond what a tool can do. They are limited in what the tool can do for them, failing to see or understand that there is more need for the tool. What you really need to get to the top is "information". Because many tools process information and make it easier to obtain, testers become overly dependent on their tools. But the information itself and how to use it is the real key to success.
Mastering information means understanding what information is available and how it will affect the test, ensuring that it is used to its fullest extent. There are several types of information that test climbers must focus on. I'm talking about two of them here: information from the application and information from previous tests.
Information from an application includes requirements, architecture, code structure, source code... and even operational information about what the application does when it executes. This type of information needs to be considered when writing and executing test cases, but the amount of information depends largely on the tester's ability, which is the ability to make testing more efficient. The more this information is used in testing, the more the testing tends to be engineering rather than guesswork.
At Microsoft, we have a Games Test Organization (GTO) that tests Xbox and PC games. When it comes to utilizing the app's information, they are the best. The game is unimaginably rich and very complicated to test. A lot of the testable stuff in the game is hidden (because having those players looking for items to trade is part of the fun of the game) o If all the testers at GTO were doing was playing the game, the problems they found wouldn't be more than end users. To make it even better, they worked with the game's developers to create dashboards that exposed what could be essentially cheating to testers. In this way, testers can know in advance where monsters will be dropped, where items will be hidden, they can see the other side of the wall, and they can control certain enemy behaviors. Their cheat tool (i.e. test harness) basically makes them gods in the game, allowing them to control the information they see for faster and smarter testing. This example is a lesson for all testers.
Information from the test means you have to pay attention to everything you do while testing and use the information you gain to influence future tests. Do you know how your tests fit into the requirements and know when a particular requirement has been adequately tested? Do you use code coverage to influence future tests? Do you know which tests will be affected when code is updated or bug fixed, or do you know to rerun all tests? Understanding how far the test has progressed and adjusting the test strategy with it is a sign of test maturity.
I used to work on a team in Microsoft's Visual Studio where we heavily used code changes (code that changed due to adding new features or fixing bugs) and code coverage to affect our tests. We put a lot of effort into informing testers of code coverage and code changes, helping them understand which test cases contribute to coverage, and helping them test changed or modified components. The end result is that when the code does change, we know exactly which tests are affected and only rerun those tests. We also know how each new test case contributes to the overall interface, feature and code coverage, guiding our testers so that everyone on the team builds on all the test cases they create and writes more meaningful tests.
What information do you use to guide your testing? How do you ensure that information is available so that it is readily available during testing? How do you make the information useful so that it affects your tests in a good way? The answers to these questions will determine how fast you will go as you descend the expert test mountain.
downhill
By the time you reach the peak of the testing mountain, you have become a very capable tester, perhaps as capable as all your colleagues in your group combined. Whatever you're doing, please don't try to be better than your entire team, no matter how good you feel about it or how tight your boss is. Once you're downhill, stop fighting for honorary titles like "People Who Find Most Defects" or "People Who Find Most Meaningful Defects." Instead, I recommend that you spend less time on testing and make innovation your top priority.
Innovating on testing means not rushing forward, but looking closely, seeing opportunities, finding bottlenecks, and improving the way everyone else on the team works. Your job becomes helping others improve. At Microsoft, we have an official position dedicated to this - Test Architect. Don't get frustrated by the lack of a cool title, though. No matter what people call you, when you're "downhill, the best thing you can do is try to make sure that as many people as possible make it up the other side of the mountain successfully.
Innovating on testing means not rushing forward, but looking closely, seeing opportunities, finding bottlenecks, and improving the way everyone else on the team works. Your job becomes helping others improve. At Microsoft, we have an official position dedicated to this - Test Architect. Don't get frustrated by the lack of a cool title, though. No matter what people call you, when you're "downhill, the best thing you can do is try to make sure that as many people as possible make it up the other side of the mountain successfully.
(over)