10 years of testing confusion and pain
In the blink of an eye, I have been working for 10 years, which means I have been doing software testing for 10 years. I have experienced 3 companies, and I have some insights. It is hard to believe that I can persist in Taobao for so long. After 7 years, people say that seven Every year, I do have a little itching, but it's not that big. No matter what, I will make some changes. I will leave Taobao BU in early July. This testing team that I have worked hard for 7 years, every year The BU, the team, and the tests are all changing. I insist on doing tests in the Taobao BU, and I will implement my ideas and thoughts little by little, watching the ups and downs and changes of the Taobao business; later I will join the merchant BU and follow the first The boss Qi Ge does what he wants to do and explores some unknowns, including business and technology and the test in his heart.
Looking back on the past ten years of testing, I have taken a lot of detours and a lot of classic paths. The first five years were mainly in the improvement of testing technology and development technology. At that time, I read a lot of books and read a lot of foreign countries about testing. I am very happy with the paper. After the team led the team for the next five years, the investment in testing technology and development has been greatly reduced. I am often confused and need to deal with some trivial matters, and I cannot find a better balance between management and technology. , so the improvement of technology in recent years is quite unsatisfactory and unhappy; but the world is like this, there will be gains and losses, and there will be gains. This is the classic old way, and most people who test in China will take This road, but many times, looking at the development of testing technology (especially mobile testing technology, big data testing technology, cloud computing testing technology), including the development of foreign technology, I can't help but feel that I really can't keep up with this change. In this era, a change must be made. What is the change?
-
Change business: Expose yourself to different businesses, different products, if you are lucky, there will be different development and testing technologies waiting, so that you can enrich yourself for a while, learn more, and let your testing experience More plump, after 1-2 years like this, it will return to the original point, the road is still so narrow, and I am old.
-
Change testing: Doing business testing is indeed a very annoying thing, a lot of miscellaneous things are troublesome, demand changes, demand rationality, user experience, test data, bug management, etc. Occasionally, there will be some automated tests, code viewing, analysis and development Designed to be implemented as an appetizer, everything else is painful and tasteless. Then I don't want to do business testing, and I don't want to bring a business testing team. What should we do? The future of testing is obvious. We must reduce testing while ensuring basic quality and improving efficiency. Will there be fewer and fewer people to lead such an elite testing team? Do the basic test service-oriented system and platform (supporting the efficient execution of those indispensable test activities)? To what extent can it be achieved and what results can be obtained?
-
Change job: I actually saw the future of testing, including career development, including the direction and the current situation in China. Can I consider quitting testing and leaving the small circle of testing, including the circle of testing platform development; in Ali, I I know that some people with high test P have successfully left the testing circle and entered the development circle, which is more developed and promising, and the future will definitely be better. There is no doubt about this; I have also thought about it dozens of times to convince myself that I must leave the testing circle. , long pain is worse than short pain, throw away those things that I think I am good at, and defeat myself again and again by inferiority, laziness and hesitation, sometimes it is difficult to believe that I am such a lack of courage. Of course, you don’t have to enter the development circle. PD is not bad, but it’s not good. It’s also very good to be HR. It’s also good to be Staffing. Anyway, don’t do testing. Do you test? Development, PD, and operations are the core team. If you go on like this, you will not be able to start a business, that is, you will not be able to become a core member of a startup company. In this life, you can only be a test manager, testing Director? Are these interesting, do you really need to know advanced technology? That will be replaced in minutes, you are so old, you have not pursued it yet, wait until the year of the monkey and the horse month; you have a family, you can't be too troublesome, just be calm; Nima, okay, I can't stand myself anymore, I Change your job, change your life, or change both? Well, let's not say, I am more painful.
Oops, I'm talking about the idol in the domestic testing session, Mr. Duan, who was the first Google testing manager in the testing field. A few years ago (2010?), he quickly jumped out of the testing circle to become Douban's technical VP of course, and then To the VP, CTO of each company, and now the CEO, let’s see, this life is full of challenges and uncertainties, the road is getting wider and wider, of course, the industry is also more and more trendy, from the Internet, to Internet games, to Internet finance; once again, I worship President Duan. After all, there is only one President Duan in China, but there are thousands of test managers and test directors.
Well, once again admit that I have been really confused in the past two years, the goals and directions are not clear, and the output and the things I get are less. If this continues, it will be hopeless, but what am I still doing in the testing field, What else to learn, what is really useful, what is really useful to the company, useful to the department, long-term or short-term, well, confused again.
10 Years of Test Reflections
Of course, I have been paying attention to the breakthroughs and innovations in Alibaba's testing related technologies. There are indeed many breakthrough results related to automated testing, online monitoring, testing services, and online and offline data comparison. Out, are more or less changed the original working mode. But what we can never forget is one of the tenets of the context-driven school, that there are no best practices at all, only good practices exist in a certain context.
Have you ever thought that the core output of development is code, which can be stored in the code repository forever, and the authority and importance of this repository are self-evident. What is the core output of testing? test code? Bug? Test scenario design? Of course, if our automated tests are doing very well, then our testcode can also have great value, but what if it doesn't? Our corresponding test scenario descriptions and bugs have not received enough attention and value. We all know that the understanding and sorting of business logic in testing should be on top of development. We often ignore the importance of the value of the integrated output of business precipitation and technology precipitation, and at the same time ignore the sustainability and iteration of this output. Performance, traceability (code actually has such characteristics).
So we can imagine, whether we can manage and map the system service capability map, test scenarios, and corresponding code on git in a unified way, the sustainable version control of TC, the process mechanism of review, the main TC and the branch TC management of TC, componentization of TC and external provision of services. We manage and share test design scenarios as an asset. Note that there are many benefits of this dynamic management of test scenarios, which will not be described here. Of course, we can also include the relationship between test data services and truly open up all aspects of the system (product logic, development code, test scenarios, machines, lines, etc.). environment, data), and can clearly know and how to improve the system bit by bit, this is a big picture, I have always wanted to see this, well, I may be in YY.
Then continue to YY, how to improve the quality of developers, if we want to achieve a 10:1 ratio of development to testing, there is a big difference between what testers do and what developers do in the current project process Yes, I myself have seen the single test code and methods written by Google's developers, which are quite good. A common sort can write more than 20 unit tests testcese, which is more comprehensive than many testers. The test sense is there, system unit tests, interface tests, and even integrated automated tests are likely to be completed by development, which is quite good. Domestic companies do not have time to develop and code, let alone write unit tests. It's time to code, is it a matter of time or a matter of quality awareness and responsibility of developers? In addition, I also found a phenomenon that everyone is very BSUI automated testing, and they all think that the maintenance cost of UI automated testing is high, and they all do interface testing. Are we going to extremes (it is difficult to see UI automation scripts continue It is integrated, it is estimated that everyone does not write UI automation scripts), the role of UI automation testing is not only in the test environment, but also the function monitoring of online links can play a key role, and we basically abandoned it. In this role, we can't find the balance point of layered automated testing, so we just choose to give up, and then think about it, if the back-end development can do unit testing and interface testing, then can our front-end development also do UI automation testing? Well, they are also better at writing automated test scripts than testing. Can we try it?
In fact, no matter how the test develops, how the test efficiency is improved, and who controls the quality of the system, these are eternal topics, who does it or not. What we have to do is to constantly change, change ourselves, change development, and change the entire R&D process. If the testers are still guarding their own three-acre field, the progress and gains are quite small, and the test needs to do a lot of things that need to be done in development, such as non-functional specialties, performance, stability, security, not necessarily It is a special test, and the entire solution must have relatively good results. Note that this is not the embodiment of business development capabilities.
Anyway, when you've been testing for 10 years, I don't think it's possible to think about bigger problems, because these problems are unavoidable and inescapable. Of course, this is not to say that we cannot do front-line testing work, but if our focus is to constantly test execution, file bugs, and write automated test scripts, this is not desirable; we still need to understand the pain of front-line testers , depression, discomfort, the place of the problem; and these abstract and enlarged problems are what we need to solve. The problem that test architects need to solve is to quickly release, improve test efficiency, and improve the ratio of development to test under the premise of balancing quality (no P1P2-level online failures). What are we going to do, what specific actions do we do, and do the goals and plans of what we do revolve around this core?
Well, in fact, I have always had some messy ideas of my own, and I have learned about the practices of many test teams and many innovative tools and processes related to testing, but what I have never thought about before is what kind of testing is in my mind. , What have I done for this kind of test, I have not done a good abstraction, summed up and planned the test in my mind. Whether this test is beautiful or realistic; whether this test is impossible to implement or can be done, I have not thought about the whole of this test very well. In the past 10 years, I have experienced a lot of points, but I have not strung it into a line, I have not connected it into a face.
Test in my heart
Well, I was too long-winded before, and now I'm back to the topic. I really thought a lot in the past two months, including reading some blogs I wrote before, selecting some good ideas from them, and adding some changes I saw. . Finally, I thought about a Liudao.com quality prevention and control system. There will definitely be some problems. Some subsequent ideas and thoughts will be built around this system, so as to improve it, truly do it well, and truly serve the entire project team. member.
why - Liudaowang Quality Control System
First of all, why there is such a Liudaowang quality control system, because we want to:
-
Comprehensive and multi-angle testing activities to ensure system quality
-
Change the way development and testing work in traditional mode
-
Improve the test maturity of the R&D team and help the organization upgrade
-
Under the condition of controllable quality, improve the development and test ratio
-
More explicit display of system quality control process and results
Big picture of Liudaowang quality control system
Liudaowang quality prevention and control system implementation strategy (removed how to do, tools, results)
Of course, each network has its own independent implementation process and strategy. I have also thought about it, but it is not convenient to take it out now, so I will not discuss it here. I will describe it in detail later when I have the opportunity. Here It should be noted that different product features will correspond to different implementation strategies, pure back-end, standard web, H5, native, SDK, ISV, big data, algorithm, etc. have their own suitable To implement strategies, what we want is to configure different projects and automate the configuration of different implementation strategies, including implementation progress, status, risk control, and so on.
Liudaowang quality control system mechanism & standard issues:
-
What is the project access standard of Liudaowang system?
-
How to know the progress of a certain network prevention and control?
-
How to know the effect and problems of a certain network prevention and control?
-
How to ensure the implementation of the specific actions of Liudao.com?
-
Could the quality risk of Liudao Network’s prevention and control not be exposed in advance?
-
Liudao network cannot make decisions based on the prevention and control results of each network?
-
How does Liudao.com obtain internal and external data in the prevention and control process?
I have not thought through many of the above questions, and what I have been thinking about is what is the core competitiveness of the Liudao network system? What is the service capability provided externally? What exactly does the system and functional architecture look like? I have to admit that I have been doing testing for 10 years. I usually don’t do much about system design, architecture design, and business abstraction design. I really can’t do platform design and architecture abstraction. For development, I cannot systematically think about what a platform looks like, nor can I accurately design the configurability, scalability, stability, and diversity of a platform.
Everyone has seen military-themed movies or TV dramas. We often see the content displayed on the big screen of the headquarters during military exercises or battles. The TMD ones are so handsome. I just want that kind of system, that kind of system. The Liudao Network prevention and control system, which provides information support for all members of the headquarters to make decisions. But I have to admit that Alibaba does not have such a large screen at present. Without such a large screen, I am very clear about the implementation information and control decisions that a project and a system should have, not only the issue of automation, but also the project we do. The problem of standard input and standard output, we have been making Internet products, we have always wanted to be fast, we have always wanted to do whatever we want, and we have always wanted our own convenience.
No matter how well the platform and idea are planned, they also need the ability to implement the implementation. For platforms and products, some people talk about top-down support and encouragement, some people talk about the problems brought by the platform, and some people talk about it. It is said that both are indispensable; everyone is afraid of change, and change means bringing new inevitable costs, unfamiliar working methods, and possibly an increase in their workload. Our goal is certain, we don't want to increase and guarantee the quality of the system by adding testers, we just want to reduce the number of testers and explore the future of testing that will definitely come.
In any case, I mentioned it at the earliest. Now that I leave the original Taobao test team and go to the merchant BU alone, I will face many problems, but I found the Liudaowang system, and I will give my long time for it. Thinking and summarizing, including learning (I often look at these architecture designs, technical thinking, etc. recently), I believe I can find a suitable test in my heart. In the next ten years of testing, the road will definitely be uneven, and there will definitely be many challenges.