In November 2015, ThoughtWorks released a new issue of Technology Radar. Technology Radar is a unique format for documenting the results of the ThoughtWorks Technology Advisory Council discussions on technology trends that have a significant impact on the industry, delivering value to stakeholders from CIOs to developers. The technical trends of this issue of radar are mainly reflected in: the popular microservice-related technologies, the gradually mature containerized ecosystem with Docker as a typical one, and the information security issues that have attracted much attention from enterprises and users. This article analyzes the impact of these new trends on software testing.
Automated testing is king
In this era of rapid change and development, if any product wants to be competitive in the market, it must be able to quickly adapt and respond to changes, requiring rapid and continuous high-quality delivery in the product development process. To achieve rapid and continuous high-quality delivery, automated testing will be essential. At the same time, automated testing is not as simple as replacing manual testing with code or tools. It has new features and trends: there are different automated technical supports for different product development technology frameworks, and different automated testing solutions are required for different business models. As a result, automated tests have better readability, lower implementation costs, higher operating efficiency and more effective coverage. The following topics from Tech Radar illustrate these characteristics of automated testing:
Consumer-driven contract testing for microservices can help solve the problems of low efficiency and instability in integration testing as services increase. Consumer-driven contract testing is a core component of a mature microservice testing strategy. REST-assured, a tool dedicated to testing and validating RESTful services, is a Java DSL that makes it easier to write tests for HTTP-based RESTful services. REST-assured supports different types of REST requests and can validate the results returned by the request from the API. It also provides a JSON verification mechanism to verify that the returned JSON data is expected. Espresso, a functional testing tool for Android systems, whose tiny kernel API hides complex implementation details and helps us write more concise, fast, and reliable tests. ThoughtWorks' open source lightweight cross-platform test automation tool Gauge supports describing test cases in business languages, supports different programming languages, and supports parallel execution of supported platforms. Pageify is a Ruby library for building page description objects for UI automation testing. The tool focuses on faster execution of tests and code readability, and works well with Webdriver or Capybara. Quick, an open source behavior-driven development and testing framework dedicated to iOS application development, supports Swift and Objective-C, and is bundled with Nimble for testing and verification. Quick is mainly used to verify the behavior of Swift and Objective-C programs. It has the same syntax style as RSpec and Jasmine, and the basic environment is easy to set up. Quick's well-structured and type assertions make it easier to test asynchronous programs. Quick has ready-made Swift and Objective-C specification file templates, so developers can quickly test their applications in just a few steps.
Tools are important, design is essential! There are many automated testing tools, but don’t be impulsive when doing automation. You need to pay attention to the following points:
Comprehensively consider the project technology stack and personnel capabilities, and use an appropriate framework to achieve automation; combine the test pyramid and the specific situation of the project, consider the appropriate test layering, if the function points that can be covered by the underlying tests must not be placed in the upper end-to-end Test to cover; automated test case design needs to consider business value, try to design test cases from the real business process/business scenario used by users, and let automation cover the most critical user scenarios first; treat test code and development code equally, let them as an integral part of the product.
Cloud technology, containerization and open source tools drive down the cost of testing
The preparation of the test environment was a cumbersome and expensive thing in the past. Many organizations did not have the conditions to prepare multiple test environments, so that the test can only be carried out in a limited environment, which may miss some very important defects, the cost and cost of the test. very high. With the development of cloud technology, multiple test environments no longer need a large number of expensive hardware devices to support, and the container technology ecosystem exemplified by Docker is gradually growing and maturing, creating and duplicating test environments has become much simpler , the cost is greatly reduced. The Phoenix Environment, recommended by Tech Radar, uses the Phoenix Server model to support the preparation of new environments required for testing, development, UAT and disaster recovery in an automated fashion. This technology has risen from the evaluation ring in the previous issue to the adoption ring, indicating that it has been verified and recognized, and it is a technology that can be used with confidence.
On the other hand is the emergence of a large number of open source tools, which are often lightweight, simple to use, and more acceptable to people than those heavyweight and expensive testing tools. With the help of these open source tools, the test work will cover the platform, environment and data to be tested more comprehensively and realistically, which will speed up the test speed and reduce the test cost; more importantly, with these tools, the testers will be able to Being able to free up more time for more interesting things like test design and exploratory testing makes testing more fun. The open source tools mentioned in the new technology radar are: Mountebank, Postman, Browsersync, Hamms, Gor and ievms, etc.
In enterprise-level applications, it is very important to test components well, especially for the separation of services and automated deployment, which are two key factors related to the success of the microservice architecture, we need more suitable tools to test them . Mountebank is a lightweight testing tool for component testing, which can be used to simulate (Mock) and stub (Stub) HTTP, HTTPS, SMTP and TCP. Postman is a REST client plugin used in Chrome. With Postman, you can create requests and analyze the information returned by the server. This tool is useful when developing new APIs or implementing client access code for existing APIs. Postman supports both OAuth1 and OAuth2, and typeset the returned JSON and XML data. By using Postman, you can view the requests you have made through Postman before, and you can edit the test data very friendly to test the API's return under different requests. At the same time, although we discourage screen-recording testing methods, Postman provides a series of extensions that allow us to use it as a tool for running tests. As the number of devices supported by a website application increases, so does the cost of cross-device testing. Browsersync greatly reduces the cost of cross-browser testing by synchronizing manual browser testing across multiple mobile or desktop browsers. By providing a command line tool as well as a UI interface, Browsersync is very friendly to CI builds and is able to automate repetitive tasks like filling out forms. In the field of software development, the blind assumption that the network will always be reliable and that the server will always respond quickly and correctly has led to many failures. Hamms can simulate a corrupted HTTP server and trigger a series of failures, including connection failures, slow responses, or malformed responses, helping us to more gracefully test how our software responds to exceptions. Gor can capture online HTTP requests in real-time and replay these HTTP requests in the test environment to help us use this production environment data to continuously test our system. Using it can greatly improve our confidence in product deployments, configuration changes or infrastructure changes. Although the usage of IE browser is shrinking day by day, the user base of IE browser still cannot be ignored for many products, and browser compatibility still needs to be tested. This is still a hassle for people who like to develop with Unix-based operating systems. To help solve this dilemma, ievms provides useful scripts to automatically set up different Windows virtual machine images to test various versions of browsers from IE6 to Microsoft Edge.
Security testing throughout the entire life cycle
"Safety is everyone's problem"! Internet security vulnerabilities frequently break out, and security issues have become an issue that every product urgently needs to pay attention to and solve. Security testing will need to run through the entire life cycle of software development. At the same time, it brings more opportunities and challenges to software testers, requiring more security-related knowledge (including more basic computer knowledge), mastering existing security testing-related technologies, so that they can be used in software development. Perform security-related analysis and testing at each stage. Although some teams have integrated security with the entire development practice, it is still quite important to cultivate everyone's security awareness at each stage, and there is still a lot of room to explore new security testing techniques and methods.
Technologies and tools related to security testing listed on the Tech Radar are: Bug bounties, Threat Modelling, ZAP, and Sleepy Puppy.
Bug bounties is a security bug reporting reward system, and more and more organizations are encouraged to record common security-related bugs through Bug bounties to help improve software quality. Threat modeling (Thread modeling) is a set of techniques, mainly from a defense perspective, to help understand and identify potential threats. When turning user stories into "evil user stories", doing so gives teams a controlled and efficient way to make their systems more secure. ZED Attack Proxy (ZAP) is an OWASP project that allows you to detect security vulnerabilities in existing sites in an automated manner. Can be used for regular security testing, or integrated into CD's Pipeline to provide an ongoing routine security vulnerability detection. Using a tool like ZAP doesn't replace careful thought about security or other system testing, but as a tool to keep our systems more secure, it's worth adding to your toolset. Sleepy Puppy is a touch-type XSS collection framework recently open sourced by Netflix. This framework can be used to test target programs for XSS vulnerabilities when attackers try to hack into second-tier systems. XSS is the top 10 security threat of OWASP, Sleepy Puppy can be used to complete automatic security scanning for several applications at the same time. It can customize touch typing and simplifies the process of capturing, managing and tracking XSS vulnerabilities. Sleepy Puppy also provides an API for integration with vulnerability scanning tools like ZAP, enabling automated security scanning.
Optimize business value
Most software is a project model, planned, implemented and delivered in different schedules. Agile development has greatly challenged this model. Through analysis and testing at various stages of the development process, new requirements can be continuously discovered, making requirements more rational and more able to reflect business value. Lean Startup techniques, such as A/B testing to observe requirements, further erode this mindset. Technology Radar recommends "Product over project", arguing that most software development work should follow the lead of the Lean Enterprise, which defines itself as building products that support business processes. Such products do not have the so-called final delivery, but more of a process of exploring how to better support and optimize business processes, which will continue as long as the business remains valuable.
As a key role in software development, QA personnel responsible for software testing will play a pivotal role in optimizing business value by testing software from the perspective of users, combined with their own understanding of software products. Software testing is not only a process of checking whether the software meets the specified requirements or clarifying the difference between the expected results and the actual results, but also needs to consciously carry out continuous verification and optimization of requirements, and analyze business trends and risks. If we can combine the idea of BDD (Behavior Driven Development) in the development process, unify the team's understanding of requirements, and use the power of the team to optimize the business, it will achieve a multiplier effect.
Traditionally, the QA role has primarily focused on ensuring the quality of software products in a product-like environment. With the advent of continuous delivery, the QA role gradually shifted to the need to analyze the quality of software products in a production environment. QA (QA in production) in the product environment is to require the QA role to do a good job in the quality analysis of software products in the product environment under the premise of doing a good job in the quality assurance work before the product is launched. Specific practices include:
(1) Introduce monitoring of the product system, formulate testing conditions, and find out the quality metrics used in the product environment. For example, use website analysis tools to collect data on users' use of applications, analyze data volume requirements, product performance trends, users' geographical characteristics, users' behavior habits, and product market share in the same type of product.
(2) Collect feedback from end users in the product environment, and classify and analyze the feedback. Such feedback may include:
Defects: It is necessary to prioritize and determine whether they need to be repaired; and to analyze the root causes of these defects, and try to avoid the recurrence of the same type of defects in the future development process. Complaints: Analyzing the reasons behind complaints may be a good opportunity to help us improve and optimize business value. Suggestion: It may be difficult for ordinary users to make high-quality suggestions, and we need to work hard when collecting feedback and collect them in a targeted manner. Once the recommendations are collected, they will be very beneficial for business value optimization.
By analyzing the software quality in the product environment, it will help to assist in the practice of "product is better than project", help optimize business value, and do a good job in enterprise product innovation. It is important to note that QA in a production environment may lead some organizations to go too far and ignore pre-launch quality assurance, it is only valuable to those organizations that have implemented and have some degree of continuous delivery practices.
Summarizing software testing is a technical job, but the problems in the field of software testing are not just technical. With the increasing degree of automation, some people continue to doubt the necessity of QA. From the previous analysis, we can see that new trends have put forward higher requirements for QA, bringing more opportunities and challenges. QA cannot simply be replaced.