For testing practitioners, manual testing and automated testing are two terms that accompany the testing profession throughout their lives. Today, let’s talk about their respective advantages and limitations, and how to use them together reasonably.
Definition of manual testing and automated testing
Manual Testing is a software testing method in which testers manually execute predetermined test cases to test the system to find errors and ensure software quality. Manual testing does not require writing automated scripts. Test cases are executed through manual operations and rely on the experience and skills of testers to design test cases and judge test results. Manual testing can test non-functional requirements such as interface, ease of use, etc.
Automation Testing is a method of testing software by using automated tools to execute predetermined test scripts. It can automatically perform test tasks that are highly repetitive and have relatively fixed logic, without the need for manual intervention. Automated testing requires writing test scripts (no matter how they are written) to perform tests instead of manual operations, but writing scripts requires programming, which makes the threshold for automated testing a little higher than manual testing. Automated testing has high execution efficiency, can perform tests repeatedly, has wide coverage, and can be returned multiple times to reduce duplication of work.
Advantages and limitations of manual and automated testing
Advantages of manual testing
Manual testing only requires manual operation, does not require writing test scripts, and has low requirements on testers' programming skills; manual testing can more comprehensively test the non-functional requirements of the software, such as user interface, ease of use, etc., which are often difficult to just Testing is completed through automated scripts; testers can conduct more flexible test designs and operations based on experience, and conduct exploratory tests that are difficult to cover in scripts; testers can find some boundary conditions that cannot be covered by automated tests with the intervention of subjective experience and other defects; another more important thing is that manual testing does not require research and development and purchase of tools, and the initial investment cost is low.
Limitations of manual testing
Manual testing completely relies on manual operations, and the preparation and execution of test cases requires a lot of time and cost; when the same manual test case is executed multiple times, the results may be inconsistent and the reproducibility is poor; the test scenarios covered by manual testing are relatively limited; it is required after code modification Re-carry out large-scale testing, manual testing is not suitable for frequent regression testing; test data needs to be prepared manually, and the quality of test data may vary between different testers; test result determination relies on personal experience, and the conclusions of different testers may be inconsistent and lack objectivity Standard; manual testing lacks a convenient recording mechanism, and it is difficult to quantify test progress and defects; manual testing cannot conduct long-term stability testing, and it is difficult to cover stability-related defects.
Advantages of automated testing
Automated testing writes test cases into scripts, which can be executed repeatedly and efficiently; automated testing can test all function points and code branches, covering a wide range of tests; each step of automated testing can be precisely controlled and is not affected by the external environment. The execution results are consistent; automated testing can repeat regression testing, reducing duplication of work; automated testing can perform long-term stress testing, stability testing, etc., and can reveal defects that are difficult to find in manual testing; test scripts can be reused, and new people only need to maintain them Scripts are not affected by personnel changes; automated testing can be integrated with the continuous integration (CI/CD) environment to achieve more frequent testing.
Limitations of automated testing
The maintenance cost of automated testing is high. When test cases are changed, the scripts need to be modified simultaneously, otherwise a large number of invalid tests will appear. The coupling between test scripts and the system is high, and system changes may cause a large number of automated test cases to fail. It is difficult for automated testing to cover all scenarios, such as business Process, user interaction, etc.; automated testing needs to simulate the real system environment, and changes in the test environment may affect script execution; when a test script fails, locating the cause is more complicated and debugging is more difficult; automation environment failures may cause a large number of use cases to fail, making it difficult to locate the problem More difficult.
How should manual testing and automated testing be used reasonably together?
In the functional testing stage, requirements and designs are still being optimized and changed, so manual testing should be the main method, and automated testing should be used in conjunction. After entering the system testing stage, the requirements are relatively stable and a large number of automated test scripts can be written for regression testing. UI testing, involving user interface, interaction, etc., is difficult to fully automate and manual testing should be the main method. For core business processes and workflows, automated test scripts should be written first to ensure quality. During the running process of automated testing, manual testers can conduct exploratory testing and adopt a two-pronged approach. If defects can be found, the automated scripts can be modified in a timely manner. Establish a modular and maintainable automated testing framework to reduce use case writing and maintenance costs.
Common automated testing tools
Finally, I would like to recommend some common automated testing tools to you.
function test:
Longtest AI-TestOPs cloud platform
Interface testing tools:
postman、jmeter、
Automated testing framework:
Selenium、Robot Framework、appium、UIautomater、Robotium、QTP、
Performance Testing:
LoadRunner、jmeter、GT、iTest
Mobile terminal (android):
adb、android sdk
Unit frame:
java ：junit/testng
python ：unittest/pytest