Several common automated testing frameworks
In the field of software testing, there are many automated testing frameworks. Here are some commonly used automated testing frameworks.
1.pytest
pytest is a unit testing framework for Python. It is similar to the unittest testing framework that comes with Python, but it is simpler and more efficient to use than the unittest framework. It mainly has the following characteristics:
- Simple and flexible, easy to use.
- Parameterization is supported.
- It can support simple unit tests and complex functional tests, and can also be used for automated testing such as selenium / appnium and interface automated testing (pytest+requests).
- pytest is not a python built-in library and needs to be installed separately.
When using pytest to write test cases, the following rules must be followed:
- The test file name must
test_start or_testend with (eg:test_ab.py) - Test methods must
teststart with . - Test class names
Teststart .
2.Junit
Junit is a unit testing framework for the Java language. It was founded by Kent Beck and Erich Gamma and has grown to be the most successful of the xUnit family, which originated from Kent Beck's sUnit. Junit has its own ecosystem of JUnit extensions. Most Java development environments have integrated Junit as a tool for unit testing.
Junit is a regression testing framework written by Erich Gamma and Kent Beck. Junit testing is programmer testing, so-called white box testing, because programmers know how (How) and what (What) function the software under test completes. Junit is a set of framework, inheriting the TestCase class, you can use Junit for automatic testing.
3.selenium
Selenium is a tool for web application testing. Supported browsers include IE, Mozilla Firefox, Mozilla Suite, etc. The main functions of this tool include: Test compatibility with browsers-test your application to see if it can work well on different browsers and operating systems. Selenium is a complete web application testing system, including test recording (selenium IDE), writing and running (Selenium Remote Control) and parallel processing of tests (Selenium Grid). The core of Selenium, Selenium Core, is based on JsUnit and is written entirely in JavaScript, so it can be used on any browser that supports JavaScript. Selenium can simulate real browsers, automated testing tools, and support multiple browsers. It is mainly used to solve JavaScript rendering problems in crawlers.
Selenium 1.0 consists of the following two parts: selenium server and Client Libraries.
Selenium 2 encapsulates the browser's native API into WebDriver API, which can directly operate the elements in the browser page, and even operate the browser itself (screen capture, window size, start, close, install plug-ins, configure certificates, etc.), so just Just like a real user is operating.
4.Appium
Appium is one of the mainstream automated testing tools on the current mobile platform.
Appium is a compound word made up of the first three letters of "application" and the last three letters of "Selenium". application is "application", and we generally refer to the application on the mobile platform as App for short. Selenium is the current mainstream Web UI automation testing tool. There is an inheritance relationship between appium and Selenium. The meaning of appium is: Selenium automated testing tool on the mobile side .
appium is an open source automated testing tool that supports native applications, web applications, and hybrid applications on iOS and Android platforms.
- Native application : Refers to those applications (App) developed with iOS or Android SDK.
- Web App : Refers to an app that can be accessed using a mobile browser such as Safari on iOS and Chrome on Android.
- Hybrid application : refers to an application that encapsulates webpage views with native code, and interacts with native code and web content. For example, WeChat applets can help developers use web technology to develop applications, and then package them with native code. More importantly, appium is a cross-platform testing tool that allows testers to write automated test scripts using the same set of APIs on different platforms (iOS, Android), which greatly increases the code complexity between iOS and Android test suites. usability.
5.LoadRunner
LoadRunner, is a load testing tool that predicts system behavior and performance. By simulating tens of millions of users implementing concurrent loads and real-time performance monitoring to identify and troubleshoot issues, LoadRunner enables testing of entire enterprise architectures. Enterprises use LoadRunner to minimize testing time, optimize performance and accelerate the release cycle of application systems.
5.1 Virtual user
Using LoadRunner's Virtual User Generator, you can easily create system loads. The engine can generate virtual users and simulate the business operation behavior of real users in the way of virtual users. It first records the business process (such as an order or flight reservation), and then converts it into a test script. With virtual users, you can simultaneously generate thousands of user visits on Windows, UNIX or Linux machines. So LoadRunner can greatly reduce the hardware and human resources required for load testing.
After creating a test script with the Virtual User Generator, you can parameterize it, which allows you to test your application with several different sets of actual data that reflect the system's load capacity. Taking an order entry process as an example, parameterization can replace fixed data in a record, such as order number and customer name, with variable values. Enter possible order numbers and customer names in these variables at will to match the operation behavior of multiple actual users.
5.2 Real load
After the Virtual users are established, you need to set your load scheme, business process composition and the number of virtual users . With LoadRunner's Controller, you can quickly organize multi-user testing scenarios. The Controller's Rendezvous feature provides an interactive environment in which you can both establish continuous and cyclic loads, and manage and drive load test scenarios.
Furthermore, you can use its scheduling service to define when users access the system to generate load. In this way, you can automate the testing process. You can also use the Controller to limit your load scenarios in which all users simultaneously perform an action, such as logging into an inventory application, to simulate peak . In addition, you can also monitor the performance of various components in the system architecture, including servers, databases, network equipment, etc., to help customers determine the configuration of the system.
5.3 Positioning performance
LoadRunner includes an integrated real-time monitor so that you can observe the operating performance of the application system at any point during the load testing process. These performance monitors show you real-time transaction performance data (such as response time) and real-time performance of other system components including application server, web server, network equipment and database. This allows you to assess the performance of these system components from both the client and server sides during testing, allowing you to identify problems more quickly.
With LoadRunner's ContentCheck TM , you can determine whether your application is functioning properly under load. ContentCheck detects the network packet content of the application when Virtual users are running, and determines whether there is any wrong content sent out. Its real-time browser helps you observe program performance from an end-user perspective.
5.4 Analysis results
Once testing is complete, LoadRunner collects and aggregates all test data and provides advanced analysis and reporting tools to quickly find and trace performance issues. Using LoadRunner's Web Transaction Detail Monitor, you can see how long it takes to download all images, frames, and text to each web page.
For example, this transaction detail analysis mechanism can analyze whether the running speed of the application system is slowed down due to a large-size graphics file or a third-party data component. Additionally, the Web Transaction Detail Monitor breaks down end-to-end response times for client, network, and server, making it easy to identify problems and locate the components that are really at fault. For example, you can break down network latency to determine DNS resolution time, the time it takes to connect to a server, or SSL authentication. By using LoadRunner's analysis tools, you can quickly find out where and why something went wrong and make adjustments accordingly.
5.5 Repeat the test
Load testing is an iterative process. Every time you handle an error condition, you need to load test your application again under the same scenario. Use this to check whether your fixes improve performance.
LoadRunner fully supports load testing of EJBs. These Java-based components run on application servers and provide a wide range of application services. By testing these components, you can identify and resolve potential issues early in application development.
Using LoadRunner, you can easily understand the performance of the system. Its Controller allows you to repeat the same test scenario as it was before the error modification. Its HTML-based reports give you the baseline you need to compare performance results, measure improvement over time, and ensure application success. Since these reports are HTML-based text, you can post them on your company intranet for easy access at any time.
6.Jmeter
Apache JMeter is a Java-based stress testing tool developed by the Apache organization for stress testing software.
JMeter was originally designed for web application testing, but has since expanded into other testing areas and can be used to test static and dynamic resources such as static files, Java servlets, CGI scripts, Java objects, databases, and FTP servers to name a few. JMeter can simulate , test their strength and analyze overall performance under different stress categories. Additionally, JMeter enables functional/regression testing of applications by creating scripts with assertions to verify that the program is returning expected results. For maximum flexibility, JMeter allows the use of regular expressions to create assertions.
The role of JMeter
- Capable of stress and performance testing HTTP and FTP servers, as well as any database (via JDBC).
- Complete portability and 100% pure java.
- Full Swing and lightweight component support (precompiled JARs using javax.swing.*) packages.
- The fully multi-threaded framework allows concurrent sampling by multiple threads and simultaneous sampling of different functions by separate thread groups.
- Careful GUI design allows quick operation and more precise timing.
High Scalability of JMeter
- Linkable samplers allow for unlimited testing capabilities.
- Various load statistics tables and linkable timers are available.
- Data analysis and visualization plugins provide great scalability and personalization.
- Has functionality to provide dynamic input to tests (including Javascript).
- A sampler that supports scripting (BeanShell is supported in 1.9.2 and above).
- Caching and offline analysis/playback of test results.
References
[1] Summary of automated testing framework
[2] Baidu Encyclopedia, LoadRunner
[3] Jmeter Chinese website