I am Huang Caicai, and I am inspiring to become the big belly boy who will be the most tested in boxing session. I chatted with the group friends of Hapi for a day and found that many group friends did not know where to do automated software testing, so I roughly wrote this article. ,I hope to be helpful.
If you want to discuss with my hapi group friends how to learn python automated software testing, you can click here ✔←
Since being selected as a programming language in 2018, Python has always been among the best in the major rankings. Currently, it ranks third in the Tiobe Index, second only to Java and C. With the widespread use of this programming language, automated testing frameworks based on Python have also emerged, and continue to develop and enrich.
Therefore, developers and testers need to consider many factors when choosing a test framework for the project at hand, including: the script quality of the framework, the simplicity of the test cases, and the possible technical weaknesses of the running modules. In order to avoid the "difficult choice syndrome", I have prepared five Python-type automated testing frameworks for you to compare and discuss.
- 1.Robot Framework
As one of the most important Python testing frameworks, Robot Framework is mainly used in test-driven development and acceptance. Although it was developed by Python, it can also run on .Net-based IronPython and Java-based Jython. At the same time, as a Python framework, Robot is also compatible with platforms such as Windows, MacOS, and Linux.
- Use prerequisites
Before using Robot Framework (RF), you need to install Python 2.7.14 and above. I recommend that you use Python 3.8 to ensure that appropriate comments can be added to the code segment and to be able to track program changes. At the same time, you also need to install the Python package manager-pip.
Of course, you must also download the corresponding development framework, for example: PyCharm Community Edition, which is popular among developers. In addition, since the code segment itself does not depend on any IDE, you can follow the IDE tools you have at hand.
- advantage
By using the keyword-driven-test method, it can help testers easily create readable test cases, thus simplifying the entire automated process.
Users can easily test the grammar in the data.
Because it is composed of various common tools and test libraries, it not only has a huge ecosystem, but also can use its various elements in a single project.
Since there are many types of APIs, the framework is highly extensible.
Although not a built-in function, Robot can use Selenium Grid to help users run various parallel tests.
- Disadvantage
Despite the above-mentioned conveniences, Robot Framework is more cumbersome in creating custom HTML reports. At most you can use it to generate short reports in xUnit format.
In addition, Robot Framework is not good at parallel testing.
The characteristics
of Robot compared with competing products. Because Robot has a rich built-in library and can use a simpler test-oriented DSL (Domain Specific Language), if you are a novice in the field of automated testing and lack development experience, you will As a Python testing framework, it is easier to use than the Pytest or Pyunit mentioned below. Of course, if you need to develop a complex automation framework, then please use Pytest or other types of Python frameworks.
- 2.Pytest
Pytest, which is suitable for a variety of software testing, is another Python-type automated testing framework. With its open source and easy-to-learn characteristics, this tool is often used by QA (Quality Analysis) teams, development teams, individual teams, and various open source projects. Given that Pytest has practical functions such as "assert rewriting", many large Internet applications, such as Dropbox and Mozilla, have switched from the unittest (Pyunit) mentioned below to Pytest.
Prerequisites for use
In addition to basic Python knowledge, users do not need more technical reserves. In addition, users only need to have a test device with a command-line interface, and install the Python package manager and IDE tools that can be used for development.
- advantage
In the past, developers had to include their tests in various large classes. Today, Pytest allows users to write more compact test suites.
Other testing tools require developers or testers to use a debugger, or use log checking to detect the source of a particular value. When users use Pytest to write test cases, they can store all the values in the test case until they learn which values have failed and which values have been asserted.
Since there is not much boilerplate code involved, users can easily write and understand various tests.
The Fixture function is often used to add a parameter to the test function and return a different value. In Pytest, you can modularize one fixture by using another. At the same time, users can use multiple fixtures to cover all parameter combinations without rewriting test cases.
Pytest developers maintain the scalability of the framework by publishing various useful plug-ins. For example: pytest-xdist can be used to perform parallel testing without using other testers. At the same time, unit tests can also be parameterized without copying any code.
By providing various special routines for developers, it can make the writing of test cases easier and less error-prone, and the code will become shorter and easier to understand.
- Disadvantage
The special routines mentioned above also mean that users must give up certain compatibility. Although it is convenient for users to write test cases, these use cases cannot be used with any other testing framework.
Comparison of Pytest's Features and Competitive Products In
terms of writing functional test cases and developing complex frameworks, Pytest outperforms UnitTest, which will be mentioned below. However, it is similar to Robot Framework and is only suitable for developing simple test frameworks.
If you are considering using Pytest, please refer to "Using Pytest and Selenium WebDriver to Realize Test Automation".
- 3.UnitTest/PyUnit
UnitTest/PyUnit, inspired by JUnit, is also a standardized Python automated testing framework for unit testing. Its base class TestCase provides various assertion methods, as well as all routines for cleaning and setting. Therefore, each method in the TestCase subclass is prefixed with "test" to identify that they can be run as a test case. Users can use the load method and TestSuite class to group and load various tests.
Of course, you can also build a custom test runner through joint use. Just as we use Junit to test Selenium, UnitTest also uses UnitTest-sml-reporting and can generate various XML reports.
Use prerequisites
Since UnitTest uses Python by default, we don't need any prerequisites. In addition to the basic knowledge of Python framework, you can also install pip and IDE tools for development.
- advantage
Developers do not need to install any other modules.
UnitTest is a derivative product of xUnit, and its working principle is very similar to other xUnit frameworks. Therefore, for those who do not have a solid Python background, they can get started quickly.
Users can run a single test case in a simpler way. You only need to reserve a name on the terminal, and the framework can flexibly execute tests for various use cases and produce refined output.
It can generate various test reports within a few milliseconds.
- Disadvantage
Although the framework often uses snake_case to name various Python codes, since it is derived from Junit, it still retains some traditional camelCase naming methods. This is often confusing.
Because it supports too much abstract methods, the purpose of the test code is sometimes not clear enough. A lot of boilerplate code is required.
The characteristics of PyUnit and the comparison of competing products
are similar to the views of some Python developers. I think: Pytest can train testers to write better automation code in a very compact way, this programming habit. Although UnitTest is Python's default automated testing framework, its working principle and naming rules are still slightly different from standard Python code. At the same time, it also requires too much boilerplate code. Therefore, the framework is not very popular.
- 4.Behave
We all know: Behavior-driven development (BDD) is a method based on agile software development. It can encourage collaboration among developers, business participants, and QA personnel. As another Python testing framework, Behave allows teams to avoid complex situations and perform BDD testing. In essence, the framework is very similar to SpecFlow and Cucumber and is often used to perform automated tests. Users can write test cases in a simple and easy-to-read language and can paste them into the code during their execution. Moreover, the established behavioral norms and procedures can also be reused in other test programs.
Prerequisites for Use
Anyone with basic knowledge of Python can use Behave. Other prerequisites include:
You must first install Python 2.7.14 and above.
You need to use the Python package manager or pip to collaborate with Behave.
Most developers will choose Pycharm as the development environment, of course you can also choose other IDE tools.
advantage
Because system behavior uses semi-formal language and domain vocabulary, Behave helps maintain consistent behavior in the organization.
Let those development teams working on different modules have similar characteristics through coordinated processing.
Each functional block constructed by it can execute various test cases.
Since there are more details worthy of user reasoning and thinking, the target product will have better norms.
According to a similar specification format, it enables project managers and stakeholders to have a clearer understanding of the output of the development team and QA team.
- Disadvantage
Only applicable to black box testing.
The characteristics of Behave compared with competing products
Behave is most suitable for black box testing such as simple Web testing. However, Behave is not a good choice for integration testing and unit testing with complex scenarios. As an alternative to Behave, some developers and testers will recommend using pytest-bdd. This tool combines all the advantages of Pytest and implements various behavior-driven testing scenarios.
If you are considering using Behave, then you can borrow "Behave with Selenium" (see https://www.lambdatest.com/support/docs/display/TD/Behave+with+Selenium±+Running+Behave+Automation+ Scripts+on+LambdaTest+Selenium+Grid) mentioned in the automation script.
5.Lettuce
Lettuce is another behavior-driven automation tool based on Cucumber and Python. Lettuce focuses on common tasks with behavior-driven development characteristics. It is not only easy to use, but also can make the whole testing process smoother and even more interesting.
Prerequisites for use
You need to install Python 2.7.14 and above with IDE. Of course, you can also use Pycharm or any other IDE tool. At the same time, you also need to install the Python package manager.
- advantage
Similar to other BDD testing frameworks, Lettuce enables developers to create multiple scenarios and use simple natural language to describe different functions.
According to a similar specification format, development and QA teams can coordinate and work together.
For black box testing, Lettuce is very suitable for behavior-driven test cases.
- Disadvantage
In order to successfully implement behavior-driven testing, continuous communication between the development team, QA personnel, and stakeholders is required. The lack of such communication can create ambiguity in the process and cause problems for the team. Comparing Competitive Products
In the eyes of ordinary developers and automation testers, Cucumber is more practical in performing BDD testing. From the perspective of Python developers and QA personnel, Pytest-bdd is. The power of Pytest is that its framework implements compact and easy-to-understand code and can be closely integrated into behavior-driven testing.
Conclusion
In summary, among the above five automated testing Python frameworks, Pytest, Robot Framework, and UnitTest can be mainly used for functional and unit testing, while Lettuce and Behave are only suitable for behavior-driven testing. Through further comparison, we believe that Pytest is true for functional testing. If you are new to automated testing based on Python, Robot Framework is the entry tool. Although its functions are limited, it is very easy to use. For Python-based BDD testing, Lettuce and Behave are equally good. However, if you already have a certain amount of Pytest experience, please use Pytest-bdd. I hope this article can help you select a suitable Python testing framework and carry out the testing smoothly.
Follow, don’t get lost, if it helps, remember to like and support!