Since it was selected as a programming language in 2018, Python has always been at the top of the major rankings.
It currently ranks third in the Tiobe Index, behind Java and C. With the widespread use of this programming language, a Python-based automated testing framework has also emerged, and continues to develop and enrich.
Therefore, developers and testers need to consider many factors when choosing a test framework for the project at hand, including: the script quality of the framework, the simplicity of test cases, and the possible technical weaknesses of running modules. In order to avoid "choice difficulty", I have prepared five types of Python-type automated testing frameworks for comparison and discussion.
Robot Framework
As one of the most important Python testing frameworks, Robot Framework is mainly used in test-driven development and acceptance. Although developed in Python, it also runs on .Net-based IronPython and Java-based Jython. At the same time, as a Python framework, Robot is also compatible with platforms such as Windows, MacOS, and Linux.
Use prerequisites
Before using Robot Framework(RF), you need to install Python 2.7.14 and above. I recommend you use Python 3.6.4 to ensure proper comments are added to code segments and to track program changes. At the same time, you also need to install the Python package manager --pip.
Of course, you must also download the corresponding development framework, for example: PyCharm Community Edition, which is more popular in the developer circle. In addition, since the code snippet itself does not depend on any IDE, you can completely use the existing IDE tools at hand.
advantage
- By using the keyword-driven-test method, it can help testers easily create readable test cases, thus simplifying the entire automation process.
- Users can easily test the syntax in the data.
- Because it is composed of various common tools and testing libraries, it not only has a large ecosystem, but also its individual elements can be used in separate projects.
- The framework is highly extensible due to having many types of APIs.
- Although not a built-in feature, Robot can live with an online-selenium-grid-for-automated-cross- browser- testing/ ), to help users run various parallel tests.
shortcoming
Despite all the above conveniences, Robot Framework is cumbersome in creating custom HTML reports. At most you can use it to generate short reports in xUnit format.
In addition, Robot Framework is not good at parallel testing.
Robot's features compared with competing products
Since Robot has a rich built-in library and can use a simpler test-oriented DSL (Domain Specific Language), if you are new to the field of automated testing and lack development experience, it is better to use it as a Python testing framework than Pytest or Pyunit, which will be mentioned below, is easier to get started. Of course, if you need to develop a complex automation framework, please use Pytest, or other types of Python frameworks.
现在我也找了很多测试的朋友，做了一个分享技术的交流群，共享了很多我们收集的技术文档和视频教程。
如果你不想再体验自学时找不到资源，没人解答问题，坚持几天便放弃的感受
可以加入我们一起交流。而且还有很多在自动化，性能，安全，测试开发等等方面有一定建树的技术大牛
分享他们的经验，还会分享很多直播讲座和技术沙龙
可以免费学习！划重点！开源的！！！
qq群号：110685036
Pytest
Pytest, which is suitable for a variety of software testing, is another Python-type automated testing framework. With its open source and easy-to-learn features, this tool is often used by QA (quality analysis) teams, development teams, personal teams, and various open source projects. Given that Pytest has practical functions such as "assert rewriting (assert rewriting)", many large Internet applications, such as Dropbox and Mozilla, have switched from the unittest (Pyunit) mentioned below to Pytest.
Use prerequisites
Apart from basic Python knowledge, users do not need more technical reserves. In addition, users only need to have a test device with a command line interface, and have installed the Python package manager and IDE tools for development.
advantage
- In the past, developers had to wrap their tests in various large classes. Today, Pytest allows users to write more compact test suites.
- Other testing tools require the developer or tester to use a debugger, or log inspection to detect the source of a particular value. When users use Pytest to write test cases, they can store all the values in the test case until they learn which values failed and which values were asserted.
- Since there is not much boilerplate code involved, users can easily write and understand various tests.
- Test fixture (Fixture) function is often used to add a parameter to the test function, and return a different value. In pytest, you can modularize one fixture by using another. At the same time, users can use multiple fixtures to cover all combinations of parameters without rewriting test cases.
- The pytest developers keep the framework extensible by releasing various utility plugins. For example: pytest-xdist can be used to execute parallel tests without using other testers. At the same time, unit tests can also be parameterized without duplicating any code.
- By providing developers with various special routines, it can make the writing of test cases easier and less error-prone, and the code will become shorter and easier to understand.
shortcoming
The special routines mentioned above also mean that users must give up certain compatibility. Although it is convenient for users to write test cases, these use cases cannot be used with any other testing framework.
The characteristics of pytest are compared with competing products
In terms of writing functional test cases and developing complex frameworks, Pytest outperforms UnitTest, which will be mentioned below. However, it is similar to Robot Framework and is only suitable for developing simple test frameworks.
If you are considering using Pytest, please refer further to "Test Automation with Pytest and Selenium WebDriver".
UnitTest/PyUnit
UnitTest/PyUnit, inspired by JUnit, is also a standardized Python-like automated testing framework for unit testing. Its base class TestCase provides various assertion methods, and all cleanup and setup routines. Therefore, every method in the TestCase subclass is prefixed with "test" to indicate that they can be run as test cases. Users can use the load method and the TestSuite class to group and load various tests.
Of course, you can also build a custom test runner by combining them. Just as we use Junit to test Selenium, UnitTest will also use UnitTest-sml-reporting and can generate reports of various XML types.
Use prerequisites
Since UnitTest uses Python by default, we don't need any prerequisites. In addition to the basic knowledge of the Python framework, you can also additionally install pip and IDE tools for development.
advantage
- Developers do not need to install any other modules.
- UnitTest is a derivative of xUnit, and its working principle is very similar to other xUnit frameworks. So for those who don't have a strong Python background, they can get started quickly.
- Users can run individual test cases in a much simpler manner. You only need to pre-order the name on the terminal, and the framework flexibly executes tests for various use cases and produces refined output.
- It is able to generate various test reports within milliseconds.
shortcoming
Although the framework commonly uses snake_case to name various Python codes, since it is derived from Junit, it still retains some traditional camelCase naming methods. This tends to be confusing.
Because it supports too many abstract methods, the purpose of the test code is sometimes unclear. Requires a lot of boilerplate code.
The characteristics of PyUnit compared with competing products
Similar to the views of some Python developers, I think: Pytest can cultivate the programming habit of testers to write better automation code in a very compact way. Although UnitTest is the default automated testing framework for Python, its working principle and naming rules are still slightly different from standard Python code. At the same time, it also requires too much boilerplate code. So the framework is not very popular.
Behave
We all know: behavior-driven development (behavior-driven development, BDD), is a method based on agile software development. It encourages collaboration among developers, business actors, and QA personnel. As another Python testing framework, Behave allows teams to avoid various complex situations to perform BDD testing. Essentially, the framework is very similar to SpecFlow and Cucumber, and is often used to perform automated testing. Users can write test cases in an easy-to-read language that can be pasted into the code during its execution. Moreover, those behavioral norms and steps that are set can also be reused in other test solutions.
Use prerequisites
Anyone with basic knowledge of Python can use Behave. Other prerequisites include:
You must first install Python 2.7.14 and above.
Requires a Python package manager or pip to work with Behave.
Most developers will choose Pycharm as the development environment, of course you can also choose other IDE tools.
advantage
- Behave helps maintain consistent behavior across an organization because the system behaves using a semi-formal language and a domain vocabulary.
- Let those development teams working on different modules have similar characteristics through coordinated processing.
- The individual functional blocks it builds are capable of executing various test cases.
- Since there are more details worthy of user reasoning and thinking, the target product will have better specification.
- Based on a similar specification format, it enables project managers and stakeholders to have a clearer understanding of the output of the development and QA teams.
shortcoming
Only suitable for black box testing.
Behave's features compared with competing products
Behave is best suited for black box testing like simple web testing. However, Behave is not a good choice for integration tests and unit tests with complex scenarios. As an alternative to Behave, some developers and testers will recommend pytest-bdd. This tool combines all the advantages of Pytest and implements various behavior-driven testing scenarios.
Lettuce
Lettuce is another behavior-driven automation tool based on Cucumber and Python. Lettuce mainly focuses on those mundane tasks that are characteristic of behavior-driven development. Not only is it easy to use, but it also makes the entire testing process smoother and even more fun.
Use prerequisites
You need to install Python 2.7.14 and above with IDE. Of course, you can also use Pycharm or any other IDE tool. At the same time, you also need to install the Python package manager.
advantage
- Similar to other BDD testing frameworks, Lettuce enables developers to create multiple scenarios and describe different functions in simple natural language.
- Based on a similar specification format, development and QA teams can coordinate and work together.
- For black-box testing, Lettuce is very suitable for those behavior-driven test cases.
shortcoming
To successfully implement behavior-driven testing, there needs to be ongoing communication between the development team, QA staff, and stakeholders. Lack of such communication can create process ambiguity and lead to problems for teams. Competitive comparison
In the opinion of average developers and automation testers, Cucumber is more practical in performing BDD tests. In the eyes of Python developers and QA personnel, Pytest-bdd is the best. The power of pytest lies in: its framework enables compact and easy-to-understand code, and can be tightly integrated into behavior-driven tests.
epilogue
To sum up, among the above five automated testing Python frameworks, Pytest, Robot Framework and UnitTest can be mainly used for functional and unit testing, while Lettuce and Behave are only suitable for behavior-driven testing. Through further comparison, we think: For functional testing, Pytest is the best. If you are new to Python-based automated testing, Robot Framework is the best tool to get you started. Although its functionality is limited, it is very easy to use. For Python-based BDD testing, Lettuce and Behave are equally good. However, if you already have some experience with Pytest, please use Pytest-bdd. I hope this article can help you choose a suitable Python testing framework and carry out the testing work smoothly.
The following are supporting learning materials. For friends who do [software testing], it should be the most comprehensive and complete preparation warehouse. This warehouse also accompanied me through the most difficult journey. I hope it can help you too!
Software testing interview applet
The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!
The following interview question sections are covered:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics
Information acquisition method:
