problems encountered
Today, I will share a problem encountered in the process of using the metersphere platform.
Let me first talk about our requirements when we use the platform for automation:
To ensure the stability of use case execution as much as possible, assertions cannot be made too simple.
For example, you cannot just assert that the returned status code is 200. At least a few fields have to be checked.
In addition, if the test environment often deletes data and the like, then the test data in the interface may be affected, and it is necessary to ensure the stable and normal execution of the use case as much as possible.
Next, let's look at a problem encountered in the assertion when debugging a use case today:
The return result of the interface is a list of ids in the following format:
For the data returned above, my idea of assertion is: go to the database to find the data that meets the requirements, and then splice such a list out. Then make an assertion at the assertion position of the platform.
Then we found that if you use jsonpath to make an assertion on the platform, for this comparison, you can choose equal to, contain or regular to try, and then find that the assertion results all fail:
I don’t know if the careful friends in front of the screen have discovered the inconsistency that caused the assertion to fail. It turns out that after the interface returns the result obtained using jsonpath, there is no space between each element in the list, and then a list generated by the python script has spaces between the elements, so the assertion fails.
solution
After finding the problem, the method I use here is: convert the list into a string, and ensure that the printed string is in the same format as the string obtained using jsonpath:
quoteIds = '[{0}]'.format(','.join(map(str, quoteIds)))
A few questions arise from here:
1. Although many operations of testers can be simplified after platformization, some new problems will also be introduced to a certain extent. For example, the type of data extracted through jsonpath becomes a string and returned, which leads to the need to do additional format conversion when asserting.
2. Similar to the list data comparison function above, you can try to use script assertion to solve it. It's just that I personally find it a bit cumbersome. If the platform can support the option of custom assertion, it is more convenient to encapsulate an assertion option yourself.
You are also welcome to join the fan exchange group of the official account. Learning resources will be provided from time to time in the group, and some industry information will also be shared from time to time. We look forward to growing together with you. In order to ensure the quality of group members, please add me as a friend first, and briefly introduce yourself (open circle of friends, which city you are in, and what position you are in). After confirming the identity of the test peers, I will invite you to join the group. Avoid some advertisers from mixing into the group, which will bring you a bad experience~