[The most detailed in the entire network] Automated testing precautions + summary of problem points, don’t take any more detours...

---

Preface

1. How is the WEB automated testing framework built?

The technology stack we use for web automation testing is:
Python+Selenium+Pytest+Parametrices+Excel+Allure+Jenkins. The framework uses Excel-based keyword drivers, which separates the maintenance framework and usage framework. When conducting automated testing, the elements The positioning expression and the operation to be performed can be written into Excel, which significantly reduces the difficulty of implementing automated testing.

Main functions:
built-in commonly used keywords to realize keyword-driven testing, with the potential to be upgraded to BDD;
automatically determines the browser type and version, and automatically downloads and starts the appropriate browser driver;
built-in automatic waiting to avoid UI testing under normal circumstances error conditions;

By executing js, special scene interactions are realized, such as forced click, forced input, drag-and-drop upload, etc.;
with string as the core assertion strategy, it supports multiple assertion methods such as equals, inclusion, regular matching, content combination, etc.;
automatically generates allure Test report, the built-in report matches the excel content one-to-one;

Supports parallel testing and distributed test file architecture:

conf # 项目配置 
data # 数据驱动测试文件 
action.py # 关键字驱动封装
case.py # 用例管理和封装
data.py # 数据驱动封装
pages # PO 封装 
report # allure测试报告 
tests # 存放 excel文件作为测试用例框架用法：
1）创建execl文件，每个sheet页看作一个TestSuite
2）在sheet页中申明测试用例，填写测试用例的名称
3）在单元格中填写步骤名、关键字、关键字参数，完成测试步骤

2. What is the value of WEB automated testing? Why do we need to do WEB automated testing?

Web automated testing is to simulate manual testers to do functional testing.
Replace human operations with machine automation.
Mainly used for smoke testing and regression testing of the core functions of the product.

Start with the core functions of the system, and then slowly expand based on the situation.
After introducing automated testing, it can replace a lot of tedious regression testing work and free up business testers, allowing business testers to focus on complex business function modules. Automated testing generally automates stable functions to ensure There will be no bugs in previously stabilized functions due to product updates.

3. How does the company implement WEB automated testing in projects?

1) The project team conducts a feasibility analysis of automation, including to what extent can the automation rate be implemented?
Main considerations:
whether the project time is relatively long, generally requiring more than one year of projects or long-term products; whether
the requirements will change frequently;
whether the automated script can be used continuously and repeatedly;

2) The project team investigated the selection of automation tools and demo demonstrations, mainly demonstrating selenium and robotframework. It would be better if there are previous mature framework demonstrations.

3) Gradually implement automation in the project, discover problems and improve them.
Mainly includes:
writing automated test plans;
extracting or writing automated test cases;
writing automated test framework scripts by the leader;

Write, debug and maintain automated test scripts (code specifications and write corresponding automated test scripts according to the division of labor, and submit them to the Git server after the self-test has no problems) Jenkins continuous integration unattended
testing;
later maintenance scripts (mainly because use cases are added due to version updates) );

4) Process, standardize, and document automation. The automation framework requires the preparation of usage documents and specification documents.
5) Generate customized reports. Continue to improve the framework. and promote it to other projects

4. How many cases are there for web automation? What is the coverage? How long does it take to execute everything?

The number of WEB automation test cases depends on the number of functional test cases. Automation use case coverage is generally about 30% of functional test cases. It takes
about 30-60 minutes to complete the execution of all WEB automation use cases.

5. How to design web automated test cases?

WEB automated test cases are extracted from manual test cases, mainly smoke cases and regression test cases. The selection of automated test cases requires more attention to the rigor of the use cases. When selecting use cases, follow the following principles:

Prioritize use cases that cover the core business processes of the product;
select use cases that require repeated execution or cumbersome verification functions, such as field verification and prompt information verification; give
priority to forward test cases, reverse use cases are generally complex and large in number;
do not Select use cases where the process is too complex (except for the main process);

6. How to ensure (improve) the stability of automated testing?

The stability of automated testing is mainly reflected in two aspects: one is the stability of element positioning, and the other is the stability of dependencies between use cases: Element positioning stability issues:

1) Try to use xpath expressions with relative paths;
2) Position elements using smart wait + display wait to avoid the problem of elements not being loaded;
3) Add a retry mechanism after script execution fails to improve the stability of the use case;
4) Use cases After execution, restore the test scenario to avoid affecting the execution of other use cases;
5) Try to ensure a separate test environment to avoid simultaneous execution of other tests;

Dependency stability issues between use cases:
Try to avoid dependencies between use cases, and use cases can be executed independently as much as possible; let each use case start executing from a common page, such as the homepage, which requires post-processing in the test framework This way, each use case will return to the homepage after execution.

7. Do automated tests find many bugs?

Not many, because the use cases for WEB automation are basic functions that have been tested by the previous project team, and then the automated scripts are written and executed. It is mainly to ensure that the functions that have passed the test will not be affected by the new functions after the new version is updated. This causes errors in the original basic functions.

8. What is PO model? Why use it?

PO model, the full name is Page Object Model, referred to as PO, is the page object model.

It means treating a page as an object, the elements of the page are the attributes of the object, and the operations of the page are the behaviors (methods) of the object.

PO mode generally uses a three-layer architecture, which are: base encapsulation layer BasePage, PO page object layer, and TestCase test case layer.

Using the PO mode can make our test cases simpler and clearer. In many cases, we can encapsulate many business operation methods in the page object, and the test script only needs to call the relevant methods. The other is that if a page element changes, we only need to modify the element positioning and related methods of the page object, without modifying other scripts. Increase code maintainability.

9. Do you understand data-driven and keyword-driven?

Data-driven is to read the input and output test data from a certain data file (such as Excel file, Csv file, YAML file, etc.), and then pass it into the automated test script through data-driven methods (ddt, parameterize, etc.).

During the entire process, automated test scripts realize data reading, test status changes, result judgments, etc., thereby achieving separation of data and code. This approach is called data-driven.

Keyword-driven is based on the object-oriented idea. The same business logic will be written into a function as a keyword to be called by different test scripts. When all business logic tests can be combined by written functions, it is a keyword-driven framework.

At this time, the development of test cases becomes a combination of test data and keywords, and this combination work is simplified into a form filling task that everyone is familiar with, so as to ultimately achieve an effect where the entire test is driven by data and keywords. .

10. Which Python libraries are mainly used in the automated testing process? What exceptions have you encountered?

Web automated testing:
webdriver, WebDriverWait, By, os, xlrd, xlwt, unittest/pytest, time, logging, HTMLTestRunner, etc.

Interface automated testing:
requests, time, logging, json, csv, jsonpath, pyyaml, re, unittest/pytest, allure

Common exceptions include the following:

NoSuchElementException： 没有如此元素异常
NoSuchAttributeException ： 没有如此属性异常
NoSuchFrameException ： 没有如此frame异常
ElementNotSelectableException ：元素不能选择异常
ElementNotVisibleException ：元素不可见异常
Element not visible at this point ：在当前点元素不可见
TimeoutException ： 超时异常；

11. What bugs were found during the interface testing process? How many? (In other words: Are there any problems with the interface test?)

Most of the bugs discovered during interface testing are related to issues such as the interface not returning results as agreed, parameters being empty, parameter length or type verification, parameter boundary values, code logic, data errors, or not returning reasonable error prompts.

1) Generally, there will be many bugs found when performing interface testing during the front-end and back-end joint debugging phase (when developing and writing code before, all ajax data was not the real data returned by the back-end, but the fake data simulated by ourselves through the interface mock. After the front-end code has been written and the back-end interface has been written, we need to remove the mock data and try to use the data provided by the back-end to perform joint debugging of the front-end and front-end. This process is called the interface connection of the front-end and the front-end. Tune.)

Front-end and back-end joint debugging interface bug cases:
For example 1: Query list page interface, the front-end wants to paginate, Mock writes three test data, is there a problem with paging when calling the back-end interface?
For example 2: Authentication is not considered when the front-end accesses mock data. There is no permission to access the real backend interface.

2) Then when the interface test is performed during the interface smoke test and regression testing phases, there will be fewer bugs.
General interface bug cases:
For example 1: Add a new promotion interface, and the full discount amount can be saved successfully even if it is empty. The reason is that the back-end code does not make a null value judgment on the full discount amount parameter.

For example 2: In the activity list interface, the activity data queried is missing the first item. The reason is that the starting sequence number passed in the limit condition in SQL is 1 instead of 0.

For example 3: Update activity interface, the interface prompts that the update is successful, but the update_time field in the database is not updated to the latest time. The reason is that the developer forgot to update this field. For example 4: For example, in the order interface, the price parameter of the product is 300 yuan, then I When submitting the order, I changed the price of the product to 3 yuan, but the back-end did not verify it. To be more cruel, I changed the price to -3, and my balance still increased?

Interface authentication bug case:
For example 1: For example, if you modify the product information interface, only the seller's permission can be modified. I can modify it successfully if I upload it to an ordinary user, and I can modify it successfully if I upload it to another seller user.

For example 2: There was a surrender interface before, and the downstream system added ID card validity period verification, which caused the interface call of the system under test to not work. The problems were discovered through automation and the impact on the system under test was evaluated in a timely manner.

The following is the most comprehensive software testing engineer learning knowledge architecture system diagram in 2023 that I compiled.

1. Python programming from entry to proficiency

2. Practical implementation of interface automation projects

3. Web automation project actual combat

4. Practical implementation of App automation project

5. Resumes of first-tier manufacturers

6. Test and develop DevOps system

7. Commonly used automated testing tools

8. JMeter performance test

9. Summary (little surprise at the end)

When you can break through yourself and overcome difficulties, your dreams are no longer far away. Only by continuing to struggle can we write a bright chapter in life and realize our own value and meaning. Keep at it and never give up!

Be brave enough to step out of your comfort zone and break through the constraints and limitations. Only by experiencing challenges can you achieve unlimited possibilities. Struggle is the source of strength for us to realize our dreams. With firm faith and hard work, we will eventually reap brilliant achievements!

Embrace every challenge and bravely pursue your dreams. Struggle is not only a choice, but also an attitude. Only by burning the passion in your heart can life bloom into a brilliant chapter. Firm your faith, work hard, and reap endless possibilities!