From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

1 Layered testing

Layered testing: In different time periods, different teams or teams use different test cases to test different focuses of the product. The first thing we see about a system/product is the UI layer, which is the appearance or the whole. These are the top layers. The top layer depends on the service layer below, which is the interface or module. The bottom layer is the unit. This unit is a function or method. According to these three layers, different time periods are selected, and testing performed by different teams and different test cases is called layered testing.

After reading through the above concepts, you should first have a general impression of hierarchical testing. Let’s explain it in detail in conjunction with the testing pyramid model:

1.1 Unit (Unit) testing

Unit testing is a test for a unit of code (usually a class/method). The value of unit testing is that it can provide the fastest feedback and verify the logical unit during the development process.

1.2 Interface (Service/Service/API) Testing

Interface testing is a test for business interfaces, mainly testing whether the internal interface function implementation is complete. For example, whether the main business flow can be passed through, whether exception handling is correct, verification when the data is empty, etc. The main value of interface testing is that the interface definition is relatively stable, unlike the interface or underlying code that changes frequently, so interface testing is easier to write and the maintenance cost of use cases is relatively low. Preparing tests at the interface level is relatively cost-effective.

1.3 Integration (UI) testing

Integration testing verifies the correctness of product functions from the user's perspective, tests the end-to-end process, and adds user scenarios and data to verify the entire business flow. Integration testing has the highest business value. It verifies a complete process. However, because it needs to verify the complete process, the cost of environment deployment, use case preparation and implementation is high, and it is not easy to implement.

1.4 Summary of layered testing

Google’s investment in automated layering is as follows: unit testing (Unit): accounting for 70%; interface testing (Service): accounting for 20%; integration testing (UI): accounting for 10%.

During the testing process, it is necessary to intervene in testing as early as possible and conduct thorough testing on key module functions. According to the pyramid model, the higher you go up and closer to QA, business and end users, the higher the cost of solving problems after discovering them. Using tiered testing has the following advantages:

Try to move testing forward, find problems and solve them in the early stages of development, and development costs will drop quickly.
Different attention is paid to different time periods, with key testing and layer-by-layer protection.
It is easy to locate the problem. It is very clear which layer is being measured. If a problem occurs, it is the problem on that layer.
Layered testing is more targeted when designing and executing tests on use cases, making the thinking clearer and less likely to be missed.
Strengthening testing's understanding of code implementation can better develop testing skills.

Finally, in the specific implementation, it is necessary to design how to divide the levels, design the test cases corresponding to the levels, and continuously track the use cases when they are executed. The previous work must play a practical role in the subsequent work.

2 UI automation

UI automated testing is an automated testing method that implements automatic operation and verification in code by simulating manual operation of the user UI interface. From the testing channel, it can be divided into WebUI testing and App testing. WebUI includes two directions: PC and H5.

2.1 UI automation function

Repeatable functional testing and verification;
Avoid human test omissions during fatigue operation;
The ability to obtain other test data through UI automation operations.

2.2 Advantages of UI automation

Use cases are easy to write, lowering the threshold for getting started;
Save manual testing costs and improve testing efficiency of functional testing and regression testing;
A means and method to ensure software quality.

2.3 Disadvantages of UI automation

Frequent changes in UI controls lead to control positioning;
The maintenance cost of use case scripts is high and the ratio of input to output is low;
The instability of element positioning leads to poor efficiency and stability of use cases.

3 Analysis of Common UI Automation Frameworks

Commonly used WebUI automated testing tools mainly include the powerful, free and open source Selenium family, the QTP tool with good experience but very expensive, and the emerging Cypress, as well as other tools.

3.1 Comparison of Cypress and Selenium users

Comparative analysis of Cypress and Selenium downloads: Selenium is relatively stable. Cypress downloads officially surpassed Selenium in 2021, and the difference continues to widen.

3.2 Comparison of Cypress and Selenium implementation architectures

The architecture of the Selenium system: the code communicates with the driver through the JSON Wire network protocol, the driver interacts with the real browser, and finally returns the operation results to the code.

Cypress system architecture: Use webpack to bundle all modules in the test code into a js file. The test code and the program under test are in different iframes in the same browser and do not need to be accessed through the network.

3.3 Comparison of Cypress and Selenium environment frameworks

Comparison of Cypress and Selenium automated environment construction: Cypress only needs to be installed and used. Selenium is provided as a library package. You need to choose the corresponding framework, assertions and additional dependencies.

3.4 Comparison summary of Cypress and Selenium environments

4 How to do UI automation well

4.1 N reasons why I don’t want to write UI automation

The cost of automated script writing is high. Selenium needs to build its own framework. Cypress can only be written in js, and both require some knowledge of the front end to write well.
It takes a long time to automate manual scripting cases. The number of written scenarios is too small and not many problems can be found.
When the page changes during recording and writing, each use case needs to be modified, requiring dedicated maintenance, and the maintenance cost is very high.
Errors caused by script problems, page loading exceptions, etc. often occur, and the written use cases are very unstable.

4.2 N reasons why I have to write UI automation

There are too many scenarios that require regression testing, and manual repeated and frequent execution is too time-consuming.
Online environment testing and wireless interface operation permissions are required.
UI automated testing is closer to users' actual usage scenarios, and quality cannot be fully guaranteed through interface testing.

4.3 How to do UI automation well—reduce code maintenance costs

For when we don’t want to write UI automation but have to do it, we need to choose a good framework to manage our automation use cases. Whether it is selenium or cypress, we need to reuse our automation script code as much as possible. If we just want to test process data, we need to encapsulate our controls and operations. Below we will mainly use warehousing Cypress automation as an example.

4.3.1 Encapsulation of basic controls

Generally, the basic package controls of the system have a certain style. For example, in the picture below, how can we quickly find the input box for the order number? Obviously, if you search directly through the "Please enter" field, you will find multiple ones and cannot locate the unique one. For the warehousing system, since the pages are all configured, there is no fixed and unique ID management.

Through DOM tree analysis, it can be concluded that an ordinary input box can find the corresponding label through the label name, then find the common parent node el-form-item, and then find the child node el-form-item__content of el-form-item. The child node el-input is used to find the input box to be entered and enter the content.

Encapsulate this operation into two basic custom commands:

//查找label所在的el-form-item控件组合根节点，正则全词匹配更加精确
Cypress.Commands.add('getElFormItemByLabel', (label) => {
  cy.get('.el-form-item__label').contains(new RegExp("^" + label + "$", "g")).first().parent('.el-form-item')
})


// 输入框填写值,根据传入的el-form-item找到el-input-inner对象进行输入，增加去除readonly事件，enter事件，以及强制输入
Cypress.Commands.add("cTypeWidthEvent", { prevSubject: 'element' }, ($elSelect, value, event = 'enter') => {
  cy.wrap($elSelect).find('.el-input__inner').then(($el)=>{
    $el.removeAttr('readonly')
    }).type(value + `{${event}}`,{force:true})
})


//后续使用输入框时候只需要这样使用
cy.getElFormItemByLabel('订单号').cTypeWidthEvent(orderNo)
cy.getElFormItemByLabel('派车单号').cTypeWidthEvent(TJNo)

According to the DOM tree analysis, the span tag content of the el-button class style can be obtained. The content of the span tag in the el-button class style is set. Find this unique node and click to realize the click event.

//封装点击自定义命令
Cypress.Commands.add('btnClick', (label) => {
    cy.get('button > span').contains(new RegExp("^" + label + "$", "g")).parent('button').click()
})

//如下使用封装所有页面的点击事件
cy.btnClick('设置')

Through the above method, the basic operating elements of the system are first encapsulated into customCommands and managed as the basic control of the system. The cost of maintaining control changes can be greatly reduced.

4.3.2 Page-Object mode manages system pages

Encapsulate the following order list page. For example, I will only search based on the order number in the outbound document center, and then click the order number to jump to the details page.

You just need to build a page class for order PO management. Since warehousing is used to run processes, you only need to encapsulate some used controls.

//创建PO管理，通过封装的基础命令来封装控件操作，此最好做到只维护控件名称数据
export default class OrderCenterListPage{
    constructor() {}
    clearReceiveOrderDate(receiveStartOrderDate,receiveEndOrderDate){
        //删除接单时间
        cy.getElFormItemByLabel('接单时间').children('.el-form-item__content').find('.el-icon-clear').first().click({force:true})
    }

    orderNoFilterInput(orderNo){
        cy.getElFormItemByLabel('订单号').cTypeWidthEvent(orderNo)
    }

    openOrderDetail(orderNo){
        cy.get('.el-table_1_column_3').contains(orderNo).click()
    }
}

//同时将此页面的操作封装成一个自定义页面操作命令
// 按订单查询明细
Cypress.Commands.add('OrderCenterListPage.openOrderDetail', (orderNo) => {
    var page = new OrderCenterListPage();
    page.clearReceiveOrderDate()
    var routeName = 'queryOrderListInfo'+ Date.now()
    cy.intercept('POST','**/order/web/queryOrderListInfo').as(routeName)
    page.orderNoFilterInput(orderNo)
    cy.wait(`@${routeName}`).then((res=>{
        page.openOrderDetail(orderNo)
    }))
})

4.3.3 Re-encapsulate the operation process

The above encapsulation is enough for page testing. If it is for process testing, there are the following steps in the outbound process similar to warehousing. Each page operation needs to be encapsulated into a process again to further improve code reuse.

Based on this, we have encapsulated the production process, as shown in the figure below:

4.3.4 Test case organization

First create a new test suite, and then encapsulate the main operations of each page into a use case of the process. Why do we do this? Split the test cases for verification to facilitate failure retry. If you write it in an it, it means that the failure retry needs to be triggered in full. On the contrary, you only need to retry one of the steps, which greatly improves the efficiency of success and shortens the time of retry execution. . You can also quickly find the location of the problem.

import ExceptionInBoundFlow from '../../support/flow/exceptionInBoundFlow'
import passBackList from '../../fixtures/0_990/passback.json'

describe('5.0到6.0切仓出库全流程验证', () => {

  const flow = new ExceptionInBoundFlow()
  const ibOrderNo = 'UAT_'+Date.now()
  const oBOrderNo = 'WMSESL140760105630761'   //WMSESL140760105632249

  before(()=>{
    cy.clearCookies()
  })

  after(()=>{
    //cy.clearCookies()
  })

  beforeEach(() => {
    //登陆系统
    cy.intercept('**/*',(req) => {
      req.headers['origin'] = 'http://sunlon.wms.jdl.cn'
    }).as('headers')

    cy.visit(Cypress.env('baseUrl'))

  })

  it('1.查询生产流程和生产状态',  () => {
    flow.getOrderProductionInfo(oBOrderNo)
  })

  it('2.根据定位异常下单',  () => {
    flow.receiveIbOrder({oBOrderNo:oBOrderNo,ibOrderNo:ibOrderNo})
  })

  it('3.扫描收货',  () => {
    flow.scanReceiving({orderId:ibOrderNo,locationNo:Cypress.env('locationNo').pickLocation})
  })

  it('4.重新定位',  () => {
    flow.reLocate(oBOrderNo)
  })

  it('5.是否手工定位',  () => {
    flow.manualLocation(oBOrderNo)
  })

  it('6.任务分配',  () => {
    flow.createOutboundTask(oBOrderNo)
  })

  it('7.拣货',  () => {
    flow.pickNew(oBOrderNo,Cypress.env('locationNo').pickLocation)
  })

  it('8.前合流',  () => {
    flow.confluenceBeforeCheck()
  })

  it('9.复核',  () => {
    flow.check({platformNo:Cypress.env('review').defaultPlatformNo,containerNo:Cypress.env('review').defaultContainerNo, palletNo:null, defaultConsumableCode:Cypress.env('review').defaultConsumableCode})
  })

  it('10.后合流上架',  () => {
    flow.upToShipmentLocation(oBOrderNo,Cypress.env('locationNo').fahuoLocation)
  })

  it('11.客单生成包裹',  () => {
    flow.createPackage(oBOrderNo)
  })

  it('12.发货',  () => {
    flow.quickShip(oBOrderNo)
  })

  describe('13.校验生产单回传',  () => {
    for(const index in passBackList){
        const node = passBackList[index].node
        const whiteList = passBackList[index].whiteList
        const desc = passBackList[index].desc
        it(`校验生产单回传节点${index},订单号=${oBOrderNo},回传节点=${node}，回传名称${desc},校验内容校验字段=${whiteList}`,  () => {
          cy.log(`校验生产单回传节点,订单号=${oBOrderNo},回传节点=${node},屏蔽校验字段=${whiteList}`).then(()=>{
            flow.passBackCompare(oBOrderNo, passBackList[index].node,passBackList[index].whiteList)
          })
        })
    }
  })
})

The above is a method for UI automation (actually not limited to UI) to reduce code script maintenance costs.

4.4 How to do UI automation well—improve script efficiency and stability

4.4.1 Eliminate waiting

When we write scripts, because some operations need to wait for the return of the background interface before proceeding to the next operation, we may increase the cy.wait(10000) setting to wait for 10 seconds for processing. If the interface does not return within 10 seconds, the use case will fail. For this purpose, we used cy.intercept to set up the interception interface route, and used wait to wait for the background interface to return before proceeding to the next step.

   //设置路由名称
    var routeName = 'queryOrderListInfo'+ Date.now()
    cy.intercept('POST','**/order/web/queryOrderListInfo').as(routeName)
 //操作页面按钮触发请求
    page.orderNoFilterInput(orderNo)
    //等待页面请求结束后进行下一步操作
    cy.wait(`@${routeName}`).then((res=>{
        page.openOrderDetail(orderNo)
    }))

In addition to solving operational problems, using cy.intercept can also be used to judge and assert the interface return value, and retry operations based on the interface return value to enhance stability.

cy.intercept('**/queryWaitTaskAssignOrderInfo').as('queryWaitTaskAssignOrderInfo1')
    //点击查询
    page.search()
    cy.wait('@queryWaitTaskAssignOrderInfo1').then((res) => {
        // 针对响应进行断言
        if(res.response.body.resultValue.total != 1){
            console.log('查找待组单的订单不成功，可能是未定位完成，再等待一分钟')
            cy.wait(30000)
            cy.intercept('**/queryWaitTaskAssignOrderInfo').as('queryWaitTaskAssignOrderInfo2')

            page.search()
            cy.wait('@queryWaitTaskAssignOrderInfo2').then((res) => {
                // 针对响应进行断言
                if(res.response.body.resultValue.total != 1){
                    console.log('查找待组单的订单不成功，可能是未定位完成，再等待一分钟')
                    cy.wait(60000)
                    cy.intercept('**/queryWaitTaskAssignOrderInfo').as('queryWaitTaskAssignOrderInfo3')
                    page.search()
                    cy.wait('@queryWaitTaskAssignOrderInfo3')
                }
            })
        }
    })

Finally, you can modify the request attributes through cy.intercept and set the interface mock result to solve the external interface dependency problem.

//设置所有请求添加请求origin
cy.intercept('**/*',(req) => {
req.headers['origin'] = 'http://sunlon.wms.jdl.cn'
}).as('headers')

//将widgets接口mock掉
cy.intercept('POST', 'http://example.com/widgets', {
  statusCode: 200,
  body: 'it worked!'
})

4.4.2 Data transfer

Cypress automation designs use cases through the above 4.3.4 method to improve stability issues, and also brings about the problem of how to transfer data between use cases. Synchronous data transfer in Selenium can be solved using a global variable. In Cypress, since every operation is asynchronous, the global variable method is not feasible. Here we can read and write files, read and write cookies, and read static variables in configuration files. Here we introduce the cookie method (note: Chinese is not supported in cookies, and if there is Chinese, the system will report an exception error).

//配置cookie全局生效：
Cypress.Cookies.defaults({
    preserve:/^testData*/
})

//获取生产流程和生产状态写入cookie：
getOrderProductionInfo(oBOrderNo){
            var json = JSON.parse(res.resultValue[0].json)
            //获取单据类型映射
            this.#testData.shipmentOrderType=json.ruleDetail[0].value[0] 
            //获取生产流程
            this.#testData.productionInfo.location.mode=json.outboundProcessDto.locatingRuleVo.operationMode
            this.#testData.productionInfo.splitOrder.mode=json.outboundProcessDto.splitOrderRuleVo.operationMode
             //单据生产流程和状态存入缓存，注意缓存不能放汉字
             cy.setCookie('testData.info.productInfo',JSON.stringify(this.#testData))
            })
 })

//cookie读取使用：
    receiveIbOrder({oBOrderNo,ibOrderNo,wait=30000}){
        //获取生产流程和上一步测试数据
        cy.getCookie('testData.info.productInfo').then(cookie=>{
             //存储sku和是否需要采购收货状态
             cy.setCookie('testData.info.productInfo',JSON.stringify(testData))
         })
    })

Through dynamic data acquisition and delivery, it not only implements common execution operations for different production processes for the same order, but also implements status-based retry, so that when the entire use case fails, re-execution can be set to ensure that the test passes.

4.4.3 Asynchronous Promise acquisition method

By introducing asynchronous Promise, the methods in the code that originally required then to be advanced layer by layer are asynchronously flattened and returned to the result. (It should be noted that it cannot be changed to async asynchronous attribute in the test case)

 export async function doTaskAsign({orderNo,orderType,pickType}){
    var batchNo = await promisify(cy['assembleFormCreat.doTaskAssign']())
    return batchNo
}

4.4.4 Shielding system abnormalities to improve stability

By introducing the following configuration in support, we can solve the problem of system error reporting and failure caused by non-cypress assertion errors.

Cypress.on('uncaught:exception', (err, runnable) => {
    // returning false here prevents Cypress from
    // failing the test
    return false
})

4.4.5 Use recording to assist in positioning complex controls

As a beginner user, you may learn grammar better and faster through recording. Cypress also supports recording.
Just configure: "experimentalStudio": true

4.4.6 Beautiful engineering structural framework

5 ways to learn UI automation

Selenium is not introduced in this article. There are a lot of project cases on the Internet for relatively mature frameworks, and various companies also have similarly developed recording tools. Cypress can learn from the 101 summary of Little Pineapple Test Notes. The boss basically introduced all the API usage practices of Cypress. The test team also has a physical book for reference and study. If you encounter problems, you can go to the Cypress community group for help. .

Not much else to say. Everyone is welcome to learn Cypress automation and contribute to our automation career.

Author: JD Logistics Xu Guigui

Source: JD Cloud Developer Community Ziyuanqishuo Tech Please indicate the source when reprinting

A brief discussion on the road to warehousing UI automation | JD Logistics Technology Team