Talk about UI automation testing of Dojo application

Dojo is a popular JavaScript framework for building rich client-side web applications. UI automation testing is crucial when developing and maintaining Dojo applications. It helps testers verify the app's functionality and user interface and ensure its consistency across browsers and devices.

Although the topic of this article is UI automation testing for Dojo applications, much of the material is applicable to other Web 2.0 applications as well. I hope that readers and friends can benefit more or less from it.

What is Dojo?

Dojo is an open source DHTML toolkit implemented in JavaScript. It was built with contributions from several projects (nWidgets, f(m), Burstlib). Dojo's original goal was to solve some long-standing historical problems encountered in developing DHTML applications, and now Dojo has become a powerful tool for developing RIA applications:

Dojo makes it easier for you to add dynamic capabilities to Web pages, and you can also use Dojo in other environments that support JavaScript;
With the components provided by Dojo, you can improve the usability and interactivity of Web applications;
Dojo largely shields the differences between browsers, so you don't have to worry about whether a Web page is available in some browsers;
With the tools provided by Dojo, you can also write command-line unit tests for your code.

Dojo's packaging tools help you optimize your JavaScript code and generate only the minimal set of Dojo packages needed to deploy your application.

Dojo Application UI Automation Testing Challenges

The challenges faced by Dojo application UI automation testing can be classified into two categories: development phase challenges and maintenance phase challenges.

Challenges in the development phase:

Handling of asynchronous requests
element positioning
Dojo complexity
product complexity

Challenges in the maintenance phase:

frequent UI changes
Dojo upgrade

Below, we describe each challenge in detail. In order to easily correspond to these challenges when explaining the design principles later, we mark each challenge with an English letter.

A. Processing of asynchronous requests

In traditional web applications, every time a user clicks a hyperlink on a page, the browser will send a request to the server, wait for the server to respond, and then return a completely new web page, but in most cases the user has to endure the page Blinks and long waits. In order to solve the problems in traditional web applications, Ajax appeared. By using Ajax pages and applications to ask the server for what they really need, that is, the part that needs to be modified in the page, the server will provide that part, which makes the traffic less, the update less, the user waits for the page to refresh shorter, and the user is more comfortable. satisfy.

However, the arrival of Ajax may not be good news for Web UI automation testing. The execution method of the Web UI automated testing framework is similar to a "queue": the user defines a series of page operations and verifications, and then the Web UI automated testing framework executes them one by one in order. In traditional web applications, since each request to the server will cause a page jump, the web UI automation testing framework can easily determine the appropriate next action to execute. However, the behavior of Ajax is asynchronous, that is to say, the user's operation and the server-side processing can now be parallel, and there is no strict order. At first, this caused trouble for developers of web UI automation testing frameworks, and this problem has not been well solved for a long time.

At first, there were mainly two solutions for Web UI automation test developers: hard-coded wait time and condition-based wait.

Hard-coded waiting time : This method refers to specifying a static waiting time in the code, such as specifying to wait for 30 seconds before performing the next operation. Usually, the setting of this value comes from experience. It is conceivable that if the waiting time is smaller than the actual waiting time, the automation script execution will fail. And, unfortunately, it does happen quite often. As the test environment changes, the actual wait time required is actually a variable. Several factors can cause it to change:

Whether the test server is remote or local: I have no idea that due to the influence of the network, if the server is remote, the waiting time will increase.
Browser used: The execution speed of the browser's JavaScript engine also has an impact on the latency. As a rule of thumb, both Firefox and Chrome are faster than IE. Therefore, if you are using IE, the waiting time will also increase.
The network environment when the test is running: If the network speed is slow when the test is running, the waiting time will increase.
Test server load: The test server may be exclusive or shared. If the server load is high when the automated test is executed, it will also lead to increased waiting time.

For the above-mentioned problem of hard-coded waiting time, Web UI automation test developers generally have the following countermeasures.

Try to set the waiting time as large as possible. There are some problems with doing this. First, it may not be guaranteed that the value is large enough in any case; second, it means that no matter whether the server is local or the remote browser is fast or slow, the execution time is the same. This is a test for the local server and the relatively fast browser The environment is a big waste of time cost.
Set the magnification factor. This method is to multiply the waiting time by a factor when encoding. This factor is either hard-coded, eg defined in a parameter file. When the test environment is better, the setting is smaller; when the test environment is worse, the setting is larger. For advanced ones, this factor can be set dynamically. For example, the execution time of a certain operation can be used as a reference for calculating the value of the factor. This approach partially improves the situation, but still does not maintain the stability of its execution.

Condition-based waiting: Due to the instability of the hard-coded waiting time approach, Web UI automation testing frameworks began to support "condition-based waiting". This method uses a "polling" mechanism to determine whether an operation can be started. Assume that the current operation to be performed is a button click action. This button is disabled by default and is enabled when the page loads data from the server. Therefore, if you want to ensure that the operation succeeds, you need to "poll" the state of the button - until it is found to be enabled, then perform the click operation. What if the button has been disabled due to a bug? Therefore, the condition-based waiting method needs to set a "timeout period". If the button is not enabled after this time, the test case is considered to have failed. Therefore, everyone has discovered that the setting of this timeout time is actually a "hard-coded waiting time", so to some extent, this method still has the disadvantages of the hard-coded waiting time method. Another disadvantage of this method is that it brings additional coding costs, and the automated test code may be filled with many conditional judgment codes, and they do not accomplish our "business" goals by themselves, but are only auxiliary codes.

B. Element positioning

General Web UI automation test coding consists of several steps: locating and obtaining elements, performing operations, and verifying results. Also, usually Web UI automation test developers spend a considerable amount of time on element positioning and fetching steps. Most web UI automation test developers expect the required element to have a static id attribute. According to the specification, the value of the id attribute must be unique in the DOM tree, so the element can be easily and accurately located through the id attribute. However, such requirements are often not product requirements, but just to meet the convenience of Web UI automated test code development. Therefore, for product developers, adding a static id attribute to each element that needs to be used in Web UI automation testing is not a high priority. Also, technically, some elements in a Dojo application cannot have a static id attribute set.

For Dojo applications, the situation is a little more complicated. Figure 1 shows the DOM structure rendered by a Dojo button widget in Firebug. There is a widgetid attribute on the outermost SPAN element, and the id attribute is defined on the inner SPAN element. What they have in common is that they are composed of a prefix and a dynamic number. This number is determined by the position of the Dojo widget in the entire DOM tree at runtime, and the index starts from 0. So you can see that the Button widget in Figure 1 is the second button on the entire DOM tree. Widgetid is an attribute of every Dojo widget, and if not specified, it is the dynamic structure as shown in the figure.

Therefore, how to locate and obtain the elements in the page through the id and widgetid in this dynamic format becomes another challenge for the development of Dojo application UI automation test.

Figure 1. The Dojo button widget

C. Dojo Complexity

Don't get me wrong, the "Dojo complexity" mentioned here does not mean that Dojo is complicated to use, but that the DOM structure of the page generated by the Dojo application is more complicated. As shown in Figure 1, the button originally implemented with an INPUT element is now nested with four layers of SPAN elements. Therefore, a Dojo widget is no longer a simple HTML element, but a combination of many HTML elements. The Dojo button widget is just a simple example, for Dojo widgets like tables, calendars, etc., their structures are more complex. The operation for a Dojo widget is no longer as simple as operating a common HMTL element (it can be realized by calling a JavaScript method), but it is necessary to specifically locate the internal corresponding element of the Dojo widget that responds to an event (such as a click event) , and then trigger specific events.

Most web UI automation testing frameworks can only handle basic HTML elements. This is reasonable, because the framework developers do not know what kind of front-end framework the final automated test developers use, so they can only provide the most basic "little bricks". If the automated test developer of the specific application has any needs, secondary development can be carried out on top of it.

For developers of UI automation tests for Dojo applications, the option is to directly locate and manipulate elements inside Dojo widgets. However, the disadvantage of doing so is obvious - lack of reusability. Therefore, Dojo widgets should be considered as the smallest unit of operation. How to modularize Dojo widgets is yet another challenge.

D. Product Complexity

Product complexity has two meanings.

One is that the product itself has many layers of pages, many interface elements and complex functional logic. In the development process of Web UI automation test, writing code for positioning elements usually takes up a large part of the time, and the workload of writing this part of code is directly affected by the number of page elements. Functional complexity generally does not increase the coding workload linearly, but it may increase the debugging workload.

The second is that the DOM structure of the product page is complex. The development model of Web 2.0 applications is very different from that of traditional Web applications. Traditional web application development is basically a page corresponding to a physical program file (for example, .jsp file). However, in the development of Web 2.0 applications, there are usually only a few physical program files in the entire application, and the page jump and presentation are completed by updating the DOM tree structure through JavaScript. Therefore, the jump page in the browser is actually only the DOM tree structure being updated. In this way, there may be such a problem: DOM child nodes of multiple pages exist on the DOM tree at the same time. "Mispositioning" can easily occur if other pages have elements similar to the one currently being positioned.

How to reduce the development workload as much as possible while the product becomes more and more complex is also a problem that developers of Dojo application UI automation tests must solve.

E. Frequent UI changes

Frequent UI changes have become another headache for Web UI automation test developers after "agile development" became popular. The product UI in iterations is often constantly modified with customer feedback, so the automated test code needs to be updated accordingly. The resulting huge maintenance workload of the Web UI automation test script has also become one of the important reasons why many people oppose the Web UI automation test.

Is it because of this that we give up Web UI automation testing? In my opinion, as a Web UI automation test developer, what needs to be considered should be: under such a reality, how to minimize the code maintenance workload caused by frequent UI changes. From experience, not every little change on the Web UI will lead to modification of the automated test code. The solution to this problem depends heavily on the testing framework's ability to adapt to changes in the DOM structure and the method used when positioning elements.

F. Dojo upgrades

Dojo upgrades may cause changes in the DOM structure of Dojo widgets, which in turn will increase the maintenance workload of automated test code. Although empirically, the impact of this is not very significant, because some common Dojo widgets (such as button input boxes and the like) are very mature, but it still needs to be considered. If a widget's DOM structure changes, how can you minimize the impact on your automated test code? This is a question for Dojo application UI automation test developers to consider.

Dojo application UI automation testing framework selection (design) principles

After explaining all the challenges, let's take a look at the factors and principles that need to be considered when we want to design or choose a Dojo application UI automation testing framework.

To sum up, there are mainly the following eight principles. In brackets are the challenges that the principle addresses.

implicit wait (A)
Position relational operations (B, E)
Encapsulates Dojo widgets (C, D, F)
IBM framework (E)
code generation (D)
Management of test data (D)
Recording and Encoding (D)

Next, we explain each principle one by one. In order to help readers understand, part of the content of Water Bear, an internal self-developed Dojo application UI automation testing framework, will be quoted during the explanation.

implicit wait

Challenge A mentions the problem of asynchronous request processing and describes several solutions, none of which are ideal. The ideal situation is that the automated testing framework can handle the problem of asynchronous requests by itself, and does not require developers to write any code. This is the so-called "implicit wait" mechanism.

At present, many Web UI automation testing frameworks have the "implicit wait" function, such as Sahi and Selenium. Water Bear uses Sahi. Of course, Sahi also provides API support for "hard-coded wait time" and "condition-based wait" (because, in some cases, you have to use "hard-coded wait time" or "condition-based wait"). According to experience, more than 90% of situations can be covered by Sahi's "implicit wait" function, but functions like "progress bar" have to use "conditional wait". Before the progress bar reaches 100%, the front-end code continuously sends Ajax requests to the background to poll the progress status. In this case, Sahi cannot intelligently know which Ajax request is the "last one", so it cannot decide when to execute the next action.

Listing 1. "Progress bar" code example

BrowserCondition cond = new BrowserCondition(getBrowser()) {
public boolean test() throws ExecutionException {
return getBrowser().isVisible(getBrowser().div("100%").in(es));
}
};
getBrowser().waitFor(cond, waitTime);

Listing 1 shows part of the code that implements the progress bar functionality in Water Bear, which is Sahi's "condition-based wait" code. The waitFor method keeps polling until the test method returns "true" (that is, the code finds that the text "100%" appears). waitTime is a timeout. If this time is reached, waitFor returns even though the test method still returns "false". Usually, at this point it can already be determined that there is a problem with the functional execution of the Web UI.

The reason why I put "implicit wait" first is because whether the Web UI automation test framework has this function is crucial to the success or failure of the Dojo application UI automation test implementation.

position relation operation

A Dojo application UI automation testing framework should have the ability to locate other objects based on a given object. This relative positional relationship usually includes: inside, near, on the left (right) and above (below) and so on. Currently, Sahi supports all of these positional lookups, in addition to "above or below" and "to the left or right". Among them, "inside" is used most frequently.

Why use positional relational operations? There are several reasons:

Accurately position page objects. In Challenge D, we mentioned that it is possible to have an object in a DOM tree whose properties and DOM structure are exactly the same as those of the object being looked up. This is especially true for "OK" or "Cancel" buttons. At this time, it is necessary to narrow the search scope by restricting the parent object. Generally speaking, as long as the boundary elements limited to the current page can prevent cross-page DOM node lookups (because it is clear at a glance which objects are on the current page, but the DOM nodes of other pages are controlled by runtime operations and cannot be predicted) .
Encapsulates a Dojo widget. This will be explained in detail below.
Provides richer lookup methods, such as finding Dojo widgets based on "tags". Typically, there will be a text to the left of the text box, such as "Name:". If the text box does not have any other suitable static attributes for positioning, you can use the "nearby" search method to first locate the element where the "name:" is located, and then find the text box.
Avoid automated test code changes when the DOM tree structure has not undergone "functional" changes. The so-called "functional" change refers to the addition, replacement or removal of controls on the page, such as buttons or input boxes. When these changes occur, the automated test code has to be modified. However, if only changes in the DOM tree structure occur, such as the addition of invisible DIV or SPAN elements, as long as the positional relationship operations are used properly, the automated test code can be modified without modification.

Let's look at an example. The left side of Figure 2 is the current state of the application under test: a Dojo button widget surrounded by a DIV element. In the code, we position the button with the text "OK" and constrain it within the DIV of the icon. Afterwards, the DOM tree structure changed, and became the right side of Figure 2, and two layers of DIV elements were added between the original DIV element and the button. At this point, the code doesn't need to change because the "OK" button is still inside the previous DIV element. However, if you use XPath to locate elements, this situation is likely to change.

Figure 2. Example of positional relationship operations

Encapsulates Dojo widgets

In Challenge C, the complexity of the Dojo widget DOM structure was mentioned. The best way is to encapsulate the Dojo widgets and operate on each Dojo widget as a whole.

The programming language used by Water Bear is Java. It maps a Dojo widget in a page to a Java class, thereby encapsulating all of the Dojo widget's behavior. Because Sahi supports "inside" positional operations, encapsulation is easier - only the outermost HTML element (usually DIV) of the Dojo widget needs to be positioned, and other elements are positioned within this element. Therefore, even if there are two Dojo widgets of the same type on the page at the same time, there will be no wrong operation across components.

Figure 3 shows some of the classes defined in Water Bear that correspond to Dojo widgets. The topmost parent class WebElement.java represents a general HTML element, which encapsulates the methods of the basic HTML elements. DojoWidget.java is an abstraction for all Dojo widget classes. It defines methods common to all Dojo widget classes, such as getWidgetId(), etc. All Dojo widget classes inherit from DojoWidget.java and define their own methods, for example, ComboBox.java needs to define the selectByText method.

Figure 3. The classes corresponding to the Dojo widgets in Water Bear

So, how do the Dojo widgets on the page correspond to these classes? The two are associated in Water Bear through the .wdef file.

Listing 2. Example .wdef file

org.waterbear.core.widgets.dijit.Textbox=div|textbox|labelnear|dijit_form_ValidationTextBox,\
evo_form_ObjectNameTextBox,dijit_form_NumberTextBox,dijit_form_TextBox,aspen_config_directory_DNTextBox
org.waterbear.core.widgets.dijit.Checkbox=div|checkbox|labelnear|dijit_form_CheckBox
org.waterbear.core.widgets.dijit.Select=table|table|labelnear|dijit_form_Select
org.waterbear.core.widgets.dijit.Button=span|span|labelinside|dijit_form_Button
org.waterbear.core.widgets.dijit.ComboBox=div|textbox|labelnear|dijit_form_ComboBox

Listing 2 shows part of the .wdef file. It is a plain text file, and each line defines a set of mapping relationships. The left side of the equal sign is the Java class name mapped by the Dojo widget, and the right side of the equal sign is the property description of the Dojo widget on the page. Among them, the last attribute plays a key role, which is the prefix part of widgetid. A Dojo widget class can be associated with various page Dojo widgets, because although some Dojo widgets are extended, the extended functions will not be involved in Web UI automation testing, so they can be represented by the same class.

IBM framework

The IBM framework, formerly known as the ITCL framework, was developed in collaboration between Quality Software Engineering and the experienced automation teams at IBM. The framework consists of a three-tier architecture, and the implementation of the architecture runs through application objects, tasks, and test case packages (IBM packages).

The rationale underlying application object, task and test case packages are:

Hierarchical Architecture
Separate the "what" from the "how"
code reuse
Consistent and clear organizational structure
Ability to rapidly increase
quick debugging
Organize files efficiently
enable collaboration
learn from others

The following is an explanation of the application objects, tasks and test cases:

Application Objects: Store information about GUI elements in your application. At the same time, you can also write your Getter methods here. These Getter methods can return objects, so that the caller can query and operate these GUI elements. Typically, these methods are invoked in the Task layer.
Tasks: Here you will write reusable methods that perform common functionality in your application. Also here, you'll write methods that can handle and query complex application-specific controls. Methods in tasks can be called by test cases.
Test cases: methods for navigating an application, verifying its state, and recording its results.

Applying this idea to Java programming is the use of different "packages" to organize classes for application objects, tasks, and test cases. No matter what the specific underlying Web UI automation testing framework is, it is helpful to use the concept of the IBM framework to structure the classes of the upper-level test cases. Adoption of the IBM framework is an effective solution to Challenge E because high reusability keeps code changes localized and one change corrects all referenced code. In addition to adopting the IBM framework, how to actually plan the granularity of the task method reasonably is also critical. However, this is very application-dependent.

code generation

Generally speaking, writing the code used to obtain interface elements (that is, the code of the application object layer) occupies a large proportion of the entire development workload. Usually, it requires the developer to use some tools that can display the DOM tree structure (such as Firebug) to observe the attributes of the elements, and then decide which attributes to use to locate the elements is more appropriate. In addition, some codes are relatively "mechanical" or patterned, such as the operation of assigning values to interface elements. Such code should be generated in some automated way as much as possible, which can greatly save development time.

In Water Bear, two code generation tools were developed for the above two problems.

A tool called AppObjs CodeGen, it can automatically traverse the current DOM tree, thereby generating the code of the application object layer. It is implemented with Sahi script and outputs the results in its log window (Figure 4). Developers simply copy the code into their own class files. Although the code may still require some manual modification, such as renaming to improve the readability of the method, it saves a lot of time compared to coding it completely by hand.

Figure 4. AppObjs CodeGen generated results

The second tool is called Skeleton CodeGen. Its input is the class of the application object layer, and the output is part of the code of the task and test case layer. This tool is more effective for Dojo applications that conform to certain patterns and have interfaces with many UI controls. The generated code still needs to be modified twice, because some logic cannot be automatically generated.

Management of test data

It is best to separate test data and test case code, so that the same test case code can be run with N different test data. In addition, this separation makes it easy to define different test data for different test environments – sometimes, some test data is environment dependent.

Water Bear leverages the open source software XStream to separate test data from test case code. First, the test data is defined in a HashMap. When the test case runs for the first time, the HashMap will be automatically serialized into an .xml file through XStream. When the automation script runs again, if the .xml file already exists, it directly reads the existing file and deserializes the file content into a HashMap with the help of XStream. In subsequent runs, as long as the data structure of the HashMap has not changed, there is no need to regenerate the .xml file. The storage directory of the data file can be specified through the configuration file, so for different environments, it is only necessary to modify the configuration file to point to a different path.

Recording and Encoding

Another way to improve development efficiency is to provide recording capabilities. But the recording function usually has the following problems:

Recording generated code is usually "procedural". "Procedural" code is hard to reuse. Even different test case codes recorded for the same page function module are isolated. If the functionality of the page changes, all test cases need to be re-recorded. Therefore, on the surface, the recording function seems to greatly reduce development time, but when the page changes, it may not be as good as we imagined.
The recorded code is usually less readable. In general, variables still need to be renamed after recording to improve code readability.

Therefore, ideally, the code generated by the recording function should be modular, that is, generated according to the three levels of application object, task and test case. Such code is easy to integrate with existing code even if it is easy to modify twice. From the point of view of use, it is unrealistic to generate all test case codes completely with the recording function. A better way is to only use the recording function to assist in the development of application object layer and task layer code. When non-functional changes to the UI occur, only the affected task methods need to be re-recorded and the application object layer updated. In theory, the test case layer does not need to be modified (unless the logic of the test case has to be modified).

Water Bear currently only supports "Procedural" recording. Figure 5 shows the code generated by Water Bear Recorder. Developers can copy and paste the generated code into their own test classes to run directly in Water Bear.

Figure 5. Water Bear Recorder

As someone who has been here, I also hope that everyone will avoid some detours

Here I will share with you some necessities of the way forward in automated testing, hoping to help you.

(WEB automated testing, app automated testing, interface automated testing, continuous integration, automated test development, big factory interview questions, resume templates, etc.)

I believe it can make you better progress!

Click on the small card below