In the last chapter, we learned how to start and close the App, but in the operation of the automation use case, the most important thing is the operation of the UI elements;
In this chapter, we will mainly learn how to locate elements, the use of positioning tools, and what are the basic element operations.
【一】Locating tool
There are two commonly used positioning tools:
1. Android sdk comes with uiautomator viewer
2. Appium's own inspector
The use and usage of these two tools have their own advantages and disadvantages and characteristics. Next, we will introduce them one by one:
uiautomator viewer
The uiautomator viewer is located in the android-sdk\tools directory, double-click uiautomatorviewer.bat to run it.
Note: If there is no jdk or jre, or the environment variables are not configured properly after installing jdk, uiautomator viewer will not run.
As shown in the figure, the tool display is roughly divided into 4 areas, the menu bar, the screenshot area, the xml file code area, and the element information area;
Click the second or third button in the menu bar to take a screenshot of the phone screen and locate the elements in it.
The basic usage is:
1. Get a screenshot;
2. Move the mouse over the element to be positioned, click the left mouse button, and the border will change from a dotted line to a solid line;
3. Check the relevant element attributes in the element information area in the lower right corner.
Note: If you want to cancel the selection of the element, you only need to click the left mouse button again on the clicked element
inspector
To use the inspector tool, you need to start the appium server first, and click the magnifying glass icon in the upper right corner.
Probably you need to set two places:
1. Click Automatic Server
2. Fill in the desired capabilities of the App, which is the desired_caps set in python before starting the app, which can basically be copied.
Then click start session in the lower right corner to start the inspector's session.
As shown in the figure above, the inspector is roughly divided into 4 areas: menu bar, screenshot area, xml file source code, element positioning method and attribute information.
The method of selecting elements is similar to that of uiautomator viewer. It is also the mouse coordinate click to select, check the element information on the right, and click again to cancel the selection.
Note: The uiautomator viewer is easy to use and quick to start. It does not need to start the session to communicate with the mobile phone. However, the framework used by some apps (such as Jingdong) may not be parsed and obtained, resulting in the following errors.
Note: The inspector is relatively more powerful. Almost all apps can get screenshots and xml source files, but the startup is slow and there are many settings. The most troublesome thing is that every time a use case is run, the inspector needs to be restarted One session.
[2] Element's positioning strategy
There are two basic element positioning strategies:
1. Element-based attributes
2. Based on coordinate values
There are complementary features between these two positioning strategies. It is recommended to use attribute positioning first, and use coordinate positioning when not possible.
Element-based attribute positioning
There are three main ways of positioning based on element attributes:
1. id
2. xpath
3. accessibility_id
id
id is the resource-id attribute value of the corresponding element. Generally, some more important elements have id, and most elements do not:
xpath
xpath is a grammar for locating nodes in a markup language, similar to the path expression for file locating in operating systems
Xpath has two forms of expression:
1. Absolute path
2. Relative path
The original uiautomator viewer cannot display the xpath of the element. You need to modify its jar package or replace it with a third-party jar package to show that the inspector can display xpath, but they are absolute paths, usually very long, as shown below:
/hierarchy/android.widget.FrameLayout/android.view.View/android.widget.FrameLayout[2]/android.widget.GridLayout/android.widget.Button[3]
Note: For xpath positioning, there is a third solution. After learning the xpath syntax, write it yourself; write a relative path expression similar to //android.widget.Button[text()="1"].
accessibility_id
accessibility_id is the content-desc in the element attribute, not every element has this attribute, many elements do not
Note: For the above three positioning methods, id and xpath positioning are recommended first, followed by accessibility_id positioning
Positioning based on coordinate values
In some cases, if the positioning method based on element attributes is not available or inconvenient to use, you can consider using coordinate values to locate the operating element.
First of all, we have to clarify what the coordinate system of the App interface is:
1. The upper left corner is the origin of the screen (0, 0), to the right is the positive direction of the x-axis, and downward is the positive direction of the y-axis
2. The more you go to the right, the greater the value of x, until it reaches the screen boundary 720
3. The lower the y value, the larger the value, until it reaches the screen boundary of 1280
4. The boundary value of the screen depends on the screen resolution
How to view the coordinates of an element:
uiautomator viewer
As shown in the figure above, in the element information, bounds represents the coordinates of the upper left corner and the lower right corner of the element in the screen coordinate system.
inspector
First click the second swipe by coordinates in the menu bar; then move the mouse in the screenshot area, the upper left corner of the screenshot area will display the coordinates of the mouse.
