1. Introduction to pywinauto

The official pywinauto document link is as follows.

Dialog is a window that contains several other GUI elements/controls, such as buttons, edit boxes, etc. The dialog is not necessarily the main window. The message box at the top of the main form is also a dialog box. The main window is also regarded as a dialog box by pywinauto.
A control is a GUI element at any level of the hierarchy. The definition includes windows, buttons, edit boxes, tables, table cells, toolbars, etc.
Win32 API technology (the "win32" backend in pywinauto) provides an identifier for each control. This is a unique integer called handle .
The UI Automation API (the "uia" backend in pywinauto) may not provide a window handle for every GUI element . No such element can be seen in the "win32" backend. But the Inspect.exeattribute can be displayed NativeWindowHandleif it is available.

2. Access Technology Selection

To operate an application, pywinauto first needs to access the application. There are two main access technologies. WIN32 access technology supports applications developed by MFC, VB6, VCL, and simple WinForms controls. MS UI Automation access technology supports applications developed by WinForms, WPS, QT5, WPF, Store apps, browsers, etc.

To analyze which access technology an application software can use, there are the following common tools:

1. SPY++ , used for WIN32 API. When SPY++ can display all the controls, the access technology should select "win32"

2. Inspect.exe : If the mode of Inspect.exe is set to UIA mode, which can display more controls than SPY++, the access technology should select "uia"

3. py_inspect : supports two access technologies, win32 and uia, and is an alternative to SWAPY.

https://github.com/pywinauto/py_inspect

4. UISPY: Support uia access technology

5. SWAPY: only supports win32 access technology. https://github.com/pywinauto/SWAPY

The figure below is the secureCRT control viewed with py_inspect. You can see that if you select uia format, you can see more controls, so the access technology should select'uia'

Other commonly used

Three, access application method

You can open an application or connect to an existing application instance. All are done through the Application object. It is not a clone of subprocess.Popen, but an access point for application automation. Here is mainly to limit the scope of the automated control process. If a program has multiple instances, one instance is automatically controlled, and other instances (processes) are not affected. There are two main types of objects that can establish this kind of entry point Application() and Desktop(). The scope of Application is a process, such as general desktop applications. The scope of Desktop can span processes. It is mainly used for programs that contain multiple processes like the calculator of win10. This is currently relatively rare.

1. Application object

1.1, open an application

connect(self, **kwargs)  # instance method:

The timeout parameter is optional, and only needs to be used when the application takes a long time to start.

from pywinauto.application import Application
app = Application(backend="uia").start('notepad.exe')
# describe the window inside Notepad.exe process
dlg_spec = app.UntitledNotepad
# wait till the window is really open
actionable_dlg = dlg_spec.wait('visible')

1.2, connect an application

connect(self, **kwargs)  # instance method:

connect() Used when you want to start an automated application. To specify an application that is already running, you need to specify one of the following:

process: the process ID of the application, for example, app = Application().connect(process=2341)
handle: the window handle of the application window, for example, app = Application().connect(handle=0x010f0c)
path: the executable of the process The path of the file (GetModuleFileNameEx is used to find the path of each process and compare it with the value passed in), for example: app = Application().connect(path=r"c:\windows\system32\notepad.exe")

Or any combination of the parameters of the specified window, these are all passed to the pywinauto.findwindows.find_elements() function. E.g

app = Application().connect(title_re=".*Notepad", class_name="Notepad")

2. Desktop object

from subprocess import Popen
from pywinauto import Desktop
Popen('calc.exe', shell=True)
dlg = Desktop(backend="uia").Calculator
dlg.wait('visible')

Four, access window method

This is the core concept of the advanced pywinauto API. You can describe any window or control in approximate or more detail, even if it does not yet exist or has been closed. The window specification also retains information about the matching/search algorithm that will be used to obtain the real window or control.

A detailed window specification is as follows:

>>> dlg_spec = app.window(title='Untitled - Notepad')
>>> dlg_spec
<pywinauto.application.WindowSpecification object at 0x0568B790>
>>> dlg_spec.wrapper_object()
<pywinauto.controls.win32_controls.DialogWrapper object at 0x05639B70>

The actual window search is wrapper_object()performed by the method. It returns some wrapper or trigger of the actual existing window/control ElementNotFoundError. This wrapper can handle windows/controls by sending actions or retrieving data.

But Python can hide this wrapper_object()call, so you can have more compact code in the program. The following descriptions are exactly the same:

dlg_spec.wrapper_object().minimize() # while debugging
dlg_spec.minimize() # in production

There are many possible standards for creating window specifications. The following are just a few examples. pywinauto.findwindows.find_elements()A list of possible criteria can be found in the function.

# 可以是多层次的
app.window(title_re='.* - Notepad$').window(class_name='Edit')

# 可以结合标准
dlg = Desktop(backend="uia").Calculator
dlg.window(auto_id='num8Button', control_type='Button')

1.通过窗体名确定窗体

dlg = app.Notepad
dlg = app['Notepad']

2、top_window()

This will return the window with the highest Z order of the top-level window of the application.

3. Specify a window through a multi-level description, such as (a), or use a combination of parameters to specify a window such as (b), the following two statements determine the same window

dlg = app.window(title_re="Page Setup").window(class_name="#32770")
dlg = app.window(title_re="Page Setup", class_name="#32770")

4. Through findwindows

pywinauto.findwindows.find_windows()

pywinauto.findwindows.find_elements() returns win32_element_info.HwndElementInfo of all running programs

5、dialogs = app.windows()

This will return a list of all visible, enabled top-level windows of the application. Then, you can use handlepropssome of the methods in the module to select the desired dialog box. Once you have the required handle, you can use

app.window(handle=win)

Note : If the title of the dialog is very long-then the attribute access entered may be very long, in which case it is usually easier to use

app.window(title_re=".*Part of Title.*")

Five, access controls

1. Attribute analysis magic

Python simplifies the specification of creating windows by dynamically parsing object attributes. But an attribute name has the same restrictions as any variable name: no spaces, commas, or other special symbols. But fortunately pywinauto uses the "best match" algorithm to find spelling errors and small changes.

app.UntitledNotepad
# 相当于
app.window(best_match='UntitledNotepad')

Access through a dictionary-like item, you can use Unicode characters and special symbols.

app['Untitled - Notepad']
# 是相同的
app.window(best_match='Untitled - Notepad')

2. How to know the name of the magic attribute

There are several principles for how to attach the "best match" to the control. If the window specification is close to one of the names, you will get a successful name match.

By title (window text, name): app.Properties.OK.click()
By title and control type: app.Properties.OKButton.click()
By control type and number: app.Properties.Button3.click() ( Note : Button0 and Button1 match the same button, Button2 is the next one, etc.)
Press the upper left corner label and control type: app.OpenDialog.FileNameEdit.set_text("")
By control type and item text:app.Properties.TabControlSharing.select("General")

Usually not all of these matching names can be used at the same time. To check these names of the specified dialog box, you can use the print_control_identifiers()method. The possible "best_match" names are displayed as a Python list for each control in the tree. You can also copy more detailed window specifications from the method output. For example app.Properties.child_window(title="Contains:", auto_id="13087", control_type="Edit").

3. How to specify controls on the dialog box

There are many ways to specify controls, the easiest way is

app.dlg.control
app['dlg']['control']

The second one is more suitable for non-English operating systems, you need to pass unicode strings, for example app [u'对话框标题'] [u'控件标题']

The code builds multiple identifiers for each control based on the following:

title

friendly class

title + friendly class

If the title text of the control is empty (after removing non-char characters), this text is not used. Instead, we look for the closest title text above and to the right of the control. And add friendly classes. So the list becomes

friendly class

closest text + friendly class

Once a set of identifiers have been created for all the controls in the dialog box, we will disambiguate them.

Use WindowSpecification.print_control_identifiers() method

Note that the identifier printed by this method has undergone the process of making the identifier unique. Therefore, if you have two edit boxes, "Edit" will be listed in their identifiers. Actually, although the first one can be called "Edit", "Edit0", "Edit1" and the second one should be called "Edit2"

Six, operation controls

https://pywinauto.readthedocs.io/en/latest/code/code.html#controls-reference

The specific operation method refers to the official introduction, pay attention to two points

1. For the same control, the access path found with win32 is different from the access path queried with uia

2. For the same control, the methods available for win32 access are different from those available for uia access.

Seven, other

1. How to use pywinauto with application languages other than English

Since Python does not support unicode identifiers in the code, you cannot use attribute access to refer to controls, so you must use item access or window()make explicit calls to it.
So don't write

app.dialog_ident.control_ident.click()

You must write

app['dialog_ident']['control_ident'].click()

Or use explicitlywindow()

app.window(title_re="非Ascii字符").window(title="非Ascii字符").click()

2. How to deal with controls that do not respond as expected (such as OwnerDraw controls)

Some controls (especially Ownerdrawn controls) will not respond to events as expected. For example, if you view any HLP file and go to the "Index" tab (click the "Search" button), you will see a list box. Running Spy or Winspector here will show you that it is indeed a list box-but it is ownerdrawn. This means that the developers tell Windows that they will override how the project is displayed and do it themselves. In this case, they have made it impossible to retrieve the string :-(.

What's the problem?

app.HelpTopics.ListBox.texts()                # 1
app.HelpTopics.ListBox.select("ItemInList")   # 2

Will return an empty list of strings, all this means that pywinauto cannot get the strings in the list box
This will fail with IndexError because ListBox's select(string) method looks for items in the text to know the index of the item it should select.

The following solutions will apply to this control

app.HelpTopics.ListBox.select(1)

This will select the second item in the list box, because it is not a string search, and it works fine.

Unfortunately, even this will not help. Developers can make the control not respond to standard events such as Select. In this case, the only way you can select items in the list box is to use the keyboard emulation of TypeKeys().

This allows you to send any keystroke to the control. So choose the third item you want to use

app.Helptopics.ListBox1.type_keys("{HOME}{DOWN 2}{ENTER}")

{HOME} Will ensure that the first item is highlighted.
{DOWN 2} Then two items will be highlighted
{ENTER} The highlighted item will be selected

If your application makes extensive use of similar control types, you can use it more easily by deriving a new class from ListBox, which can use additional knowledge about your specific application. For example, in the WinHelp example, every time an item is highlighted in the list view, its text is inserted into the Edit control above the list, and you can get the item's text from there, for example:

# 打印列表框中当前所选项目的文本
# （只要你没有输入编辑控件！）
print app.HelpTopics.Edit.texts()[1]

Introduction to pywinauto