Python operates the windows desktop to implement mouse and keyboard operations. Detailed explanation of python's pyautogui library documentation

I. Overview

1 Overview

PyAutoGUI is a pure Python GUI automation tool. Its purpose is to automatically control mouse and keyboard operations with programs. It supports multiple platforms (Windows, OS X, Linux).

Source code address:https://github.com/asweigart/pyautogui

Official documentation:https://github.com/asweigart/pyautogui/blob/master/docs/simplified-chinese.ipynb

PyAutoGUI can simulate mouse movement, clicking, dragging, keyboard key input, press and hold operations, and mouse + keyboard hotkey press and hold operations at the same time. It can be said that all hands can be moved.

PyAutoGUI supports Python 2.x and Python 3.x.

2. Installation

# Windows：PyAutoGUI没有任何依赖，因为它用Python的ctypes模块所以不需要pywin32
pip install pyautogui

# OS X：PyAutoGUI需要PyObjC运行AppKit和Quartz模块。这个模块在PyPI上的按住顺序是pyobjc-core和pyobjc
sudo pip3 install pyobjc-core
sudo pip3 install pyobjc
sudo pip3 install pyautogui

# Linux：PyAutoGUI需要python-xlib（Python 2）、python3-Xlib（Python 3）
sudo pip3 install python3-xlib
sudo apt-get scrot
sudo apt-get install python-tk
sudo apt-get install python3-dev
sudo pip3 install pyautogui```

Introduced in python (no additional import code will be posted in subsequent sample codes):

import pyautogui

2. Screen operation

1. Get the screen resolution

# 获取屏幕分辨率（宽高） Size(width=1920, height=1080)
screenWidth, screenHeight = pyautogui.size()

2. Whether a certain coordinate is on the screen

#  (x,y)是否在屏幕上
x, y = 122, 244
pyautogui.onScreen(x, y)

3. Get the current mouse position

# Point(x=1184, y=744)
pyautogui.position()

3. Mouse operation

The screen position uses a Cartesian coordinate system for the X and Y axes. The origin (0,0) is in the upper left corner, increasing to the right and downward respectively.

If the screen pixels are 1920 * 1080, then the coordinates of the lower right corner are (1919, 1079)

1. Move the mouse

The moveTo() function will move the mouse cursor to the specified XY axis coordinates. If a None value is passed in, it means using 当前the object axis coordinate value of the cursor.

pyautogui.moveTo(100, 200)     # 光标移动到(100, 200)位置
pyautogui.moveTo(None, 500)   # 光标移动到(100, 500)位置
pyautogui.moveTo(600, None)   # 光标移动到(600, 500)位置

Generally, the mouse cursor moves to the specified position instantaneously. If you want the mouse to move slower, you can set the duration: the
default duration pyautogui.MINIMUM_DURATIONis 0.1 seconds. If you set the time shorter than the default value, it will instantaneously. implement.

pyautogui.moveTo(100, 200, duration=2)     # 用2秒把光标移动到(100, 200)位置

If you want the cursor to move relative to the current position as the origin, use the pyautogui.moveRel() function. For example:

pyautogui.moveTo(100, 200) #把光标移动到(100, 200)位置
pyautogui.moveRel(0, 50)   #向下移动50
pyautogui.moveRel(30, 0, 2)   #向右移动30
pyautogui.moveRel(30, None)   #向右移动30

#  用num_seconds秒的时间把光标的X轴（水平）坐标移动xOffset，
#  Y轴（竖直）坐标向下移动yOffset。
xOffset, yOffset = 50, 100
pyautogui.moveRel(xOffset, yOffset, duration=num_seconds)

#  用缓动/渐变函数让鼠标2秒后移动(500,500)位置
pyautogui.moveRel(500, 500, duration=2, tween=pyautogui.easeInOutQuad)

2. Click operation

For convenience of operation, PyAutoGUI provides doubleClick(), tripleClick() and rightClick() to implement double-click, triple-click and right-click operations.

# 在鼠标位置单击
pyautogui.click()
# 在鼠标位置双击
pyautogui.doubleClick()

# click()函数就是让鼠标点击，默认是单击左键，参数可以设置：button属性可以设置成left，middle和right
pyautogui.click(x=moveToX, y=moveToY, clicks=num_of_clicks, interval=secs_between_clicks, button='left')

#如果单机之前要先移动，可以把目标的XY坐标值传入函数：
#  先移动到(100, 200)再单击
pyautogui.click(x=100, y=200, duration=2)

# 所有的点击都可以用这个函数，不过下面的函数可读性更好：
pyautogui.rightClick(x=moveToX, y=moveToY)
pyautogui.middleClick(x=moveToX, y=moveToY)
pyautogui.doubleClick(x=moveToX, y=moveToY)
pyautogui.tripleClick(x=moveToX, y=moveToY)

# 可以通过button参数设置left，middle和right三个键。例如：
pyautogui.click(button='right')

# 要做多次单击可以设置clicks参数，还有interval参数可以设置每次单击之间的时间间隔。例如：
#  双击左键
pyautogui.click(clicks=2)
#  两次单击之间停留0.25秒
pyautogui.click(clicks=2, interval=0.25)
#  三击右键
pyautogui.click(button='right', clicks=2, interval=0.25)

The mouseDown() and mouseUp() functions can implement mouse press and mouse release operations. Both have the same parameters, x, y and button. For example:

# 鼠标每个按键按下和松开两个事件可以分开处理：
pyautogui.mouseDown(x=moveToX, y=moveToY, button='left')
pyautogui.mouseUp(x=moveToX, y=moveToY, button='left')
#  鼠标左键按下再松开
pyautogui.mouseDown(); pyautogui.mouseUp() 
#  按下鼠标右键
pyautogui.mouseDown(button='right') 
#  移动到(100, 200)位置，然后松开鼠标右键
pyautogui.mouseUp(button='right', x=100, y=200) 

3. Roller operation

Mouse wheel scrolling can be simulated using the scroll() function and the number of clicks parameters. The number of clicks on different platforms is different. There are also x and y parameters that can be positioned to the (x, y) position before scrolling. For example:

# scroll函数控制鼠标滚轮的滚动，amount_to_scroll参数表示滚动的格数。正数则页面向上滚动，负数则向下滚动：
pyautogui.scroll(clicks=amount_to_scroll, x=moveToX, y=moveToY)
#  向上滚动10格
pyautogui.scroll(10)
#  向下滚动10格
pyautogui.scroll(-10)
#  移动到(100, 100)位置再向上滚动10格
pyautogui.scroll(10, x=100, y=100)

On OS X and Linux platforms, PyAutoGUI can also use hscroll() to achieve horizontal scrolling. For example:

#  向右滚动10格
pyautogui.hscroll(10)
#  向左滚动10格
pyautogui.hscroll(-10)

The scroll() function is a wrapper of vscroll() and performs vertical scrolling.

4. Record cursor applet

# ! python 3
import pyautogui
print('Press Ctrl-C to quit')
try:
    while True:
        x, y = pyautogui.position()
        positionStr = 'X: {} Y: {}'.format(*[str(x).rjust(4) for x in [x, y]])
        print(positionStr, end='')
        print('\b' * len(positionStr), end='', flush=True)
except KeyboardInterrupt:
    print('\n')

5. Mouse drag

PyAutoGUI's dragTo() and dragRel() functions are similar to the moveTo() and moveRel() functions. In addition, they have a button parameter that can be set to the left, middle, and right keys. For example:

#  按住鼠标左键，把鼠标拖拽到(100, 200)位置
pyautogui.dragTo(100, 200, button='left')
#  按住鼠标左键，用2秒钟把鼠标拖拽到(300, 400)位置
pyautogui.dragTo(300, 400, 2, button='left')
#  按住鼠标右键，用2秒钟把鼠标拖拽到(30,0)位置
pyautogui.dragTo(30, 0, 2, button='right')

6. Tween/Easing function

The purpose of the easing/gradient function is to make the cursor movement more dazzling. You can ignore these if you don't need them.

The easing/gradient function can change the speed and direction of the cursor movement. Usually the mouse moves in a straight line at a constant speed, which is the linear easing/gradient function. PyAutoGUI has 30 easing/gradient functions, which can be pyautogui.ease*?viewed through. Among them, pyautogui.easeInQuad()the function can be used for the moveTo(), moveRel(), dragTo() and dragRel() functions. The cursor moves first slowly and then quickly, and the entire process time is still the same as before. The function has the opposite pyautogui.easeOutQuadeffect: the cursor starts moving very fast, and then slowly slows down. pyautogui.easeOutElasticIt's a spring effect that first crosses the end point and then bounces back. For example:

#  开始很慢，不断加速
pyautogui.moveTo(100, 100, 2, pyautogui.easeInQuad)
#  开始很快，不断减速
pyautogui.moveTo(100, 100, 2, pyautogui.easeOutQuad)
#  开始和结束都快，中间比较慢
pyautogui.moveTo(100, 100, 2, pyautogui.easeInOutQuad)
#  一步一徘徊前进
pyautogui.moveTo(100, 100, 2, pyautogui.easeInBounce)
#  徘徊幅度更大，甚至超过起点和终点
pyautogui.moveTo(100, 100, 2, pyautogui.easeInElastic)

99. Protection measures (FAILSAFE)

At that timepyautogui.FAILSAFE = True , if the mouse cursor was in the upper left corner of the screen, the PyAutoGUI function would generate pyautogui.FailSafeExceptionan exception. If it gets out of control and you need to interrupt the PyAutoGUI function, put the mouse cursor in the upper left corner of the screen. To disable this feature, set FAILSAFE to False:

pyautogui.FAILSAFE = False

99. Delay operation (PAUSE)

pyautogui.PAUSEYou can add a delay to all PyAutoGUI functions by setting it to float or int time (seconds). The default delay time is 0.1 seconds. This can make PyAutoGUI run slower when the function loop is executed, which is very useful. For example:

pyautogui.PAUSE = 2.5
pyautogui.moveTo(100,100); pyautogui.click()

All PyAutoGUI functions are blocked until the delay is completed. (Future plans include adding an optional non-blocking mode for calling functions.)

建议PAUSE和FAILSAFE一起使用。

4. Keyboard operation

1. Input operation

The main function of keyboard control is typewrite(). This function can implement character input. To increase the time interval between two inputs, you can use the interval parameter. For example:

#  输入Hello world!
pyautogui.typewrite('Hello world!')
#  每次输入间隔0.25秒，输入Hello world!
pyautogui.typewrite('Hello world!', interval=0.25)

# 多个键也可以：
pyautogui.typewrite(['a', 'b', 'c', 'left', 'backspace', 'enter', 'f1'], interval=secs_between_keys)

2. Key operation

# 按下并抬起esc
pyautogui.press('esc')
# 按下shift（一直按着）
pyautogui.keyDown('shift')
# 连续操作
pyautogui.press(['left', 'left', 'left', 'left', 'left', 'left'])
# 抬起shift
pyautogui.keyUp('shift')
# 组合键 ： ctrl+c
pyautogui.hotkey('ctrl', 'a') # 全选
pyautogui.hotkey('ctrl', 'c') # 复制
pyautogui.hotkey('ctrl', 'v') # 粘贴

3. Button list:

pyautogui.KEYBOARD_KEYS

['\t', '\n', '\r', ' ', '!', '"', '#', '$', '%', '&', "'", '(', ')', '*', '+', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '<', '=', '>', '?', '@', '[', '\\', ']', '^', '_', '`', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '{', '|', '}', '~', 'accept', 'add', 'alt', 'altleft', 'altright', 'apps', 'backspace', 'browserback', 'browserfavorites', 'browserforward', 'browserhome', 'browserrefresh', 'browsersearch', 'browserstop', 'capslock', 'clear', 'convert', 'ctrl', 'ctrlleft', 'ctrlright', 'decimal', 'del', 'delete', 'divide', 'down', 'end', 'enter', 'esc', 'escape', 'execute', 'f1', 'f10', 'f11', 'f12', 'f13', 'f14', 'f15', 'f16', 'f17', 'f18', 'f19', 'f2', 'f20', 'f21', 'f22', 'f23', 'f24', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9', 'final', 'fn', 'hanguel', 'hangul', 'hanja', 'help', 'home', 'insert', 'junja', 'kana', 'kanji', 'launchapp1', 'launchapp2', 'launchmail', 'launchmediaselect', 'left', 'modechange', 'multiply', 'nexttrack', 'nonconvert', 'num0', 'num1', 'num2', 'num3', 'num4', 'num5', 'num6', 'num7', 'num8', 'num9', 'numlock', 'pagedown', 'pageup', 'pause', 'pgdn', 'pgup', 'playpause', 'prevtrack', 'print', 'printscreen', 'prntscrn', 'prtsc', 'prtscr', 'return', 'right', 'scrolllock', 'select', 'separator', 'shift', 'shiftleft', 'shiftright', 'sleep', 'space', 'stop', 'subtract', 'tab', 'up', 'volumedown', 'volumemute', 'volumeup', 'win', 'winleft', 'winright', 'yen', 'command', 'option', 'optionleft', 'optionright']

5. Message pop-up window

If you need a message pop-up, pause the program by clicking OK, or display some information to the user, the message pop-up function will have JavaScript-like functions:

pyautogui.alert('这个消息弹窗是文字+OK按钮')
pyautogui.confirm('这个消息弹窗是文字+OK+Cancel按钮')
# 在prompt()函数中，如果用户什么都不输入，就会返回None。
pyautogui.prompt('这个消息弹窗是让用户输入字符串，单击OK')

1. alert() function

Displays a simple message popup with text and an OK button. Returns the text of the button after the user clicks it.

pyautogui.alert(text='text内容', title='title内容', button='OK')

2. confirm() function

Displays a simple message pop-up window with text, OK and Cancel buttons. After the user clicks, it returns the text of the clicked button. It supports custom lists of numbers and text.

#  OK和Cancel按钮的消息弹窗
pyautogui.confirm(text='', title='', buttons=['OK', 'Cancel'])
#  10个按键0-9的消息弹窗
pyautogui.confirm(text='', title='', buttons=range(10))

3. prompt() function

A message pop-up window that can be entered, with OK and Cancel buttons. The user clicks the OK button to return to the entered text, and clicks the Cancel button to return None.

pyautogui.prompt(text='', title='' , default='')

4. password() function

The style is the same as prompt(), used to enter password, and the message is represented by *. With OK and Cancel buttons. The user clicks the OK button to return the entered text, and clicks the Cancel button to return None.

pyautogui.password(text='', title='', default='', mask='*')

6. Screen capture function

PyAutoGUI uses the Pillow/PIL library to implement image-related recognition and operations.

In Linux, you must execute sudo apt-get install scrotto use the screenshot feature.

1. Image processing

#  返回一个Pillow/PIL的Image对象
pyautogui.screenshot()
pyautogui.screenshot('foo.png')

# 如果你有一个图片文件想在上面做点击操作，你可以用locateOnScreen()函数来定位。
#  返回(最左x坐标，最顶y坐标，宽度，高度) (0, 1040, 48, 40)
pyautogui.locateOnScreen('pyautogui/looks.png')

# locateAllOnScreen()函数会寻找所有相似图片，返回一个生成器： (0, 1040, 48, 40)
for i in pyautogui.locateAllOnScreen('pyautogui/looks.png'):
    print(i)
list(pyautogui.locateAllOnScreen('pyautogui/looks.png')) # [(0, 1040, 48, 40)]

# locateCenterOnScreen()函数会返回图片在屏幕上的中心XY轴坐标值：(24, 1060)
pyautogui.locateCenterOnScreen('pyautogui/looks.png')

If no image is found, None will be returned.

Positioning is relatively slow, usually taking 1 to 2 seconds.