Python-based pyautogui module (detailed summary of mouse and keyboard operations)

Python basic mouse operation

1. GUI control function

The modules used to control the mouse and keyboard are: When pyautoguithis module operates the mouse and keyboard, it is very fast, and if the module controls the mouse, the program is more difficult to close. At this time, we have two methods specifically for the above situations:

1.1 Automatic fail-safe function

pyautogui.FAILSAFE =False  

By default, this function is True. This function means: when the mouse pointer sits on top of the screen, the program will report an error; the purpose is to prevent the program from being unable to stop;

1.2 Pause function

pyautogui.PAUSE = 1    

It means that all pyautogui commands must be paused for one second; other commands will not pause; doing so can prevent keyboard and mouse operations from being too fast;

2. Mouse operation

2.1 Control mouse movement

1. Get the screen resolution

print(pyautogui.size())   # 返回所用显示器的分辨率； 输出：Size(width=1920, height=1080)
width,height = pyautogui.size()
print(width,height)  # 1920 1080

2. Move the mouse

• Move to the specified location

pyautogui.moveTo(100,300,duration=1)   

Move the mouse to the specified coordinates; the function of duration is to set the moving time, all gui functions have this parameter, and they are all optional parameters;

• Move in direction

pyautogui.moveRel(100,500,duration=4)   # 第一个参数是左右移动像素值，第二个是上下，

Move 100px to the right and 500px down. This process lasts for 1 second;

3. Get the mouse position

print(pyautogui.position())   # 得到当前鼠标位置；输出：Point(x=200, y=800)

2.2 Control mouse click

• Mouse click

# 点击鼠标
pyautogui.click(10,10)   # 鼠标点击指定位置，默认左键
pyautogui.click(10,10,button='left')  # 单击左键
pyautogui.click(1000,300,button='right')  # 单击右键
pyautogui.click(1000,300,button='middle')  # 单击中间

• Double click mouse

pyautogui.doubleClick(10,10)  # 指定位置，双击左键
pyautogui.rightClick(10,10)   # 指定位置，双击右键
pyautogui.middleClick(10,10)  # 指定位置，双击中键

• Click & release

pyautogui.mouseDown()   # 鼠标按下
pyautogui.mouseUp()    # 鼠标释放

2.3 Control mouse drag

• Drag to the specified position

pyautogui.dragTo(100,300,duration=1)   

Drag the mouse to the specified coordinates; the function of duration is to set the moving time, all gui functions have this parameter, and they are all optional parameters;

• Drag in direction

pyautogui.dragRel(100,500,duration=4)   # 第一个参数是左右移动像素值，第二个是上下，

Drag 100px to the right and 500px down. This process lasts for 1 second;

2.4 Control mouse scrolling

The function to control the scrolling of the mouse is scroll(), passing in an integer parameter, indicating how many units to scroll up or down; the unit varies according to the operating system;

pyautogui.scroll(300)  # 向下滚动300个单位；

3. Screen processing

3.1 Take screenshots

We control the operation of the mouse and cannot be done blindly, so we need to monitor the content on the screen to decide whether to perform the corresponding operation. pyautogui provides a method screenshot() that can return a Pillow image object;

There are three commonly used functions:

im = pyautogui.screenshot(): Returns the screenshot of the screen, which is a Pillow image object
im.getpixel((500, 500)): returns the color of the pixel at (500, 500) on the im object, which is an RGB tuple
pyautogui.pixelMatchesColor(500,500,(12,120,400)): it is a contrast function, and the comparison is on the screen (500, 500) Is the color of the pixel at this point the same as the given element;

im = pyautogui.screenshot()
im.save('屏幕截图.png')

Save the screenshot;

3.2 Recognizing images

First, we need to get a screenshot. For example, if we want to like, we first save the image of the thumb; then use the function: locateOnScreen('zan.png'), if the image can be found, return the location of the image , Such as: Box(left=25, top=703, width=22, height=22); if the picture is not found, it returns None;
if there are multiple pictures on the screen that can match, you need to use locateAllOnScreen('zan .png'), if multiple values ​​are matched, a list is returned, as follows:

import pyautogui
pyautogui.PAUSE = 1

# 图像识别（一个）
btm = pyautogui.locateOnScreen('zan.png')
print(btm)  # Box(left=1280, top=344, width=22, height=22)

# 图像识别（多个）
btm = pyautogui.locateAllOnScreen('zan.png')
print(list(btm))  # [Box(left=1280, top=344, width=22, height=22), Box(left=25, top=594, width=22, height=22)]

pyautogui.center((left, top, width, height))Return to the center point of the specified position; in this way, we can click with the mouse operation to find the center of the picture; reference example: "4.4 Automatic Like Program"

4. Keyboard input

4.1 Keyboard input function

pyautogui.keyDown(): Simulate key press ;:
pyautogui.keyUp()Simulate key release
pyautogui.press();: # is to call keyDown() & keyUp() to simulate a key press;: The
pyautogui.typewrite('this',0.5)first parameter is the input content, and the second parameter is the interval time between each character;
pyautogui.typewrite(['T','h','i','s']):typewrite You can also pass in a single letter list;

For example:

pyautogui.keyDown('shift')    # 按下shift
pyautogui.press('4')    # 按下 4
pyautogui.keyUp('shift')   # 释放 shift

Output: $;

pyautogui.typewrite('$$$$', 0.5)

Slow output: $$$

4.2 Keyboard special keys

Sometimes we need to enter some special keys, such as the left arrow, which are represented by corresponding keyboard strings, for example:

pyautogui.typewrite(['T','i','s','left','left','h',])   # 输出：This

Explanation: The left here is the arrow to the left; there are many other keyboard strings, please refer to the following table:

Keyboard string	Description
enter (or return or \n)	Carriage return
esc	ESC
shiftleft, shiftright	Left and right SHIFT keys
altleft, altright	Left and right ALT keys
ctrlleft, ctrlright	Left and right CTRL keys
tab (\t)	TAB键
backspace, delete	BACKSPACE 、DELETE键
pageup, pagedown	PAGE UP 和 PAGE DOWN键
home, end	HOME and END keys
up, down, left,right	Arrow keys
f1, f2, f3 ... f12	F1…….F12 key
volumemute, volumedown,volumeup	The sound becomes louder and quieter (some keyboards do not)
pause	PAUSE key, pause key
capslock	CAPS LOCK key
numlock	NUM LOCK key
scrolllock	SCROLLLOCK key
insert	INSERT key
printscreen	PRINT SCREEN key
winleft, winright	Win 键 (windows)
command	command key (Mac OS X)
option	option（Mac OS X）

4.3 Shortcut keys

If we need to simulate the copy shortcut key ctrl+ c, if we use the previous method, the code is:

pyautogui.keyDown('ctrl')
pyautogui.keyDown('c')
pyautogui.keyUp('c')
pyautogui.keyUp('ctrl')

The order of pressing and releasing the shortcut keys is very important. At this time, we can use pyautogui.hotkey(). This function can accept multiple parameters, press them in the order in which they are passed in, and release them in the reverse order. The above shortcut key ctrl+ ccan change the code to:

pyautogui.hotkey('ctrl','c')

5. Prompt message box

5.1 Prompt Box/Warning Box

import pyautogui
a = pyautogui.alert(text='This is an alert box.', title='Test')
print(a)

The output is as follows: Click OK, the return value is'OK'

5.2 Selection box

import pyautogui
a = pyautogui.confirm('选择一项', buttons=['A', 'B', 'C'])
print(a)

The output is as follows: Click the B option, the return value is'B'

5.3 Password input

import pyautogui

a = pyautogui.password('Enter password (text will be hidden)')
print(a)

The output is as follows: input the password, it will be displayed as cipher text, click OK, the return value is the value just entered

5.4 Normal input

import pyautogui

a = pyautogui.prompt('请输入一个数字：')
print(a)

The output is as follows: display as plain text, click OK, the return value is the value just entered;

6. Examples

6.1 Mouse control mouse to draw a square

for i in range(2):   # 画正方形
    pyautogui.moveTo(200,200,duration=1)
    pyautogui.moveTo(200,400,duration=1)
    pyautogui.moveTo(400,400,duration=0.5)
    pyautogui.moveTo(400,200,duration=2)
	

6.2 Get the real-time position of the mouse

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# author：Zhang Kai time:

import pyautogui
import time

try:
    while True:
        x,y = pyautogui.position()
        posi = 'x:' + str(x).rjust(4) + ' y:' + str(y).rjust(4)
        print('\r',posi,end='')
        time.sleep(0.5)

except KeyboardInterrupt:
    print('已退出！')

display effect:

6.3 Get the color of the mouse position and the position

import pyautogui
import time

try:
    while True:
        x,y = pyautogui.position()
        rgb = pyautogui.screenshot().getpixel((x,y))
        posi = 'x:' + str(x).rjust(4) + ' y:' + str(y).rjust(4) + '  RGB:' + str(rgb)
        print('\r',posi,end='')
        time.sleep(0.5)

except KeyboardInterrupt:
    print('已退出！')

display effect:

6.4 Automatic Like Program

We need to like all articles. After the likes on this page are finished, scroll the mouse to like all the newly loaded articles;

code show as below:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# author：Zhang Kai 

import pyautogui
import time


def zan():
    time.sleep(0.5)    # 等待 0.5 秒
    left, top, width, height = pyautogui.locateOnScreen('zan.png')   # 寻找 点赞图片；
    center = pyautogui.center((left, top, width, height))    # 寻找 图片的中心
    pyautogui.click(center)    # 点击
    print('点赞成功！')


while True:
    if pyautogui.locateOnScreen('zan.png'):
        zan()   # 调用点赞函数
    else:
        pyautogui.scroll(-500)    # 本页没有图片后，滚动鼠标；
        print('没有找到目标，屏幕下滚~')

After running, they will be

liked one by one: The input in Pycharm is as follows: