1 Introduction
In a recent project, it was necessary to record keyboard and mouse commands, and then play back the recording process through injection. Generally speaking, some ordinary click commands can be completed through some python packages to complete keyboard and mouse simulation operations, such as through the pyautogui package. However, during the mouse movement, if the frequency is very high, such as sampling 50 point pixel coordinates in 1 second, then you will find that if the playback is performed through python, the entire playback process, compared to the original recording process, It's like slow playback.
The whole process is abstracted as follows:
During the recording process, a sample is taken every time T (the coordinates of the point are recorded). Since the sampling itself consumes very little time (less than 0.1ms), it can be ignored. Therefore, if n samples are sampled during the recording process point, the time required is: nT
During the playback process, you need to move to each sampling point p1, p2..., because moving to each point during the playback process is achieved through the python function moveTo, and the function moveTo is executed once The time is t (about 100ms, which cannot be ignored), so the time consumed by the playback process is: nT + nt.
The time ratio between the playback process and the recording process: (nT + nt)/nT = 1 + t/T, so the playback process is equivalent to 1 + t/T times slower than the recording process. And the higher the sampling frequency, that is, the smaller T, the greater the proportion of slow playback!
In order to make the playback process closer to the recording process, that is, to reduce the ratio of slow playback, the execution time of moveTo needs to be scaled. Since it takes a long time to implement moveTo through python, moveTo can be implemented through c/c++, which can greatly shorten the time required for moveTo.
2. Generate dll
In order to call dll in python, you need to generate the dll file of moveTo first. The corresponding source code is as follows:
// mouseMove.h
#ifdef MYDLL_EXPORTS
#define MATH_API __declspec(dllexport)
#else
#define MATH_API __declspec(dllimport)
#endif
#include <windows.h>
extern "C" {
MATH_API void mouseClick(int x, int y);
MATH_API void LeftDown(int x, int y);
MATH_API void LeftUp(int x, int y);
MATH_API void mouseMove(int x, int y); // 对应moveTo函数
}
// mouseMove.cpp
void mouseClick(int x, int y) {
INPUT input = {
0 };
input.type = INPUT_MOUSE;
input.mi.dwFlags = MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_MOVE | MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP;
// 计算屏幕上的坐标位置
input.mi.dx = x * 65535 / GetSystemMetrics(SM_CXSCREEN);
input.mi.dy = y * 65535 / GetSystemMetrics(SM_CYSCREEN);
SendInput(1, &input, sizeof(INPUT));
}
void LeftDown(int x, int y) {
INPUT input = {
0 };
input.type = INPUT_MOUSE;
input.mi.dwFlags = MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_MOVE | MOUSEEVENTF_LEFTDOWN;
// 计算屏幕上的坐标位置
input.mi.dx = x * 65535 / GetSystemMetrics(SM_CXSCREEN);
input.mi.dy = y * 65535 / GetSystemMetrics(SM_CYSCREEN);
SendInput(1, &input, sizeof(INPUT));
}
void LeftUp(int x, int y) {
INPUT input = {
0 };
input.type = INPUT_MOUSE;
input.mi.dwFlags = MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_MOVE | MOUSEEVENTF_LEFTUP;
// 计算屏幕上的坐标位置
input.mi.dx = x * 65535 / GetSystemMetrics(SM_CXSCREEN);
input.mi.dy = y * 65535 / GetSystemMetrics(SM_CYSCREEN);
SendInput(1, &input, sizeof(INPUT));
}
void mouseMove(int x, int y) {
INPUT input = {
0 };
input.type = INPUT_MOUSE;
input.mi.dwFlags = MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_MOVE;
// 计算屏幕上的坐标位置
input.mi.dx = x * 65535 / GetSystemMetrics(SM_CXSCREEN);
input.mi.dy = y * 65535 / GetSystemMetrics(SM_CYSCREEN);
SendInput(1, &input, sizeof(INPUT));
}
Note: Not only mouseMove (corresponding to moveTo) is implemented in the dll file, but mouseClick and other operations are also implemented incidentally.
3. Call dll through pyton
To call a dll file in python, you only need to import the ctypes package first, then load the dll file through the ctypes package, and then call the function in the dll.
import ctypes
import pyautogui
# 加载dll文件
lib = ctypes.cdll.LoadLibrary("E:\\program\\cplusplus\\myDll\\x64\\Release\\myDll.dll")
# 调用dll中的函数
lib.mouseMove(100,200)
4. Comparison of running speed between python library and dll library
We all know that python is certainly not as fast as c/c++, but how much is the difference in running speed? If we were just talking about the function of moving the mouse mentioned in this question, it might be scary to say it! ! !
No picture to tell. . . . ,
import time
import ctypes
import pyautogui
lib = ctypes.cdll.LoadLibrary("E:\\program\\cplusplus\\myDll\\x64\\Release\\myDll.dll")
t = time.time()
# dll执行1000次mouseMove
for i in range(0,1000):
lib.mouseMove(i,2)
t1 = time.time()
print("mouseMove 1000 times by dll: ",t1 - t)
# 通过pyautogui执行100次moveTo
t = time.time()
for i in range(0,100):
pyautogui.moveTo(i,300)
t1 = time.time()
print("moveTo 100 times by pyautogui:",t1 - t)
As can be seen from the figure, the time consumed by executing 1000 mouseMoves through dll is 0.11s, and the average time is about 0.1ms. The
time consumed by executing 100 mouseTo through pyautogui is 10s. The average consumption time of 100ms
is actually a difference of 1000 times? ? ?
No wonder the recording process becomes slower when converted to python playback~
--------------------- Update ---------- -----------------------
It has always been hard to believe that the speeds of python and c++ dlls are so different, but in fact there is not such a big gap. , it should essentially be that pyautogui also calls Windows API related dlls. So what's the problem?
Studying the pyautogui source code found that the moveTo interface of pyautogui has a decorator, and most of the time is spent on the decorator.
If the decorator is removed, it will be equivalent to the speed of calling dll.
Reference articles:
[1] python calls the c++ dll library
[2] C++ simulates mouse movement and click implementation code
[3] [python] python calls the C++ program