Dear readers and bloggers:
Hello everyone, I am Hanzi. Today we need to control the screen mouse with gestures using cv2, mediapipe and pyautogui.
Table of contents
You can read the previous article first.
1. Preparation
First pip all packages:
pip install opencv-python
pip install mediapipe
pip install pyautoguiIf there is a problem with pip, please go to this article to learn:
https://blog.csdn.net/B20111003/article/details/125536325?spm=1001.2014.3001.5502If you haven't read the previous article, please go to the column interface to select and read it.
Tips：
If you are using a mac , you need to set in the settings to allow python to control the computer
Windows do not need oh. If you are using a desktop or an old computer without a camera, please prepare an external camera in advance.
Two, the code
Because the content is too difficult, so I only give the code...
Ideas:
1. Import library
2. Let the program know what is a hand (hands)
3. Write a function that allows the program to judge the finger (index finger)
4. Turn on the camera
5. Display the camera and judge and control it
6. Loop and check
7. If you exit, close the window and stop the process
import mediapipe as mp
import pyautogui as pa
import cv2
hands = mp.solutions.hands.Hands(min_detection_confidence=0.7,
min_tracking_confidence=0.7)
def isIndexFinger():
fingerDict = {}
for idx in range(21):
fingerDict[idx] = hand_landmarks.landmark[idx].y
fingerOrder = sorted(fingerDict, key = fingerDict.__getitem__)
if fingerOrder[:3] == [8, 7, 6]:
return True
else:
return False
controlFlag = 0
cam = cv2.VideoCapture(0)#打开摄像头
ret,frame = cam.read()#读取第一帧图像
print("Runing Program...")
while ret:#当图像读取成功的时候
img = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) # 图像颜色模式转换
width, height = img.shape[1], img.shape[0]
result = hands.process(img) # 用手部模型处理图片
if result.multi_hand_landmarks: # 两只手是否存在
for hand_landmarks in result.multi_hand_landmarks:
for i in range(21):
x = int(hand_landmarks.landmark[i].x * width)
y = int(hand_landmarks.landmark[i].y * height)
cv2.circle(img,(x,y),5,(0,255,0),-1)
x = int(hand_landmarks.landmark[8].x * width)
y = int(hand_landmarks.landmark[8].y * height)
if (hand_landmarks.landmark[8].y * height < hand_landmarks.landmark[12].y * height and hand_landmarks.landmark[16].y * height):
if controlFlag == 0:
controlFlag = 1
x_p, y_p = x, y
x_n, y_n = x, y
else:
x_p, y_p = x_n, y_n
x_n, y_n = x, y
pa.moveRel(x_p - x_n, y_n - y_p)
else:
pass
img = cv2.cvtColor(img,cv2.COLOR_RGB2BGR)
cv2.imshow("camera",img)
cv2.waitKey(10)
ret, frame = cam.read()#读取下一帧图像
cam.release()
cv2.destroyAllWindows()Finally, please pay attention to everyone, thank you!
Well, that's all for today's article, see you in the next issue!
Hanzi
2023/2/19