[Python • Project Combat] pytesseract+pyqt realizes small project of picture recognition software - (2) realizes QQ screenshot function

This article assumes that you have already studied the previous article . If you haven't studied it yet, go and learn it quickly. Follow bloggers to learn more.


foreword

After the last study, we installed the tesseract recognition engine, and quickly recognized the content of the picture through pytesseract. Then PyQt5DesignModewe created our project through the project template, and the interface of the software has been drawn, and a simple click event has been added for the screenshot button.

In this article, we will continue to improve this project and realize the function of QQ screenshot.


1. Task purpose

On the basis of the previous section, realize the function of QQ screenshot, save the picture and display it on the software interface.

Require

  • You can use shortcut keys to take screenshots
  • You can click the screenshot button to take a screenshot
  • Screenshots are displayed on the software

2. Implement screenshot function

1. Screenshot function analysis

Theoretical basis

1. Screenshot function

Qt provides a way to take screenshots - grabWindow, the function prototype

def grabWindow(WID window, x=0, y=0, width=-1, height=-1)

The arguments are 窗口IDand 要截取的区域(a rectangular area of ​​x, y, width, and height).
窗口IDIt can be obtained through winId() of QWidget. If the window ID of the entire screen is intercepted, pass in 0.

2. Support mouse move event

Qt's QWidgets provides a method that enables the window to support mouse movement events - mouseTracking, indicating whether the mouse tracking property of the window is in effect. function prototype

def setMouseTracking(self, bool)

The parameter has only one booltype of parameter, indicating whether to enable mouse tracking events.

3. Window without borders

Qt's QWidgets provides a method that can make the window borderless - setWindowFlagthe Flag of the window needs to be passed in, that is, Qt_WindowTypeto enable windows of different styles. function prototype

def setWindowFlag(self, Qt_WindowType, on=True)

The main parameter we use here is Qt_WindowTypeto indicate the style of the window, and its options are as follows

  • Qt::Widget: The default value of the QWidget constructor. If the new widget has no parent widget, it is an independent window, otherwise it is a child widget.
  • Qt::Window: Whether or not there is a parent widget, the new widget is a window, usually with a window border and a title bar.
  • Qt::Dialog : The new widget is a dialog
  • Qt::Sheet: The new widget is a Macintosh form.
  • Qt::Drawer: The new widget is a Macintosh drawer.
  • Qt::Popup: The new widget is a pop-up top-level window.
  • Qt::Tool: The new widget is a tool window, which is usually a small window used to display tool buttons.
    If a tool window has a parent widget, it will be displayed on top of the parent widget, otherwise, it will be equivalent to using
  • Qt::WindowStaysOnTopHintexhibit.
  • Qt::Tooltip: The new widget is a prompt window without title bar and window border.
  • Qt::SplashScreen: The new widget is a welcome window, which is the default value of the QSplashScreen constructor.
  • Qt::Desktop: The new widget is the desktop, which is the default for the QDesktopWidget constructor.
  • Qt::SubWindow: The new widget is a child window, regardless of whether the widget has a parent widget.
  • Qt::X11BypassWindowManagerHint: Completely ignore the window manager. Its function is to generate a window without window frame that is not managed at all. At this time, the user cannot use the keyboard to input unless the QWidget::ActivateWindow() function is called manually.
  • Qt::FramelessWindowHint: Produces a window without window borders, at this time the user cannot move the window and change its size.
  • Qt::CustomizeWindowHint: Turn off the default window title hint.

4. Window full screen

Qt's QWidgets provides a method that can make the window full screen display - setWindowState, which means setting the state of the window. function prototype

def setWindowState(self, Union, Qt_WindowStates=None, Qt_WindowState=None):

When we use it, we mainly use it Qt_WindowState, and its options are as follows

  • Qt: :WindowNoStatenormal status
  • Qt: :WindowMinimizedwindow minimized
  • Qt:: WindowMaximizedmaximize window
  • Qt::WindowFullScreenThe window fills the entire screen and has no borders
  • Qt:: Window ActiveA window that becomes active, e.g. can receive keyboard input

Implementation ideas

Inherit the QWidgets class to implement a borderless full-screen window, covering the entire screen, and giving a black transparent background.

Handle the following events as shown in the figure below

  • Press the left mouse button to record the start coordinates
  • Press the right mouse button to cancel the operation
  • Mouse movement, record mouse coordinates
  • Release the left mouse button and record the end coordinates of the mouse movement
  • Double-click the mouse to save the screenshot content
  • Drawing event, draw the box dragged by the mouse in real time

The logic here should be something like this. When the left mouse button is pressed, the coordinates of the clicked position of the mouse start to be recorded. When the mouse moves, a new coordinate is updated. The new coordinate and the previous coordinate can calculate the x, y, width, and height of the rectangular area in real time. When the left mouse button is released, the coordinates of this point will be recorded and calculated with the first starting coordinates to obtain the coordinates of the screenshot area.

The mouse movement event is always running, because we want to preview the effect of our screenshots in real time, so we need to always get the position of the mouse. At the same time, the drawing event also needs to run all the time. It needs to calculate the rectangular area we preview in real time and draw it on the interface.

The event of the right mouse button is very simple, which is to cancel the current operation. Its logic is that if an area is currently selected, then cancel this area, and if no area is selected, then close the window. The final effect is as follows.

2. Realization of screenshot function

The following code ideas come from CSDN users @Karbob, thank you for your ideas.

1. Create a window

Create a window that inherits QWidget, and we set the window basicly. init_windowThe method is what we use to initialize the window, we enable the mouse tracking function, set the mouse cursor, set the window borderless and window full screen.

class CaptureScreen(QWidget):

    def __init__(self):
        super(QWidget, self).__init__()
        self.init_window()  # 初始化窗口
        self.capture_full_screen()  # 获取全屏

    def init_window(self):
        self.setMouseTracking(True)  # 鼠标追踪
        self.setCursor(Qt.CrossCursor)  # 设置光标
        self.setWindowFlag(Qt.FramelessWindowHint)  # 窗口无边框
        self.setWindowState(Qt.WindowFullScreen)  # 窗口全屏


It is worth mentioning that we took a screenshot of the full screen of the desktop when the window was initialized, and saved it in the field of the window, which will be used when we draw the screenshot later.

self.capture_full_screen()  # 获取全屏

His implementation is to call grabWindowthe method to take a screenshot of the desktop. The implementation code is as follows,

    def capture_full_screen(self):
        self.full_screen_image = QGuiApplication.primaryScreen().grabWindow(QApplication.desktop().winId())

There are also some variables that need to be initialized

    begin_position = None
    end_position = None
    full_screen_image = None
    capture_image = None
    is_mouse_pressLeft = None
    painter = QPainter()
  • begin_positionRepresents the starting coordinates of the mouse
  • end_positionRepresents the end coordinates of the mouse
  • full_screen_imageImage to store full screen screenshots
  • capture_imageCaptured picture
  • is_mouse_pressLeftWhether the left mouse button is pressed
  • painterthe object to draw

2. Implement the mouse press event

The mouse down event mainly does two things,

  1. Determine whether the left mouse button or the right mouse button is pressed
  2. If it is the left mouse button, record the starting position of the mouse, otherwise cancel the current operation

The event handler flowchart is shown below,

Created with Raphaël 2.3.0 开始 是否左键? 记录当前坐标 设置属性is_mouse_pressLeft为True 结束 处理右键 是否选择截图区域? 取消截图区域 取消当前操作 yes no yes no

The implementation code is as follows,

    def mousePressEvent(self, event):
        if event.button() == Qt.LeftButton:
            self.begin_position = event.pos()
            self.is_mouse_pressLeft = True
        if event.button() == Qt.RightButton:
            # 如果选取了图片,则按一次右键开始重新截图
            if self.capture_image is not None:
                self.capture_image = None
                self.paint_background_image()
                self.update()
            else:
                self.close()

3. Implement the mouse movement event

The main job of the mouse movement event is to obtain the real-time mouse coordinates, and then call the update method of widgets to update the interface. The code here is as follows

    def mouseMoveEvent(self, event):
        if self.is_mouse_pressLeft is True:
            self.end_position = event.pos()
            self.update()

4. Implement the mouse release event

The main job of the mouse release event is to record the coordinates when the mouse is released, and then is_mouse_pressLeftset it to false to indicate that the mouse is no longer pressed. The code is as follows

    def mouseReleaseEvent(self, event):
        self.end_position = event.pos()
        self.is_mouse_pressLeft = False

5. Implement the drawing event

The drawing event actually executes two things. One is to set the background picture as a screenshot of the desktop and darken the color. The other is to place the rectangular picture without darkening color in the area selected by the mouse.

    def paintEvent(self, event):
        self.painter.begin(self)  # 开始重绘
        self.paint_background_image()
        pen_color = QColor(30, 144, 245)  # 画笔颜色
        self.painter.setPen(QPen(pen_color, 1, Qt.SolidLine, Qt.RoundCap))  # 设置画笔,蓝色,1px大小,实线,圆形笔帽
        if self.is_mouse_pressLeft is True:
            pick_rect = self.get_rectangle(self.begin_position, self.end_position)  # 获得要截图的矩形框
            self.capture_image = self.full_screen_image.copy(pick_rect)  # 捕获截图矩形框内的图片
            self.painter.drawPixmap(pick_rect.topLeft(), self.capture_image)  # 填充截图的图片
            self.painter.drawRect(pick_rect)  # 画矩形边框
        self.painter.end()  # 结束重绘

In the code, self.paint_background_image()it is the method of drawing a gray-black background. His implementation code is as follows

    def paint_background_image(self):
        shadow_color = QColor(0, 0, 0, 100)  # 黑色半透明
        self.painter.drawPixmap(0, 0, self.full_screen_image)
        self.painter.fillRect(self.full_screen_image.rect(), shadow_color)  # 填充矩形阴影

self.get_rectangle(self.begin_position, self.end_position)Calculate the mouse frame selection area rectangle, the implementation code is as follows,

    def get_rectangle(self, begin_point, end_point):
        pick_rect_width = int(qAbs(begin_point.x() - end_point.x()))
        pick_rect_height = int(qAbs(begin_point.y() - end_point.y()))
        pick_rect_top = begin_point.x() if begin_point.x() < end_point.x() else end_point.x()
        pick_rect_left = begin_point.y() if begin_point.y() < end_point.y() else end_point.y()
        pick_rect = QRect(pick_rect_top, pick_rect_left, pick_rect_width, pick_rect_height)
        # 避免高度宽度为0时候报错
        if pick_rect_width == 0:
            pick_rect.setWidth(2)
        if pick_rect_height == 0:
            pick_rect.setHeight(2)

        return pick_rect

Calculating the rectangle is actually very simple. His idea is to record the start coordinates and end coordinates, so that the width and height of the rectangle can be calculated through the end coordinates, so the following two formulas are obtained

H ( z ) = y 1 − y , W ( z ) = x 1 − x H(z) = y1-y, W(z) = x1-xH(z)=y 1y,W(z)=x 1x


There are two more important points here, which are to intercept the graphics of the mouse rectangle area, calculate the mouse frame selection area through the above method, and then copy the original picture of the mouse frame selection area, which we use to full_screen_image.copyachieve

pick_rect = self.get_rectangle(self.begin_position, self.end_position)  # 获得要截图的矩形框
self.capture_image = self.full_screen_image.copy(pick_rect)  # 捕获截图矩形框内的图片

The actual implementation process here is as follows

Through the start coordinates, and the w and h of the rectangle, we can determine which area we want, and pass it in full_screen_image.copyto get a screenshot of the corresponding area. QRect is such a data type, so we only need Just pass in data of type QRect.

6. Mouse double-click event

In the mouse double-click event, we call the function of saving the picture, and then close the window.

This is related to the linkage between the following window programs, so it should be read in conjunction with the follow-up content

Its implementation code is as follows

    def mouseDoubleClickEvent(self, event):
        if self.capture_image is not None:
            self.save_image()
            self.close()

3. Pass the screenshot to the main interface

Create a file under the controller package CaptureScreen.pyand inherit it UI_CaptureScreen.py. Let's write the logic between windows.

We call save_image() in the double-click event of the mouse. Here we implement it

    def save_image(self):
        self._signal[QPixmap].emit(self.capture_image)

A signal is triggered here, the following is the definition of the signal

_signal = pyqtSignal(QPixmap)

The trigger signal can transfer data between windows, here we define a signal of type QPixmap, and then we use a slot in MainWindow to receive this data

self.screenWindow._signal[QPixmap].connect(self.handle_capture_picture)

Then there is the method of processing this data, that is, after receiving the data, display it on the Label

    @pyqtSlot(QPixmap)
    def handle_capture_picture(self, img):
        print("获取到图片", img)
        self.img_raw = img
        local_img = QPixmap(img).scaled(self.picture_label.width(), self.picture_label.height())
        # self.picture_label.setScaledContents(True)
        self.picture_label.setPixmap(local_img)

The above logic is that when the save_image triggers the signal, the handle_capture_picture of MainWIndow will process the signal, and then put the picture on the interface.

4. Realize shortcut key screenshot

Realize the screenshot of the shortcut key The system_hotkey library is used here, which is also the global hotkey implemented by the signal slot used. First, define a signal

sig_keyhot = pyqtSignal(str)

and then connect to the handler function

self.sig_keyhot[str].connect(self.MKey_pressEvent)

Initialize two hotkeys and register hotkeys

self.hk_start, self.hk_stop = SystemHotkey(), SystemHotkey()
self.hk_start.register(('control', '1'), callback=lambda x: self.send_key_event("capture_start"))
self.hk_stop.register(('control', '2'), callback=lambda x: self.send_key_event("None"))

Then there is the handler function

    @pyqtSlot(str)
    def MKey_pressEvent(self, i_str):
        if i_str == 'capture_start':
            self.screenWindow.show()
        elif i_str == 'None':
            QMessageBox.information(self, '温馨提示', '其他功能请等待后续添加哦')

Summarize

The above is the whole content of this article. This article completes the screenshot function of this project, uses pyqt's native method to realize the screenshot, and responds to the main interface, which is also one of the powerful functions of PyQt5DesignMode , which improves the development efficiency of the program.

This small tool is only a part of this project. In fact, we will add tools such as pdf format conversion later until this project can be published. Please look forward to the following articles.

PyQt5DesignMode is a project template implemented by me combining MVC ideas with pyqt5. It aims to realize multi-window applications with pyqt5. If you are interested, please give me one star.

Welcome to subscribe to this column to learn more about python.

Guess you like

Origin blog.csdn.net/weixin_47754149/article/details/127470534