Introduction
In automated testing, such as using selenium to test web pages, when locating elements, identifying elements, and confirming element content, you will encounter that the target element cannot be obtained, the target element is a picture and the text cannot be obtained, and the dynamic change of the target element cannot define the expected value, etc. , Encountering all of the above will hinder the progress of the test script, so choose image recognition to achieve the above actions that cannot be completed.
Principle and environment construction
Outside the original python environment (the author uses python3.7), to achieve image recognition, first install the tesseract back-end engine to support OCR recognition, mainly involving two environments (screenshots, OCR recognition).
tesseract
Since Textshot's OCR recognition needs to call the tesseract backend engine, first, tesseract needs to be installed.
1. Windows version installation can directly access the download link [https://sourceforge.net/projects/tesseract-ocr/].
Configure two environment variables, one path and one TESSDATA_PREFIX.
(1) Add a path under path to the path where Tesseract-OCR is located, for example: F:\SocTest\dist\Tesseract-OCR.
(2) Create a new TESSDATA_PREFIX and put it in the path where tessdata is located, for example, F:\SocTest\dist\Tesseract-OCR\tessdata.
(3) After the configuration is complete, the computer needs to be restarted.
2. Mac can use Homebrew to install (brew install tesseract)
Snipping Tool Installation
The screenshot tool is a tool we often use. There are many libraries or functions in Python that can realize screenshots. For example, pyscreenshot or the Image function in PIL, you only need to pass the coordinates of the starting point and end point of the mouse frame selection to the grab method. You can realize the screenshot function.
It can be installed through the pip install tool such as: pip install pyscreenshot.
Image recognition verification code Demo implementation
Image recognition function
Create a SocImage.py file, import os, PIL-Image, pytesseract, time, the function needs to pass in the browser and element as the main parameters, and use the incoming element to get the x and y coordinate axes of the upper left corner where the entire verification code is located, and then Calculate the coordinate axes of the other three points by obtaining the width and height of the element, realize the screenshot and name it screen, after the image is prepared, call the image_to_string function of pytesseract to convert the image into a string, and finally return the recognized string For the test script function, the code sample is as follows:
test script function
