[Unity] OCR recognition--Tesseract articles

        By chance (actually, it was accidentally discovered on GitHub), I found a Unity project of Tesseract. The test found that not only the recognition effect is very good, but also supports multiple platforms.

1. Download project

GitHub - Neelarghya/tesseract-unity: Standalone OCR plugin for Unity using Tesseract Standalone OCR plugin for Unity using Tesseract. Contribute to Neelarghya/tesseract-unity development by creating an account on GitHub. https://github.com/Neelarghya/tesseract- After the unity      download is complete, unzip it. It can be opened directly with Unity, I am using 2020.3.3. (The project name is changed by myself)

     The Main in the Scenes directory is the demonstration path of the project. Run it directly, and you can see that the recognition effect is very good. 

 

 The effect is as follows.

     The next thing we have to do is to migrate him into the OpenCV project.

2. Migration plugin

         I will not export the plugin here, just copy the folder directly. The three folders "Plugins", "Scripts" and "StreamingAssets" we need, copy them out and paste them into the OpenCV project. For example, the main OCR folder

3. Use of plug-ins

       Create a new script class, mine is called EasyOCR, as follows

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using OpenCvSharp;
using OpenCvSharp.Util;
using OpenCvSharp.XFeatures2D;
using System;


public delegate void RecoginzeCall(string a);

public class EasyOCR
{
    private TesseractDriver _tesseractDriver;
    private bool IsSetUp = false;//确保已经启用成功
    private Texture2D mainTexture;//待识别图片
    private RecoginzeCall CallBack = null;

    public EasyOCR()
    {
        _tesseractDriver= _tesseractDriver = new TesseractDriver();
        //启用OCR，
        _tesseractDriver.Setup(OnSetupCompleteRecognize);
        IsSetUp = false;
    }
    public void Recoginze(Mat Img, RecoginzeCall _call)
    {
        Mat tempImg = Img.CvtColor(ColorConversionCodes.BGR2GRAY).MedianBlur(5).Threshold(185, 255, ThresholdTypes.BinaryInv);
        Texture2D temp = OpenCvSharp.Unity.MatToTexture(tempImg);
        CallBack = _call;
        mainTexture = temp;
        OnSetupCompleteRecognize();
    }
    public void Recoginze(Texture2D _texture, RecoginzeCall _call)
    {
        Texture2D texture = new Texture2D(_texture.width, _texture.height, TextureFormat.ARGB32, false);
        texture.SetPixels32(_texture.GetPixels32());
        texture.Apply();
        mainTexture = texture;
        OnSetupCompleteRecognize();
    }
    ///懒，就写在一个函数里了
    private void OnSetupCompleteRecognize()
    {
        if(!IsSetUp)
        {
            IsSetUp = true;
            return;
        }
        //识别，结果会以string形式返回，
        string data=_tesseractDriver.Recognize(mainTexture);
        if(CallBack!=null)
        {
            CallBack(data);
        }
    }
}

        The above code gives two detection functions, because Tesseract has a much better detection effect on black characters on a white background, so if your image to be detected is not black and white, you need to use OpenCV to binarize it . This point is very important.

        Next, give a sample code, I call the OCR function in Update to achieve "pseudo-real-time" detection

private EasyOCR OCR;
private Mat CheckMat;

void Start()
{
       
     OCR = new EasyOCR();
     ///打开相机
     StartCoroutine(OpenCamera());
}
void Update()
{
       
    if (Tex != null && Tex.didUpdateThisFrame)
    {
        Mat Frame = OpenCvSharp.Unity.TextureToMat(Tex);
        OpenCvSharp.Rect Roi = new OpenCvSharp.Rect(760, 440, 400, 400);

        CheckTimes++;
        if (CheckTimes>24)
        {
            CheckTimes = 0;
            CheckMat = Frame.SubMat(Roi).Clone().Resize(new Size(200, 200));
            StartCoroutine(FrameCheck());
        }
           
        Cv2.Rectangle(Frame, Roi, new Scalar(0, 255, 255));
        Destroy(Capture.texture);
        Capture.texture = OpenCvSharp.Unity.MatToTexture(Frame);
     }
     
}
IEnumerator FrameCheck()
{
    yield return new WaitForSeconds(0.0f);
    //开始检测并给个回调函数
    OCR.Recoginze(CheckMat, OnChecked);
}

    The above part is mainly to do a Resize, which changes the size of the Roi area and speeds up the detection speed. It takes a long time for Tesseract detection, so I detect it every once in a while . For clear fonts that take up a large proportion of the image screen, you can properly resize them to speed up the detection process.