[Unity]OCR recognition--OCRHMMDecoder articles

The effect of this case is particularly bad at present, and only letters and numbers can be recognized 

1. Demo running

        Open the OpenCV content in the previous article, find the Demo, then find the OCR.Alphabet Demo, and double-click to enter the Scene.

 

Run it, and you will find that the detection results are awesome.

 Two, code interpretation

        The code will not be posted here, let’s explain the overall logical relationship.

1. AlphabetOCRScene (mounted on RawImage)

         

         It has two inputs, one is a test image and the other is a trained knn model. as follows

public UnityEngine.Texture2D texture;
public UnityEngine.TextAsset model;

        It mainly relies on the AlphabetOCR class to implement detection. To create this class, you need to input the bytes of the model. Therefore, the original format of the model file is xml. In Unity, you need to change the xml suffix to bytes, and then drag it to the Model column.

2.AlphabetOCR

        It needs a Model to be successfully created, because the creation of OCRHMMDecoder requires an xml file path. In order to use the OpenCV plug-in in Android devices, the author specially wrote a conversion function to convert bytes into an xml file and load it. I almost didn’t understand it at the beginning. or something. as follows

public AlphabetOCR(byte[] model)
		{
			// use temporary file as OpenCV::Contrib::Text doesn not have in-memory reading functions
			string folder = UnityEngine.Application.persistentDataPath;
			string filename = null;
			while (null == filename || System.IO.File.Exists(System.IO.Path.Combine(folder, filename)))
				filename = System.IO.Path.GetRandomFileName() + ".xml";

			// flush
			var tempFile = System.IO.Path.Combine(folder, filename);
			System.IO.File.WriteAllBytes(tempFile, model);

			// read classifier callback and clean file
			ocr = new OCRHMMDecoder.ClassifierCallback(tempFile, CvText.OCRClassifierType.KNN);
			System.IO.File.Delete(tempFile);
		}

        After the creation is successful, the image is first grayscaled, filtered, binarized, the outline of the text is found, RotatedRect is obtained, the image is segmented, corrected, and finally classified and detected. The general process is as follows

const string vocabulary = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";

int[] classes;//类别
double[] confidences;//准确度
OCRHMMDecoder.ClassifierCallback ocr = null;
ocr = new OCRHMMDecoder.ClassifierCallback(tempFile, CvText.OCRClassifierType.KNN);
ocr.Eval(二值化分割后图像, out classes, out confidences);
// process
if (confidences.Length > 0)
{
	var unit = new RecognizedLetter();
	unit.Data = vocabulary[classes[0]].ToString();
	unit.Confidence = confidences[0];
	
}