I do not know at the time as a sweep with ZXing code base, there is no thought "ZXing is how each frame captured from the camera to get a picture of the two-dimensional code and resolve it?", And if you've thought about from source in know the answer, then this article is not necessary that you read on, if you thought about it but do not know the answer, then this article is for you, I believe after you've read have a clear answer.
In order not so unexpected, or the first source to follow a step by step explanation, first look at how to get to the data in the camera captured images of.
Get the camera to capture data
Since the previous article has analyzed ZXingthe steps of decoding, the focus here look, the camera captures the image of the subsequent steps, the source code is as follows
public void restartPreviewAndDecode() {
if (state == State.SUCCESS) {
state = State.PREVIEW;
cameraManager.requestPreviewFrame(decodeThread.getHandler(), R.id.decode);
activity.drawViewfinder();
}
}
复制代码The above code is CaptureActivityHandlercalled in the constructor, i.e. the CaptureActivityHandlercalling instantiation. Then, a call to cameraManagerthe requestPreviewFramemethod, the following code
/**
* A single preview frame will be returned to the handler supplied. The data will arrive as byte[]
* in the message.obj field, with width and height encoded as message.arg1 and message.arg2,
* respectively.
*
* @param handler The handler to send the message to.
* @param message The what field of the message to be sent.
*/
public synchronized void requestPreviewFrame(Handler handler, int message) {
OpenCamera theCamera = camera;
if (theCamera != null && previewing) {
previewCallback.setHandler(handler, message);
theCamera.getCamera().setOneShotPreviewCallback(previewCallback);
}
}
复制代码Now let's analyze the above code, look at the focus of the sentence
theCamera.getCamera().setOneShotPreviewCallback(previewCallback);
复制代码The role of this code is to set a callback preview frame, meaning that each camera will capture a data call, set here previewCallbackin the method, the analysis, the final call previewCallbackmethod is the public void onPreviewFrame(byte[] data, Camera camera)first argument here is that every i.e., the image data of a byte array. Google Android in support of common YUV format Camera Preview CallBack, there are two: one is NV21, one is YV12, the general default Android is YCbCR_420_sp (NV21), of course, you can also set the format they need through the following code.
Camera.Parameters parameters = camera.getParameters();
parameters.setPreviewFormat(ImageFormat.NV21);
camera.setParameters(parameters);
复制代码ZXingThe library is not formatted, so here is the default NV21format. So the question is, NV21in the end what does it mean? For more information, please read on
Detailed YUV image format
YUV color encoding is a method, and also its equivalent RGB color coding methods.
RGB image, each pixel has red, green, and blue three primary colors, wherein each primary color are occupied by 8 bit, which is a byte, a pixel will take up 24 bit, which is three bytes . A 1280 * 720 size image, on the occupation of 1280 * 720 * 3/1024/ 1024 = 2.63 MB of storage space. YUV color encoding is used to specify the brightness and chromaticity of the color pixel . Wherein, Y represents the brightness (Luminance, Luma), and U and V represent chrominance (Chrominance, Chroma). The color has defined two aspects of color: hue and saturation.
Above NV21, and YV12it is stored in YUV format.
- NV21 belongs YUV420SP format type. It is also previously stored the Y component, but then again not store all of the U or V component, the UV component but alternately continuously stored.
- YV12 format belong YUV420P type, i.e., Y component stored beforehand, then stored U, V components, YV12 and then after the first Y V U.
About YUV format, there is a good article online, click here . After a certain understanding of the YUV format, we continue to analyze the source code, look, is how to identify the two-dimensional code from the picture.
Picture two-dimensional code recognition
As already known, a camera data are each acquired callback PreviewCallbackclass onPreviewFramemethod, in this method, use of the mechanisms Handler, converts the image into a byte array to a DecodeHandlerclass, then call a decodemethod, as follows
private void decode(byte[] data, int width, int height) {
long start = System.nanoTime();
//...省略部分代码
Result rawResult = null;
PlanarYUVLuminanceSource source = activity.getCameraManager().buildLuminanceSource(data, width, height);
if (source != null) {
BinaryBitmap bitmap = new BinaryBitmap(new GlobalHistogramBinarizer(source));
try {
rawResult = multiFormatReader.decodeWithState(bitmap);
} catch (ReaderException re) {
// continue
Log.e(TAG, "decode: 没有发现二维码" );
} finally {
multiFormatReader.reset();
}
}
//...省略部分代码
}
复制代码It can be said that part of the code ZXingto decode the core code, and now a little bit of analysis, look at the
PlanarYUVLuminanceSource source = activity.getCameraManager().buildLuminanceSource(data, width, height);
复制代码This code, instantiates PlanarYUVLuminanceSourcean object, the main purpose is acquiring image data of the scan code frame. When the binary image of this object calls the method, and will be introduced later in the source code. Look at this code
BinaryBitmap bitmap = new BinaryBitmap(new GlobalHistogramBinarizer(source));
复制代码This code, ah, look at new GlobalHistogramBinarizer(source)this code, GlobalHistogramBinarizerdata image is binarized in this class, of course, a HybridBinarizerclass that also the binary image, and that what the main difference is it? The main difference is the HybridBinarizertype of treatment than GlobalHistogramBinarizeraccurate, but processing speed is slow, it is recommended to use the phone in a relatively good performance, while GlobalHistogramBinarizerprocessing less precise, if any of the shadow, the picture might be addressed there will be problems, but faster is recommended in the performance is not good phone. Here, we are using GlobalHistogramBinarizerto binary image processing, because, after my test found that the speed quickly.
Look sentence code is an instance of a BinaryBitmapclass, then the GlobalHistogramBinarizerobject is injected.
The following code is found and resolved two-dimensional code from the image, as follows
try {
rawResult = multiFormatReader.decodeWithState(bitmap);
} catch (ReaderException re) {
// continue
Log.e(TAG, "decode: 没有发现二维码" );
} finally {
multiFormatReader.reset();
}
复制代码Tracking down and found eventually calls QRCodeReaderclass decode(BinaryBitmap image, Map<DecodeHintType,?> hints)method. code show as below
public final Result decode(BinaryBitmap image, Map<DecodeHintType,?> hints)
throws NotFoundException, ChecksumException, FormatException {
DecoderResult decoderResult;
ResultPoint[] points;
if (hints != null && hints.containsKey(DecodeHintType.PURE_BARCODE)) {
BitMatrix bits = extractPureBits(image.getBlackMatrix());
decoderResult = decoder.decode(bits, hints);
points = NO_POINTS;
} else {
// 会进入这段代码
DetectorResult detectorResult = new Detector(image.getBlackMatrix()).detect(hints);
decoderResult = decoder.decode(detectorResult.getBits(), hints);
points = detectorResult.getPoints();
}
// If the code was mirrored: swap the bottom-left and the top-right points.
if (decoderResult.getOther() instanceof QRCodeDecoderMetaData) {
((QRCodeDecoderMetaData) decoderResult.getOther()).applyMirroredCorrection(points);
}
Result result = new Result(decoderResult.getText(), decoderResult.getRawBytes(), points, BarcodeFormat.QR_CODE);
List<byte[]> byteSegments = decoderResult.getByteSegments();
if (byteSegments != null) {
result.putMetadata(ResultMetadataType.BYTE_SEGMENTS, byteSegments);
}
String ecLevel = decoderResult.getECLevel();
if (ecLevel != null) {
result.putMetadata(ResultMetadataType.ERROR_CORRECTION_LEVEL, ecLevel);
}
if (decoderResult.hasStructuredAppend()) {
result.putMetadata(ResultMetadataType.STRUCTURED_APPEND_SEQUENCE,
decoderResult.getStructuredAppendSequenceNumber());
result.putMetadata(ResultMetadataType.STRUCTURED_APPEND_PARITY,
decoderResult.getStructuredAppendParity());
}
return result;
}
复制代码View
DetectorResult detectorResult = new Detector(image.getBlackMatrix()).detect(hints);
复制代码This code. image.getBlackMatrix()Is to call GlobalHistogramBinarizerthe class getBlackMatrix, wherein the code is not read, getBlackMatrixthe main role is to process image binarization processing, the binarization is critical to define the limits of black and white, we have converted the image to grayscale images , each point is represented by a gray scale value, it is necessary to define a gray value greater than this value will be white (0), the value is below the black (1). A specific processing method is as follows
In GlobalHistogramBinarizer, the uniform image is taken from the line 5 (covering the entire height of the image), each line of four-fifths of the intermediate as a sample taken; gray value to the X-axis, the number of pixels per gray scale value of the Y-axis establishing a histogram, most points taken from a gray value histogram, and then go to other fractional calculated gradation value, multiplied by the number of points according to distance from the maximum number of points to the square of the gray scale value of scoring, selected from a highest gray value score. Next, in the middle of these two gray values of distinguishing a selection principle is taken as close to the middle as possible and to points. With boundaries after it is easy, compared with each point of the whole image, if the grayscale value is smaller than the limit of the black matrix in the new set point, the rest is white, to 0.
A code above, calls Detectorthe detect(Map<DecodeHintType,?> hints)method, as follows
/**
* <p>Detects a QR Code in an image.</p>
*
* @param hints optional hints to detector
* @return {@link DetectorResult} encapsulating results of detecting a QR Code
* @throws NotFoundException if QR Code cannot be found
* @throws FormatException if a QR Code cannot be decoded
*/
public final DetectorResult detect(Map<DecodeHintType,?> hints) throws NotFoundException, FormatException {
resultPointCallback = hints == null ? null :
(ResultPointCallback) hints.get(DecodeHintType.NEED_RESULT_POINT_CALLBACK);
FinderPatternFinder finder = new FinderPatternFinder(image, resultPointCallback);
FinderPatternInfo info = finder.find(hints);
return processFinderPatternInfo(info);
}
复制代码Can be learned from the comments of this code, the effect of this method is the "result of the two-dimensional code detection package," If no two-dimensional code will throw an NotFoundExceptionexception if it can not resolve the two-dimensional code will throw an FormatExceptionexception. Now, we look at how to find the two-dimensional code in an image.
It features two-dimensional code
Before introducing the method of two-dimensional code image was found, first look at the characteristics of the two-dimensional code, as in FIG.
There are three two-dimensional code "back" -shaped pattern, it is very obvious. Located at a point intermediate the upper left corner of the pattern, if the image deflection correction may be two-dimensional code.
Two-dimensional code identification, identification is three point two-dimensional code, and gradually analyze the characteristics of these three points
- Each point has two profiles. That is, two mouths, inside a large "mouth" There is a small "mouth", so the two profiles.
- If this "back" into a white background, from left to right, top to bottom, or draw a line. This line through the ratio of black and white pattern is about: black and white ratio of 1: 1: 3: 1: 1. As shown below
- How to find the top left corner of the vertex? The apex angle of the other two vertices is 90 degrees.
By the above steps, we can identify three vertices of the two-dimensional code, and identify the top left vertex.
ZXing two-dimensional code image is recognized
Has been described above features two-dimensional code, also found two-dimensional code describes how the "return", and now we look at ZXingis how to identify the two-dimensional code in the picture, the main code is as follows
final FinderPatternInfo find(Map<DecodeHintType,?> hints) throws NotFoundException {
boolean tryHarder = hints != null && hints.containsKey(DecodeHintType.TRY_HARDER);
int maxI = image.getHeight();
int maxJ = image.getWidth();
// 在图像中寻找黑白像素比例为1:1:3:1:1
int iSkip = (3 * maxI) / (4 * MAX_MODULES);
if (iSkip < MIN_SKIP || tryHarder) {
iSkip = MIN_SKIP;
}
boolean done = false;
int[] stateCount = new int[5];
for (int i = iSkip - 1; i < maxI && !done; i += iSkip) {
// 获取一行的黑白像素值
clearCounts(stateCount);
int currentState = 0;
for (int j = 0; j < maxJ; j++) {
if (image.get(j, i)) {
// 黑色像素
if ((currentState & 1) == 1) { // Counting white pixels
currentState++;
}
stateCount[currentState]++;
} else { // 白色像素
if ((currentState & 1) == 0) { // Counting black pixels
if (currentState == 4) { // A winner?
if (foundPatternCross(stateCount)) { // Yes 是否是二维码左上角的回字
boolean confirmed = handlePossibleCenter(stateCount, i, j);
if (confirmed) {
// Start examining every other line. Checking each line turned out to be too
// expensive and didn't improve performance.
iSkip = 2;
if (hasSkipped) {
done = haveMultiplyConfirmedCenters();
} else {
int rowSkip = findRowSkip();
if (rowSkip > stateCount[2]) {
// Skip rows between row of lower confirmed center
// and top of presumed third confirmed center
// but back up a bit to get a full chance of detecting
// it, entire width of center of finder pattern
// Skip by rowSkip, but back off by stateCount[2] (size of last center
// of pattern we saw) to be conservative, and also back off by iSkip which
// is about to be re-added
i += rowSkip - stateCount[2] - iSkip;
j = maxJ - 1;
}
}
} else {
shiftCounts2(stateCount);
currentState = 3;
continue;
}
// Clear state to start looking again
currentState = 0;
clearCounts(stateCount);
} else { // No, shift counts back by two
shiftCounts2(stateCount);
currentState = 3;
}
} else {
stateCount[++currentState]++;
}
} else { // Counting white pixels
stateCount[currentState]++;
}
}
}
if (foundPatternCross(stateCount)) {
boolean confirmed = handlePossibleCenter(stateCount, i, maxJ);
if (confirmed) {
iSkip = stateCount[0];
if (hasSkipped) {
// Found a third one
done = haveMultiplyConfirmedCenters();
}
}
}
}
FinderPattern[] patternInfo = selectBestPatterns();
ResultPoint.orderBestPatterns(patternInfo);
return new FinderPatternInfo(patternInfo);
}
复制代码The above code does the following main thing
1, looking locator
In the image sampling on every other row iSkip,
int iSkip = (3 * maxI) / (4 * MAX_MODULES);
复制代码 The number of pixels in successive lines of the same color included in the array, the array length is 5 bits, i.e., go to black \ white \ black \ white \ black images (such as black starts to be detected included in the array [0] , until the detected value +1 are white array [0]; detecting the white starts counting in the array [1], and so on). After detecting which fills 5 5 whether the number of pixels in a ratio of 1: 1: 3: 1: 1 (there may be an error range of 50%), if the condition described to find the approximate location of the locator, this the image to the handlePossibleCentermethod of the locator to find the center point, the vertical direction is detected whether the start condition is satisfied locator, thereby determined so as to satisfy Y axis coordinate value of the center point, and then use this value to detect the coordinate in the horizontal direction again whether locator meet conditions, such as the center coordinate values thereby determined to meet the X axis. At this point to find the center coordinates a locator.
According to the above mentioned steps to find the center coordinates of all three locators, the next start positions of the three positioning locators symbol, i.e., upper left (B point), lower left (A point), upper right (C point) three positions. First by pairwise distance between the upper left fix what is the point (upper left point to other distance between two points that should be not far off), and then by calculating BA, BC vector cross product of two points A and C fix.
2, looking correctors
ABC calculate the coordinates of the three possible locations correctors, and then to AlignmentPatternFinder去look for that correction character closest to the bottom right corner, and seek to find ways to substantially the same manner locators, if found return correctors center coordinates, if not find it does not matter, the decoding program can continue.
Through the above two steps can determine whether there is a two-dimensional code image acquired by the camera frame, if there are two-dimensional code to parse the two-dimensional code, there is no two-dimensional code will throw an exception, then continue to resolve the next frame data.
to sum up
By analyzing the above explanation and source code, we can know if there is a two-dimensional code image frame judgment need to go through the following steps:
- Acquired image frame data, the YUV format;
- The two-dimensional code scan code frame gradation processing image data;
- The grayscale image after binarizing process;
- According to a feature locator to find a two-dimensional code;
- Looking correctors two-dimensional code.
If found correctors in step "4", then the frame picture contains a two-dimensional codes, two-dimensional code can be resolved, otherwise it throws an exception, continue to parse the data of the next frame image.
Conclusion
I do not look at the source code before, I am more confused, do not know how to determine whether there is a two-dimensional barcode image, although that can be judged based on two-dimensional code in the "return", but do not know how to find the "return" it ! Read the source code to know, you can picture a "binary" of treatment, and then to find the "return" character according to the proportion of black and white pixels, feel learned a lot. So, we do not know a function of a library is how to achieve, the best solution is to read the source code, the answers are in the source code.
In the study the source code when deleting a lot of independent code and analysis of two-dimensional code, the last code here .
This series of articles:
A source ZXing resolve: to let run up source
ZXing source analytic II: master decoding step
ZXing source code parsing III: Working with camera configuration and data
This article has been public number "AndroidShared" episode