1. Identification process
Receipt Identification Workflow
Working with Images in MATLAB
Import, display, and manipulate color and grayscale images.
Segmenting an Image
Create binary images by thresholding pixel intensity values.
Pre- and Postprocessing Techniques
Improve an image segmentation using common pre- and postprocessing techniques.
Classification and Batch Processing
Develop a metric to classify an image, and apply that metric to a set of image files.
二、Working with Images in MATLAB
1.Getting Images into MATLAB
Importing images into MATLAB
In all MATLAB image processing workflows, the first step is importing images into the MATLAB workspace.
Once there, you can use MATLAB functions to display, process, and classify the images.
You can import an image into memory and assign it to a variable in the MATLAB workspace for later manipulation, display, and export.
A = imread("imgFile.jpg");
Ending the line with a semicolon stops MATLAB from displaying the values stored in A, which is helpful since images typically contain millions of values.
I = imread("IMG_001.jpg");In this training, you'll work with a mix of receipt and non-receipt images. The file names don't indicate if an image is a receipt image or not, but you can find out by displaying the image.
Image data stored in a variable A can be displayed using the imshow function.
imshow(A)imshow(I)
I2 = imread("IMG_002.jpg");
imshow(I2)
You often want to view multiple images. For example, you might want to compare an image and a modified version of that image.
You can display two images together using the imshowpair function.
imshowpair(A1,A2,"montage")
The "montage" option places the images A1 and A2 side-by-side, with A1 on the left and A2 on the right.
imshowpair(I,I2,"montage")
When you're done processing an image in MATLAB, you can save it to a file. The function imwrite exports data stored in the MATLAB workspace to an image file.
imwrite(imgData,"img.jpg")
The output file format is automatically identified based on the file extension.
Try using imwrite to export I as a PNG file. Then use imread to load the image file into a new variable, and display it.
imwrite(I,"myImage.png")
Inew = imread("myImage.png");
imshow (Inew)
Remember that you can execute your script by clicking the Run or Run Section button in the MATLAB Toolstrip.
When you're finished practicing, you may move on to the next section.
2.Grayscale and Color Images
Color planes in receipt images
While receipts contain mostly grayscale colors, receipt images are still stored with three color planes like other color images. Like sunlight, which is composed of a full spectrum of colors, the color planes combine to create bright white paper.
Extract Color Planes and Intensity Values
Many image processing techniques are affected by image size. The images you'll be classifying all have the same size: 600 pixels high and 400 pixels wide.
You can find the size of an array using the size function.
sz = size(A)
The first value returned by size is the number of rows in the array, which corresponds to the height of the image. The second value is the number of columns in the array, which corresponds to the width of the image.
I = imread("IMG_003.jpg");
imshow(I)
sz = size(I)
Did the size of the array match the size of the image in pixels? Almost. You probably noticed that there was an additional value returned by the size function: 3.
That's because the imported image is a color image, and so it needs a third dimension to store the red, green, and blue color planes.
You can access individual color planes of an image by indexing into the third dimension. For example, you can extract the green (second) color plane using the value 2 as the third index.
Ig = I(:,:,2);R = I(:,:,1);
imshow(R)
Most images use the unsigned 8-bit integer (uint8) data type, which stores integers from 0 to 255. Bright or brightly colored images contain pixel intensity values near 255 in one or more color planes.
The red plane of this receipt image has some pretty bright areas. Do you think it will reach the maximum value of 255?
You can find the largest value in an array using the max function.
Amax = max(A,[],"all")
Using the "all" option finds the maximum across all values in the array. The brackets are required; they are placeholders for an unused input.
Rmax = max(R,[],"all")
Dark areas, like receipt text, contain values close to zero. Based on the image of the red plane from Task 3, do you think the red plane has any elements with a value of 0?
You can find the smallest value in an array using the min function.
Amin = min(A,[],"all")Rmin = min(R,[],"all")
Many common tasks can be completed more quickly using functions in Image Processing Toolbox. For example, if you want to extract all three color planes of an image array, you can use imsplit instead of indexing into each plane individually.
[R,G,B] = imsplit(A);
You can display all three color planes at once using montage.
montage({R,G,B})
Try using imsplit to extract the color planes of I. Display all three color planes using montage.
Convert from Color to Grayscale
Why convert to grayscale?
- When loaded into memory, a grayscale image occupies a third of the space required for an RGB image.
- Because a grayscale image has a third of the data, it requires less computational power to process and can reduce computation time.
- A grayscale image is conceptually simpler than an RGB image, so developing an image processing algorithm can be more straightforward when working with grayscale.
In receipt images, like the one shown below, little information is lost when converting to grayscale. The essential characteristics, like the dark text and bright paper, are preserved.
On the other hand, if you wanted to classify the produce visible in the background, the color planes would be essential. After converting to grayscale, it's hard to see the broccoli, let alone classify it.
The algorithm that you’ll implement in this training is built around identifying text, so color is not necessary. Converting the images to grayscale eliminates color and helps the algorithm focus on the black and white patterns found in receipts.
You can convert an image to grayscale using the im2gray function.
Ags = im2gray(A);I = imread("IMG_002.jpg");
imshow(I)
gs = im2gray(I);
imshow(gs)
Did you notice that the purple color of the receipt text is lost during the conversion to grayscale? However, since not all receipts have purple text, the purple values aren't useful in a classification algorithm.
This negligible loss of information has an upside: gs is one third the size of I, having only two dimensions instead of three. You can verify this with the size function.
sz = size(gs)
The grayscale image created by im2gray is a single plane of intensity values. The intensity values are computed as a weighted sum of the RGB planes, as described here.
If you’ll be needing your converted grayscale images later, you can save them to a file. Try using imwrite to save gs to the file gs.jpg.
3.Contrast Adjustment
Contrast and Intensity Histograms
Even though these two receipt images are grayscale, the contrast is different.
If you are analyzing a set of images, normalizing the brightness can be an important preprocessing step, especially for identifying the black and white patterns of text in receipt images.
An intensity histogram separates pixels into bins based on their intensity values. Dark images, for example, have many pixels binned in the low end of the histogram. Bright regions have pixels binned at the high end of the histogram.
The histogram often suggests where simple adjustments can be made to improve the definition of image features. Do you notice any adjustments that could be made to the receipt image so that the text is easier to identify?
One of the essential identifying features of a receipt image is text. Receipt images should have good contrast so that the text stands out from the paper background.
You can investigate the contrast in an image by viewing its intensity histogram using the imhist function.
imhist(I)
If the histogram shows pixels binned mainly at the high and low ends of the spectrum, the receipt has good contrast. If not, it could probably benefit from a contrast adjustment.
I = imread("IMG_001.jpg");
I2 = imread("IMG_002.jpg");
gs = im2gray(I);
gs2 = im2gray(I2);
imshowpair(gs,gs2,"montage")
imhist(gs)
Did the histogram of gs suggest high contrast between the text and receipt paper, or do you believe it could use an adjustment?
A second image, gs2, is displayed in the script. Does it appear washed out? While a visual inspection can be useful, it’s often easier to assess the contrast by viewing the histogram.
imhist(gs2)
The intensity histogram of gs2 shows lower contrast between the text and the background. Most of the dark pixels have intensity values around 100, and not many bright pixels have intensity values above 200. That means the contrast is about half of what it could be if the image used the full intensity range (0 to 255).
Increasing image contrast brightens light pixels and augments dark pixels. You can use the imadjust function to adjust the contrast of a grayscale image automatically.
Aadj = imadjust (A);
gs2Adj = imadjust(gs2);
imshowpair(gs2,gs2Adj,"montage")