In the morning, I searched for "color recognition" on Baidu, and I have a little bit of a general concept. It is the old way, hands-on, more experiments, and going forward. I would like to ask teachers to give me some advice.
OpenCV (Baidu Encyclopedia)
is a cross-platform computer vision library based on BSD license (open source) that can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient—consisting of a series of C functions and a small number of C++ classes, it also provides interfaces to languages such as Python, Ruby, and MATLAB, and implements many common algorithms in image processing and computer vision. OpenCV is written in C++, and its main interface is also C++, but it still retains a large number of C language interfaces. The library also has a large number of Python, Java and MATLAB/OCTAVE (version 2.5) interfaces. The API interface functions of these languages can be obtained through online documentation. Now it also provides support for C#, Ch, Ruby, GO.
OpenCV was established by Intel in 1999 and is now supported by Willow Garage. OpenCV is a cross-platform computer vision library based on the BSD license (open source) that can run on Linux, Windows and Mac OS operating systems. It is lightweight and efficient—consisting of a series of C functions and a small number of C++ classes, it also provides interfaces to languages such as Python, Ruby, and MATLAB, and implements many common algorithms in image processing and computer vision. OpenCV has a cross-platform middle and high-level API including more than 500 C functions. It does not depend on other external libraries-although some external libraries can also be used. OpenCV provides a transparent interface for Intel® Integrated Performance Primitives (IPP). This means that if there are IPP libraries optimized for a specific processor, OpenCV will automatically load these libraries at runtime. (Note: The code of OpenCV version 2.0 has been significantly optimized, no IPP is needed to improve performance, so version 2.0 no longer provides IPP interface)
Overview of OpenCV
Its full name is Open source Computer Vision Library, an open source computer vision library. In other words, it is a set of open source API function libraries for computer vision. This means:
(1) Whether it is scientific research or commercial application, you can use it for development;
(2) The source code of all API functions is public, and you can see the program steps of its internal implementation ;
(3) You can modify the source code of OpenCV, compile and generate the specific API functions you need. However, as a library, it only provides APIs for some commonly used, classic, and popular algorithms.
A typical computer vision algorithm should include the following steps:
(1) Data acquisition (for OpenCV, it is a picture);
(2) Preprocessing;
(3) Feature extraction;
(4) Feature selection;
(5) Classification Design and training of the device;
(6) Classification and discrimination;
and OpenCV provides APIs for these six parts respectively (remember the word).
Color recognition based on OpenCV
Color model The
commonly used models in digital image processing are RGB (red, green, blue) model and HSV (hue, saturation, brightness). RGB is widely used in color monitors and color video cameras. Our usual pictures are generally RGB model. The HSV model is more in line with the way people describe and interpret colors. The color description of HSV is natural and very intuitive to people.
HSV model
HSV model color parameters are: hue (H: hue), saturation (S: saturation), brightness (V: value). A color space created by AR Smith in 1978, also known as Hexcone Model.
Hue (H: hue): Measured by an angle, the value range is 0°～360°, starting from red and counting in a counterclockwise direction, red is 0°, green is 120°, and blue is 240°. Their complementary colors are: yellow is 60°, cyan is 180°, and magenta is 300°;
saturation (S: saturation): the value range is 0.0 to 1.0, the larger the value, the more saturated the color.
Brightness (V: value): The value range is 0 (black) to 255 (white).
Convert RGB to HSV.
Let (r, g, b) be the red, green, and blue coordinates of a color, and their values are real numbers between 0 and 1. Let max be equivalent to the largest of r, g, and b. Let min be equal to the smallest of these values. To find the value of (h, s, v) in HSV space, where h ∈ [0, 360) is the hue angle of the angle, and s, v ∈ [0,1] is the saturation and brightness. There is a function under OpenCV that can directly convert RGB model to HSV model. In OpenCV, H ∈ [0, 180), S ∈ [0, 255], V ∈ [0, 255]. We know that the H component can basically represent the color of an object, but the values of S and V must also be within a certain range, because S represents the degree of mixing of the color represented by H and white, that is, the smaller S is, The whiter the color, the lighter the color; V represents the mixing degree of the color represented by H and black, that is, the smaller the V, the darker the color. Roughly identifying the value of blue is H in the range of 100 to 140, S and V in the range of 90 to 255.
OpenCV color recognition ideas
1. Create a slider: used to adjust the threshold and identify different colors.
2. Color space conversion: Convert RGB to HSV model, so the color can be identified by different thresholds of HSV of different colors. It can be achieved by using cvtcolor() in Opencv. Generally, color images are in RGB color space, and HSV color space model is a color system that is even more commonly used in people’s lives. When adjusting the brightness on the TV remote control, in the palette of paintings, and when watching TV It's very common, because it is more in line with the way people describe colors-what color it is, how dark it is, and how bright it is. It should be noted that in opencv, the value ranges of H, S, and V are [0,180), [0,255), and [0,255), respectively, instead of the actual model [0,360], [0,1], [0,1].
3. Histogram equalization: Due to the influence of light, each frame of the picture read by the mobile phone may be too bright or too dark. Histogram equalization can make the distribution of pixels in each interval more balanced and make the image layered. Stronger. Opencv can be implemented with equalizeHist() function. Histogram equalization is to stretch the original histogram to make it evenly distributed in the entire gray scale, thereby enhancing the contrast of the image. The central idea of histogram equalization is to change the grayscale histogram of the original image from a relatively concentrated area to a uniform distribution in the entire grayscale range.
4. Binarization: Set the gray value of the pixels on the image to 0 or 255, which will make the entire image show a clear black and white effect. Binarization of grayscale images can highlight a certain range of information. It is to set the color value of the pixel within the set interval (such as ab) to 255, and set it to 0 outside the range. But for different values of ab and, the binarization effect will be great different.
5. Open operation: used to remove noise in the image, that is, interference information. Opencv can use getStructuringElement() function to make corresponding settings. The opening operation is based on the expansion and erosion of the image. Expansion is the "field expansion" of the highlighted part of the image. The effect image has a larger highlight area than the original image; the erosion is that the highlight area in the original image is eaten away. , The effect picture has a smaller highlight area than the original picture. The open operation is to corrode the image first and then expand to eliminate small objects. The mathematical principle is to define a convolution kernel B and convolve it with the target image to achieve the corresponding effect. Nuclei of different shapes and sizes will have different effects.
6. Close operation: After the open operation, there may be some disconnected areas. The closed operation can close these unconnected areas to make the image more complete. The closing operation is the opposite of the opening operation. It expands first and then corrodes. It is used to eliminate small black holes. The principle is the same as the opening operation.
There is a Rubik's Cube with five colors, so I use it as an experimental prop for identifying colors.
Open MaixPy IDE, select Tools-Machine Vision-Value Editor
Open the source image position, select the frame buffer
Adjust the LAB value, mainly in the binary image column. The white pixels are the pixels
to be tracked. Thoroughly understand the
name of the Lab color space.
Before starting, please clarify the name of the Lab color space:
The full name of Lab is CIELAB. Sometimes it is also written as CIE Lab*
where CIE stands for International Commission on Illumination (International Commission on Illumination), which is an international authority on lighting and color.
The channel
Lab is composed of a luminance channel (channel) and two color channels. In the Lab color space, each color is represented by three numbers L, a, and b. The meaning of each component is as follows:
L stands for brightness
a stands for the component from green to red
b stands for the component from blue to yellow
Perceptual uniform
Lab is designed based on people's perception of color, more specifically, it is perceptual uniform. Perceptual uniform means that if the number (that is, the three numbers mentioned above, L, a, and b) varies by the same amount, then it will bring about the same visual variation. Compared with RGB and CMYK color spaces, Lab is more in line with human vision and easier to adjust: if you want to adjust the brightness (regardless of the Helmholtz–Kohlrausch effect, see the note below), adjust the L channel, and if you want to adjust only the color balance, adjust them separately. a and b.
Note: The Helmholtz–Kohlrausch effect is an illusion of the human eye-when the color saturation is high, the color will look brighter.
Device independent
Lab has a very good feature-device-independent (device-independent). In other words, given the white point of the color space (the white point of a color space is shown in the figure below), this color space can clearly determine how each color is created and displayed. It has nothing to do with the display medium used. It should be noted that Lab defines the color relative to the white point. Only after defining what color the white point is (for example, defined as CIE standard illuminant D50) can we know other colors.
Numerical Range
Theoretically, L, a, and b are all real numbers, but in practice they are generally limited to an integer range: the
larger the L, the higher the brightness. When L is 0, it represents black, and when L is 100, it represents white.
When a and b are 0, both represent gray.
When a changes from a negative number to a positive number, the corresponding color changes from green to red.
b changes from a negative number to a positive number, and the corresponding color changes from blue to yellow.
In practical applications, we often set the range of the color channel between -100~+100 or -128127.
Visualization As
you can see, Lab* has three components, so it can be presented in three-dimensional space. In a two-dimensional space, a chromaticity diagram is often used to visualize it, that is, to fix the brightness L and see the changes in a and b. Note that these visualizations are not precise, they just help people understand.
LAB color model
It is established in accordance with an international standard for color measurement established by Commission International Eclairage (CIE) in 1931. It was improved in 1976 and named a color mode. The Lab color model makes up for the deficiencies of the RGB and CMYK color modes. It is a device-independent color model and a color model based on physiological characteristics. The Lab color model consists of three elements, one element is brightness (L), and a and b are two color channels. a includes colors ranging from dark green (low brightness value) to gray (medium brightness value) to bright pink (high brightness value); b is from bright blue (low brightness value) to gray (medium brightness value) and then To yellow (high brightness value). Therefore, this color mixing will produce bright colors.
The Lab mode does not rely on light or paint. It is a color mode determined by the CIE organization that theoretically includes all colors that can be seen by the human eye. Lab mode makes up for the shortcomings of RGB and CMYK color modes. Compared with the RGB color space, Lab is an uncommon color space. It was established on the basis of the international standard for color measurement established by the International Commission on Illumination (CIE) in 1931. In 1976, it was officially named CIELab after modification. It is a device-independent color system and a color system based on physiological characteristics. This also means that it uses digital methods to describe human visual perception. The L component in the Lab color space is used to represent the brightness of the pixel, the value range is [0,100], which means from pure black to pure white; a means the range from red to green, and the value range is [127,-128]; b represents the range from yellow to blue, and the value range is [127,-128]. The following figure shows the icon of Lab color space;
The Lab color space is larger than the color gamut of computer monitors and even human vision. It is expressed that Lab bitmaps require more pixel data to obtain the same accuracy than RGB or CMYK bitmaps. Lab mode defines the most colors, and has nothing to do with light and equipment, and the processing speed is as fast as RGB mode, much faster than CMYK mode. Therefore, you can safely use Lab mode in image editing. Moreover, the color is not lost or replaced when the Lab mode is converted to CMYK mode. Therefore, the best way to avoid color loss is: use Lab mode to edit the image, and then convert to CMYK mode for printout.
#MicroPython hands-on (08)-zero-based learning MaixPy recognition color
#Experimental program one: find red blob dynamically recognize the red block
Try to select the best color tracking threshold, adjust the threshold through the slide bar, and highlight the red block on the binary image (display white).
The measured LAB values are: 55, 70, 42, 65, 52, 8
#MicroPython Do it by hand (08)——Zero-based learning MaixPy's recognition color
# One of the experimental procedures: find red blob dynamic recognition of red blocks
#MicroPython动手做（08）——零基础学MaixPy之识别颜色
#实验程序：find red blob 动态识别红色块
import sensor
import image
import lcd
import time
lcd.init(freq=15000000)
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.run(1)
red_threshold = (55, 70, 42, 65, 52, 8)
while True:
img=sensor.snapshot()
blobs = img.find_blobs([green_threshold])
if blobs:
for b in blobs:
tmp=img.draw_rectangle(b[0:4])
tmp=img.draw_cross(b[5], b[6])
c=img.get_pixel(b[5], b[6])
lcd.display(img)
The Rubik’s Cube for experimentation has 5 colors
After running, you can see the frame circle and + sign, which can be accurately identified
The experimental scene in front of the window in the cloudy
sky #MicroPython hands-on (08)-zero-based learning MaixPy to identify colors
#Experimental program two: find green blob dynamically identify green blocks
Get the green LAB threshold value 0, 88, -42, -6, -9, 13
#MicroPython动手做（08）——零基础学MaixPy之识别颜色
#实验程序之二：find green blob 动态识别绿色块
import sensor
import image
import lcd
import time
lcd.init(freq=15000000)
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.run(1)
green_threshold = (0, 88, -42, -6, -9, 13)
while True:
img=sensor.snapshot()
blobs = img.find_blobs([green_threshold])
if blobs:
for b in blobs:
tmp=img.draw_rectangle(b[0:4])
tmp=img.draw_cross(b[5], b[6])
c=img.get_pixel(b[5], b[6])
lcd.display(img)
#MicroPython Do it by hand (08)——Zero-based learning MaixPy's recognition color
#Experimental program three: find orange blob dynamically recognize orange blocks
Get the orange LAB threshold value of 0, 80, 66, -20, 80, 50 (intersection with red)
#MicroPython动手做（08）——零基础学MaixPy之识别颜色
#实验程序之三：find orange blob 动态识别橙色块
import sensor
import image
import lcd
import time
lcd.init(freq=15000000)
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.run(1)
orange_threshold = (0, 80, 66, -20, 80, 50)
while True:
img=sensor.snapshot()
blobs = img.find_blobs([orange_threshold])
if blobs:
for b in blobs:
tmp=img.draw_rectangle(b[0:4])
tmp=img.draw_cross(b[5], b[6])
c=img.get_pixel(b[5], b[6])
lcd.display(img)
The blue LAB threshold is 0, 80, -128, 35, -128, -18
The yellow LAB threshold value is 88, 95, 0, -44, 93, 48.
Note:
This experimental case is based on the color of an ordinary Rubik's cube (cloudy light), and the experimental data (LAB threshold value) is only reference
Blue (0, 80, -128, 35, -128, -18)
Red (55, 70, 42, 65, 52, 8)
Green (0, 88, -42, -6, -9, 13)
Orange (0, 80, 66, -20, 80, 50)
yellow (88, 95, 0, -44, 93, 48)
#MicroPython hands-on (08)-the recognition color of MaixPy
#Experimental procedure four: Identify the red and flowery parts on the picture
Open MaixPy IDE, select tools-machine vision-value editor-source image location-image file (recognize the picture on the computer)
Carefully adjust the values of the six sliders, the reference results are shown in the binary image below
The LAB threshold for identifying red and flower hearts is (0, 80, 90, 24, 75, -10)
#MicroPython动手做（08）——零基础学MaixPy之识别颜色
#实验程序之四：识别图片上的红心和花心部分
import sensor
import image
import lcd
import time
lcd.init(freq=15000000)
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.run(1)
Five_threshold = (0, 80, 90, 24, 75, -10)
while True:
img=sensor.snapshot()
blobs = img.find_blobs([Five_threshold])
if blobs:
for b in blobs:
tmp=img.draw_rectangle(b[0:4])
tmp=img.draw_cross(b[5], b[6])
c=img.get_pixel(b[5], b[6])
lcd.display(img)
