[Study Notes] Commonly used opencv operations in ROS

• Subscribe to the camera topic and perform callback processing

self.sub = rospy.Subscriber("/camera/rgb/image_raw", Image, self.image_cb, queue_size=1)

---

• CvBridge().imgmsg_to_cv2(): Conversion of ROS image message type to opencv type

cv_image = bridge.imgmsg_to_cv2(image_message, desired_encoding="passthrough")

cv_image1 = CvBridge().imgmsg_to_cv2(data, "bgr8")

data: image name

bgr8: BGR8 format

There is also conversion from Opencv type to ROS Image type:

image_message = cv2_to_imgmsg(cv_image, encoding="passthrough")

not_img=cv2.bitwise_not(img1)

---

• cv2.cvtColor(): Convert RGB color to HSV space

The cvtcolor() function is a color space conversion function that can convert RGB colors to HSV, HSI and other color spaces. Can also be converted to grayscale.

cvtColor(Mat src, Mat dst, int code)

src	A matrix representing the source.
dst	A matrix representing the destination.
code	An integer code representing the type of conversion, such as RGB to grayscale.

pic_hsv = cv2.cvtColor(pic0, cv2.COLOR_BGR2HSV)
cv2.imshow("hsv_pic",pic_hsv)

Additional knowledge:

cv::cvtColor() supports conversion between multiple color spaces. The supported conversion types and conversion codes are as follows:

1. Conversion of RGB and BGR (the default color space of opencv color images is BGR) color space

cv::COLOR_BGR2RGB
cv::COLOR_RGB2BGR
cv::COLOR_RGBA2BGRA
cv::COLOR_BGRA2RGBA

2. Add alpha channel to RGB and BGR images

cv::COLOR_RGB2RGBA
cv::COLOR_BGR2BGRA

3. Remove alpha channel from RGB and BGR images

cv::COLOR_RGBA2RGB
cv::COLOR_BGRA2BGR

4. Convert from RBG and BGR color space to grayscale space

cv::COLOR_RGB2GRAY
cv::COLOR_BGR2GRAY

cv::COLOR_RGBA2GRAY
cv::COLOR_BGRA2GRAY

5. Convert from grayscale space to RGB and BGR color space

cv::COLOR_GRAY2RGB
cv::COLOR_GRAY2BGR

cv::COLOR_GRAY2RGBA
cv::COLOR_GRAY2BGRA

6. Conversion between RGB and BGR color spaces and BGR565 color space

cv::COLOR_RGB2BGR565
cv::COLOR_BGR2BGR565
cv::COLOR_BGR5652RGB
cv::COLOR_BGR5652BGR
cv::COLOR_RGBA2BGR565
cv::COLOR_BGRA2BGR565
cv::COLOR_BGR5652RGBA
cv::COLOR_BGR5652BGRA

7. Conversion between grayscale space domain BGR565

cv::COLOR_GRAY2BGR555
cv::COLOR_BGR5552GRAY

8. Conversion between RGB and BGR color spaces and CIE XYZ

cv::COLOR_RGB2XYZ
cv::COLOR_BGR2XYZ
cv::COLOR_XYZ2RGB
cv::COLOR_XYZ2BGR

9. Conversion between RGB and BGR color spaces and uma chromaticity (YCrCb space)

cv::COLOR_RGB2YCrCb
cv::COLOR_BGR2YCrCb
cv::COLOR_YCrCb2RGB
cv::COLOR_YCrCb2BGR

10. Conversion between RGB and BGR color spaces and HSV color space

cv::COLOR_RGB2HSV
cv::COLOR_BGR2HSV
cv::COLOR_HSV2RGB
cv::COLOR_HSV2BGR

11. Conversion between RGB and BGR color spaces and HLS color space

cv::COLOR_RGB2HLS
cv::COLOR_BGR2HLS
cv::COLOR_HLS2RGB
cv::COLOR_HLS2BGR

12. Conversion between RGB and BGR color spaces and CIE Lab color space

cv::COLOR_RGB2Lab
cv::COLOR_BGR2Lab
cv::COLOR_Lab2RGB

cv::COLOR_Lab2BGR

13. Conversion between RGB and BGR color spaces and CIE Luv color space

cv::COLOR_RGB2Luv
cv::COLOR_BGR2Luv
cv::COLOR_Luv2RGB
cv::COLOR_Luv2BGR

14. Conversion of Bayer format (raw data) to RGB or BGR color space

cv::COLOR_BayerBG2RGB
cv::COLOR_BayerGB2RGB
cv::COLOR_BayerRG2RGB
cv::
COLOR_BayerGR2RGB cv::COLOR_BayerBG2BGR
cv::COLOR_BayerGB2BGR
cv::COLOR_BayerRG2BGR
cv::COLOR_BayerGR2BGR
————————————————
感谢： https://blog.csdn.net/weixin_51105360/article/details/113941015

Here is another definition of hsv:

 in:

        Hue: (hue, hue)

        Saturation (saturation, color purity)

        Value

The cross section of a cylinder can be regarded as a polar coordinate system, H is represented by the polar angle of the polar coordinates, S is represented by the polar axis length of the polar coordinates, and V is represented by the height of the central axis of the cylinder.

---

• np.arrray(): Construct an array to store the HSV threshold range of the color patch

The role of the np.array function: There is no dimensionality problem with lists, but arrays have dimensions, and the role of np.array() is to convert the list into an array, which can also be said to be used to generate an array.

LowerBlue = np.array([95, 90, 80])
UpperBlue = np.array([130, 255, 255])

 Two arrays are set here to determine the blue threshold

Generally, effective processing of images in color space is done in HSV space, and then a strict range needs to be given for the corresponding HSV component in the basic color. The following is the fuzzy range calculated through experiments (the accurate range is not available online. given).

H:  0 — 180

S:  0 — 255

V:  0 — 255

Here some reds are classified into the purple range:

---

• cv2.inRange(): Thresholding

mask = cv2.inRange(pic_hsv, LowerBlue, UpperBlue)
cv2.imshow("black&white",mask)

cv_image2: original image or image to be processed

LowerBlue: Processed by the threshold determined above. Anything below this value will become 0, which is black.

UpperBlue: It is also the threshold determined above. Above this value, it will turn into 0, which is black.

The value between LowerBlue and UpperBlue will be converted to 255, which is white

Because I use blue detection, after the processing is completed, the blue part of the graph will turn to white, and the other parts will turn to black, forming a black and white binary image. The mask is the processed image.

---

•  cv2.bitwise_and(): bitwise operation, masking the image

dst=cv2.bitwise_and（src1,src2[,mask]]）

Implement bitwise AND operations

•  dst represents an array output value with the same size as the input value.
•  src1 represents the first input value of array or scalar type.
•  src2 represents the second input value of array or scalar type.
•  Mask represents an optional operation mask, an 8-bit single-channel array.

Features:

•  Performing a bitwise AND operation on any value N and the value 0 will result in the value 0.
•  Performing a bitwise AND operation on any value N (only 8-bit values ​​are considered here) and the value 255 (8-bit binary number is 1111 1111) will result in the value N itself.

Based on the above characteristics, a mask image M can be constructed. There are only two values ​​in the mask image M: one is the value 0, and the other is the value 255. Perform a bitwise AND operation on the mask image M and a grayscale image G, and in the resulting image R:

•      The value at the position corresponding to the value 255 in the mask image M comes from the grayscale image G.
•      The value at the position corresponding to the value 0 in the mask image M is zero (black).

Thanks: https://blog.csdn.net/weixin_45335726/article/details/122415833

pic_and = cv2.bitwise_and(pic_hsv, pic_hsv, mask=mask)
cv2.imshow("and_pic",pic_and)

cv_image2: The original hsv image obtained

mask: Binary image after thresholding

The final graphic should be that the blue squares are still in the original color, and the other binarized parts will become black.

Understand the negation operation

The negation operation is very simple, that is, black becomes white, and white becomes black. Of course, this is not rigorous, but it reflects the result of the negation operation well; for example, the negation of 0 is 1, and the negation of 1 is 0. . To negate, use the bitwise_not method, which receives an image parameter. The following method still uses the image named 1bit.

      

 not_img = cv2.bitwise_not(img1)

Effect:

Thanks to: [python opencv computer vision from zero to practical] 7. Logical operations and applications_1_bit's blog-CSDN blog

---

• X[:,:,0]: is to take all the data in the first dimension of the three-dimensional matrix

Operations on Array objects in Python:

For X[:,0]: takes all the data in the first dimension of the two-dimensional array

For X[:,1]: it is to take all the data in the second dimension of the two-dimensional array

For X[:,m:n]: it is to take all the data from the mth dimension to the n-1th dimension in the two-dimensional array

For X[:,:,0]: it is to take all the data in the first dimension of the three-dimensional matrix

For X[:,:,1]: is to take all the data in the second dimension of the three-dimensional matrix

For X[:,:,m:n]: it is to take all the data from the mth dimension to the n-1th dimension in the three-dimensional matrix

#!usr/bin/env python
#encoding:utf-8
from __future__ import division
  
'''
__Author__:沂水寒城
学习Python中的X[:,0]、X[:,1]、X[:,:,0]、X[:,:,1]、X[:,m:n]和X[:,:,m:n]
'''
  
import numpy as np
  
def simple_test():
  '''
  简单的小实验
  '''
  data_list=[[1,2,3],[1,2,1],[3,4,5],[4,5,6],[5,6,7],[6,7,8],[6,7,9],[0,4,7],[4,6,0],[2,9,1],[5,8,7],[9,7,8],[3,7,9]]
  # data_list.toarray()
  data_list=np.array(data_list)
  print 'X[:,0]结果输出为：'
  print data_list[:,0] 
  print 'X[:,1]结果输出为：'
  print data_list[:,1]
  print 'X[:,m:n]结果输出为：'
  print data_list[:,0:1]
  data_list=[[[1,2],[1,0],[3,4],[7,9],[4,0]],[[1,4],[1,5],[3,6],[8,9],[5,0]],[[8,2],[1,8],[3,5],[7,3],[4,6]],
        [[1,1],[1,2],[3,5],[7,6],[7,8]],[[9,2],[1,3],[3,5],[7,67],[4,4]],[[8,2],[1,9],[3,43],[7,3],[43,0]],
        [[1,22],[1,2],[3,42],[7,29],[4,20]],[[1,5],[1,20],[3,24],[17,9],[4,10]],[[11,2],[1,110],[3,14],[7,4],[4,2]]]
  data_list=np.array(data_list)
  print 'X[:,:,0]结果输出为：'
  print data_list[:,:,0] 
  print 'X[:,:,1]结果输出为：'
  print data_list[:,:,1]
  print 'X[:,:,m:n]结果输出为：'
  print data_list[:,:,0:1]
  
  
if __name__ == '__main__':
  simple_test()

The output of X[:,0] is:
[1 1 3 4 5 6 6 0 4 2 5 9 3]
The output of X[:,1] is:
[2 2 4 5 6 7 7 4 6 9 8 7 7]
The result output of X[:,m:n] is:
[[1]
 [1]
 [3]
 [4]
 [5]
 [6]
 [6]
 [0]
 [4]
 [2]
 [5]
 [9]
 [  3 ]  ] The  result  output
of ]  [ 8 1 3 7 43]  [ 1 1 3 7 4]  [ 1 1 3 17 4]  [11 1 3 7 4]] X[:,:,1] The result output is: [[ 2 0 4 9 0]  [4 5 6 9 0]  [2 8 5 3 6]













 [ 1 2 5 6 8]
 [ 2 3 5 67 4]
 [ 2 9 43 3 0]
 [ 22 2 42 29 20]
 [ 5 20 24 9 10]
 [ 2 110 14 4 2]]
X[:,:,m :n] The result output is:
[[[ 1]
  [ 1]
  [ 3]
  [ 7]
  [ 4]]
 
 [[ 1] [ 1]
  [ 3]
  [
  8]
  [ 5]]
 
 [[ 8]
  [ 1]
  [ 3]
  [ 7]
  [ 4]]
 
 [[ 1]
  [ 1]
  [ 3]
  [ 7] [ 7]
  ]
 
 [[ 9]
  [ 1]
  [ 3] [
  7]
  [ 4]]
 
 [[ 8]
  [ 1]
  [3]
  [7]
  [43]]
 
 [[ 1]
  [ 1]
  [ 3]
  [ 7]
  [ 4]]
 
 [[ 1]
  [ 1]
  [ 3]
  [17]
  [ 4]]
 
 [[11]
  [ 1]
  [ 3]
  [ 7]
  [ 4]]]
[Finished in 0.6s]

Thanks: https://www.jb51.net/article/180301.htm

pic_gray = pic_and[:, :, 0]
# print("pic_gray:",pic_gray)
cv2.imshow("gray_pic",pic_gray)

---

• cv2.blur(): mean filtering

dst = blur(src, ksize, dst=None, anchor=None, borderType=None)

Mean filter, also called low-pass filter
. As the name suggests, the mean filter averages the data in the filter core and then assigns this value to the core position of the matrix.
The mean filter can be implemented using the cv2.blur() method

src:image

ksize: Filter kernel size, represented by tuples, such as (a, b) a represents height and b represents width.

anchor: wave core anchor point

borderType: border type

Comparison of mean filtering effects:

Original picture:

3*3 filter:

dst1 = cv2.blur(img, (3, 3))

5*5 filter:

dst2 = cv2.blur(img, (5, 5))

10*10 filter:

dst3 = cv2.blur(img, (10, 10))

It can be seen that the larger the filter kernel size, the blurr the image.

Thanks to: OpenCV Filter Longmen Grottoes [Python-Open_CV Series (9)] (Mean Filter, Median Filter, Gaussian Filter, Bilateral Filter)_Hou Xiaojiu's Blog-CSDN Blog

pic_lowblur = cv2.blur(pic_gray, (9, 9))
cv2.imshow("lowblur_pic",pic_lowblur)

 Perform 9*9 mean filtering to complete image smoothing and reduce noise.

"Introduction to Robotics--Analysis, Control and Application" Saeed B.Niku

9.12.1 Neighborhood averaging using convolution masks

• "It is possible to create some masks that act as low-pass filters to attenuate the high-frequency parts of the image without changing the low-frequency parts, thus achieving the purpose of noise reduction."
• "The mask acts as a low-pass filter, attenuating obvious differences between adjacent pixels but having little effect on pixels of lesser intensity."
• "The histogram of the image also changes accordingly...the edges of the image become smoother, making the processed image softer and less focused."
• "As the noise is eliminated, the edges of the object also become blurred"----so the median filter is introduced

For other median filters cv2.medianBlur(), Gaussian filter cv2.GussianBlur(), and bilateral filters cv2.bilateralFilter(), please refer to:

OpenCV filter Longmen Grottoes [Python-Open_CV series (9)] (mean filter, median filter, Gaussian filter, bilateral filter)_Hou Xiaojiu's blog-CSDN blog

---

• cv2.threshold(): simple threshold function

The function of the threshold is to process the grayscale value of the image according to the set value. For example, pixels whose grayscale is greater than a certain value are retained. Thresholds and related algorithms can be used to capture specific graphics from images, such as background removal.

The threshold related functions in cv2 are:

    Ordinary threshold function threshold
    Adaptive threshold function adaptivthreshold

First introduce the simple threshold function: cv2.threshold(src, thresh, maxval, type[, dst]), the return value is retval, dst

Among them:
src is the grayscale image
thresh is the starting threshold
maxval is the maximum value
type defines how to process the relationship between the data and the threshold. There are the following types:

Options	pixel value>thresh	Other cases
cv2.THRESH_BINARY	maxval	0
cv2.THRESH_BINARY_INV	0	maxval
cv2.THRESH_TRUNC	thresh	Current gray value
cv2.THRESH_TOZERO	Current gray value	0
cv2.THRESH_TOZERO_INV	0	Current gray value

Other values ​​are:

• cv2.THRESH_OTSU
• cv2.THRESH_TRIANGLE
• cv2.THRESH_MASK

cv2.THRESH_OTSU uses the least squares method to process pixels, while cv2.THRESH_TRIANGLE uses the trigonometric algorithm to process pixels. In general, cv2.THRESH_OTSU is suitable for bimodal plots. cv2.THRESH_TRIANGLE is suitable for unimodal plots. A unimodal or bimodal graph refers to a grayscale histogram.

Thanks to: CV2 simple threshold function: cv2.threshold()_Fenghua Mingyuan's blog-CSDN blog_cv2.threshold()

    (_, pic_thresh_binary) = cv2.threshold(pic_lowblur, 90, 255, cv2.THRESH_BINARY)
    cv2.imshow("pic_binary",pic_thresh_binary)

This will set pixels with pixel values ​​above 90 to maxval

---

•      cv2.getStructuringElement: Get structuring element

Mat cv::getStructuringElement	(	int	shape,
		Size	ksize,
		Point	anchor = Point(-1,-1)
	)

Python:
	cv.getStructuringElement(	shape, ksize[, anchor]	) ->	retval

Returns a structuring element of a specified size and shape for morphological operations.
This function constructs and returns structural elements that can be further passed to erosion, dilation, or morphology. But you can also construct arbitrary binary masks yourself and use them as structure elements.

The parameter
shape element can be one of the shapes of MorphShapes
ksize The size of the structural element.
Anchor the anchor position within the component. The default value (−1.−1) means the anchor point is at the center. Note that only the shape of the cross member depends on the anchor position. In other cases, the anchor only adjusts the offset of the result of the morphological operation.

MorphShapes: Shapes of structural elements

MORPH_RECT  Python: cv.MORPH_RECT	a rectangular structuring element: No=1
MORPH_CROSS  Python: cv.MORPH_CROSS	a cross-shaped structuring element: Eij={10if i=anchor.and orj=anchor.xotherwise
MORPH_ELLIPSE  Python: cv.MORPH_ELLIPSE	an elliptic structuring element, that is, a filled ellipse inscribed into the rectangle Rect(0, 0, esize.width, 0.esize.height)

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25, 25))

---

• morphologyEx(): perform advanced morphology transformation

void cv::morphologyEx	(	InputArray	src,
		OutputArray	dst,
		int	op,
		InputArray	kernel,
		Point	anchor = Point(-1,-1),
		int	iterations = 1,
		int	borderType = BORDER_CONSTANT,
		const Scalar &	borderValue = morphologyDefaultBorderValue()
	)

The function cv::morphologyEx can perform advanced morphological transformations using erosion and dilation as basic operations.

Any operation can be done in place. For multi-channel images, each channel is processed independently.

Parameters

src	Source image. The number of channels can be arbitrary. The depth should be one of CV_8U, CV_16U, CV_16S, CV_32F or CV_64F.
dst	Destination image of the same size and type as source image.
op	Type of a morphological operation, see MorphTypes
kernel	Structuring element. It can be created using getStructuringElement.
anchor	Anchor position with the kernel. Negative values mean that the anchor is at the kernel center.
iterations	Number of times erosion and dilation are applied.
borderType	Pixel extrapolation method, see BorderTypes. BORDER_WRAP is not supported.
borderValue	Border value in case of a constant border. The default value has a special meaning.

pic_morphologyEx = cv2.morphologyEx(pic_thresh_binary, cv2.MORPH_CLOSE, kernel)
cv2.imshow("morphologyEx_pic",pic_morphologyEx)

---

• cv2.erode():腐蚀操作

void cv::erode	(	InputArray	src,
		OutputArray	dst,
		InputArray	kernel,
		Point	anchor = Point(-1,-1),
		int	iterations = 1,
		int	borderType = BORDER_CONSTANT,
		const Scalar &	borderValue = morphologyDefaultBorderValue()

使用特定的结构元素侵蚀图像。

The function erodes the source image using the specified structuring element that determines the shape of a pixel neighborhood over which the minimum is taken:

dst(x,y)=min(x′,y′):element(x′,y′)≠0src(x+x′,y+y′)

The function supports the in-place mode. Erosion can be applied several ( iterations ) times. In case of multi-channel images, each channel is processed independently.

Parameters

src	input image; the number of channels can be arbitrary, but the depth should be one of CV_8U, CV_16U, CV_16S, CV_32F or CV_64F.
dst	output image of the same size and type as src.
kernel	structuring element used for erosion; if element=Mat(), a 3 x 3 rectangular structuring element is used. Kernel can be created using getStructuringElement.
anchor	position of the anchor within the element; default value (-1, -1) means that the anchor is at the element center.
iterations	number of times erosion is applied.
borderType	pixel extrapolation method, see BorderTypes. BORDER_WRAP is not supported.
borderValue	border value in case of a constant border

pic_erode = cv2.erode(pic_morphologyEx, None, iterations=4)
cv2.imshow("erode_pic",pic_erode)

 

---

•  cv2.dilates():扩张操作

void cv::dilate	(	InputArray	src,
		OutputArray	dst,
		InputArray	kernel,
		Point	anchor = Point(-1,-1),
		int	iterations = 1,
		int	borderType = BORDER_CONSTANT,
		const Scalar &	borderValue = morphologyDefaultBorderValue()
	)

使用特定的结构元素放大图像。

The function dilates the source image using the specified structuring element that determines the shape of a pixel neighborhood over which the maximum is taken:

dst(x,y)=max(x′,y′):element(x′,y′)≠0src(x+x′,y+y′)

The function supports the in-place mode. Dilation can be applied several ( iterations ) times. In case of multi-channel images, each channel is processed independently.

Parameters

src	input image; the number of channels can be arbitrary, but the depth should be one of CV_8U, CV_16U, CV_16S, CV_32F or CV_64F.
dst	output image of the same size and type as src.
kernel	structuring element used for dilation; if elemenat=Mat(), a 3 x 3 rectangular structuring element is used. Kernel can be created using getStructuringElement
anchor	position of the anchor within the element; default value (-1, -1) means that the anchor is at the element center.
iterations	number of times dilation is applied.
borderType	pixel extrapolation method, see BorderTypes. BORDER_WRAP is not suported.
borderValue	border value in case of a constant border

pic_dilate = cv2.dilate(pic_erode, None, iterations=4)
cv2.imshow("dilate_pic",pic_dilate)

 Key references:  OpenCV: OpenCV modules https://docs.opencv.org/4.5.5/index.html