Reference: Gu Yue College and ROS robot development practice
Goal : To realize the ROS system to read the image of the camera, the image data read by ROS is converted into the image in opencv, opencv processes the received image, and finally returns it to the ROS system for visual output.
Install the opencv library and related interface packages
Since I use the ROS-Melodic version, roscore can only be executed in python2, and the vision part needs to be executed in python3, so the package is installed in two pythons. (Important operation, because other vision-related libraries, such as pytorch, require python3. If the default environment is python2 and the corresponding package is not installed into python3, a missing dependency error will be reported.)
(1 message) ROS modification: ubuntu system changes the default python version (important operation 
) 1001.2014.3001.5501 The specific method of switching the python version is in the above article.
sudo apt-get install ros-melodic-vision-opencv libopencv-dev ros-melodic-vision-opencv python-opencvDownload the corresponding source code and put it in the src of the workspace
guyueclass/planning&perception/robot_vision_beginner/robot_vision at main guyuehome/guyueclass (github.com) https://github.com/guyuehome/guyueclass/tree/main/planning%26perception/robot_vision_beginner/robot_vision this Guyue Academy and ROS robot development practice source code.
running code
The first is usb_cam.launch
<launch>
<node name="usb_cam" pkg="usb_cam" type="usb_cam_node" output="screen" >
<param name="video_device" value="/dev/video0" />
<param name="image_width" value="640" />
<param name="image_height" value="480" />
<param name="pixel_format" value="yuyv" />
<param name="camera_frame_id" value="usb_cam" />
<param name="io_method" value="mmap"/>
</node>
</launch>Create a node, and then serve as the publisher, the image information of the camera, the parameters must correspond to the camera parameters, or an error will not be reported.
Followed by cv_bridge_test.py
Among them, the python version declaration needs to be modified, because the default python version of the molodic version is python2, but now the visual algorithms are all used in python3, which has practical significance in python3.
Modified statement:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import rospy
import cv2
from cv_bridge import CvBridge, CvBridgeError
from sensor_msgs.msg import Image
class image_converter:
def __init__(self):
# 创建cv_bridge，声明图像的发布者和订阅者
self.image_pub = rospy.Publisher("cv_bridge_image", Image, queue_size=1)
self.bridge = CvBridge()
self.image_sub = rospy.Subscriber("/usb_cam/image_raw", Image, self.callback)
def callback(self,data):
# 使用cv_bridge将ROS的图像数据转换成OpenCV的图像格式
try:
cv_image = self.bridge.imgmsg_to_cv2(data, "bgr8")
except CvBridgeError as e:
print(e)
# 在opencv的显示窗口中绘制一个圆，作为标记
(rows,cols,channels) = cv_image.shape
if cols > 60 and rows > 60 :
cv2.circle(cv_image, (60, 60), 30, (0,0,255), -1)
# 显示Opencv格式的图像
cv2.imshow("Image window", cv_image)
cv2.waitKey(3)
# 再将opencv格式额数据转换成ros image格式的数据发布
try:
self.image_pub.publish(self.bridge.cv2_to_imgmsg(cv_image, "bgr8"))
except CvBridgeError as e:
print(e)
if __name__ == '__main__':
try:
# 初始化ros节点
rospy.init_node("cv_bridge_test")
rospy.loginfo("Starting cv_bridge_test node")
image_converter()
rospy.spin()
except KeyboardInterrupt:
print("Shutting down cv_bridge_test node.")
cv2.destroyAllWindows()According to the calculation graph generated by rqt_graph, /usb_cam is the publisher node generated by the launch file, and /cv_bridge_test is the node cv_bridge_test initialized if __name__ == '__main__': as a subscriber.
import cv2
from cv_bridge import CvBridge, CvBridgeError
from sensor_msgs.msg import Imagecv2 - opencv library
cv_bridge——Convert ROS image (imgmsg) and opencv image (cv2) by calling the api in this function.
sensor_msgs.msg——sensor data type, where the data type of Image image is imported here.
self.image_pub = rospy.Publisher("cv_bridge_image", Image, queue_size=1)
self.bridge = CvBridge()
self.image_sub = rospy.Subscriber("/usb_cam/image_raw", Image, self.callback)According to the calculation graph above:
The first is the subscriber, whose role is to receive the original image information published by usb_cam , the topic name /usb_cam/image_raw , the topic name cannot be changed here, because in the topic communication, the topic names of the publisher and the subscriber must be the same to communicate.
The second is the publisher. Note that this publisher is not the same as the above subscriber, because the topic names of the two are different, which means that the two cannot communicate on the topic. Among them, the topic cv_bridge_image published by the publisher has a data type of Image , which can be observed through the rqt_image_view command.
self.bridge = CvBridge() defines a handle, and then can flexibly call related conversion interfaces.
def callback(self,data):
# 使用cv_bridge将ROS的图像数据转换成OpenCV的图像格式
try:
cv_image = self.bridge.imgmsg_to_cv2(data, "bgr8")
except CvBridgeError as e:
print(e)
# 在opencv的显示窗口中绘制一个圆，作为标记
(rows,cols,channels) = cv_image.shape
if cols > 60 and rows > 60 :
cv2.circle(cv_image, (60, 60), 30, (0,0,255), -1)
# 显示Opencv格式的图像
cv2.imshow("Image window", cv_image)
cv2.waitKey(3)
# 再将opencv格式额数据转换成ros image格式的数据发布
try:
self.image_pub.publish(self.bridge.cv2_to_imgmsg(cv_image, "bgr8"))
except CvBridgeError as e:
print(e)
This is the callback function, which is the parameter of the topic communication subscriber, and the callback function is called through the last parameter.
self.image_sub = rospy.Subscriber("/usb_cam/image_raw", Image, self.callback)At the same time, in the main function, the spin turns back non-stop to achieve the purpose of non-stop cyclic subscription.
rospy.spin()In the callback function:
In the first step, ROS original image information is converted into image information in opencv.
cv_image = self.bridge.imgmsg_to_cv2(data, "bgr8")The second step, image processing in opencv
# 在opencv的显示窗口中绘制一个圆，作为标记
(rows,cols,channels) = cv_image.shape
if cols > 60 and rows > 60 :
cv2.circle(cv_image, (60, 60), 30, (0,0,255), -1)
# 显示Opencv格式的图像
cv2.imshow("Image window", cv_image)
cv2.waitKey(3)
In the third step, the image processed by opencv is converted back to ROS image information and released at the same time. The released image information can be viewed through ROS commands.
self.image_pub.publish(self.bridge.cv2_to_imgmsg(cv_image, "bgr8"))Analysis is complete, run the code
roslaunch robot_vision usb_cam.launchrosrun robot_vision cv_bridge_test.pyrqt_image_viewIf the second part is the ROS-melodic version, it will definitely report an error. Of course, it can be solved by modifying the python version declaration and running with python2, but now python2 has been eliminated, and it is meaningless even if it runs successfully.
ROS error: cv_bridge error in ROS-Melodic method.
operation result:
This is the image output by opencv .
This is the converted and released image, where the topic name is cv_bridge_image , which is traceable and logical in the code.
This is the raw image, the topic name is /usb_cam/image_raw , and the image data type message that the subscriber receives in the code. (rqt_image_view)
This is the calculation graph (rqt_graph), in which only two nodes are seen