This article briefly introduces the Point Cloud Library (PCL), an open-source library for processing 2D and 3D data, such as LiDAR point clouds. By familiarizing yourself with some basic knowledge of using PCL, you can use PCL for subsequent positioning. It mainly covers the following contents:
Point cloud data Point Cloud Data (PCD) file
点云库The Point Cloud Library(PCL)
PCL Viewer (Viewer)
Create and use lidar objects in PCL
Templates and different point clouds
Adjust the LiDAR parameters to check the point cloud in PCL
1. Point cloud data (PCD) file
The storage format of lidar data is called point cloud data (PCD). The pcd file is a list of Cartesian coordinates (x, y, z) and intensity value i, which is a single snapshot after each scan of the environment. This means that for a VLP 64 lidar, the pcd file will have about 256,000 (x,y,z,i) values.
A PCD of a city block, parked cars, and a passing van. Intensity values are displayed in different colors. The black area in the middle is where the car with the lidar sensor is located.
PCD coordinates
The coordinate system of the point cloud data is the same as the local coordinate system of the car. In this coordinate system, the x-axis points to the front of the car and the y-axis points to the left side of the car. Since this coordinate system uses the right-hand rule, the z-axis of the coordinate system points up the car.
2. Point cloud library PCL
In this lesson learn to process point cloud data to find obstacles. All code will be done in a c++ environment, PCL is an open source c++ library for working with point clouds, use it to visualize data, render shapes, and some built-in processing functions. Some documentation of PCL can be found here: Point Cloud Library | The Point Cloud Library (PCL) is a standalone, large scale, open project for 2D/3D image and point cloud processing.
PCL is widely used in the robotics community for processing point cloud data, and there are many tutorials online for using it. There are many built-in functions in PCL that can help detect obstacles, such as segmentation, extraction, and clustering.
3. Startup code
All the code (and the content about obstacle detection) are included in the GitHub repository, you can clone the repo and run it on your computer according to the README. Reference link: GitHub - udacity/SFND_Lidar_Obstacle_Detection . It mainly contains two main files, namely environment.cpp and processPointClouds.cpp. The environment.cpp file contains the main function, which generates a runnable executable. The processPointClouds.cpp file contains placeholders for all functions that process pcds. There are also sensors/lidar.h, which is used to simulate lidar perception and create point cloud data; render.cpp and render.h, which have functions for rendering objects to the screen.
Code structure
Top level: CMakeLists.txt
Readme
src/
render/
box.h - structure definition of box object
render.h
render.cpp - this file together with the header file defines the classes and methods of the render object.
sensors/
data/ - This directory contains the PCD data used in this paper.
lidar.h - has functions to create pcds using rays.
environment.cpp - main file: process and visualize pcds using the PCL viewer.
processPointClouds.h
processPointClouds.cpp - Functions for filtering, segmenting, clustering, binning, loading and saving pcds.
4. Compile the lidar simulator
Compilation guide in Ubuntu environment (Ubuntu 16.04)
-
克隆github repo-https://github.com/udacity/SFND_Lidar_Obstacle_Detection :
cd ~ git clone https://github.com/udacity/SFND_Lidar_Obstacle_Detection.git
-
Edit CMakeLists.txt as follows:
cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
add_definitions(-std=c++14)
set(CXX_FLAGS "-Wall")
set(CMAKE_CXX_FLAGS, "${CXX_FLAGS}")
project(playback)
find_package(PCL 1.11 REQUIRED)
include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})
list(REMOVE_ITEM PCL_LIBRARIES "vtkproj4")
add_executable (environment src/environment.cpp src/render/render.cpp src/processPointClouds.cpp)
target_link_libraries (environment ${PCL_LIBRARIES})
3. Execute the following command in the terminal
sudo apt install libpcl-dev cd ~/SFND_Lidar_Obstacle_Detection mkdir build && cd build cmake .. make ./environment
Execute ./environment, and see a pop-up window, the picture is as follows:
A simple highway simulation environment, the cars in the middle lane are green (customizable cars), other traffic cars are blue. Everything is rendered using PCL with simple boxes, lines and colors. You can use the mouse to move around the environment, try holding the left mouse button to rotate around the scene, hold the middle mouse button to move the scene; to zoom, use the middle mouse scroll key or the right mouse button while moving.
5. PCL Viewer
A pcl viewer is created in environment.cpp, the viewer is used to handle the visualization of all objects on the screen. The functions that use the pcl viewer in environment.cpp are initCamera and initHighway. The initCamera function is used to set different viewing angles in the window. There are 5 different options: XY, TopDown, Side and FPS---XY provides 45 degrees Perspective, FPS is the first-person perspective, giving people a feeling of sitting in the driver's seat of a car. In addition, the rendering function also makes extensive use of the viewer, note that the viewer is usually passed in as a reference. This way, the process is much more streamlined since nothing needs to be returned.
6. Create the lidar object
The lidar object is defined by the header file of src/sensors/lidar.h and included at the top of environment.cpp. There is also a simpleHighway function in environment.cpp that accepts a reference parameter to the PCL visualizer viewer.
Instantiate a pointer to the Lidar object in the simpleHighway function. Use the new keyword to create a Lidar pointer object on the heap. The Lidar constructor has two parameters: the car and the slope of the ground - these parameters are necessary for modeling light collisions. The created Lidar object should have a slope of 0.
Note that
the lidar parameter is necessary to model ray collisions. LiDAR objects will hold point cloud data, which can be very large. By instantiating on the heap, we can use more memory than 2MB on the stack. However, object lookups take longer on the heap, whereas stack lookups are very fast.
In environment.cpp, simpleHighway function:
/ TODO: Create lidar sensor
Lidar* lidar = new Lidar(cars, 0);
7. Using lidar objects
To further use the newly created Lidar object, have a look at src/sensors/Lidar.h to see how everything is defined. In this header file, the ray object is defined. LiDAR will use these rays to perform ray casting to perceive the surrounding environment. The scan function inside the Lidar structure performs ray casting.
Now call the lidar scan function to see what the lidar rays look like. Back in the environment file, after calling the Lidar constructor, you can use the scan function to render the lidar rays.
LiDAR Sensing
To create a point cloud, call the lidar scan() method on the lidar object.
The result is stored in a PointCloud pointer object, the
point type of pcl::PointCloud<pcl::PointXYZ>::Ptr PointCloud will be pcl::PointXYZ.
Call the renderRays function with the resulting PointCloud pointer.
Note that
PointCloud's template syntax is similar to that of vector or other standard container libraries: ContainerName<ObjectName>.
The Ptr type from PointCloud indicates that the object is actually a pointer—a 32-bit integer containing the memory address of the PointCloud object. Many functions in pcl take point cloud pointers as arguments, so it is convenient to return inputCloud in this form.
The renderRays function is defined in src/render. It contains functions that allow rendering of points and shapes to the pcl viewer. Use this in the viewer to render lidar rays as line segments.
The parameter of the renderRays function is viewer, which is passed in by reference. This means that any changes to the viewer in the body of the renderRays function will directly affect the viewer outside the scope of the function. The position of the lidar is also passed, along with the point cloud generated by the scan function. The point type of PointCloud will be pcl::PointXYZ. We'll also cover some other different types of point clouds.
In environment.cpp, in the simpleHighway function, add two more lines after the previous lidar object instantiation:
// TODO: Create lidar sensor
Lidar* lidar = new Lidar(cars, 0);
pcl::PointCloud<pcl::PointXYZ>::Ptr inputCloud = lidar->scan();
renderRays(viewer, lidar->position, inputCloud);8. Templates and different point cloud data
Why use templates?
The lidar scan function used earlier produced a pcl PointCloud object with pcl::PointXYZ points. Objects use templates because there are many different types of point clouds: some are 3D, some are 2D, and some include color and intensity. A normal 3D point cloud is used here, so PointXYZ is used. However, there are also some points that have an intensity component. Instead of defining two separate functions, one with a parameter of PointXYZ and the other with a parameter of PointXYZI, the template can automate this process. With templates, you only need to write the function once, and use the template as a parameter to specify the point type.
templates and pointers
If you haven't used templates with pointers before, you may have noticed in your code that whenever you use a template-dependent pointer, typename is used. For example here's the function signature:
typename pcl::PointCloud<PointT>::Ptr ProcessPointClouds<PointT>::FilterCloud(typename pcl::PointCloud<PointT>::Ptr cloud, float filterRes, Eigen::Vector4f minPoint, Eigen::Vector4f maxPoint)The reason for this is as follows: given a piece of code with a type name parameter, such as pcl::PointCloud<PointT>::Ptr, without knowing the value of the type name parameter, the compiler cannot determine that this piece of code is a value Still type. The compiler will assume that the code represents a value. If the code actually means typename, then typename needs to be specified.
Nine, adjust the lidar parameters
You can rotate and move around the scene to see the different lights being cast. However, your current LiDAR setup will limit what you can do. The resolution is very low, as seen in the scene, only one ray touches a car. So by increasing the resolution of the lidar, you can clearly see other vehicles around you.
These include increasing the minimum distance so that the contact point with the roof is not included; as well as improving the resolution of the horizontal and vertical angles and adding noise. The added noise is actually quite high since the units are meters, but results in much more interesting and realistic point data in the scene. Feel free to experiment and tune these lidar hyperparameters!
Improve the resolution of lidar by increasing the number of vertical layers and the angular resolution around the z-axis.
numLayers: changed from 3 to 8.
horizontalLayerIncrement: changed from pi/6 to pi/64.
Set minDistance to 5 (meters) to remove points on the roof of the car.
Add noise, around 0.2 for more interesting pcds.
After completing the above settings, the output is as shown in the following figure:
Increase LiDAR Range
10. Detect point cloud
Now you can see what the lidar rays look like, so what about the actual point cloud data that will be used and processed? In fact, you can use the renderPointCloud function to view the point cloud data in rendering, or you can choose to turn off the highway scene Render so you can see what the point cloud looks like by itself.
The results in the image above are noise-free, with lidar minDistance set to zero. When increasing the lidar minDistance, you can remove the points that hit the roof of the car, because these points cannot help detect other vehicles. Also, some noise variance helps create more interesting point clouds, and adding noise will help develop more robust point processing functions.
To achieve the above effect, call renderPointCloud instead of renderRays in the simpleHighway function.
The point cloud can be viewed without hindrance by setting renderScene to false in environment.cpp.
When complete, the output should look like the image below:
Analog PCD
In environment.cpp, in simpleHighway, remove the previous call to renderRays and use renderPointCloud instead:
Lidar* lidar = new Lidar(cars, 0);
pcl::PointCloud<pcl::PointXYZ>::Ptr inputCloud = lidar->scan();
//renderRays(viewer, lidar->position, inputCloud);
renderPointCloud(viewer, inputCloud, "inputCloud"); // You can also use other names than just "inputCloud"Also, in the function, modify renderScene to hide object boundaries:
// RENDER OPTIONS
bool renderScene = false;11. Summary
Through this article, I learned the following:
Point Cloud Data (PCD) files
Point Cloud Library (PCL)
PCL Viewer
Create and use LiDAR object templates in PCL
with different point clouds
Adjust LiDAR parameters
Inspect point clouds in PCL