Smart transportation sand table self-driving car equipped with Baidu Apollo9.0 technology

With the vigorous development of intelligent network transportation, the introduction of Baidu Apollo 9.0 technology has injected new vitality into the field of autonomous driving. This article will provide an in-depth analysis of the Baidu Apollo 9.0 technical framework and discuss how to successfully transplant it to the smart traffic sandbox self-driving car platform to implement the self-driving algorithm.

1. Analysis of Baidu Apollo 9.0 technical framework

First, we will introduce the core technology framework of Baidu Apollo 9.0. The framework includes multiple modules such as perception, planning, and control. It achieves efficient and accurate autonomous driving through a combination of deep learning and traditional algorithms.

1.1 Perception module

The perception module is the eyes of autonomous driving. Its main task is to obtain data from various sensors and generate an understanding of the vehicle's surrounding environment. In Apollo 9.0, deep learning technology has been widely used in the perception module. Deep learning algorithms such as convolutional neural networks (CNN) are used for tasks such as target detection and semantic segmentation, allowing the perception module to efficiently and accurately identify roads, vehicles and pedestrians.

In the sandbox car, due to limited computing resources, we optimized the perception module. We will detail the streamlined design of this module and the strategies to reduce computational cost while maintaining accuracy.

1.2 Planning module

The planning module is the core of path planning and determines how the vehicle should travel from the starting point to the ending point. In Baidu Apollo 9.0, the planning module uses advanced trajectory optimization technology, combining local obstacle avoidance and global path planning, allowing vehicles to respond to various traffic scenarios flexibly and efficiently.

We will deeply analyze the working principle of Apollo 9.0 planning module, including the specific algorithms of local path planning and global path planning. This section will show in detail how the planning module ensures the safety and rapid arrival of vehicles at their destination in complex urban environments.

1.3 Control module

The control module is the most critical link in autonomous driving. It is responsible for adjusting the vehicle's speed, steering and other control signals based on the planned path and real-time perceived environmental information. In Apollo 9.0, the control module comprehensively uses advanced algorithms such as model predictive control (MPC) to ensure that the vehicle can execute the planned path smoothly and accurately.

We will introduce the working principle of the control module in detail, including the generation of control signals, real-time adjustment strategies, etc. Through the in-depth analysis of this module, readers will have a more comprehensive understanding of automatic driving control algorithms.

2. Transplantation and implementation

2.1 Apollo 9.0 transplantation in sandbox car

In the process of transplanting the Baidu Apollo 9.0 technical framework to the smart traffic sandbox self-driving car, we faced many challenges such as hardware adaptation and sensor data fusion. To ensure smooth migration and efficient operation, each step of the migration was discussed and implemented in detail.

2.1.1. Hardware adaptation

There are differences between the hardware of the sandbox car and the real vehicle, so the first task is to adapt the hardware dependencies of Baidu Apollo. Among them, we mainly focus on the following aspects:

2.1.1.1 Sensor adaptation

The data format of the lidar, camera and other sensors on the sand table car may be different from Apollo's default format. We write adapters to convert sensor data into a format that Apollo can recognize and process.

// CyberRT Sensor Adapter

#include <cyber/cyber.h>
#include <cyber/time/time.h>
#include <sensor_msgs/PointCloud2.h>
#include <sensor_msgs/Image.h>

using apollo::cyber::Node;
using apollo::cyber::Time;

class SensorAdapter {
public:
  SensorAdapter() {
    // Initialize CyberRT node
    node_ = apollo::cyber::CreateNode("sensor_adapter");
    
    // Subscribe to LiDAR data
    lidar_sub_ = node_->CreateReader<sensor_msgs::PointCloud2>(
        "/sensor/lidar",
        [this](const std::shared_ptr<sensor_msgs::PointCloud2>& msg) {
          ProcessLidarData(msg);
        });
    
    // Subscribe to camera data
    camera_sub_ = node_->CreateReader<sensor_msgs::Image>(
        "/sensor/camera",
        [this](const std::shared_ptr<sensor_msgs::Image>& msg) {
          ProcessCameraData(msg);
        });
  }

private:
  void ProcessLidarData(const std::shared_ptr<sensor_msgs::PointCloud2>& msg) {
    // Lidar data processing logic
  }

  void ProcessCameraData(const std::shared_ptr<sensor_msgs::Image>& msg) {
    // Camera data processing logic
  }

private:
  std::shared_ptr<Node> node_;
  std::shared_ptr<apollo::cyber::Reader<sensor_msgs::PointCloud2>> lidar_sub_;
  std::shared_ptr<apollo::cyber::Reader<sensor_msgs::Image>> camera_sub_;
};

2.1.1.2 Controller adaptation

The chassis control method of the sandbox car may be different from the default control method of Apollo. By implementing a custom controller, we ensure that control instructions can be sent to the car correctly.

// CyberRT Controller Adapter

#include <cyber/cyber.h>
#include <control_msgs/ControlCommand.h>

using apollo::cyber::Node;
using apollo::cyber::Time;

class ControllerAdapter {
public:
  ControllerAdapter() {
    // Initialize CyberRT node
    node_ = apollo::cyber::CreateNode("controller_adapter");
    
    // Subscribe to control command
    control_sub_ = node_->CreateReader<control_msgs::ControlCommand>(
        "/control/command",
        [this](const std::shared_ptr<control_msgs::ControlCommand>& msg) {
          ProcessControlCommand(msg);
        });
  }

private:
  void ProcessControlCommand(const std::shared_ptr<control_msgs::ControlCommand>& msg) {
    // Control command processing logic
  }

private:
  std::shared_ptr<Node> node_;
  std::shared_ptr<apollo::cyber::Reader<control_msgs::ControlCommand>> control_sub_;
};

2.2. Data fusion

On the sand table trolley, the data generated by various sensors need to be fused to provide more accurate perception information. This involves the fusion of point cloud data and image data to ensure that the planning and control modules can obtain complete environmental information.

2.2.1 Data fusion algorithm

Data fusion algorithms need to combine lidar and camera data to achieve all-round perception of the environment. Here is the simplified code for data fusion:

// CyberRT Data Fusion

#include <cyber/cyber.h>
#include <sensor_msgs/PointCloud2.h>
#include <sensor_msgs/Image.h>
#include <fusion_msgs/FusedPointCloudImage.h>

using apollo::cyber::Node;

class DataFusion {
public:
  DataFusion() {
    // Initialize CyberRT node
    node_ = apollo::cyber::CreateNode("data_fusion");
    
    // Subscribe to LiDAR data
    lidar_sub_ = node_->CreateReader<sensor_msgs::PointCloud2>(
        "/sensor/lidar",
        [this](const std::shared_ptr<sensor_msgs::PointCloud2>& lidar_msg) {
          lidar_data_ = ProcessLidarData(lidar_msg);
          FuseData();
        });
    
    // Subscribe to camera data
    camera_sub_ = node_->CreateReader<sensor_msgs::Image>(
        "/sensor/camera",
        [this](const std::shared_ptr<sensor_msgs::Image>& camera_msg) {
          camera_data_ = ProcessCameraData(camera_msg);
          FuseData();
        });
    
    // Publish fused data
    fused_data_pub_ = node_->CreateWriter<fusion_msgs::FusedPointCloudImage>("/fusion/data");
  }

private:
  // LiDAR data processing logic
  LidarData ProcessLidarData(const std::shared_ptr<sensor_msgs::PointCloud2>& lidar_msg) {
    // ...
  }

  // Camera data processing logic
  CameraData ProcessCameraData(const std::shared_ptr<sensor_msgs::Image>& camera_msg) {
    // ...
  }

  // Data fusion logic
  void FuseData() {
    if (lidar_data_ && camera_data_) {
      // Fusion logic
      fusion_msgs::FusedPointCloudImage fused_msg;
      // Populate fused_msg with fused data
      fused_data_pub_->Write(fused_msg);
    }
  }

private:
  std::shared_ptr<Node> node_;
  std::shared_ptr<apollo::cyber::Reader<sensor_msgs::PointCloud2>> lidar_sub_;
  std::shared_ptr<apollo::cyber::Reader<sensor_msgs::Image>> camera_sub_;
  std::shared_ptr<apollo::cyber::Writer<fusion_msgs::FusedPointCloudImage>> fused_data_pub_;

  LidarData lidar_data_;
  CameraData camera_data_;
};

Through the above hardware adaptation and data fusion work, we successfully transplanted the Baidu Apollo 9.0 technology framework to the smart traffic sandbox self-driving car, achieving accurate perception of data and all-round perception of the environment. The following chapters will delve into the practical application of path planning and control algorithms based on this transplant.

3. Practical application of path planning and control algorithms

In smart traffic sand table autonomous vehicles, path planning and control algorithms are the core to achieve autonomous navigation and obstacle avoidance. Based on the transplantation of Baidu Apollo 9.0 technical framework, we have deeply optimized the path planning and control modules to adapt to the special environment of the sand table car.

3.1. Path planning

The path planning of the sand table car needs to take into account the particularity of the sand table environment, including miniature sand table roads, miniature sand table vehicles and other factors. Based on Baidu Apollo, we have carried out the following path planning module optimization.

3.1.1 Environment modeling

In the self-driving car project of the smart traffic sandbox, environment modeling is a key step to ensure that the self-driving algorithm can run accurately in the actual miniature traffic sandbox. As in the real world, we need to model roads, intersections, traffic signs and other vehicles in the actual miniature sandbox with high accuracy.

road modeling

In order to enable the car to accurately navigate in the miniature sandbox, we accurately modeled the road. This includes information such as the geometry of the road, the location of lane lines, lane widths, and more. With the help of sensors such as laser sensors and cameras, we conduct high-precision three-dimensional point cloud scanning of the road to ensure the authenticity and accuracy of the model.

// 道路建模代码示例

RoadModel BuildRoadModel(const PointCloud& road_points) {
  RoadModel road_model;

  // 道路几何形状建模
  road_model.geometry = FitCurve(road_points);

  // 提取车道线信息
  road_model.lane_markings = ExtractLaneMarkings(road_points);

  // 其他道路特征建模...

  return road_model;
}

Intersection modeling

Intersections are one of the complex parts of Miniature Traffic Sandbox. In order to ensure that the self-driving car can correctly identify the intersection and take corresponding driving decisions, we conducted intersection modeling.

// 交叉口建模代码示例

IntersectionModel BuildIntersectionModel(const PointCloud& intersection_points) {
  IntersectionModel intersection_model;

  // 交叉口几何形状建模
  intersection_model.geometry = FitPolygon(intersection_points);

  // 提取交叉口内部道路信息
  intersection_model.roads = ExtractRoadsInIntersection(intersection_points);

  // 其他交叉口特征建模...

  return intersection_model;
}

Traffic sign modeling

Traffic signs are crucial for self-driving cars to recognize and make decisions. We use computer vision algorithms to identify and model traffic signs in miniature sandboxes in real time.

// 交通标志建模代码示例

TrafficSignModel BuildTrafficSignModel(const CameraImage& image) {
  TrafficSignModel traffic_sign_model;

  // 使用计算机视觉算法识别交通标志
  traffic_sign_model.sign_type = RecognizeTrafficSign(image);

  // 获取交通标志位置信息
  traffic_sign_model.position = LocateTrafficSign(image);

  // 其他交通标志特征建模...

  return traffic_sign_model;
}

Through the above modeling work, we created a realistic and highly restored driving scene in the miniature traffic sandbox. This provides reliable input for the self-driving car's perception and decision-making, allowing it to complete various driving tasks efficiently and accurately in this simulated environment.

3.1.2 Lattice Planner

Lattice Planner is a grid-based path planning algorithm in Apollo. We have improved on this algorithm to adapt to the actual sandbox environment. Here is a code example:

// CyberRT Lattice Planner

#include <cyber/cyber.h>
#include <planning/lattice/lattice_planner.h>

using apollo::cyber::Node;
using apollo::cyber::Time;

class LatticePlanner {
public:
  LatticePlanner() {
    // Initialize CyberRT node
    node_ = apollo::cyber::CreateNode("lattice_planner");
    
    // Subscribe to high-precision map
    map_sub_ = node_->CreateReader<planning::HDMap>(
        "/map/hdmap",
        [this](const std::shared_ptr<planning::HDMap>& map_msg) {
          hd_map_ = ProcessMap(map_msg);
        });
    
    // Subscribe to localization data
    localization_sub_ = node_->CreateReader<localization::LocalizationEstimate>(
        "/localization/pose",
        [this](const std::shared_ptr<localization::LocalizationEstimate>& pose_msg) {
          current_pose_ = ProcessLocalization(pose_msg);
        });
    
    // Subscribe to planning request
    planning_request_sub_ = node_->CreateReader<planning::PlanningRequest>(
        "/planning/request",
        [this](const std::shared_ptr<planning::PlanningRequest>& request_msg) {
          ProcessPlanningRequest(request_msg);
        });
    
    // Publish planning result
    planning_result_pub_ = node_->CreateWriter<planning::ADCTrajectory>("/planning/trajectory");
  }

private:
  HDMap ProcessMap(const std::shared_ptr<planning::HDMap>& map_msg) {
    // HD map processing logic
    return hd_map;
  }

  Pose ProcessLocalization(const std::shared_ptr<localization::LocalizationEstimate>& pose_msg) {
    // Localization data processing logic
    return current_pose;
  }

  void ProcessPlanningRequest(const std::shared_ptr<planning::PlanningRequest>& request_msg) {
    // Planning request processing logic
    planning::ADCTrajectory trajectory = LatticePlanningAlgorithm(hd_map, current_pose, request_msg);
    planning_result_pub_->Write(trajectory);
  }

private:
  std::shared_ptr<Node> node_;
  std::shared_ptr<apollo::cyber::Reader<planning::HDMap>> map_sub_;
  std::shared_ptr<apollo::cyber::Reader<localization::LocalizationEstimate>> localization_sub_;
  std::shared_ptr<apollo::cyber::Reader<planning::PlanningRequest>> planning_request_sub_;
  std::shared_ptr<apollo::cyber::Writer<planning::ADCTrajectory>> planning_result_pub_;

  HDMap hd_map_;
  Pose current_pose_;
};

3.2. Control algorithm

After the path planning obtains the planned trajectory, the control algorithm is responsible for stably and efficiently navigating the car to the destination according to the planned trajectory. For smart transportation sandbox self-driving cars, we have optimized the control algorithm to adapt to the miniature sandbox environment.

3.2.1 MPC controller

Introduction to Model Predictive Control (MPC) Controller

MPC (Model Predictive Control) is an advanced control strategy that optimizes a series of future control inputs by using a system model at each control step. MPC handles constraints better than traditional control methods and performs well in nonlinear and multivariable systems.

Fundamentals of MPC Controllers

The core idea of the MPC controller is to model the dynamic model of the system, predict the system behavior in the future, and calculate the optimal control input at each control step. This optimization problem usually involves minimizing performance indicators, such as vehicle offset, speed deviation, etc.

The general solution steps for MPC are as follows:

System Modeling: Building a mathematical model that describes the dynamic behavior of a system, which can be a linear or nonlinear model.
Prediction: Using a system model to predict the system state for a period of time in the future based on the current state and control inputs.
Performance index definition: Define a performance index to measure how well the system behaves, such as minimizing the error of the vehicle deviating from the trajectory.
Optimization problem solving: Incorporate performance indicators and system constraints into the optimization problem, and obtain the optimal control input by solving this problem.
Execution: Apply calculated optimal control inputs to guide the system onto the predicted trajectory.

Application of MPC in autonomous driving

In autonomous driving systems, MPC is widely used in path planning and vehicle control. By combining high-precision map information, sensing data and planning goals, the MPC controller can achieve efficient and smooth vehicle motion. We used the MPC controller from Apollo and adapted it. Here is a simplified code example:

// CyberRT MPC Controller

#include <cyber/cyber.h>
#include <control/mpc_controller.h>

using apollo::cyber::Node;

class MPCController {
public:
  MPCController() {
    // Initialize CyberRT node
    node_ = apollo::cyber::CreateNode("mpc_controller");
    
    // Subscribe to planning result
    planning_result_sub_ = node_->CreateReader<planning::ADCTrajectory>(
        "/planning/trajectory",
        [this](const std::shared_ptr<planning::ADCTrajectory>& trajectory_msg) {
          current_trajectory_ = ProcessTrajectory(trajectory_msg);
        });
    
    // Subscribe to vehicle state
    vehicle_state_sub_ = node_->CreateReader<canbus::Chassis>(
        "/canbus/chassis",
        [this](const std::shared_ptr<canbus::Chassis>& chassis_msg) {
          current_vehicle_state_ = ProcessChassis(chassis_msg);
        });
    
    // Publish control command
    control_command_pub_ = node_->CreateWriter<control::ControlCommand>("/control/command");
  }

private:
  Trajectory ProcessTrajectory(const std::shared_ptr<planning::ADCTrajectory>& trajectory_msg) {
    // Trajectory processing logic
    return current_trajectory;
  }

  VehicleState ProcessChassis(const std::shared_ptr<canbus::Chassis>& chassis_msg) {
    // Vehicle state processing logic
    return current_vehicle_state;
  }

  void ControlLoop() {
    // MPC control loop
    control::ControlCommand command = MPCControlAlgorithm(current_trajectory, current_vehicle_state);
    control_command_pub_->Write(command);
  }

private:
  std::shared_ptr<Node> node_;
  std::shared_ptr<apollo::cyber::Reader<planning::ADCTrajectory>> planning_result_sub_;
  std::shared_ptr<apollo::cyber::Reader<canbus::Chassis>> vehicle_state_sub_;
  std::shared_ptr<apollo::cyber::Writer<control::ControlCommand>> control_command_pub_;

  Trajectory current_trajectory_;
  VehicleState current_vehicle_state_;
};

Through the optimization of the above path planning and control algorithms, we have achieved more accurate and efficient navigation and driving control on the smart traffic sandbox self-driving car. This system can not only complete diverse scene driving in a miniature sandbox environment, but also provide strong support for the development and testing of actual autonomous vehicles.

4. Teaching and practice

Teaching and practice of intelligent network-connected smart transportation autonomous driving sandbox in higher vocational colleges and universities

Overview

As an advanced teaching tool, the intelligent connected autonomous driving sandbox system is committed to providing a comprehensive and practical teaching experience for students in automotive majors, transportation engineering majors, artificial intelligence majors and other fields in higher vocational colleges and universities. Combined with the self-driving car in the sand table, students can have an in-depth understanding and mastery of the core technology of the self-driving system, and at the same time, through practical operations, transform theoretical knowledge into practical applications.

Features of sand table system

Comprehensive coverage of multiple professional fields: For multiple professional fields such as automotive, transportation engineering, and artificial intelligence, the sandbox system provides comprehensive and in-depth teaching content to meet the needs of students in different majors.
Open source code supports secondary development: The car system in the sandbox uses open source code, providing students with a wealth of academic research and secondary development opportunities. Students can gain an in-depth understanding of the underlying principles of the system through independent development and adaptation.
Distributed program architecture: The sand table system adopts a distributed program architecture, which enables multiple cars to work together and increases the scalability of the system. This also provides students with a practical platform for team collaboration and system integration.

Teaching application scenarios

1. Automobile major

In the automotive major, the sandbox system can be used for teaching in the following aspects:

Automotive Electronic Systems: Students can learn the design and debugging of automotive electronic control units (ECUs), as well as the application of sensors and actuators related to autonomous driving.
Vehicle Dynamics: By simulating the dynamic behavior of a vehicle under various driving scenarios, students can gain an in-depth understanding of the vehicle's handling characteristics.

2. Transportation Engineering Major

Applications of sand table systems in traffic engineering majors include:

Intelligent traffic management: Students can simulate traffic flow through the sandbox, design and test intelligent traffic management algorithms, and improve road traffic efficiency.
Transportation planning: The sand table system can be used to simulate the implementation effects of different transportation planning plans and help students understand the complexity of urban transportation planning.

3. Artificial Intelligence Major

For the artificial intelligence major, the sandbox system has the following applications:

Machine learning and perception: Students can understand the principles of sensors such as lidar and cameras, and perform target detection and tracking through machine learning algorithms.
Path planning and decision-making: The self-driving car in the sandbox provides actual path planning and decision-making scenarios to help students deeply understand the key role of artificial intelligence in self-driving.

4. Other majors

The sand table system is also suitable for other professional fields:

Intelligent Manufacturing Major: Students can learn the principles of automated control and collaborative operations in intelligent manufacturing.
Computer major: The sandbox system provides a wealth of computer vision, perception and control system application scenarios and is suitable for students majoring in computer science.

Practical operations and scientific research projects

Students can participate in practical and research projects through:

System reverse engineering: Students can delve into the source code of the sandbox system, conduct reverse engineering, and understand the implementation principles of each module.
Application of new algorithms: For autonomous driving scenarios, students can try to apply new path planning, perception or decision-making algorithms and verify their performance through experiments.
Teamwork projects: Students can form teams to carry out self-designed practical projects, including system integration, algorithm optimization, etc.

Conclusion

The intelligent network-connected smart transportation autonomous driving sandbox system provides a unique platform for teaching and scientific research in higher vocational colleges and universities due to its openness, practicality and interdisciplinary nature. Through the self-driving car in the sand table, students can be exposed to the most cutting-edge self-driving technology, improve their comprehensive literacy, and cultivate more professional talents for the future development of the smart transportation field.