Autopilot perception inventory system
Perception (sensing) system is a high precision map data as input various sensors, and processing through a series of calculations, the surrounding environment of the vehicle accurately senses the autopilot system.
It can provide a wealth of information to the downstream module, including the position of the obstacle, the shape, type and speed information, including semantic understanding of some special scenarios (e.g., construction zones, traffic lights and traffic signs, etc.).
Composition and sub-systems of perception
◆ sensors: a sensor related to the installation, viewing angle, distance detection, data throughput, calibration accuracy, time synchronization. Because more sensors autopilot used, time synchronization solutions is critical.
◆ target detection and classification: In order to ensure the safety autopilot, perception system needs to reach approximately one hundred percent recall rate and a very high accuracy rate. Target detection and classification often involves working depth learning, including 3D point cloud and object detection on the (Pictures) 2D Image and depth of multi-sensor fusion.
◆ multi-target tracking: follow-up information on multiple frames to calculate and predict the trajectory of the obstacle.
◆ scene understanding: including traffic lights, road signs, construction area, as well as special categories, such as school buses, police cars.
◆ machine learning and distributed training infrastructure related evaluation system
◆ data: a large number of annotation data here, including 3D point cloud data and the 2D image data.
Detailed sensor sensors are currently autopilot applications are divided into three categories: laser radar ( LiDAR), camera ( Camera), millimeter-wave radar ( Radar).
Mentioned at the beginning, the sensing system for a variety of sensor input data and high-precision map, while the figure shows the output sensing system for object detection, i.e., capable of detecting an obstacle around the vehicle, such as vehicles, pedestrians, bicycles, etc., combined with high-precision map, the system will be perceived on the surrounding background (environmental context) information output.
As shown above, the block representative of a passenger green color, orange for a motorcycle, yellow for a pedestrian, gray is detected environmental information, such as vegetation.
Sensing system incorporates a plurality of frames of information (upper panel), but also on the speed of the vehicle and pedestrian movement, direction, and so precise trajectory prediction output.
2 sensor is configured with a multi-sensor fusion depth
Learn about the perceptual system from input to output of general introduction, Next, I briefly introduce the third generation pony Chi autopilot system PonyAlpha sensor installation programs and solutions for multi-sensor fusion depth.
Sensor mounting scheme is currently a perceived distance PonyAlpha sensor mounting scheme can cover the peripheral drive 360 degrees, the range of less than 200m.
Specifically, this embodiment uses three laser radar, the top and sides of the vehicle. Meanwhile, the plurality of wide-angle cameras to cover the field of view of 360 degrees. View distant aspect, the millimeter-wave radar and forward telephoto camera to expand the perceived distance range of 200 meters, so that it can detect more distant object information. This sensor configuration can ensure that our autonomous vehicles autopilot In such a scenario residential, commercial, industrial zone.
Multi-sensor depth of integration solutions for basic multi-sensor depth of integration of multi-sensor depth of integration programs primary solution is to calibrate data from different sensors into a single coordinate system, including the camera's internal reference calibration, laser radar to external camera parameters calibration, millimeter-wave radar to reach the GPS external reference calibration like. Sensor fusion is an important prerequisite for the calibration accuracy to achieve a very high level, regardless of the outcome of the level sensor fusion or metadata-level sensor fusion, this is the necessary foundation.
By the figure you will find that our system will perceive a laser precise 3D point cloud projected onto the image, the visible sensor calibration accuracy is sufficiently high. Calibration solutions of different sensors entire sensor calibration work basically be completely automated way.
First, the camera is calibrated internal reference (above), which is for correcting image distortion caused by the characteristics of the camera itself and the like. Internal reference calibration camera platform so that each camera sensor calibration can be completed within two to three minutes.
Followed by laser radar and external reference calibration of GPS / IMU (above), the original data is based on radar laser radar coordinate system, so we need to point the radar coordinate system converted from the world coordinate system, which involves radar and laser calculating a positional relationship between the GPS / IMU opposite. Our calibration tool through optimized solution that can quickly find the optimal position in relation outdoors.
The third is the integration of the camera to the laser radar (above). Lidar perceived environment 360-degree rotatable manner, per revolution is 100 ms, while the camera is a momentary exposure, in order to ensure the rotation of the camera to ensure exposure of the laser radar synchronization requires both time synchronization, i.e., by Lidar to trigger the camera exposure. For example, the corresponding position of the camera may be triggered by the exposure time of the position information of the laser radar, in order to achieve precise synchronization of the camera and the laser radar. 3D (lidar) and 2D (camera) complementary to each other, so that both can be perceived better integration to obtain a more accurate output.
Finally, the millimeter-wave radar ( Radar) and GPS / IMU calibration (above), is also the Radar data from the Local (local) coordinate system is converted into the world coordinate system, we will pass the real to calculate the 3D environment Radar and the relative positional relationship between the GPS / IMU's. Good calibration results sensing system is given to ensure that the lane information obstacle within 200 meters from the vehicle (e.g., located within a lane, or the lane line pressure, etc.) and the like. The following demo video vividly concisely shows a partial depth of the treatment effect multisensor fusion.
3 -vehicle sensing system architecture of the vehicle-mounted sensing system architecture is what? What is its solution?
The figure shows the architecture of the whole system of perception car. First a laser radar, a camera, a millimeter wave radar sensor three kinds of data required for time synchronization, error control all the time in milliseconds. Combined sensor data, sensing system frame basis ( Frame-based) detection ( Detection), segmentation ( Segmentation), classification ( Classification) computing, using the multi-frame information in the last multi-target tracking, the correlation result output. This process will involve multi-sensor fusion depth technical details and depth of learning-related, I do not do too much discussion here. Perceptual system solution should guarantee the following five points:
◆ The first is security, almost one hundred percent guarantee of detection ( Detection) recall ( Recall).
◆ Precision ( Precision) is very high, if below a certain threshold, causing False Positive (false), it will cause the vehicle to travel very uncomfortable in autopilot.
◆ try to be helpful to all output traffic information, including road signs, traffic lights and other scene understanding.
◆ ensure the efficient operation of the sensing system can handle a large number of near real-time sensor data.
◆ Scalability ( Scalability) is also important. Deep learning ( Deep Learning) rely on large amounts of data, generalization of their training is very important for perception model system. The future, we hope that model ( Model) and new algorithms capable of adapting road more cities and countries.
4 challenge perception technology
Perception Challenge precision and recall balanced
The figure shows the evening peak period crossroads of busy scene, this time a large number of pedestrians, motorcycle through the intersection.
By 3D point cloud data (above), can be seen at this time the corresponding original data sensing. Here the challenge is that after the calculation process, perception system needs to output the correct division all obstacles (in this environment segmentation) results and obstacles categories. In addition to a busy intersection, perceptual system in some special treatment or adverse weather conditions, also faces no small challenge.
Dump heavy rain or prolonged rains often cause surface water, the vehicle passes naturally splash. If the perception of the spray system can not accurately identify and filter, which will cause the autopilot trouble. And laser radar and the camera ( data Lidar & Camera), and our perception of the spray system has a high recognition rate. Nagao scene challenge sprinkler on the map is two types of sprinkler (on the map) we had encountered in the road test. Mist spray gun sprinkler using left upward, whereas the right side is sprayed to both sides of the sprinkler.
Human encounters sprinkler driver can easily judge and more than sprinkler, but the perception system, you need to take some time to identify and deal with such scenes and vehicles, our autopilot encountered a similar scene It has been better ride experience. Detecting small objects
Meaning that a small object detection, face unexpected road test events, such as stray cats, suddenly appeared on the road, perception systems for such small objects can have accurate recall, in order to ensure safe little life.
traffic light
To carry out as more and more regions and countries autopilot drive test, perceptual system in dealing with traffic lights will always encounter new long tail scene.
For example, the problem of backlight (upper panel) or suddenly after exit from the bridge opening in the camera exposure problems, we can solve the problem by dynamically adjusting the exposure or the like of the camera.
There are scenes of traffic lights countdown (above), perceptual system can recognize the digital countdown, this can allow autonomous vehicles in the face of the yellow / front gives better planning decisions to respond, optimize travel experience.
When rain, the camera (the camera) will drops clouds (above), the system needs to deal with perception of the scene under such special climatic conditions, accurate identification of the traffic lights.
