LiDAR and Automotive Technology Route
For today's cars, intelligent driving is reflected in specific intelligent usage scenarios such as valet parking and different levels of assisted driving. The core representative capability is autonomous driving.
To make a car intelligent, it is necessary to build an intelligent system with the trinity of "perception, decision-making, and execution". Perception is mainly detected by various environmental monitoring sensors installed on the car, decision-making mainly depends on algorithms, software and chips, and execution is completed by various controllers and components.
In autonomous driving scenarios, smart cars need to accurately identify the driving environment such as vehicles, pedestrians, traffic lights, obstacles, etc., which must be realized with the help of on-board sensors such as on-board cameras, ultrasonic radar, millimeter-wave radar, and lidar.
This article refers to the following two articles:
Yanzhi New Energy Vehicle
https://mp.weixin.qq.com/s/yOVWNVepQru0GlDvZG6LHA
and Yiou.com
https://baijiahao.baidu.com/s?id=1715559306625289798&wfr=spider&for=pc
Different on-board sensors have very different characteristics.
The camera can collect external data and perform image recognition according to the algorithm. The technology is mature and the price is cheap. At the same time, image recognition is also an implementation path of artificial intelligence, with greater potential. However, it is easily affected by bad weather, and the effect is poor in the complex environment of backlight and light and shadow, and it is difficult to accurately measure the distance.
Millimeter-wave radars measure distances through millimeter waves and are not affected by weather, with a detection range of up to 200 meters. However, the cost is high, the detection ability of non-metallic objects is weak, and it is difficult to identify pedestrians.
Ultrasonic radar uses ultrasonic ranging, and the cost is low, but the effective detection distance is usually less than 5 meters, the angular resolution is poor, and it is easily affected by the weather.
Lidar uses laser ranging, which has the highest cost, but has a long detection distance and extremely high angle measurement accuracy. It can build a 3D model in real time. There is no effective alternative to this function.
These different on-board sensors have their own advantages and can complement each other to a certain extent. However, for the purpose of realizing intelligent driving, the choices of various car companies are completely different, and basically two types of program routes have been formed:
the first type is represented by Tesla, and adopts the "pure vision + algorithm scheme" scheme Generally, only cameras and low-cost radars are used, and the powerful algorithm computing power of the decision-making layer is used to accurately judge the driving conditions.
This solution relies on a large amount of data, as well as top-level image recognition algorithms, which can form a barrier to advantage, and at the same time, the cost is extremely low. However, the shortcomings are also obvious. The current image recognition algorithm is prone to misjudgment, which leads to traffic accidents.
The second type is a perception solution with lidar as the core, which is a common choice for OEMs other than Tesla. Lidar can measure distances at a longer distance, supplemented by other sensors, and can basically achieve full-range perception of the entire vehicle. Due to the longer-distance road condition perception, the car has longer time to complete further detection, algorithm analysis and decision-making, which can make up for the shortcomings of the algorithm.
From the current point of view, the second type of route is the choice of most OEMs, and lidar has naturally become an indispensable sensing device for L3 and above autonomous driving systems.
2022, the first year of mass production of automotive high-standard lidar
After the rise of autonomous driving, visual cameras, millimeter-wave radar, and lidar are all rapidly evolving.
For lidar, three mainstream routes are gradually differentiated, namely: mechanical rotary lidar, hybrid solid-state lidar, and solid-state lidar.
The mechanical rotating lidar first appeared, and it was commonly used in early self-driving test vehicles such as Waymo, Baidu Apollo, and Pony.ai.
The solid-state lidar has just entered the early stage of mass production, and it has begun to be bet by OEMs such as SAIC.
Hybrid solid-state lidar is the most worth mentioning, and it is more competitive in terms of performance, meeting vehicle regulations, safety and mass production. Therefore, in this technical route, Xiaopeng P5, the world's first mass-produced lidar car, appeared, as well as the later Polar Fox Alpha S HI version, Feifan R7, Weilai ET7, Ideal L9 and Salon Mecha Dragon, etc.
Although the positioning and price of these models are different, and the lidar products they carry have their own considerations, this does not prevent a general comparison of the strength of each lidar technology from the perspective of performance.
• From the two indicators of ranging and angular resolution, the lidars on NIO ET7 and Feifan R7 are tied for first place.
• In terms of mass production time, Xiaopeng P5 is the earliest, NIO ET7 is in second place, and most of other LiDAR mass production vehicles such as Feifan R7 will be delivered this year and next.
With the official delivery of NIO ET7, in 2022, the first year of mass-production pre-installation of vehicle-mounted high-standard LiDAR has officially begun.
Next, let's start with NIO ET7 and see how the lidar on the market is rolled in.
01
Image-level lidar, what is it?
Before understanding the lidar of Weilai ET7, let's take a look at the technical principle.
The technical principle of lidar is that a beam of ultra-short laser pulses is emitted by a laser transmitter, the laser is projected on the target, and diffuse reflection occurs, and then the laser receiver receives the diffusely reflected light, and measures the flight of the laser beam in the air by measuring the flight of the laser beam in the air. time, calculate the distance from the target problem to the sensor.
In the process of the evolution of the automotive industry to high-level autonomous driving, it is constantly necessary to accurately perceive the surrounding complex road environment. Visual cameras and millimeter-wave radars have their own hardware limitations, such as for large light ratio, dark light, stationary obstacles, etc. It is difficult to identify reliably, but lidar has become the "darling" of autonomous driving perception hardware because it is less disturbed by these factors.
By collecting reflected light pulses and reconstructing three-dimensional spatial data, lidar can not only identify the vehicle ahead, but also effectively identify targets such as road bumps, missing manhole covers, throwing objects, and large stationary obstacles, which are difficult for current cameras to complete. task.
At present, various routes have emerged on the lidar track, but there are three main types of routes: mechanical rotating lidar, hybrid solid-state lidar, and solid-state lidar.
Each route has its own advantages and disadvantages, but hybrid solid-state LiDAR is the most suitable for mass production of autonomous driving.
Because the mechanical rotating lidar is mounted on many Robotaxi, the architecture is difficult to be applied on mass-produced vehicles with strict cost requirements. Solid-state LiDAR has a lot of room for improvement because it cannot achieve the performance required for autonomous driving, that is, long distance and high resolution.
In hybrid solid-state lidar, the transmitter and receiver are fixed, the light source and detector are also fixed, and the space is scanned through a rotating or vibrating mirror. The advantage is that, on the one hand, the diameter of the collecting optical path is larger, which makes it easier to achieve a longer detection distance and has a high resolution. On the other hand, the upstream industry chain is relatively mature, such as motors, mirrors, lenses, etc., which are relatively mature and suitable for large-scale mass production at this stage.
As a result, considering factors such as performance, safety, and mass production, hybrid solid-state LiDAR has developed the fastest and has become a product on many new cars.
The lidar on NIO ET7 is a hybrid solid-state lidar provided by Innovusion.
This lidar and 32 other high-performance perception hardware include:
• 7 8-megapixel high-definition cameras;
• 4 3-megapixel high-sensitivity surround view cameras;
• 5 mmWave radars;
• 12 ultrasonic radars;
• 2 1 high-precision positioning unit;
• 1 vehicle-road cooperative perception and 1 enhanced driver perception, etc.
These together constitute NIO's super-sensing system Aquila.
The LiDAR of NIO ET7 adopts a polygonal prism design. The patent diagram is as follows:
Aside from complicated explanations, let’s look directly at some of the characteristics of this lidar. Weilai calls ET7's lidar "image-level lidar". The so-called image level means that this lidar generates sufficiently high-definition "point cloud" images.
The two core features of this lidar are:
• Long detection range;
• High resolution.
According to the information released by Weilai, the farthest detection distance of ET7's lidar can reach 500 meters, and the detection distance can reach 250 meters under 10% reflectivity, with a 120° ultra-wide viewing angle and 0.06° x 0.06° ultra-high resolution It is the world's first 1550 nm lidar to achieve mass production, and the world's first ultra-long-range high-precision lidar delivered to end users.
The ability of the hardware itself to detect data up to 500 meters away.
According to industry sources, “This lidar does not deliberately sacrifice other performance in order to detect 500 meters. It can achieve centimeter-level accuracy when detecting different targets such as 5 meters, 10 meters, and 250 meters. It is the characteristics of the hardware itself."
What is the level of the NIO ET7 compared to the lidars on other models?
02Who
is the strongest lidar?
The technical core of a lidar is: detection distance and resolution.
When the vehicle enters the automatic driving state, the lidar will continuously perceive the vehicle ahead in real time. When encountering danger (the vehicle), it will choose to automatically change lanes or automatically decelerate, or even automatically emergency braking.
Bao Junwei, CEO of Tudatong, once said, “If the vehicle wants to brake safely, it needs to leave an early warning distance of 100-150 meters. When the lidar collects data, it can collect 10 to 15 frames per second. In order to reduce the false alarm rate And the false negative rate, multiple frames of data must be collected, which requires the accumulation of 0.5 seconds to 1 second of data.”
During this process:
• Perceptual planning decision, which takes 0.5 seconds to 1 second;
• 2.5 seconds to 4 seconds to execute command braking;
• When the vehicle is traveling at 120 km/h, the braking distance needs 50 to 60 seconds Meter.
So, the whole distance needs to be around 100 - 150 meters. To ensure the early warning distance of 100 meters, a standard detection distance of 250 meters is required.
Then look at the comparison chart below.
At present, there are only two LiDAR mass-produced vehicles that can meet the detection distance of 250 meters, the NIO ET7 and Feifan R7. Among the remaining lidars, except for the ideal L9, which reaches 200 meters, the detection distance of other lidars is mainly 150 meters.
In terms of safety, the detection distance that can be provided by hardware is a quantifiable basis. In theory, the longer the detection distance, the more redundant the time left for vehicle decision-making.
Another indicator is angular resolution.
The angular resolution is mainly determined by the scanning mode. As shown in the figure above, the angular resolution of lidars currently on the market is mainly 0.2°x0.2°.
For example, a box with a size of 20 cm x 20 cm has an opening angle of about 0.1° at a distance of 100 meters. If the resolution is only 0.2°, there is a high probability that the box will be missed at this distance. If the resolution is 0.2°, the box needs to be seen at a distance of 40 meters, and then multiple frames of data are accumulated. It may be that the car has already arrived in front of the box just after entering the decision-making process, which will lead to an accident.
Unlike the lidars on the market, the angular resolution of NIO ET7's lidars is up to 0.06°. When encountering the same box, the lidar can ensure that at least 4 points are detected on the box. After accumulating 5 to 10 frames of data, it will take about 1 second, and there are 70 meters left to the box, and the decision-making brake will start, and stay There is plenty of time to brake.
As for the wavelength of lidar, each has its own advantages and disadvantages.
Lidar wavelengths on cars are typically 905 nm and 1550 nm.
According to the research of "Nine Chapters Zhijia":
At present, the lasers of TOF lidar are mainly 905 nm, and the lasers of Luminar, Tudatong and Yijing Technology are 1550 nm, and the lasers of FMCW lidar are all uniform. 1550 nm.
The reason is:
According to Luowei CTO Andy Sun: to make any product, first of all, it depends on the maturity of the supply chain, it is impossible to make all the parts by yourself. Specifically for FMCW lidar, in principle, the laser does not have to use 1550 nm or 905 nm. There is no essential technical difference between using 1550 nm or 905 nm. However, in the field of optical communication, it is matched with FMCW. They are all devices based on the 1550 nm band. The supply chain of these devices is relatively mature and the cost is controllable. In contrast, the cost of FMCW based on 905 nm is too high.
Considering the eye safety requirements of continuous light, the upper limit of optical power at 1550 nm is 40 times greater than that at 905 nm. Under the principle of FMCW coherent amplification, theoretically, 1550 nm can test 40 times longer distances than 905 nm.
According to an article written by Zhou Yanwu, an industry veteran, "high-power laser sources need to consider human eye safety, so only 1550-nanometer lasers can be selected. 1550-nanometer lidars are 100,000 times safer than traditional 905-nanometer lidars."
Even in TOF, 1550 nm overcomes sunlight noise better than 905 nm. However, the disadvantage of 1550 nm wavelength lidar is that it is expensive.
In the entire perception system, the direct impact of lidar on sunlight is lower than that of visual cameras, so in some special scenarios, lidar has natural advantages to supplement visual shortcomings.
The last question, the number of lidars, is it better to have 1, or 2, or even 3 or 4? This is the final factor that affects the actual effect of lidar.
Weilai ET7 is equipped with 1 lidar, Xiaopeng G9 is equipped with 2, Avita 11 is equipped with 3, and Salon Mecha Dragon is even equipped with 4 lidars.
In theory, carrying more lidars will definitely bring more accurate perception capabilities. Multiple lidars can be redundant with each other, but it will also bring objective cost increases. If the cost is not considered, the more the number of lidars is placed, it will definitely help to improve perception. However, after all, mass-produced vehicles are in the civilian market, and factors such as maintenance and calibration must be considered.
According to the person in charge of an overseas autonomous driving company: "The quantity depends on the definition of the whole vehicle, and what problems do the OEMs need lidar to solve, as long as this lidar can meet the life cycle of the vehicle and continue to support Software OTA capability, one or two depends on the situation, the radar potential of ET7 and Feifan R7 is great.”
Based on the above analysis, in terms of performance parameters, Weilai ET7 and Feifan R7 have the strongest lidars. The ideal L9 and the Xpeng G9 equipped with two lidars are next, and the remaining models will perform relatively poorly on a single lidar. As for the performance improvement brought by multiple lidars, we need to wait for the mass-produced car to actually hit the road before further comparison.
03
On the road, how does the strongest lidar perform?
NIO ET7 equipped with lidar, what kind of changes can users feel?
At the end of this month, NIO ET7 will be officially delivered to users, and the lidar on the car will be activated simultaneously. I believe everyone will soon be able to experience it in the test drive.
Before it is officially delivered to users, it is still analyzed from a theoretical point of view.
First, compare it with the traditional visual perception scheme.
Compared with the traditional models that use visual cameras and millimeter-wave radar as the main perception solutions, there are some special features of Lidar itself.
At present, visual cameras face the industry problem of "large light ratio".
When a vehicle enters the tunnel during the day, or suddenly enters the tunnel when there is no night light on the road at night, the light in the tunnel will suddenly brighten the light, and the camera will expose several to ten frames. Data is invalid data, which has a certain impact on driving safety.
Due to the characteristics of lidar, the lidar of NIO ET7 is basically not affected by light. This is what lidar production vehicles can do.
When dealing with special-shaped vehicles, the perception of lidar will be more accurate. According to Weilai, “When facing some conventional special-shaped vehicles, such as flatbed trailers, the camera will make errors in judging the distance. The Weilai ET7 equipped with lidar will detect the distance of these special-shaped vehicles. very precise."
When entering the city for automatic assisted driving, there are usually large vehicles (buses, large trucks) cut in at close range. For visual cameras, it is difficult to identify completely, or distortion will be formed when collecting data, so for the visual camera. The error of the position and attitude of the target will be relatively large, resulting in a relatively slow response of the vehicle system. Because Weilai ET7 does not need to estimate the distance through the camera, and directly detects the distance through the lidar, the measurement results will be very accurate.
Since the imaging quality of the original point cloud of Tudatong's lidar is very high, it is not prone to jumps, and the prediction of the target's cut-in route trajectory will be better.
It can be reasonably guessed that in the future, when pedestrians at urban intersections are waiting for red lights, the movement and posture of pedestrians will change. Weilai ET7 may perceive and predict through lidar, and predict who will cross the intersection. These functions will eventually be used. Pushed to users via OTA.
In extreme scenarios, lidar will also perform better.
For example, in a city street without street lights, if you happen to encounter the opposite vehicle with the high beam turned on, and a pedestrian suddenly appears across the street, it is commonly known as a "ghost probe", and it is difficult for the camera to capture this target. Lidar has little effect on light, so it can capture the target in time and warn the user. If the user does not respond, the vehicle will trigger the AEB automatic emergency braking function.
Of course, Weilai ET7's lidar also faces a major industry problem, which is "smearing". This means that when scanning highly reflective objects, some lidars will not only image the real location, but may also produce false images of similar shapes at other locations. Smear is also known as "ghosting".
According to industry sources, Weilai ET7's lidar does not have a 100% guarantee to solve the "smear" problem from the hardware level. If the lidar cannot be handled well, there will be certain problems for the automatic driving algorithm that follows. challenge. However, in testing, a very small amount of ghosting did not have much impact on object detection.
Compared with other lidar vehicles, NIO ET7 will also have better performance in theory.
Since the detection distance and angular resolution of the Weilai ET7 are better, it is guessed that the false alarm rate of the target object will be lower in the actual automatic driving state of this car. Since the car's lidar hardware is good enough, it will bring higher resolution and help reduce the false alarm rate.
Bao Junwei once gave an example, if a 20 cm x 40 cm pillow is thrown from the car, at a distance of 100 meters, Tudatong's lidar can return 5 to 7 points after detection. Tudorton's lidar can detect a 20cm x 40cm projectile from 100 meters away with a clear point cloud. Under the same conditions, the standard detection distance of the lidar is 150 meters, and it is difficult for the point cloud to detect the projectiles at the same distance.
In the face of the same thrown object, the NIO ET7 with stronger detection ability will perceive the target more accurately, providing the basis for the automatic driving to accurately give instructions for automatic deceleration or automatic lane change, and finally let the automatic driving operate. Closer to the old human driver.
Summary
In addition to Tesla's insistence on relying on visual perception solutions, more car companies are joining the lidar army.
Weilai ET7's lidar will always be activated during the driving process of the vehicle, scanning and perceiving the surrounding environment in real time, and redundant with the visual camera, thus complementing the supplement of a single perception hardware. After mass production, it is believed that the cost of lidar can be diluted.
Looking to the future, lidar hardware is developing rapidly.
According to industry sources, if the laser hardware capability is insufficient, the information that needs to be detected cannot be captured. This generation of NIO ET7 lidar hardware will not be outdated in the next three to five years, or even ten years. The next thing to wait for is the continuous iteration of NIO ET7's autonomous driving software. As for other lidar models, hardware replacement factors may have to be considered.
As a car that benchmarks against the BMW 7 series, the positioning of the NIO ET7 is determined. The lidar of this car basically represents the capability ceiling of the existing mass-produced cars. With this car running in front, we will see the large-scale popularization of the autonomous driving era earlier.
Reference link
https://mp.weixin.qq.com/s/yOVWNVepQru0GlDvZG6LHA
https://baijiahao.baidu.com/s?id=1715559306625289798&wfr=spider&for=pc