Exchange group | Enter "sensor group/skateboard chassis group", please add WeChat ID: xsh041388
Exchange group | Enter the "Automotive Basic Software Group", please add WeChat account: Faye_chloe
Remarks: group name + real name, company, position
A few months ago, the author asked a question to a friend who is also engaged in lidar and 4D millimeter-wave radar: you both make these two types of products, so how do you see 4D millimeter-wave radar "killing low-line-count lidar" What about the possibility?
This friend did not answer me directly, but added an additional question: The question is, can we still see the so-called "low line count lidar" now?
The author suddenly realized a problem: the presence of 16-line, 32-line or even 64-line lidar is indeed getting weaker and weaker. In fact, as the number of lines of lidar becomes higher and higher, in the future, 96 lines and 128 lines may become "low line number lidar".
So, is it the 96-line, 128-line lidar, or the 4D millimeter-wave radar that kills the 16-line and 32-line lidar? This is an interesting topic.
Jiuzhang Zhijia recently launched the " 4D Millimeter Wave Radar Report "——
Foreword:
The position of millimeter-wave radar in the configuration of autonomous driving sensors is rapidly rising.
With the continuous improvement of autonomous driving capabilities, the degree of participation of the automatic driving system in the driving process of the vehicle continues to increase, and the traditional millimeter-wave radar is becoming more and more powerless. In order to meet the needs of the perception module of the advanced automatic driving system to achieve all-target, all-working conditions, and all-weather coverage, millimeter-wave radar must go in the direction of "high-definition".
Millimeter-wave radar with "high-definition" characteristics is called imaging radar, or 4D millimeter-wave radar.
"4D" refers to the addition of height-dimensional data analysis of the target on the basis of the original distance, azimuth, and speed, which can realize information perception in the four dimensions of "3D+height"; and the concept of "imaging" refers to its ultra-high Resolution, which can effectively analyze the outline, category, and behavior of the target.
Facing the same obstacles ahead, millimeter-wave radar can only receive limited return information points, and can only judge that there are obstacles ahead, while 4D millimeter-wave radar can receive dozens of times the return information points, which has evolved into a laser-like radar. The same high-density point cloud can further detect the shape of the object, and even recognize the object by combining algorithms.
This means that, compared to traditional millimeter-wave radars, the 4D millimeter-wave radar system can adapt to more complex road conditions, including identifying smaller objects, partially occluded objects, and detection of stationary objects and laterally moving obstacles.
And it is precisely because of these characteristics that the emergence of 4D millimeter-wave radar instantly raised the "forceful" of millimeter-wave radar
Active players in the 4D millimeter-wave radar market include traditional Tier 1 players such as Continental, ZF, and Aptiv, technology giants like Waymo, Mobileye, and Huawei, as well as Arbe, Aoku, Senstech, and Nava Electronics. , Geometry Partners and many other startups.
one. Millimeter wave radar is striving to rise from a supporting role to a leading role
Since the second half of 2020, if a newly released mass-produced model is about to be equipped with lidar, then car companies will definitely highlight this "bonus item" in a very high-profile PR, especially the lidar. The name of the supplier must also be "big book". In contrast, millimeter-wave radar, which has accompanied the automotive industry for decades, has not enjoyed such an honor.
Because, from the perspective of car companies, "star parts" and suppliers such as lidar can enhance their brand value; in contrast, millimeter wave radar is "too common", but this situation is becoming change. For example, in March 2021, when the SAIC R brand released the new SUV ES33, it said in a very high profile that "it will be equipped with ZF's 4D millimeter-wave radar PREMIUM".
Behind this change is that compared with the traditional millimeter-wave radar, the more powerful 4D millimeter-wave radar is striving to become a "stand alone" sensor. It is hoped that the millimeter-level radar can turn from a supporting role to main character.
Compared with cameras, traditional millimeter-wave radars have strong ranging and speed measuring capabilities, and are not affected by weather and visibility. However, due to insufficient resolution, it is necessary to detect obstacles in front (especially small targets) When the outlines of trees and roadsides are clearly outlined, traditional millimeter-wave radars often fail.
In contrast, 4D millimeter-wave radar has more antennas, higher angle resolution, velocity resolution and distance resolution, so it can analyze the outline of the target more effectively without the participation of lidar , category, behavior, and thus easier to know when you must brake and when you don't.
For example, the automatic driving system needs to obtain information about the offset position of the target vehicle’s driving lane 200 meters in front of the ego vehicle for decision-making, while the azimuth angle accuracy of the traditional front radar is about 0.3 degrees. Large position error. In contrast, the azimuth accuracy of 4D millimeter-wave radar can reach 0.1 degrees, which is three times higher than that of traditional front radar. It can output more accurate vehicle lane deviation information at 200 meters to the decision-making system.
For example, for hidden vehicles, the traditional millimeter-wave radar has only a 20% chance of being detected, while the 4D millimeter-wave radar ARS 540 is said to have an 80% chance of being able to do so; another example is ZF's long-range 4D Millimeter wave radar is said to be able to receive about 10 data points from pedestrians. It can even analyze the movement trajectory of individual limbs by measuring the moving speed of these data points, thereby identifying the walking direction of pedestrians.
Usually, millimeter-wave radars need to rely on angle dimension, height dimension, distance dimension and speed dimension to work together to detect targets. The "relatively static" state, therefore, the velocity dimension is invalid. At this time, if the resolution of the millimeter-wave radar is not high enough, it is easy to regard the two cars as "the same car".
However, because of the relatively high point cloud density of 4D millimeter-wave radar, even if the velocity and distance dimensions are invalid, the probability of "guessing" the target is still relatively high.
Moreover, unlike the recognition accuracy of common algorithms that is easily constrained by the incomplete target sample library, 4D millimeter-wave radar is more efficient in classifying targets by increasing the point cloud density—reducing the time spent on calculations , that is, the traditional millimeter-wave radar + algorithm may need to scan many frames to identify the obstacle with a relatively weak signal, while the 4D millimeter-wave radar may only need to scan 1-2 frames to get it done.
There is also a common scenario of "no distinction between people and vehicles": a person is standing next to a parked vehicle.
The camera does not have penetrating power and cannot be seen. However, the traditional millimeter-wave radar is used for detection, because the resolution is not enough, and the energy density and reflection intensity of the car are higher than that of the human. Therefore, the point cloud scanned by the millimeter-wave radar on the human body It is easy to be "sucked up" by the car, and as a result, the person is mistaken for "part of the car". Similarly, when a small car and a large car are very close, they will also be mistaken for "part of the large car" by the millimeter-wave radar.
Obviously, people or cars that are "merged" are actually ignored by traditional millimeter-wave radars, which is extremely dangerous. However, 4D millimeter-wave radar has "high dynamic resolution" and can distinguish various obstacles with great differences in reflection intensity in the same occasion, that is, a large car is a large car, a small car is a small car; a car is a car, and a person is a car. people. In the case of a clearer analysis of the target, the millimeter-wave radar is more capable of supporting the decision-making system.
Under the multi-sensor fusion scheme, 4D millimeter-wave radar can also "guidance" cameras and lidars to potential risk areas, which will greatly improve safety performance.
Another example is the detection of stationary objects. Since there is no vertical antenna, it is impossible to obtain height-dimensional data (Bosch and Continental's forward millimeter-wave radars can output height-dimensional information, but the accuracy is not enough), and traditional millimeter-wave radars are difficult to judge whether a stationary object in front is on the ground or in the air. Then it is easy to separate low and small "obstacles" (no need to brake) such as manhole covers, speed bumps, and roadside metal, and high "air obstacles" (no need to brake) such as traffic signs, gantry, overpasses, etc. static obstacles (requiring braking) to confuse.
In view of this, if the traditional millimeter-wave radar is used as the main sensor, it may lead to frequent false braking. In order to avoid false braking, the AEB algorithm decides to reduce the confidence weight of the millimeter-wave radar (even to filter out static obstacles), focusing on visual perception results. This is the truth of the widely circulated statement that "millimeter wave radar cannot identify static obstacles".
However, the challenge of visual perception is that the monocular and trinocular cameras must first recognize (classify) the target before they can be detected, but the recognition not only requires good lighting, but also highly depends on the target model library, and the model library cannot be poor For all types, this means that many static obstacles have become "fish that slipped through the net" for visual perception. Therefore, it often happens that there is a static obstacle ahead, but the self-driving car still crashes into it.
A friend who worked as an ADAS algorithm engineer in a traditional car company told the author that the project he worked on three years ago was stuck on "millimeter wave radar cannot recognize static obstacles" - at that time, millimeter wave radar The limitation of the physical performance of radar, this problem is unsolvable, but the leader does not understand, the leader asked him to "must solve it". Ultimately, the engineer was forced to resign.
But now, with 4D millimeter-wave radar, the tragedy of engineers being forced to leave their jobs due to uncertainty about static obstacle recognition can be avoided, at least, the probability of occurrence will be greatly reduced. Because the 4D millimeter-wave radar with the "standard" longitudinal antenna (specially used for height measurement) can provide data dimensions with vertical resolution, the return signals of obstacles in front are no longer roughly arranged on the two-dimensional ground, but presented in three-dimensional space. This helps static obstacles of various heights be "differentiated".
In short, from the perspective of the decision-making system, compared with traditional millimeter-wave radar, the detection results of 4D millimeter-wave radar have higher confidence, so the decision-making system does not have to worry too much about referring to the output of 4D millimeter-wave radar. Doing path planning based on the perception results will frequently lead to false braking. Therefore, the weight of 4D millimeter-wave radar in many functions can be ranked above the camera.
What impact will the weight of 4D millimeter wave radar have on driving safety? We imagine a scenario where the "vehicle in front of the vehicle in front" has already braked, and the "vehicle in front" has not reacted, and will not have time to brake in the future, how will the "self-car" react?
Traditional millimeter-wave radar can penetrate the "vehicle in front" and detect what happened to the "vehicle in front of the vehicle in front", but unfortunately, this detection result is often "put into limbo" by the decision-making system; and the camera as a sensor , but does not have the penetrating power to see clearly what happened to the "car in front of the car in front". Therefore, when the vehicle speed is relatively high, a series of rear-end collisions is a high probability event.
But if 4D millimeter-wave radar is used, it will be another result: because the "reliability rate" is higher than that of traditional millimeter-wave radar, the detection results of 4D millimeter-wave radar can easily cause "high attention" from the decision-making system. Therefore, in When the "vehicle in front" hits the "vehicle in front of the vehicle in front" because it is too late to brake, the "self-vehicle" will brake in time because the decision-making system has received and believed the notification of "danger ahead".
With the help of 4D millimeter-wave radar, the "self-car" not only avoids reckless collision with the front car, but also its braking action will give positive feedback to the "rear car", preventing itself from being "rear-ended"; Saved the "back car" and "the back car of the back car".
This is a capability that even lidar does not have. (However, a research and development engineer of a new car-making force said: "This method is theoretically feasible, but it has not yet become a routine detection method in mass production projects.")
The French market research institute Yole released the "2020 Radar Industry Situation Report: Manufacturers, Applications and Technology Trends" released last year, pointing out that after the application scenarios become more stringent, millimeter-wave radar is moving towards more accurate description of the front and rear of the vehicle. The 4D millimeter wave radar advances.
Currently, many companies are exploring the possibility of replacing low-line-number lidar with 4D millimeter-wave radar.
In addition, DMS and vital body monitoring in the cockpit are also becoming important application scenarios for 4D millimeter-wave radar.
Tesla CEO Musk's "kill millimeter wave radar" campaign in 2021 once made many people question the application of 4D millimeter wave radar in the autonomous driving market, but in fact, Musk is denouncing millimeter wave radar When discussing the various disadvantages of the radar, he mainly aimed at the traditional millimeter-wave radar, and he did not reject the "high-precision millimeter-wave radar".
As early as October 2020, Musk mentioned the plan to adopt 4D imaging radar; in September this year, Tesla was found to have submitted a self-developed millimeter-wave radar certification application to the FCC (Federal Communications Commission of the United States) and passed. Because the confidentiality order was not lifted until December this year, the specific parameters and uses of this millimeter-wave radar are still unclear, but from the public test report, it is known that this is a 77Ghz radar, and the antenna setting uses 6 to 8 hair plan.
In China, as of now, several models with prices in the range of 250,000 to 400,000 have been planned to be equipped with 4D millimeter-wave radar on the front bumper. I believe that when the time is right, more and more car companies will have a deep understanding of the value of 4D millimeter-wave radar, and then they will eagerly use 4D millimeter-wave radar to "talk about things" in future new car launches. of.
2. Main players and technical routes of 4D millimeter wave radar
Currently, players in the 4D millimeter-wave radar market mainly fall into the following categories: traditional Tier 1 companies such as Continental, ZF, Bosch, and Aptiv; autonomous driving solution companies such as Waymo, Mobileye, and Huawei; Aoku, Arbe, Geometry Partners, Chu Start-up companies such as Aviation Technology and Sensitech.
On the technical route of increasing the resolution by increasing the number of antennas, there are currently three main solutions: "cascading", cascading + virtual aperture imaging, and integrated chips.
1. Cascading
The so-called cascading refers to adding the 77G and 79G standard radar chips (MMIC chips) of Infineon, Texas Instruments, NXP and other companies through two-stage/four-cascade/eight-cascade connection to increase the physical antenna MIMO (the number of receiving antennas and the number of transmitting antennas) The number of virtual channels obtained by multiplying the number of antennas).
Two-level cascade is to connect two 3T4R chips together to form 6T8R; four-cascade is to connect four 3T4R chips together to form 12T16R to form 192 virtual receiving channels, such as ARS 540 in mainland China; The electronic 18T24R product is 6 cascaded. Bosch, ZF, Waymo, and Huawei all use cascading methods.
The advantage of this solution is that it is less difficult to develop in the early stage, so the time to market is relatively short, but the disadvantages are large size, high cost, high power consumption (multi-chip simultaneous operation will increase power consumption), and insufficient signal-to-noise ratio (multiple MMIC chips) There is crosstalk between them), algorithm adaptation and other issues.
In addition, compared with the traditional millimeter-wave radar, the multi-chip cascading scheme not only has more complex antenna layout, but also has a much more complex hierarchical structure of the PCB board. For example, if there are 6 layers of boards in total, the material of each layer may be different, so the expansion coefficient is also different, which will cause the board surface to warp, thus affecting the energy utilization rate.
In addition, Jiuzhang Zhijia also learned that many 4D millimeter-wave radar manufacturers using the cascaded technology route have encountered the technical problem of "intermediate frequency synchronization"-taking the quadruple connection as an example, the four chips have an intermediate frequency of about 20G The signals must be synchronized, and the two boards are pressed together, so the yield rate cannot increase.
Therefore, if the cascading solution is adopted, it is easier to do a demo based on the reference design and SDK provided by the chip manufacturer, but the threshold for mass production is very high, and only those companies with strong technical and engineering capabilities can do it well.
2. Cascade + virtual aperture imaging technology
The so-called cascading + virtual aperture imaging technology refers to the high-multiple virtual MIMO based on the existing chip in the cascading way through the unique virtual aperture imaging software algorithm and antenna design, in order to achieve the original number of physical antennas. On the Internet, ten times and dozens of times the number of antennas are virtualized, and the angular resolution is directly increased from 10 degrees to 1 degree.
The waveform of traditional radar is single-frequency, repetitive, and non-adaptive. The only way to generate multiple waveforms is to increase the number of receiving antennas; while the virtual aperture imaging waveform is adaptive phase modulation (frequency modulation + phase modulation + amplitude modulation), each The receiving antenna produces different phase responses at different times, and the data is then interpolated and extrapolated to create a "virtual aperture" that improves angular resolution by orders of magnitude.
According to public information, Aoku is the representative manufacturer that adopts this cascade plus virtual aperture imaging technology. Aoku is positioned as Tier 2, and it provides 4D millimeter-wave radar signal processing algorithms without hardware (hardware is provided by Hella and other partners). In October 2021, Aoku's self-developed 4D millimeter-wave radar AI algorithm and AD4D millimeter-wave radar technology will be acquired by Ambarella.
The more virtual channels, the more complete the received signal and the clearer the detection result. Qie Jianjun, head of the Aoku marketing department, said: "After adopting the virtual aperture imaging technology, our single chip can achieve the effect of other companies' four-cascade products in terms of resolution, and the two-level cascade can reach other companies' six-level resolution. Cascading effects."
There are also several 4D millimeter-wave radar manufacturers questioning that this technology "does not conform to the laws of physics" in principle.
For example, someone said: "Increasing the number of antennas through frequency modulation, amplitude modulation, and phase modulation can optimize product performance locally, but according to common sense, optimization through software will definitely not be able to fully compensate for the lack of hardware capabilities. Because, in point Given the frequency (how many electromagnetic waves can be emitted in one second), what the software adjustment can do is to 'scatter' these electromagnetic waves, but no matter how you adjust, the total number of electromagnetic waves will not be too many. If it’s dense here, it’s sparse there.”
However, a Tier 1 engineer who has worked closely with Aoku said: From the perspective of principle, I don’t believe what Aoku said is true, but after product testing, the angular resolution is indeed quite high, and, The ranging accuracy of the target within 50 meters can reach 0.1 meters.
The barriers to virtual aperture imaging technology mainly lie in the layout and waveform of the antenna. Among them, the antenna layout mainly affects the size of the virtual aperture, and the waveform mainly affects the number of channels. In addition, the increase in the number of antennas also puts forward higher requirements for subsequent data processing capabilities.
The virtual antenna technology has completely solved the problem that has plagued the automotive millimeter-wave radar industry for decades and can only increase the angular resolution by increasing the number of physical antennas. It can make the cost of the product be controlled at a reasonable level while greatly improving the angular resolution.
3. Integrated chip
The so-called integrated chip solution refers to the realization of the above functions by integrating the multi-transmission and multi-reception antennas into one chip and forming an ASIC chip. At present, the representative companies of this technology mainly include Arbe, Uhnder, Vayaar, SteradianSemi, RFISee and so on. The most typical is the 4D millimeter-wave radar RFIC chip developed by Arbe, which integrates 48 transmitters and receivers and has more than 2300 virtual channels.
The integrated chip can greatly reduce the size of the 4D millimeter wave radar, and realize the most advanced radio frequency performance at the lowest cost per channel in the market. However, the implementation difficulty of the integrated chip solution is much higher than that of the cascade solution. The main challenges are:
1. How to arrange so many antennas in a very small confined space;
2. How to overcome the mutual interference between antennas;
3. How to reduce power consumption and how to dissipate heat;
4. How to improve the signal-to-noise ratio, if the signal-to-noise ratio cannot be improved, the effective detection distance will be very short (Uhnder said that their integrated chip is a digital FM chip, which has anti-interference ability);
5. The chip solution is ASIC. Once the chip is taped out, the algorithm will be solidified. After that, the algorithm can only be modified for individual parameter configurations for specific scenarios, but the functions cannot be greatly adjusted.
The first few problems can still be overcome by means of engineering technology, but the last problem is unsolvable.
The person in charge of a 4D millimeter-wave radar manufacturer said that currently, 4D millimeter-wave radar is still a new product, and OEMs have not yet used it on a large scale. Therefore, 4D millimeter-wave radar manufacturers cannot determine which parameters the OEMs prefer in the end. "Once a millimeter-wave radar manufacturer tapes out a certain version of the integrated chip, later on, if the OEM finds that the algorithm of the 4D millimeter-wave radar needs to be greatly changed during the test, Arbe will have to tape it out again, which will not only increase the cost, but also increase the cost. And it also affects the progress of boarding.”
The above-mentioned people believe that the most suitable time for mass production of Arbe's integrated chip solution may be five years after the 4D millimeter-wave radar is put on the car in batches, and 100 models of 20 OEMs are using it, and the algorithm has been solidified. At this time, it is easy for manufacturers to quickly reduce costs by doing integrated chip solutions, and they can also build a moat. "We may also make integrated chip solutions in the future, but not now, because as long as the model is changed, the product may not be usable."
There is also a "metamaterial route". Because the current research on metamaterials is still in the laboratory stage, it is still difficult to achieve commercialization in the short term, so we will not discuss it in this article.
three. Software and hardware integration and machine learning
No matter how good the product is, it will be more troublesome if the car company "can't use it".
With the trend of upgrading traditional millimeter-wave radars to 4D millimeter-wave radars, the hardware ceiling has been broken through, and the system has higher requirements for algorithm capabilities.
Compared with 3D millimeter-wave radar, the number of point clouds of 4D millimeter-wave radar has increased significantly. Therefore, how to eliminate false alarms or unnecessary point clouds, how to select the point clouds that need to be used and apply them to the functional level, That's a big challenge.
In the past, the hardware did not have the ability to detect small obstacles anyway, so the way the algorithm solves the problem is usually "guessing". Now, with the improvement of hardware capabilities, the algorithm ability must also be upgraded from "guessing" to "analyzing", otherwise it will be Will "live up" the hardware.
For example, how to separate similar targets, how to find failure scenarios, how to filter interference, and how to reasonably set the threshold of signal-to-noise ratio (threshold is too high, small targets are easy to be missed; threshold is too low, it is easy to have false detection ), etc., all require powerful software algorithm capabilities.
But the status quo is that at present, most car companies do not have the algorithm capabilities of millimeter-wave radar.
A project manager of a millimeter-wave radar manufacturer said: "The mass-produced projects from 2022 to 2023 basically use the results of the millimeter-wave radar after data processing. Really use it."
For a long time, millimeter-wave radar manufacturers have often provided software-hardware integration solutions, that is, the algorithms have been integrated into the hardware. For car companies, the millimeter-wave radar directly outputs the perception results, and they only need to fuse the results with the recognition results of other sensors.
At present, although car companies are clamoring for "software and hardware decoupling" and hope to make their own algorithms, in fact, the barriers to millimeter-wave radar algorithms are extremely high, and only a very small number of car companies can handle them. Therefore, for most cars For enterprises, the software and hardware integration solutions provided by manufacturers are still the first choice.
Judging from the research results of Jiuzhang Zhijia, at present, the algorithms of the data processing part of 4D millimeter-wave radar, such as ACC, AEB, BSD, and LCA, are basically done by radar manufacturers. "There are also some OEMs working on the development of millimeter-wave radar algorithms, but judging from the current overall development of the industry, it is still relatively difficult for OEMs to make the transition from point cloud to target to function."
However, to the embarrassment of car companies, the algorithm of 4D millimeter-wave radar is much more complicated than that of traditional millimeter-wave radar. It is also difficult for manufacturers to figure out the algorithm themselves, and some manufacturers can only deliver hardware. It is said that in the case of cooperation between an international OEM and a German Tier 1, the OEM had to do algorithms that it was not good at.
So, why is it difficult for Tier 1, which has accumulated decades of experience in the algorithm of traditional millimeter-wave radar, to handle the algorithm of 4D radar?
A big difference is: traditional millimeter wave radar defines a target as a "point target", while 4D millimeter wave defines a target as an "extended target". Therefore, the signal processing and point cloud processing architectures of the two are different.
In addition, the technical director of a 4D millimeter-wave radar manufacturer said that the traditional millimeter-wave radar algorithm only needs to do some simple data clustering processing, while the 4D millimeter-wave radar algorithm needs to focus on AVP, HWP, TJA performs functions, and these functions are usually performed by algorithm companies or hardware technology companies with strong algorithms.
It is understood that when bidding with some foreign companies for a project of a domestic OEM, the reason why a domestic millimeter-wave radar manufacturer was able to win quickly was that, in addition to the higher quantity and quality of point clouds, it could also provide 4D millimeter-wave radar algorithms. is also a key reason.
The pain point of 4D millimeter-wave radar algorithm is difficult to write, which just became an opportunity for BlueSpace.ai, which specializes in providing predictive perception software technology solutions for 4D radar, lidar and other sensors.
Of course, millimeter-wave radar manufacturers are not willing to hand over the "soul" to their partners. In fact, many companies in the 4D millimeter-wave radar industry chain have plans to build algorithm capabilities as their core competitiveness in the future. The introduction of machine learning algorithms is one of the biggest highlights of this series of actions.
In a previous report by "Gaogong Smart Car", an industry insider said: "In the past, because the radar resolution was very low, you couldn't do any post-processing related to machine learning. Now because 4D millimeter-wave radar can produce laser-like Radar point cloud data, machine learning can be used to train the radar perception system to recognize objects, especially to help solve the edge detection problem that traditional radar cannot overcome."
This trend has also been verified by more and more companies. For example, NXP has launched a vehicle-level AI toolkit. In addition to being applied in the traditional vision field, 4D millimeter-wave radar will also use neural networks to classify road users and obstacles based on their point cloud images.
According to the plans of radar chip manufacturers such as NXP and Texas Instruments, the next step for 4D millimeter-wave radar is to improve the machine learning capabilities of similar camera algorithms.
Arbe has thought from the very beginning how to put signal processing and artificial intelligence on top of off-the-shelf RF chipsets and digital signal processors DSP to achieve real-time clustering, tracking, self-localization, false target filtering, radar-based and radar-based + Camera object classification.
This artificial intelligence algorithm can recognize whether the detected object is a person and not a tree, and calculates where it will be in a second, and also fuses with the camera and other sensors in the suite to analyze the Detect objects for classification and matching.
According to the report of "Gaogong Intelligent Car", Aptiv has also given a series of data of 4D millimeter-wave radar combined with machine learning capabilities.
For example, for small objects or debris on the road, machine learning can increase the detection distance by more than 50%, and can track small objects within 200 meters; Missed detections; machine learning can also reduce position error and target heading error by more than 50%, which means the vehicle is better able to identify vehicles parked in other lanes, as well as stationary or slow-moving objects.
As another example, tunnels are a challenging environment for traditional mmWave radars—tunnel walls are a huge reflective surface, which can lead to very many return points, which may even exceed the ability of the radar to process the target. But Aptiv believes that machine learning can help vehicles understand when they have entered a tunnel, filtering out noise in detection with greater precision than classical methods; at the same time, it can also better interpret radar returns in tunnels and other closed environments , to classify objects such as sectors.
Additionally, Qualcomm itself does not produce mmWave radar, but they claim that the radar's performance can be amplified by performing deep learning on the radar. For example, higher resolution and 3D scanning are achieved through the use of enhanced radar through the use of a "radar deep neural network" developed in-house by Qualcomm.
Accordingly, the competitiveness of a 4D millimeter-wave radar will be largely assisted or restricted by the deep learning algorithm capabilities of manufacturers and partners.
Four. The road is long and difficult: the problem in the application of 4D mass production
After Xiaopeng announced at the end of 2020 that it would adopt lidar on the P5, other OEMs (both new forces and traditional OEMs) followed suit. It can be said that lidar has triggered a round of armament among auto companies. However, in the 4D millimeter wave radar market, neither the cooperation between BMW and Continental, nor the cooperation between SAIC R brand and ZF seems to have had much influence on the decisions of other OEMs.
According to Jiuzhang Zhijia's research, only a few car companies will use 4D millimeter-wave radar on new models launched in the next one or two years.
Why do most OEMs still hold a wait-and-see attitude towards 4D mmWave? What obstacles are still facing the mass production of 4D millimeter-wave radar? (Difficulties at the product development level have been introduced in the second section of this article. Here we will focus on the difficulties encountered in the application)
1. Technical and engineering problems
(1) Multiple indicators need to meet the conditions at the same time
A perception engineer from an OEM said that many 4D millimeter-wave radar manufacturers emphasize the advantages of their products in single indicators such as ranging resolution, angular resolution, and velocity resolution, but the significance of a single indicator to the final imaging Not much, "In fact, it is necessary to improve the distance resolution, angle resolution, and velocity resolution at the same time to achieve a better imaging effect, and the confidence of the results will be higher."
( 2 ) Pre-fusion is difficult to do
In order to fully utilize the technical advantages of the 4D millimeter-wave radar, it is necessary to perform front fusion with the camera - there is still a problem of confidence in the post-fusion, that is, both sides have seen the target, or a single sensor has seen the target. Who do you trust? Therefore, pre-fusion is necessary. However, pre-convergence is not an easy task. The reasons are as follows—
A. For a long time, under the mode that millimeter-wave radar manufacturers only provide a black box integrating software and hardware, most OEMs have never "seen" the original data of millimeter-wave radar, and they don't know much about the characteristics of these data at all ( The data format of the millimeter-wave radar is different from that of the camera), so there are too few talents who are really familiar with the point cloud attributes of the millimeter-wave radar.
Nowadays, most companies are learning millimeter-wave radar from scratch (5-6 years later than learning lidar), and even find it difficult to write millimeter-wave radar algorithms, let alone integrate 4D millimeter-wave radar with cameras. What's more, at present, there are not many samples of 4D millimeter-wave radar on the market, and there are not many learning opportunities for downstream customers.
B. 4D millimeter-wave radar has a large number of channels and a relatively large amount of data. The pre-fusion with vision requires relatively high computing power, and the computing power at the sensor side is not enough. The memory of the millimeter-wave radar chip is limited, and the processor It needs to deal with FFT transformation, CFAR, filtering, etc., and cannot handle too many point clouds. Therefore, if the point cloud density is relatively high, the pre-fusion needs to be done in the domain controller.
At present, Aoku's new generation of 4D millimeter-wave radar is following the technical route of algorithm + central domain control. It is said that a single radar can achieve an angular resolution of 0.1°*0.1° and a point cloud of hundreds of thousands of points per second. .
But if the main control chip is placed in the domain controller, not only the high data rate and data compression of 4D millimeter wave radar will bring challenges to the centralized architecture, but also the bandwidth and rate of signal transmission between the antenna and the processor will affect to the detection accuracy. This means that the idea that data operations are carried out in the domain controller is also difficult to work.
To solve the above contradictions, 4D millimeter-wave radar manufacturers need to have a deep enough understanding of the central domain controller, or be deeply bound with a domain controller manufacturer or chip manufacturer. And Aoku can put the algorithm of 4D millimeter-wave radar in the central domain controller, which has a lot to do with their acquisition by Ambarella - Ambarella's domain control chip CV3 will take out a small piece to make millimeter-wave radar All signal processing (can deal with 6 radars).
C. Pre-fusion requires joint calibration of 4D millimeter-wave radar and camera, but joint calibration is difficult--4D millimeter-wave radar does not understand semantic information accurately enough, and the classification of targets is not accurate; in addition, 4D millimeter-wave radar has distance information , while the camera does not. Then, when the two are jointly calibrated, how to achieve confidence and reliability at the level of vision and 4D millimeter-wave radar point clouds, and under what circumstances which sensor is more accurate is a big problem.
( 3 ) EMC is difficult to pass
The electronic compatibility of 4D millimeter-wave radar, that is, EMC, is difficult to pass. The key is to consider how to avoid interference to the outside world and how to fight against interference from the outside world. Among them, interference to the outside world includes interference with objects outside the car and the car radio in the car, etc., which is related to the emission frequency of electromagnetic waves (including EMI and EMF). EMC problems are generally difficult to find in the early simulation, but need to be found in the experimental process.
( 4 ) The test equipment is not yet mature
Usually, car companies or Tier 1 need to use radar simulators to test the hard conditions in the laboratory before road testing. 3D millimeter-wave radar is mature and the test equipment is mature, but 4D millimeter-wave radar is currently only released by Schwartz. I installed a 4D radar simulator. Although the design was designed according to the indicators, there is no equipment for verification.
( 5 ) The test standard is not yet clear
It turns out that 3D millimeter-wave radar only needs to measure the detection ability in the horizontal direction. After 4D millimeter-wave radar has altitude information, it needs to add high-dimensional tests, and the standards and methods of altitude testing still need to be explored.
( 6 ) Not easy to install
The size of 4D millimeter-wave radar is generally larger than that of traditional millimeter-wave radar, and the shape is also different. Therefore, the installation position is not easy to design.
2. Difficulties at the policy level: lack of national standards
At this stage, 4D millimeter-wave radar lacks national standards, and many OEMs are worried that they will not be able to use it after large-scale installation.
For example, in sensitive areas such as roads within one kilometer from the observatory, 77G millimeter-wave radar is not allowed to be turned on, but if 4D millimeter-wave radar is turned on most of the time, how to turn it off in these sensitive areas is also a problem .
3. Difficulties at the commercial level: the price/performance ratio is not high enough
4D millimeter-wave radar and lidar are heterogeneous, and many engineers do not believe that it can replace lidar.
Even, some engineers who have worked in the autonomous driving perception team of 4D millimeter-wave radar manufacturers and OEMs said frankly: "From our test data, at this stage, 4D millimeter-wave radar not only cannot replace lidar, but also , Compared with the mature 3D millimeter-wave radar, its advantages are not so obvious. I feel that 4D millimeter-wave radar will have to wait at least 2-4 years to mature."
Another person in charge of the marketing department of a millimeter-wave radar manufacturer said: "The imaging effect of 4D millimeter-wave radar is much worse than that of laser radar. There is no need to use its own weakness to compete with laser radar; There is no way to replace lidar as a primary sensor.”
Although the product is not easy to use, the price is not low. As a result, after comparison, people found that the current cost performance of 4D millimeter wave radar is not high enough.
The price information of 4D millimeter-wave radar is still relatively confidential, and it is inconvenient for us to disclose too much in the article.
Unfinished words: To survive, or to poetry and distance ?
The opinion of a broker who has been paying attention to the autonomous driving track for a long time——
Currently, start-up companies on the 4D millimeter-wave radar track are currently in the stage of desperately raising funds. Since last year, whether investors invest in you or not mainly depends on whether you have mass production orders. So, from the perspective of financing, They focus on the 3D millimeter-wave radar that is easier to mass-produce and land, and it is more beneficial to get orders than to focus on 4D millimeter-wave radar.
But on the battlefield of 3D millimeter-wave radar, they still have to compete with companies such as Bosch, Continental, and ZF, and the pressure will be very high, so they may not have much energy and funds to attack 4D millimeter-wave radar.
For more information about 4D millimeter-wave radar, please refer to the latest report of Jiuzhang Zhijia——
2023 report topic selection schedule (a total of 6 copies throughout the year)
Discrimination and analysis of the necessity of full-stack self-development of autonomous driving
Layout, planning and development trend of smart car vehicle architecture/domain controller
Laser radar technology route and technology trend
Topics on Smart Car Software Development
Analysis on the Development Trend of Vehicle Camera Core Technology
Challenges and solutions for mass production of autonomous driving products
how to buy
Nine-chapter special reports are priced at 10,000 yuan/article
The first trial issue report——
" 4D Millimeter Wave Radar Technology Trend and Application Analysis "
Only 999 yuan / article
If you need to purchase a report, please scan the QR code to add staff
References:
4D imaging is not enough! Millimeter wave radar also needs a new capability blessing to "debut"
https://mp.weixin.qq.com/s/ajjq8Tax8tmWIlhPBeaIcQ
Mobileye launched the most powerful 4D millimeter wave image radar in history https://mp.weixin.qq.com/s/kxAEC5fAkwnye8_xCXw9Vg
write at the end
About Contribution
If you are interested in contributing to "Nine Chapters Smart Driving" ("knowledge accumulation and sorting" type articles), please scan the QR code on the right and add staff WeChat.
Note: Be sure to note your real name, company, and current position when adding WeChat
And the information about the position of interest, thank you!
Quality requirements for "knowledge accumulation" manuscripts:
A: The information density is higher than most reports of most brokerages, and not lower than the average level of "Nine Chapters Smart Driving";
B: Information needs to be highly scarce, and more than 80% of the information needs to be invisible on other media. If it is based on public information, it needs to have a particularly powerful and exclusive point of view. Thank you for your understanding and support.
Recommended reading:
◆ Nine Chapters - A Collection of Articles in 2021
◆Detailed explanation of Inceptio’s mass production technology methodology for self-driving trucks
◆ Lidar: the war between 905 and 1550
◆ Single SoC chip solution, or will accelerate the large-scale mass production application of the mobile and parking integrated solution