Exploration and practice of autonomous driving tool chain from R&D domain to mass production domain

17120203:

From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

Introduction

This article is compiled from the keynote speech "Exploration and Practice of Autonomous Driving Tool Chain from R&D Domain to Mass Production Domain" by Xu Peng, Senior Manager of Baidu Smart Cloud Autonomous Driving Cloud R&D at Baidu Cloud Intelligence Conference - Smart Car Sub-forum on September 5, 2023. .

The middle paragraph of the full text is accompanied by the complete versions of the two product demonstration videos in the speech, which are not to be missed.

(Video viewing: https://mp.weixin.qq.com/s/qsgrgirWa_UiSPkF_P7NYQ )

The full text is 3580, and the estimated reading time is 9 minutes.

Baidu, as the first domestic company to deploy autonomous driving, has been actively exporting autonomous driving-related products, technologies and services to the industry. I am honored to have such an opportunity today to share with you Baidu's practical experience in the field of autonomous driving tool chains. We also hope to discuss our understanding of autonomous driving tool chains with you.

Autonomous driving is a complex subject that integrates multiple capabilities. Without solid technology accumulation and industrial layout, it will be difficult to solve the problem of rapid implementation of autonomous driving. Baidu has been laying out core R&D technologies for autonomous driving since 2013. In 2021, we opened up to the industry the experience accumulated in the field of autonomous driving for many years to form systematic tool chain products. Within two years, we have been widely recognized by many domestic and foreign OEMs and Tier1.

Here are a few cases to share with you:

The first case is a cloud simulation platform. Together with our customers, we have accumulated 500,000+ customized scenarios. In less than a year, we have helped car companies complete more than 700 versions of autonomous driving algorithm iterations and achieved nearly 10 million kilometers of test verification, allowing car companies to increase their volume. The time for the implementation of industrial intelligent driving has been greatly advanced.

The second case is data closed loop. Within one year, we helped customers host a total of 50 PB of data, and embedded more than 500 autonomous driving data mining models in the platform, completing the efficient processing of hundreds of millions of frames of data to achieve data value enhancement and transformation.

The third one is data annotation. For the research and development and training of autonomous driving, high-quality and large-scale data sets are generally obtained manually and frame-by-frame under traditional conditions. Baidu's self-developed AI intelligent annotation model has helped customers annotate tens of millions of frames of data, saving tens of millions of labor costs.

These three cases correspond to data application, data management and data production process from left to right. In the process of implementation with car companies, we believe that the new model of self-driving research and development is a model that can make the application, production, and management of massive self-driving data more efficient. This is also the value of the tool chain that we have gradually found in the process of gradually communicating with customers, and it has also been recognized by customers.

Just now we shared the early tool chain requirements based on the research and development domain, as well as some thoughts on exploration and implementation with customers. With the gradual development of smart cars, especially the second half with intelligence as the core, smart cars are about to cross the critical point of development and usher in a real explosion.

Therefore, if we want to win the smart driving market, we must carry out mass production R&D layout in advance.

However, most of the current autonomous driving cloud products in the industry mainly provide R&D capabilities for autonomous driving functions from scratch. However, in the mass production stage, car companies are more concerned about developing their own algorithms from scratch to excellence, and the core problem to be solved is the long-tail problem. Although the proportion of long tail problems may only be 10%, the cost and price of solving the long tail problem are very high, which also brings four major challenges to car companies:

The first is data compliance. According to the requirements of Document No. 1 issued by the Ministry of Natural Resources last year, the operation of intelligent connected cars on the road has been clearly defined as a surveying and mapping activity, and it is necessary to entrust a qualified map dealer to manage and control it to ensure the security of geographical information data and meet compliance requirements. Therefore, for car companies, how to transmit data returned from mass production to the cloud in compliance with regulatory policies to help back-end research and development has become the first challenge.
Second is the issue of efficiency. For mass-produced vehicles, massive data of millions of vehicles needs to be transmitted back every day, which places extremely high requirements on the processing efficiency of the platform. So, how to mine high-quality data from massive and mixed business data and solve the long-tail problem has become a huge challenge for car companies.
The third is the service issue. We need to continue to improve the intelligent driving effect and ride experience based on feedback from different users, so as to achieve personalized service capabilities for thousands of people.
Finally, there is the issue of cost. Urban road scenes are complex and testing is particularly difficult. Automobile mass production needs to be expanded to all parts of the country in a low-cost manner. How to quickly adapt to different urban scenarios also requires corresponding solutions.

Therefore, autonomous driving in the mass production era requires new tools and new services.

As the first company in China to lay out autonomous driving, Baidu has taken the lead in completing product upgrades from R&D domain tool chains to mass production domain services. It is committed to serving the mass production of smart driving, overcoming long-tail problems, and winning new market growth points for car companies.

As one of the first companies to enter autonomous driving, Baidu has taken the lead in completing the comprehensive upgrade of the autonomous driving tool chain from the research and development domain to the mass production domain.

The picture below is Baidu Intelligent Cloud’s complete solution for the fully upgraded autonomous driving tool chain for mass-produced vehicles.

Baidu's upgraded autonomous driving tool chain solution has built an intelligent driving production line for vehicle intelligent development for car companies, and built a cloud service for the entire life cycle of autonomous driving. This solution provides car companies with a full-stack tool platform for model development, model training, data collection, data annotation, simulation testing, operation and supervision in the intelligent driving research and development process, making autonomous driving development smarter, more efficient, and It is simpler to help car companies quickly develop and use autonomous driving, and realize closed loop of data, closed loop of problems and closed loop of scenarios.

Baidu's autonomous driving tool chain has been fully verified in practice. Through a large amount of autonomous driving road test data, a scenario library of tens of millions has been formed; based on the leading AI base, autonomous driving simulation tests of tens of millions of kilometers per day have been achieved; at the same time, it has also supported Baidu's 6000w+ kilometers The measured mileage of autonomous driving provides effective support for the rapid iteration and effect verification of Baidu’s autonomous driving technology.

The set of solutions provided by Baidu Intelligent Cloud, based on the tool chain, also exports to customers some of Baidu's practices and experiences in the self-driving research and development process, and integrates the product Know-How we have accumulated in practice with users. Sharing helps car companies stay at the forefront of smart driving and seize new markets for smart driving services.

Below I will talk about some of the main considerations in the practice of the autonomous driving tool chain from three aspects.

The first point is tool chain + compliance services. We know that compliance is the bottom line. Only with data security can the smart driving industry develop healthily and rapidly. As the only company in the industry that understands compliance, provides comprehensive infrastructure, and is also proficient in autonomous driving business, Baidu accurately understands data compliance requirements while building an autonomous driving tool chain while meeting customer business innovation needs. , to achieve the effect of "the original data does not leave the vehicle, the surveying and mapping data does not leave the cloud, the surveying and mapping results are not related, and the qualified map dealers are fully controlled", helping car companies achieve compliance upgrades and transformations throughout the entire process.

Behind "precise compliance", Baidu has established a professional compliance assurance team that can provide full life cycle safety services and multi-department "consultation" services to solve various difficult scenarios and provide car companies with smart driving safety and reliability. Guarantee and protect sustainable development.

The second point is data services. In the past few years of rapid development, car companies have gradually accumulated a large amount of data. How to quickly and accurately mine valuable information from massive data is the key to the iteration speed of smart driving.

The traditional data mining process is mostly carried out through manual annotation or algorithmic marking, which can no longer meet customers' needs for massive long-tail data.

Based on Baidu's years of accumulation in the search field and combined with the Wenxin large model, we have implemented an autonomous driving "data intelligent search engine". Data services are upgraded from "process-based" to "retrieval-based" to achieve the "needle in the haystack" of data mining.

With the help of retrieval data mining capabilities, tag capabilities are even richer. At the same time, through the precise definition of data scenarios, more precise demand definitions and retrieval capabilities can be made based on pictures and text, so that data assets can be quickly developed from scratch and improved. The mining of traditional special scenes takes about a week to complete, but through retrieval services, scenes can basically be obtained with one click.

In the process of autonomous driving research and development, some customized scenarios are often needed to reproduce problems. The traditional solution and prevention is to ask R&D personnel to write some mining algorithms based on the required scenarios, and then arrange the corresponding task workflow to run them. Now, as shown in the video, you can directly search for the required data from massive data through functions such as image search, text search, or scene retrieval, turning disordered data into valuable data resources. .

The third point is simulation services. We have observed that many partners in the industry actually lack data accumulation. We have also been thinking about how to enable industry partners who lack data accumulation to quickly conduct autonomous driving R&D, testing and operation at a low cost. At this time, high-precision and multi-scenario city-level simulation is the key to solving this problem.

Baidu has accumulated massive amounts of autonomous driving data over the years. On the one hand, we automatically build twin cities based on Baidu Map's large-scale road network, achieving high-precision simulation restoration in the twin cities, fully covering differentiated scenarios in hundreds of cities; on the other hand, Baidu has currently accumulated more than 6,000,000+ kilometers of autonomous driving test mileage data and a tens of millions of scenario libraries.

Baidu Smart Cloud's simulation service will provide car companies with real road networks in more than 100 cities and tens of millions of kilometers of scene data, and support large-scale simulation tests of tens of millions of kilometers per day. Car companies can easily verify their autonomous driving capabilities in different urban scenarios on the cloud, solving the problem of "difficulty in driving smart cars out of the city" caused by differences in geographical environment and road conditions, releasing data worth tens of billions, helping car companies quickly accumulate test mileage, and significantly reduce costs. R&D costs and R&D efficiency are increased by more than 10 times.

Since its official release in 2021, Baidu Smart Cloud's simulation platform has gone through multiple versions of iterations. In this process, I am very grateful to all partners in the industry for their trust. At the same time, based on the development trend of large model technology, our simulation platform has also added many new functions, including automatic generation of simulation scenarios, which will be gradually released later and shared with more industry partners.

The development of smart cars is about to cross a critical point and usher in a real explosion. Baidu Intelligent Cloud Autonomous Driving Toolchain effectively solves many challenges faced by mass production of intelligent driving by providing "housekeeper-style" cloud services. Baidu looks forward to working with industry partners to overcome the technical difficulties of autonomous driving, seize growth opportunities, and enter a new era of smart driving mass production.

- - - - - - - - - - END - - - - - - - - - -

Recommended reading

Application practice of large models in the field of code defect detection

Support OC code reconstruction practice through Python scripts (2): Data items provide code generation for module access data paths

Talk to InfoQ about Baidu’s open source high-performance search engine Puck

A brief discussion on search presentation layer scenario technology-tanGo practice

First introduction to search: Baidu search product manager’s first lesson

Exploration and practice of autonomous driving tool chain from R&D domain to mass production domain

Guess you like