I. Introduction
With the shocking release of the new retail project of Autohome not long ago, we directly proposed the ABC new experience mode for the first time to address the pain points of users when looking at cars and choosing cars. A "useful" experience.
in:
A: It refers to the one-stop car selection and test drive service (All in one). Users can view cars through holographic comparison and test drive real cars, making the product comparison experience more intuitive and time-saving;
B: It refers to the exclusive space-time objective evaluation (Belong to customers), which is intended to provide users with exclusive service space without disturbing, and at the same time, they can see the professional, objective and fair vehicle evaluation edited by Autohome, so as to achieve a more worry-free experience;
C: Refers to the immersive meta-universe experience (Creative technology) of Autohome Energy Space Station. Through holographic technology, functions such as 1:1 car viewing, car dismantling, and comparison of cars in the same frame are realized, bringing users in-depth product analysis and updates. Real technology experience.
At the same time, relying on the actual test data of Autohome to create "the most realistic virtual car experience" allows users to experience car acceleration, turning, braking, collision and other car scenes with less effort.
Shocking release of new retail
Shanghai Energy Space Station
Two business requirements
As a one-stop car life service provider integrating "content ecology + tool service + trading platform", Autohome has been thinking about helping users make better decisions about viewing, buying, using and replacing cars. How to break the dimensional wall between online and offline, think about how to connect offline users with OEM dealers, and help users make better car viewing, car selection and car buying decisions. This requires that Autohome needs to have a strong platform capability for the whole process of viewing, buying and buying offline cars.
The first thing we think about is how to enable users to experience and connect multiple cars in depth in one site space, reducing the pain of users running back and forth in the 4S store in the physical space, and at the same time, it cannot be a simple page information retrieval. The car viewing experience close to the real car, but also to give users an experience that even the real car can't give, such as the explosion of the car body, dismantling and refitting, and even a mind-blowing entertainment experience that breaks the dimensional wall, such as the knowledge of new energy vehicles. Science popularization, so the requirements of the business are quite demanding, requirements: multiple cars, a single space, real, objective, beyond imagination, etc.
New retail products
Three technical solutions
3.1 Scheme selection
Combining the core requirements of these businesses above, we investigated some domestic and foreign automotive 3D virtualization presentation technologies, and finally integrated the maturity and effect requirements of the hardware implementation solution, and chose to use holographic film + Unity3D engine technology to present users with an immersive experience. Naked-eye 3D car viewing experience.
3.2 Holographic Technology Selection
The first category is holographic technology in a narrow sense, such as typical holographic technology, which uses the interference principle of light to convert the direction and intensity data of each point of light into interference fringes that can be recorded by development, and conducts photosensitive media on the surface of the lens carrier. Exposure processing, to obtain a holographic grating that records all the information of light, narrow holographic technology is not common, it originated from holographic technology, applied to holographic grating optical waveguide technology, the process of holographic experiment is very complicated, it is difficult for ordinary consumers to use , At present, there are no consumer-grade chivalrous holographic products at home and abroad.
One is the "perceived holography" presented by near-eye display devices and 3D real-time rendering technology, such as Oculus and Hololens, which currently have good consumer-grade products at home and abroad.
Another type is what we call a holographic flat projection, which creates a three-dimensional effect by reflecting the content of the flat-panel display into the eyes of the observer with the help of a semi-transparent and semi-reflective film layer. The advantage is that there is no requirement for user equipment, and it is more suitable for displaying effects in exhibition halls.
Holographic Film Principle Demonstration-1
Holographic Film Principle Demonstration-2
We finally chose the technical solution of holographic planar projection, which is currently the most commonly used technical solution for large-scale exhibition hall performances in China. It can give users a near-real naked-eye 3D viewing experience in a relatively fixed space.
3.3 3D engine selection
We finally chose the current domestic mainstream Unity3D engine to implement. The Unity engine supports forward and delayed rendering for the HDRP high-definition rendering pipeline on the PC side, and uses physically-based lighting and materials. It is ideal for pushing graphics to high-end hardware like PC, XBox and Playstation for stunning HD visuals. It's geared toward high-fidelity gaming, graphics demos and architectural rendering, hyper-realistic effects, and the best graphics you need. It is also much faster than the built-in renderer when targeting high-end graphics. At the same time, we considered that a set of codes can support both HDRP pipelines and URP pipelines in the early stage of design, so that we can quickly upgrade and migrate our capabilities to mobile devices, including mobile phones, AR glasses and VR glasses. At the same time, we use With the assistance of motion capture equipment, it can easily drive mobile 3D digital humans. At present, the domestic Unity technology community is mature, and there are many practical and proven achievements of predecessors that can be directly used for reference.
3.4 Technical difficulties
Our final presentation depends on the complete coordination of the entire set of software and hardware equipment in the physical space. There are high requirements for the overall architecture of hardware equipment, 3D programs, central control consoles, and lighting control. It is necessary to ensure that users In the process of immersive experience of holographic vehicles, there should not be too many technical interference factors to make users feel fake. At the same time, the holographic cabin of the entire energy space station must consider the requirements of supporting large-scale commercial use and fast replicability, which puts very high requirements on the standards of software and hardware and technical implementation.
At the same time, the holographic cabin also needs to serve as an immersive virtual experience technology platform to export its capabilities to other business lines, such as the company's live broadcast platform and online auto show linkage scenarios, to promote the horizontal empowerment of the company's internal technology.
3.5 Holographic Cabin Architecture
In response to the technical difficulties of the appeal, we must think carefully in the early stage of the technology's implementation. Our overall thinking can be summed up in nine words: standardization, high efficiency, and real experience. Therefore, in terms of architecture design, we will gather technical standards, including the standards for accessing holographic cabins through external channels, holographic experience evaluation standards, 3D virtual human and 3D vehicle model standards, etc. Our architecture design will also attach great importance to the efficiency of R&D and online delivery, hoping to minimize the time cost of accessing new cars. At the same time, all standards and high efficiency are based on the real experience, which requires us to continuously upgrade our holographic experience from multiple aspects to ensure that users can always feel "real".
Overall structure
3.6 Unified Standardization
We have designed a set of software and hardware integrated architecture deployment solutions, and polished a set of the best 3D visual presentation specifications, including the physical space distance between the user and the holographic film, screen clarity, lighting brightness, audio surround sound and background video A series of indicators such as selection, etc., all of which form a complete set of SOP specifications, can finally create a holographic immersive viewing effect for users. When everything is standardized and constructed according to our standards, large-scale commercial use is possible.
Unified standard for hardware equipment
All hardware devices in the holographic cabin must be uniformly standardized, including the configuration of the GPU game console, the resolution and size of the display device, the model of the projection device, the specifications of the holographic film manufacturer, the lighting equipment, and the API specifications for the central control instructions of all these devices, etc. After 8 months of continuous polishing and exploration, and back and forth communication and tearing with hardware suppliers, in order to finally achieve the real and shocking user experience of the holographic cabin, we have exported the most stringent set of hardware device access specifications in the industry. The subsequent large-scale commercial promotion of the energy space station has made hardware preparations.
3D model material high standard
After 20+ times of iterative polishing, we finally unified the 3D car model standards in the holographic cabin, and used the highest model standards to control all links from model production, art acceptance to program acceptance, with an average of 3D car models Both have more than 40 million triangular faces, and the size of the model package exceeds 1.5G, laying the foundation for the ultimate shocking experience in the holographic cabin. The model standard specification supports the display of high-precision exterior and interior models comparable to real cars on 4K and 8K screens, and also meets the ten times the number of faces of the general model on the mobile terminal to achieve higher accuracy. The display accuracy requirements for interior leather, wood grain, plastics, screens, etc. are comparable to those of real cars. The exterior material of the vehicle requires a 1:1 comparison with the real car and the real picture, and the lighting effect requires more realistic lighting effects such as ambient light, volume light, halo, running water light, and lantern dance. From the model specification level, it can support 80% of the vehicle holographic experience effect guarantee.
Holographic Experience Standardization
In order to ensure the unity of the overall holographic cabin experience and prepare for the large-scale open cabin to be replicated, we have standardized the hardware and software links in the holographic cabin, such as the brightness of the lights in different experience scenarios, and the sound in different scenarios. We have dealt with all the details of the experience, such as the volume level, the transition effect of the 3D car model in different scenes, etc.
External signal access standardization
As a platform for the holographic cabin, there will be many external cooperative businesses that want to access it, and interact with the vehicles and materials in the holographic cabin in real time. Considering the uniformity of technology implementation and the stability of experience, we will It is required that external businesses want to access the holographic cabin in the form of high-definition 4K video stream signals. For this reason, we have developed a set of general multi-channel video signal acquisition and encoding and decoding systems, which are specially used for front-end It is equipped to process externally accessed video signals to ensure the signal stability and clarity of the external materials experienced in the holographic cabin.
3.6 High Efficiency Architecture
In terms of the overall architecture design, we have gradually explored a set of technical architecture solutions that meet business needs and can also greatly improve R&D efficiency through nearly 8 months. The core includes three aspects: instruction layered design, complete data Driver design and video stream I/O design.
For the module instruction layer design, we imitate the network communication protocol design and separate the instruction encoding layer, communication layer and UI display layer. From the layered isolation of the code implementation level, each layer can be optimized separately, and a long connection between the central control and the command can be established to ensure high transmission efficiency. At the same time, the full-duplex communication mechanism of the WebSocket protocol can effectively control the execution status of the command. and progress, the user can interrupt the execution of the command at any time, so that the user can really control the screen presentation content in the holographic cabin through the central control command.
Command communication layer design
infrastructure layering
In order to achieve the ultimate R&D delivery efficiency, we expect low-code configuration delivery to be completely driven by background data. For this reason, we redesigned the code structure of U3D and disassembled the holographic cabin into 9 modules with 30+ command functions.
Holographic Nine Modules
We split the lecturer module into multiple instructions, with independent entry and exit logic at the module level, and separate life cycle control at the instruction level, abstracting out: Module->Command->Action three-layer data structure, through the division Layer isolation, we can directly assemble a complete command through data assembly, all commands are executed according to a common timeline logic, and users can control the display effect and stage of holographic content in real time with the help of long connection communication and command life cycle status .
Talking Car Module Architecture
For a single command, in order to facilitate data configuration, we launched a command visual editor separately. The core idea is to simulate the design idea of video editing software. It is expected to "edit" a fully functional command in the editor by simply dragging and dropping. With the holographic cabin command, visual editor and background data configuration function, we have extremely efficient function generation efficiency, so as to ensure the efficiency of bicycle delivery.
Command instruction structure
Through a series of optimized designs such as command module splitting and isolation, reasonable data structure design, visual command editor, and fully functional data configuration drivers, the final effect is: the delivery efficiency of holographic bicycles has been shortened from 12 days per car to now. 3 days a car.
command visual editor
We imagine the holographic cabin as a platform. Judging from the business planning, this platform needs to have two characteristics at the same time: it is convenient for external business access, and it is convenient for access to external business.
The core is to deal with these two business scenarios:
Cabin access: Support external services and functions to be connected to the holographic cabin, such as virtual digital human, real-time interview signals, live video signals, external picture-in-picture and other scenarios. For such needs, we design video collection according to the above standardization The access process is collected and accessed through the HDMI physical line and the real-time video stream URL. After being synthesized by our specially developed video transcoding synthesis program, it is output as a video signal to the holographic large screen.
Out-of-cabin output: It can be output to the live broadcast platform, and the live broadcast platform pushes the stream to the CDN node, and the user pulls the live signal data in real time. We can achieve a live broadcast delay of about 5 seconds.
Video Streaming I/O Architecture
Holographic command system architecture
3.6 Virtual Digital Human
In order to achieve the ultimate human-vehicle interaction in the holographic cabin, we joined forces with the team that created Teresa Teng's real-life image for Jay Chou's concert and spent 5 months creating a 3D digital human image of the image ambassador of Autohome, Gu Ailing. Voiceprint collection and machine learning training with a large number of big data samples in the car knowledge base finally polished out an interesting, realistic and informative "digital real person" of Gu Ailing. We can have various interesting interactions with Gu Ailing at zero distance in the holographic cabin.
We are still planning to implement more black technology functions for digital humans, such as intelligent expression and body drive and control, real-time voice delivery, etc. Our ultimate goal is to make digital humans more like real people in terms of appearance, shape and intelligence. Realistic and emotional feedback from users.
3D digital human image (restricted by commercial rights, the face has been blurred)
Four business highlights
4.1 3D holographic real experience + real data
3D car model extremely realistic restoration
Cool vehicle explosion effect
On the basis of cool holographic 3D vehicle display effects, we also rely on Autohome's deep mining and learning of massive user data in the vertical automotive industry, which enables us to understand user demands better than users themselves, superimposing Autohome professional editors and With the professional evaluation data continuously output by the big V, we can output complete neutral and objective evaluation data for each car, and give users the most realistic suggestions for car purchase decisions, so that users can see the real car and see the real point of view.
Motor Highlights Introduction
Suspension characteristics introduction 
Two-car comprehensive PK comparison (data content in the picture is test data, for reference only)
4.2 Holographic Interactive Live Streaming of Internet Celebrities
Holographic multi-channel mixed stream live broadcast
In order to allow more users to experience our cool holographic lecture car without going to the energy space station in person, and to expand the network volume of the new retail energy space station and attract more user traffic to the store for test drives, we have polished a set of A new live broadcast technology system - holographic interactive live broadcast, we superimpose the 3D background of the holographic cabin vehicle and mix the video signals of multiple live anchors into a complete holographic immersive live broadcast signal stream, creating a complete integration of people, vehicles and fields The extremely realistic effect allows us to arbitrarily switch between multiple environments and multiple vehicles in a live broadcast for detailed vehicle explanations, bringing users an unprecedented live broadcast experience.
Our first holographic interactive live broadcast received close to 40,000 viewers, and user comments and interactions were very positive. We will continue to deeply optimize and upgrade the holographic live broadcast experience in the future, hoping to bring complete benefits to both users and the industry. The same new live broadcast experience.
4.3 Holographic Interactive Dream Theater
With the help of the holographic technology of the energy space station, in addition to vividly displaying the details of the vehicle in naked-eye 3D, it can also display more interesting and informative material content in the form of a theater, such as energy science popularization video programs, cultural and artistic popularization promotion programs, and digital human interactive entertainment, etc. And other rich and interesting program forms, allowing users to experience more three-dimensional and realistic visual effects than movie theaters.
popular science program on the origin of the universe
At the same time, we have also developed more interesting functions that allow users to directly participate in the interaction through the integration of MR interactive technology and the 3D space of the holographic cabin, including the interaction between real people and cars, real people and Gu Ailing’s digital human interaction, and real voice and digital human dialogue. Wait for the scene.
A group photo of a virtual digital human and a real person (the face of the digital human image has been blurred due to commercial rights and interests)
Five conclusions
You can go to the Shanghai Energy Space Station to experience the holographic cabin we built with 3D metaverse technology. The effect you see now is just the beginning. Our R&D team has been working overtime to continuously optimize and upgrade product functions and experience. The two-week iterative version is advancing at a speed, and we have also planned more interesting and fun functions waiting for everyone to discover.
The holographic cabin is a brand-new product that our production and research team of the main engine factory has polished from scratch step by step after 10 months. Home to a new journey.