Editor's note: From an Internet company to a smart terminal solution company, Lu Qiming's transformation may be difficult to understand. However, the impact of the economic environment and personal technical difficulties still made him go to an unknown world without hesitation. As Huang Renxun said a few days ago, "retreat" is not easy for smart people. However, strategic retreats, sacrifices, and deciding what to let go are at the very heart of success. Today's Lu Qiming is exploring a gorgeous immersive world like "Ready Player One". The following is Lu Qiming’s narration——
01 The career past of a veteran
I'm a veteran of the PC Internet era. I started working in 2000, so the length of service is super easy to calculate. So far, I have worked in 5 companies. Since the first company to do digital video surveillance system, I have formed an indissoluble bond with audio and video. After that, every company has never left the multimedia field.
My entire career can be divided into three stages: the first 10 years in a foreign company, writing code, learning technology, leading a team, experiencing international culture, accumulating various project experience, writing blogs, publishing books, and closely interacting with technical communities; In the second 10 years, I have been "running in small steps" in domestic Internet companies. I have witnessed the decline of the PC Internet and the vigorous development of the mobile Internet. I have experienced company mergers and listings... the work is stressful, but also full of sense of accomplishment. Because each of our function iterations or optimizations affects tens of millions of daily active users; now I am working in a smart terminal solution company, and the company is in the process of transformation, and I am also starting my third 10-year struggle.
In 2000, I graduated from Nanjing University and came to Shanghai, where I joined a start-up company making video surveillance systems. In that company, I came into contact with DirectShow for the first time and wrote many articles about multimedia development technology. It was also at that time that I published my first book "DirectShow Development Guide".
DirectShow is a set of application framework and development kit for streaming media processing on the Windows platform provided by Microsoft. It can be used to realize functions such as audio and video data collection, transcoding, and multimedia file playback. Because I use Microsoft technology, my role in promoting the technology community has been recognized by Microsoft. In those years, he was still very active in the technical community and had a lot of output, so he was re-elected several times as Microsoft MVP.
I have always been grateful for the Microsoft MVP program, because of this program, I went abroad (to Singapore) for the first time in my life, and met other MVP friends who I still have contact with today.
Some friends told me that they got into this business after reading my book. As a technical person, I think the growth of personal ability is very important: no matter how busy the project is, don't forget personal growth! Beyond that, building a personal brand is just as important.
Blogging, publishing a book, teaching a class, launching an open source project, etc. are all excellent ways to build your personal brand. When you become famous one day and come to a new environment, you don’t have to introduce yourself too much, or prove yourself too much. Everyone knows who you are (and your public achievements), so start with a little more respect , making it easier for you to integrate into the team and advance your work. This is more deeply felt in the second 10 years of my career. It can be regarded as a surprise. Because when I first started blogging, it was purely a personal technical summary and sharing, and no one could have imagined what happened later.
Lu Qiming's CSDN blog homepage, he has a cute name: happydeer
In the first 10 years, my technology grew the fastest, and I basically learned from the direct leadership, or followed the CTO. I was lucky to meet a lot of good leaders. Their behavior, pure love for technology, and pursuit of the ultimate performance experience always inspire me. If you are taking a leadership position, be sure to lead by example and be a good leader who helps the team grow.
02 "Sometimes, the solution to difficulties is not to persist, but to jump out"
During the period when I encountered a bottleneck in my career, my heart was actually very tormented and painful. In a large factory, each department is relatively independent, and the division of labor is different, so it is easy to solidify. I am a PC client, focusing on Windows technology, and have done it for many years. But with the end of the PC era and the popularity of the mobile Internet, the technology of mobile apps is changing with each passing day, but it has nothing to do with me; no matter how hot AI is, it has nothing to do with me. Feel so lost!
The platform and technology I am good at are becoming obsolete, and the business that the team is responsible for is deteriorating. As a second-line manager, I can’t get up and down, and I am in an environment of "if you don’t advance, you will retreat". That kind of anxiety and hesitation is Very tough.
Sometimes, the solution to difficulties is not to persist, but to jump out.
Later, I chose to come to AAC Technologies. From an Internet company to a smart terminal solution company, this span is difficult for ordinary people to understand. Frankly speaking, behind this change is the impact of the economic environment on the one hand, and the breaking of my personal technical difficulties on the other hand.
Although AAC Technology started out with hardware products, it has now transformed into a technology company that can provide integrated software and hardware solutions for smart terminals. I am still in charge of software development here. The platforms cover Windows, macOS, Android and iOS involve a wide range of technologies, including native development, cross-platform frameworks, H5, applets and game engines, etc., all of a sudden, there is no restriction, and there is a feeling of "the sea is wide and the fish is leaping"!
AAC Technologies RichTap official website: https://www.richtap-haptics.com/
My previous work experience was all in the software field, and my understanding of audio and video technology was a bit narrow. It was only after I came to a company with "hard technology" that I realized that basic components are also very important. Hardware is the economic base, and software is the superstructure. The hardware provides capabilities, and the software fully utilizes the hardware capabilities to bring users the ultimate experience.
03 In AAC, challenge your next self
At present, my goal is divided into two aspects: on the one hand, to improve everyone's awareness of haptic/vibration, and on the other hand, to minimize the difficulty of designing haptics and application integration. Simply put, it is to answer two questions: What is the sense of touch? How to do it?
We are now developing a haptic design tool suite called RichTap Creator, which allows designers to create various expressive vibration effects. We also have a powerful algorithm for automatic adaptation of audio and video. We also provide a multi-platform SDK, allowing developers to easily integrate rich haptics into their apps.
RichTap Creator Haptic Design Toolkit
The advantage of this is that it truly achieves one-time design and multi-terminal implementation . The toolchain is currently complete. Our goal in the next stage is to support dual-motor or even multi-motor scenarios. One of our brand customers has a tablet equipped with dual motors. If you use them well, the vibration can be more three-dimensional and rich. Playing games on this tablet should not be too cool!
In the past year, I realized that audio and video technology is relatively lonely, and it is time to introduce a new partner to them - we dedicate gorgeous video to our eyes, we dedicate beautiful audio to our ears, then our What about the hands or other parts of the body? We can give them a tactile experience. The sense of touch is generated by a small motor. In the past two years, more than 60% of the mid-to-high-end mobile phones launched each year are equipped with AAC’s high-quality X-axis linear motor. Don't let them sleep for too long!
"Ace Racing" equipped with dual X-axis linear motors
In addition to mobile phones and tablets, AR/VR/game controllers/new energy vehicles are all landing scenarios that we pay close attention to. We will provide a full set of design tools and integrated SDK. From the perspective of research and development, I will continue to pay attention to the international standards related to touch, so that our tools and SDK can keep up with the international trend and be the best in the world!
04 Touch, and the future of immersion
Regarding the future of the multimedia ecology, I think the metaverse is a very promising direction, which will involve a lot of technology, but no one knows when the relevant software and hardware technologies and infrastructure will mature, and when the metaverse will be really come.
Therefore, technicians should never forget to innovate and optimize, use new technologies, explore new user scenarios, and strive to create new value. Innovation is not something that happens at a specified time and place, so a little more patience is required.
As I mentioned earlier, more and more mobile phones on the market are equipped with high-quality X-axis linear motors, which can express rich vibration effects. How to use this ability? App developers need to work together! At present, IEEE is in the process of formulating the tactile control interface standard for mobile game scenarios.
In addition to mobile phones, AR/VR devices are the place where tactile experience can play a big role. I have seen the SDKs of various VR all-in-one machines, and the vibration interface functions provided by everyone are still relatively rudimentary. At the same time, I read the OpenXR 1.0 specification, and I feel that it is not satisfactory. Our company is working with a tactile industry association to promote the evolution of the OpenXR standard, so that more VR wearable peripherals such as vests and gloves can work together to bring users a more immersive experience.
Meta's haptic prototype glove featuring 'gas brakes' to be released in 2021
I also noticed the efforts of the MPEG standard working group in this direction. The standard for immersive experience is called MPEG-I. In the future, an MP4 file may contain not only audio or video data, but also tactile data, or even only tactile data. , because in January 2022, the ISO/IEC 14496-12 (ISOBMFF) standard-7th edition has designated haptics as a first-level media type, on an equal footing with audio and video. This trend is very worthy of everyone's attention.
Therefore, the audio and video experience should still be developed in a higher-definition and more immersive direction. It’s just that many people may not realize that the immersive experience does not necessarily have to continue to “dig and dig” on the audio and video. We can also use the sense of touch to make the experience more immersive. The answer lies outside the question...
After more than a year of study and research in AAC, I realized that the hardware and software technology of haptics is gradually maturing, but everyone’s understanding of haptics is still relatively shallow, and the development of the application ecology is far from enough.
From the perspective of application, iQiyi can be said to be at the forefront. Several popular dramas such as "Basic Law of Genius", "Hurricane", and the variety show "China Rap Showdown" have turned on the vibration function, which has played a very good role. The role of exaggerating the atmosphere of the plot and enhancing the sense of presence. There are also several games, such as "Peace Elite", "Awakening of Dawn", "King Racing", etc. In some scenes, racing drift, shooting, impact and footsteps, etc., are integrated with vibration to make the player experience more realistic and immersive. . However, these head applications alone are not enough. I hope that through my sharing at this conference, everyone's awareness of haptics will be improved, and then the haptic ecology will be prosperous together. The sense of touch can be used not only on mobile phones, but also in AR/VR and even the Metaverse, where more immersive experiences are pursued.
In 2018, there was a movie called "Ready Player One", and the story it told was more in line with my imagination of the metaverse. It's just that the current VR devices still have a lot of room for improvement in terms of wearing comfort, and the perception experience is still limited, and there is still a long way to go.
1 easter egg
LVS : Spoiler for your exclusive LiveVideoStackCon content!
Lu Qiming: At the LiveVideoStack conference, no one should have talked about touch. At this conference, I will give you the basic physiological knowledge of tactile science, as well as the realization technology (including hardware and software) of tactile sensation in reality, and show you how to make tactile sensation on various platforms from the perspective of application development, and how to achieve it better experience. I will also introduce international standards related to haptics and their latest developments, hoping to help you broaden your horizons and find some directions for future work.
▲Scan the QR code in the picture or click " Read the original text " ▲
Check out more exciting topics of LiveVideoStackCon 2023 Shanghai Station