Today, I would like to introduce you to a group of Tencent people who are not doing their jobs properly. They are called "noise hunters" in the arena.
They were immersed in the laboratory, but also shuttled in the vegetable market and wandered along the road .
Noise is their prey, sniped, captured, and destroyed.
The superb hunting skills are making a group of special people say goodbye to the noise and let them "hear" the world.
Main writer: jar
Edit: cross
Co-produced: Tencent Industry Internet Official Account Tencent News
"The boss of the pork shop has started to chop minced meat, let's go!"
Wang Yannan's eyes glowed and rushed over like a hungry tiger. His good partner held the computer in his hand and almost missed it.
A radio stick stretched out to the table, listening to the violent collision of the machete and the cutting board, Wang Yannan was like a treasure.
"Young man, I've seen you for a few days. Why don't you have two catties of pork belly?" The butcher shop owner glanced at Wang Yannan, who was dressed in a shirt and was gentle and gentle, and joked.
This is an ordinary working day afternoon. On the fifth day of Wang Yannan's shuttle in the vegetable market, these days he has caught a lot of shouts, meat chopping, footsteps, and the sound of their blending together.
But these are far from enough. He will go to Shennan Avenue to catch the roar of cars.
This group of "paid grocery shopping" engineers comes from a team in Tencent's Multimedia Lab researching AI noise reduction technology. Their daily job is to deal with noise-to collect it and eliminate it.
Colleagues jokingly called them "noise hunters".
Fight against noise!
Why do you want to make noise?
"The telephone was invented more than a hundred years ago, but mankind has not solved the problem of call noise." Wang Yannan said, shaking his head.
Wang Yannan is a member of the AI noise reduction team, a PhD from the University of Science and Technology of China. From undergraduate to master's to Ph.D., years of research in the audio field have made him extremely sensitive to sound, deeply aware of the impact of noise, and sure that noise reduction technology can bring changes to people's lives.
Looking at the history of more than one hundred years of human voice calls, it is actually a history of fighting noise.
The communication equipment is constantly updated and iterated. We can talk anytime and anywhere on the road, in the crowd, and even hope to have a normal conversation a few meters away from the microphone. These scenarios place higher requirements on noise reduction technology.
Wang Yannan gave an example, “In the vegetable market, the reason why the boss of the pork shop can be heard clearly is because the ears selectively shield the meat. What our team has to do is to make the machines and equipment behave like people. "
Therefore, if you want to eliminate noise, you only need to identify the noise and then actively intervene.
This seemingly simple answer has troubled technicians for hundreds of years, because the difficulty lies in identifying what is noise. Wang Yannan and his team have long realized that sound processing is the most difficult—sound data is one-dimensional, images are two-dimensional, and video is three-dimensional. The fewer dimensions, the more difficult it is.
They got the blessing of AI.
"We will collect a large amount of sound data, cut and clean it, extract features, and then add model training. If the model accuracy rate is less than 99%, we will continue to collect it until the standard is reached."
This is the beginning of this article. Wang Yannan captures sounds and samples at the vegetable market.
It's just that before capturing the "high-level" noise in the vegetable market, they experimented more in the office.
At the beginning of the project, colleagues would see Wang Yannan wandering around with a radio stick every day.
He captured the keyboard sound of his colleagues, made the sound of closing doors, played back the sound of cups, and the sound of paper towels. He collected almost all the noises he could think of in the office.
Through massive collection and application, the AI noise reduction model based on machine learning, the team increased the recognition rate to 96.2%, surpassing most open source models.
At this time, they still don’t know that a pandemic sweeping the world is coming. The demand for remote office has spawned a product with hundreds of millions of users-the birth of Tencent Conference. The core technology behind the scenes is the AI drop developed by these engineers. Noise technology.
It is precisely because of these engineers who are not doing their jobs properly, in Foshan, 200 kilometers away from them, the life of a girl with severe hearing impairment, Xiaoting, has undergone tremendous changes.
Just to wake up the "sleeping ears"
Back in time fifteen years ago, a little girl from an ordinary family in Foshan was born, her name was Xiaoting.
Xiaoting couldn't hear her voice since she was born, but no one noticed it. She likes to dance, and she dances to the music of the store outside when she is over one year old. Mom learned later that she couldn't hear, but was watching the lights on the speakers flicker and dance.
She couldn't speak until she was two years old. I went to the hospital to find out that she was deaf from birth.
Drumming became her only way to connect with the dance world.
She couldn't hear the music, she could only feel the vibration of the music drum through her toes, trying to remember the action. Sometimes, she couldn't even feel the vibration of music, so she silently counted the beats in her heart to keep up with the rhythm.
Nevertheless, they soon learned a more difficult fact: as Xiaoting grows up, her hearing will continue to attenuate, and the only drumbeat will disappear from her hearing.
In 2018 , a turning point emerged.
Guangdong included cochlear implants in social security, and Xiaoting got one of the precious free quotas. One month after the operation, Xiaoting's cochlear implant was turned on. For the first time, she heard her name from her mother's moving lips, "Your name is Song Xiaoting."
After putting on her cochlea, Xiaoting went downstairs to the supermarket, crossed the street, took the bus for the first time...a world of sound, slowly unfolding in front of her.
That year, Xiaoting appeared on the stage of CCTV Children's Spring Festival Gala. Xiaoting wore a long green dress and showed a bright smile in the center of the stage.
From the outside world, wearing a cochlear implant and hearing voices seemed to have changed the fate of this girl. She became confident, and the future began to be full of infinite possibilities. But that is not the case.
" I can hear, but I don't understand."
In fact, as Xiaoting said to her mother, “heard” does not mean “heard clearly”. The voice she heard was still far from normal. Most of the sounds from the cochlear implant to her ears were noises, without details.
You can think of a misty bathhouse, everything is smoky, you can't see or feel it. This kind of sound, which is blocked by a layer of cloth and cannot be determined, is the world that the hearing impaired people hear.
In 2020, Xiaoting left Foshan and went to Guangzhou to study in high school.
In the same year, the fate of the little girl met with a group of engineers.
Magical Noise Hunter
"For us able-bodied people, we can imagine how painful it is to look at things in misty scenes. For the hearing impaired, they will be trapped in such an environment 24 hours a day for their entire lives. If our Technology can provide them with some help and make them hear more clearly, which is really a very meaningful thing."
Because of work needs, Shang Shidong, who has been dealing with noise all the year round, came up with this idea after he came into contact with the "hearing impaired".
Shang Shidong is the head of AI noise reduction technology research in Tencent's Multimedia Lab. He has been rooted in the audio field for 25 years. Witnessed the history of audio technology iteration. "Technology can improve lives and make up for shortcomings." This time, he set his sights on the hearing impaired.
"Public information shows that there are more than 85 million disabled people in China, but in life, you rarely perceive their presence, just as you rarely see blind people walking on the blind path," Shang Shidong believes, "this is because we The'barrier-free construction' is not good enough."
After initiating this idea, Shang Shidonghe's team did a lot of research on the hearing impaired, and analyzed many domestic and foreign research on the application of noise reduction technology in the hearing impaired, and found that "noise" really troubled the hearing impaired. A big obstacle.
Shang Shidong quickly found the domestic cochlear implant manufacturer Nuoerkang. The two sides hit it off and decided to develop a new generation of cochlear implants supported by AI noise reduction technology.
Xiaoting has become an early tester of a new generation of cochlear implants.
"She is very excited, very happy, and the whole person is in very good condition. She herself is a pretty lively little girl." Shang Shidong recalled the scene after seeing Xiaoting put on a new generation of cochlear implants.
After using AI noise reduction technology, Xiaoting can not only hear the sounds of birds and wind, but can even hear her mother's voice through the noisy environment.
AI noise reduction technology awakened Xiaoting's sleeping ears, and more delicate and vivid voice details poured into her world. She said, "As if by magic, the noise in my ears was taken away by the hunter."
However, the initial research and development of the magic noise hunter was not smooth.
From the proposal of IDEA to algorithm verification and product DEMO implementation, the technical team of Shang Shidong and Nuoerkang spent nearly a year and experienced numerous version iterations during this period.
Cochlear implant chips are limited in size, poorly compatible, and weak in computing power, and are not enough to carry a large number of calculations. After repeated discussions and verifications, they found a solution-a mobile phone companion plus a cochlear implant.
In short, the calculation process is transferred to the mobile phone, the mobile phone processes and filters the signal, and then sends the signal to the cochlear implant through the Bluetooth device.
The mobile phone companion's solution requires extremely high latency, and people will feel uncomfortable once it exceeds 200 milliseconds. Just like watching a movie, if the sound delays the video, the audience will feel obviously uncomfortable.
Faced with huge challenges, Shang Shidong led the team to stay up all night to tackle tough problems, trying to find a faster AI algorithm. Finally, combined with the experience and configuration data in the Tencent meeting, the delay was controlled to less than 150 milliseconds.
On September 27, 2020, which coincides with the International Day of the Deaf, Xiaoting, a new-generation cochlear implant wearer Xiaoting, came to Shenzhen with her parents and finally met the "hunter" who caught the noise in her ears-Shang Shidong And his team.
On the same day, Tencent Multimedia Lab announced that, in conjunction with Shenzhen Barrier-free Information Research Association, Tencent Charity Foundation and other institutions, it will open Teana Audio AI noise reduction technology to the industry and launch "Teana Action", hoping for more manufacturers and developers like Nuoerkang Participants join the industry that focuses on hearing impaired people.
"In the past, our work was more focused on technological algorithm breakthroughs and product development. We did not expect that our technology could also bring some changes to the lives of the hearing impaired. This made my team and I greatly encouraged." Shang Shidongcong Xiaoting saw the infinite possibilities of technology.
"Next, we hope to make the algorithm better. While helping more hearing impaired people hear more clearly, we can also explore more application scenarios of AI noise reduction technology, such as elderly people in their 50s and 60s. After hearing loss, it can also be improved through our technology."
This group of noise hunters is also a microcosm of Tencent's "undoing business" industry people. They capture noises between streets and lanes, they also snipe the noises and wake up their sleeping ears.
They are using technology to continuously improve the world and make the details of the corners of the world clearer.
19:30, January 7
Tencent programmer video account live broadcast
Scan the QR code to make an appointment tomorrow night live broadcast