Charming Voice Family Group (10)
@大玲, I just met the new Sichuan baby - Cherry, so cute!
No, this girl is really rare, I will drag her into our big group right now
"Da Ling" invites "Cherry" to join the group chat
@Everybody Our big family welcomes the first baby, everyone welcomes @cherry
Hello brothers and sisters! ! I am Cherry from Chengdu, Sichuan. I am 7 years old. I am very happy to be a family with you!
Welcome to the cute little cherry, if you are interested in cherry, you can come to the recording studio to play with us
Want it! Cherry has just learned a nursery rhyme, waiting for Ha Er to sing it to her sister.
Our child is versatile, don’t wait to go to the recording studio, let’s give you a song first
Tintin the cat in a red dress
Miss Gao, a matchmaker
pockmarked empress drummer
Banzhu Yaya lifts up and walks away
Car lifter, lift slowly
look trip the girl
Embroidered shoes worn by girls...
Click below to listen to the full nursery rhyme~
It's like that, isn't it all the hot pot flavor? 
The Sichuan tone cherry launched by Xiaoai in December last year is the industry's first dialect tone created from real children's recording data. At the same time, we also added super anthropomorphic technology to this tone, and reproduced a variety of modal particle expressions to make Xiao Ai's reply more anthropomorphic and natural.
As the only child newly added to the "Voice Family", the voice is full of milk, yet domineering and cute, so authentic "Sichuan dialect", who made it? Let’s get to the point, please let the engineers of the technical team come and introduce us to the implementation~
01
"Sound" enters the hearts of the people, where does the authentic rural sound come from?
In order to allow users to experience the fun brought by sounds of different timbres, the Xiaoai Sound Store has continuously launched new sounds of different timbres for the public to choose from. There are currently 10 voices in the voice store, which are divided into 6 series: dialect, girlfriend, translation, ancient style, IP, and technology.
As a major cultural art of local culture, dialect is not only the soil that nourishes folk culture, but also a living fossil of culture that is passed on orally. Hereby, we have developed a series of "dialect timbres" , which are currently available in Northeast dialect and Sichuan dialect. The birth of the dialect series was a deliberate decision for the voice team.
On the one hand, the divergent pronunciation of dialects has brought great challenges to the accuracy of dialect synthesis. It should be known that the main differences between modern Chinese and various dialects are pronunciation, vocabulary, grammar and other aspects, and pronunciation is particularly prominent. Unlike Mandarin, which has a unified national standard, there is no unified pronunciation of dialects. Not only are there many types of dialects (eight dialects with the largest number of users, including Northeast dialect, Beijing dialect, Jilu dialect, Jiaoliao dialect, Central Plains dialect, Lanyin dialect, Jianghuai dialect, and Southwest Mandarin dialect), each dialect has internal Not unified, with several dialects and many "local languages" distributed.
On the other hand, due to the influence of Mandarin, the pronunciation of some words in some dialects has some differences among people of different age groups. This makes it more difficult for dialect synthesis to be "authentic". After all, if the dialect is to be in place, it must be "authentic and colloquial", which is also one of the important criteria for us to measure whether the timbre is up to standard.
In order to protect this "culture" and promote the spread of local culture carried by dialects, Xiaomi engineers continue to explore and challenge more difficult voice technologies.
02
Teana's "Children's Voice", a look at hard-core technology
The recently added Sichuan children's dialect has received widespread praise from users and friends as soon as it was launched. I think it won the favor of all brothers, sisters, uncles and aunts!
At present, most of the dialect voice libraries on the market are mature female voices and male voices, and there is a lack of children's voices. Restricted by many factors, Sichuan Children's Cherry Voice is the most difficult one in the current dialect series.
On the one hand, the Sichuan dialect is different from the Mandarin phoneme set and the pronunciation methods are different. From a technical point of view, it generally requires >3000 sound materials to directly train the sound model, but faced with a corpus of only more than 500 sentences , we lack a large amount of children’s Sichuan dialect data as the basic model, technically it is impossible to directly train the Sichuan dialect sound model, and the technical implementation is relatively difficult.
In order to overcome the difficulty of dialect synthesis on small-scale data, the engineer proposed a method based on transfer learning, which divides the synthesis of children's voice dialects into two stages. First, low-resource dialect (cross-language) synthesis technology is used to obtain a children's timbre data model, and then on the basis of more than 500 sentence corpora, iteratively trains a basic model dedicated to Sichuan dialect.
The difference in pronunciation between Mandarin and dialects determines that it is difficult to directly use the basic model of Mandarin timbre, and use the more easily obtained pronunciation model of adult dialects for timbre migration. The pronunciation categories are more matched, and it is easier to adapt to training, and then complete the overall children's voice An augmentation of , used to train child speech rate style models. Of course, when the training data is reconstructed and amplified through technical means, a small amount of TTS background noise may be introduced, and the noise reduction coder can be used to reduce the noise of the synthesized speech appropriately.
On the other hand, dialect recording is difficult. For children who are in the learning and imitating stage, her recording materials are collected through adults leading and reading, which greatly limits the number of recordings and increases the difficulty of recording. In the end, more than 500 sentences of audio recordings were collected. Although the number can be further expanded through technical means, the impact of original recordings on the quality of synthesis is the most important. Moreover, the child's pronunciation stability is also poor, the speech speed is slower, and the intonation is higher. The pronunciation of each word changes too much in different sentences, and the degree of freedom of expression is large, so the difficulty of modeling will "increase with the tide".
In order to further improve the reasoning speed, the method of distillation learning is introduced , and the knowledge of the teacher model is transferred to the student model. The reasoning speed is faster and the sound quality is almost not reduced. The overall process of model training is shown in the figure below:
After obtaining the model's initial synthesized voice, in order to further improve the naturalness of the synthesis, the engineers also used the "super anthropomorphic technology" on this basis , so that the artificially synthesized voice (the voice of cherry) is different in terms of intonation, sentence breaks, and speech speed changes. It is more like a real person's pronunciation, and also reproduces the expression of a variety of modal particles, so as to alleviate the rigidity of the electronic mechanical sound and make the sound more natural.
With the blessing of AI technology, sound synthesis is no longer a static voice package, but a dynamic voice assistant with a "brain", which is more human and realistic. The synthesis of children's dialect timbres is a breakthrough in intelligent voice technology, bringing users a better "interactive" experience.
03
"Sound" into the ear, technology back to basics
"Voice" is a special medium. In the past, there was Su Shi's "hometown accent without accompaniment and hard thinking", and later there was Gao Qi's "hometown accent is true return to the ear". It can endow it with intimacy that cannot be seen and touched at any time, just like "the local people do not know the local accent", carrying emotions to the listener.
In the future, Xiaomi engineers will continue to research and develop, covering wider and more dialect timbres, focusing on "authentic and colloquial" sound technology, to create the ultimate experience in dialect mode for users! At the same time, the Xiaoai Sound Store will continue to launch new timbres, so that more people can experience and feel the charm of dialects. What will the next sound look like? Let's stay tuned!
Finally, Cherry invites you to come to Sichuan to have fun!
