Xiaohongshu community content is mainly UGC, and experience-sharing advertising creative formats are more in line with the consumption habits of community users and can also achieve better advertising effects. How to help customers lower the threshold for advertising creative production, produce a large number of high-quality advertising creatives at low cost, high efficiency, and sustainably, and rationally optimize advertising creatives and traffic adaptation, are key issues that the intelligent creative direction is committed to solving .
After more than a year of exploration and construction, we have built a complete set of advertising creative production and selection capabilities to help customers place advertising creatives in one stop. This article will introduce our capability building and technical solutions in the intelligent creative direction of Xiaohongshu Advertising .
In Xiaohongshu, users share and discover the wonders of the world through rich media notes. As an important part of the Xiaohongshu ecosystem, commercial advertising also needs to conform to the aesthetics and tone of the community. In this very large UGC community, sincere sharing based on personal experience is more popular among users, and advertising information is no exception.
Advertising creativity is the main carrier of advertising information. On the one hand, it carries the content that advertisers want to promote and market. On the other hand, it is the first place where users are reached by commercial information. Therefore, the quality of advertising creativity directly affects the effectiveness and effectiveness of advertising . User experience . Whether the advertising picture highlights the product features concisely and directly, whether the title shows the applicable scenarios of the product, etc., are all factors that affect users' attention and are also part of the advertising creativity. The production cost of high-quality creatives is relatively high. If Xiaohongshu can help customers lower the threshold for advertising creative production, and help customers optimize creatives and adapt traffic, it will be of great significance to improve the efficiency of advertising and satisfy user experience.
After years of development, other companies in the industry have provided intelligent capabilities in many aspects such as titles, pictures, and display styles of advertising creatives. Although Xiaohongshu's commercialization started late, it is catching up in the direction of intelligent creativity.
Xiaohongshu Advertising has started to carry out intelligent creative-related construction in 2021, and has successively realized productization in the two main scenarios of search and information flow, and gradually formed a complete functional matrix:
At the material level, because Xiaohongshu is naturally in the form of rich media notes, and the notes themselves contain high-quality picture materials, we took the lead in implementing the ability to select pictures.
Subsequently, the automatically generated titles are provided to the customer as a candidate set. After being screened by the customer, the system selects them during the delivery process.
The materials supported by Preferred have also been expanded from ordinary note advertising to lead advertising, and will also support product advertising in the future.
The intelligent optimization strategy not only builds an efficient E&E algorithm, but also builds a large-scale deep learning model for thousands of people.
Behind the launch of these product functions and the improvement of effects are the solutions to technical problems one by one.
“It’s hard for a good woman to make a meal without rice.” Rich and high-quality creative materials are the source of intelligent creativity to improve the effect. In order to ensure user experience, Xiaohongshu’s advertising format is very native and in the form of notes, so the creative material mainly consists of two parts: cover image and title . There are usually multiple pictures in a graphic note, which have been carefully selected by bloggers or advertisers and can be directly used as material for cover images. For video notes, we can extract key frames from the videos provided by bloggers as material for the cover image. However, whether it is a video note or a graphic note, the blogger only provides one title. How to get more title materials without increasing the burden on the blogger?
Technical Difficulties
Text generation technology has been used in industry a few years ago. The technical difficulty lies in how to balance controllability and diversity at the same time . In order to ensure controllability, early solutions were mainly based on templates or rules for text generation, which caused poor diversity. With the emergence of large NLP models, open natural language generation has gradually become feasible, but it has also brought about a new problem: although the large model seems to be very intelligent in generating smoother content, it is actually easy to apply it. The phenomenon of showing off one's talents to others appears. How to make the text generated by large NLP models controllable but not restricted, free but not lax is a very technically challenging matter.
Based on the most cutting-edge controllable text generation technology and Xiaohongshu's massive high-quality text big data, we have built a Xiaohongshu featured title generation system based on content understanding.
Controlled generation
In terms of generation paradigm, we mainly use two generation paradigms based on language model (GPT2) and Seq2Seq (T5).
● Among them, the GPT2 model uses the rich text of notes as input to fully learn the context information of the notes. At the same time, it combines the feature control information (keywords, title length, whether it contains emoticons, etc.) to generate titles, which greatly guarantees that the generated titles are consistent with the original text. correlation.
Keyword/feature control signal + note text -> generate title
● The Seq2Seq model mainly rewrites the original title by mining high-frequency query, bidword, and using information such as brand, function points, and benefit points as prompts. At the same time, it integrates different style elements to rewrite the original title while maintaining the original title. The core content increases the diversity of titles. The two types of generation models are used together online to ensure better generation results for different advertising notes.
Original title core content + original note text features | Style model -> style title (including title core content)
Pre-training technical base
Good generation capabilities are inseparable from a powerful pre-trained language model that is familiar with Xiaohongshu’s marketing style. Based on the industry's most cutting-edge large-scale language model theory and practical experience, we have built the RED series - pre-training model technology base: providing RED-BERT (understanding), RED-GPT2 (generative) based on Xiaohongshu's internal data ), RED-T5 (seq2seq) and other pre-training models make full use of large-scale language models to conduct unsupervised learning of Xiaohongshu’s massive text knowledge.
Among them, RED-BERT supports the requirements related to content understanding such as selling point extraction, relevance assessment, machine review quality control, offline estimation, etc. of generated dependencies, while the core controllable generation capabilities are supported by RED-GPT2 and RED-T5. The open source generation models on the market are all trained based on public corpus, such as Chinese Wikipedia, etc., which are very different from the language style of Xiaohongshu Notes. Our "RED-" series of pre-training models are based on Xiaohongshu It is trained on internal 1 billion note content and can better learn the language characteristics of Xiaohongshu notes. At the same time, we have also improved the vocabulary adaptability problems in the public model, such as the vocab is too large, the lack of focus on specific Chinese corpus, the lack of emoji and Xiaohongshu expressions, etc.
The following are the offline evaluation indicators of the two generative models. Compared with the open source model, the generative model based on Xiaohongshu scene training has significant improvements in relevant indicators:
An efficient optimization strategy is the key for these creative materials to play their role and reflect their value. The quality of the optimal strategy is reflected in two aspects:
1. Is it a good choice? Being able to select the creative ideas that users like the most is the key to improving the efficiency of advertising.
2. Is it quick to choose? Being able to select good ideas as quickly as possible is the key to the perceived effect by customers and users.
Without prior knowledge, selecting the best one from multiple creative materials in an advertising note is actually a MAB (Multi-Armed Bandit) problem. Commonly used algorithm strategies in the industry for this type of problem include epsilon greedy (ɛ-greedy), Thompson sampling (Tompson sampling), UCB (Upper confidence bound), etc. By comprehensively analyzing factors such as the convergence speed of each algorithm and sensitivity to feedback data, we adopted the UCB strategy.
UCB Strategy Introduction
Multi-armed gambling machine problem (MAB): The slot machine has a rocker (similar to an arm). Shaking the rocker will spit out a certain amount of money according to a certain probability. A gambler faces multiple slot machines and does not know that the slot machine spits out money. How to maximize returns given probability distribution?
The idea of UCB strategy to solve the MAB problem is to use confidence intervals: give each slot machine a confidence interval, the middle value of the interval is the average profit of the machine (the average amount of money spit out), and the width is proportional to the logarithm of the total number of times played. Inversely proportional to the number of times played on that machine. Faced with multiple slot machines, choose the machine with the largest upper bound every time.
In the scenario of creative material selection: for each material, use the revenue (Reward) of multiple exposures of the material + the width of the confidence interval (Bonus) of the material to calculate the upper bound of the interval (Score), and the material with the highest Score for each exposure .
Among them, Reward is the comprehensive income of the cumulative exposure of a certain material (such as the increase in ctr and income, which can also be combined). The calculation of Bonus is related to the number of exposures of the material (Imp) and the number of exposures of all materials (totalImp). , the formula is as follows:
After a certain amount of exploration, the width of the confidence interval will gradually become smaller. At this time, the Reward will also tend to be stable, and it will tend to expose materials with greater comprehensive benefits. If a certain material is less exposed, as the total number of exposures increases, the Bonus of the material will become larger, and the Score will also become larger, and there will also be a certain exposure probability for exploration.
When designing the optimal strategy for Xiaohongshu’s advertising creatives based on UCB, we need to consider the following three issues at the same time:
1. How to balance user experience?
2. How to help advertisers save optimization costs?
3. What impact does browsing depth have on selection?
How to balance user experience
The Xiaohongshu community attaches great importance to user experience, and changes to the advertising display style may affect user experience. When designing algorithms, while striving to maximize revenue on the business side, it is also necessary to take into account user experience indicators.
We use user length of stay (avgViewTime) as a comprehensive indicator to measure quality of service (QoS, Quality of Service). How to calculate user experience (QoE, Quality of Experience) through QoS?
Relevant research on QoE points out that when QoS is low, the user's QoE is already very low. Even if QoS continues to deteriorate, QoE will not drop much further. In the same way, when QoS is high, the user's QoE is already very high, and further improvements in QoS will not increase QoE, as shown in the following figure:
Therefore, the expression of QoE we get is:
Among them is the enhancement coefficient that can be dynamically adjusted. Finally we set the Reward in UCB to:
Among them, is the weight of ctr,
which is used to adjust the dimension of the stay duration part, and
is the weight of the average stay duration.
Experiments show that this Reward design can significantly improve the advertising ctr indicator while ensuring that the user-side indicator does not drop.
How to help advertisers save optimization costs?
After data analysis, it was found that when using the UCB strategy for optimization, the ctr of the material will converge to a more stable value in a short period of time, and after a longer period of observation, the stable performance of the optimal material is better than other materials, so You can consider designing an exit mechanism to reduce exploration costs. Take picture selection as an example:
Exit mechanism : Consider only exposing the picture with the highest Reward for notes that meet a certain condition (sufficient exposure, clicks, or CTR stable in a range). At the same time, within a sliding window time, a threshold is selected to ensure that each picture has a certain exposure. The implementation logic of the exit mechanism is as follows:
Experiments show that the UCB optimization strategy with an exit mechanism can reduce the number of explorations, save advertisers’ optimization costs, and have a positive effect on platform revenue. (Note: The exit mechanism only applies to static materials, and the Reward difference between the materials needs to be obvious)
How does ad browsing depth affect selection?
In the search scenario, users have a strong purpose of looking for specific information. As the browsing depth increases, the more information users accumulate, the lower the need for clicks on materials. Therefore, when an advertisement is exposed in different locations, its ctr will vary greatly, which will have a greater impact on the ctr part of the Reward. In this case, we consider correcting CTR based on advertising pits.
COEC (Click on Expected Click) is commonly used in the industry to measure the quality of two materials. Furthermore, it can measure the difference between two preferred elements with different exposure pits. The calculation method is as follows:
Among them, it represents the exposure of the nth pit,
the click volume of the nth pit, and
the click-through rate of the nth pit.
But in our scenario, we hope that this difference can be attributed to ctr to facilitate the calculation of Reward in the UCB strategy. This requires adding additional hyperparameters to map the COEC value to ctr and ensuring that its distribution is approximate, which increases A certain degree of uncontrollability.
Based on the current business situation, we innovatively proposed a method for correcting CTR pits, named ECOI (Expected Click on Impression).
ECOI (Expected Click on Impression) : Drawing on the idea of COEC (calculating the expected number of clicks based on the average ctr of different pit positions), the clicks at all positions are calibrated and aligned with the first position. The calculation method is as follows:
ECOI can also correct ctr. Compared with COEC, its meaning and value range are basically consistent with the actual ctr, which makes it convenient to directly replace ctr in the model, and there is no need to introduce hyperparameters and find the mapping relationship with ctr. However, ECOI also has its shortcomings. Just looking at the calibrated number of clicks on a certain pit, the value may be larger than the number of exposures (the chance of this happening is very small, especially in the advertising system where the number of clicks is much smaller than the number of exposures. , so it can be ignored).
After comparative experiments, it was found that the effect of the experimental group using ECOI was slightly better than that of using COEC, and was much better than the effect of not performing pit correction.
UCB relies on feedback from posterior data, but many creative materials (including cover images and titles) are distributed in the long tail, and it may not be possible to collect enough feedback data in a limited time to obtain confident results. So we built a large-scale discrete-valued DNN model to supplement creative generalization capabilities.
In addition, for popular creative materials, we also take the user's personalization into account in the model, so that the system has the ability to select thousands of people, thereby further improving the selection effect.
How to improve generalization ability
Features are the learning basis of large-scale discrete value DNN. In the first phase of the project, we carefully characterized the creative materials and users, and built a model with strong generalization and personalization capabilities.
● Creative material generalization: Explore basic creative generalization features, such as image categories, OCR results, text segmentation, entity words, etc. On the one hand, it prevents the model from overfitting on the self-explanatory features. On the other hand, for new ideas in the cold start stage, when the self-explanatory features have not completely converged, the generalization features can provide corresponding information to improve the prediction effect.
● User personalized characterization: Use creative side features to characterize user interests, construct a click behavior sequence based on the picture ID of the notes that the user has clicked in history, and use the generalized characteristics of the picture ID to expand the user behavior sequence, including OCR sequences, picture categories Head sequence, entity word sequence, etc. In addition, based on the user's generalized feature sequence in each feature dimension of the image, we take the top 3 features that appear in each sequence to describe the features that the user is most interested in.
● Creative-side feature crossover: Cross user-side features with current creative features to characterize the user's preference for the current creative. Through feature crossover, the model's discrimination of candidate creatives under personalized prediction can be enhanced. On the query, the text and creative title recognized by the OCR of the current query and the candidate image are matched at character granularity, word granularity and entity words. In terms of user behavior, the current candidate image is matched with the image category and OCR sequence in the user's historical click behavior sequence.
How to Solve Compute Scaling Issues
The inference granularity of the deep model is a deeper material level than the note, that is, a note will have multiple candidate titles and pictures. In order to improve the accuracy of model prediction, we perform the Cartesian product of the title and the picture. Inference allows the model to directly select the optimal combination of title images. But this also increases the computational complexity of model inference by an order of magnitude.
In order to perform model inference at this computational level, we designed a cascaded twin-tower structure to ensure the prediction effect of the model, as follows:
● The left tower in the two-tower model is a complete click-through rate model structure. The input layer only receives the embedding of advertising features. The model structure is much larger in complexity and parameter volume than the right tower.
● The right side is a shallow DNN network, and the input layer adds embedding of creative side features. At the same time, we pass the output of the left advertising tower and the upper hidden layer vector to the right tower to guide the learning of creative features on click-through rates.
● During training, the right tower receives information from the left tower and integrates creative features to learn together. The gradient of the creative tower on the right is not passed back to the left tower, ensuring that the left tower learns the overall performance of the advertisement and does not include information on creative features in terms of features and training.
● During online inference, each advertisement is calculated only once for the advertising tower on the left, and the click-through rate of N creatives is calculated for the creative tower on the right. Compared with the click-through rate model, it only increases the calculation amount of N creative towers, avoiding the explosion of calculation amount caused by creative expansion.
Since there are fewer features on the creative side, it is easy to overfit on the id type features. We have refined the network structure on the creative side:
● Creative features use SENet to learn the weight of the embedding vector of each feature.
● For image ID features, additional regularization is added to SENet, and adaptive dropout is added to reduce the impact of ID features.
● For generalization features, we add user_id, query, and embeddings passed from the advertising side and merge them together, and use DeepNet to enhance the intersection ability of creative features and advertising features with the embedding of generalization features on the creative side.
Due to its generalization and personalization capabilities, the deep optimization model further improves the optimization effect of advertising creatives.
After more than a year of hard iteration, Xiaohongshu commercialization has completed the construction of some basic capabilities in intelligent creative technology, and customer usage has begun to take shape. In the future, we will continue to carry out technological innovation and capability upgrades in creative production, multi-modal understanding, optimal strategies, optimal models, etc.
Author of this article: Business Technology Department (in no particular order)
Liao Fan (Li Peilong) Yu Bo (Kang Chaoming) Henry (Zhang Baoru)
Laoban (Guo Tai) Shenzong (Xie Mingchen) Qiu Ge (Deng Yongchao)
Kunpeng (Wang Jincheng) Treasure House (Zhang Jingpeng) Special Picture (Huang Yanhua)