1 Derived from Senkawa
Dewu’s recommended scenarios, in addition to several relatively large scenarios such as the waterfall on the homepage, there are also many small long-tail scenarios, including: channels, venues, in-purchase and post-purchase scenarios, brand walls, etc. In this type of scene, the volume of a single scene is small (both UV and GMV are small), the scene is scattered, and the types are diverse. Individual optimization of such scenarios involves far more cost than output. With the development of business, there will only be more and more such long-tail scenarios, and the optimization of such scenarios needs to be solved urgently. Therefore, we need such a general recommendation platform to undertake these small scenarios, and be able to continuously optimize and bring benefits. "Turning parts into whole", "compatibility", "unified platform", this is Qianchuan.
2 Problems Qianchuan needs to solve
Combining various needs and positioning, Qianchuan, as a unified recommendation system, faces many difficulties and needs to have at least five abilities.
3 Engineering and Algorithmic Solutions
In response to the above difficulties, Qian Chuan proposed a unified recommendation framework. On the basis of Qianchuan ID system, the unified optimization of multi-scenario recommendation is realized.
Qianchuan's recommendation framework is generally divided into five layers
-
APP service layer: docking with various long-tail scenarios, currently accessing scenarios including topics, channels, venues, independent streams during and after purchases, and image output
-
Qianchuan access layer: Currently, two access methods are provided, one is to access through the product delivery service, and the other is to register for Qianchuan direct connection. Qianchuan establishes the Qianchuan ID system based on the differences in access scenarios, and will provide a specific Qianchuan ID (or a collection of Qianchuan IDs) for each access scenario.
-
Qianchuan DPP layer: Provides a variety of DPP recommendation modules to meet multi-type recommendation needs, including product recommendation DPP, multi-type recommendation DPP, floor recommendation DPP, and brand recommendation DPP. The framework of each DPP module is basically the same, and a differentiated recommendation strategy will be designed according to the type of recommendation.
-
Algorithm layer: build a complete recommendation link, and optimize efficiency and experience in the entire process of recall, rough sorting, fine sorting, and strategy.
-
Recall stage: Design 5 types of recalls including I2I, U2I, etc., deal with scenarios, behaviors, and interest deviations as much as possible, and recall users’ favorite products.
-
Rough sorting stage: To meet high-performance requirements, provide single-target and multi-target rough sorting capabilities to provide room for subsequent fine sorting.
-
Refinement stage: Carry out a series of model iterations in terms of scene differences, user interests, multiple goals, and promotional applications.
-
Strategy phase: Combined with business needs, it provides capabilities such as policy intervention, scenario differentiated configuration, flow control and power adjustment, diversity rearrangement, and multi-type distribution.
-
Infrastructure layer: Relying on strong capabilities including machine learning platform, indexing platform, feature service, flow control platform, etc., a complete set of Qianchuan recommendation framework can be created.
-
4 Algorithm iteration process
4.1 Evolution of Recall Rough Sorting
Due to the characteristics of Qianchuan's multiple business scenarios, the recall and rough sorting stages faced a series of challenges, including: differences in scene behavior preferences, user interest deviations in multiple scenarios, and differences in scene target positioning.
Challenge 1: Differences in Scene Behavior Preferences
The behavior of Qianchuan users is relatively scattered, and the behaviors in different scenarios are sparse and vary greatly. In order to deal with this kind of deviation, Qianchuan collects the long-term and short-term behaviors of users in all scenarios, and designs a series of strategies such as I2I, redirection, Trigger selection, and vectorization.
- I2I, Redirect Capture Behavioral Preferences
- Different trigger selection strategies take into account long-term and short-term behaviors
- GraphEmb, RankI2I realize the vectorization of behavior preference
Challenge 2: User Interest Bias in Multiple Scenarios
The interests of users in different scenarios are not completely consistent. The specific performance is as follows: male user A pays more attention to products such as shoes, sports, 3C, etc. on the theme landing page, and pays more attention to cosmetics related products on the gift channel; female user B pays more attention to the theme landing page The page is more inclined to bags, dolls, etc., and the gift channel pays more attention to basketball, casual clothes and other products. The interests of users in different scenarios have both commonality and differences. Qianchuan uses the DSSM vector representation of the whole scene and the MIND vector combined with the scene features to represent the interest deviation of users in multiple scenarios.
- DSSM introduces full-scenario data to represent users' basic interest in commodities
- MIND combines scene features to expand user representation in multiple scenes and multiple interests
Challenge 3: Differences in scene target positioning
Due to the different scene positioning, the corresponding goals are also inconsistent. The whole can be summarized as dpv-oriented scene and uv value-oriented scene. The dpv-oriented scene corresponds to the estimation of clicks, and the uv value-oriented scene needs to estimate clicks and conversions at the same time. For the two types of scene targets, Qian Chuan designed a rough row of twin towers and a rough row of ESMM models to achieve differentiated predictions by target and eliminate the difference in scene target positioning.
- Rough sorting twin-tower model realizes click prediction
- Coarse sorting ESMM realizes multi-objective estimation of clicks and conversions
4.2 Sorting Model Iteration
Challenge 1: Modeling User Interests
User interest modeling has always been one of the most important optimization points in recommender systems, and the user's historical behavior is the most direct expression of the user's potential interest.
Previous work mainly focused on modeling users' real-time and short-term behaviors. Only using recent behaviors cannot model users' long-term stable interests and periodic behaviors. At the same time, the data feedback loop of the recommendation system is limited to local popular ones. content.
On the other hand, it is the lack of feature crossover. The model migrates from the deepFM single-objective paradigm to the multi-objective model based on the dmt paradigm. The fm side structure is removed. Although the user behavior sequence features can be fully mined, the sparse features on the model There are still few crossovers, and there is a certain room for optimization.
Challenge 2: Modeling Scene Differences
Because different scenarios often have their own unique positioning, the users of the service and the products included are quite different. However, the user distribution and behavior preferences of small scenarios also fluctuate greatly with changes in activities and operating strategies.
Newcomer landing pages, newcomer channels and other scenarios, the proportion of new users is relatively high, the click rate is high and the conversion rate is obviously low; the subsidy channel mainly focuses on cost-effective products, and the click and conversion intentions are good, but aov is significantly higher than the overall The audience of the women’s channel is basically women. The product collection of this scene is significantly different from that of the mainstream scene. Female users also have obvious preferences. According to the data, the proportion of women’s clothing and bags in this scene has increased significantly; while the venue, etc. Scenarios, daily and big promotion user distribution and commodity pools have changed a lot, and user behavior has also changed accordingly. During the warm-up period, the willingness to collect continues to increase until the day of the big promotion is concentrated and converted. For post-purchase, post-payment and other scenarios, since user needs have been partially met, the browsing depth is correspondingly low.
Iteration 1: Modeling User Interests
In order to fully model the differences in user interests, on the basis of constructing various explicit cross-statistical features of users and products in the scene, we further improve our model by optimizing the full modeling of user behavior sequences and implicit feature crossing. Accuracy in characterizing user preferences.
-
First, we added a transformer structure to process the user's long-term and short-term user behavior sequences, and merged and deduplicated the behavior sequences to increase the information capacity.
-
We have added an explicit intersection of user statistical features and sparse features to improve the model effect.
-
Add the intersection features of Qianchuan id, product, and user attributes to user behavior, and use the co-action structure to do implicit intersection.
a. The user behavior sequence is used as the Feed Feature, and the sequence embedding before the attention is reused
b. cspu, qcid, gender, brand, etc. are used as Induction Features to build a 3-layer mlp
c. Do 3 factorials to increase the high-order feature crossover
Iteration 2: Multi-scene difference modeling
In order to fully model the differences in different scenarios, on the basis of constructing the cross-statistical features of commodities, brands, and categories in the scenarios, we further optimize the model structure to fully learn the differences in user preferences in different scenarios.
- Describe scene preferences and scene efficiency differences by constructing features
-
Characterize user activity, user life cycle tags and other characteristics by constructing features
a. User label - life cycle
b. User-different scenarios-activity [exp｜clk｜buy｜clickbuy && cspu｜brand｜cate]
c. User-full scene-activity [exp｜clk｜buy｜clickbuy && cspu｜brand｜cate]
-
Through the mmoe structure, the model can further learn the difference of the scene.
a. Although the features are enriched, if the samples of different scenes are mixed, using only one model will cause different scenes to interfere with each other and cover each other, and it is difficult to achieve the optimal effect. Therefore, the model structure is further adjusted and the structure of mmoe is reused.
b. In order to highlight the differences of the scenes, it is necessary to adjust the input of the original MMoE Gate network. For this reason, only the information of Qianchuan id is selected as the feature, and the information of the scene id is used to select the experts, so that different scenes are output through softmax Different Gate weights.
c. For different scenarios, the model can perceive the differences of the scenarios, and different scenarios can choose different combinations of experts sub-networks, so as to realize the differential modeling of different scenarios.
-
Through the poso structure, the model can further learn the differences of user groups.
a. In the initial use of user features, it was found that the features of new users were not fully learned and utilized by the network because of the scarcity of data (new user samples only accounted for 4.6% of all samples), resulting in almost no influence on the network even if these features were masked Parametric distribution.
b. In order to solve the processing of unbalanced distribution features, the POSO structure is adopted, and the user label features are mainly input on the Gate side, such as whether it is a new customer, whether it is a seller, gender and other characteristics. Then when POSO is used in the fully connected structure, the output of each layer of the fully connected and the output of the gate are multiplied bit by bit.
According to the estimated click-through rate, we can see that the accuracy of the model in estimating new customers has been significantly improved, and from the perspective of the overall pcoc, the modeling effect of the model has also been significantly improved.
5 Future Prospects
5.1 Business
Qianchuan, as always, supports the development of various businesses:
-
Standardize the docking process, continue to expand new scenarios, and achieve accuracy and efficiency, such as: the main map of the venue, the King Kong position map, etc.
-
Summarize business demands, refine the commonality of requirements, and further expand capabilities, such as: multi-type distribution capabilities, diversity rearrangement capabilities, etc.
-
Maintain the stability of the system, improve monitoring and Aegis inspection, and find and deal with related problems in a timely manner.
5.2 Algorithm
Qianchuan’s recall iteration will continue to focus on dealing with scenarios, behaviors, and interest deviations: on the one hand, start with scenario-related features, and iterate the recall model around the scenario features, such as: introduce the scenario features as an independent tower, and add scenarios, users, and products. On the one hand, continue to dig deep into the characteristics of users and products, strengthen the proportion of price factors, enrich price features, deepen the relationship between prices and scenarios, users, and products, and achieve accurate modeling.
Qianchuan Jingpai iteration will continue to focus on the direction of multi-scenario difference modeling, on the one hand, explore the advanced direction of existing modeling paradigms, such as PepNet and so on. On the one hand, continue to mine features, such as user behavior sequences in the scene, and explore the model structure adapted to it, such as SAR-Net.
In addition, in many practical applications of Qianchuan, there are many non-waterfall scenarios, and K products may be presented to the user at one time. For example, scenes such as King Kong position, Zhongtong position, inside the venue floor, product classification, etc. At this time, the K products are presented on a card, and the K recommendation positions are mutually influenced. Will try to explore the algorithm direction of full screen optimization such as generating rearrangement.
The product recommendation at the Qianchuan venue is easily affected by external intervention and the schedule of the big promotion. The ranking model often performs better than the baseline on a daily basis, but the efficiency drops sharply when it comes to the big promotion. This is both a challenge and an opportunity for algorithmic technology innovation. Directions such as continuous learning, real-time ODL, and LTR will be further explored.
5.3 Versatility
In addition, we will continue to upgrade to make Qianchuan more versatile, and strive to achieve componentization, building algorithm libraries, flexible expansion and replicability.
Scan the QR code to add WeChat Assistant
If you have any questions, or want to know more technical information, please add the assistant WeChat:
Text: Tmac, Ah Shou, Yaobik
This article belongs to Dewu technology original, source: Dewu technology official website
It is strictly forbidden to reprint without the permission of Dewu Technology, otherwise legal responsibility will be investigated according to law!