Multi task video recommendation scheme, baidu engineers' actual combat experience sharing

The recommended system has a wide range of application scenarios , For example, very popular short video recommendations 、 E-commerce platform commodity recommendation 、 Search recommendation, etc . But you know ？ Short video APP Before showing you a video that interests you , It's usually about predicting whether you're interested in the video , And predict how long you'll watch , Like it or not , Will you secretly collect it and watch it next time . Only when multiple dimensions receive positive feedback , Short video APP Will push this video to your mobile phone . But here comes the problem , There are so many information dimensions you want to predict , If each indicator is regarded as a task , Each task has a model to learn , The online resources will be very expensive . This is the time , Multitasking model came into being , Multi task learning of one model can be completed .

This industrial practice example introduces how to use Flying propeller Large scale recommended algorithm library PaddleRec , Take the short video recommended scene as an example , Provide recommended application solutions for multitasking technology . This technology can also be recommended in the commodity 、 Information flow recommendation 、 Search and other scenario applications , thus , We're done 20+ Implementation of projects within the company . We provide data preparation from 、 The whole process reusable scheme of model training and optimization , Lower the threshold of industrial landing .

Click to read the original text GET Project links

https://github.com/PaddlePaddle/PaddleRec/tree/master/application/multitask_on_video

All source code and tutorials have been open source , Welcome to use ,star encourage ~

The project design

We apply multitasking in this example (multitask). There are many definitions of multitasking learning , We think that more intuitive understanding is a machine learning method as opposed to single task learning . Take short video recommendation as an example to introduce the difference between single task and multi task . Short video recommended , Common multiple tasks include the completion rate 、 Do you like it 、 Collection, etc .

• Single task learning ： Ignore the possible relationship between tasks and learn each task separately ;

• Multi task learning ： Value the connection between tasks , Through joint learning , Learn and optimize multiple tasks at the same time , Both the differences between tasks are taken into account , Considering the connection between tasks , This is also one of the most important ideas of multitasking learning .

The design of recommendation system , It is often a process from simple to complex , Comprehensive accuracy and performance considerations , The most common path is the linear model /LR、 The twin tower model (user/item)、DNN Full connection 、 Joint learning of multiple objectives / More complex feature learning (wide&deep, deepfm etc. ). When selecting models for short video recommendation tasks , The upgrade route in history is from the most basic share-bottom Start to gradually upgrade to expert gating algorithm mmoe and ple, Step by step optimization , Let the model find out the most suitable short video for us .

After the above three rounds of calculation , We use the same short video data set （ Very important , Comparable ） Three models were trained . It also predicts the rate of liking and completion of short video . The results are as follows ：

Optimization experience summary

The recommended business scenarios are very rich , We provide the following optimization ideas ：

How to determine the hyperparameter , Such as expert Number and expert model ？

Actually expert Models and gate Models are not fixed , The most common is multilayer FC The network consists of mlp, Specific implementation can refer to PaddleRec/MMoE. In our practice , When gate The network consists of one layer fc Upgrade to multi tier fc（gate Network parameters increase ,expert Network parameters are reduced ） The effect is obviously improved .

CGC still ple, and mmoe How about the contrast effect ？

Single layer in practice CGC Under the same parameter scale, the effect is generally better than mmoe, There are many layers in our practice ple Only in a few scenarios will it be better than CGC, So in many cases, we directly adopt CGC upgrade mmoe.

Hot start solution when adding new subtasks ？

In practice, it is often necessary to add tasks or modify models . The following solutions have been tried in practice ：

• Complete cold start training . This method is suitable for fast convergence , A scene in which the model changes greatly , Especially for sparse Partial modification .

• Hot start only sparse part (embedding). This is the most common way , Hot start the online training directly sparse Tables can converge quickly , It's also more stable .

• Hot start sparse Part and part dense Model . Re convergence from previous expert model parameters is not necessarily better than complete cold start expert The effect of expert model is better , There may be local convergence traps , This requires the comparison of multiple groups of experiments under specific scenarios .

Model deployment

We provide the following three-step deployment process .

STEP 1： Use save_inference_model Interface saves the model

Use... On the server side python Deployment requires the use of save_inference_model Interface saves the model .

STEP 2： Dynamic static derived model

If you use dynamic graph training to complete , You want to convert the saved model into a static diagram inference, Then you can refer to our to_static.py Script .

STEP 3： Use inference base to predict

PaddleRec Provide tools/paddle_infer.py Script , For your convenience inference The prediction library can predict the model efficiently .

Flying propeller Complete project instructions are also provided 、 Developing documents 、 Data and code , You can refer to the following example tutorials for quick learning ：

Sample tutorial Links

https://aistudio.baidu.com/aistudio/projectdetail/4180842

If you want to know more details , Welcome to our live course , Hand in hand whole process teaching waiting for you to watch .

Flying propeller Examples of industrial practice

Help enterprises to stride forward AI Landing gap

Flying propeller Examples of industrial practice , Committed to accelerating AI In the forward path of industrial landing , Reduce the gap between theoretical technology and industrial application . The example comes from the real business scenario of the industry , Through complete code , Provide solution process analysis from data preparation to model deployment , It can be called an industrial landing “ Automatic pilot ”.

• Real industrial scene ： With the actual AI Enterprise cooperation and co construction of application , Select the high-frequency demand of the enterprise AI Application scenarios such as smart city - Helmet detection 、 Intelligent manufacturing - Meter reading, etc ;

• Complete code implementation ： Provide code that can be run with one key , stay “AI Studio One stop development platform ” Use the free power one button on Notebook function ;

• Detailed process analysis ： Deep parsing starts with data preparation and processing 、 Model selection 、 Model optimization and deployment AI The whole process of landing , Share reusable model tuning and optimization experience ;

• Direct project landing ： Baidu senior engineer teaches users the whole process code practice , Easy access to the project POC Stage .

Wonderful course preview

In order to make the kids more convenient to apply the multi task video recommended example tutorial , Baidu senior R & D Engineer prepares data for in-depth analysis 、 The whole development process from scheme design to model optimization deployment , Hand in hand to teach you code practice .

Scan code to obtain live playback / Video course link

Join the technology exchange group

Focus on 【 Flying propeller PaddlePaddle】 official account

Get more technical content ~

This article is shared in Blog “ Flying propeller PaddlePaddle”（CSDN）.
If there is any infringement , Please contact the [email protected] Delete .
Participation of this paper “OSC Source creation plan ”, You are welcome to join us , share .