阅读说明：本技术 一种基于多模态的深度兴趣网络推荐方法 (Deep interest network recommendation method based on multiple modes ) 是由 万振民 胡彬 于 2021-08-30 设计创作，主要内容包括：本发明提供一种基于多模态的深度兴趣网络推荐方法,步骤包括：数据采集、数据预处理、网络模型训练、模型在线推理。本发明利用新模型提取微博画像特征、上下文特征、用户画像特征、用户历史行为序列、微博文本、微博图像、社交关系等信息构建个性化的深度学习微博推荐模型,满足用户个性化阅读,解决现有技术中的深度学习模型无法解决图像、文字、声音、动画等多种模态形式的特征对模型的影响,从而提高排序推荐模型的效果。(The invention provides a deep interest network recommendation method based on multiple modes, which comprises the following steps: data acquisition, data preprocessing, network model training and model on-line reasoning. According to the method, the novel model is used for extracting information such as microblog picture features, context features, user picture features, user historical behavior sequences, microblog texts, microblog images and social relations to construct the personalized deep learning microblog recommendation model, so that the personalized reading of a user is met, the problem that the deep learning model in the prior art cannot solve the influence of the features of various modal forms such as images, characters, sounds and animations on the model is solved, and the effect of the sequencing recommendation model is improved.)

1. A deep interest network recommendation method based on multiple modes is characterized by comprising the following steps:

s1, data acquisition;

s2, preprocessing data;

s3, training a network model;

s4, model online reasoning.

2. The multi-modality-based deep interest network recommendation method of claim 1,

step S1 specifically includes embedding points for user behavior according to the front end, collecting user behavior data, storing the user behavior data in hive, and triggering data preprocessing.

3. The multi-modality-based deep interest network recommendation method of claim 1,

step S2 specifically includes reading the hive original data set, performing data preprocessing, sample cleaning, deduplication, missing value processing, reconstructing the sample into a data set suitable for the feature structure of the network model, splitting the data set into a training set and a sample set, and triggering network model training.

4. The multi-modality-based deep interest network recommendation method of claim 1,

step S3 specifically includes constructing a network structure of a new CTR ranking model using tensorflow, and obtaining the new CTR ranking model using training set training.

5. The multi-modality-based deep interest network recommendation method of claim 1,

step S4 specifically includes providing a prediction service for the trained ranking model through a docker container and a tensoflow serving mirror, and the user remotely requests the iterative ranking model to perform the prediction service through HTTP or GRPC, and ranks the microblog resources recalled by the user.

6. The multi-modality-based deep interest network recommendation method according to claim 3, wherein the step S2 specifically comprises the following steps:

s21: obtaining the image characteristics of the microblog through a Resnet34 network model;

s22: pre-training a text of a microblog to get embedding, and then performing microblog text sequence feature processing on the text through a GRU (general packet radio unit);

s23: obtaining the imbedding vector characteristic of each user by the user relation graph through the node2 vec;

s24: performing concat on the output results of the steps S21, S22 and S23, and then inputting the output results into a 2-layer full-connection layer network model;

s25: extracting user interest characteristics of history sequences related to reading, praise, comment and collection of a user through an Attention network;

s26: concat the context characteristic, the microblog portrait characteristic, the user portrait characteristic and the characteristic extracted in the step S25 and inputting the concat into a 2-layer full-connection grid model;

s27: inputting the context feature, the microblog portrait feature, the user portrait feature and the features extracted in the steps S21, S22, S23 and S25 into an FM model;

s28: and (5) concat the output results of the models of the steps S24, S26 and S27, and processing the output results by using a sigmod function after passing through a single neuron.

7. The deep interest network recommendation method based on multi-modal in accordance with claim 6, wherein the FM model function formula in step S27 is:

wherein the content of the first and second substances,

representing the operation result of the FM model;

x represents a feature vector;

w₀is a constant term coefficient representing the offset of the FM model;

n is the number of all the features,

w_ix_irepresenting the multiplication of the ith eigenvector by the eigenvalue;

v_iis a vector representation of the ith feature,

<v_i,v_j>representing the inner product of the ith feature vector and the jth feature vector to represent feature intersection;

x_i,x_jrepresenting second order features of two mutually different feature combinations.

Technical Field

The invention relates to the technical field of intelligent recommendation, in particular to a deep interest network recommendation method based on multiple modes.

Background

Compared with the traditional machine learning model, the Deep learning model has stronger expression capability, can mine more data hidden models, and currently, the mainstream Deep learning recommendation models comprise AutoRec, DeepCross, PNN, NeuralCF, FNN, NFM, Wide & Deep, DeepFM, AFM, DIN and DIEN. However, none of these deep learning recommendation models takes into account the influence of the features of various modal forms such as images, characters, sounds, and animation on the models, and the recommendation effect is not satisfactory. In fact, the forms of images, characters, sounds, and animations easily affect the user's choice.

Therefore, the prior art has drawbacks and needs further improvement.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a deep interest network recommendation method based on multiple modes. The method and the device improve the effect of the sequencing recommendation model by collecting the characteristics of images, characters, sounds, animations and other multi-modal forms.

In order to achieve the purpose, the invention adopts the following specific scheme:

the invention provides a deep interest network recommendation method based on multiple modes, which comprises the following steps:

s1, data acquisition;

s2, preprocessing data;

s3, training a network model;

s4, model online reasoning.

Further, in step S1, the data acquisition specifically includes embedding points for user behavior according to the front end, collecting user behavior data, storing the user behavior data in hive, and triggering data preprocessing.

Further, step S2, the data preprocessing specifically includes reading the hive original data set, performing data preprocessing, sample cleaning, duplicate removal, missing value processing, reconstructing the sample into a data set suitable for the feature structure of the network model, splitting the data set into a training set and a sample set, and triggering network model training.

Further, in step S3, the network model training specifically includes constructing a network structure of a new CTR ranking model using tensorflow, and obtaining the new CTR ranking model using training set training.

Further, in step S4, the model online inference specifically includes providing a prediction service for the trained ranking model through a docker container and a tensoflow serving mirror image, and the user remotely requests the iterative ranking model to perform the prediction service through HTTP or GRPC, and ranks the microblog resources recalled by the user.

Further, step S2 specifically includes the following steps:

s21: obtaining the image characteristics of the microblog through a Resnet34 network model;

s22: pre-training a text of a microblog to get embedding, and then performing microblog text sequence feature processing on the text through a GRU (general packet radio unit);

s23: obtaining the imbedding vector characteristic of each user by the user relation graph through the node2 vec;

s24: performing concat on the output results of the steps S21, S22 and S23, and then inputting the output results into a 2-layer full-connection layer network model;

s25: extracting user interest characteristics of history sequences related to reading, praise, comment and collection of a user through an Attention network;

s26: concat the context characteristic, the microblog portrait characteristic, the user portrait characteristic and the characteristic extracted in the step S25 and inputting the concat into a 2-layer full-connection grid model;

s27: inputting the context feature, the microblog portrait feature, the user portrait feature and the features extracted in the steps S21, S22, S23 and S25 into an FM model;

s28: and (5) concat the output results of the models of the steps S24, S26 and S27, and processing the output results by using a sigmod function after passing through a single neuron.

Further, in step S27, the FM model function formula is:

wherein the content of the first and second substances,

representing the operation result of the FM model;

x represents a feature vector;

w₀is a constant term coefficient representing the offset of the FM model;

n is the number of all the features,

w_ix_irepresenting the multiplication of the ith eigenvector by the eigenvalue;

v_iis a vector representation of the ith feature,

<v_i,v_j>representing the inner product of the ith feature vector and the jth feature vector to represent feature intersection;

x_i,x_jrepresenting second order features of two mutually different feature combinations.

By adopting the technical scheme of the invention, the invention has the following beneficial effects:

the invention provides a deep interest network recommendation method based on multiple modes, which comprises the following steps: data acquisition, data preprocessing, network model training and model online reasoning. The novel model is used for extracting information such as microblog portrait features, context features, user portrait features, user historical behavior sequences, microblog texts, microblog images, social relations and the like to construct an individualized deep learning microblog recommendation model, so that the requirement of individualized reading of a user is met, the problem that the deep learning model in the prior art cannot solve the influence of the features of various modal forms such as images, characters, sounds and animations on the model is solved, and the effect of the sequencing recommendation model is improved.

Drawings

FIG. 1 is a general flow diagram of an embodiment of the present invention;

FIG. 2 is a flowchart of a process for extracting microblog data according to an embodiment of the invention;

fig. 3 is a schematic diagram of processing microblog data according to an embodiment of the invention.

Detailed Description

The invention is further described below with reference to the following figures and specific examples.

The invention is explained in detail in connection with figures 1-3,

the invention provides a deep interest network recommendation method based on multiple modes, which comprises the following steps:

s1, data acquisition → S2, data preprocessing → S3, network model training → S4, and model online reasoning.

The specific contents are as follows:

s1, data acquisition: and embedding points for user behaviors according to the front end, collecting user behavior data, storing the user behavior data into hive, and triggering data preprocessing.

S2, preprocessing data: reading a hive original data set, and performing data preprocessing: cleaning a sample, removing the weight, processing missing values and the like, reconstructing the sample into a data set which is suitable for the characteristic structure of the network model, splitting the data set into a training set and a sample set, and triggering the training of the network model.

S3, training a network model: constructing a network structure of a new CTR sequencing model by using tensoflow, and training by using a training set to obtain the new CTR sequencing model;

s4, model online reasoning: the trained sequencing model provides prediction service through a docker container and a tensoflow serving mirror image, and a user remotely requests the iterative sequencing model prediction service through HTTP or GRPC to sequence microblog resources recalled by the user.

The novel model of the scheme has the characteristics that:

(1) multimodal exploitation: acquiring the embedding characteristic of the image by using a Resnet34 network model and extracting the serial number characteristic of the microblog text by using a GRU (general packet Unit), so that the new model can learn morphological characteristic information such as the image and the character;

(2) learning the deep interest of the user: extracting user interest characteristics through an Attention network according to the historical behaviors of the user, and then putting the user interest characteristics into a new model to learn the medium and long term interests of the user;

(3) deep neural network and FM integration: by combining the deep neural network and the FM, the memory capacity and the generalization capacity of the whole model can be improved, and the recommendation effect is better.

The model structure of the multi-modal deep interest network based on community microblog recommendation is detailed, as shown in fig. 2 and 3:

s21: obtaining the image characteristics of the microblog through a Resnet34 network model;

s22: pre-training a text of a microblog to get embedding, and then performing microblog text sequence feature processing on the text through a GRU (general packet radio unit);

s23: obtaining the imbedding vector characteristic of each user by the user relation graph through the node2 vec;

s24: performing concat on the output results of the steps S21, S22 and S23, and then inputting the output results into a 2-layer full-connection layer network model;

s25: extracting user interest characteristics of history sequences related to reading, praise, comment and collection of a user through an Attention network;

s26: concat the context characteristic, the microblog portrait characteristic, the user portrait characteristic and the characteristic extracted in the step S25 and inputting the concat into a 2-layer full-connection grid model;

s27: inputting the context feature, the microblog portrait feature, the user portrait feature and the features extracted in the steps S21, S22, S23 and S25 into an FM model;

s28: and (5) concat the output results of the models of the steps S24, S26 and S27, and processing the output results by using a sigmod function after passing through a single neuron.

The formula of the FM model function in step S27 is:

wherein the content of the first and second substances,

representing the operation result of the FM model;

x represents a feature vector;

w₀is a constant term coefficient representing the offset of the FM model;

n is the number of all the features,

w_ix_irepresenting the multiplication of the ith eigenvector by the eigenvalue;

v_iis the direction of the ith featureThe amount is expressed in terms of,

<v_i,v_j>representing the inner product of the ith feature vector and the jth feature vector to represent feature intersection;

x_i,x_jrepresenting second order features of two mutually different feature combinations.

According to the scheme, a new model of a user and microblog relation recommendation model is established under a deep learning framework, the problem that the user is inaccurate in acquiring microblog information is well solved, and all indexes are increased to a certain extent:

TABLE 1 comparison of AUC (area Under client) and superiority and inferiority during model training

As can be seen from table 1, after the new model comes online, the microblog CTR (click through rate) in the microblog recommendation service is improved by about 50%, and the approval, comment, forwarding and user retention rate of the microblog are also improved to some extent.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.