阅读说明：本技术 文本摘要与关键词抽取方法、装置、设备及介质 (Text abstract and keyword extraction method, device, equipment and medium ) 是由 张剑 程刚 刘代琴 黄仁杰 于 2021-11-02 设计创作，主要内容包括：本申请涉及一种文本摘要与关键词抽取方法,包括：获取包括若干文本分句的训练文本数据；将多个文本分句转换为句向量,并标记多个句向量的关键词标签及摘要句标签；对标记后的多个句向量进行上下文信息嵌入,得到标准向量集；利用标准向量集对预构建的摘要句分析模型以及关键词分析模型进行联合训练；利用训练后的摘要句分析模型和关键词分析模型抽取获取到的待处理文本的文本摘要与关键词。此外,本申请还涉及一种文本摘要与关键词抽取装置、设备及介质。本申请可解决利用单独训练的摘要模型和关键词抽取模型时无法同时精确地对摘要与关键词进行抽取的问题。(The application relates to a text abstract and keyword extraction method, which comprises the following steps: acquiring training text data comprising a plurality of text clauses; converting a plurality of text clauses into sentence vectors, and marking keyword labels and abstract sentence labels of the sentence vectors; embedding context information into the marked sentence vectors to obtain a standard vector set; performing joint training on the pre-constructed abstract sentence analysis model and the keyword analysis model by using a standard vector set; and extracting the text abstract and the keywords of the acquired text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model. In addition, the application also relates to a text abstract and keyword extraction device, equipment and medium. The method and the device can solve the problem that the abstract and the keywords cannot be extracted simultaneously and accurately when the abstract model and the keyword extraction model which are trained independently are utilized.)

1. A method for extracting text abstracts and key words is characterized by comprising the following steps:

acquiring training text data, wherein the training text data comprises a plurality of text clauses;

converting a plurality of text clauses into sentence vectors, and marking keyword labels and abstract sentence labels of the sentence vectors;

embedding context information into the marked sentence vectors through a preset coding model to obtain a standard vector set;

performing joint training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the standard vector set, wherein the joint training performs parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset joint loss function, and the joint loss function comprises a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model;

and extracting the text abstract and the keywords of the acquired text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model.

2. The method for extracting text abstract and keyword according to claim 1, wherein the performing a joint training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the set of standard vectors, the joint training performing parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset joint loss function, the joint loss function including a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model, comprises:

assigning initial values to the parameters of the abstract sentence analysis model and the keyword analysis model, and generating abstract prediction labels and keyword prediction labels of a plurality of standard vectors by using the abstract sentence analysis model and the keywords after the initial values are assigned;

calculating the first loss value according to the abstract prediction label and the marked actual abstract label, and calculating the second loss value according to the keyword prediction label and the marked actual keyword label;

calculating a joint loss value of the joint loss function according to the first loss value and the second loss value, and calculating an update gradient of the abstract sentence analysis model and the keyword analysis model according to the joint loss value by using a gradient descent algorithm;

and adjusting and optimizing parameters of the abstract sentence analysis model and the keyword analysis model according to the updating gradient to obtain the trained abstract sentence analysis model and the trained keyword analysis model.

3. The method of claim 2, wherein the joint loss function comprises:

wherein the content of the first and second substances,is the joint loss value of the joint loss function,in order to be said first loss value,in order to be said second loss value,is a balance factor.

4. The method for extracting text summaries and keywords according to any one of claims 1 to 3, wherein the embedding of context information on the marked sentence vectors through a preset coding model to obtain a standard vector set comprises:

encoding position information of text clauses corresponding to the sentence vectors in the training text data into sentence position vectors by using a preset encoding model, and combining the sentence position vectors with the sentence vectors to obtain sentence standard vectors; or

The position information of a plurality of text participles contained in text clauses corresponding to a plurality of sentence vectors in the training text data is encoded into word position vectors by using the encoding model, and the word position vectors are embedded into the word vectors of the text participles to be combined to obtain word standard vectors;

and collecting the sentence standard vectors or the word standard vectors into the standard vector set.

5. The method of any of claims 1 to 3, wherein the coding model is a BERT model and the keyword analysis model is a CRF model.

6. The method of any of claims 4, wherein converting the plurality of text clauses into sentence vectors comprises:

one of the text clauses is selected as a target clause from the text clauses in sequence;

splitting the target clause into a plurality of text participles, and coding each text participle into a word vector;

and splicing the word vectors into sentence vectors according to the sequence of the positions of the text participles in the target clause.

7. The method of claim 2, wherein the generating a plurality of abstract predictive labels and keyword predictive labels of the standard vectors by using the abstract sentence analysis model and the keywords with initial values comprises:

performing convolution and pooling processing on each standard vector for preset times by using the abstract sentence analysis model to obtain vector characteristics;

calculating sentence probability values of text clauses corresponding to the standard vectors as abstract sentences according to the vector characteristics;

marking the text clauses with the sentence probability value larger than or equal to a preset probability threshold value by using abstract prediction labels belonging to abstract clauses;

and marking the text clauses with the sentence probability value smaller than the preset probability threshold value by using abstract prediction labels which do not belong to abstract sentences.

8. An apparatus for abstracting text abstract and key words, the apparatus comprising:

the text clause module is used for acquiring training text data, and the training text data comprises a plurality of text clauses;

the vector conversion module is used for converting the text clauses into sentence vectors, marking keyword labels and abstract sentence labels of the sentence vectors, and embedding context information into the marked sentence vectors through a preset coding model to obtain a standard vector set;

the model optimization module is used for carrying out combined training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the standard vector set, wherein the combined training carries out parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset combined loss function, and the combined loss function comprises a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model;

and the text processing module is used for extracting the text abstract and the keywords of the acquired text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model.

9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for implementing the steps of the text summarization and keyword extraction method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the text summarization and keyword extraction method according to any one of claims 1 to 7.

Technical Field

The present application relates to the field of text processing, and in particular, to a method and an apparatus for extracting text summaries and keywords, an electronic device, and a computer readable storage medium.

Background

With the development of the network era, the information explosion which follows the development of the network era leads people to spend a great deal of time and energy on browsing and analyzing the contacted data, and further leads people to be tired and tired in looking up information, so that the key words and abstract extraction in the data become important points which are more and more concerned, and how to accurately extract the abstract and the key words from a great amount of data becomes a problem to be solved urgently.

Disclosure of Invention

The application provides a text abstract and keyword extraction method, a text abstract and keyword extraction device and a storage medium, which are used for solving the problem that an abstract and a keyword cannot be simultaneously and accurately extracted when an independently trained abstract model and a keyword extraction model are utilized.

In a first aspect, the present application provides a method for extracting text summaries and keywords, including:

acquiring training text data, wherein the training text data comprises a plurality of text clauses;

converting a plurality of text clauses into sentence vectors, and marking keyword labels and abstract sentence labels of the sentence vectors;

embedding context information into the marked sentence vectors through a preset coding model to obtain a standard vector set;

performing joint training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the standard vector set, wherein the joint training performs parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset joint loss function, and the joint loss function comprises a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model;

and extracting the text abstract and the keywords of the acquired text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model.

In detail, the performing combined training on the pre-constructed abstract sentence analysis model and the keyword analysis model by using the standard vector set, wherein the combined training performs parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset combined loss function, and the combined loss function includes a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model, and includes:

assigning initial values to the parameters of the abstract sentence analysis model and the keyword analysis model, and generating abstract prediction labels and keyword prediction labels of a plurality of standard vectors by using the abstract sentence analysis model and the keywords after the initial values are assigned;

calculating the first loss value according to the abstract prediction label and the marked actual abstract label, and calculating the second loss value according to the keyword prediction label and the marked actual keyword label;

calculating a joint loss value of the joint loss function according to the first loss value and the second loss value, and calculating an update gradient of the abstract sentence analysis model and the keyword analysis model according to the joint loss value by using a gradient descent algorithm;

and adjusting and optimizing parameters of the abstract sentence analysis model and the keyword analysis model according to the updating gradient to obtain the trained abstract sentence analysis model and the trained keyword analysis model.

In detail, the joint loss function includes:

wherein the content of the first and second substances,is the joint loss value of the joint loss function,in order to be said first loss value,in order to be said second loss value,is a balance factor.

In detail, the embedding context information into the marked sentence vectors through a preset coding model to obtain a standard vector set includes:

encoding position information of text clauses corresponding to the sentence vectors in the training text data into sentence position vectors by using a preset encoding model, and combining the sentence position vectors with the sentence vectors to obtain sentence standard vectors; or

The position information of a plurality of text participles contained in text clauses corresponding to a plurality of sentence vectors in the training text data is encoded into word position vectors by using the encoding model, and the word position vectors are embedded into the word vectors of the text participles to be combined to obtain word standard vectors;

and collecting the sentence standard vectors or the word standard vectors into the standard vector set.

In detail, the coding model is a BERT model, and the keyword analysis model is a CRF model.

In detail, the converting the plurality of text clauses into sentence vectors includes:

one of the text clauses is selected as a target clause from the text clauses in sequence;

splitting the target clause into a plurality of text participles, and coding each text participle into a word vector;

and splicing the word vectors into sentence vectors according to the sequence of the positions of the text participles in the target clause.

In detail, the generating a plurality of abstract prediction labels and keyword prediction labels of the standard vector by using the abstract sentence analysis model and the keyword after assigning initial values includes:

performing convolution and pooling processing on each standard vector for preset times by using the abstract sentence analysis model to obtain vector characteristics;

calculating sentence probability values of text clauses corresponding to the standard vectors as abstract sentences according to the vector characteristics;

marking the text clauses with the sentence probability value larger than or equal to a preset probability threshold value by using abstract prediction labels belonging to abstract clauses;

and marking the text clauses with the sentence probability value smaller than the preset probability threshold value by using abstract prediction labels which do not belong to abstract sentences.

In a second aspect, the present application provides a text summarization and keyword extraction apparatus, the apparatus comprising:

the text clause module is used for acquiring training text data, and the training text data comprises a plurality of text clauses;

the vector conversion module is used for converting the text clauses into sentence vectors, marking keyword labels and abstract sentence labels of the sentence vectors, and embedding context information into the marked sentence vectors through a preset coding model to obtain a standard vector set;

the model optimization module is used for carrying out combined training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the standard vector set, wherein the combined training carries out parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset combined loss function, and the combined loss function comprises a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model;

and the text processing module is used for extracting the text abstract and the keywords of the acquired text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model.

In a third aspect, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

the processor is configured to implement the steps of the text summarization and keyword extraction method according to any embodiment of the first aspect when executing the program stored in the memory.

In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, implements the steps of the text summarization and keyword extraction method according to any one of the embodiments of the first aspect.

Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:

the method provided by the embodiment of the application can embed the context information by carrying out word vector and sentence vector corresponding to the text data, so as to ensure that the vector contains rich context information, is beneficial to improving the accuracy of keyword and abstract analysis, further, the vector containing the position information and the joint loss function are used for simultaneously optimizing and training the abstract sentence analysis model and the keyword analysis model, the joint loss function including the first loss value of the abstract sentence analysis model and the second loss value of the keyword analysis model can avoid a situation that the accuracy of one model is improved while the accuracy of the other model is reduced, the accuracy of the two models can be synchronously improved, and then the text abstract and the key words of the text to be processed are extracted by utilizing the optimized models, the method can solve the problem that the abstract and the keywords can not be extracted simultaneously and accurately when the abstract model and the keyword extraction model which are trained independently are utilized.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic flowchart of a method for extracting text summaries and keywords according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a process of splitting training text data into a plurality of text clauses according to an embodiment of the present application;

fig. 3 is a schematic flowchart of a process of converting each text clause into a sentence vector according to an embodiment of the present application;

fig. 4 is a schematic block diagram of an apparatus for text summarization and keyword extraction according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of an electronic device for text summarization and keyword extraction according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a schematic flowchart of a method for extracting text summaries and keywords according to an embodiment of the present application. In this embodiment, the method for extracting text summaries and keywords includes:

and S1, acquiring training text data, wherein the training text data comprises a plurality of text clauses.

In the embodiment of the present application, the training text data is data that is obtained in advance and can be used for training a given text processing model.

In an embodiment of the present application, the acquiring training text data includes:

acquiring original text data;

extracting the symbol position of a target punctuation mark in the training text data by utilizing a regular expression compiled by punctuation marks;

and splitting the original text data into training text data containing a plurality of text clauses according to the symbol positions.

The regular expression generated by punctuation coding can be used for identifying the symbolic positions of punctuation in the text data, and further splitting the training text data according to the identified symbolic positions to obtain a plurality of text clauses.

In other embodiments of the present application, the clause processing may be performed on the original text data by a send _ token () method in an NLTK library in Python, so as to obtain a plurality of text clauses corresponding to the original text data.

Illustratively, the original text data may be split into several text clauses using the following send _ token () method:

from nltk.tokenize import sent_tokenize

data = "x"

print(sent_tokenize(data))

wherein x is the original text data.

In the embodiment of the application, the original text data is split into the text clauses, so that the overall efficiency of analyzing the training text data subsequently is reduced, and the analysis efficiency is improved.

And S2, converting the text clauses into sentence vectors, and marking the keyword labels and abstract sentence labels of the sentence vectors.

In the embodiment of the application, because the natural language processing model cannot directly analyze and process the text, a plurality of text clauses can be converted into sentence vectors, and the text clauses are further enabled to have model processability.

In detail, word segmentation processing can be performed on each text clause, words contained in the text clauses are coded into word vectors one by one, and then the word vectors are used for splicing the sentence vectors of each text clause.

In this embodiment, referring to fig. 2, the converting the text clauses into sentence vectors includes:

s21, selecting one of the text clauses as a target clause from the text clauses in sequence;

s22, splitting the target clause into a plurality of text participles, and coding each text participle into a word vector;

and S23, splicing the word vectors into sentence vectors according to the sequence of the positions of the text participles in the target clause.

For example, the target clause is: word and no play, splitting the target clause to obtain text clauses as follows: word, and, no, and play, wherein word vectors obtained by encoding the four text participles are sequentially: "token _ 1", "token _ 2", "token _ 3", and "token _ 4", so that the four word vectors can be spliced into the following sentence vectors according to the sequence of the four text participles of "word", "and", "no", and "play" in the target sentence:

sent_i={ token_1，token_2，token_3，token_4}

and sending _ i is the ith target clause selected from the text clauses.

In detail, the target clause can be divided into a plurality of text clauses by word segmentation tools such as jieba, SnowNLP, PKUse, THULAC and the like; alternatively, the target clause may be split into several text participles by word token () method in NLTK library in Python.

Illustratively, the target clause may be split into several text participles using the following word token () method:

from nltk.tokenize import word_tokenize

data = "y"

print(word_tokenize(data))

wherein y is the target clause.

Specifically, each of the text participles may be encoded into a word vector using a transform encoder, a word2vec encoder, an Autoencoder, or the like having a word encoding function.

In one practical application scenario of the present application, because the generated sentence vectors need to be used for analyzing the training text data in the following, each sentence vector needs to be labeled by using the keyword labels and abstract sentence labels in the training text data.

In detail, the keyword tag may be E _ Si, where E _ Si may be 0 or 1, when a text segment in the ith target sentence in the text clause is a keyword, the text segment may be marked with E _ Si =1, and when the text segment in the ith target sentence in the text clause is a keyword, the text segment may be marked with E _ Si = 0.

The abstract sentence label may be E _ ABi, where E _ ABi may be 0, 1, or 2, and when the ith target sentence in the text clause is an abstract sentence and the target sentence contains a keyword, the target sentence may be marked by E _ Abi = 2; when the ith target clause in the text clauses is an abstract clause and the target clauses do not contain keywords, marking the target clauses by utilizing E _ Abi = 1; when the ith target clause in the text clauses is not a summary clause, the target clause can be marked by using E _ Abi = 0.

And S3, embedding context information of the marked sentence vectors through a preset coding model to obtain a standard vector set.

In one practical application scenario of the method and the device, because the text has certain continuity and context semantic relevance, the position information of each marked sentence vector can be embedded through a preset coding model, so that the sentence vector contains abundant context information, and the accuracy of subsequent keyword and abstract analysis according to the sentence vector is improved.

In detail, the coding model may be a BERT model, where, for a single word in a sentence vector, the coding model may be used to code the position information of the word in the sentence vector and the position information of the word in the training text data, and information obtained after coding is used as the word context information; for a sentence vector, the position information of the sentence vector in the training text data and the position information of each word in the sentence vector can be encoded, and the information obtained after encoding is used as the context information of the sentence vector.

Specifically, the position information includes position information of a sentence vector corresponding to each text clause in the training text data, and position information of a text clause included in each text clause in the training text data.

In this embodiment of the present application, the embedding context information into the marked multiple sentence vectors through a preset coding model to obtain a standard vector set includes:

encoding position information of text clauses corresponding to the sentence vectors in the training text data into sentence position vectors by using a preset encoding model, and combining the sentence position vectors with the sentence vectors to obtain sentence standard vectors; or

The position information of a plurality of text participles contained in text clauses corresponding to a plurality of sentence vectors in the training text data is encoded into word position vectors by using the encoding model, and the word position vectors are embedded into the word vectors of the text participles to be combined to obtain word standard vectors;

and collecting the sentence standard vectors or the word standard vectors into the standard vector set.

Specifically, the sentence position vector may be embedded in each of the sentence vectors, and the word position vector may be embedded in each of the text participles, by means of vector concatenation as in step S2.

And when each sentence vector and the word vector in the sentence vectors are embedded into the position information, obtaining the standard vector.

In the embodiment of the application, context information embedding is carried out on each marked sentence vector through the preset coding model, so that the sentence vectors contain abundant context information, and further, the problem that each sentence vector is independently analyzed when the sentence vectors are analyzed subsequently is avoided, the relevance among the sentence vectors is reflected, and the analysis accuracy is improved.

And S4, performing joint training on the pre-constructed abstract sentence analysis model and the keyword analysis model by using the standard vector set, wherein the joint training performs parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset joint loss function, and the joint loss function comprises a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model.

In the embodiment of the present application, the abstract sentence analysis model includes, but is not limited to, a Convolutional Neural Network (CNN) model having an abstract sentence recognition function, and a Support Vector Machine (SVM) model; the keyword analysis model includes, but is not limited to, a Conditional Random Field (CRF) model with a keyword recognition function, and a Lstm (Long short-term memory) model.

In the embodiment of the application, a BERT model is used when context information is embedded, a CRF model is used when an abstract sentence model and a keyword model are built, the model is built by combining the BERT model and the CRF model, and an artificial characteristic template and a rule do not need to be built, so that the model can be built conveniently and quickly.

In this embodiment of the application, referring to fig. 3, the method of performing joint training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the standard vector set includes, performing parameter optimization on the abstract sentence analysis model and the keyword analysis model through a preset joint loss function, where the joint loss function includes a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model, and includes:

s31, assigning initial values to the parameters of the abstract sentence analysis model and the keyword analysis model, and generating a plurality of abstract predictive labels and keyword predictive labels of the standard vectors by using the initially assigned abstract sentence analysis model and the keywords;

s32, calculating the first loss value according to the abstract prediction label and the marked actual abstract label, and calculating the second loss value according to the keyword prediction label and the marked actual keyword label;

s33, calculating a joint loss value of the joint loss function according to the first loss value and the second loss value, and calculating the update gradient of the abstract sentence analysis model and the keyword analysis model according to the joint loss value by using a gradient descent algorithm;

and S34, adjusting and optimizing the parameters of the abstract sentence analysis model and the keyword analysis model according to the updating gradient to obtain the trained abstract sentence analysis model and keyword analysis model.

Wherein the labeled actual digest label and the actual keyword label are the labels labeled in S2.

In this embodiment of the application, the generating a plurality of abstract prediction labels and keyword prediction labels of the standard vector by using the abstract sentence analysis model and the keyword after assigning the initial values includes:

performing convolution and pooling processing on each standard vector for preset times by using the abstract sentence analysis model to obtain vector characteristics;

calculating sentence probability values of text clauses corresponding to the standard vectors as abstract sentences according to the vector characteristics;

marking the text clauses with the sentence probability value larger than or equal to a preset probability threshold value by using abstract prediction labels belonging to abstract clauses;

and marking the text clauses with the sentence probability value smaller than the preset probability threshold value by using abstract prediction labels which do not belong to abstract sentences.

In detail, the preset activation function may be the same as an activation function of a softmax layer in the abstract sentence analysis model.

Specifically, after the word probability value that each word vector included in each standard vector is a keyword is calculated, the calculated secondary probabilities may be arranged as a probability matrix according to the sequence in which each word vector is arranged in the standard vector, and then the probability matrix is input into the keyword analysis model (such as a CRF model), and the keyword prediction label of the text segmentation corresponding to each word vector is predicted by using the keyword analysis model according to the probability matrix.

Specifically, a softmax layer in the abstract sentence analysis model can be used for calculating the probability value of the text clause corresponding to each standard vector as the abstract sentence according to the vector characteristics.

Preferably, the probability threshold is 0.5.

In detail, the parameter assigning initial values to the abstract sentence analysis model and the keyword analysis model may be performed by generating different random numbers using a preset random number generator, and assigning initial values to the parameters of the abstract sentence analysis model and the keyword analysis model using the generated random numbers; alternatively, all the parameters of the abstract sentence analysis model and the keyword analysis model may be set to zero, so as to assign initial values to the parameters of the abstract sentence analysis model and the keyword analysis model.

Experiments show that certain internal connection exists between abstract extraction and keyword analysis, the training result of one model can promote the accuracy of the result of the other model, in the embodiment, the two models are trained simultaneously by using the loss values of the two tasks of the preset joint loss function, the abstract sentence analysis model and the keyword analysis model can be jointly optimized simultaneously, and the accuracy of the abstract sentence analysis model and the accuracy of the keyword analysis model are improved simultaneously.

In detail, the first loss value is a loss value between the digest prediction tag output by the digest prediction model and the actual digest tag of the label, and the second loss value is a loss value between the keyword prediction tag output by the keyword analysis model and the actual keyword tag of the label.

Specifically, the gradient descent algorithm includes, but is not limited to, a batch gradient descent algorithm, a random gradient descent algorithm, a small batch gradient descent algorithm; when the joint loss value is obtained through calculation, the joint loss value can be calculated through a preset gradient descent algorithm to obtain an updating gradient of the abstract sentence analysis model and the keyword analysis model, the updating gradient is a vector value and has a direction, and therefore parameters of the abstract sentence analysis model and the keyword analysis model can be adjusted and optimized according to the direction of the updating gradient until the joint loss value is smaller than a preset loss threshold value.

Specifically, the gradient descent calculation can be used for calculating the updating gradients of the parameters in the abstract sentence analysis model and the keyword analysis model in turn, and then the abstract sentence analysis model and the keyword analysis model are updated in turn according to the updated gradients until the joint loss value calculated by the output results of the abstract sentence analysis model and the keyword analysis model is smaller than the preset loss threshold value.

In an embodiment of the present application, the joint loss function includes:

wherein the content of the first and second substances,is the joint loss value of the joint loss function,in order to be said first loss value,in order to be said second loss value,is a balance factor.

In detail, the balance factorThe preset constant may be set, or the adjusted value may be calculated according to the verification result after the abstract sentence analysis model and the keyword analysis model are verified by the preset verification set after each round of training, for example, after the abstract sentence analysis model and the keyword analysis model are trained for one round, the preset verification is performed by using the trained abstract sentence analysis model and the trained keyword analysis modelAnalyzing the set, and analyzing the balance factor according to the accuracy of the abstract sentence analysis model and the keyword analysis model to the result obtained by analyzing the verification setUpdating (e.g. balance factor)A preset proportional value of the accuracy).

In the embodiment of the application, the abstract prediction label and the keyword prediction label are calculated by using the joint loss function to calculate the joint loss value, and the parameters of the two models are subjected to back propagation optimization through a gradient descent algorithm based on the joint loss value, so that the synchronous optimization of the abstract sentence analysis model and the keyword analysis model is realized, and the efficiency of model optimization is improved.

And S5, extracting the text abstract and the keywords of the acquired text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model.

In the embodiment of the application, the text to be processed can be any pre-acquired text, and abstract sentences contained in the text to be processed can be extracted by using an abstract sentence analysis model (namely a trained abstract sentence analysis model) after parameter optimization; and extracting the keywords in the text to be processed by using the keyword analysis model after parameter optimization (namely extraction of the trained keyword analysis model).

The method provided by the embodiment of the application can embed the context information by carrying out word vector and sentence vector corresponding to the text data, so as to ensure that the vector contains rich context information, is beneficial to improving the accuracy of keyword and abstract analysis, further, the vector containing the position information and the joint loss function are used for simultaneously optimizing and training the abstract sentence analysis model and the keyword analysis model, the joint loss function including the first loss value of the abstract sentence analysis model and the second loss value of the keyword analysis model can avoid a situation that the accuracy of one model is improved while the accuracy of the other model is reduced, the accuracy of the two models can be synchronously improved, and then the text abstract and the key words of the text to be processed are extracted by utilizing the optimized models, the method can solve the problem that the abstract and the keywords can not be extracted simultaneously and accurately when the abstract model and the keyword extraction model which are trained independently are utilized.

As shown in fig. 4, an embodiment of the present invention provides a module schematic diagram of a text abstract and keyword extraction apparatus 10, where the text abstract and keyword extraction apparatus 10 includes: a text clause module 11, a vector conversion module 12, a model optimization module 13 and a text processing module 14.

The text clause module 11 is configured to obtain training text data, where the training text data includes a plurality of text clauses;

the vector conversion module 12 is configured to convert the text clauses into sentence vectors, mark keyword labels and abstract sentence labels of the sentence vectors, and embed context information of the marked sentence vectors through a preset coding model to obtain a standard vector set;

the model optimization module 13 is configured to perform joint training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the standard vector set, where the joint training performs parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset joint loss function, and the joint loss function includes a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model;

the text processing module 14 is configured to extract the text abstract and the keywords of the obtained text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model.

In detail, in the embodiment of the present application, each module in the text abstract and keyword extraction apparatus 10 adopts the same technical means as the text abstract and keyword extraction method described in fig. 1 to 3, and can produce the same technical effect, and details are not repeated here.

As shown in fig. 5, an electronic device provided in the embodiment of the present application includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, where the processor 111, the communication interface 112, and the memory 113 complete communication with each other through the communication bus 114;

a memory 113 for storing a computer program;

in an embodiment of the present application, when the processor 111 is configured to execute the program stored in the memory 113, the method for extracting the text abstract and the keyword provided in any one of the foregoing method embodiments is implemented, including:

acquiring training text data, wherein the training text data comprises a plurality of text clauses;

converting a plurality of text clauses into sentence vectors, and marking keyword labels and abstract sentence labels of the sentence vectors;

embedding context information into the marked sentence vectors through a preset coding model to obtain a standard vector set;

performing joint training on a pre-constructed abstract sentence analysis model and a keyword analysis model by using the standard vector set, wherein the joint training performs parameter optimization on the abstract sentence analysis model and the keyword analysis model simultaneously through a preset joint loss function, and the joint loss function comprises a first loss value of the abstract sentence analysis model and a second loss value of the keyword analysis model;

and extracting the text abstract and the keywords of the acquired text to be processed by using the trained abstract sentence analysis model and the trained keyword analysis model.

Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the text summarization and keyword extraction method provided in any of the foregoing method embodiments.

It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.