阅读说明：本技术 一种话题分析方法、装置和存储介质 (Topic analysis method and device and storage medium ) 是由 耿雪芹 王晓斌 焦梦姝 黄三伟 于 2020-01-20 设计创作，主要内容包括：本发明公开了一种话题分析方法,包括：获取待处理文本语料,并获取每一个待处理文本语料所对应的分词结果和对应的词性；获取过滤后文本语料；通过依存句法对过滤后的每一文本语料的分词结果和对应的词性进行分析,获得分词的语法成分和分词之间的依存关系以及获得每一个文本语料对应的依存对；根据组合句式结构和依存对,获得每一个文本语料对应的话题；获取相似话题,并根据相似话题的数量进行排序。本发明还同时公开了话题分析装置和存储介质,通过分词的基础上使用句法分析,来分析文本语句中的语法结构和分词结果之间的依存关系,再按照预设的多种汉语常见组合句式结构,提取出通顺、准确的话题,能够从海量文本中分析话题。(The invention discloses a topic analysis method, which comprises the following steps: acquiring text corpora to be processed, and acquiring word segmentation results and corresponding parts of speech corresponding to each text corpora to be processed; obtaining filtered text corpora; analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain the grammar components of the word segmentation and the dependency relationship among the words and obtain the dependency pair corresponding to each text corpus; obtaining topics corresponding to each text corpus according to the combined sentence pattern structure and the dependency pairs; and acquiring similar topics and sequencing according to the number of the similar topics. The invention also discloses a topic analysis device and a storage medium, which analyze the dependency relationship between the syntactic structure and the word segmentation result in the text sentence by using syntactic analysis on the basis of word segmentation, and extract smooth and accurate topics according to the preset structure of various common Chinese combined sentences, thereby being capable of analyzing topics from massive texts.)

1. A topic analysis method, characterized in that the method comprises:

acquiring text corpora to be processed, and acquiring word segmentation results and corresponding parts of speech corresponding to each text corpora to be processed;

filtering the text corpus to be processed according to the word segmentation result, and acquiring the filtered text corpus;

analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain the grammar components of the word segmentation and the dependency relationship among the words and obtain the dependency pair corresponding to each text corpus;

obtaining topics corresponding to each text corpus according to the combined sentence pattern structure and the dependency pairs;

acquiring similar topics, and sequencing according to the number of the similar topics;

the obtaining of similar topics and ranking according to the number of similar topics include:

calculating similarity values with the acquired other topics aiming at each topic;

carrying out similarity combination according to the similarity value;

merging according to document id distribution of topics, and merging the two topics into one topic if the same id number in the document id lists of the two topics exceeds a preset number;

and sequencing the combined topics, and selecting the topics with the target number according to the frequency and outputting the topics.

2. The topic analysis method of claim 1, wherein the step of obtaining the corpus of texts to be processed and obtaining the segmentation result corresponding to each corpus of texts to be processed comprises:

performing sentence division processing on the text corpus according to punctuation marks;

and performing word segmentation processing on each text corpus to be processed to obtain word segmentation results.

3. The topic analysis method according to claim 2, wherein the step of filtering the text corpus to be processed according to the word segmentation result and obtaining the filtered text corpus comprises:

calculating the document frequency of each word segmentation in the word segmentation result, and performing descending order arrangement;

acquiring the keywords of the topics arranged in the front;

acquiring a text corpus to be filtered, wherein the text corpus to be filtered is a text corpus which does not contain any participle in topic keywords;

and removing the text corpus to be filtered from the text corpus to be processed to obtain the filtered text corpus.

4. The topic analysis method according to claim 2 or 3, wherein the step of performing sentence division processing on the text corpus according to punctuation marks comprises:

randomly distributing numbers to all documents in the text corpus, deleting preset punctuations in all documents in the text corpus to obtain target text sentences, and marking the document numbers of the text sentences;

adopting punctuation marks to segment the target text sentences, counting the frequency of the segmented sentences, and marking the document number where the segmented sentences are located as the document id of the sentences, wherein the punctuation marks at least comprise: comma, semicolon, period, question mark, exclamation mark, ellipsis;

and taking the segmented text corpus marking the frequency number and the document id as a text corpus to be processed.

5. The topic analysis method according to claim 2 or 3, wherein the step of performing the segmentation processing on each text corpus to be processed to obtain a segmentation result comprises:

performing word segmentation processing on each text corpus to be processed;

removing stop words, special symbols, letters and emoticons in the word segmentation processing result;

and obtaining word segmentation results.

6. The topic analysis method according to claim 1, wherein the step of analyzing the segmentation result and the corresponding part of speech of each filtered text corpus by dependency syntax to obtain the grammatical component of the segmentation and the dependency relationship between the segmentation and obtain the dependency pair corresponding to each text corpus comprises:

analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain a dependency relationship between grammar components and the word segmentation, wherein the grammar components comprise a subject, a predicate, an object, a fixed language, a shape and a complement, and the dependency relationship is a phrase relationship;

determining the participles forming the dependency relationship as a dependency pair;

wherein the dependency syntax is: graph-based analysis methods, transfer-based analysis methods, or deep learning-based analysis methods.

7. The topic analysis method of claim 6, wherein the step of obtaining the topic corresponding to each text corpus according to the combined sentence structure and the dependency pair comprises:

obtaining a core word of each text statement according to dependency syntax analysis;

determining participles which are in a dominance relation and a move-guest relation with the core word dependency relation;

combining the determined participles according to a preset sequence to obtain a topic main stem, wherein the preset sequence is as follows: a combination of a master-predicate-guest sequence and other relationships, wherein the other relationships are: the combination of the main and auxiliary relation words, the core words and the moving and guest relation words;

and filling the determined topic main stems to obtain the topics.

8. The topic analysis method of claim 7, wherein the step of populating the determined topic stems to obtain topics comprises:

filling by finding words with a relationship in a fixed relationship or a state with subject, predicate and object dependence;

and keeping the filling result with the length within the preset length interval as a topic.

9. A topic analysis apparatus, characterized in that the apparatus comprises a processor, and a memory connected with the processor through a communication bus; wherein the content of the first and second substances,

the memory is used for storing a topic analysis program;

the processor for executing the topic analysis program to implement the topic analysis step of any one of claims 1 to 8.

10. A storage medium, in particular a computer readable storage medium, storing one or more programs, which are executable by one or more processors to cause the one or more processors to perform the topic analysis step of any one of claims 1 to 8.

Technical Field

The present invention relates to the field of topic analysis and processing, and in particular, to a topic analysis method, device and storage medium.

Background

With the rapid development of information technology, the internet has become a main channel for people to acquire and distribute information. Because of the large amount of network information, wide sources and high transmission speed, it becomes more and more troublesome for the general netizens how to quickly and accurately find the desired network information. Therefore, how to analyze and extract the hot topics concerned by netizens from massive network information quickly, accurately and comprehensively becomes a current very hot research direction.

At present, the network topics still take texts as main expression modes, and the technical means for finding topics from the texts at present is still limited to lexical levels, namely, information related to the topics is searched by means of identification of keywords, hot words, co-occurring words, sensitive words, emotional tendency words, entity words and the like, and on the other hand, most of the existing topic analysis algorithms are based on clustering algorithms and gather texts of the same topic into one class. However, only the analysis is performed at the word level, and the obtained information is often local, and complete semantic information cannot be obtained. In addition, sometimes an article has not only one topic but also related sub-topics. That is, topics and articles are not in one-to-one relationship, and the clustering algorithm considers that one text has only one topic, so that the core content of the whole text cannot be completely summarized.

Disclosure of Invention

In view of the above, the present invention is directed to a topic analysis method, device and storage medium, which are used for analyzing the dependency relationship between the syntactic structure and the word segmentation result in a text sentence by using syntactic analysis on the basis of word segmentation, and then extracting a smooth and accurate topic according to a preset structure of a plurality of common Chinese combination sentences, so as to analyze topics from a mass of texts.

In order to achieve the purpose, the technical scheme of the invention is realized as follows: the invention provides a topic analysis method, which comprises the following steps:

acquiring text corpora to be processed, and acquiring word segmentation results and corresponding parts of speech corresponding to each text corpora to be processed;

filtering the text corpus to be processed according to the word segmentation result, and acquiring the filtered text corpus;

analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain the grammar components of the word segmentation and the dependency relationship among the words and obtain the dependency pair corresponding to each text corpus;

obtaining topics corresponding to each text corpus according to the combined sentence pattern structure and the dependency pairs;

acquiring similar topics, and sequencing according to the number of the similar topics;

the obtaining of similar topics and ranking according to the number of similar topics include:

calculating similarity values with the acquired other topics aiming at each topic;

carrying out similarity combination according to the similarity value;

merging according to document id distribution of topics, and merging the two topics into one topic if the same id number in the document id lists of the two topics exceeds a preset number;

and sequencing the combined topics, and selecting the topics with the target number according to the frequency and outputting the topics.

In the foregoing solution, the step of obtaining the linguistic data of the text to be processed and obtaining the word segmentation result corresponding to each linguistic data of the text to be processed includes:

performing sentence division processing on the text corpus according to punctuation marks;

and performing word segmentation processing on each text corpus to be processed to obtain word segmentation results.

In the foregoing solution, the step of filtering the corpus of the text to be processed according to the word segmentation result, and obtaining the filtered corpus includes:

calculating the document frequency of each word segmentation in the word segmentation result, and performing descending order arrangement;

acquiring the keywords of the topics arranged in the front;

acquiring a text corpus to be filtered, wherein the text corpus to be filtered is a text corpus which does not contain any participle in topic keywords;

and removing the text corpus to be filtered from the text corpus to be processed to obtain the filtered text corpus.

In the foregoing solution, the step of performing sentence division processing on the text corpus according to the punctuation marks includes:

randomly distributing numbers to all documents in the text corpus, deleting preset punctuations in all documents in the text corpus to obtain target text sentences, and marking the document numbers of the text sentences;

and dividing the target text sentence by adopting punctuation marks, counting the frequency of the divided sentence, and marking the document number of the divided sentence as the document id of the sentence. Wherein the punctuation mark comprises at least: comma, semicolon, period, question mark, exclamation mark, ellipsis;

and taking the segmented text corpus marking the frequency number and the document id as a text corpus to be processed.

In the foregoing solution, the step of performing word segmentation processing on each text corpus to be processed to obtain word segmentation results includes:

performing word segmentation processing on each text corpus to be processed;

removing stop words, special symbols, letters and emoticons in the word segmentation processing result;

and obtaining word segmentation results.

In the foregoing solution, the step of analyzing the segmentation result and the corresponding part-of-speech of each filtered text corpus by using dependency syntax to obtain dependency relationships between the grammar components of the segmentation and obtain the dependency pairs corresponding to each text corpus includes:

analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain a dependency relationship between grammar components and the word segmentation, wherein the grammar components comprise a subject, a predicate, an object, a fixed language, a shape and a complement, and the dependency relationship is a phrase relationship;

determining the participles forming the dependency relationship as a dependency pair;

wherein the dependency syntax is: graph-based analysis methods, transfer-based analysis methods, or deep learning-based analysis methods.

In the foregoing solution, the step of obtaining the topic corresponding to each text corpus according to the combined sentence structure and the dependency pair includes:

obtaining a core word of each text statement according to dependency syntax analysis;

determining participles which are in a dominance relation and a move-guest relation with the core word dependency relation;

combining the determined participles according to a preset sequence to obtain a topic main stem, wherein the preset sequence is as follows: a combination of a master-predicate-guest sequence and other relationships, wherein the other relationships are: the combination of the main and auxiliary relation words, the core words and the moving and guest relation words;

and filling the determined topic main stems to obtain the topics.

In the above scheme, the step of filling the determined topic stems to obtain topics includes:

filling by finding words with a relationship in a fixed relationship or a state with subject, predicate and object dependence;

keeping the filling result with the length within the preset length interval as a topic;

the step of obtaining similar topics and sorting according to the number of the similar topics comprises the following steps:

calculating similarity values with the acquired other topics aiming at each topic;

carrying out similarity combination according to the similarity value;

and re-sequencing the combined topics, and selecting the topics with the target number according to the frequency and outputting the topics.

In order to achieve the above object, the present invention also provides a topic analysis device, which includes a processor, and a memory connected to the processor through a communication bus; wherein the content of the first and second substances,

the memory is used for storing a topic analysis program;

the processor for executing the topic analysis program,

the one or more processors perform the topic analysis steps of any of the above aspects.

To achieve the above object, the present invention also provides a computer readable storage medium, specifically a computer readable storage medium, storing one or more programs, which are executable by one or more processors to cause the one or more processors to perform the topic analysis step of any one of the above aspects.

According to the topic analysis method, the topic analysis device and the storage medium, the dependency relationship between the syntactic structure and the word segmentation result in the text sentence is analyzed by using syntactic analysis on the basis of word segmentation, and then the smooth and accurate topic is extracted according to the preset structure of various common Chinese combined sentences, so that the topic can be analyzed from massive texts.

Drawings

FIG. 1 is a schematic flow chart of a topic analysis method in an alternative embodiment of the present invention;

FIG. 2 is a schematic diagram of dependencies in an alternative embodiment of the invention;

FIG. 3 is a diagram illustrating a visualization of the result of dependency parsing in an alternative embodiment of the invention;

FIG. 4 is an exemplary diagram of a combined sentence structure in an alternative embodiment of the present invention;

fig. 5 is a schematic structural diagram of a topic analysis device in an alternative embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.

Fig. 1 is a schematic flow diagram of a topic analysis method in an embodiment of the present invention, please refer to fig. 1, the embodiment of the present invention provides a topic analysis method, and the method includes:

s101: and acquiring the linguistic data of the texts to be processed, and acquiring the word segmentation result and the corresponding part of speech corresponding to each linguistic data of the texts to be processed.

It should be noted that the text corpus is a corpus set for data capture, and may include a pathological sentence or a sentence including a special symbol. Therefore, it is necessary to process text containing special symbols.

In one implementation mode of the invention, sentence division processing is carried out on the text corpus according to the punctuation marks, so that specific punctuation marks contained in the sentences are removed.

Specifically, the step of performing sentence division processing on the text corpus according to the punctuation marks includes: randomly distributing numbers to all documents in the text corpus, deleting preset punctuations in all documents in the text corpus to obtain target text sentences, and marking the document numbers of the text sentences; and dividing the target text sentence by adopting punctuation marks, counting the frequency of the divided sentence, and marking the document number of the divided sentence as the document id of the sentence. Wherein the punctuation mark comprises at least: comma, semicolon, period, question mark, exclamation mark, ellipsis; and taking the segmented text corpus marking the frequency number and the document id as a text corpus to be processed.

In one implementation, the sentence processing procedure includes: the following punctuation marks are first deleted: 【】 { }, "" "," [ ], and "; the text is then segmented by punctuation as follows: comma, semicolon, period, question mark, exclamation mark, ellipsis.

It can be understood that the text can be processed into individual text sentences by performing text segmentation through punctuations, and the segmented text sentences are used as text corpora to be processed.

And then performing word segmentation processing on each text corpus to be processed to obtain word segmentation results. The specific implementation comprises the following steps: performing word segmentation processing on each text corpus to be processed; removing stop words, special symbols, letters and emoticons in the word segmentation processing result; and obtaining word segmentation results.

It should be noted that word segmentation is an important step of preprocessing of text mining, and can be implemented through language models such as LSTM, which is the prior art, and after word segmentation is completed, some other feature engineering can be continuously performed, such as vectorization (vectorize), TF-IDF, and Hash trigk.

S102: and filtering the text corpus to be processed according to the word segmentation result, and acquiring the filtered text corpus.

It is understood that the text corpus to be processed may be obtained by text mining, where not all sentences have associations, for example, there may be completely different content between sentences, and then the sentences need to be filtered to filter out those completely irrelevant sentences.

In the embodiment of the present invention, the step of filtering the corpus of the text to be processed according to the word segmentation result and acquiring the filtered corpus includes: calculating the document frequency of each word segmentation in the word segmentation result, and performing descending order arrangement; acquiring the keywords of the topics arranged in the front; acquiring a text corpus to be filtered, wherein the text corpus to be filtered is a text corpus which does not contain any participle in topic keywords; and removing the text corpus to be filtered from the text corpus to be processed to obtain the filtered text corpus.

It can be understood that after the word segmentation processing is finished, the document frequency of the words is calculated, the words are sorted according to the document frequency of the words, and the words with the highest sorting order are output as topic keywords according to the descending order. And removing the text corpora which do not contain the topic keywords, and filtering the text corpora which do not contain the topic keywords, so that the text corpora to be processed are all the text corpora related to the topic keywords.

S103: and analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain the grammar components of the word segmentation and the dependency relationship among the words and obtain the dependency pair corresponding to each text corpus.

In one implementation mode of the invention, the word segmentation result and the corresponding part of speech of each filtered text corpus are analyzed through dependency syntax to obtain the dependency relationship between grammar components and the word segmentation, wherein the grammar components comprise a subject, a predicate, an object, a fixed language, a subject and a complement, and the dependency relationship is a phrase relationship; determining the participles forming the dependency relationship as a dependency pair; wherein the dependency syntax is: graph-based analysis methods, transfer-based analysis methods, or deep learning-based analysis methods.

And inputting the output words and parts of speech of each sentence into a dependency syntax analysis algorithm to obtain the grammar components of the words and the dependency relationship among the words. In the invention, grammatical components of the adopted words comprise subjects, predicates, objects, determinants, subjects, complements and the like, the adopted dependence relationship is a common phrase relationship in a Chinese phrase structure, as shown in FIG. 2, the dependence relationship is a subject-predicate relationship, for example, I likes; animate relationships, such as written compositions; centering relationships, such as safflower; relationships in the shape, such as very likes; dynamic complement relationships, e.g., written; juxtaposition, such as blue sky and white cloud; pre-object, e.g., job written; mediate relationships, such as a task; left and right additional relationships, such as the word "comprising"; the core relationships, such as the core words of the sentence, are generally verbs, and other relationships, and are not specifically limited herein.

The result of the dependency syntax analysis is shown in fig. 3. Dependency parsing is an algorithm that analyzes an input text sentence to obtain the syntactic structure of the sentence or the dependency between words in the sentence. In the dependency syntax, dependency relationships occur between words to form a dependency pair, where the dependency pair includes two words (one of which is a core word and the other is a modifier) and a dependency arc (a directed arc is used to represent the dependency relationship between the two words, and the direction of the dependency arc is from the modifier to the core word).

Therefore, a dependency pair corresponding to each text corpus can be obtained, and each text corpus can correspond to a plurality of dependency pairs.

S104: and obtaining topics corresponding to each text corpus according to the combined sentence pattern structure and the dependency pairs.

After obtaining the dependency pair corresponding to each text corpus, obtaining the core word of each text statement according to dependency syntactic analysis; determining participles which are in a dominance relation and a move-guest relation with the core word dependency relation; combining the determined participles according to a preset sequence to obtain a topic main stem, wherein the preset sequence is as follows: a combination of a master-predicate-guest sequence and other relationships, wherein the other relationships are: the combination of the main and auxiliary relation words, the core words and the moving and guest relation words; and filling the determined topic main stems to obtain the topics.

Filling can also be carried out by finding words with a relationship in a fixed relationship or a state with subject, predicate and object dependence; and keeping the filling result with the length within the preset length interval as a topic.

Through the dependency relationship between two words in each sentence and the combination of related words according to various combination sentence pattern structures as topics, the embodiment of the invention provides a principal and predicate object combination sentence pattern, and the others are similar in extraction mode.

Specifically, the syntactic combination step is to find the core word of the sentence according to the result of the dependency syntactic analysis, for example, the core word is "make" in fig. 4; then, words with the dependency relationship between the core words and the core words being the 'main and subordinate relationship' and the 'moving object relationship' are searched, and the words are respectively 'police' and 'report' in the graph 4; then, the words are combined together according to the sequence of the main predicate object and the predicate relation words, the core words and the mobile predicate object relation words, and the words are combined as follows in the figure 4: police party [ subject ] makes [ predicate ] to notify [ object ], so, topic main stem is obtained, because topic main stem component is not complete, filling is needed, and topic branch is obtained.

The branch filling step is as follows:

1) modification-defined filling: find and relate to subject, predicate, object dependency for "relation in center" or "relation in shape" word. In fig. 4, only the predicate "make" has two dependency relationships, and the corresponding term is "pair";

2) filling other components: and if the words are not filled in the previous step, ending. If yes, continuing to fill the words filled in the last step, and searching for words with dependence relationship with the filling words being other relationships in the graph 2. In fig. 4, a filler "pair" exists, and a dependency "betweent" exists, and the corresponding word is "event".

3) Repeat 2) until there are no filler words. FIG. 4 continues to fill in the word "event" resulting in the "centering relationship" word "violence".

4) And combining the filled phrases to form a short sentence which is the topic. In fig. 4: "police notify of violence events".

5) And for the topics, reserving the topics with the length within a preset length interval.

S105: and acquiring similar topics and sequencing according to the number of the similar topics.

In the case of many topics, it is necessary to perform classification or merging processing by determining which topics are similar. For many similar topics, a topic is randomly selected, and the frequency is the accumulation of the frequency of all similar topics.

The step of obtaining similar topics and sorting according to the number of the similar topics comprises the following steps: calculating similarity values with the acquired other topics aiming at each topic; carrying out similarity combination according to the similarity value; merging according to the document id distribution of topics: if the number of the same ids in the document id lists of the two topics exceeds the preset number, combining the two topics into one topic; and re-sequencing the combined topics, and selecting the topics with the target number according to the frequency and outputting the topics.

After the processing of the steps S101-S104, all text corpora output corresponding topics, the frequency count of sentences is recorded as the frequency count of topics generated by the sentences, the document id of sentences is recorded as the document id of topics generated by the sentences, and similarity combination is performed on all topics, wherein the similarity combination method comprises clustering, editing distance, word vectors and the like. And re-sequencing the combined topics, and selecting a certain number of topics according to the frequency number to be output (for example, performing descending order arrangement according to the frequency number of the topics, and selecting a certain number of topics arranged in front) as final topics.

In order to achieve the above object, the present invention further provides a topic analysis device, please refer to fig. 5, which includes a processor 501, and a memory 503 connected to the processor 501 through a communication bus 502; the memory 503 is used for storing a topic analysis program; the processor 501 is configured to obtain text corpora to be processed, and obtain a word segmentation result and a corresponding part of speech corresponding to each text corpus to be processed; filtering the text corpus to be processed according to the word segmentation result, and acquiring the filtered text corpus; analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain the grammar components of the word segmentation and the dependency relationship among the words and obtain the dependency pair corresponding to each text corpus; obtaining topics corresponding to each text corpus according to the combined sentence pattern structure and the dependency pairs; and acquiring similar topics and sequencing according to the number of the similar topics.

Here, the processor 501 is configured to execute the topic analysis program to implement the following topic analysis steps: performing sentence division processing on the text corpus according to punctuation marks; and performing word segmentation processing on each text corpus to be processed to obtain word segmentation results.

Here, the processor 501 is configured to execute the topic analysis program to implement the following topic analysis steps: calculating the document frequency of each word segmentation in the word segmentation result, and performing descending order arrangement; acquiring the keywords of the topics arranged in the front; acquiring a text corpus to be filtered, wherein the text corpus to be filtered is a text corpus which does not contain any participle in topic keywords; and removing the text corpus to be filtered from the text corpus to be processed to obtain the filtered text corpus.

Here, the processor 501 is configured to execute the topic analysis program to implement the following topic analysis steps: randomly distributing numbers to all documents in the text corpus, deleting preset punctuations in all documents in the text corpus to obtain target text sentences, and marking the document numbers of the text sentences; and dividing the target text sentence by adopting punctuation marks, counting the frequency of the divided sentence, and marking the document number of the divided sentence as the document id of the sentence. Wherein the punctuation mark comprises at least: comma, semicolon, period, question mark, exclamation mark, ellipsis; and taking the segmented text corpus marking the frequency number and the document id as a text corpus to be processed.

Here, the processor 501 is configured to execute the topic analysis program to implement the following topic analysis steps: performing word segmentation processing on each text corpus to be processed; removing stop words, special symbols, letters and emoticons in the word segmentation processing result; and obtaining word segmentation results.

Here, the processor 501 is configured to execute the topic analysis program to implement the following topic analysis steps: analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain a dependency relationship between grammar components and the word segmentation, wherein the grammar components comprise a subject, a predicate, an object, a fixed language, a shape and a complement, and the dependency relationship is a phrase relationship; determining the participles forming the dependency relationship as a dependency pair; wherein the dependency syntax is: graph-based analysis methods, transfer-based analysis methods, or deep learning-based analysis methods.

Here, the processor 501 is configured to execute the topic analysis program to implement the following topic analysis steps: obtaining a core word of each text statement according to dependency syntax analysis; determining participles which are in a dominance relation and a move-guest relation with the core word dependency relation; combining the determined participles according to a preset sequence to obtain a topic main stem, wherein the preset sequence is as follows: a combination of a master-predicate-guest sequence and other relationships, wherein the other relationships are: the combination of the main and auxiliary relation words, the core words and the moving and guest relation words; and filling the determined topic main stems to obtain the topics.

Here, the processor 501 is configured to execute the topic analysis program to implement the following topic analysis steps: filling by finding words with a relationship in a fixed relationship or a state with subject, predicate and object dependence; keeping the filling result with the length within the preset length interval as a topic; calculating similarity values with the acquired other topics aiming at each topic; carrying out similarity combination according to the similarity value; merging according to the document id distribution of topics: if the number of the same ids in the document id lists of the two topics exceeds the preset number, combining the two topics into one topic; and re-sequencing the combined topics, and selecting the topics with the target number according to the frequency and outputting the topics.

Optionally, the processor 501 may be a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component. Here, the program executed by the processor 501 may be stored in a memory 503 connected to the processor 501 through a communication bus 502, and the memory 503 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM, Double Data Synchronous Random Access Memory), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), Synchronous linked Dynamic Random Access Memory (SLDRAM, Synchronous Random Access Memory), Direct Memory bus (DRmb Access Memory, Random Access Memory). The memory 503 described in connection with the embodiments of the invention is intended to comprise, without being limited to, these and any other suitable types of memory 503. The memory 503 in the present embodiment is used to store various types of data to support the operation of the processor 501. Examples of such data include: any computer programs for the processor 501 to operate on, such as an operating system and application programs; contact data; telephone book data; a message; a picture; video, etc. The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, and is used for implementing various basic services and processing hardware-based tasks.

To achieve the above object, the present invention further provides a computer readable storage medium, specifically a computer readable storage medium, storing one or more programs, which are executable by one or more processors 501, so as to cause the one or more processors 501 to perform the topic analysis step of any one of the above schemes: acquiring text corpora to be processed, and acquiring word segmentation results and corresponding parts of speech corresponding to each text corpora to be processed; filtering the text corpus to be processed according to the word segmentation result, and acquiring the filtered text corpus; analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain the grammar components of the word segmentation and the dependency relationship among the words and obtain the dependency pair corresponding to each text corpus; obtaining topics corresponding to each text corpus according to the combined sentence pattern structure and the dependency pairs; and acquiring similar topics and sequencing according to the number of the similar topics.

Optionally, the one or more programs may be executable by the one or more processors 501 to cause the one or more processors 501 to perform the following topic analysis steps: performing sentence division processing on the text corpus according to punctuation marks; and performing word segmentation processing on each text corpus to be processed to obtain word segmentation results and corresponding parts of speech.

Optionally, the one or more programs may be executable by the one or more processors 501 to cause the one or more processors 501 to perform the following topic analysis steps: calculating the document frequency of each word segmentation in the word segmentation result, and performing descending order arrangement; acquiring the keywords of the topics arranged in the front; acquiring a text corpus to be filtered, wherein the text corpus to be filtered is a text corpus which does not contain any participle in topic keywords; and removing the text corpus to be filtered from the text corpus to be processed to obtain the filtered text corpus.

Optionally, the one or more programs may be executable by the one or more processors 501 to cause the one or more processors 501 to perform the following topic analysis steps: randomly distributing numbers to all documents in the text corpus, deleting preset punctuations in all documents in the text corpus to obtain target text sentences, and marking the document numbers of the text sentences; and dividing the target text sentence by adopting punctuation marks, counting the frequency of the divided sentence, and marking the document number of the divided sentence as the document id of the sentence. Wherein the punctuation mark comprises at least: comma, semicolon, period, question mark, exclamation mark, ellipsis; and taking the segmented text corpus marking the frequency number and the document id as a text corpus to be processed.

Optionally, the one or more programs may be executable by the one or more processors 501 to cause the one or more processors 501 to perform the following topic analysis steps: performing word segmentation processing on each text corpus to be processed; removing stop words, special symbols, letters and emoticons in the word segmentation processing result; and obtaining word segmentation results.

Optionally, the one or more programs may be executable by the one or more processors 501 to cause the one or more processors 501 to perform the following topic analysis steps: analyzing the word segmentation result and the corresponding part of speech of each filtered text corpus through dependency syntax to obtain a dependency relationship between grammar components and the word segmentation, wherein the grammar components comprise a subject, a predicate, an object, a fixed language, a shape and a complement, and the dependency relationship is a phrase relationship; determining the participles forming the dependency relationship as a dependency pair; wherein the dependency syntax is: graph-based analysis methods, transfer-based analysis methods, or deep learning-based analysis methods.

Optionally, the one or more programs may be executable by the one or more processors 501 to cause the one or more processors 501 to perform the following topic analysis steps: obtaining a core word of each text statement according to dependency syntax analysis; determining participles which are in a dominance relation and a move-guest relation with the core word dependency relation; combining the determined participles according to a preset sequence to obtain a topic main stem, wherein the preset sequence is as follows: a combination of a master-predicate-guest sequence and other relationships, wherein the other relationships are: the combination of the main and auxiliary relation words, the core words and the moving and guest relation words; and filling the determined topic main stems to obtain the topics.

Optionally, the one or more programs may be executable by the one or more processors 501 to cause the one or more processors 501 to perform the following topic analysis steps: filling by finding words with a relationship in a fixed relationship or a state with subject, predicate and object dependence; keeping the filling result with the length within the preset length interval as a topic; calculating similarity values with the acquired other topics aiming at each topic; carrying out similarity combination according to the similarity value; merging according to the document id distribution of topics: if the number of the same ids in the document id lists of the two topics exceeds the preset number, combining the two topics into one topic; and re-sequencing the combined topics, and selecting the topics with the target number according to the frequency and outputting the topics.

Alternatively, the computer-readable storage medium may be a volatile memory, such as a random access memory; or a non-volatile memory, such as a read-only memory, flash memory, hard disk, or solid state disk; or may be a respective device, such as a mobile phone, computer, tablet device, personal digital assistant, etc., that includes one or any combination of the above-described memories 503.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.