阅读说明：本技术 音频标注的检错方法和装置 (Error detection method and device for audio annotation ) 是由 张晴晴 朱冬 贾艳明 何淑琳 于 2020-11-12 设计创作，主要内容包括：本申请公开了一种音频标注的检错方法,包括：获取音频数据,并将音频数据切分为多个音频片段；对音频片段进行标注,得到初始标注文本；采用通用文本检错模型对初始标注文本进行检错处理,以得到第一标注文本；确定通用文本检错模型的混淆词典；采用文本分类模型识别第一标注文本的领域类别；根据领域类别,采用领域类别对应的领域文本检错模型对第一标注文本进行检错处理,以得到第二标注文本；将通用文本检错模型的混淆词典与领域文本检错模型的第二标注文本作为微调模型的数据库；根据第二标注文本的语义,采用微调模型对第二标注文本进行微调处理,以得到最终的第三标注文本。(The application discloses an error detection method for audio annotation, which comprises the following steps: acquiring audio data and cutting the audio data into a plurality of audio segments; labeling the audio clip to obtain an initial labeling text; adopting a general text error detection model to perform error detection processing on the initial labeling text to obtain a first labeling text; determining a confusion dictionary of a universal text error detection model; identifying the field type of the first labeled text by adopting a text classification model; according to the field type, performing error detection processing on the first labeling text by adopting a field text error detection model corresponding to the field type to obtain a second labeling text; taking the confusion dictionary of the universal text error detection model and the second labeled text of the field text error detection model as a database of the fine tuning model; and performing fine tuning processing on the second labeled text by adopting a fine tuning model according to the semantic meaning of the second labeled text to obtain a final third labeled text.)

1. An error detection method for audio annotation, comprising:

acquiring audio data and segmenting the audio data into a plurality of audio segments;

labeling the audio clip to obtain an initial labeling text;

adopting a general text error detection model to perform error detection processing on the initial labeling text to obtain a first labeling text;

determining a confusion dictionary of the universal text error detection model;

identifying the field type of the first labeled text by adopting a text classification model;

according to the field type, adopting a field text error detection model corresponding to the field type to perform error detection processing on the first labeling text to obtain a second labeling text;

taking the confusion dictionary of the universal text error detection model and the second labeled text of the field text error detection model as a database of a fine tuning model;

and according to the semantics of the second labeled text, carrying out fine adjustment processing on the second labeled text by adopting the fine adjustment model so as to obtain a final third labeled text.

2. An error detection method as defined in claim 1, wherein the confusion dictionary comprises a personal confusion dictionary and a shared confusion dictionary, and wherein the determining the confusion dictionary of the generic text error detection model specifically comprises:

after modification and confirmation of a specific labeling person, recording the text with the labeling error and the frequency of the occurrence of the labeling error;

when the frequency is higher than a threshold value, adding the text with the labeling errors into a personal confusion dictionary of the specific labeling person;

and counting the personal confusion dictionary of a plurality of labeling personnel, and adding the wrongly labeled text into the shared confusion dictionary when the occurrence frequency of the wrongly labeled text is higher than the preset frequency.

3. The error detection method according to claim 1, wherein performing error detection processing on the initial labeled text by using a universal text error detection model to obtain a first labeled text, specifically comprises:

finding out the position of the marking error by adopting a universal text error detection model;

acquiring a candidate item list for replacing the error label from the confusion dictionary;

acquiring a candidate item from the candidate item list to replace the error label;

calculating the fluency and the puzzlement of the replaced marked text by adopting an N-gram model;

and determining the best target candidate item according to the fluency and the confusion degree so as to obtain a first labeling text.

4. The error detection method according to claim 1, wherein the error detection processing is performed on the first labeled text by using a domain text error detection model corresponding to the domain category according to the domain category to obtain a second labeled text, and then the method further comprises:

generating error detection information under the condition that the first annotation text has errors;

wherein the error detection information comprises an audio segment index, an error position index and a candidate word.

5. The error detection method of claim 1 wherein the domain categories include: economy, education, science and technology, society, games, and entertainment.

6. An apparatus for error detection of audio annotations, comprising:

the acquisition module is used for acquiring audio data and segmenting the audio data into a plurality of audio segments;

the marking module is used for marking the audio clip to obtain an initial marking text;

the first error detection module is used for carrying out error detection processing on the initial labeling text by adopting a universal text error detection model so as to obtain a first labeling text;

the determining module is used for determining an confusion dictionary of the universal text error detection model;

the identification module is used for identifying the field type of the first labeled text by adopting a text classification model;

the second error detection module is used for carrying out error detection processing on the first labeling text by adopting a field text error detection model corresponding to the field type according to the field type so as to obtain a second labeling text;

the storage module is used for taking the confusion dictionary of the universal text error detection model and the second labeled text of the field text error detection model as a database of a fine tuning model;

and the fine tuning module is used for performing fine tuning processing on the second labeled text by adopting the fine tuning model according to the semantic meaning of the second labeled text so as to obtain a final third labeled text.

7. The error detection apparatus of claim 6, wherein the confusion dictionary comprises a personal confusion dictionary and a shared confusion dictionary, and wherein the determination module specifically comprises:

the recording submodule is used for recording the text with the wrong label and the frequency of the wrong label after the modification and the confirmation of a specific label person;

the personal dictionary sub-module is used for adding the text with the labeling errors into the personal confusion dictionary of the specific labeling person when the frequency is higher than a threshold value;

and the shared dictionary submodule is used for counting the personal confusion dictionaries of a plurality of labeling personnel, and adding the wrongly labeled texts into the shared confusion dictionary when the times of the wrongly labeled texts are higher than the preset times.

8. The error detection apparatus of claim 6, wherein the first error detection module specifically comprises:

the searching submodule is used for searching out the position of the labeling error by adopting a universal text error detection model;

the obtaining submodule is used for obtaining a candidate item list for replacing the error label from the confusion dictionary;

the replacing submodule is used for acquiring a candidate item from the candidate item list to replace the error label;

the calculation submodule is used for calculating the fluency and the confusion degree of the replaced marked text by adopting an N-gram model;

and the determining submodule is used for determining the best target candidate item according to the fluency and the confusion degree so as to obtain the first labeling text.

9. The error detection apparatus of claim 1, further comprising:

the generating module is used for generating error detection information under the condition that the first annotation text has errors;

wherein the error detection information comprises an audio segment index, an error position index and a candidate word.

10. The error detection apparatus of claim 1 wherein the domain categories include: economy, education, science and technology, society, games, and entertainment.

Technical Field

The application belongs to the field of voice recognition, and particularly relates to an error detection method and device for audio labeling.

Background

With the development of speech recognition technology, speech recognition technology is increasingly applied to various fields, such as: the intelligent home in daily life, the intelligent application in the education field, and the intelligent robot in the medical or financial fields. The current speech recognition technology depends on a speech recognition model of deep learning training to transcribe speech into a text, and then the text is subjected to subsequent processing. Efficient and accurate speech recognition models rely on large amounts of high quality speech data.

However, in the process of implementing the present application, the inventor finds that, in general, the speech data required for training the speech recognition model is obtained by a manual labeling method.

At present, at least the following problems exist: the marking quality of each voice is influenced by the fatigue degree and knowledge cognitive level of current marking personnel, and the situation that wrongly written characters exist in a marked text is inevitable in the marking process. Even if subsequent quality control personnel strictly control the data, the finally obtained labeled data has text errors, the use of the data can cause the process of the trained speech recognition model to be tortuous, and the recognition effect is poor. Of course, the quality inspection cost is increased, and the quality inspection pressure of quality inspectors is increased.

Disclosure of Invention

The embodiment of the application aims to provide an error detection method and device for audio annotation, and the technical problems that the accuracy of a voice recognition model is low and the recognition effect is poor due to the fact that the quality of the existing voice annotation is easily affected by the fatigue degree and knowledge cognition level of annotating personnel can be solved.

In order to solve the technical problem, the present application is implemented as follows:

in a first aspect, an embodiment of the present application provides an error detection method for audio annotation, including:

acquiring audio data and segmenting the audio data into a plurality of audio segments;

labeling the audio clip to obtain an initial labeling text;

adopting a general text error detection model to perform error detection processing on the initial labeling text to obtain a first labeling text;

determining a confusion dictionary of the universal text error detection model;

identifying the field type of the first labeled text by adopting a text classification model;

according to the field type, adopting a field text error detection model corresponding to the field type to perform error detection processing on the first labeling text to obtain a second labeling text;

taking the confusion dictionary of the universal text error detection model and the second labeled text of the field text error detection model as a database of a fine tuning model;

and according to the semantics of the second labeled text, carrying out fine adjustment processing on the second labeled text by adopting the fine adjustment model so as to obtain a final third labeled text.

Further, the confusion dictionary comprises a personal confusion dictionary and a shared confusion dictionary, and the determining the confusion dictionary of the universal text error detection model specifically comprises:

after modification and confirmation of a specific labeling person, recording the text with the labeling error and the frequency of the occurrence of the labeling error;

when the frequency is higher than a threshold value, adding the text with the labeling errors into a personal confusion dictionary of the specific labeling person;

and counting the personal confusion dictionary of a plurality of labeling personnel, and adding the wrongly labeled text into the shared confusion dictionary when the occurrence frequency of the wrongly labeled text is higher than the preset frequency.

Further, performing error detection processing on the initial labeling text by using a general text error detection model to obtain a first labeling text, which specifically comprises:

finding out the position of the marking error by adopting a universal text error detection model;

acquiring a candidate item list for replacing the error label from the confusion dictionary;

acquiring a candidate item from the candidate item list to replace the error label;

calculating the fluency and the puzzlement of the replaced marked text by adopting an N-gram model;

and determining the best target candidate item according to the fluency and the confusion degree so as to obtain a first labeling text.

Further, according to the field type, performing error detection processing on the first labeled text by using a field text error detection model corresponding to the field type to obtain a second labeled text, and then, the method further includes:

generating error detection information under the condition that the first annotation text has errors;

wherein the error detection information comprises an audio segment index, an error position index and a candidate word.

Further, the domain categories include: economy, education, science and technology, society, games, and entertainment.

In a second aspect, an embodiment of the present application provides an apparatus for detecting an error of an audio annotation, including:

the acquisition module is used for acquiring audio data and segmenting the audio data into a plurality of audio segments;

the marking module is used for marking the audio clip to obtain an initial marking text;

the first error detection module is used for carrying out error detection processing on the initial labeling text by adopting a universal text error detection model so as to obtain a first labeling text;

the determining module is used for determining an confusion dictionary of the universal text error detection model;

the identification module is used for identifying the field type of the first labeled text by adopting a text classification model;

the second error detection module is used for carrying out error detection processing on the first labeling text by adopting a field text error detection model corresponding to the field type according to the field type so as to obtain a second labeling text;

the storage module is used for taking the confusion dictionary of the universal text error detection model and the second labeled text of the field text error detection model as a database of a fine tuning model;

and the fine tuning module is used for performing fine tuning processing on the second labeled text by adopting the fine tuning model according to the semantic meaning of the second labeled text so as to obtain a final third labeled text.

Further, the confusion dictionary comprises a personal confusion dictionary and a shared confusion dictionary, and the determining module specifically comprises:

the recording submodule is used for recording the text with the wrong label and the frequency of the wrong label after the modification and the confirmation of a specific label person;

the personal dictionary sub-module is used for adding the text with the labeling errors into the personal confusion dictionary of the specific labeling person when the frequency is higher than a threshold value;

and the shared dictionary submodule is used for counting the personal confusion dictionaries of a plurality of labeling personnel, and adding the wrongly labeled texts into the shared confusion dictionary when the times of the wrongly labeled texts are higher than the preset times.

Further, the first error detection module specifically includes:

the searching submodule is used for searching out the position of the labeling error by adopting a universal text error detection model;

the obtaining submodule is used for obtaining a candidate item list for replacing the error label from the confusion dictionary;

the replacing submodule is used for acquiring a candidate item from the candidate item list to replace the error label;

the calculation submodule is used for calculating the fluency and the confusion degree of the replaced marked text by adopting an N-gram model;

and the determining submodule is used for determining the best target candidate item according to the fluency and the confusion degree so as to obtain the first labeling text.

Further, the error detection apparatus further includes:

the generating module is used for generating error detection information under the condition that the first annotation text has errors;

wherein the error detection information comprises an audio segment index, an error position index and a candidate word.

Further, the domain categories include: economy, education, science and technology, society, games, and entertainment.

In the embodiment of the application, through the universal text error detection model, the field text error detection model and the fine tuning model, automatic error detection of audio data is realized, the rapid and accurate advantages of the universal text error detection model are fully utilized, meanwhile, the field category and context semantics are further considered, the influence of the fatigue degree and knowledge cognitive level of a labeling person on the labeling quality is avoided, the labeling quality is improved, and the accuracy and the recognition effect of the voice recognition model are further improved.

Drawings

Fig. 1 is a flowchart illustrating an error detection method for audio annotation according to an embodiment of the present application;

FIG. 2 is a flowchart illustrating another method for detecting an error in an audio annotation according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an error detection apparatus for audio annotation according to an embodiment of the present disclosure.

Description of reference numerals:

30-error detection device, 301-acquisition module, 302-labeling module, 303-first error detection module, 3031-searching submodule, 3032-acquisition submodule, 3033-replacement submodule, 3034-calculation submodule, 3035-determination submodule, 304-determination module, 3041-recording submodule, 3042-personal dictionary submodule, 3043-shared dictionary submodule, 305-recognition module, 306-second error detection module, 307-warehousing module, 308-fine tuning module and 309-generation module.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings in combination with embodiments.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one.

The following describes the speech processing method provided by the embodiment of the present application in detail through a specific embodiment and an application scenario thereof with reference to the accompanying drawings.

Example one

Referring to fig. 1, a flow chart of an error detection method for an audio annotation provided in an embodiment of the present application is shown, where the error detection method for an audio annotation includes:

s101: audio data is acquired and sliced into a plurality of audio segments.

Optionally, the audio data is subjected to voice dotting by using a voice detection system, and the audio data is segmented according to the dotting.

Alternatively, the audio data may be segmented according to a preset time length, for example, 3 s. The audio data may also be segmented according to phoneme length, e.g. 6 phoneme units.

S102: and labeling the audio clip to obtain an initial labeling text.

The labeling can be performed by using the existing audio labeling method, which is not described herein again.

S103: and carrying out error detection processing on the initial labeling text by adopting a universal text error detection model to obtain a first labeling text.

In particular, the generic text error detection model includes at least a language model of N-Gram.

The database of the universal text error detection model comprises a confusion dictionary, wherein the confusion dictionary comprises a word-level pinyin confusion set, a font confusion set and a word-level confusion set.

It is to be understood that the location of the annotation error can be found by the general error detection model, and the candidate list can be found from the confusion dictionary.

The first labeled text processed by the universal text error detection model preliminarily solves some basic confusion errors.

S104: an obfuscation dictionary of the generic text error detection model is determined.

Specifically, the confusion dictionary of the universal text error detection model can be determined by recording the error-labeled words and the corresponding candidate words in the background.

Further, the confusion dictionary can include a personal confusion dictionary for a particular tagged person and a shared confusion dictionary for all tagged persons.

S105: and identifying the field category of the first labeled text by adopting a text classification model.

Specifically, the text classification models include, but are not limited to, TextCNN, TextRNN, TextRCNN, BERT.

The domain categories may include economy, education, science, society, games, and entertainment.

S106: and according to the field type, performing error detection processing on the first labeling text by adopting a field text error detection model corresponding to the field type to obtain a second labeling text.

It will be appreciated that each domain category corresponds to a respective domain text error detection model.

In particular, the domain text error detection model includes, but is not limited to, BERT and transformations.

In this embodiment, BERT is taken as an example of a classification model, and a domain text error detection model is further described.

Adopting BERT (bidirectional Encoder reproduction from transformations) to realize the discovery of error words and the filtering of candidate words in the first labeled text, utilizing a language model (Masked LM) Masked in the BERT to mask the first labeled text word by word, and finally utilizing a decoder of the BERT to obtain candidate items from a font confusion set of a confusion dictionary.

The second labeled text processed by the domain text error detection model has higher accuracy compared with the first labeled text due to the consideration of the domain to which the text belongs.

S107: taking the confusion dictionary of the universal text error detection model and the second labeled text of the field text error detection model as a database of the fine tuning model;

s108: and performing fine tuning processing on the second labeled text by adopting a fine tuning model according to the semantic meaning of the second labeled text to obtain a final third labeled text.

It should be noted that, the fine tuning model extracts semantic information of the second labeled text by using the confusion dictionary of the general text error detection model and the second labeled text of the field text error detection model, and further performs error detection according to context semantics to obtain a final third labeled text.

For example, the second annotation text processed by the generic text error detection model and the domain text error detection model is "in order to escape the surrounding ring of the enemy, his success is strong", and it is correct when the judgment is made by using the audio clip. However, after the context semantic recognition, the "winning and wanting" can be checked as "living and wanting".

In the embodiment of the application, through the universal text error detection model, the field text error detection model and the fine tuning model, automatic error detection of audio data is realized, the rapid and accurate advantages of the universal text error detection model are fully utilized, meanwhile, the field category and context semantics are further considered, the influence of the fatigue degree and knowledge cognitive level of a labeling person on the labeling quality is avoided, the labeling quality is improved, and the accuracy and the recognition effect of the voice recognition model are further improved.

Example two

Referring to fig. 2, a flow chart of another audio annotation error detection method provided in this embodiment of the present application is shown, where the speech processing method includes:

s201: audio data is acquired and sliced into a plurality of audio segments.

S202: and labeling the audio clip to obtain an initial labeling text.

S203: and finding out the position of the labeling error by adopting a universal text error detection model.

S204: a list of candidates for replacing the false callout is obtained from the confusion dictionary.

S205: and acquiring a candidate item from the candidate item list to replace the error label.

Optionally, the candidate with the highest priority in the candidate list is selected for replacement.

S206: and calculating the fluency and the puzzlement of the replaced marked text by adopting an N-gram model.

It should be understood that the higher the fluency and the lower the confusion, the higher the accuracy of the annotated text. Conversely, the lower the fluency and the higher the confusion, the lower the accuracy of the labeled text.

S207: and determining the optimal target candidate item according to the fluency and the confusion degree so as to obtain the first labeling text.

It should be appreciated that the best target candidate has the highest fluency and/or the lowest confusion after replacement.

Through the comparison of the fluency and the confusion degree, the accuracy of selecting the best target candidate item can be improved.

S208: and after the modification confirmation of the specific labeling personnel, recording the text with the labeling error and the frequency of the occurrence of the labeling error.

S209: and when the frequency is higher than the threshold value, adding the text with the labeling errors into the personal confusion dictionary of the specific labeling personnel.

S210: and counting the personal confusion dictionaries of a plurality of labeling personnel, and adding the wrongly labeled texts into the shared confusion dictionary when the occurrence frequency of the wrongly labeled texts is higher than the preset frequency.

It should be noted that the confusion dictionary in the present embodiment generally includes a personal confusion dictionary and a shared confusion dictionary.

The individual confusion dictionary and the sharing confusion dictionary can achieve the effect of combining personalized error correction and common error sharing.

S211: and identifying the field category of the first labeled text by adopting a text classification model.

S212: and according to the field type, performing error detection processing on the first labeling text by adopting a field text error detection model corresponding to the field type to obtain a second labeling text.

S213: and generating error detection information in the case that the first annotation text has an error.

Wherein the error detection information comprises an audio segment index, an error position index and a candidate word.

The specific position where the error occurs can be quickly positioned through the audio segment index, the error position index and the candidate words, and the error detection efficiency is improved.

S214: and taking the confusion dictionary of the universal text error detection model and the second labeling text of the field text error detection model as a database of the fine tuning model.

S215: and performing fine tuning processing on the second labeled text by adopting a fine tuning model according to the semantic meaning of the second labeled text to obtain a final third labeled text.

In the embodiment of the application, the accuracy of selecting the best target candidate item is improved through the comparison of the fluency and the confusion degree, the effect of combining personalized error correction and common error sharing is achieved through the forms of the personal confusion dictionary and the shared confusion dictionary, the influence of the fatigue degree and the knowledge cognitive level of a marking person on the marking quality can be further avoided, and the marking quality is improved.

EXAMPLE III

Referring to fig. 3, which shows a schematic structural diagram of an error detection apparatus for audio annotation provided in an embodiment of the present application, the error detection apparatus 30 includes:

an obtaining module 301, configured to obtain audio data and divide the audio data into a plurality of audio segments;

a labeling module 302, configured to label the audio segment to obtain an initial labeling text;

the first error detection module 303 is configured to perform error detection processing on the initial labeling text by using a general text error detection model to obtain a first labeling text;

a determining module 304, configured to determine an obfuscation dictionary of the universal text error detection model;

an identifying module 305, configured to identify a domain category of the first labeled text by using a text classification model;

the second error detection module 306 is configured to perform error detection processing on the first labeled text by using a field text error detection model corresponding to the field type according to the field type to obtain a second labeled text;

the entering module 307 is configured to use the confusion dictionary of the general text error detection model and the second labeled text of the domain text error detection model as a database of the fine tuning model;

and the fine-tuning module 308 is configured to perform fine-tuning processing on the second labeled text by using a fine-tuning model according to the semantic meaning of the second labeled text, so as to obtain a final third labeled text.

Further, the confusion dictionary includes a personal confusion dictionary and a shared confusion dictionary, and the determining module 304 specifically includes:

the recording submodule 3041 is configured to record the text with the labeling error and the frequency of the occurrence of the labeling error after the modification and the confirmation of the specific labeling personnel;

the personal dictionary sub-module 3042 is used for adding the text with the labeling error into the personal confusion dictionary of the specific labeling person when the frequency is higher than the threshold value;

the shared dictionary submodule 3043 is configured to count individual confusion dictionaries of multiple labeling personnel, and add a text with a labeling error to the shared confusion dictionary when the occurrence frequency of the text with a labeling error is higher than a preset frequency.

Further, the first error detection module 303 specifically includes:

the searching submodule 3031 is used for searching out the position of the marking error by adopting a universal text error detection model;

an obtaining submodule 3032, configured to obtain a candidate item list for replacing the error label from the confusion dictionary;

a replacement submodule 3033, configured to obtain a candidate item from the candidate item list and replace the error label;

a calculation submodule 3034, configured to calculate fluency and confusion of the replaced annotation text by using an N-gram model;

and the determining sub-module 3035 is configured to determine an optimal target candidate item according to the fluency and the confusion degree to obtain the first annotation text.

Further, the error detection apparatus 30 further includes:

a generating module 309, configured to generate error detection information when the first annotation text has an error;

wherein the error detection information comprises an audio segment index, an error position index and a candidate word.

Further, the domain categories include: economy, education, science and technology, society, games, and entertainment.

The error detection apparatus 30 provided in this embodiment of the present application can implement each process implemented in the foregoing method embodiments, and is not described here again to avoid repetition.

In the embodiment of the application, through the universal text error detection model, the field text error detection model and the fine tuning model, automatic error detection of audio data is realized, the rapid and accurate advantages of the universal text error detection model are fully utilized, meanwhile, the field category and context semantics are further considered, the influence of the fatigue degree and knowledge cognitive level of a labeling person on the labeling quality is avoided, the labeling quality is improved, and the accuracy and the recognition effect of the voice recognition model are further improved.

The virtual device in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal.

The above description is only an example of the present invention, and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.