阅读说明：本技术 语种识别方法、装置、电子设备和存储介质 (Language identification method and device, electronic equipment and storage medium ) 是由 方昕 李晋 刘俊华 于 2020-03-17 设计创作，主要内容包括：本发明实施例提供一种语种识别方法、装置、电子设备和存储介质,其中方法包括：确定待识别语音数据；将待识别语音数据输入至语种识别模型中,得到语种识别模型输出的语种识别结果；语种识别模型是基于样本语音数据、样本语音数据的语种,以及多个语种的描述文本训练得到的；多个语种包括集内语种和集外语种,集内语种为样本语音数据的语种。本发明实施例提供的语种识别方法、装置、电子设备和存储介质,语种识别模型基于集内语种和集外语种的描述文本,对待识别语音数据进行语种识别,实现了包含集外语种在内的准确的语种识别。(The embodiment of the invention provides a language identification method, a language identification device, electronic equipment and a storage medium, wherein the method comprises the following steps: determining voice data to be recognized; inputting the voice data to be recognized into the language recognition model to obtain a language recognition result output by the language recognition model; the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages; the plurality of languages includes an intra-collection language and an extra-collection language, and the intra-collection language is a language of the sample voice data. According to the language identification method, the language identification device, the electronic equipment and the storage medium provided by the embodiment of the invention, the language identification model is used for carrying out language identification on the voice data to be identified based on the description texts of the language in the set and the language out of the set, so that accurate language identification including the language in the set is realized.)

1. A language identification method, comprising:

determining voice data to be recognized;

inputting the voice data to be recognized into a language recognition model to obtain a language recognition result output by the language recognition model;

the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages; the languages include an intra-collection language and an extra-collection language, and the intra-collection language is the language of the sample voice data.

2. The language identification method according to claim 1, wherein the language identification model is configured to determine a speech language representation vector corresponding to the speech data to be identified, and perform language identification based on the speech language representation vector and the text language representation vectors of multiple languages, and a text language representation vector of any language is determined based on the description text of any language.

3. The language identification method according to claim 2, wherein the inputting the speech data to be identified into a language identification model to obtain a language identification result output by the language identification model specifically comprises:

inputting the voice data to be recognized to a voice language representation layer of the language recognition model to obtain the voice language representation vector output by the voice language representation layer;

and inputting the voice language representation vector and the text language representation vector of each language into a similarity judgment layer of the language identification model to obtain the language identification result output by the similarity judgment layer.

4. The language identification method according to claim 3, wherein the inputting the speech data to be identified into a speech language representation layer of the language identification model to obtain the speech language representation vector outputted by the speech language representation layer specifically comprises:

inputting the voice data to be recognized to a voice feature extraction layer of the voice language representation layer to obtain a voice feature vector output by the voice feature extraction layer;

and inputting the voice feature vector to a spatial transformation layer of the voice language representation layer to obtain the voice language representation vector output by the spatial transformation layer.

5. The language identification method of claim 4, wherein the speech language characterization layer further comprises a language classification layer, and the language classification layer is configured to determine a language corresponding to the speech feature vector;

the speech feature extraction layer and the language classification layer form a language classification model, and the language classification model is obtained based on sample speech data and language training of the sample speech data.

6. The language identification method according to claim 3, wherein said speech language representation layer constitutes a speech language representation model, and said speech language representation model is trained based on sample speech data and a text language representation vector corresponding to a description text of a language of said sample speech data.

7. The language identification method according to claim 6, wherein the speech language representation model is obtained based on sample speech data, text language representation vectors corresponding to description texts of the languages of the sample speech data, and a language representation discriminator;

and the voice language representation model and the language representation discriminator form a generation countermeasure network.

8. The language identification method according to any one of claims 1 to 7 wherein a text language representation vector of any language is specifically determined based on language attribute information extracted from a description text of said any language; the language attribute information includes at least one of language family, use area, use crowd and pronunciation characteristics of any language.

9. A language identification device, comprising:

a determination unit configured to determine voice data to be recognized;

the language identification unit is used for inputting the voice data to be identified into a language identification model to obtain a language identification result output by the language identification model;

the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages; the languages include an intra-collection language and an extra-collection language, and the intra-collection language is the language of the sample voice data.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the language identification method according to any of claims 1 to 8 when executing said program.

11. A non-transitory computer readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the language identification method according to any one of claims 1 to 8.

Technical Field

The present invention relates to the field of natural language processing technologies, and in particular, to a language identification method and apparatus, an electronic device, and a storage medium.

Background

Language identification means that a machine automatically judges the language type to which voice data belongs, for example, chinese, english, french, japanese, or the like, based on the input voice data.

The current language identification methods, such as a language identification method based on a phoneme recognizer or a language identification method based on a deep neural network, are all language identification schemes for a closed set, that is, only the language corresponding to the sample speech in the training set can be identified. If the language corresponding to the voice data to be recognized never appears in the training set, the language of the voice data cannot be recognized accurately by the current language recognition method.

Disclosure of Invention

The embodiment of the invention provides a language identification method, a language identification device, electronic equipment and a storage medium, which are used for solving the problem that the conventional language identification method cannot identify languages which are not in a training set.

In a first aspect, an embodiment of the present invention provides a language identification method, including:

determining voice data to be recognized;

inputting the voice data to be recognized into a language recognition model to obtain a language recognition result output by the language recognition model;

the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages; the languages include an intra-collection language and an extra-collection language, and the intra-collection language is the language of the sample voice data.

Optionally, the language identification model is configured to determine a speech language characterization vector corresponding to the speech data to be identified, and perform language identification based on the speech language characterization vector and the text language characterization vectors of multiple languages, where a text language characterization vector of any language is determined based on a description text of any language.

Optionally, the inputting the speech data to be recognized into a language recognition model to obtain a language recognition result output by the language recognition model specifically includes:

inputting the voice data to be recognized to a voice language representation layer of the language recognition model to obtain the voice language representation vector output by the voice language representation layer;

and inputting the voice language representation vector and the text language representation vector of each language into a similarity judgment layer of the language identification model to obtain the language identification result output by the similarity judgment layer.

Optionally, the inputting the speech data to be recognized into a speech language representation layer of the language recognition model to obtain the speech language representation vector output by the speech language representation layer specifically includes:

inputting the voice data to be recognized to a voice feature extraction layer of the voice language representation layer to obtain a voice feature vector output by the voice feature extraction layer;

and inputting the voice feature vector to a spatial transformation layer of the voice language representation layer to obtain the voice language representation vector output by the spatial transformation layer.

Optionally, the speech language characterization layer further includes a language classification layer, where the language classification layer is configured to determine a language corresponding to the speech feature vector;

the speech feature extraction layer and the language classification layer form a language classification model, and the language classification model is obtained based on sample speech data and language training of the sample speech data.

Optionally, the speech language representation layer constitutes a speech language representation model, and the speech language representation model is obtained by training based on sample speech data and a text language representation vector corresponding to a description text of a language of the sample speech data.

Optionally, the speech language characterization model is obtained based on sample speech data, a text language characterization vector corresponding to a description text of a language of the sample speech data, and a language characterization discriminator;

and the voice language representation model and the language representation discriminator form a generation countermeasure network.

Optionally, the language representation vector of the text in any language is specifically determined based on language attribute information extracted from the description text in any language; the language attribute information includes at least one of language family, use area, use crowd and pronunciation characteristics of any language.

In a second aspect, an embodiment of the present invention provides a language identification apparatus, including:

a determination unit configured to determine voice data to be recognized;

the language identification unit is used for inputting the voice data to be identified into a language identification model to obtain a language identification result output by the language identification model;

the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages; the languages include an intra-collection language and an extra-collection language, and the intra-collection language is the language of the sample voice data.

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a bus, where the processor and the communication interface, the memory complete mutual communication through the bus, and the processor may call a logic command in the memory to perform the steps of the method provided in the first aspect.

In a fourth aspect, an embodiment of the present invention provides a non-transitory computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the method as provided in the first aspect.

According to the language identification method, the language identification device, the electronic equipment and the storage medium, the language identification model is based on the description texts of the language in the set and the language out of the set, the language identification is carried out on the voice data to be identified, and accurate language identification including the language in the set is realized.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a flowchart illustrating a language identification method according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a language identification model operation method according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating a speech language characterization method according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating a language identification method according to another embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a language identification device according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

With the continuous development of speech recognition technology, language recognition is widely applied as a front-end system of speech recognition. For example, in the field of telephone service, the language identification system can automatically forward the telephone corresponding to different languages to corresponding service personnel or systems, or the one-key translation function on the translator can automatically judge the language of the speech to be translated, so as to automatically select the corresponding speech recognition and translation model, and the like.

The current language identification methods mainly comprise two types: a language identification method based on a phoneme recognizer and a language identification method based on bottom acoustic characteristics. The language identification method based on the phoneme recognizer utilizes the difference of phoneme collocation relationship between different languages as a feature to carry out language identification, and the language identification method based on the bottom acoustic feature utilizes the statistical characteristic difference of acoustic units which can be described by the bottom acoustic feature to classify the languages.

However, the above-mentioned language identification methods are all language identification schemes for a closed set, that is, the above-mentioned language identification method can identify a certain language only when sample voice data of the language is learned in the training process. However, when the sample speech data of a certain language never appears in the training set, i.e. the model never learns the sample speech data of the language in the training process, the current language identification method cannot identify the language.

Accordingly, an embodiment of the present invention provides a language identification method. Fig. 1 is a schematic flow chart of a language identification method according to an embodiment of the present invention, as shown in fig. 1, the method includes:

step 110, determining voice data to be recognized.

Here, the speech data to be recognized is speech data that needs to be language-recognized. The voice data to be recognized may be obtained through a sound pickup device, where the sound pickup device may be a smart phone, a tablet computer, or an intelligent electrical appliance such as a sound, a television, an air conditioner, and the like, and the sound pickup device may further amplify and reduce noise of the voice data to be recognized after acquiring the voice data to be recognized through sound pickup by the microphone array, which is not specifically limited in the embodiment of the present invention.

Step 120, inputting the voice data to be recognized into the language recognition model to obtain a language recognition result output by the language recognition model;

the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages; the plurality of languages include an intra-collection language which is a language of the sample voice data and an extra-collection language which is a language other than the language of the sample voice data.

Here, the description text of any language is a natural language description text including language attribute information corresponding to the language, and the language attribute information may specifically be geographical distribution, language history, word source, grammar, and the like of the language. The description text of any language may be obtained from related data for introducing language knowledge, such as encyclopedia pages corresponding to each language, which is not specifically limited in this embodiment of the present invention.

For example, the descriptive text for malaysia may be: "Malaysia is the official language of Malaysia and Welan, and is also one of the official languages of Singapore. Malaysia was also used in many places other than the island Sumenda wax by 1945. Belongs to the Nandamo system Indonesian language family. Distributed in Sumengla, Liao and Linga islands of Malaysia, Singapore, Wenlai, south thailand and Indonesia. Malay has 6 vowels, 3 diphthongs, and 24 consonants. Borrowing is from Sanskrit and Arabic ".

The language identification model analyzes and identifies the voice data to be identified based on the language attribute information learned from the description texts of a plurality of languages, thereby determining the language identification result. Here, the language identification result is the language of the voice data to be identified.

Because the language identification model learns the language attribute information of the corresponding language from the description text of each language in advance, even if the language of the voice data to be identified belongs to the foreign language collection, namely the voice training set of the language identification model does not contain the sample voice data of the language, the language identification model never learns the sample voice data of the language in the training process, and the language identification model can obtain an accurate language identification result based on the voice data to be identified and the description texts of the languages in and out of the collection, thereby realizing the accurate identification of the foreign language collection.

In addition, before step 120 is executed, the language identification model may be obtained by training in advance, and specifically, the language identification model may be obtained by training in the following manner: first, sample speech data of a large number of known languages and description texts of a plurality of languages are collected. The plurality of languages include a language corresponding to the sample voice data, i.e., a language in the set, and a language other than the language in the set, i.e., a foreign language in the set. Then, the initial model is trained based on the sample voice data, the language of the sample voice data and the description texts of a plurality of languages, so that a language identification model is obtained. The initial model may be a single neural network model or a combination of multiple neural network models.

For example, mandarin, english, and russian are used as the languages in the set, sample voice data corresponding to mandarin, english, and russian are collected, and description texts corresponding to mandarin, english, and russian are obtained. And taking Spanish, German and French as the foreign language collection, and acquiring description texts respectively corresponding to Spanish, German and French. And applying the sample voice data respectively corresponding to Mandarin, English and Russian and the description texts respectively corresponding to Mandarin, English, Russian, Spanish, German and French to the training of the initial model, thereby obtaining the language identification model. The language identification model obtained by the method not only can identify mandarin, English and Russian, but also can identify Spanish, German and French which have not learned sample voice data.

According to the method provided by the embodiment of the invention, the language identification model identifies the languages of the voice data to be identified based on the description texts of the language in the set and the language out of the set, so that the accurate language identification including the language in the set is realized.

Based on the above embodiment, the language identification model is used to determine the speech language characterization vector corresponding to the speech data to be identified, and perform language identification based on the speech language characterization vector and the text language characterization vectors of multiple languages, where the text language characterization vector of any language is determined based on the description text of the language.

Specifically, the language identification model is used for determining a speech language representation vector corresponding to the speech data to be identified. Here, the speech language representation vector corresponding to the speech data to be recognized is a vector representation of the language attribute information of the corresponding language obtained based on the speech data to be recognized, wherein the speech language representation vector can be used to distinguish the languages.

Then, the language identification model matches the voice language representation vector with the text language representation vectors of a plurality of languages respectively to obtain a language identification result. The language identification result is the language of the voice data to be identified.

The language representation vector of any language is determined based on the description text of the language and is used for representing the language attribute information of the language. Optionally, a text language representation vector of any language is extracted from the description text of the language through a pre-trained Word vector model, for example, a Word2vec model or a Glove model, or a sentence vector model, for example, an Elmo model or a Bert model, which is not specifically limited in this embodiment of the present invention.

Further, if a pre-trained word vector model is adopted, an average vector of all word vectors in a description text of any language can be used as a text language representation vector of the language, or based on TF-IDF (term frequency-inverse document frequency), a weighted average is performed on the word vectors of the description text of any language to obtain the text language representation vector of the language. If a pre-trained sentence vector model is adopted, the description text of any language can be divided into sentences, and the average vector of all the sentence vectors in the description text of the language is used as the text language representation vector of the language. The embodiment of the invention does not specifically limit the acquisition process of the text language representation vector. It should be noted that, for the description text of each language, the same method is adopted to obtain the corresponding text language representation vector.

The language type characterization vector corresponding to the speech data to be recognized determined by the language type recognition model can accurately characterize the language attribute information contained in the speech data to be recognized, and in addition, the text language type characterization vector of each language can characterize the language attribute information of the language. Therefore, the speech language representation vector corresponding to the speech data to be recognized and the text language representation vector of each language can be used for representing the language attribute information of the corresponding language, and the language recognition result of the speech data to be recognized can be determined by matching the speech language representation vector with the text language representation vector of each language. Therefore, even if the language of the speech data to be recognized belongs to the foreign language collection, i.e. the speech training set of the language recognition model does not contain the sample speech data of the language, and the language recognition model never learns the sample speech data of the language in the training process, but the description texts of a plurality of languages contain the description texts of the foreign language collection, so that the language recognition model can obtain an accurate language recognition result based on the representation vectors of the speech language and the representation vectors of the text languages of the foreign language collection and the language collection, thereby realizing the accurate recognition of the foreign language collection.

The method provided by the embodiment of the invention determines the voice language representation vector of the voice data to be recognized through the language identification model, and then performs language identification based on the voice language representation vector and the text language representation vectors of the language in the set and the language out of the set, thereby realizing accurate identification including the language in the set.

Based on any of the above embodiments, fig. 2 is a schematic flow chart of a language identification model operation method provided by an embodiment of the present invention, as shown in fig. 2, in the method, step 120 specifically includes:

step 121, inputting the voice data to be recognized into a voice language representation layer of a language recognition model to obtain a voice language representation vector output by the voice language representation layer;

and step 122, inputting the voice language representation vector and the text language representation vector of each language into a similarity judgment layer of the language identification model to obtain a language identification result output by the similarity judgment layer.

Specifically, the speech language characterization layer is configured to determine, based on the speech data to be recognized, a speech language characterization vector corresponding to the speech data to be recognized. The similarity judging layer is used for calculating the similarity between the voice language representation vector and the text language representation vector of each language, and obtaining a language identification result based on the similarity between the voice language representation vector and the text language representation vector of each language. Optionally, the language with the highest similarity to the speech language characterization vector may be determined and output as the language identification result, or the probability that the speech data to be identified corresponds to each language may be determined based on the similarity between the speech language characterization vector and the text language characterization vector of each language, and the language with the highest probability is output as the language identification result.

The method provided by the embodiment of the invention determines the voice language type representation vector of the voice data to be recognized through the voice language type representation layer, and the similarity judgment layer determines the language type recognition result based on the similarity between the voice language type representation vector and the text language type representation vectors of a plurality of languages, thereby realizing the accurate recognition of foreign language collection.

Based on any of the above embodiments, fig. 3 is a schematic flow chart of the speech language characterization method provided by the embodiment of the present invention, and as shown in fig. 3, step 121 specifically includes:

step 1211, inputting the voice data to be recognized into the voice feature extraction layer of the voice language representation layer, and obtaining the voice feature vector output by the voice feature extraction layer.

Specifically, the voice feature extraction layer is used for extracting voice feature vectors of voice data to be recognized. The speech feature vector is used to represent features that can distinguish different languages in the speech data to be recognized, such as prosodic features, spectral features, phoneme collocation relationships, included vocabularies or grammars, and the like. The speech feature extraction layer may extract the speech feature vector of the speech data to be recognized by using a Neural Network model such as DNN (Deep Neural Network), RNN (Recurrent Neural Network), or CNN (convolutional Neural Network), which is not specifically limited in this embodiment of the present invention.

Step 1212, inputting the speech feature vector to a spatial transformation layer of the speech language representation layer to obtain a speech language representation vector output by the spatial transformation layer.

Specifically, the spatial transform layer is configured to analyze a language characteristic in the speech feature vector, and transform the speech feature vector into a speech language characterization vector that can characterize the language characteristic of the speech data to be recognized. In the training process, the space conversion layer learns the mapping relation between the voice feature vector and the voice language characterization vector, so that the input voice feature vector can be converted into the voice language characterization vector. The spatial transform layer may use a neural network model such as DNN, RNN, or CNN to convert the speech feature vector into a speech language feature vector, which is not specifically limited in this embodiment of the present invention.

The method provided by the embodiment of the invention provides a basis for language identification by converting the voice feature vector of the voice data to be identified into the voice language representation vector.

Based on any of the above embodiments, in the method, the speech language characterization layer further includes a language classification layer, where the language classification layer is configured to determine a language corresponding to the speech feature vector;

the speech feature extraction layer and the language classification layer form a language classification model, and the language classification model is obtained based on sample speech data and language training of the sample speech data.

Specifically, in order to improve the language distinguishing capability of the speech feature vector extracted by the speech feature extraction layer, a language classification layer for determining the language corresponding to the speech feature vector is further arranged in the speech language characterization layer, wherein the speech feature extraction layer and the language classification layer form a language classification model. Here, the language classification model may be trained as follows: first, a large amount of sample voice data is collected while determining the language of the sample voice data. And then, training the language classification model based on the sample voice data and the language of the sample voice data, and updating the parameters of the language classification model. Further, Cross Entropy (CE) between the language output by the language classification model and the language of the predetermined sample speech data may be used as a loss function to perform model training with the aim of reducing the difference between the two languages, thereby improving the accuracy of the language output by the language classification model.

After training is finished, the speech feature vectors extracted by the speech feature extraction layer in the language classification model have enough language distinguishing capability, and the method is favorable for improving the recognition accuracy of the language recognition model. It should be noted that the language classification layer may only exist in the training stage of the language classification model, and when performing language identification on the speech data to be identified based on the language identification model, the language classification layer does not participate in the language identification process.

The method provided by the embodiment of the invention enhances the language distinguishing capability of the speech feature vector in the training process by adding the language classification layer, and is beneficial to improving the recognition accuracy of the language recognition model.

Based on any of the above embodiments, in the method, the speech language characterization layer forms a speech language characterization model, and the speech language characterization model is obtained by training based on the sample speech data and a text language characterization vector corresponding to a description text of the language of the sample speech data.

Specifically, the speech language representation layer can be used as a speech language representation model to be trained independently, and can be trained in the following way: firstly, a large amount of sample voice data is collected, and a text language representation vector corresponding to a description text of the language of the sample voice data is determined. The determination method of the text language representation vector is the same as that of the text language representation vector in any of the above embodiments, and is not repeated here. Then, the sample voice data is input into the voice language representation model, and the voice language representation vector corresponding to the sample voice data and the text language representation vector corresponding to the description text of the language of the sample voice data are output through the voice language representation model, so that the parameters of the voice language representation model are updated. During specific training, the Minimum Mean Square Error (Minimum Mean Square Error) between the text language representation vector and the speech language representation vector output by the speech language representation model can be used as a loss function, and a gradient descent method can be used for updating the parameters of the speech language representation model.

In the training process of the speech language representation model, the speech language representation vector corresponding to the sample speech data output by the speech language representation model is as close as possible to the text language representation vector corresponding to the description text of the language of the sample speech data. Therefore, after the speech language representation model is trained, the speech language representation vector corresponding to the sample speech data output by the speech language representation model is very similar to the text language representation vector corresponding to the description text of the language of the sample speech data. When language identification is actually carried out, the similarity between the voice language representation vector corresponding to the voice data to be identified and output by the voice language representation model and the text language representation vector corresponding to the language of the voice data to be identified is higher than the similarity between the voice language representation vector corresponding to the voice data to be identified and the text language representation vectors of other languages, so that an accurate language identification result is obtained.

According to the method provided by the embodiment of the invention, the speech language characterization model is trained, so that the similarity between the speech language characterization vector corresponding to the output speech data to be recognized and the text language characterization vector corresponding to the language of the speech data to be recognized is highest, and the recognition accuracy of the language recognition model is improved.

Based on any of the above embodiments, in the method, the speech language characterization model is obtained based on the sample speech data, the text language characterization vector corresponding to the description text of the language of the sample speech data, and the language characterization discriminator through training; the language characterization model and the language characterization discriminator form a generation countermeasure network.

In order to further improve the similarity between the speech language characterization vector corresponding to the speech data to be recognized and the text language characterization vector corresponding to the language of the speech data to be recognized, so as to improve the accuracy of language recognition, the embodiment of the invention also provides a language characterization discriminator, so that a confrontation network is generated by the speech language characterization model and the language characterization discriminator. In the generation of the countermeasure network, a voice language representation model is a generator, and a language representation discriminator is a discriminator. The language characterization discriminator may be implemented by using a neural network model such as DNN, RNN, or CNN, which is not specifically limited in the present invention.

In the training process, a speech language representation vector corresponding to sample speech data output by the speech language representation model and a text language representation vector corresponding to a description text of the language of the sample speech data are input into the language representation discriminator. Here, the output result of the language representation discriminator is a vector type, specifically, a text language representation vector or a speech language representation vector, and a cross entropy between the output result of the language representation discriminator and the vector type of the actually input language representation discriminator may be used as a loss function to perform model training with the purpose of reducing a difference between the two, so that the trained language representation discriminator can accurately distinguish the type of the input vector.

In this process, the purpose of the speech language representation model is to generate speech language representation vectors that are very similar to the text language representation vectors corresponding to the description texts of the language of the sample speech data as much as possible, so that the language representation discriminator cannot distinguish the two. The language representation discriminator aims to distinguish the speech language representation vector generated by the speech language representation model from the text language representation vector corresponding to the description text of the language of the sample speech data as much as possible.

The speech language characterization model and the language characterization discriminator continuously play in the training process, so that after the training is finished, the speech language characterization model can generate speech language characterization vectors extremely similar to text language characterization vectors corresponding to description texts of the languages of the sample speech data, and the language characterization discriminator cannot distinguish the speech language characterization vectors generated by the speech language characterization model from the text language characterization vectors corresponding to the description texts of the languages of the sample speech data.

Therefore, the trained speech language representation model can further improve the similarity between the speech language representation vector corresponding to the speech data to be recognized and output by the speech language representation model and the text language representation vector corresponding to the language of the speech data to be recognized, so that the similarity is obviously higher than the similarity between the speech language representation vector corresponding to the speech data to be recognized and the text language representation vectors of other languages, and the accuracy of language recognition is finally improved.

In the method provided by the embodiment of the invention, the language representation discriminator is added to form a confrontation network with the voice language representation model, so that the similarity between the voice language representation vector corresponding to the voice data to be recognized and output by the voice language representation model and the text language representation vector corresponding to the language of the voice data to be recognized is further improved, and the accuracy of language recognition is improved.

Based on any of the above embodiments, in the method, the text language representation vector of any language is specifically determined based on language attribute information extracted from the description text of the language; the language attribute information includes at least one of language family, use area, use crowd and pronunciation characteristics of the language.

Specifically, a large amount of redundant information or noise information may exist in the description text of any language, and all information in the description text of the language is indiscriminately processed based on the information to extract the text language representation vector of the language, which may slow down the convergence rate of the language identification model and reduce the performance of the language identification model. Therefore, on the basis of the description text of any language, language attribute information capable of distinguishing the language from other languages, such as one or more of the language family, the use region, the use population and the pronunciation characteristics of the language, is extracted. Then, based on the language attribute information, a text language representation vector of the language is determined.

According to the method provided by the embodiment of the invention, the language attribute information of any language is extracted, and the text language representation vector of the language is determined based on the language attribute information, so that the performance of the language identification model is improved.

Based on any of the above embodiments, fig. 4 is a schematic flow chart of a language identification method according to another embodiment of the present invention, as shown in fig. 4, the method includes the following steps:

firstly, acquiring voice data to be recognized, wherein the voice data to be recognized is voice data of Indonesian;

then, inputting the voice data to be recognized into a voice feature extraction layer of the language recognition model to obtain a voice feature vector of the voice data to be recognized, wherein the voice feature vector is used for representing features capable of distinguishing various languages; the voice language characterization layer can be DNN, RNN or CNN;

then, inputting the voice feature vector of the voice data to be recognized to a spatial transformation layer of a language recognition model to obtain a voice language representation vector output by the spatial transformation layer, wherein the voice language representation vector is used for representing language attribute information of the language of the voice data to be recognized;

and finally, inputting the voice language representation vector and the text language representation vector of each language into a similarity judgment layer of the language identification model, calculating the similarity between the voice language representation vector and the text language representation vector of each language by the similarity judgment layer, selecting the language corresponding to the text language representation vector with the maximum similarity to the voice language representation vector as a language identification result and outputting the language identification result.

It should be noted that the language identification model is trained based on the sample voice data, the language of the sample voice data, and the description texts of a plurality of languages. As an example, 1000 pieces of sample speech data are collected for training of the language identification model, wherein 500 pieces of sample speech data corresponding to malaysia and 500 pieces of sample speech data corresponding to dutch are included. The description texts of multiple languages include description texts of three languages, namely Malaysia language, Dutch language and Indonesia language.

When a language recognition model is trained by using sample voice data of Malaysia and Dutch and description texts of three languages, the spatial transformation layer can convert voice feature vectors of voice data to be recognized, which are output by the voice feature extraction layer, into voice language characterization vectors so as to characterize language attribute information of the languages of the voice data to be recognized. Meanwhile, the method can ensure that the voice language representation vector is very similar to the text language representation vector corresponding to the language of the voice data to be recognized, and the similarity is obviously higher than the similarity between the voice language representation vector corresponding to the voice data to be recognized and the text language representation vectors of other languages.

At this time, even if the sample voice data of the language identification model does not contain the voice data corresponding to the Indonesia language, the trained language identification model can extract the voice language characterization vector of the voice data to be identified, so that the similarity of the voice language characterization vector of the voice data to be identified and the text language characterization vector of the Indonesia language is obviously higher than the similarity between the voice language to be identified and the Malaysia language or the Dutch language, and finally the language corresponding to the voice data to be identified is the Indonesia language.

The method provided by the embodiment of the invention determines the voice language representation vector of the voice data to be recognized through the voice feature extraction layer and the spatial conversion layer, and determines the similarity between the voice language representation vector of the voice data to be recognized and the text language representation vector of each language based on the voice language representation vector of the voice data to be recognized and the text language representation vector of each language, thereby determining the language recognition result, realizing the recognition of foreign languages and improving the recognition accuracy.

Based on any of the above embodiments, fig. 5 is a schematic structural diagram of a language identification device according to an embodiment of the present invention, and as shown in fig. 5, the device includes a determining unit 510 and a language identification unit 520.

The determining unit 510 is configured to determine voice data to be recognized;

the language identification unit 520 is configured to input the speech data to be identified into the language identification model to obtain a language identification result output by the language identification model;

the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages; the plurality of languages includes an intra-collection language and an extra-collection language, and the intra-collection language is a language of the sample voice data.

According to the device provided by the embodiment of the invention, the language identification model identifies the languages of the voice data to be identified based on the description texts of the language in the set and the language out of the set, so that the accurate language identification including the language in the set is realized.

Based on any of the above embodiments, the language identification model is configured to determine a speech language characterization vector corresponding to the speech data to be identified, and perform language identification based on the speech language characterization vector and a plurality of language text characterization vectors, where the text language characterization vector of any language is determined based on the description text of the language.

The device provided by the embodiment of the invention determines the voice language representation vector of the voice data to be recognized through the language identification model, and then performs language identification based on the voice language representation vector and the text language representation vectors of the language in the collection and the language out of the collection, thereby realizing accurate identification including the language in the collection.

Based on any of the above embodiments, the language identification unit 520 specifically includes:

the speech language characterization unit is used for inputting the speech data to be recognized into a speech language characterization layer of the language recognition model to obtain a speech language characterization vector output by the speech language characterization layer;

and the similarity judging unit is used for inputting the voice language representation vector and the text language representation vector of each language into a similarity judging layer of the language identification model to obtain a language identification result output by the similarity judging layer.

The device provided by the embodiment of the invention determines the voice language type representation vector of the voice data to be recognized through the voice language type representation layer, and the similarity judgment layer determines the language type recognition result based on the similarity between the voice language type representation vector and the text language type representation vectors of a plurality of languages, thereby realizing the accurate recognition of foreign language collection.

Based on any of the above embodiments, the speech language characterization unit specifically includes:

the voice feature extraction unit is used for inputting the voice data to be recognized into a voice feature extraction layer of the voice language characterization layer to obtain a voice feature vector output by the voice feature extraction layer;

and the space transformation unit is used for inputting the voice feature vector to a space transformation layer of the voice language representation layer to obtain the voice language representation vector output by the space transformation layer.

The device provided by the embodiment of the invention provides a basis for language identification by converting the voice feature vector of the voice data to be identified into the voice language representation vector.

Based on any of the above embodiments, in the apparatus, the speech language characterization layer further includes a language classification layer, and the language classification layer is configured to determine a language corresponding to the speech feature vector; the speech feature extraction layer and the language classification layer form a language classification model, and the language classification model is obtained based on sample speech data and language training of the sample speech data.

The device provided by the embodiment of the invention enhances the language distinguishing capability of the speech feature vector step by step in the training process by adding the language classification layer, and is beneficial to improving the recognition accuracy of the language recognition model.

Based on any of the above embodiments, in the apparatus, the speech language characterization layer forms a speech language characterization model, and the speech language characterization model is obtained by training based on the sample speech data and a text language characterization vector corresponding to a description text of a language of the sample speech data.

According to the device provided by the embodiment of the invention, the speech language characterization vector corresponding to the speech data to be recognized and output by the speech language characterization model is enabled to have the highest similarity with the text language characterization vector corresponding to the language of the speech data to be recognized by training the speech language characterization model, so that the recognition accuracy of the language recognition model is improved.

Based on any of the above embodiments, in the apparatus, the speech language characterization model is obtained based on the sample speech data, the text language characterization vector corresponding to the description text of the language of the sample speech data, and the language characterization discriminator; the language characterization model and the language characterization discriminator form a generation countermeasure network.

The device provided by the embodiment of the invention forms a confrontation network with the voice language representation model by adding the language representation discriminator, so that the similarity between the voice language representation vector corresponding to the voice data to be recognized and output by the voice language representation model and the text language representation vector corresponding to the language of the voice data to be recognized is further improved, and the accuracy of language recognition is improved.

Based on any of the above embodiments, in the apparatus, the text language representation vector of any language is specifically determined based on language attribute information extracted from the description text of the language; the language attribute information includes at least one of language family, use area, use crowd and pronunciation characteristics of any language.

The device provided by the embodiment of the invention improves the performance of the language identification model by extracting the language attribute information of any language and determining the text language representation vector of the language based on the language attribute information.

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 6, the electronic device may include: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call logical commands in the memory 630 to perform the following method: determining voice data to be recognized; inputting the voice data to be recognized into a language recognition model to obtain a language recognition result output by the language recognition model; the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages.

In addition, the logic commands in the memory 630 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic commands are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of commands for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Embodiments of the present invention further provide a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is implemented to perform the method provided in the foregoing embodiments when executed by a processor, and the method includes: determining voice data to be recognized; inputting the voice data to be recognized into a language recognition model to obtain a language recognition result output by the language recognition model; the language identification model is obtained by training based on sample voice data, the language of the sample voice data and description texts of a plurality of languages.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes commands for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.