阅读说明：本技术 一种空中交通管制指令的语音合成方法及系统 (Voice synthesis method and system for air traffic control instruction ) 是由 杨群 高洁 刘绍翰 于 2021-10-19 设计创作，主要内容包括：本发明公开一种空中交通管制指令的语音合成方法及系统,涉及语音合成技术领域,方法包括：获取训练数据集；训练数据集包括多种空中交通管制指令的文本样本以及与文本样本对应的语音样本；利用训练数据集对语音合成模型进行训练和优化；获取参考音频；参考音频包括默认参考音频和自定义参考音频；默认参考音频为任意一个语音样本；自定义参考音频的特征与语音样本的特征相同；特征包括声音、语速、语调和背景音；将参考音频和待生成空中交通管制指令的语音对应的文本均输入优化后的语音合成模型中,生成具有参考音频的特征的空中交通管制指令的语音。本发明能够使生成的空中交通管制指令的语音更接近于真实的空中交通管制员的录音。(The invention discloses a voice synthesis method and a system of an air traffic control instruction, which relate to the technical field of voice synthesis, and the method comprises the following steps: acquiring a training data set; the training data set comprises text samples of various air traffic control instructions and voice samples corresponding to the text samples; training and optimizing the speech synthesis model by using a training data set; acquiring a reference audio; the reference audio comprises default reference audio and custom reference audio; defaulting the reference audio to be any one voice sample; the characteristics of the self-defined reference audio are the same as those of the voice sample; the characteristics comprise sound, speech speed, intonation and background sound; and inputting the reference audio and the text corresponding to the voice of the air traffic control instruction to be generated into the optimized voice synthesis model, and generating the voice of the air traffic control instruction with the characteristics of the reference audio. The invention can make the generated voice of the air traffic control instruction more approximate to the real recording of the air traffic controller.)

1. A method for speech synthesis of air traffic control commands, the method comprising:

acquiring a training data set; the training data set comprises text samples of various air traffic control instructions and voice samples corresponding to the text samples;

training and optimizing a voice synthesis model by using the training data set to obtain an optimized voice synthesis model;

acquiring a reference audio; the reference audio comprises default reference audio and custom reference audio; the default reference audio is any one of the voice samples; the characteristics of the self-defined reference audio are the same as those of the voice sample; the characteristics comprise sound, speech speed, intonation and background sound;

and inputting the reference audio and the text corresponding to the voice of the air traffic control instruction to be generated into the optimized voice synthesis model, and generating the voice of the air traffic control instruction with the characteristics of the reference audio.

2. The method according to claim 1, wherein the training and optimizing a speech synthesis model using the training data set to obtain an optimized speech synthesis model specifically comprises:

inputting a text sample of the air traffic control instruction into a text coder, and respectively inputting a voice sample corresponding to the text sample into a reference coder and a voice coder for training and optimization to obtain an optimized voice synthesis model; the speech synthesis model includes a text encoder, a reference encoder, and a vocoder.

3. The method of claim 1, wherein the obtaining reference audio further comprises:

acquiring a scene to which voice of an air traffic control instruction to be generated belongs; the scene comprises a clearance, a tower and an approach;

and generating a text of the air traffic control instruction corresponding to the scene by using the air traffic control grammar.

4. The method of claim 1, wherein the obtaining reference audio further comprises:

acquiring a synthesis mode; the synthesis mode comprises a default mode and a custom mode;

when the synthesis mode is a default mode, acquiring the default reference audio, and taking the default reference audio as the reference audio;

and when the synthesis mode is a user-defined mode, acquiring the user-defined reference audio, and taking the user-defined reference audio as the reference audio.

5. A system for speech synthesis of air traffic control commands, the system comprising:

the training data set acquisition module is used for acquiring a training data set; the training data set comprises text samples of various air traffic control instructions and voice samples corresponding to the text samples;

the model training and optimizing module is used for training and optimizing a voice synthesis model by utilizing the training data set to obtain an optimized voice synthesis model;

the reference audio acquisition module is used for acquiring reference audio; the reference audio comprises default reference audio and custom reference audio; the default reference audio is any one of the voice samples; the characteristics of the self-defined reference audio are the same as those of the voice sample; the characteristics comprise sound, speech speed, intonation and background sound;

and the voice generation module is used for inputting the reference audio and the text corresponding to the voice of the air traffic control instruction to be generated into the optimized voice synthesis model and generating the voice of the air traffic control instruction with the characteristics of the reference audio.

6. The system of claim 5, wherein the model training and optimization module comprises:

the model training and optimizing unit is used for inputting the text sample of the air traffic control instruction into a text encoder, and respectively inputting the voice sample corresponding to the text sample into a reference encoder and a voice encoder for training and optimizing to obtain an optimized voice synthesis model; the speech synthesis model includes a text encoder, a reference encoder, and a vocoder.

7. The system of claim 5, wherein the reference audio acquisition module further comprises, before:

the scene acquisition module is used for acquiring a scene to which the voice of the air traffic control instruction to be generated belongs; the scene comprises a clearance, a tower and an approach;

and the instruction text generation module is used for generating the text of the air traffic control instruction corresponding to the scene by using the air traffic control grammar.

8. The system of claim 5, wherein the reference audio acquisition module further comprises, before:

a synthesis mode acquisition module for acquiring a synthesis mode; the synthesis mode comprises a default mode and a custom mode;

a default reference audio acquiring module, configured to acquire the default reference audio when the synthesis mode is a default mode, and use the default reference audio as the reference audio;

and the user-defined reference audio acquisition module is used for acquiring the user-defined reference audio when the synthesis mode is the user-defined mode, and taking the user-defined reference audio as the reference audio.

Technical Field

The invention relates to the technical field of voice synthesis, in particular to a voice synthesis method and system for air traffic control instructions.

Background

In recent years, with the development of artificial intelligence technology, speech recognition technology is also more utilized in the field of air transportation. In current speech recognition research, it is easy to construct an acoustic model of a given language in the presence of sufficient corpus. However, because the dialogue linguistic data of the air traffic control personnel and the pilot are scarce and the labeling brings higher labor and time cost, the problem that the data volume of the voice of the air traffic control instruction is insufficient at present is solved, and how to collect a large amount of voice of the air traffic control instruction becomes a difficult problem.

The voice of the air traffic control instruction sent by the air traffic controller has strong professionalism, is greatly different from other common voices, and the voice of the air traffic control instruction generated by adopting the general voice synthesis technology has certain difference from the real recording of the air traffic controller in the aspects of voice, speed, tone, background voice and the like, and plays no great role in the training of voice recognition. Based on this, how to make the generated voice of the air traffic control instruction closer to the real recording of the air traffic controller in terms of speed, tone, background sound, etc. becomes a problem to be solved by those skilled in the art.

Disclosure of Invention

The invention aims to provide a voice synthesis method and a voice synthesis system for an air traffic control instruction, which can enable the generated voice of the air traffic control instruction to be closer to the real recording of an air traffic controller.

In order to achieve the purpose, the invention provides the following scheme:

a method of speech synthesis of air traffic control instructions, the method comprising:

acquiring a training data set; the training data set comprises text samples of various air traffic control instructions and voice samples corresponding to the text samples;

training and optimizing a voice synthesis model by using the training data set to obtain an optimized voice synthesis model;

acquiring a reference audio; the reference audio comprises default reference audio and custom reference audio; the default reference audio is any one of the voice samples; the characteristics of the self-defined reference audio are the same as those of the voice sample; the characteristics comprise sound, speech speed, intonation and background sound;

and inputting the reference audio and the text corresponding to the voice of the air traffic control instruction to be generated into the optimized voice synthesis model, and generating the voice of the air traffic control instruction with the characteristics of the reference audio.

Optionally, the training and optimizing a speech synthesis model by using the training data set to obtain an optimized speech synthesis model specifically includes:

inputting a text sample of the air traffic control instruction into a text coder, and respectively inputting a voice sample corresponding to the text sample into a reference coder and a voice coder for training and optimization to obtain an optimized voice synthesis model; the speech synthesis model includes a text encoder, a reference encoder, and a vocoder.

Optionally, the obtaining the reference audio further includes:

acquiring a scene to which voice of an air traffic control instruction to be generated belongs; the scene comprises a clearance, a tower and an approach;

and generating a text of the air traffic control instruction corresponding to the scene by using the air traffic control grammar.

Optionally, the obtaining the reference audio further includes:

acquiring a synthesis mode; the synthesis mode comprises a default mode and a custom mode;

when the synthesis mode is a default mode, acquiring the default reference audio, and taking the default reference audio as the reference audio;

and when the synthesis mode is a user-defined mode, acquiring the user-defined reference audio, and taking the user-defined reference audio as the reference audio.

The invention also provides the following scheme:

a system for speech synthesis of air traffic control instructions, the system comprising:

the training data set acquisition module is used for acquiring a training data set; the training data set comprises text samples of various air traffic control instructions and voice samples corresponding to the text samples;

the model training and optimizing module is used for training and optimizing a voice synthesis model by utilizing the training data set to obtain an optimized voice synthesis model;

the reference audio acquisition module is used for acquiring reference audio; the reference audio comprises default reference audio and custom reference audio; the default reference audio is any one of the voice samples; the characteristics of the self-defined reference audio are the same as those of the voice sample; the characteristics comprise sound, speech speed, intonation and background sound;

and the voice generation module is used for inputting the reference audio and the text corresponding to the voice of the air traffic control instruction to be generated into the optimized voice synthesis model and generating the voice of the air traffic control instruction with the characteristics of the reference audio.

Optionally, the model training and optimizing module specifically includes:

the model training and optimizing unit is used for inputting the text sample of the air traffic control instruction into a text encoder, and respectively inputting the voice sample corresponding to the text sample into a reference encoder and a voice encoder for training and optimizing to obtain an optimized voice synthesis model; the speech synthesis model includes a text encoder, a reference encoder, and a vocoder.

Optionally, the reference audio obtaining module further includes:

the scene acquisition module is used for acquiring a scene to which the voice of the air traffic control instruction to be generated belongs; the scene comprises a clearance, a tower and an approach;

and the instruction text generation module is used for generating the text of the air traffic control instruction corresponding to the scene by using the air traffic control grammar.

Optionally, the reference audio obtaining module further includes:

a synthesis mode acquisition module for acquiring a synthesis mode; the synthesis mode comprises a default mode and a custom mode;

a default reference audio acquiring module, configured to acquire the default reference audio when the synthesis mode is a default mode, and use the default reference audio as the reference audio;

and the user-defined reference audio acquisition module is used for acquiring the user-defined reference audio when the synthesis mode is the user-defined mode, and taking the user-defined reference audio as the reference audio.

According to the specific embodiment provided by the invention, the invention discloses the following technical effects:

the invention discloses a voice synthesis method and a system of air traffic control instructions, which train and optimize a voice synthesis model by utilizing a voice sample of the air traffic control instructions, and generate the voice of the air traffic control instructions with the characteristics of the sound, the speed, the tone, the background sound and the like of reference audio by adopting the optimized voice synthesis model, so that the voice of the generated air traffic control instructions is closer to the real recording of an air traffic controller.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

FIG. 1 is a flow chart of a method for speech synthesis of air traffic control commands according to an embodiment of the present invention;

FIG. 2 is a block diagram of an embodiment of a speech synthesis system for air traffic control commands in accordance with the present invention;

FIG. 3 is a schematic diagram of a speech synthesis system for air traffic control commands in accordance with the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention aims to provide a voice synthesis method and a voice synthesis system for an air traffic control instruction, which can enable the generated voice of the air traffic control instruction to be closer to the real recording of an air traffic controller.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

FIG. 1 is a flowchart of a speech synthesis method for air traffic control commands according to an embodiment of the present invention. Referring to fig. 1, the speech synthesis method of the air traffic control instruction includes:

step 101: acquiring a training data set; the training data set includes text samples of a plurality of air traffic control instructions and speech samples corresponding to the text samples.

Step 102: and training and optimizing the voice synthesis model by using the training data set to obtain the optimized voice synthesis model.

Step 103: acquiring a reference audio; the reference audio comprises default reference audio and custom reference audio; defaulting the reference audio to be any one voice sample; the characteristics of the self-defined reference audio are the same as those of the voice sample; features include voice, speech rate, intonation, and background sounds.

Step 104: and inputting the reference audio and the text corresponding to the voice of the air traffic control instruction to be generated into the optimized voice synthesis model, and generating the voice of the air traffic control instruction with the characteristics of the reference audio.

Specifically, step 102 specifically includes:

inputting a text sample of the air traffic control instruction into a text encoder, and respectively inputting a voice sample corresponding to the text sample into a reference encoder and a voice encoder for training and optimizing to obtain an optimized voice synthesis model; the speech synthesis model includes a text encoder, a reference encoder, and a vocoder.

Step 103 further comprises:

acquiring a scene to which voice of an air traffic control instruction to be generated belongs; the scene includes clearance, tower and approach.

And generating a text of the air traffic control instruction corresponding to the scene by using the air traffic control grammar.

Acquiring a synthesis mode; the composition mode includes a default mode and a custom mode.

And when the synthesis mode is the default mode, acquiring default reference audio, and taking the default reference audio as the reference audio.

And when the synthesis mode is the self-defined mode, obtaining the self-defined reference audio, and taking the self-defined reference audio as the reference audio.

When the scene to which the voice of the air traffic control instruction to be generated belongs is obtained, the number of the voices of the air traffic control instruction to be generated can be obtained at the same time, that is, the user can select the scene to which the air traffic control instruction to be synthesized belongs, the synthesized number and the synthesis mode.

The air traffic control grammar comprises air traffic control command grammar rules under each scene, namely air traffic control commands corresponding to each scene, and corresponding amount of air traffic control command texts are generated according to the air traffic control command grammar rules under each scene, namely the texts of the air traffic control commands.

After the user selects the model for synthesizing the voice (the voice synthesis model), the user can select a default mode or a self-defined mode, when the user selects the default mode, the stored default reference audio is input into the synthesis model, when the user selects the self-defined mode, the audio with the air traffic control style to be synthesized is also required to be input into the voice synthesis model as the reference audio, and the output result of the voice synthesis model is the characteristics of the voice, the speed, the tone and the like of a controller in the simulated user reference audio.

The speech synthesis model includes an encoder, a decoder, and an vocoder, the encoder including a text encoder and a reference audio encoder. The speech synthesis model needs to be trained using speech for at least 20 hours of real air traffic control during the training phase. After the encoder receives data input by a user, the text encoder converts an input text sequence into a corresponding potential representation, and the reference encoder extracts characteristics of the air traffic controller, such as voice, speech speed, tone, environment background (background sound) and the like in reference audio and converts the characteristics into the corresponding potential representation. The potential representation output by the encoder is input to a decoder, which performs spectral prediction based on the upsampled potential representation. The spectrum sequence generated by the decoder is input to a vocoder for voice synthesis, and a waveform is synthesized from the predicted spectrogram by using the vocoder.

The invention provides a voice synthesis method and a voice synthesis system based on a controller style, namely an air traffic control style. If the user selects the custom mode, the reference audio with the desired synthesis style needs to be input into the model, otherwise the model will use the default reference audio. And generating a corresponding number of standard air traffic control commands under the scene according to the scene selected by the user. The generated air traffic control command text is input into a speech synthesis model, which outputs a spectrogram of the predicted synthesized speech and inputs the spectrogram into a vocoder. The vocoder outputs an audio wav file of the synthesized voice based on the input synthesized voice.

The method and the device generate the text of the air traffic control instruction corresponding to the scene by using the air traffic control grammar, generate the standard air traffic control instruction under the scene according to the scene selected by the user, synthesize different audio frequencies at one time, and ensure that the user does not need to manually input the command text to the voice synthesis model. Compared with the current general speech synthesis technology, the method has the advantages that the reference audio encoder is added, and the style characteristic extraction is carried out on the speech audio with the air traffic control style, so that the synthesized speech is closer to the speech under the real air control scene in the aspects of sound, speech speed, tone, background sound and the like.

Fig. 2 is a block diagram of an embodiment of a speech synthesis system for air traffic control commands according to the present invention, fig. 3 is a schematic diagram of a speech synthesis system for air traffic control commands according to the present invention, and referring to fig. 2 and fig. 3, the speech synthesis system for air traffic control commands includes:

a training data set obtaining module 201, configured to obtain a training data set; the training data set includes text samples of a plurality of air traffic control instructions and speech samples corresponding to the text samples.

And the model training and optimizing module 202 is configured to train and optimize the speech synthesis model by using the training data set to obtain an optimized speech synthesis model.

A reference audio obtaining module 203, configured to obtain a reference audio; the reference audio comprises default reference audio and custom reference audio; defaulting the reference audio to be any one voice sample; the characteristics of the self-defined reference audio are the same as those of the voice sample; features include voice, speech rate, intonation, and background sounds.

The speech generation module 204 is configured to input both the reference audio and a text corresponding to the speech of the air traffic control instruction to be generated into the optimized speech synthesis model, and generate speech of the air traffic control instruction with the characteristics of the reference audio.

Specifically, the model training and optimizing module 202 specifically includes:

the model training and optimizing unit is used for inputting the text samples of the air traffic control instructions into a text encoder, and respectively inputting the voice samples corresponding to the text samples into a reference encoder and a voice decoder for training and optimizing to obtain an optimized voice synthesis model; the speech synthesis model includes a text encoder, a reference encoder, and a vocoder.

The reference audio obtaining module 203 further includes:

the scene acquisition module is used for acquiring a scene to which the voice of the air traffic control instruction to be generated belongs; the scene includes clearance, tower and approach.

And the instruction text generation module is used for generating the text of the air traffic control instruction corresponding to the scene by using the air traffic control grammar.

A synthesis mode acquisition module for acquiring a synthesis mode; the composition mode includes a default mode and a custom mode.

The default reference audio acquisition module is used for acquiring default reference audio when the synthesis mode is the default mode, and taking the default reference audio as the reference audio; that is, when the user selects the default mode, the system will enter default reference audio within the system into the speech synthesis model.

And the user-defined reference audio acquisition module is used for acquiring the user-defined reference audio when the synthesis mode is the user-defined mode and taking the user-defined reference audio as the reference audio.

The voice synthesis system of the air traffic control command is added with a command generation model, namely a command text generation module, so that a user does not need to manually input a command text, the system can generate a standard air control command in a scene according to the scene selected by the user, and different audio frequencies can be synthesized at one time.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.