阅读说明：本技术 一种针对语音唤醒后提升自动语音识别准确率的方法 (Method for improving automatic voice recognition accuracy rate after voice awakening ) 是由 尹钧 赵亚丽 缪炜 于 2021-09-27 设计创作，主要内容包括：本发明涉及智能语音交互技术领域,且公开了一种针对语音唤醒后提升自动语音识别准确率的方法,将N个语音采集设备以间距d线性排列构成语音采集模块,N为大于等于2的正整数,将采集到的多通道带噪语音数据经傅里叶变换后输入固定波束形成模块,根据预先设计的多个导向矢量生成多个方向的固定波束。该针对语音唤醒后提升自动语音识别准确率的方法,通过提供一种简单有效的提升识别正确率的方法,利用了唤醒加识别的这种常用语音交互模式,将看似没有关系的两者有效结合在一起,以固定波束形成方式,且只在唤醒后识别前触发噪声统计的更新,无需再估计语音统计特性,避免了复杂的计算以及参数估计错误带来的语音畸变。(The invention relates to the technical field of intelligent voice interaction, and discloses a method for improving the accuracy of automatic voice recognition after voice awakening, which comprises the steps of linearly arranging N voice acquisition devices at a distance d to form a voice acquisition module, inputting acquired multi-channel noisy voice data into a fixed beam forming module after Fourier transformation, and generating fixed beams in multiple directions according to a plurality of pre-designed guide vectors, wherein N is a positive integer greater than or equal to 2. The method for improving the accuracy of automatic voice recognition after voice awakening utilizes a common voice interaction mode of awakening and recognition, effectively combines the two which seem unrelated together in a fixed beam forming mode, triggers the updating of noise statistics only before the recognition after the awakening, does not need to estimate the voice statistical characteristics, and avoids the voice distortion caused by complex calculation and parameter estimation errors.)

1. A method for improving the accuracy of automatic voice recognition after voice awakening is characterized in that: the method comprises the following steps:

1) arranging N voice acquisition devices linearly at a distance d to form a voice acquisition module, wherein N is a positive integer greater than or equal to 2;

2) the method comprises the steps that collected multi-channel voice data with noise are input into a fixed beam forming module after Fourier transformation, and fixed beams in multiple directions are generated according to multiple pre-designed guide vectors;

3) inputting the multi-path fixed beam data into a wake-up module for wake-up scoring;

4) after successful awakening, locking a steering vector S (k, theta i) used by the fixed beam with the highest awakening confidence coefficient, and recording an awakening time t0, wherein k is a corresponding sub-frequency band, and theta i is an expected direction angle;

5) estimating and updating a noise covariance matrix Rn in a time period after awakening and before identification;

6) and after reconstructing the objective function by the noise covariance matrix Rn, generating a constraint condition based on the guide vector S (k, theta i), calculating a beam weight W (k), stopping updating the weight when the identification state is started, and synthesizing beam data by using the estimated weight for identification until the identification state is closed.

2. The method according to claim 1, wherein the method comprises the following steps: the fixed beam forming module in the second step comprises the step of dividing the plane space into at least 2 possible direction angles or guide vectors in advance, the specific dividing mode can be designed according to the number and the distance of the voice acquisition devices in the first step, at least one guide vector is subject to the target voice direction, the calculation of the fixed beam weight is completed in advance according to different guide vectors, and the pre-designed fixed beam is independent of the environment, so that the fixed beam forming module can be designed according to the white noise gain maximization, the directivity maximization or any other reasonable target.

3. The method according to claim 1, wherein the method comprises the following steps: the fixed beam data includes enhanced speech data obtained by beamforming filtering and post-filtering.

4. The method according to claim 1, wherein the method comprises the following steps: the fifth step includes determining a time t1 at which the recognition request occurs, where the time can be determined according to the existence probability of the speech, and the current noise covariance matrix is estimated by using the time from t0 to t1, and the updating is stopped after the time t1, or the noise covariance matrix can be directly and simply updated according to a fixed length of time (usually less than 1s) after the wake-up time t0, where the wake-up time t0 generally refers to the time at which the wake-up is triggered, and can also be adjusted to the time at which the wake-up word is spoken by integrating the wake-up score (when the wake-up score is lower than a preset threshold value).

5. The method according to claim 1, wherein the method comprises the following steps: the updating of the beam weight in the sixth step is characterized in that the fixed beam with the highest confidence coefficient in the fourth step is updated based on the statistical characteristic of the current noise, so that the updated beam can better suppress the current noise (i.e., beam side lobes), the implementation mode is based on a linear constraint minimum variance method, a target function { WH Rn W } is reconstructed according to the noise covariance matrix Rn, and the beam weight W is calculated through a guide vector design constraint condition, wherein WH is the conjugate transpose of W.

6. The method according to claim 1, wherein the method comprises the following steps: the update of the beam weight refers to an update of the noise covariance matrix Rn after wake-up and before identification, and may also be not limited thereto, such as an update of a steering vector and a design of a corresponding constraint.

7. The method according to claim 1, wherein the method comprises the following steps: the updating of the beam weight is stopped when the identification is started, the beam weight is stored, the enhanced voice data is obtained in a fixed beam forming mode and is sent to the identification, the noise covariance matrix Rn is updated at the moment different from the fixed beam described in the step four, the covariance matrix Rn used by the fixed beam in the step four is designed in advance, can be an identity matrix, and can also be specially constructed according to a sine (.) function or other functions, and meanwhile, under some special environments, the updated noise covariance matrix Rn can also be degenerated into the covariance matrix corresponding to the fixed beam in the step four.

Technical Field

The invention relates to the technical field of intelligent voice interaction, in particular to a method for improving the accuracy of automatic voice recognition after voice awakening.

Background

With the more and more intensive development of far-field intelligent voice interaction applications, the experience of a user on an intelligent voice device is directly affected by the effect of voice recognition, generally speaking, the voice recognition technology relates to awakening of the device, voice control of the device after awakening, man-machine conversation with the device and the like, and for convenience of description, the term "recognition" refers specifically to recognition of a request of the user after awakening through a cloud automatic voice recognition (ASR) system (for example, "what weather is today"), and in the recognition, an error of one word may also cause an erroneous request.

In the application of actual products, factors such as environmental noise and room reverberation can cause serious reduction of voice recognition accuracy, voice signal processing is one of core technologies in the fields of modern communication, artificial intelligence and the like, signals are collected by an acoustic sensor, namely a microphone, and target voice quality is improved through a front-end signal processing technology, so that the method is an effective method for improving the voice recognition rate, wherein a microphone array technology can enhance target voice through a beam forming mode by utilizing information collected by a plurality of microphones in space, the beam forming methods are many, and can be simply divided into fixed beam forming and self-adaptive beam forming, generally speaking, the fixed beam forming is to generate corresponding pickup beams to space through a fixed weight design mode, and the method is stable and low in computation complexity, but cannot obtain real-time room reverberation, and the like, Information such as noise statistical characteristics and the like causes that performance consistency is difficult to maintain in different environments, adaptive beam forming updates the weight in real time through estimation of the noise statistical characteristics and a transfer function, so that the performance has certain adaptability to environment changes, but an algorithm is not light enough, in contrast, the power consumption of equipment is greatly increased, and when the beam weight is updated, a voice or noise section needs to be accurately distinguished, otherwise, target voice is possibly damaged, errors such as word dropping and word changing can be caused in voice recognition seriously, and user experience is influenced.

Disclosure of Invention

Technical problem to be solved

Aiming at the defects of the prior art, the invention provides a method for improving the accuracy of automatic voice recognition after voice awakening, which has the advantages of improving the recognition rate through simple and effective array signal processing and the like, and solves the problems that the accuracy of voice recognition is seriously reduced due to factors such as environmental noise and room reverberation, target voice is possibly damaged, errors such as word dropping and word changing are seriously caused in voice recognition, and user experience is influenced.

(II) technical scheme

Another technical problem to be solved by the present invention is to provide a method for improving the accuracy of automatic speech recognition after speech awakening, which comprises the following steps:

1) arranging N voice acquisition devices linearly at a distance d to form a voice acquisition module, wherein N is a positive integer greater than or equal to 2;

2) the method comprises the steps that collected multi-channel voice data with noise are input into a fixed beam forming module after Fourier transformation, and fixed beams in multiple directions are generated according to multiple pre-designed guide vectors;

3) inputting the multi-path fixed beam data into a wake-up module for wake-up scoring;

4) after successful awakening, locking a steering vector S (k, theta i) used by the fixed beam with the highest awakening confidence coefficient, and recording an awakening time t0, wherein k is a corresponding sub-frequency band, and theta i is an expected direction angle;

5) estimating and updating a noise covariance matrix Rn in a time period after awakening and before identification;

6) and after reconstructing the objective function by the noise covariance matrix Rn, generating a constraint condition based on the guide vector S (k, theta i), calculating a beam weight W (k), stopping updating the weight when the identification state is started, and synthesizing beam data by using the estimated weight for identification until the identification state is closed.

Further, the fixed beam forming module in the second step may divide the planar space into at least 2 possible direction angles or steering vectors in advance, the specific division manner may be designed according to the number and the spacing of the voice collecting devices in the first step, based on at least one steering vector including the target voice direction, and the calculation of the fixed beam weight is completed in advance according to different steering vectors, and the pre-designed fixed beam is independent of the environment, and therefore may be designed according to the white noise gain maximization, the directivity maximization, or any other reasonable target.

Further, the fixed beam data includes enhanced voice data obtained through beam forming filtering and post-filtering.

Further, the fifth step includes determining a time t1 at which the recognition request occurs, where the time can be determined according to the existence probability of the speech, and the current noise covariance matrix is estimated by using the time from t0 to t1, and the updating is stopped after the time t1, or the noise covariance matrix can be directly and simply updated according to a fixed length of time (usually less than 1s) after the wake-up time t0, where the wake-up time t0 generally refers to the time at which the wake-up is triggered, and the wake-up score (when the wake-up score is lower than a preset threshold) can be adjusted to the time at which the wake-up word is spoken.

Further, the updating of the beam weight in the sixth step is characterized in that the fixed beam with the highest confidence coefficient in the fourth step is updated based on the statistical characteristic of the current noise, so that the updated beam has better suppression on the current noise (i.e., beam side lobes), the implementation mode is based on a linear constraint minimum variance method, a target function { WH _ Rn _ W } is reconstructed according to the noise covariance matrix Rn, and the beam weight W is calculated through a guide vector design constraint condition, wherein WH is a conjugate transpose of W.

Further, the update of the beam weight refers to an update of the noise covariance matrix Rn after wake-up and before identification, and may also be not limited thereto, such as an update of a steering vector and a design of a corresponding constraint.

Further, the updating of the beam weights stops when the identification starts, the beam weights are stored, the enhanced voice data is obtained in a fixed beam forming mode and is sent to be identified, the noise covariance matrix Rn is updated at the moment different from the fixed beam described in the fourth step, the covariance matrix Rn used by the fixed beam in the fourth step is designed in advance, can be an identity matrix, and can also be specially constructed according to a sinc (sine) function or other functions, and meanwhile, under some special environments, the updated noise covariance matrix Rn can also be degenerated into the covariance matrix corresponding to the fixed beam in the fourth step.

(III) advantageous effects

Compared with the prior art, the invention provides a method for improving the accuracy of automatic voice recognition after voice awakening, which has the following beneficial effects:

1. the method for improving the accuracy of automatic voice recognition after voice awakening utilizes the common voice interaction mode of awakening and recognition to effectively combine the two which are seemingly irrelevant by providing a simple and effective method for improving the recognition accuracy.

2. The method for improving the accuracy of automatic voice recognition after voice awakening is implemented in a fixed beam forming mode, and the noise statistics is triggered to be updated only before recognition after awakening, so that the voice statistics characteristic does not need to be estimated again, and the voice distortion caused by complex calculation and parameter estimation errors is avoided.

3. The method for improving the accuracy of automatic voice recognition after voice awakening can be used for updating the weight in real time in the recognition process and possibly seriously damaging the voice.

4. The method for improving the accuracy of automatic voice recognition after voice awakening is simple and effective, not only refers to an updating mode of a noise covariance matrix, but also includes the step of selecting a guide vector with the highest confidence coefficient by utilizing the awakening score, so that redundant calculation of the guide vector is avoided, and the beam weight can be updated by using the extracted guide vector and the noise covariance matrix.

Drawings

FIG. 1 is a flow chart of improving the accuracy of speech recognition according to the present invention;

FIG. 2 is a block diagram of a fixed beamforming module in the present invention;

fig. 3 is a diagram illustrating updating a fixed beam according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention and the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1 to 3, a method for improving automatic speech recognition after wake-up includes the following steps:

1) the method comprises the following steps that N voice acquisition devices are linearly arranged at intervals d to form a voice acquisition module, N is a positive integer larger than or equal to 2, compared with the traditional keyboard input and character input, the speed of voice input is higher, and the efficiency of voice input is at least three times of that of the traditional input mode;

2) the method comprises the steps of inputting collected multi-channel voice data with noise into a fixed beam forming module after Fourier transformation, generating fixed beams in multiple directions according to a plurality of pre-designed guide vectors, designing a specific dividing mode according to the number and the spacing of voice collecting equipment in the first step, taking the condition that at least one guide vector contains a target voice direction as the criterion, and completing the calculation of the weight of the fixed beams in advance according to different guide vectors, wherein the Fourier transformation is the basic operation in digital signal processing and is widely applied to the field of representing and analyzing discrete time domain signals, but because the operand of the Fourier transformation is in direct proportion to the square of the number N of the transformation points, when N is larger, the DFT algorithm is directly applied to perform spectrum transformation without practical application, however, the appearance of the fast fourier transform technology fundamentally changes the situation, so that the practical performance of the method is widely applied;

3) the method comprises the steps that multi-path fixed beam data are input into a wake-up module to be woken up and scored, the woken-up fixed beam data are sent, the woken-up fixed beam data comprise enhanced voice data obtained through beam forming filtering and post filtering, the beam forming has the advantages that space information can be used for space filtering (Spatialfiltering), for example, single-microphone noise reduction is taken as an example, signals received by a single microphone cannot distinguish the incoming wave direction, and stable noise is mainly suppressed in noise suppression, because voice signals are unstable, how to accurately distinguish unstable noise and voice is difficult to distinguish microphone arrays, at least two microphones are arranged in an array, the incoming wave direction can be distinguished to a certain degree, and interfering voice or other unstable noise in an unexpected direction can be linearly attenuated;

4) after successful awakening, locking a steering vector S (k, theta i) used by the fixed beam with the highest awakening confidence coefficient, and recording an awakening time t0, wherein k is a corresponding sub-frequency band, and theta i is an expected direction angle, the performance of the microphone array is mainly evaluated by white noise gain and directivity, the former is used for evaluating the suppression capability of the array in a white noise scene, the latter is used for evaluating the array gain of the array in a diffusion field noise, the essence of the array gain is array gain, and the output signal-to-noise ratio can be divided by the input signal-to-noise ratio and can also be expressed as a transfer function of a signal;

5) estimating and updating the noise covariance matrix Rn in the time period after wake-up and before recognition, wherein the step comprises determining the time t1 when the recognition request occurs, the time can be judged according to the existence probability of the voice, and the current noise covariance matrix is estimated by using the time from t0 to t1, the updating is stopped after the time t1, or the noise covariance matrix can be updated directly and simply according to a fixed length of time (usually less than 1s) after the wake-up time t0, the wake-up time t0 generally refers to the time when wake-up is triggered, or the wake-up score (below a preset threshold) can be synthesized and adjusted to the time when the wake-up word is finished, in the statistics and probability theory, each element of the covariance matrix is the covariance between vector elements, and is the natural generalization variance from scalar random variables to high-dimensional random vectors, the covariance matrix is the covariance between different dimensions, rather than between different samples, the standard deviation and variance are typically used to describe one-dimensional data, and a covariance matrix may be used to compute a dataset for multidimensional data;

6) after the noise covariance matrix Rn reconstructs an objective function, constraint conditions are generated based on a guide vector S (k, theta i), beam weight W (k) can be calculated, stopping updating the weights when the identification state is started, synthesizing beam data by using the estimated weights for identification until the identification state is closed, wherein the updating of the beam weights refers to updating of the fixed beam with the highest confidence coefficient in the fourth step based on the current noise statistical characteristics, so that the updated beam has better suppression to the current noise (i.e. beam side lobe), the implementation mode is based on a linear constraint minimum variance method, reconstructing an objective function { WH Rn W } according to the noise covariance matrix Rn, calculating the beam weight W through a guide vector design constraint condition, WH is the conjugate transpose of W, and the updated noise covariance matrix Rn may also be degenerated into the covariance matrix corresponding to the fixed beam in step four under some special circumstances.

The invention has the beneficial effects that: the invention provides a simple and effective method for improving the recognition accuracy, which utilizes a common voice interaction mode of wake-up plus recognition to effectively combine the two which are seemingly irrelevant together in a fixed beam forming mode, triggers the updating of noise statistics only before recognition after wake-up without estimating the voice statistical characteristics again, avoids the voice distortion caused by complex calculation and parameter estimation errors, and can seriously damage the voice by updating the weight in real time in the recognition process. Therefore, redundant calculation of the guide vector is avoided, and the beam weight can be updated by using the extracted guide vector and the noise covariance matrix.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.