Wind noise processing method, device and system based on multiple microphones and storage medium
阅读说明：本技术 基于多麦克风的风噪处理方法、装置、系统及存储介质 (Wind noise processing method, device and system based on multiple microphones and storage medium ) 是由 吴晟 王文涛 边云锋 徐紫薇 于 2018-11-14 设计创作，主要内容包括：本申请提供一种基于多麦克风的风噪处理方法、装置、系统及存储介质。包括：分别从K个麦克风获取一个第一数字信号,K为大于1的整数；针对每个第一数字信号,对第一数字信号进行分离处理得到一个第一信号变换域谱和一个第二信号变换域谱；对第一信号变换域谱进行风噪修复处理,得到第三信号变换域谱；对第三信号变换域谱与第二信号变换域谱合并,得到第一变换域谱；对第一变换域谱进行重建处理,得到第二数字信号,从而一方面可以降低风噪干扰,另一方面可以防止信号失真。(The application provides a multi-microphone-based wind noise processing method, device and system and a storage medium. The method comprises the following steps: respectively acquiring a first digital signal from K microphones, wherein K is an integer greater than 1; for each first digital signal, carrying out separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum; wind noise restoration processing is carried out on the first signal transform domain spectrum to obtain a third signal transform domain spectrum; combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum; and reconstructing the first transform domain spectrum to obtain a second digital signal, so that wind noise interference can be reduced on one hand, and signal distortion can be prevented on the other hand.)
1. A multi-microphone based wind noise processing method is characterized by comprising the following steps:
respectively acquiring a first digital signal from K microphones, wherein K is an integer greater than 1;
for each first digital signal, performing separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum;
wind noise restoration processing is carried out on the first signal transform domain spectrum to obtain a third signal transform domain spectrum;
combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum;
and reconstructing the first transform domain spectrum to obtain a second digital signal.
2. The method of claim 1, wherein before performing the separation process on each first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum, further comprising:
transforming the first digital signal to obtain a second transform domain spectrum;
correspondingly, the separating the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum includes:
and carrying out spectrum separation processing on the second transform domain spectrum to obtain the first signal transform domain spectrum and the second signal transform domain spectrum.
3. The method of claim 2, wherein transforming the first digital signal to obtain a second transform domain spectrum comprises:
generating a vector with the frame length of N by the first digital signal according to an interframe interval L, wherein L is a positive number, and N is an integer greater than 1;
and performing windowed discrete Fourier transform on the vector to obtain a second transform domain spectrum corresponding to the first digital signal, wherein the second transform domain spectrum corresponding to the first digital signal comprises N/2+1 elements.
4. The method of claim 2, wherein said spectrally separating said second transform-domain spectrum to obtain said first signal transform-domain spectrum and said second signal transform-domain spectrum comprises:
the first k of the N/2+1 elements comprised by the second transform domain spectrum LThe +1 elements form a first signal transform domain spectrum corresponding to the second transform domain spectrum, and the second transform domain spectrum comprises the post k of N/2+1 elements HEach element constituting a second signal transform domain spectrum corresponding to said second transform domain spectrum, wherein k L+k H＝N/2。
5. The method of claim 4,
k is LIs determined by said N and the frequency fs of said first digital signal.
6. The method according to any of claims 2-4, wherein the first signal transform domain spectrum is a low frequency signal transform domain spectrum and the second signal transform domain spectrum is a high frequency signal transform domain spectrum.
7. The method according to any one of claims 1 to 5, wherein the performing a wind noise restoration process on the first signal transform domain spectrum to obtain a third signal transform domain spectrum comprises:
normalizing the real part and the imaginary part of the first signal transform domain spectrum to obtain a normalized real part and a normalized imaginary part of the first signal transform domain spectrum;
determining the minimum value of the modulus of the K first signal transformation domain spectrums under all the domain spectrums;
and obtaining the third signal transform domain spectrum according to the normalized real part, the normalized imaginary part and the minimum value of the modulus of the K first signal transform domain spectrums under each domain spectrum.
8. The method of claim 7, wherein determining the minimum of the modulus of the K first signal transform domain spectra under all domain spectra comprises:
determining a minimum of a sum of real and imaginary parts of the K first signal transform frequency domains at all domain spectra.
9. The method according to any of claims 1-5, wherein said combining said third signal transform domain spectrum with said second signal transform domain spectrum to obtain a first transform domain spectrum comprises:
forming the third signal transform domain spectrum into a top k in the first transform domain spectrum L+1 elements, and composing the second signal transform domain spectrum into a k-last in the first transform domain spectrum HAnd (4) each element.
10. The method according to any one of claims 1-5, wherein said reconstructing the first transform domain spectrum to obtain a second digital signal comprises:
and performing time domain signal reconstruction processing on the first transform domain spectrum to obtain the second digital signal.
11. The method of claim 10, wherein the performing a time-domain signal reconstruction process on the first transform-domain spectrum to obtain the second digital signal comprises:
repairing the first transform domain spectrum to obtain a third transform domain spectrum corresponding to the first transform domain spectrum;
performing windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum;
and accumulating the target signals in each time domain to obtain the second digital signal.
12. The method according to claim 11, wherein the performing the repair process on the first transform-domain spectrum to obtain a third transform-domain spectrum corresponding to the first transform-domain spectrum comprises:
forming first N/2 elements of the first transform-domain spectrum into first N/2 elements of the third transform-domain spectrum, and forming conjugates of last N/2 elements of the first transform-domain spectrum into last N/2 elements of the third transform-domain spectrum.
13. A multi-microphone based wind noise processing apparatus, comprising:
the acquisition module is used for respectively acquiring a first digital signal from K microphones, wherein K is an integer greater than 1;
the separation processing module is used for separating the first digital signals to obtain a first signal transform domain spectrum and a second signal transform domain spectrum aiming at each first digital signal;
the wind noise restoration processing module is used for performing wind noise restoration processing on the first signal transform domain spectrum to obtain a third signal transform domain spectrum;
a merging module, configured to merge the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum;
and the reconstruction processing module is used for reconstructing the first transform domain spectrum to obtain a second digital signal.
14. The apparatus of claim 13, wherein the separation processing module comprises:
the transformation unit is used for transforming the first digital signal to obtain a second transform domain spectrum;
and the spectrum separation unit is used for performing spectrum separation processing on the second transform domain spectrum to obtain the first signal transform domain spectrum and the second signal transform domain spectrum.
15. The apparatus according to claim 14, wherein the transformation unit is specifically configured to:
generating a vector with the frame length of N by the first digital signal according to an interframe interval L, wherein L is a positive number, and N is an integer greater than 1;
and performing windowed discrete Fourier transform on the vector to obtain a second transform domain spectrum corresponding to the first digital signal, wherein the second transform domain spectrum corresponding to the first digital signal comprises N/2+1 elements.
16. The apparatus according to claim 14, wherein the spectral separation unit is specifically configured to:
the first k of the N/2+1 elements comprised by the second transform domain spectrum LThe +1 elements form a first signal transform domain spectrum corresponding to the second transform domain spectrum, and the second transform domain spectrum comprises the post k of N/2+1 elements HEach element constituting a second signal transform domain spectrum corresponding to said second transform domain spectrum, wherein k L+k H＝N/2。
17. The apparatus of claim 16,
the kL is determined by the N and the frequency fs of the first digital signal.
18. The apparatus of any of claims 15-17, wherein the first signal transform domain spectrum is a low frequency signal transform domain spectrum and the second signal transform domain spectrum is a high frequency signal transform domain spectrum.
19. The apparatus according to any one of claims 13-17, wherein the wind noise restoration processing module comprises:
the normalization processing unit is used for performing normalization processing on the real part and the imaginary part of the first signal transform domain spectrum to obtain a normalized real part and a normalized imaginary part of the first signal transform domain spectrum;
a determining unit for determining a minimum of the modes of the K first signal transform domain spectra under all domain spectra;
and the processing unit is used for obtaining the third signal transform domain spectrum according to the normalized real part, the normalized imaginary part and the minimum value of the modulus of the K first signal transform domain spectrums under each domain spectrum.
20. The apparatus according to claim 19, wherein the determining unit is specifically configured to:
determining a minimum of a sum of real and imaginary parts of the K first signal transform frequency domains at all domain spectra.
21. The apparatus according to any one of claims 13 to 17, wherein the merging module is specifically configured to:
and forming the third signal transform domain spectrum into the first kL +1 elements in the first transform domain spectrum, and forming the second signal transform domain spectrum into the second kH elements in the first transform domain spectrum.
22. The apparatus according to any one of claims 13-17, wherein the reconstruction processing module is specifically configured to:
and performing time domain signal reconstruction processing on the first transform domain spectrum to obtain the second digital signal.
23. The apparatus of claim 22, wherein the reconstruction processing module comprises:
the restoration processing unit is used for restoring the first transform domain spectrum to obtain a third transform domain spectrum corresponding to the first transform domain spectrum;
the inverse discrete Fourier transform unit is used for performing windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum;
and the accumulation processing unit is used for carrying out accumulation processing on the target signals in each time domain to obtain the second digital signals.
24. The apparatus according to claim 23, wherein the repair processing unit is specifically configured to:
forming first N/2 elements of the first transform-domain spectrum into first N/2 elements of the third transform-domain spectrum, and forming conjugates of last N/2 elements of the first transform-domain spectrum into last N/2 elements of the third transform-domain spectrum.
25. A multi-microphone based wind noise processing apparatus, comprising: the filter comprises a processing unit and K first filters, wherein the processing unit is respectively connected with the K first filters;
the processing unit is configured to:
respectively acquiring a first digital signal from K microphones, wherein K is an integer greater than 1;
for each first digital signal, performing separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum;
wind noise restoration processing is carried out on the first signal transform domain spectrum to obtain a third signal transform domain spectrum;
combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum;
the first filter is to:
and reconstructing the first transform domain spectrum to obtain a second digital signal.
26. The apparatus of claim 25, further comprising: the processing unit is respectively connected with the K second filters;
the second filter is to: before the first digital signal is subjected to separation processing to obtain a first signal transform domain spectrum and a second signal transform domain spectrum, the first digital signal is transformed to obtain a second transform domain spectrum;
correspondingly, the processing unit is specifically configured to:
and carrying out spectrum separation processing on the second transform domain spectrum to obtain the first signal transform domain spectrum and the second signal transform domain spectrum.
27. The apparatus of claim 26, wherein the second filter is specifically configured to:
generating a vector with the frame length of N by the first digital signal according to an interframe interval L, wherein L is a positive number, and N is an integer greater than 1;
and performing windowed discrete Fourier transform on the vector to obtain a second transform domain spectrum corresponding to the first digital signal, wherein the second transform domain spectrum corresponding to the first digital signal comprises N/2+1 elements.
28. The apparatus according to claim 26, wherein the processing unit is specifically configured to: the first k of the N/2+1 elements comprised by the second transform domain spectrum LThe +1 elements form a first signal transform domain spectrum corresponding to the second transform domain spectrum, and the second transform domain spectrum comprises the post k of N/2+1 elements HEach element constituting a second signal transform domain spectrum corresponding to said second transform domain spectrum, wherein k L+k H＝N/2。
29. The apparatus of claim 28 wherein kL is determined by said N and a frequency fs of said first digital signal.
30. The apparatus of any of claims 26-29, wherein the first signal transform domain spectrum is a low frequency signal transform domain spectrum and the second signal transform domain spectrum is a high frequency signal transform domain spectrum.
31. The apparatus according to any one of claims 25 to 29, wherein the processing unit is specifically configured to:
normalizing the real part and the imaginary part of the first signal transform domain spectrum to obtain a normalized real part and a normalized imaginary part of the first signal transform domain spectrum;
determining the minimum value of the modulus of the K first signal transformation domain spectrums under all the domain spectrums;
and obtaining the third signal transform domain spectrum according to the normalized real part, the normalized imaginary part and the minimum value of the modulus of the K first signal transform domain spectrums under each domain spectrum.
32. The apparatus according to claim 31, wherein the processing unit is specifically configured to:
determining a minimum of a sum of real and imaginary parts of the K first signal transform frequency domains at all domain spectra.
33. The apparatus according to any one of claims 25 to 29, wherein the processing unit is specifically configured to:
and forming the third signal transform domain spectrum into the first kL +1 elements in the first transform domain spectrum, and forming the second signal transform domain spectrum into the second kH elements in the first transform domain spectrum.
34. The apparatus according to any of claims 25-29, wherein the first filter is specifically configured to:
and performing time domain signal reconstruction processing on the first transform domain spectrum to obtain the second digital signal.
35. The apparatus of claim 34, wherein the first filter is specifically configured to:
repairing the first transform domain spectrum to obtain a third transform domain spectrum corresponding to the first transform domain spectrum;
performing windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum;
and accumulating the target signals in each time domain to obtain the second digital signal.
36. The apparatus of claim 35, wherein the first filter is specifically configured to:
forming first N/2 elements of the first transform-domain spectrum into first N/2 elements of the third transform-domain spectrum, and forming conjugates of last N/2 elements of the first transform-domain spectrum into last N/2 elements of the third transform-domain spectrum.
37. A multi-microphone based wind noise processing system, comprising: a wind noise processing apparatus as claimed in any of claims 13 to 24 and K microphones; wherein the K microphones are connected with the wind noise processing device.
38. A multi-microphone based wind noise processing system, comprising: a wind noise processing apparatus as claimed in any of claims 25 to 36 and K microphones; wherein the K microphones are connected with the wind noise processing device.
39. A computer storage medium, comprising: computer instructions for implementing a multi-microphone based wind noise processing method according to any of claims 1-12.
Technical Field
The embodiment of the application relates to the technical field of noise reduction, in particular to a multi-microphone-based wind noise processing method, device and system and a storage medium.
Background
Wind noise is caused by air turbulence near the pick-up site of the microphone, which is converted into turbulent pressure fluctuations that are picked up by the microphone along with the sound waves. Since the fluctuations are often much larger than the sound waves, they will result in a substantial distortion of the recorded signal of the microphone. Wind noise is very common when a microphone is used outdoors for audio acquisition, has a very large influence on the quality of recording, and can greatly destroy the fidelity of the recording.
The current method for overcoming the wind noise interference in the recording is as follows: the method of physical protection is adopted to prevent the pickup part of the microphone from forming air turbulence, for example, a wind-proof sponge ball or a wind-proof hair ball is adopted to wrap the microphone, however, although the mode can effectively reduce wind noise interference, the mode causes great attenuation of high-frequency signals and distortion of the signals.
Disclosure of Invention
The embodiment of the application provides a wind noise processing method, a wind noise processing device, a wind noise processing system and a storage medium based on multiple microphones. According to the technical scheme, on one hand, wind noise interference can be reduced, and on the other hand, signal distortion can be prevented.
In a first aspect, the present application provides a method for processing wind noise based on multiple microphones, including: respectively acquiring a first digital signal from K microphones, wherein K is an integer greater than 1; for each first digital signal, carrying out separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum; wind noise restoration processing is carried out on the first signal transform domain spectrum to obtain a third signal transform domain spectrum; combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum; and reconstructing the first transform domain spectrum to obtain a second digital signal.
In a second aspect, the present application provides a multi-microphone based wind noise processing apparatus, comprising: the device comprises an acquisition module, a separation processing module, a wind noise restoration processing module, a combination module and a reconstruction processing module. The acquisition module is used for acquiring a first digital signal from K microphones respectively, wherein K is an integer greater than 1; the separation processing module is used for separating the first digital signals to obtain a first signal transform domain spectrum and a second signal transform domain spectrum aiming at each first digital signal; the wind noise restoration processing module is used for performing wind noise restoration processing on the first signal transform domain spectrum to obtain a third signal transform domain spectrum; the merging module is used for merging the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum; the reconstruction processing module is used for reconstructing the first transform domain spectrum to obtain a second digital signal.
In a third aspect, the present application provides a multi-microphone based wind noise processing apparatus, including: the filter comprises a processing unit and K first filters, wherein the processing unit is respectively connected with the K first filters; the processing unit is used for: respectively acquiring a first digital signal from K microphones, wherein K is an integer greater than 1; for each first digital signal, carrying out separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum; wind noise restoration processing is carried out on the first signal transform domain spectrum to obtain a third signal transform domain spectrum; combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum; the first filter is used for reconstructing the first transform domain spectrum to obtain a second digital signal.
In a fourth aspect, the present application provides a multi-microphone based wind noise processing system, comprising: the wind noise processing apparatus according to the second aspect and K microphones; wherein, K microphones are connected with the wind noise processing device.
In a fifth aspect, the present application provides a multi-microphone based wind noise processing system, comprising: a wind noise processing apparatus according to the third aspect and K microphones; wherein, K microphones are connected with the wind noise processing device.
In a sixth aspect, the present application provides a computer storage medium comprising: computer instructions for implementing the multi-microphone based wind noise processing method described above.
In a seventh aspect, the present application provides a computer program product comprising: computer instructions for implementing the multi-microphone based wind noise processing method described above.
The application provides a multi-microphone-based wind noise processing method, device and system and a storage medium. The method comprises the following steps: respectively acquiring a first digital signal from K microphones, wherein K is an integer greater than 1; for each first digital signal, carrying out separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum; wind noise restoration processing is carried out on the first signal transform domain spectrum to obtain a third signal transform domain spectrum; combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum; and reconstructing the first transform domain spectrum to obtain a second digital signal. According to the technical scheme, on one hand, wind noise interference can be reduced, and on the other hand, signal distortion can be prevented.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
Fig. 1 is an application scenario diagram of the present application;
fig. 2 is a flowchart of a multi-microphone based wind noise processing method according to an embodiment of the present disclosure;
fig. 3 is a flowchart of a multi-microphone based wind noise processing method according to another embodiment of the present application;
fig. 4 is a flowchart of a multi-microphone based wind noise processing method according to still another embodiment of the present application;
fig. 5 is a flowchart of a multi-microphone based wind noise processing method according to another embodiment of the present application;
fig. 6 is a flowchart of a multi-microphone based wind noise processing method according to still another embodiment of the present application;
fig. 7 is a signal waveform diagram of a dual microphone provided by an embodiment of the present application under wind noise interference;
fig. 8 is a signal waveform diagram of a dual microphone provided in an embodiment of the present application after being processed by wind noise;
fig. 9 is a schematic diagram of a multi-microphone based wind noise processing apparatus 90 according to an embodiment of the present application;
fig. 10 is a schematic diagram of a multi-microphone based wind
fig. 11 is a schematic diagram of a multi-microphone based wind
fig. 12 is a schematic diagram of a multi-microphone based wind
fig. 13 is a schematic diagram of a multi-microphone based wind noise processing apparatus 130 according to an embodiment of the present application;
fig. 14 is a schematic diagram of a multi-microphone based wind
fig. 15 is a schematic diagram of a multi-microphone based wind
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As mentioned above, the current method of overcoming the wind noise interference in sound recordings is: the method of physical protection is adopted to prevent the pickup part of the microphone from forming air turbulence, for example, a wind-proof sponge ball or a wind-proof hair ball is adopted to wrap the microphone, however, although the mode can effectively reduce wind noise interference, the mode causes great attenuation of high-frequency signals and distortion of the signals. In order to solve the technical problem, the application provides a wind noise processing method, a wind noise processing device, a wind noise processing system and a storage medium based on multiple microphones.
Fig. 1 is a view of an application scenario of the present application, and as shown in fig. 1, a wind
Fig. 2 is a flowchart of a multi-microphone-based wind noise processing method according to an embodiment of the present disclosure, where an execution main body of the method is a wind noise processing apparatus, and the wind noise processing apparatus may be part or all of a smart device such as a computer, a tablet computer, and a mobile phone. As shown in fig. 2, the method comprises the steps of:
step S21: the wind noise processing device respectively obtains a first digital signal from K microphones, wherein K is an integer larger than 1.
Step S22: the wind noise processing device is used for separating the first digital signals to obtain a first signal transform domain spectrum and a second signal transform domain spectrum aiming at each first digital signal.
Step S23: and the wind noise processing device carries out wind noise restoration processing on the first signal transform domain spectrum to obtain a third signal transform domain spectrum.
Step S24: and the wind noise processing device combines the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum.
Step S25: and the wind noise processing device carries out reconstruction processing on the first transform domain spectrum to obtain a second digital signal.
Step S21 is explained below:
optionally, the wind noise processing device may perform signal acquisition on the K microphones to obtain K first digital signals, which are respectively denoted as x 1(t)，x 2(t)……x K(t), t represents time. The K first digital signals may be the same or different, and this is not limited in this embodiment of the application. The sampling frequencies of the K first digital signals are the same, and the sampling frequency is denoted as fs.
Step S22 is explained below:
optionally, fig. 3 is a flowchart of a method for processing wind noise based on multiple microphones according to another embodiment of the present application, and as shown in fig. 3, before step S22, the method further includes:
step S31: and the wind noise processing device transforms the first digital signal to obtain a second transform domain spectrum.
Accordingly, step S22 includes:
step S32: and the wind noise processing device performs spectrum separation processing on the second transform domain spectrum to obtain a first signal transform domain spectrum and a second signal transform domain spectrum.
The transformation performed by the wind noise processing device on the first digital signal may be Discrete Fourier Transform (DFT), Discrete cosine Transform (DCT for DCT cosine Transform), short-time Fourier Transform, and the like, which is not limited in this application.
For example: the first digital signal is transformed by a following transformation mode to obtain a second transform domain spectrum, and the wind noise processing device generates a vector with the frame length of N according to an inter-frame interval L, wherein L is a positive number, and N is an integer larger than 1. And performing windowed discrete Fourier transform on the vector to obtain a second transform domain spectrum corresponding to the first digital signal, wherein the second transform corresponding to the first digital signalThe domain spectrum includes N/2+1 elements. Specifically, the wind noise processing device converts any one of the first digital signals x c(t), c is 1,2, …, K, forming a vector with the frame length N according to the inter-frame interval L, and performing windowing discrete Fourier transform on the vector to obtain a second transform domain spectrum X with N/2+1 elements c[k] n：
Wherein j is an imaginary unit, and N has a value in the range of int [0.005fs]<＝N<＝int[fs]E.g. N-int [0.032fs]，int[]Is an integer fetch operation. Wherein the value range of L is int [0.005fs ]]<＝L<＝int[fs]And L is<N, typically L ═ int [0.016fs]. Wherein h is ana[l]1,2, …, N is an N-point analysis window function, h syn[l]1,2, …, N being N-point synthesis window functions, which satisfy the following conditions:
N＝L z+2L t+L O，L＝L t+L Owherein L is zAnd L OIs a non-negative integer, L tIs an integer other than 0.
h ana[l]＝h syn[l]＝0，l＝1,…,L zIf L is zIs not 0.
h ana[l]＝h syn[l]＝1，l＝L z+L t+1，…,L z+L t+L OIf L is zIs not 0.
h ana[L z+l]＝h ana[N+1-l]，l＝1,2,…,L t。
h syn[L z+l]＝h syn[N+1-l],l＝1,2,…,L t。
h ana[L z+l]h syn[L z+l]+h ana[L z+L t+L O+l]h syn[L z+L t+L O+l]＝1，
l＝1,2,…,L t。
Further, the wind noise processing device performs spectrum separation processing on the second transform domain spectrum to obtain a first signal transform domain spectrum and a second signal transform domain spectrum.
Alternatively, the wind noise processing means may include the first k of the N/2+1 elements of the second transform domain spectrum LThe +1 elements form a first signal transform domain spectrum corresponding to the second transform domain spectrum, and the second transform domain spectrum comprises the post k of N/2+1 elements HThe elements form a second signal transform domain spectrum corresponding to the second transform domain spectrum, where k L+k HN/2. Wherein k is LIs determined by N and the frequency fs of the first digital signal. For example:
Specifically, the spectrum separation operation is carried out on the K second transform domain spectrums to obtain K high-frequency transform domain spectrums X H1[k] n,…,X HK[k] n,k＝1,2,…,k HAnd K low frequency transform domain spectra X L1[k] n,…,X LK[k] n,k＝1,2,…,k L+1, the operation of obtaining the low frequency transform domain spectrum is:
X Lc[k] n＝X c[k] n，k＝1,2,…,k L+1，c＝1,2,…,K。
the operation of obtaining the high frequency transform domain spectrum is:
X Hc[k] n＝X c[k L+1+k] n，k＝1,2,…,k H，c＝1,2,…,K。
another alternative is: the wind noise processing device may combine odd-numbered elements of the N/2+1 elements included in the second transform domain spectrum into a first signal transform domain spectrum corresponding to the second transform domain spectrum, and combine even-numbered elements of the N/2+1 elements included in the second transform domain spectrum into a second signal transform domain spectrum corresponding to the second transform domain spectrumA transposed spectrum, where k L+k H＝N/2。
It should be noted that: how to perform spectrum separation processing on the second transform domain spectrum to obtain the first signal transform domain spectrum and the second signal transform domain spectrum is not limited to the two options.
Step S23 is explained below:
optionally, fig. 4 is a flowchart of a method for processing wind noise based on multiple microphones according to still another embodiment of the present application, and as shown in fig. 4, step S23 includes the following steps:
step S41: the wind noise processing device normalizes the real part and the imaginary part of the first signal transform domain spectrum to obtain a normalized real part and a normalized imaginary part of the first signal transform domain spectrum.
Step S42: the wind noise processing means determines the minimum of the modes of the K first signal transform domain spectra under all domain spectra.
Step S43: and the wind noise processing device obtains a third signal transform domain spectrum according to the normalized real part, the normalized imaginary part and the minimum value of the modulus of the K first signal transform domain spectrums under each domain spectrum.
Specifically, a normalized real part X corresponding to the first signal transform domain spectrum is obtained Rc[k] nAnd normalizing the imaginary part X Ic[k] nIt satisfies the following conditions:
k＝1,2,…,k L+1，c＝1,2,…,K
real () is an operation taking the real part of a complex number, imag () is an operation taking the imaginary part of a complex number, and abs () is an operation taking the absolute value. Secondly, the wind noise processing device respectively calculates the minimum absolute value of the real part and the minimum absolute value of the imaginary part of each spectrum sequence number K of the K first signal transform domain spectrums:
R L[k] n＝min{abs[real(X L1[k] n),…,abs[real(X LK[k] n)}
I[k] n＝min{abs[imag(X L1[k] n),…,abs[imag(X LK[k] n)}
k＝1,2,…,k L+1, where min () is the minimum operation.
And finally, acquiring K third signal transform domain spectrums by the wind noise processing device:
step S24 is explained below:
alternatively, fig. 5 is a flowchart of a method for processing wind noise based on multiple microphones according to another embodiment of the present application, and as shown in fig. 5, step S24 includes the following steps:
step S51: the wind noise processing device forms the third signal transform domain spectrum into the front k in the first transform domain spectrum L+1 elements, and composing the second signal transform domain spectrum into the last k in the first transform domain spectrum HAnd (4) each element.
Specifically, a first transform domain spectrum X' c[k] nThe method comprises the following specific steps:
X' c[k] n＝X' LC[k] n，k＝1,2,…,k L+1，c＝1,2,…,K
X' c[k L+1+k] n＝X Hc[k] n，k＝1,2,…,k H，c＝1,2,…,K。
another alternative is: the wind noise processing means may combine elements of the third signal transform domain spectrum into elements of odd bits in the first transform domain spectrum and elements of the second signal transform domain spectrum into elements of even bits in the first transform domain spectrum.
The wind noise processing device may be configured to perform a combination processing of the third signal transform domain spectrum and the second signal transform domain spectrum according to a spectrum separation processing performed on the second transform domain spectrum, for example: if the spectrum separation processing method performed by the wind noise processing apparatus on the second transform domain spectrum adopts the first optional method, the spectrum combination processing method performed on the third signal transform domain spectrum and the second signal transform domain also adopts the first optional method in the spectrum combination processing. If the second optional manner is adopted by the spectrum separation processing method of the second transform domain spectrum performed by the wind noise processing device, the second optional manner is also adopted by the spectrum combination processing method of the third signal transform domain spectrum and the second signal transform domain.
Step S25 is explained below:
step S25 includes: and the wind noise processing device carries out time domain signal reconstruction processing on the first transform domain spectrum to obtain a second digital signal.
Optionally, fig. 6 is a flowchart of a wind noise processing method based on multiple microphones according to yet another embodiment of the present application, and as shown in fig. 6, the wind noise processing apparatus performs time domain signal reconstruction processing on the first transform domain spectrum to obtain a second digital signal includes the following steps:
step S61: and the wind noise processing device carries out restoration processing on the first transform domain spectrum to obtain a third transform domain spectrum corresponding to the first transform domain spectrum.
Step S62: and the wind noise processing device performs windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum.
Step S63: and the wind noise processing device performs accumulation processing on the target signals in each time domain to obtain a second digital signal.
The wind noise processing device combines the first N/2 elements in the first transform domain spectrum into the first N/2 elements of the third transform domain spectrum, and combines the conjugate of the last N/2 elements in the first transform domain spectrum into the last N/2 elements of the third transform domain spectrum.
In particular, for K first transform domain spectra
where denotes a conjugate operation.
Further, the wind noise processing device performs windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum. The specific process is as follows:
further, the wind noise processing device pair d c[l] nPerforming overlap accumulation operation to obtain repaired L-point time domain audio signal
Wherein z is c[l] nThe initial value of the buffer is zero, and the buffer needs to be updated after each overlap accumulation: z is a radical of c[l] n＝d c[L z+L t+L O+l] n,l＝1,2,…,L t。
The embodiment of the application provides a wind noise processing method based on multiple microphones, which comprises the following steps: the wind noise processing device respectively acquires a first digital signal from K microphones, and separates the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum aiming at each first digital signal. And carrying out wind noise restoration processing on the first signal transform domain spectrum to obtain a third signal transform domain spectrum. And combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum. And reconstructing the first transform domain spectrum to obtain a second digital signal. The wind noise processing method provided by the application can reduce wind noise interference and simultaneously can not cause the signal distortion problem.
The following explains the effect of the wind noise processing method based on a dual-microphone audio acquisition system: the sampling frequency fs of the dual-microphone audio acquisition system is 48000Hz, N is 2048, and L is 1024, and the analysis window function and the synthesis window function are respectively as follows:
fig. 7 is a waveform diagram of signals of a dual microphone in a case of wind noise interference, as shown in fig. 7, two paths of digital audio signals (i.e., the first digital signal) obtained by a dual-microphone audio acquisition system are subjected to severe wind noise interference, and digital overload occurs to the signals due to excessive interference in some time periods. Fig. 8 is a waveform diagram of signals of a dual microphone after being subjected to wind noise processing according to an embodiment of the present application, and as shown in fig. 8, after being subjected to wind noise processing, the amplitude of a repaired signal (i.e., the second digital signal) becomes very flat, and there is no overload distortion.
Fig. 9 is a schematic diagram of a multi-microphone based wind noise processing apparatus 90 according to an embodiment of the present disclosure, where the wind noise processing apparatus may be part or all of a smart device such as a computer, a tablet computer, or a mobile phone. As shown in fig. 9, the wind noise processing apparatus includes:
the acquiring
The
And the wind noise
A merging
And a
Optionally, fig. 10 is a schematic diagram of a multi-microphone based wind
and a transforming
The
Optionally, the
Optionally, the
Alternatively, k LIs determined by N and the frequency fs of the first digital signal.
Optionally, the first signal transform domain spectrum is a low frequency signal transform domain spectrum and the second signal transform domain spectrum is a high frequency signal transform domain spectrum.
Optionally, fig. 11 is a schematic diagram of a multi-microphone based wind
the
A determining
The
Optionally, the determining
Optionally, the merging
Optionally, the
Optionally, fig. 12 is a schematic diagram of a multi-microphone based wind
a
An inverse discrete
And the
Optionally, the
The wind noise processing apparatus provided in the present application may be used to execute the wind noise processing method, and the content and effect of the wind noise processing apparatus may refer to the method section, which is not described herein again.
Fig. 13 is a schematic diagram of a multi-microphone based wind noise processing apparatus 130 according to an embodiment of the present application, and as shown in fig. 13, the wind noise processing apparatus 130 includes: the processing unit 131 and the K first filters 132, wherein the processing unit 131 is connected to the K first filters 132 respectively.
The processing unit 131 is configured to: a first digital signal is respectively obtained from K microphones, and K is an integer larger than 1. And for each first digital signal, carrying out separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum. And carrying out wind noise restoration processing on the first signal transform domain spectrum to obtain a third signal transform domain spectrum. And combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum.
The first filter 132 is configured to: and reconstructing the first transform domain spectrum to obtain a second digital signal.
Optionally, the wind noise processing apparatus 130 further includes: k second filters 133, wherein the processing unit 131 is connected to the K second filters 133, respectively. The second filter 133 is configured to: before the first digital signal is separated to obtain a first signal transform domain spectrum and a second signal transform domain spectrum, the first digital signal is transformed to obtain a second transform domain spectrum. Correspondingly, the processing unit 131 is specifically configured to: and carrying out spectrum separation processing on the second transform domain spectrum to obtain a first signal transform domain spectrum and a second signal transform domain spectrum.
Optionally, the second filter 133 is specifically configured to: and generating a vector with the frame length of N by the first digital signal according to the inter-frame interval L, wherein L is a positive number, and N is an integer greater than 1. And performing windowed discrete Fourier transform on the vector to obtain a second transform domain spectrum corresponding to the first digital signal, wherein the second transform domain spectrum corresponding to the first digital signal comprises N/2+1 elements.
Optionally, the processing unit 131 is specifically configured to: the first k of the N/2+1 elements comprising the second transform domain spectrum LThe +1 elements form a first signal transform domain spectrum corresponding to the second transform domain spectrum, and the second transform domain spectrum comprises the post k of N/2+1 elements HThe elements form a second signal transform domain spectrum corresponding to the second transform domain spectrum, where k L+k H＝N/2。
Alternatively, k LIs determined by N and the frequency fs of the first digital signal.
Optionally, the first signal transform domain spectrum is a low frequency signal transform domain spectrum and the second signal transform domain spectrum is a high frequency signal transform domain spectrum.
Optionally, the processing unit 131 is specifically configured to: and normalizing the real part and the imaginary part of the first signal transform domain spectrum to obtain a normalized real part and a normalized imaginary part of the first signal transform domain spectrum. The minimum of the modes of the K first signal transform domain spectra under all domain spectra is determined. And obtaining a third signal transform domain spectrum according to the normalized real part, the normalized imaginary part and the minimum value of the modulus of the K first signal transform domain spectrums under each domain spectrum.
Optionally, the processing unit 131 is specifically configured to: the minimum of the sum of the real and imaginary parts of the K first signal transform frequency domains at all domain spectra is determined.
Optionally, the processing unit 131 is specifically configured to: forming the third signal transform domain spectrum into the first k in the first transform domain spectrum L+1 elements, and composing the second signal transform domain spectrum into the last k in the first transform domain spectrum HAnd (4) each element.
Optionally, the first filter 132 is specifically configured to: and performing time domain signal reconstruction processing on the first transform domain spectrum to obtain a second digital signal.
Optionally, the first filter 132 is specifically configured to: and repairing the first transform domain spectrum to obtain a third transform domain spectrum corresponding to the first transform domain spectrum. And carrying out windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum. And accumulating the target signals in each time domain to obtain a second digital signal.
Optionally, the first filter 132 is specifically configured to: the first N/2 elements of the first transform-domain spectrum are combined into the first N/2 elements of the third transform-domain spectrum, and the conjugate of the last N/2 elements of the first transform-domain spectrum are combined into the last N/2 elements of the third transform-domain spectrum.
The wind noise processing apparatus provided in the present application may be used to execute the wind noise processing method, and the content and effect of the wind noise processing apparatus may refer to the method section, which is not described herein again.
Fig. 14 is a schematic diagram of a multi-microphone based wind
Wherein, this wind processing apparatus of making an uproar includes:
the acquisition module is used for acquiring a first digital signal from K microphones respectively, wherein K is an integer larger than 1.
And the separation processing module is used for separating the first digital signals to obtain a first signal transform domain spectrum and a second signal transform domain spectrum aiming at each first digital signal.
And the wind noise restoration processing module is used for performing wind noise restoration processing on the first signal transform domain spectrum to obtain a third signal transform domain spectrum.
And the merging module is used for merging the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum.
And the reconstruction processing module is used for reconstructing the first transform domain spectrum to obtain a second digital signal.
Optionally, the separation processing module comprises:
and the transformation unit is used for transforming the first digital signal to obtain a second transform domain spectrum.
And the spectrum separation unit is used for performing spectrum separation processing on the second transform domain spectrum to obtain a first signal transform domain spectrum and a second signal transform domain spectrum.
Optionally, the transformation unit is specifically configured to: and generating a vector with the frame length of N by the first digital signal according to the inter-frame interval L, wherein L is a positive number, and N is an integer greater than 1. And performing windowed discrete Fourier transform on the vector to obtain a second transform domain spectrum corresponding to the first digital signal, wherein the second transform domain spectrum corresponding to the first digital signal comprises N/2+1 elements.
Optionally, the spectral separation unit is specifically configured to: the first k of the N/2+1 elements comprising the second transform domain spectrum LThe +1 elements form a first signal transform domain spectrum corresponding to the second transform domain spectrum, and the second transform domain spectrum comprises the post k of N/2+1 elements HThe elements form a second signal transform domain spectrum corresponding to the second transform domain spectrum, where k L+k H＝N/2。
Alternatively, k LIs determined by N and the frequency fs of the first digital signal.
Optionally, the first signal transform domain spectrum is a low frequency signal transform domain spectrum and the second signal transform domain spectrum is a high frequency signal transform domain spectrum.
Optionally, the wind noise restoration processing module includes:
and the normalization processing unit is used for performing normalization processing on the real part and the imaginary part of the first signal transform domain spectrum to obtain a normalized real part and a normalized imaginary part of the first signal transform domain spectrum.
A determining unit for determining a minimum of the modes of the K first signal transform domain spectra under all domain spectra.
And the processing unit is used for obtaining a third signal transform domain spectrum according to the normalized real part, the normalized imaginary part and the minimum value of the modulus of the K first signal transform domain spectrums under each domain spectrum.
Optionally, the determining unit is specifically configured to: the minimum of the sum of the real and imaginary parts of the K first signal transform frequency domains at all domain spectra is determined.
Optionally, the merging module is specifically configured to: forming the third signal transform domain spectrum into the first k in the first transform domain spectrum L+1 elements, and composing the second signal transform domain spectrum into the last k in the first transform domain spectrum HAnd (4) each element.
Optionally, the reconstruction processing module is specifically configured to: and performing time domain signal reconstruction processing on the first transform domain spectrum to obtain a second digital signal.
Optionally, the reconstruction processing module includes:
and the restoration processing unit is used for restoring the first transform domain spectrum to obtain a third transform domain spectrum corresponding to the first transform domain spectrum.
And the inverse discrete Fourier transform unit is used for performing windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum.
And the accumulation processing unit is used for carrying out accumulation processing on the target signals in each time domain to obtain second digital signals.
Optionally, the repair processing unit is specifically configured to: the first N/2 elements of the first transform-domain spectrum are combined into the first N/2 elements of the third transform-domain spectrum, and the conjugate of the last N/2 elements of the first transform-domain spectrum are combined into the last N/2 elements of the third transform-domain spectrum.
The wind noise processing system provided by the present application includes a wind noise processing apparatus, which can be used to execute the wind noise processing method, and the content and effect of the wind noise processing apparatus can be referred to in the method section, which is not described again in the present application.
Fig. 15 is a schematic diagram of a multi-microphone based wind
Wherein, this wind processing apparatus of making an uproar includes:
the filter comprises a processing unit and K first filters, wherein the processing unit is respectively connected with the K first filters.
The processing unit is used for: a first digital signal is respectively obtained from K microphones, and K is an integer larger than 1. And for each first digital signal, carrying out separation processing on the first digital signal to obtain a first signal transform domain spectrum and a second signal transform domain spectrum. And carrying out wind noise restoration processing on the first signal transform domain spectrum to obtain a third signal transform domain spectrum. And combining the third signal transform domain spectrum and the second signal transform domain spectrum to obtain a first transform domain spectrum.
The first filter is for: and reconstructing the first transform domain spectrum to obtain a second digital signal.
Optionally, the wind noise processing apparatus further includes: and the processing unit is respectively connected with the K second filters. The second filter is for: before the first digital signal is separated to obtain a first signal transform domain spectrum and a second signal transform domain spectrum, the first digital signal is transformed to obtain a second transform domain spectrum. Correspondingly, the processing unit is specifically configured to: and carrying out spectrum separation processing on the second transform domain spectrum to obtain a first signal transform domain spectrum and a second signal transform domain spectrum.
Optionally, the second filter is specifically configured to: and generating a vector with the frame length of N by the first digital signal according to the inter-frame interval L, wherein L is a positive number, and N is an integer greater than 1. And performing windowed discrete Fourier transform on the vector to obtain a second transform domain spectrum corresponding to the first digital signal, wherein the second transform domain spectrum corresponding to the first digital signal comprises N/2+1 elements.
Optionally, the processing unit is specifically configured to: the first k of the N/2+1 elements comprising the second transform domain spectrum LThe +1 elements form a first signal transform domain spectrum corresponding to the second transform domain spectrum, and the second transform domain spectrum comprises the post k of N/2+1 elements HThe elements form a second signal transform domain spectrum corresponding to the second transform domain spectrum, where k L+k H＝N/2。
Alternatively, k LIs determined by N and the frequency fs of the first digital signal.
Optionally, the first signal transform domain spectrum is a low frequency signal transform domain spectrum and the second signal transform domain spectrum is a high frequency signal transform domain spectrum.
Optionally, the processing unit is specifically configured to: and normalizing the real part and the imaginary part of the first signal transform domain spectrum to obtain a normalized real part and a normalized imaginary part of the first signal transform domain spectrum. The minimum of the modes of the K first signal transform domain spectra under all domain spectra is determined. And obtaining a third signal transform domain spectrum according to the normalized real part, the normalized imaginary part and the minimum value of the modulus of the K first signal transform domain spectrums under each domain spectrum.
Optionally, the processing unit is specifically configured to: the minimum of the sum of the real and imaginary parts of the K first signal transform frequency domains at all domain spectra is determined.
Optionally, the processing unit is specifically configured to: forming the third signal transform domain spectrum into the first k in the first transform domain spectrum L+1 elements, and composing the second signal transform domain spectrum into the last k in the first transform domain spectrum HAnd (4) each element.
Optionally, the first filter is specifically configured to: and performing time domain signal reconstruction processing on the first transform domain spectrum to obtain a second digital signal.
Optionally, the first filter is specifically configured to: and repairing the first transform domain spectrum to obtain a third transform domain spectrum corresponding to the first transform domain spectrum. And carrying out windowing inverse discrete Fourier transform on the third transform domain spectrum to obtain a target signal on a time domain corresponding to the first transform domain spectrum. And accumulating the target signals in each time domain to obtain a second digital signal.
Optionally, the first filter is specifically configured to: the first N/2 elements of the first transform-domain spectrum are combined into the first N/2 elements of the third transform-domain spectrum, and the conjugate of the last N/2 elements of the first transform-domain spectrum are combined into the last N/2 elements of the third transform-domain spectrum.
The wind noise processing system provided by the present application includes a wind noise processing apparatus, which can be used to execute the wind noise processing method, and the content and effect of the wind noise processing apparatus can be referred to in the method section, which is not described again in the present application.
The Processor according to the present Application may be a Motor controller MCU (MCU), a Central Processing Unit (CPU), or other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.
The present application further provides a computer storage medium comprising: the contents and effects of the computer instructions for implementing the multi-microphone based wind noise processing method as described above can be referred to the method part, and will not be described herein.
The present application further provides a computer program product comprising: the contents and effects of the computer instructions for implementing the multi-microphone based wind noise processing method as described above can be referred to the method part, and will not be described herein.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.