阅读说明：本技术 信号处理装置、远程会议装置以及信号处理方法 (Signal processing apparatus, teleconference apparatus, and signal processing method ) 是由 川合窒登 金森光平 井上贵之 于 2017-06-12 设计创作，主要内容包括：信号处理装置具备第1麦克风、第2麦克风、以及信号处理部。信号处理部对所述第1麦克风的收音信号或所述第2麦克风的收音信号的至少任意一个进行回声消除处理,并且使用通过该回声消除处理消除回声后的信号来求出所述第1麦克风的收音信号以及所述第2麦克风的收音信号的相关分量。(The signal processing device includes a 1 st microphone, a 2 nd microphone, and a signal processing unit. The signal processing unit performs echo cancellation processing on at least one of the sound pickup signal of the 1 st microphone and the sound pickup signal of the 2 nd microphone, and obtains a correlation component between the sound pickup signal of the 1 st microphone and the sound pickup signal of the 2 nd microphone by using a signal from which an echo is cancelled by the echo cancellation processing.)

1. A signal processing device is provided with:

a 1 st microphone;

a 2 nd microphone; and

and a signal processing unit that performs echo cancellation processing on at least one of the sound pickup signal of the 1 st microphone and the sound pickup signal of the 2 nd microphone, and obtains a correlation component between the sound pickup signal of the 1 st microphone and the sound pickup signal of the 2 nd microphone using a signal from which an echo is cancelled by the echo cancellation processing.

2. The signal processing apparatus according to claim 1,

the signal processing unit obtains the correlation component by performing a filtering process based on an adaptive algorithm using a current input signal or the current input signal and several past input signals.

3. The signal processing apparatus of claim 1 or 2,

the signal processing section performs audio enhancement processing using the correlation component.

4. The signal processing apparatus of any one of claims 1 to 3,

the signal processing unit performs a process of removing the correlation component using the correlation component.

5. The signal processing apparatus according to claim 4,

the signal processing means performs a process of eliminating the noise component using spectral subtraction,

the signal after the cancellation processing of the correlation component is used as the noise component.

6. The signal processing apparatus according to claim 5,

the signal processing unit further performs enhancement processing of a higher harmonic component in the spectral subtraction.

7. The signal processing apparatus of claim 5 or 6,

the signal processing unit sets a different gain for each frequency or for each time in the spectral subtraction.

8. The signal processing device according to any one of claims 1 to 7, comprising:

a distance estimation unit for estimating the distance of the sound source,

the signal processing unit adjusts a gain of the sound pickup signal of the 1 st microphone or the sound pickup signal of the 2 nd microphone according to the distance estimated by the distance estimation unit.

9. The signal processing apparatus according to claim 8,

the distance estimating unit estimates the distance of the sound source based on a ratio of a signal subjected to audio enhancement processing using the correlation component to a noise component extracted by the cancellation processing of the correlation component.

10. The signal processing apparatus of any one of claims 1 to 9,

the 1 st microphone is a directional microphone,

the 2 nd microphone is a non-directional microphone.

11. The signal processing apparatus of any one of claims 1 to 10,

the signal processing unit performs the echo cancellation processing on the sound pickup signal of the 2 nd microphone.

12. A teleconference device further includes:

the signal processing apparatus of any one of claims 1 to 10; and

a loudspeaker.

13. A method for processing a signal, which comprises the steps of,

echo cancellation processing is performed on at least either one of the sound pickup signal of the 1 st microphone or the sound pickup signal of the 2 nd microphone, and the signal from which echo is cancelled by the echo cancellation processing is used to determine the correlated component of the sound pickup signal of the 1 st microphone and the sound pickup signal of the 2 nd microphone.

14. The signal processing method according to claim 13,

the correlation component is obtained by performing a filtering process based on an adaptive algorithm using the current input signal or the current input signal and several past input signals.

15. The signal processing method according to claim 13 or 14,

audio enhancement processing is performed using the correlated components.

16. The signal processing method according to any one of claims 13 to 15,

the correlation component is used to perform a cancellation process of the correlation component.

17. The signal processing method according to claim 16,

the elimination process of the noise component is performed using spectral subtraction,

the signal after the cancellation processing of the correlation component is used as the noise component.

18. The signal processing method according to claim 17,

and further performing enhancement processing of higher harmonic components in the spectral subtraction.

19. The signal processing method according to claim 16 or 17,

in the spectral subtraction, different gains are set for each frequency or each time.

20. The signal processing method according to any one of claims 13 to 19,

the distance of the sound source is estimated and,

and adjusting the gain of the sound collecting signal of the 1 st microphone or the sound collecting signal of the 2 nd microphone according to the estimated distance.

21. The signal processing method according to claim 20,

estimating the distance of the sound source based on a ratio of a signal subjected to audio enhancement processing using the correlation component to a noise component extracted by cancellation processing of the correlation component.

Technical Field

One embodiment of the present invention relates to a signal processing apparatus, a teleconference apparatus, and a signal processing method for acquiring sound of a sound source using a microphone.

Background

Patent document 1 and patent document 2 disclose a configuration for enhancing a target sound by a spectral subtraction method. The configurations of patent documents 1 and 2 extract correlated components of 2 microphone signals as target sounds. In both of the configurations of patent documents 1 and 2, noise estimation is performed by filtering processing based on an adaptive algorithm, and enhancement processing of a target sound by spectral subtraction is performed.

Disclosure of Invention

Problems to be solved by the invention

In the case of a device that acquires sound of a sound source using a microphone, there is a case where sound output from a speaker is surrounded as an echo component. Since echo components are input as the same component into 2 microphone signals, the correlation becomes very high. Therefore, there is a possibility that the echo component becomes a target sound and the echo component is enhanced.

Therefore, an object of one embodiment of the present invention is to provide a signal processing apparatus, a teleconference apparatus, and a signal processing method that can obtain a correlation component more accurately than before.

Means for solving the problems

The signal processing device includes a 1 st microphone, a 2 nd microphone, and a signal processing unit. The signal processing unit performs echo cancellation processing on at least one of the sound pickup signal of the 1 st microphone and the sound pickup signal of the 2 nd microphone, and obtains a correlation component between the sound pickup signal of the 1 st microphone and the sound pickup signal of the 2 nd microphone by using a signal from which an echo is cancelled by the echo cancellation processing.

Effects of the invention

According to one embodiment of the present invention, the correlation component can be obtained with higher accuracy than in the conventional art.

Drawings

Fig. 1 is a schematic diagram showing the configuration of a signal processing apparatus 1.

Fig. 2 is a plan view showing the directivity of the microphone 10A and the microphone 10B.

Fig. 3 is a block diagram showing the configuration of the signal processing device 1.

Fig. 4 is a block diagram showing an example of the configuration of the signal processing unit 15.

Fig. 5 is a flowchart showing the operation of the signal processing unit 15.

Fig. 6 is a block diagram showing a functional configuration of the noise estimation unit 21.

Fig. 7 is a block diagram showing a functional configuration of the noise suppression unit 23.

Fig. 8 is a block diagram showing a functional configuration of the distance estimating unit 24.

Detailed Description

Fig. 1 is a schematic diagram showing an external appearance of the configuration of the signal processing device 1. Fig. 1 shows the main structure related to sound reception and sound reproduction, and does not show other structures. The signal processing device 1 includes a cylindrical case 70, a microphone 10A, a microphone 10B, and a speaker 50. As an example, the signal processing apparatus 1 according to the present embodiment is used as a teleconference apparatus by picking up audio, outputting a picked-up signal related to the picked-up audio to another apparatus, inputting a playback signal from another apparatus, and outputting the playback signal from a speaker.

The microphones 10A and 10B are disposed at the outer peripheral positions of the case 70 on the upper surface of the case 70. The speaker 50 is provided on the upper surface of the casing 70 so that the sound reproduction direction is the upper surface direction of the casing 70. However, the shape of the case 70, the arrangement of the microphone, and the arrangement of the speaker are examples, and are not limited to these examples.

Fig. 2 is a plan view showing the directivity of the microphone 10A and the microphone 10B. As shown in fig. 2, the microphone 10A is a directional microphone having the strongest sensitivity to the front (left direction in the drawing) of the apparatus and no sensitivity to the rear (right direction in the drawing). The microphone 10B is a non-directional microphone having uniform sensitivity in all directions. However, the directivity of the microphone 10A and the microphone 10B shown in fig. 2 is an example. For example, both microphone 10A and microphone 10B may be omnidirectional microphones.

Fig. 3 is a block diagram showing the configuration of the signal processing device 1. The signal processing device 1 includes a microphone 10A, a microphone 10B, a speaker 50, a signal processing unit 15, a memory 150, and an interface (I/F) 19.

The signal processing unit 15 is constituted by a CPU or a DSP. The signal processing unit 15 performs signal processing by reading out and executing a program 151 stored in a memory 150 as a storage medium. For example, the signal processing unit 15 controls the level of the sound pickup signal Xu of the microphone 10A or the sound pickup signal Xo of the microphone 10B to output the signals to the I/F19. In the present embodiment, the description of the a/D converter and the D/a converter is omitted, and all of the various signals are digital signals unless otherwise specified.

The I/F19 transmits the signal input from the signal processing section 15 to other devices. Further, a sound reproduction signal is input from another device and is input to the signal processing unit 15. The signal processing unit 15 adjusts the level of a sound emission signal input from another device, and outputs audio from the speaker 50.

Fig. 4 is a block diagram showing a functional configuration of the signal processing unit 15. The signal processing unit 15 realizes the configuration shown in fig. 4 by the above-described program. The signal processing unit 15 includes an echo cancellation unit 20, a noise estimation unit 21, an audio enhancement unit 22, a noise suppression unit 23, a distance estimation unit 24, and a gain adjuster 25. Fig. 5 is a flowchart showing the operation of the signal processing unit 15.

The echo cancellation unit 20 receives the sound pickup signal Xo from the microphone 10B, and cancels an echo component from the received sound pickup signal Xo (S11). The echo cancellation unit 20 may cancel the echo component from the picked-up sound signal Xu of the microphone 10A, or may cancel the echo component from both the picked-up sound signal Xu of the microphone 10A and the picked-up sound signal Xo of the microphone 10B.

The echo canceller 20 receives a signal (sound emission signal) output to the speaker 50. The echo cancellation unit 20 performs echo cancellation processing by an adaptive filter. That is, the echo canceller 20 estimates a feedback component of the sound signal output from the speaker 50 and reaching the microphone 10B through the acoustic space. The echo cancellation unit 20 processes the playback signal with an FIR filter that simulates an impulse response in the acoustic space, and estimates a return component. The echo cancellation section 20 cancels the estimated feedback component from the sound reception signal Xo. The echo cancellation unit 20 updates the filter coefficients of the FIR filter using an adaptive algorithm such as LMS or RLS.

The noise estimation unit 21 receives the sound pickup signal Xu of the microphone 10A and the output signal of the echo cancellation unit 20. The noise estimation unit 21 estimates a noise component based on the sound pickup signal Xu of the microphone 10A and the output signal of the echo cancellation unit 20.

Fig. 6 is a block diagram showing a functional configuration of the noise estimation unit 21. The noise estimation unit 21 includes a filter calculation unit 211, a gain adjuster 212, and an adder 213. The filter calculation unit 211 calculates the gain W (f, k) for each frequency in the gain adjuster 212 (S12).

The noise estimation unit 21 performs fourier transform on the sound pickup signal Xo and the sound pickup signal Xu to convert the signals to signals Xo (f, k) and Xu (f, k) on the frequency axis, respectively. "f" denotes frequency and "k" denotes a frame number.

The gain adjuster 212 extracts the target sound by multiplying the sound pickup signal Xu (f, k) by the gain W (f, k) for each frequency. The gain of the gain adjuster 212 is updated by the filter calculation unit 211 using an adaptive algorithm. However, the target sound extracted by the processing of the gain adjuster 212 and the filter calculation unit 211 is only a component related to the direct sound that reaches the microphones 10A and 10B from the sound source, and the impulse response corresponding to the indirect sound component is ignored. Therefore, the filter calculation unit 211 performs update processing that takes into account only a few frames in the update processing based on the adaptive algorithm such as NLMS or RLS.

Then, as shown in the following equation, the noise estimation unit 21 subtracts the output signals W (f, k) · Xu (f, k) of the gain adjuster 212 from the picked-up sound signal Xo (f, k) in the adder 213, thereby eliminating the direct sound component from the picked-up sound signal Xo (f, k) (S13).

[ numerical formula 1]

E(f，k)＝X_o(f，k)-W(f，k)X_u(f，k)

Thus, the noise estimation unit 21 can estimate the noise component E (f, k) in which the correlation component of the direct sound is eliminated from the sound pickup signal Xo (f, k).

Next, the signal processing unit 15 performs noise canceling processing by spectral subtraction using the noise components E (f, k) estimated by the noise estimation unit 21 in the noise suppression unit 23 (S14).

Fig. 7 is a block diagram showing a functional configuration of the noise suppression unit 23. The noise suppression unit 23 includes a filter calculation unit 231 and a gain adjuster 232. As shown in equation 2 below, the noise suppression unit 23 obtains the spectral gain | Gn (f, k) | using the noise component E (f, k) estimated by the noise estimation unit 21 in order to perform noise cancellation processing by spectral subtraction.

[ numerical formula 2]

Here, β (f, k) is a coefficient that multiplies the noise component, and has a different value for each time and each frequency. β (f, k) is set as appropriate according to the usage environment of the signal processing device 1. For example, the value of β can be set so as to increase for frequencies at which the level of the noise component increases.

In the present embodiment, the signal to be subtracted by the spectral subtraction is the output signal X' o (f, k) of the audio enhancement unit 22. As shown in equation 3 below, before the noise canceling process by the noise suppression unit 23, the audio enhancement unit 22 obtains an average of the echo-cancelled signal Xo (f, k) and the output signal W (f, k) · Xu (f, k) of the gain adjuster 212 (S141).

[ numerical formula 3]

X′_o(·f，k)＝0.5×{X_o(f，k)+W(f，k)X_u(f，k)}

The output signal W (f, k) · Xu (f, k) of the gain adjuster 212 is a component correlated with Xo (f, k), and corresponds to the target sound. Therefore, the audio enhancement unit 22 obtains the average of the echo-cancelled signal Xo (f, k) and the output signal W (f, k) · Xu (f, k) of the gain adjuster 212, thereby enhancing the audio of the target sound.

The gain adjuster 232 multiplies the spectral gain | Gn (f, k) | calculated by the filter calculation unit 231 by the output signal X' o (f, k) of the audio enhancement unit 22 to obtain an output signal Yn (f, k).

As shown in equation 4 below, the filter calculation unit 231 may further calculate a spectral gain G' n (f, k) for enhancing the higher harmonic component.

[ numerical formula 4]

|G′n(f，k)|＝max{|G_n1(f，k)|，|G_n2(f，k)|，...，|G_ni(f，k)|}

Here, i is an integer. According to this equation 4, the integral multiple components (i.e., higher harmonic components) of each frequency component are enhanced. However, when the value of f/i is a decimal number, interpolation processing is performed as shown in the following expression 5.

[ numerical formula 5]

Since high-frequency components are more subtracted in the subtraction processing of noise components based on spectral subtraction, there is a possibility that sound quality deteriorates. However, in the present embodiment, since the higher harmonic components are enhanced by the above-described spectral gain G' n (f, k), deterioration of sound quality can be prevented.

As shown in fig. 4, the gain adjuster 25 receives the output signal Yn (f, k) in which the noise component is suppressed by audio enhancement, and performs gain adjustment. The gain gf (k) of the gain adjuster 25 is determined by the distance estimating unit 24.

Fig. 8 is a block diagram showing a functional configuration of the distance estimating unit 24. The distance estimation unit 24 includes a gain calculation unit 241. The gain calculation unit 241 receives the output signal E (f, k) of the noise estimation unit 21 and the output signal X' (f, k) of the audio enhancement unit 22 as input, and estimates the distance between the microphone and the sound source (S15).

As shown in equation 6 below, the gain calculation unit 241 performs noise suppression processing by spectral subtraction. However, the multiplication coefficient γ of the noise component is a fixed value, and is a value different from the coefficient β (f, k) in the noise suppression unit 23 described above.

[ numerical formula 6]

The gain calculation unit 241 further obtains an average gth (k) of the levels of all the frequency components with respect to the signal after the noise suppression processing. Mbin is the upper limit of the frequency. The average value gth (k) corresponds to the ratio of the target sound to the noise. The farther the distance between the microphone and the sound source is, the lower the value of the ratio of the target sound to the noise is, and the closer the distance between the microphone and the sound source is, the higher the value of the ratio of the target sound to the noise is. That is, the average value gth (k) corresponds to the distance between the microphone and the sound source. Thus, the gain calculation unit 241 functions as a distance estimation unit that estimates the distance of the sound source based on the ratio of the target sound (the signal subjected to the audio enhancement processing) and the noise component.

Then, the gain calculation unit 241 changes the gain gf (k) of the gain adjuster 25 in accordance with the value of the average gth (k) (S16). For example, as shown in equation 6, when the average value gth (k) exceeds the threshold, the gain gf (k) is set to the predetermined value a, and when the average value gth (k) is equal to or less than the threshold, the gain gf (k) is set to the predetermined value b (b < a). Therefore, the signal processing device 1 can enhance the sound of the sound source close to the device as the target sound without picking up the sound of the sound source far from the device.

In the present embodiment, the audio of the sound pickup signal Xo of the omnidirectional microphone 10B is enhanced and the gain is adjusted to output the audio to the I/F19, but the audio of the sound pickup signal Xu of the directional microphone 10A may be enhanced and the gain may be adjusted to output the audio to the I/F19. However, since the microphone 10B is a non-directional microphone, sound can be collected for all surrounding sounds. Therefore, it is preferable to adjust the gain of the sound pickup signal Xo of the microphone 10B and output the sound pickup signal Xo to the I/F19.

The technical idea shown in the present embodiment is summarized as follows.

1. The signal processing device includes a 1 st microphone (microphone 10A), a 2 nd microphone (microphone 10B), and a signal processing unit 15. The signal processing unit 15 (echo cancellation unit 20) performs echo cancellation processing on at least either the sound pickup signal Xu of the microphone 10A or the sound pickup signal Xo of the microphone 10B. The signal processing unit 15 (noise estimation unit 21) obtains output signals W (f, k) · Xu (f, k) which are correlation components of the picked-up sound signal of the 1 st microphone and the picked-up sound signal of the 2 nd microphone by using the signal Xo (f, k) from which the echo is canceled by the echo cancellation processing.

As in patent document 1 (japanese patent application laid-open No. 2009-049998) and patent document 2 (international publication No. 2014/024248), when echo occurs in the case of obtaining a correlation component using 2 signals, the echo component is obtained as a correlation component, and the echo component is enhanced as a target sound. However, since the signal processing device according to the present embodiment calculates the correlation component using the echo-cancelled signal, the correlation component can be calculated with higher accuracy than in the related art.

2. The signal processing unit 15 performs filtering processing based on an adaptive algorithm using the current input signal or the current input signal and several past input signals to obtain output signals W (f, k) · Xu (f, k) as correlation components.

For example, in patent document 1 (japanese patent laid-open No. 2009-049998) and patent document 2 (international publication No. 2014/024248), an adaptive algorithm is used to estimate a noise component. For an adaptive filter using an adaptive algorithm, the larger the number of taps, the more the computational load becomes. In addition, in the processing using the adaptive filter, since the reverberation component of the audio is contained, it is difficult to estimate the noise component with high accuracy.

On the other hand, in the present embodiment, the output signal W (f, k) · Xu (f, k) of the gain adjuster 212, which is a correlated component of the direct sound, is calculated by the filter calculation unit 211 through the update processing based on the adaptive algorithm, but this update processing is update processing that ignores the impulse response corresponding to the component of the indirect sound and considers only the amount of 1 frame (the current input value) as described above. Therefore, the signal processing unit 15 of the present embodiment can significantly reduce the calculation load in the process of estimating the noise component E (f, k). Further, since the update processing of the adaptive algorithm is processing in which indirect sound components are ignored and reverberation components of audio do not affect, the correlation components can be estimated with high accuracy. However, the update processing is not limited to an amount of only 1 frame (current input value). The filter calculation unit 211 may perform update processing including several past signals.

3. The signal processing section 15 (audio enhancement section 22) performs audio enhancement processing using the correlated component. The correlation component is the output signal W (f, k) · Xu (f, k) of the gain adjuster 212 in the noise estimation unit 21. The audio enhancement unit 22 obtains an average value of the echo-cancelled signal Xo (f, k) and the output signal W (f, k) · Xu (f, k) of the gain adjuster 212, thereby enhancing the audio of the target sound.

In this case, since the audio enhancement processing is performed using the correlation component calculated by the noise estimation unit 21, the audio can be enhanced with high accuracy.

4. The signal processing unit 15 (noise suppression unit 23) performs a process of canceling the correlation component using the correlation component.

5. More specifically, the noise suppression unit 23 performs noise component cancellation processing using spectral subtraction. The noise suppression unit 23 uses the signal from which the correlation component has been eliminated in the noise estimation unit 21 as the noise component.

The noise suppression unit 23 uses the high-precision noise component E (f, k) calculated by the noise estimation unit 21 as a noise component in the spectral subtraction, and therefore can suppress the noise component more precisely than before.

6. The noise suppression unit 23 further performs enhancement processing of higher harmonic components in the spectral subtraction. Thereby, the higher harmonic component is enhanced, and therefore, the deterioration of the sound quality can be prevented.

7. The noise suppression unit 23 sets different gains β (f, k) for each frequency or each time in the spectral subtraction. Thus, the coefficient by which the noise component is multiplied is set to an appropriate value corresponding to the environment.

8. The signal processing unit 15 includes a distance estimation unit 24 that estimates the distance of the sound source. The signal processing unit 15 adjusts the gain of the sound pickup signal of the 1 st microphone or the sound pickup signal of the 2 nd microphone in the gain adjuster 25 based on the distance estimated by the distance estimating unit 24. Thus, the signal processing device 1 can enhance the sound of a sound source close to the device as a target sound without picking up the sound of a sound source far from the device.

9. The distance estimation unit 24 estimates the distance of the sound source based on the ratio of the signal X' (f, k) subjected to the audio enhancement processing using the correlation component and the noise component E (f, k) extracted by the cancellation processing of the correlation component. Thus, the distance estimation unit 24 can estimate the distance with higher accuracy.

Finally, the description of the present embodiment is illustrative in all respects and should not be taken as limiting. The scope of the present invention is not shown by the above-described embodiments but by the claims. Further, the scope of the present invention includes the scope equivalent to the claims.

Description of the reference symbols

1. Signal processing device

10A, 10B. microphone

15. Signal processing section

19···I/F

20. echo cancellation section

21. noise estimating section

22. Audio enhancement section

23. noise suppressing part

24 DEG.distance estimating section

25-gain adjuster

50. loudspeaker

70. casing

150. memory

151. program

211. filter calculation section

212. gain adjuster

213. adder

231 · filter calculation section

232-gain adjuster

241 DEG.gain calculating part