阅读说明：本技术 声音信号的能量分布修正方法及其系统 (Energy distribution correction method and system for sound signal ) 是由 杜博仁 张嘉仁 曾凯盟 于 2019-02-25 设计创作，主要内容包括：本发明提供一种声音信号的能量分布修正方法及其系统,此方法适用于具有动作传感器、左扬声器及右扬声器的头戴装置,并且包括下列步骤。利用动作传感器检测头戴装置的转动角度,并且取得对应于左扬声器及右扬声器的双声道信号。将双声道信号转换成声道数量大于或等于5的多声道信号。定义左扬声器及右扬声器的四轴声源位置,以将多声道信号转换至左扬声器的四声道信号及右扬声器的四声道信号。根据转动角度以及四轴声源位置,修正左扬声器及右扬声器的四声道信号的能量分布,以分别产生对应于左扬声器及右扬声器的左输出信号及右输出信号。(The invention provides a method and a system for correcting energy distribution of a sound signal, wherein the method is suitable for a head-mounted device with a motion sensor, a left loudspeaker and a right loudspeaker and comprises the following steps. The rotation angle of the head-mounted device is detected by the motion sensor, and binaural signals corresponding to the left speaker and the right speaker are acquired. The two-channel signal is converted into a multi-channel signal having a number of channels greater than or equal to 5. Four-axis sound source positions of the left loudspeaker and the right loudspeaker are defined so as to convert the multi-channel signals into four-channel signals of the left loudspeaker and four-channel signals of the right loudspeaker. And correcting the energy distribution of the four-channel signals of the left loudspeaker and the right loudspeaker according to the rotation angle and the positions of the four-axis sound sources so as to respectively generate a left output signal and a right output signal corresponding to the left loudspeaker and the right loudspeaker.)

1. A method for correcting the energy distribution of an audio signal, which is applied to a head-mounted device having a motion sensor, a left speaker and a right speaker, includes:

detecting a rotation angle of the head mount with the motion sensor and acquiring binaural signals corresponding to the left speaker and the right speaker;

converting the two-channel signal into a multi-channel signal, wherein the number of channels of the multi-channel signal is greater than or equal to 5;

defining four-axis sound source positions of the left loudspeaker and the right loudspeaker so as to convert the multichannel signals to four-channel signals of the left loudspeaker and four-channel signals of the right loudspeaker; and

and according to the rotation angle and the four-axis sound source position, correcting the energy distribution of the four-channel signals of the left loudspeaker and the right loudspeaker so as to respectively generate a left output signal corresponding to the left loudspeaker and a right output signal corresponding to the right loudspeaker.

2. The method of claim 1, wherein the step of converting the binaural signal to the multi-channel signal further comprises:

converting the two-channel signal to an original multi-channel signal; and

and according to the characteristics of the two-channel signals, carrying out dynamic gain adjustment on each original multi-channel signal to generate the multi-channel signals.

3. The method of claim 1, wherein the step of defining the four-axis sound source positions for the left speaker and the right speaker comprises:

and setting a connecting line of a first sound source position and a third sound source position in the four-axis sound source positions to be vertical to a connecting line of a second sound source position and a fourth sound source position in the four-axis sound source positions aiming at each of the left loudspeaker and the right loudspeaker.

4. The method according to claim 1, wherein the step of converting the multi-channel signal into the four-channel signal for the left speaker and the four-channel signal for the right speaker comprises:

assigning four of the multi-channel signals to each of the four-axis sound source positions of the left speaker; and

assigning four of the multi-channel signals to each of the four-axis sound source positions of the right speaker, wherein the multi-channel signals assigned to the left speaker are not identical to the multi-channel signals assigned to the right speaker.

5. The method of claim 4, wherein the multi-channel signal is a five-channel signal comprising a left channel signal, a right channel signal, a center channel signal, a left surround signal, and a right surround signal, wherein the left channel signal, the right channel signal, the center channel signal, and the left surround signal are respectively assigned to the four-axis sound source positions of the left speaker, and the left channel signal, the right channel signal, the center channel signal, and the right surround signal are respectively assigned to the four-axis sound source positions of the right speaker.

6. The method of claim 1, wherein the step of modifying the energy distribution of the four-channel signals for the left speaker and the right speaker according to the rotation angle and the four-axis sound source position comprises:

setting a left gain curve of the four-channel signal of the left loudspeaker according to the rotation angle and the four-axis sound source position aiming at the left loudspeaker; and

and aiming at the right loudspeaker, setting a right gain curve of the four-channel signal of the right loudspeaker according to the rotation angle and the four-axis sound source position, wherein the right gain curve is different from the left gain curve.

7. The method of claim 6, wherein the left gain curve and the right gain curve each exhibit a cardioid distribution and are oriented in different directions.

8. The method of claim 6, wherein the multi-channel signal is a five-channel signal comprising a left channel signal, a right channel signal, a center channel signal, a left surround signal, and a right surround signal, wherein a gain value corresponding to the left channel signal and a gain value corresponding to the left surround signal in the left gain curve are both greater than a gain value corresponding to the center channel signal and a gain value corresponding to the right channel signal in the left gain curve, and wherein a gain value corresponding to the right channel signal and a gain value corresponding to the right surround signal in the right gain curve are both greater than a gain value corresponding to the left channel signal and a gain value corresponding to the center channel signal.

9. The method of claim 6, wherein generating the left output signal corresponding to the left speaker and the right output signal corresponding to the right speaker comprises:

synthesizing the four-channel signal of the left speaker according to the left gain curve to generate the left output signal; and

synthesizing the four-channel signal of the right speaker according to the right gain curve to generate the right output signal.

10. A system for modifying the energy distribution of an acoustic signal, comprising:

a head-mounted device including a motion sensor, a left speaker, and a right speaker;

a processing device to:

detecting a rotation angle of the headset using the motion sensor;

obtaining binaural signals corresponding to the left speaker and the right speaker;

converting the two-channel signal into a multi-channel signal, wherein the number of channels of the multi-channel signal is greater than or equal to 5;

defining four-axis sound source positions of the left loudspeaker and the right loudspeaker so as to convert the multichannel signals to four-channel signals of the left loudspeaker and four-channel signals of the right loudspeaker;

according to the rotation angle and the four-axis sound source position, correcting the energy distribution of the four-channel signals of the left loudspeaker and the right loudspeaker to respectively generate a left output signal corresponding to the left loudspeaker and a right output signal corresponding to the right loudspeaker; and

outputting the left output signal and the right output signal using the left speaker and the right speaker, respectively.

Technical Field

The invention relates to a method and a system for correcting energy distribution of a sound signal.

Background

Virtual Reality (VR) creates a real audio and video and other sensory simulated world to reproduce a real environment or an imaginary scene. The user can integrate, explore and manipulate the virtual reality environment to feel as if he or she is in the environment. However, when the screen of the general VR headset in the market rotates along with the movement of the user, the sound signals of the earphones are not changed synchronously, and the head movement of the user and the energy distribution of the sound signals cannot be well matched.

Disclosure of Invention

The invention provides a method and a system for correcting energy distribution of a sound signal, which can enable the head movement of a user to be well matched with the energy distribution of the sound signal.

In an embodiment of the present invention, the method is applied to a head-mounted device having a motion sensor, a left speaker and a right speaker, and includes the following steps. The rotation angle of the head mount is detected by the motion sensor, and binaural signals corresponding to the left speaker and the right speaker are acquired. The two-channel signal is converted to a multi-channel signal, wherein the number of channels of the multi-channel signal is greater than or equal to 5. Four-axis sound source positions of the left speaker and the right speaker are defined to convert the multi-channel signal into a four-channel signal of the left speaker and a four-channel signal of the right speaker. According to the rotation angle and the four-axis sound source position, the energy distribution of the four-channel signals of the left loudspeaker and the right loudspeaker is corrected so as to respectively generate a left output signal corresponding to the left loudspeaker and a right output signal corresponding to the right loudspeaker.

In an embodiment of the invention, the system includes a head-mounted device and a processing device. The headset includes a motion sensor, a left speaker, and a right speaker. The processing device is used for detecting the rotation angle of the head-wearing device by using the motion sensor, acquiring the two-channel signals corresponding to the left loudspeaker and the right loudspeaker, converting the two-channel signals into multi-channel signals with the number of channels being more than or equal to 5, defining four-axis sound source positions of the left loudspeaker and the right loudspeaker, converting the multi-channel signals into four-channel signals of the left loudspeaker and four-channel signals of the right loudspeaker, and correcting the energy distribution of the four-channel signals of the left loudspeaker and the right loudspeaker according to the rotation angle and the four-axis sound source positions so as to respectively generate a left output signal corresponding to the left loudspeaker and a right output signal corresponding to the right loudspeaker.

In order to make the aforementioned and other features and advantages of the invention more comprehensible, embodiments accompanied with figures are described in detail below.

Drawings

Fig. 1 is a schematic diagram of a five-channel signal of a general stereo field;

FIG. 2 is a block diagram of an energy distribution modification system for an audio signal according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating a method for modifying an energy distribution of an audio signal according to an embodiment of the present invention;

fig. 4A and 4B are schematic diagrams of four-axis sound source positions and signals of a left speaker and a right speaker according to an embodiment of the invention;

fig. 5A and 5B are schematic diagrams illustrating gain curves of a left speaker and a right speaker according to an embodiment of the invention.

Description of the reference numerals

200: system for controlling a power supply

210: head-mounted device

212: left loudspeaker

214: right loudspeaker

216: motion sensor

220: processing apparatus

S302 to S312: step (ii) of

e_L: left channel signal

e_R: right channel signal

P11～P15、P41_L～P44_L、P41_R～P44_R、P51_L～P54_L、P51_R～P54_R: location of sound source

θ_SL、θ_L、θ_C、θ_R、θ_SR: angle of rotation

θ: rotation angle

s_L: left channel signal

s_C: center channel signal

s_R: right channel signal

Left surround signal

Right surround signal

G_L: left gain curve

G_R: right gain curve

Gain value

Detailed Description

In general, a stereo field is designed to have five-channel signals at new positions by using binaural signals, and new five-channel signals are synthesized by using the inter-aural intensity difference (IID) technique according to the relative position relationship between each new channel and the old channel, and finally the five-channel signals are converted into binaural signals to be output. Taking the schematic diagram of fig. 1 according to the five-channel signal of the stereo field shown as an example, the two-channel signal e_L、e_RThe sound source positions P11, P12, P13, P14 and P15 (or the angle theta) are synthesized_SL、θ_L、θ_C、θ_R、θ_SR) Of the five-channel signal s_L、s_C、s_R、

However, this is the best setting when the user is assumed to be facing straight ahead (i.e., θ is 0), so when θ is 0, the energy distribution of the left and right channel signals will coincide with the original signal. When the user rotates backward in situWhen the signal is received (i.e., θ is 180 °), the energy distribution of the left and right channel signals is not just opposite to that of the original signal, and the magnitude values thereof are also significantly different. Therefore, the invention can dynamically correct the energy distribution of the sound signal according to the rotation angle of the head of the user, so that the head movement of the user is well matched with the energy distribution of the sound signal.

Some embodiments of the invention will be described in detail below with reference to the drawings, wherein like reference numerals refer to like or similar elements throughout the several views. These embodiments are merely exemplary of the invention and do not disclose all possible embodiments of the invention. Rather, these embodiments are merely exemplary of the methods and systems of the present invention claimed.

FIG. 2 is a block diagram of an energy distribution modification system for an audio signal according to an embodiment of the present invention. First, fig. 2 first describes all the components and the configuration of the system, and the detailed functions will be disclosed together with fig. 3.

Referring to fig. 2, the system 200 at least includes a head-mounted device 210 and a processing device 220, wherein the processing device 220 may be built in the head-mounted device 210, or be connected to the display head-mounted device 210 wirelessly, through wires, or electrically.

In detail, the head-mounted device 210 may be a head-mounted display with a left speaker 212, a right speaker 214, a motion sensor 216, or glasses, which may be implemented as a virtual reality head-mounted device, an augmented reality head-mounted device, a mixed reality head-mounted device, for example. The left speaker 212 and the right speaker 214 are used to play audio signals. The motion sensor 216 may be an accelerometer (e.g., a gravity sensor), a gyroscope (e.g., a gyroscope sensor), or any sensor that can detect linear movement, linear movement direction, and rotational movement (e.g., rotational angular velocity or angle) of the headset 210.

The processing device 220 is used for controlling the operation of the system 200 and includes a memory and a processor. The memory may be, for example, any type of fixed or removable Random Access Memory (RAM), read-only memory (ROM), flash memory (flash memory), a hard disk or other similar device, an integrated circuit, and combinations thereof. The processor may be, for example, a Central Processing Unit (CPU), an Application Processor (AP), or other programmable general purpose or special purpose microprocessor (microprocessor), Digital Signal Processor (DSP), sound processor or other similar device, integrated circuit, and combinations thereof. For example, the processor may include a central processing unit and a Sound processor, wherein the Sound processor may also include a digital signal processor and a Sound Codec (Sound Codec).

In this embodiment, the processing device 220 may be a computing device with computing capability and a processor, such as a file server, a database server, an application server, a workstation, a personal computer, etc., and the head-mounted device 210 and the processing device 220 transmit information in a wired or wireless manner through their respective communication interfaces. In another embodiment, the processing device 220 may be built into the headset 210 as a single integrated (all-in-one) system.

Fig. 3 is a flowchart illustrating a method for modifying an energy distribution of an audio signal according to an embodiment of the invention, and the method flow of fig. 3 can be implemented by the system 200 of fig. 2.

Referring to fig. 2 and 3, the processing device 220 detects the rotation angle of the head mount 210 by the motion sensor 216 of the head mount 210 (step S302), and acquires the binaural signals corresponding to the left speaker 212 and the right speaker 214 (step S304). For a fixed sound source, the user wearing the headset 210 will have the same perception of the sound signal as the user's head-up and head-down, but the left-right rotation will have an effect. Therefore, the rotation angle herein may refer to the rotation of the head mount 210 with respect to the horizontal axis, and the binaural signal may be a binaural stereo signal (stereo signal) having a left sound signal and a right sound signal, which is used in general games and audio/video.

Next, the processing device 220 converts the binaural signal into a multichannel signal (step S306). In this embodiment, the processing device 220 may convert the two-channel signal into original multi-channel signals by using a Dolby digital algorithm (Dolby digital algorithm), and then perform dynamic gain adjustment on each original multi-channel signal according to the characteristics of the two-channel signal to generate multi-channel signals. The number of channels of the multi-channel signal is greater than or equal to 5, such as a five-channel signal, a seven-channel signal, and the like. The description will be made below with respect to a five-channel signal.

The processing device 220 defines four-axis sound source positions of the left speaker 212 and the right speaker 214 to convert the multi-channel signals into four-channel signals for the left speaker 212 and four-channel signals for the right speaker 214 (step S308), thereby converting the multi-channel signals into symmetric four-axis sound sources, wherein the four-axis sound source for the left speaker 212 will be different from the four-axis sound source for the right speaker 214. That is, the processing device 220 may allocate four channel signals of the multi-channel signal to the four-axis sound source positions of the left speaker 212 and the right speaker 214, and the four channel signals allocated to the two speakers will not be identical. Taking a five-channel signal as an example, the left speaker 212 and the right speaker 214 may cancel one surround sound source each.

Specifically, fig. 4A and 4B are schematic diagrams of four-axis sound source positions and signals of the left speaker 212 and the right speaker 214, respectively, according to an embodiment of the invention. First, it is assumed that a binaural signal can be divided into a left channel signal e_LAnd a right channel signal e_RThe two-channel signal can be converted into the original five-channel signal, and then the left-channel signal e is used_LAnd a right channel signal e_RThe correlation characteristic of the left audio signal s is used for dynamic gain adjustment of each axis to generate a left audio signal s_LCenter channel signal s_CRight channel signal s_RLeft surround signal

And right surround signal

Referring to FIG. 4A, assume that the binaural signal can be divided into left channel signals e_LAnd a right channel signal e_RThe four-axis sound source position will be set at the first sound source position P41_LSecond sound source position P42_LA third sound source position P43_LAnd a fourth sound source position P44_LWherein the first sound source position P41_LAnd a third sound source position P43_LWill be connected to the second sound source position P42_LAnd a fourth sound source position P44_LAre perpendicular to each other. Viewed from another perspective, the left channel signal e corresponding to the binaural signal_LFor the left speaker 212, the first sound source position P41_LSecond sound source position P42_LA third sound source position P43_LAnd a fourth sound source position P44_LMay be positions corresponding to 0, 90, 180, and 270 degrees, respectively_L＝0°、θ_C＝90°、θ_R＝180°、θ_S270 deg., and the left channel signal s_LCenter channel signal s_CRight channel signal s_RAnd left surround signalWill be assigned to these four sound source positions, respectively. With the left speaker 212, the right surround signal will be cancelled.

Referring back to FIG. 4B, similarly, the four-axis sound source position is set at the first sound source position P41_RSecond sound source position P42_RA third sound source position P43_RAnd a fourth sound source position P44_RWherein the first sound source position P41_RAnd a third sound source position P43_RWill be connected to the second sound source position P42_RAnd a fourth sound source position P44_RAre perpendicular to each other. Viewed from another perspective, the left channel signal e corresponding to the binaural signal_RFor the left speaker 212, the first sound source position P41_RSecond sound source position P42_RA third sound source position P43_RAnd a fourth sound source position P44_RMay be corresponding to 0 degree angle, 90 degreeAngle, 180 degree angle and 270 degree angle, which can be theta_L＝0°、θ_C＝90°、θ_R＝180°、θ_S270 deg., and the left channel signal s_LCenter channel signal s_CRight channel signal s_RAnd right surround signal

Will be assigned to these four sound source positions, respectively. With the right speaker 214, the left surround signal will be cancelled.

Referring back to fig. 3, after converting the multi-channel signal into a four-channel signal, the processing device 220 modifies the energy distribution of the four-channel signal of the left speaker 212 and the right speaker 214 according to the detected rotation angle of the head-mounted device 110 and the position of the four-axis sound source (step S310) to generate a left output signal and a right output signal (step S312). In detail, the processing device 220 adaptively adjusts the energy distribution of the left speaker 212 and the right speaker 214 according to the rotation angle of the head-mounted device 110, so that the energy distribution of the sound signal can be well matched when the head of the user rotates. For the left speaker 212, the processing device 220 will set a left gain curve of the four-channel signal of the left speaker 212 according to the rotation angle and the four-axis sound source position, and for the right speaker, the processing device 220 will set a right gain curve of the four-channel signal of the right speaker according to the rotation angle and the four-axis sound source position, wherein the right gain curve is different from the left gain curve. Taking the example of converting a five-channel signal into a four-channel signal, the gain value corresponding to the left channel signal and the gain value corresponding to the left surround signal in the left gain curve are both greater than the gain value corresponding to the center channel signal and the gain value corresponding to the right surround signal in the left gain curve, and the gain value corresponding to the left channel signal and the gain value corresponding to the left surround signal in the right gain curve are both less than the gain value corresponding to the left channel signal and the gain value corresponding to the left surround signal in the right gain curve. Processing device 220 will then synthesize the four-channel signals for left speaker 212 according to the left gain curve to produce a left output signal, and synthesize the four-channel signals for right speaker 214 according to the right gain curve to produce a right output signal. The left output signal and the right output signal are output by a left speaker 212 and a right speaker 214, respectively.

In the present embodiment, the left gain curve and the right gain curve may respectively exhibit a cardioid distribution (cardioid distribution) and respectively face different directions. Specifically, fig. 5A and 5B are schematic diagrams illustrating gain curves of the left speaker 212 and the right speaker 214, respectively, according to an embodiment of the invention.

Referring to fig. 5A and 5B, assuming that the rotation angle of the head mount 210 is θ, the four-axis sound source position of the left speaker 212 is set at P51_L、P52_L、P53_L、P54_LThe four-axis sound source position for the right speaker 212 would be set at P51_R、P52_R、P53_R、P54_R. Left gain curve G_LAnd right gain curve G_RWill exhibit a cardioid distribution, given

When in use

And

if not, then,andfor the purposes of FIG. 5A, which corresponds to the left speaker channel 212 (the user's left ear), the left channel signal s_LGain value of

And left surround signalGain value of

Are all larger than the center channel signal s_CGain value ofAnd a right channel signal s_RGain value ofFor the purposes of FIG. 5B, which corresponds to the right speaker channel 214 (the user's right ear), the right channel signal s_RGain value ofAnd corresponding to the right surround signalGain value of

Are all greater than the corresponding left channel signal s_LGain value of

And to the center channel signal s_CGain value ofThereafter, the gain values will be set at respective gain valuesTo adjust the left channel signal s_LCenter channel signal s_CRight channel signal s_RLeft surround signal

And right surround signal

To generate an adjusted signalThen, the left output signal X is generated in any synthesis manner for each channel signal_LAnd a right output signal X_R。

In summary, the method and system for modifying energy distribution of audio signals provided by the present invention convert a two-channel signal into a multi-channel signal, convert the multi-channel signal into four-channel signals corresponding to a left speaker and a right speaker, and adaptively modify energy distribution of the four-channel signals according to a rotation angle of a headset. The invention can be practically applied to general VR head-wearing devices in the market, and when the screen rotates along with the movement of the user, the energy distribution of the sound signals of the earphones can be synchronously changed, so that the user can well match the image content watched in the screen and the sound heard in the screen.

Although the present invention has been described with reference to the above embodiments, it should be understood that various changes and modifications can be made therein by those skilled in the art without departing from the spirit and scope of the invention.