阅读说明：本技术 360度音源实时回放系统 (360-degree sound source real-time playback system ) 是由 胡剑凌 唐银环 李瑶天 曹洪龙 于 2019-09-12 设计创作，主要内容包括：本发明公开了一种360度音源实时回放系统。本发明一种360度音源实时回放系统,包括：主体由一音频数据头盔(简称头盔)构成,音频数据头盔由头盔框架、麦克风阵列、耳机和处理系统构成,其中耳机根据人耳位置安置,360度音源实时回放处理硬件模块主要实现头部位置感知、音源定位以及基于HRTF的音频重建等处理功能,采用柔性制造工艺在头盔框架内部沿其内轮廓装配；麦克风阵列用于音源定位功能的音频数据采集功能,安置于头盔上。本发明的有益效果：360度音源实时回放系统以置于头盔上的麦克风阵列的TDOA算法对声源定位,结合置于头盔上的倾角传感器感知到的头部运动角度计算HRTF值实现360度音源实时回放。(The invention discloses a real-time playback system of a 360-degree sound source. The invention discloses a real-time playback system of a 360-degree sound source, which comprises: the main body is composed of an audio data helmet (helmet for short), the audio data helmet is composed of a helmet frame, a microphone array, earphones and a processing system, wherein the earphones are arranged according to the positions of human ears, a 360-degree sound source real-time playback processing hardware module mainly realizes processing functions of head position perception, sound source positioning, HRTF-based audio reconstruction and the like, and the audio data helmet is assembled inside the helmet frame along the inner contour thereof by adopting a flexible manufacturing process; the microphone array is used for the audio data acquisition function of the sound source positioning function and is arranged on the helmet. The invention has the beneficial effects that: the real-time playback system of the 360-degree sound source positions the sound source through a TDOA algorithm of a microphone array arranged on a helmet, and calculates an HRTF value by combining a head movement angle sensed by a tilt sensor arranged on the helmet, so that the real-time playback of the 360-degree sound source is realized.)

1. A real-time playback system for a 360-degree audio source, comprising: the main body is composed of an audio data helmet (helmet for short), the audio data helmet is composed of a helmet frame, a microphone array, earphones and a processing system, wherein the earphones are arranged according to the positions of human ears, a 360-degree sound source real-time playback processing hardware module mainly realizes processing functions of head position perception, sound source positioning, HRTF-based audio reconstruction and the like, and the audio data helmet is assembled inside the helmet frame along the inner contour thereof by adopting a flexible manufacturing process; the microphone array is used for the audio data acquisition function of the sound source positioning function and is arranged on the helmet;

the 360-degree sound source real-time playback processing hardware module comprises an HRTF (head related transfer function) calculation module and a signal processing module; the HRTF calculation module is used for positioning the head posture by using a position sensor, positioning a sound source signal according to the microphone array, further determining the direction of the current sound relative to the head, then retrieving an HRTF database by using position information and calculating HRTF data of the direction by using an HRTF interpolation algorithm; the HRTF database is preset and stored in a memory of the 360-degree sound source real-time playback processing hardware module when leaving a factory; the signal processing module is mainly responsible for filtering the data by utilizing the calculated HRTF so as to generate a sound effect with azimuth sensation; the read interval of the head angle information is required to be consistent with the length of one extracted frame of data, and each frame of data corresponds to one set of read head angle information.

2. The real-time 360 degree audio source playback system of claim 1 wherein the localization of the audio source signal according to the microphone array is as follows:

let P point be the sound source. Mic₂、Mic₃、Mic₄And Mic₅The four microphones are respectively arranged at the positions shown in figure 2, and form a cross matrix;

wherein o represents the origin, R₂、R₃、R₄、R₅Respectively representing the distances of the sound source P to the four microphones. The positive x-axis direction represents the direction directly in front of the head, the positive y-axis direction represents the direction directly to the left of the head, and the z-axis direction represents the direction perpendicular to the ground. Theta represents the direction angle of the sound source, and is calculated in such a way that the included angle between the projection of a line segment consisting of the sound source and the origin on the xoy plane and the positive half axis of the x axis increases from 0 degree to 360 degrees in a clockwise direction from the positive half axis of the x axis.The elevation angle of the sound source is represented by the calculation mode of the included angle between the projection of a line segment formed by the sound source and the original point on the xoy plane and the xoy plane, the elevation angle value is positive when the sound source is above the xoy plane, and the elevation angle value is negative when the sound source is below the xoy plane. The time delay relationship of the four microphones is shown in (1).

Where C represents the speed of sound. Tau is₃₂、τ₄₂、τ₅₂Respectively represent Mic₃、Mic₄、Mic₅And Mic₂Their values can be determined from the data received by the microphones.

Assuming that the position of point P is (x, y, z), the position of point P is required to be listed up as the following equation, as shown in (2), where L represents the distance between the four microphones and the origin o.

Combining (1) and (2), it is possible to obtain:

therefore, the value of (x, y, z) can be obtained according to the formula (3), and the direction angle theta and the elevation angle can be obtained according to the geometric relationship

However, if only four microphones are measured, since their installation positions are symmetrical with respect to the origin, then if the sound source P is located on the z-axis, then theoretically P point to Mic₂～Mic₅Are equal, i.e. τ₃₂＝τ₄₂＝τ₅₂When the denominator is 0 in the solving formula of R2 according to (3), the correct position of the sound source cannot be found, and in this case, Mic is equal to 0₁Then will start working, where Mic₁The distance from the origin is also L. According to Mic₁And Mic₂From the received signal, we can find τ₂₁A value of (a), wherein₂₁Represents Mic₂And Mic₁The time delay therebetween. We can therefore list the following equations:

solving according to (4):

in this case, the direction angle and the elevation angle can be obtained accordingly.

In addition, when the decision condition (6) is satisfied, we can assume that the position of the sound source P is on the coordinate axis z.

And is

Since a set of helmet systems is designed here, the case where the distance P to o is less than L is not considered; therefore, the omnidirectional sound source positioning can be completed.

3. The 360-degree audio source real-time playback system of claim 2,the method is characterized in that an HRTF (head related transfer function) calculation module of a 360-degree sound source real-time playback processing hardware module firstly judges whether a sound source position (P point) is on a Z axis according to a formula (1) and a formula (6), and if the sound source position (P point) is on the Z axis, a direction angle and an elevation angle of the sound source position can be further calculated by using formulas (4) and (5)If not on the Z axis, (x, y, Z) and hence the direction and elevation angle can be found by using the equations (2) and (3)

4. The 360-degree audio source real-time playback system of claim 1, wherein the positioning of the head pose by the position sensor specifically comprises:

an inclination sensor or gyroscope is arranged on the helmet and used for providing a direction angle and an elevation angle after the head of a person rotates. And the direction angle and the elevation angle of the sound source at the current moment relative to the head of the person in the coordinate system can be calculated by combining the direction angle and the elevation angle calculated by positioning the sound source. For example, the initial direction of the sound source is measured asThe position of the head of the person at the current moment measured by the sensor isAccording to the principle of relative movement, when (theta)_o-θ_s) At > 0, the direction of the sound source with respect to the current head position can be measured asWhen (theta)_o-θ_s) < 0, the direction of the sound source with respect to the current head position can be measured as

5. The real-time 360 degree audio source playback system of claim 4 wherein the HRTF computing module of the real-time 360 degree audio source playback processing hardware module reads the information of the tilt sensor or gyroscope to obtain the direction angle and elevation angle of the human head after rotatingThen according to the principle of relative movement, with (theta)_o-θ_s) The value of (A) is taken as a criterion for judgment: if (theta)_o-θ_s) > 0, the direction of the sound source with respect to the current head position can be measured asIf (theta)_o-θ_s) < 0, the direction of the sound source with respect to the current head position can be measured as

6. The 360 degree audio source real-time playback system of claim 1 wherein the major parameters of the HRTF database include direction angle, elevation angle, distance, and HRTF parameters; after the angle of the sound source relative to the head at the current moment is calculated, the measured related data of the HRTF database and an interpolation algorithm are used for carrying out interpolation processing on the HRTF database to calculate the HRTF of the angle.

7. The real-time 360-degree audio source playback system of claim 6, wherein the interpolation algorithm is as follows: let HRTF of P be sought, where A, B, C, D represents the 4 points already existing in the database that are closest to the position of P, and C is the point with larger direction angle and elevation angle among the four points. Δ θ represents the difference in the direction angle between the C point and the P point, andrepresenting the difference in elevation between points C and P,θ_grid、representing the resolution of the azimuth and elevation angles in the database. The interpolation formula is shown as (8):

HRTF_p＝αHRTF_C+βHRTF_A+γHRTF_D (8)

wherein the HRTF_A、HRTF_C、HRTF_DThe data already existing in the database are respectively, and α, β, and γ are interpolation weights, and the calculation method thereof is shown as (9).

Where Δ θ represents the difference between the sound source P point and the closest directional angle present in the database (Δ θ)>0)，Representing the difference in elevation angle between the sound source P point and the closest elevation angle present in the databaseθ_grid、Representing the resolution of the azimuth and elevation angles in the database.

8. The 360-degree audio source real-time playback system of claim 7, wherein distance information about the head center position of the audio source is obtained, from which the distance R of the audio source relative to both ears is found_L、R_RThereby the HRTFs can be interpolated for distance; take the left ear as an example.

Where point P is the sound source, Q1 is the farther data point present in the database, Q2 is the closer data point present in the database, and Δ r represents the distance resolution in the database. The calculation formula of the distance interpolation is shown as (10):

HRTF_P＝a₁*HRTF_Q1+a₂*HRTF_Q2 (10)

in the formula a₁、a₂The interpolation weights are respectively expressed, and the calculation method is shown in (11).

9. The real-time audio playback system of claim 8, wherein the HRTF calculating module of the real-time audio playback processing hardware module of 360 degree retrieves the HRTF database according to the calculated sound source localization information and the head position perception information, and the HRTF corresponding to the sound source (P point) can be calculated by using the interpolation algorithm shown in fig. 5 using the formula (9) and the formula (8)_P。

10. The 360-degree audio source real-time playback system of claim 9, wherein the read interval of the head angle information is required to be consistent with the length of the extracted frame data, and each frame data corresponds to a set of read head angle information. Firstly, the sound source is positioned by utilizing the algorithm of the design, so that the direction angle, the elevation angle and the distance from the sound source to the head can be calculated, and two distance points which are most adjacent to the sound source are found in a database according to the distance. Then, the direction and elevation angle of each frame data relative to the head can be calculated according to the sound source direction obtained by positioning and the head angle information obtained by each frame, and the direction and elevation angle are set asRespectively calculating the relation between the two distance points by using an interpolation algorithmThe HRTF of (1). And then the HRTF of the frame can be obtained by using a distance interpolation algorithm.

Technical Field

The invention relates to the field of sound source playback, in particular to a 360-degree sound source real-time playback system.

Background

With the progress and development of science and technology, the requirements of more and more people on the quality of audio and video are gradually improved. Thus VR (virtual reality) and AR (augmented reality) gradually move into a person's field of view. To some extent AR can count as an upgrade and development of VR, since AR adds a real element to VR. HRTF (head related Transfer Function) is a technology that can be applied to AR, and filtering sound source signals by HRTF can produce corresponding direction perception.

The traditional technology has the following technical problems:

in practical applications, on one hand, since the HRTFs are related to the spatial positions of the head, the HRTFs for different angles of the head are different from one another, and if the HRTFs for each angle are to be measured, high cost and a large capacity of memory are required. On the other hand, it is desirable to locate a sound source by receiving a real sound from the outside and to reconstruct an audio signal to be played back into a headset through HRTFs and position sensor information.

Disclosure of Invention

The invention aims to provide a 360-degree sound source real-time playback system.

In order to solve the above technical problem, the present invention provides a real-time playback system of a 360-degree sound source, comprising: the main body is composed of an audio data helmet (helmet for short), the audio data helmet is composed of a helmet frame, a microphone array, earphones and a processing system, wherein the earphones are arranged according to the positions of human ears, a 360-degree sound source real-time playback processing hardware module mainly realizes processing functions of head position perception, sound source positioning, HRTF-based audio reconstruction and the like, and the helmet frame is assembled inside along the inner contour by adopting a flexible manufacturing process; the microphone array is used for the audio data acquisition function of the sound source positioning function and is arranged on the helmet;

the 360-degree sound source real-time playback processing hardware module comprises an HRTF (head related transfer function) calculation module and a signal processing module; the HRTF calculation module is used for positioning the head posture by using a position sensor and positioning a sound source signal according to a microphone array so as to determine the direction of the current sound relative to the head, and then, the HRTF calculation module is used for retrieving an HRTF database by using position information and calculating HRTF data of the direction by using an HRTF interpolation algorithm; the HRTF database is preset and stored in a memory of the 360-degree sound source real-time playback processing hardware module when leaving a factory; the signal processing module is mainly responsible for filtering the data by utilizing the calculated HRTF so as to generate a sound effect with azimuth sensation; the read interval of the head angle information is required to be consistent with the length of the extracted frame data, and each frame data corresponds to a group of read head angle information.

In another embodiment, the localization of the sound source signal according to the microphone array is specifically as follows:

let P point be the sound source. Mic₂、Mic₃、Mic₄And Mic₅The four microphones are respectively arranged at the positions shown in figure 2, and form a cross matrix;

wherein o represents the origin, R₂、R₃、R₄、R₅Respectively representing the distances of the sound source P to the four microphones. The positive x-axis direction represents the direction directly in front of the head, the positive y-axis direction represents the direction directly to the left of the head, and the z-axis direction represents the direction perpendicular to the ground. Theta represents the direction angle of the sound source, and is calculated in such a way that the included angle between the projection of a line segment consisting of the sound source and the origin on the xoy plane and the positive semiaxis of the x axis increases from 0 degree to 360 degrees in a clockwise direction from the positive semiaxis of the x axis.The elevation angle of the sound source is represented by the calculation mode of the included angle between the projection of a line segment formed by the sound source and the original point on the xoy plane and the xoy plane, the elevation angle value is positive when the sound source is above the xoy plane, and the elevation angle value is negative when the sound source is below the xoy plane. The time delay relationship of the four microphones is shown in (1).

Where C represents the speed of sound. Tau is₃₂、τ₄₂、τ₅₂Respectively represent Mic₃、Mic₄、Mic₅And Mic₂In time betweenThe values of the other can be determined from the data received by the microphone.

Assuming that the position of point P is (x, y, z), the position of point P is required to be listed up as the following equation, as shown in (2), where L represents the distance between the four microphones and the origin o.

Combining (1) and (2), it is possible to obtain:

therefore, the value of (x, y, z) can be obtained according to the formula (3), and the direction angle can be obtained according to the geometric relationship^θAnd elevation angle

However, if only four microphones are measured, since their installation positions are symmetrical with respect to the origin, then if the sound source P is located on the z-axis, then theoretically P point to Mic₂～Mic₅Are equal, i.e. τ₃₂＝τ₄₂＝τ₅₂When the denominator is 0 in the solving formula of R2 according to (3), the sound source cannot be located exactly when Mic is determined₁Then will start working, where Mic₁The distance from the origin is also L. According to Mic₁And Mic₂Receiving the received signal, we can find τ₂₁A value of (a), wherein₂₁Represents Mic₂And Mic₁The time delay therebetween. Therefore i can list the following formula:

solving according to (4):

in this case, the direction angle and the elevation angle can be obtained accordingly.

In addition, when the decision condition (6) is satisfied, we can assume that the position of the sound source P is on the coordinate axis z.

Since a set of helmet systems is designed here, the case where the distance P to o is less than L is not considered; therefore, the omnidirectional sound source positioning can be completed.

In one embodiment, the HRTF calculating module of the real-time playback processing hardware module for a 360-degree sound source first determines whether the sound source position (point P) is on the Z-axis according to formula (1) and formula (6), and if so, the direction angle and the elevation angle can be further determined by using formulae (4) and (5)If not on the Z axis, (x, y, Z) and hence the direction and elevation angle can be found by using the equations (2) and (3)

In one embodiment, the positioning of the head pose by the position sensor specifically comprises:

an inclination sensor or gyroscope is arranged on the helmet and used for providing a direction angle and an elevation angle after the head of a person rotates. And the direction angle and the elevation angle of the sound source at the current moment relative to the head of the person in the coordinate system can be calculated by combining the direction angle and the elevation angle calculated by positioning the sound source. For example, the initial direction of the sound source is measured asThe position of the head of the person at the current moment measured by the sensor isAccording to the principle of relative movement, when (theta)_o-θ_s) When greater than 0, canBy measuring the direction of the sound source with respect to the current head positionWhen (theta)_o-θ_s) < 0, the direction of the sound source with respect to the current head position can be measured as

In one embodiment, the HRTF calculation module of the 360-degree sound source real-time playback processing hardware module reads the information of the matched tilt sensor or gyroscope, so that the direction angle and elevation angle of the human head after rotation are obtainedThen according to the principle of relative movement, with (theta)_o-θ_s) The value of (A) is taken as a criterion for judgment: if (theta)_o-θ_s) > 0, the direction of the sound source with respect to the current head position can be measured asIf (theta)_o-θ_s) < 0, the direction of the sound source with respect to the current head position can be measured as

In one embodiment, the main parameters of the HRTF database include direction angle, elevation angle, distance, and HRTF parameters; after the angle of the sound source relative to the head at the current moment is calculated, the measured related data of the HRTF database and an interpolation algorithm are used for carrying out interpolation processing on the HRTF database to calculate the HRTF of the angle.

In one embodiment, the interpolation algorithm is as follows: suppose that HRTF of point P is to be solved, where A, B, C, D represents 4 points that exist in the database and are closest to the position of point P, and point C is the point of four points with larger direction angle and elevation angle. Δ θ represents the difference in the direction angle between the C point and the P point, andrepresenting the difference in elevation angle between points C and P, θ_grid、Indicating the resolution of the azimuth and elevation angles in the database. The interpolation formula is shown as (8):

HRTF_p＝αHRTF_C+βHRTF_A+γHRTF_D (8)

wherein the HRTF_A、HRTF_C、HRTF_DThe data already existing in the database are respectively, and α, β, and γ are interpolation weights, and the calculation method is shown as (9).

Where delta theta represents the difference between the sound source P point and the closest directional angle present in the database (delta theta > 0),representing the difference in elevation angle between the sound source P point and the closest elevation angle present in the databaseθ_grid、Representing the resolution of the direction and elevation angles in the database.

In one embodiment, distance information of the sound source with respect to the head center position is derived, from which the distance R of the sound source with respect to the ears can be determined_L、R_RThereby the HRTFs can be interpolated for distance; take the left ear as an example.

Where point P is the sound source, Q1 is the farther data point present in the database, Q2 is the closer data point present in the database, and Δ r represents the distance resolution in the database. The calculation formula of the distance interpolation is shown as (10):

HRTF_P＝a₁*HRTF_Q1+a₂*HRTF_Q2 (10)

in the formula a₁、a₂The interpolation weights are respectively expressed, and the calculation method is shown in (11).

In one embodiment, the HRTF calculating module of the real-time 360-degree sound source playback processing hardware module may retrieve the HRTF database according to the calculated sound source localization information and head position perception information, and may calculate the HRTF corresponding to the sound source (P point) by using the interpolation algorithm shown in fig. 5 according to formula (9) and formula (8)_P。

In one embodiment, the read interval of the head angle information is required to be consistent with the length of the extracted frame data, and each frame data corresponds to a set of read head angle information. Firstly, the sound source is positioned by utilizing the algorithm of the design, the direction angle, the elevation angle and the distance from the sound source to the head can be calculated, and two distance points which are most adjacent to the sound source are found in a database according to the distance. Then, the direction and elevation angle of each frame data relative to the head can be calculated according to the sound source direction obtained by positioning and the head angle information obtained by each frame, and the direction and elevation angle are set asRespectively calculating the relation between the two distance points by using an interpolation algorithmThe HRTF of (1). And then the HRTF of the frame can be obtained by using a distance interpolation algorithm.

The invention has the beneficial effects that:

the 360-degree sound source real-time playback system positions a sound source by a TDOA algorithm of a microphone array arranged on a helmet, and then calculates the position of the sound source at the current moment relative to the head by combining a head movement angle sensed by a tilt sensor arranged on the helmet. And calculating the HRTF value of the angle by utilizing a triangular interpolation algorithm and a measured HRTF database with certain resolution according to the calculated sound source positioning information and head position perception information, thereby realizing real-time playback of a 360-degree sound source. Therefore, real-time playback of external signals is realized, and the data calculation amount and the measurement cost of the system are successfully reduced.

Drawings

Fig. 1 is a schematic diagram of microphone array installation in the 360-degree sound source real-time playback system of the present invention.

FIG. 2 is a schematic diagram of a head coordinate system in the 360-degree audio source real-time playback system according to the present invention.

Fig. 3 is a schematic block diagram of the overall design of the 360-degree audio source real-time playback system of the present invention.

FIG. 4 is a schematic diagram of the geometric relationship of sound source localization in the real-time 360-degree sound source playback system of the present invention

FIG. 5 is a schematic diagram of the angle interpolation in the real-time 360-degree audio playback system according to the present invention.

FIG. 6 is a schematic diagram of distance interpolation in the real-time 360-degree audio playback system according to the present invention.

Detailed Description

The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.

The 360-degree sound source real-time playback system main body is composed of an audio data helmet (helmet for short) which is composed of a helmet frame, a microphone array, earphones and a processing system, wherein the earphones are arranged according to the positions of human ears, the 360-degree sound source real-time playback processing hardware module mainly achieves processing functions of head position perception, sound source positioning, audio reconstruction based on HRTF and the like, and the 360-degree sound source real-time playback processing hardware module is assembled inside the helmet frame along the inner contour of the helmet frame by adopting a flexible manufacturing process. The microphone array is used for the audio data acquisition function of the sound source positioning function and is arranged on the helmet, wherein the arrangement method is shown in figure 1.

The hemisphere shown in fig. 1 is a helmet, and numbers 1, 2, 3, 4, and 5 are respectively indicated as five microphones, which together constitute a microphone array for sound source localization. Wherein the 5 microphones are all omni-directional microphones.

To describe the position of the microphone array, and thus the pose of the helmet, and thus more clearly to represent the HRTF, a coordinate system is established with respect to the head, as shown in fig. 2.

Fig. 2(a) is a top view of the head, and s1 and s2 are assumed sound sources, where θ represents a direction angle which increases in the order of 0 degrees clockwise from the front of the head of a person, ranging from 0 to 360 degrees, e.g., θ _1, θ _ 2. Fig. 2(b) is a right side view of the head, s3 and s4 are assumed sound sources, where phi denotes an elevation angle, the elevation angle is an angle between a sound source direction and a horizontal plane with reference to the horizontal plane, the sound source is above the horizontal plane, the elevation angle is positive, for example, phi _3 is below the horizontal plane, and the elevation angle is negative, for example, phi _ 4.

The 360-degree sound source real-time playback processing hardware module realizes the 360-degree sound source real-time playback function, and the general design principle block diagram of the system is shown in fig. 3, which mainly comprises an HRTF calculation module and a signal processing module. The HRTF calculation module is mainly used for positioning the head posture by using a position sensor and positioning a sound source signal according to a microphone array so as to determine the direction of the current sound relative to the head, and then the HRTF calculation module is used for searching an HRTF database by using position information and calculating HRTF data of the direction by using an HRTF interpolation algorithm. The HRTF database is preset and stored in a memory of the 360-degree sound source real-time playback processing hardware module when the HRTF database leaves a factory. The signal processing module is mainly responsible for filtering the data by utilizing the calculated HRTF so as to generate the sound effect with the azimuth effect. It is required that the read interval of the head angle information is kept consistent with the length of one extracted frame of data, each frame of data corresponding to one set of read head angle information.

The specific processing process of real-time playback of the 360-degree sound source can be divided into the following steps:

1. sound source localization

(1) Principle of sound source localization

As shown in fig. 4, the P point is assumed to be a sound source. Mic₂、Mic₃、Mic₄And Mic₅Arranged in the positions shown in fig. 2, respectively, four microphones forming a crossAnd (4) forming.

Wherein o represents the origin, R₂、R₃、R₄、R₅Respectively representing the distances of the sound source P to the four microphones. The positive x-axis direction represents the direction directly in front of the head, the positive y-axis direction represents the direction directly to the left of the head, and the z-axis direction represents the direction perpendicular to the ground. Theta represents the direction angle of the sound source, and is calculated in such a way that the included angle between the projection of a line segment consisting of the sound source and the origin on the xoy plane and the positive semiaxis of the x axis increases from 0 degree to 360 degrees in a clockwise direction from the positive semiaxis of the x axis.The elevation angle of the sound source is represented by the calculation mode of the included angle between the projection of a line segment formed by the sound source and the original point on the xoy plane and the xoy plane, the elevation angle value is positive when the sound source is above the xoy plane, and the elevation angle value is negative when the sound source is below the xoy plane. The time delay relationship of the four microphones is shown in (1).

Where C represents the speed of sound. Tau is₃₂、τ₄₂、τ₅₂Respectively represent Mic₃、Mic₄、Mic₅And Mic₂The values of the time delay between the two can be obtained by the data received by the microphones.

Assuming that the position of point P is (x, y, z), the position of point P is required to be listed up as the following equation, as shown in (2), where L represents the distance between the four microphones and the origin o.

Combining (1) and (2), it is possible to obtain:

therefore, the value of (x, y, z) can be obtained according to the formula (3), and the square can be obtained according to the geometric relationshipAngle of direction theta and angle of elevation

However, if only four microphones are measured, since their installation positions are symmetrical with respect to the origin, then if the sound source P is located on the z-axis, then theoretically P point to Mic₂～Mic₅Are equal, i.e. τ₃₂＝τ₄₂＝τ₅₂When the denominator is 0 in the solving formula of R2 according to (3), the sound source cannot be located exactly when Mic is determined₁Then will start working, where Mic₁The distance from the origin is also L. According to Mic₁And Mic₂Receiving the received signal, we can find τ₂₁A value of (a), wherein₂₁Represents Mic₂And Mic₁The time delay therebetween. Therefore i can list the following formula:

solving according to (4):

in this case, the direction angle and the elevation angle can be obtained accordingly.

In addition, when the decision condition (6) is satisfied, we can assume that the position of the sound source P is on the coordinate axis z.

Since a set of helmet systems is designed here, the case where the distance P to o is less than L is not considered. Therefore, the omnidirectional sound source positioning can be completed.

(2) Sound source positioning method

The HRTF calculation module of the real-time playback processing hardware module of the 360-degree sound source firstly judges whether the sound source position (P point) is on the Z axis according to the formula (1) and the formula (6), and if so, the HRTF calculation module judges whether the sound source position (P point) is on the Z axisThe direction angle and elevation angle can be further determined by the following equations (4) and (5)If not on the Z axis, (x, y, Z) and hence the direction and elevation angle can be found by using the equations (2) and (3)

2. Head position sensing

(1) Head position sensing principle

An inclination sensor or gyroscope is arranged on the helmet and used for providing a direction angle and an elevation angle after the head of a person rotates. And the direction angle and the elevation angle of the sound source at the current moment relative to the head of the person in the coordinate system can be calculated by combining the direction angle and the elevation angle calculated by positioning the sound source. For example, the initial direction of the sound source is measured asThe position of the head of the person at the current moment measured by the sensor isAccording to the principle of relative movement, when (theta)_o-θ_s) At > 0, the direction of the sound source with respect to the current head position can be measured asWhen (theta)_o-θ_s) < 0, the direction of the sound source with respect to the current head position can be measured as

(2) Head perception algorithm

Reading information of a matched tilt sensor or gyroscope by an HRTF (head related transfer function) calculation module of the 360-degree sound source real-time playback processing hardware module so as to obtain a direction angle and an elevation angle of a rotated human headThen according to the principle of relative movement, with (theta)_o-θ_s) The value of (A) is taken as a criterion for judgment: if (theta)_o-θ_s) > 0, the direction of the sound source with respect to the current head position can be measured asIf (theta)_o-θ_s) < 0, the direction of the sound source with respect to the current head position can be measured as

HRTF data acquisition

(1) HRTF data acquisition principle

Since the HRTF for any angle needs huge cost and calculation amount, a group of HRTF databases with certain resolution can be measured in advance and stored in a memory of the system. The main parameters of the HRTF database include direction angle, elevation angle, distance and HRTF parameters. After calculating the angle of the sound source relative to the head at the current moment, we can use the measured related data of the HRTF database and an interpolation algorithm to interpolate the HRTF of the angle.

A schematic diagram of the interpolation algorithm is shown in fig. 5.

Suppose that HRTF of point P is to be found, where A, B, C, D represents 4 points already existing in the database and closest to the position of point P, and point C is the point of the four points with the larger direction angle and elevation angle. Δ θ represents the difference in the direction angle between the point C and the point P, andrepresenting the difference in elevation angle between points C and P, θ_grid、Indicating the resolution of the direction and elevation angles in the database. The interpolation formula is shown as (8):

HRTF_p＝αHRTF_c+βHRTF_A+γHRTF_D (8)

wherein the HRTF_A、HRTF_C、HRTF_DThe data already existing in the database are respectively, and α, β, and γ are interpolation weights, and the calculation method is shown as (9).

Where delta theta represents the difference between the sound source P point and the closest directional angle present in the database (delta theta > 0),representing the difference in elevation angle between the sound source P point and the closest elevation angle present in the databaseθ_grid、Representing the resolution of the direction and elevation angles in the database.

Because the sound source positioning of the 5 microphones of the design can also obtain the distance information of the sound source about the head center position, the distance R of the sound source relative to the ears can be obtained according to the distance information_L、R_RThe HRTFs can thus be interpolated for distance. As shown in fig. 6, the left ear is taken as an example.

Where point P is the sound source, Q1 is the farther data point present in the database, Q2 is the closer data point present in the database, and Δ r represents the distance resolution in the database. The calculation formula of the distance interpolation is shown as (10):

HRTF_P＝a₁*HRTF_Q1+a₂*HRTF_Q2(10) in the formula a₁、a₂The interpolation weights are respectively expressed, and the calculation method is shown in (11).

(2) HRTF calculation method

The HRTF calculating module of the 360-degree sound source real-time playback processing hardware module can retrieve the HRTF database according to the calculated sound source positioning information and the head position perception information, and the HRTF corresponding to the sound source (P point) can be calculated by using the interpolation algorithm shown in fig. 5 by using the formula (9) and the formula (8)_P。

Under the condition that the positions of ears of the helmet are determined, the system can also calculate the HRTF corresponding to the sound source (P point) by adopting the formula (11) and the formula (10) and adopting the interpolation algorithm shown in FIG. 6_P。

Specifically, as shown in fig. 2, it is required that the read interval of the head angle information and the length of the extracted frame data are consistent, and each frame data corresponds to a set of read head angle information. Firstly, the sound source is positioned by utilizing the algorithm of the design, the direction angle, the elevation angle and the distance from the sound source to the head can be calculated, and two distance points which are most adjacent to the sound source are found in a database according to the distance. Then, the direction and elevation angle of each frame data relative to the head can be calculated according to the sound source direction obtained by positioning and the head angle information obtained by each frame, and the direction and elevation angle are set asRespectively calculating the relation between the two distance points by using an interpolation algorithmThe HRTF of (1). And then the HRTF of the frame can be obtained by using a distance interpolation algorithm.

4.360 degree sound source real-time playback

As shown in fig. 2, a frame of signal (20-30 ms) is sequentially extracted for processing, and the HRTF function corresponding to the frame, which is output by the HRTF calculation module of the 360-degree sound source real-time playback processing hardware module, is received. And filtering the frame of signal by using an HRTF function, outputting the frame of audio after the filtering is finished, and realizing real-time playback of the 360 sound sources.

The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.