阅读说明：本技术 动态音频感知追踪系统和方法、存储介质、耳机 (Dynamic audio perception tracking system and method, storage medium, and earphone ) 是由 宋立玮 罗海彬 王丽 于 2021-09-15 设计创作，主要内容包括：本发明公开了一种动态音频感知追踪系统和方法、存储介质、耳机,动态音频感知追踪系统包括音频感知模块、脑电波检测模块、音频分析模块和音频追踪模块；音频感知模块用于采集环境中的不同声源所分别对应的第一音频信号,并对第一音频信号进行滤波和放大处理后,传输至耳朵；脑电波检测模块用于检测人脑对于每个第一音频信号的脑电波变化,以获取人脑感兴趣的第二音频信号；音频分析模块用于对第二音频信号的波形进行切片和特征提取,并进行标记,以及判断每个第二音频信号的优先级别；音频追踪模块用于按照优先级别分别对每个第二音频信号所对应的声源进行定位追踪。根据本发明的动态音频感知追踪系统,能够对感兴趣的声音进行定位追踪。(The invention discloses a dynamic audio perception tracking system and a method, a storage medium and an earphone, wherein the dynamic audio perception tracking system comprises an audio perception module, a brain wave detection module, an audio analysis module and an audio tracking module; the audio sensing module is used for acquiring first audio signals corresponding to different sound sources in the environment respectively, filtering and amplifying the first audio signals and transmitting the first audio signals to ears; the brain wave detection module is used for detecting brain wave change of the human brain on each first audio signal so as to acquire a second audio signal which is interesting to the human brain; the audio analysis module is used for carrying out slicing and feature extraction on the waveform of the second audio signal, marking the waveform and judging the priority level of each second audio signal; and the audio tracking module is used for respectively positioning and tracking the sound source corresponding to each second audio signal according to the priority level. According to the dynamic audio perception tracking system, the interested sound can be located and tracked.)

1. A dynamic audio perceptual tracking system, comprising:

the audio sensing module is used for acquiring first audio signals corresponding to different sound sources in the environment respectively, filtering and amplifying the first audio signals and transmitting the first audio signals to ears;

the brain wave detection module is used for detecting brain wave changes of the human brain on each first audio signal so as to acquire a second audio signal which is interesting to the human brain;

the audio analysis module is used for carrying out slicing and feature extraction on the waveform of the second audio signal, marking the waveform of the second audio signal and judging the priority level of each second audio signal;

and the audio tracking module is used for respectively positioning and tracking the sound source corresponding to each second audio signal according to the priority level.

2. The dynamic audio perception tracking system of claim 1, further comprising an audio buffering module configured to store the first audio signal and transmit the first audio signal having an occurrence frequency above a threshold to the audio analysis module.

3. The dynamic audio perception tracking system according to claim 1 or 2, wherein the audio perception module includes a first audio signal filter, an audio signal collector and an audio signal amplifier electrically connected in sequence.

4. The dynamic audio perception tracking system according to claim 1 or 2, wherein the brain wave detection module comprises a detection electrode, a current timer, an electroencephalogram signal amplifier, an electroencephalogram frequency counter, a controller and a second audio signal filter which are electrically connected in sequence.

5. The dynamic audio perception tracking system of claim 1 or 2, wherein the audio tracking module includes a six-axis inertial sensor.

6. A dynamic audio perceptual tracking method based on the dynamic audio perceptual tracking system as claimed in any one of claims 1 to 5, comprising the steps of:

collecting first audio signals corresponding to different sound sources in an environment respectively, filtering and amplifying the first audio signals, and transmitting the first audio signals to ears;

detecting changes of the brain wave of the human brain for each first audio signal to obtain a second audio signal which is interesting to the human brain;

slicing and feature extraction are carried out on the second audio signals, marking is carried out on the second audio signals, and the priority level of each second audio signal is judged;

and respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.

7. The dynamic audio perceptual tracking method of claim 6, further comprising the steps of:

the audio sensing module transmits the first audio signal to an audio caching module;

and if the frequency of the first audio signal exceeds a threshold value, the audio buffer module transmits the first audio signal to an audio analysis module.

8. A computer-readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the dynamic audio perceptual tracking method of any one of claims 6 to 7.

9. Headphones, characterized in that it comprises a dynamic audio perceptual tracking system according to any of the claims 1 to 5.

Technical Field

The invention relates to the technical field of sound source positioning, in particular to a dynamic audio perception tracking system and method, a storage medium and an earphone.

Background

When a user wears the earphone, whether the user listens to music or not, the earphone can shield some environmental sound sources; therefore, when a user encounters an emergency or is interested in a certain sound source, the user has to remove the earphone to carefully listen and judge, which causes the situation that the earphone is dropped on the ground or lost carelessly. In addition, uncertainty of the direction of the sound source, the type of the sound source, the distance of the sound source, the perceived priority level of the sound source, and the like affects the ability of the user wearing the headset to determine an emergency, and particularly for users with hearing impairment or poor hearing, the user may easily ignore information useful for the user in the environmental sound source when using the headset, may miss a station, miss an important instruction, ignore an alarm sound, and the like.

Disclosure of Invention

The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a dynamic audio perception tracking system which can detect sounds in the external environment in real time under the condition that a user wears an earphone and accurately judge the distance, direction, type, importance degree and the like of the sounds.

The invention also provides a dynamic audio perception tracking method.

The invention also provides a computer readable storage medium.

The invention further provides the earphone.

In a first aspect, a dynamic audio perception tracking system according to an embodiment of the present invention includes an audio perception module, a brain wave detection module, an audio analysis module, and an audio tracking module; the audio sensing module is used for acquiring first audio signals corresponding to different sound sources in the environment respectively, filtering and amplifying the first audio signals and transmitting the first audio signals to ears; the brain wave detection module is used for detecting brain wave change of the human brain on each first audio signal so as to acquire a second audio signal which is interesting to the human brain; the audio analysis module is used for carrying out slicing and feature extraction on the waveform of the second audio signal, marking the waveform and judging the priority level of each second audio signal; the audio tracking module is used for respectively positioning and tracking the sound source corresponding to each second audio signal according to the priority level.

The dynamic audio perception tracking system according to the embodiment of the invention has at least the following beneficial effects: the earphone can help users who wear earphones to listen to music, detect sounds in the external environment in real time, and accurately judge the distance, direction, type, importance degree and the like of the sounds, so that the users are helped to dynamically position and track the sound sources interested by the users, the users are helped to accurately judge the contents of the sounds and take corresponding actions, and important information is avoided being missed.

According to some embodiments of the present invention, the audio analysis module is further configured to store the first audio signal and transmit the first audio signal with the frequency of occurrence higher than a threshold to the audio buffer module.

According to some embodiments of the present invention, the audio sensing module includes a first audio signal filter, an audio signal collector, and an audio signal amplifier, which are electrically connected in sequence.

According to some embodiments of the present invention, the brain wave detection module includes a detection electrode, a current timer, an electroencephalogram signal amplifier, an electroencephalogram frequency counter, a controller, and a second audio signal filter, which are electrically connected in sequence.

According to some embodiments of the invention, the audio tracking module comprises a six-axis inertial sensor.

In a second aspect, a dynamic audio perceptual tracking method according to an embodiment of the present invention includes the steps of: collecting first audio signals corresponding to different sound sources in an environment respectively, filtering and amplifying the first audio signals, and transmitting the first audio signals to ears; detecting changes of the brain wave of the human brain for each first audio signal to obtain a second audio signal which is interesting to the human brain; slicing and feature extraction are carried out on the second audio signals, marking is carried out on the second audio signals, and the priority level of each second audio signal is judged; and respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.

The dynamic audio perception tracking method according to the embodiment of the invention at least has the following beneficial effects: the earphone can help users who wear earphones to listen to music, monitor sounds in the external environment in real time, and accurately judge the distance, direction, type and importance degree of the sounds, so that the users are helped to dynamically position and track sound sources interested by the users, the users are helped to accurately judge the contents of the sounds and take corresponding actions, and important information is avoided being missed.

According to some embodiments of the invention, further comprising the steps of: the audio sensing module transmits the first audio signal to an audio caching module; and if the frequency of the first audio signal exceeds a threshold value, the audio buffer module transmits the first audio signal to an audio analysis module.

In a third aspect, a headset according to an embodiment of the present invention includes a dynamic audio perceptual tracking system as described in the above embodiments of the first aspect of the present invention.

The earphone according to the embodiment of the invention has at least the following beneficial effects: by adopting the dynamic audio perception tracking system, the sound in the external environment can be monitored in real time under the condition that the user wears the earphone to listen to music, and the distance, the direction, the type and the importance degree of the sound can be accurately judged, so that the user can be helped to dynamically position and track the sound source interested by the user, the user can be helped to accurately judge the content of the sound and take corresponding action, and important information is prevented from being missed.

In a fourth aspect, a computer-readable storage medium according to an embodiment of the present invention has stored thereon computer instructions, which when executed by a processor, implement a dynamic audio perceptual tracking method according to an embodiment of the second aspect of the present invention.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a block diagram of a dynamic audio perception tracking system according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a brain wave detection module according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating steps of a perceptual tracking method for dynamic audio according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to the present preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.

In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.

In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.

In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.

In a first aspect, as shown in fig. 1, a dynamic audio perception tracking system according to an embodiment of the present invention includes an audio perception module, a brain wave detection module, an audio analysis module, and an audio tracking module; the audio sensing module is used for acquiring first audio signals corresponding to different sound sources in the environment respectively, filtering and amplifying the first audio signals and transmitting the first audio signals to ears; the brain wave detection module is used for detecting brain wave change of the human brain on each first audio signal so as to acquire a second audio signal which is interesting to the human brain; the audio analysis module is used for carrying out slicing and feature extraction on the waveform of the second audio signal, marking the waveform and judging the priority level of each second audio signal; and the audio tracking module is used for respectively positioning and tracking the sound source corresponding to each second audio signal according to the priority level.

Specifically, the audio sensing module includes a first audio signal filter, an audio signal collector and an audio signal amplifier which are electrically connected in sequence. The first audio signal filter is used for filtering out sound sources far away from each other, the audio signal collector is used for collecting sound sources near to a user (such as conversation sound, whistling sound, broadcast sound, prompting sound, warning sound and the like of people near to each other), first audio signals corresponding to the sound sources are amplified through the audio signal amplifier and transmitted to the ears, and the first audio signals are transmitted to the brain through the ears. That is to say, the sound source perception module is used for perceiving sound sources from different directions in the environment, filtering out sound with a longer distance, collecting sound sources with a shorter distance, amplifying first audio signals corresponding to the sound sources, and transmitting the amplified first audio signals to ears.

As shown in fig. 2, in some embodiments of the present invention, the brain wave detecting module includes a detecting electrode, a current timer, an electroencephalogram signal amplifier, an electroencephalogram frequency counter, a controller, and a second audio signal filter, which are electrically connected in sequence. The detection electrodes are used for collecting brain waves of the human brain, the current timer and the brain wave frequency counter are used for acquiring the time domain and frequency domain change conditions of the brain waves, the brain wave signal amplifier is used for amplifying the collected brain wave signals, and the detection electrodes, the current timer, the brain wave signal amplifier and the brain wave frequency counter are used for detecting the change conditions of the brain waves in real time and transmitting the change conditions to the controller; when a user is interested in the sound heard by the user, the brain waves can generate larger fluctuation due to stress reaction, so that the user can know which audio is interested by the user by detecting the change condition of the brain waves; at this time, the controller will activate the second audio signal filter to filter out the audio frequency bands that are not of interest to the user, and only retain the audio signals of interest (i.e., the second audio signals), and transmit these second audio signals to the audio analysis module. Wherein, the controller can adopt common processors such as MCU.

After receiving second audio signals, the audio analysis module performs slicing and feature extraction on the waveforms of the second audio signals, marks the signals and respectively clusters the second audio signals by using a corresponding self-learning algorithm; meanwhile, the audio analysis module judges the priority level of the second audio signals according to the degree of interest of the user and sends the second audio signals to the audio tracking module.

The audio tracking module can adopt an MEMS six-axis inertial sensor, the sensor integrates functional parts such as an MEMS accelerometer, a gyroscope, a pressure sensor, a magnetic sensor, a signal processing circuit and the like into a silicon chip by utilizing a three-dimensional heterogeneous integration technology, and an algorithm is built in the sensor to realize functions such as chip-level guidance, navigation, positioning and the like. The audio tracking module utilizes the MEMS six-axis inertial sensor to react to physical movement of a user, takes a sound source interested by the user as a fixed reference object, converts the reaction of linear displacement and angular velocity rotation of the user into an electric signal, and realizes the function of space dynamic positioning and tracking of the interested sound source.

Therefore, the dynamic audio perception tracking system according to the embodiment of the invention can help users who wear earphones to listen to music, detect sounds in the external environment in real time, and accurately judge the distance, the type, the direction and the importance degree of the sounds, thereby helping users to dynamically position and track sound sources interested in the users, helping users to accurately judge the contents of the sounds and take corresponding actions, and avoiding missing important information.

As shown in fig. 1, in some embodiments of the present invention, the audio buffer module is further included, and the audio buffer module is configured to buffer the first audio signal and transmit the first audio signal with the occurrence frequency higher than the threshold to the audio analysis module. Wherein, the value of the threshold value can be self-defined.

Specifically, after the audio sensing module collects sound sources in the environment, first audio signals corresponding to the sound sources are sent to the audio cache module, so that the audio cache module can judge whether some sounds frequently appear. And if the audio buffer module finds that the occurrence frequency of a certain first audio signal is higher, the first audio signal is sent to the audio analysis module for analysis. This is because, when a certain sound occurs frequently, the brain fatigues the sound, and brain waves do not fluctuate significantly; therefore, the sound needs to be identified by the audio buffer module so that the audio tracking module can perform localization tracking on the sound. If the brain wave detection module does not detect brain wave fluctuation and the audio cache module does not find audio with high frequency, the audio sensing module enters a standby state, and power consumption is saved.

In a second aspect, as shown in fig. 3, a dynamic audio perception tracking method according to an embodiment of the present invention includes the following steps:

s100: the method comprises the steps of collecting first audio signals corresponding to different sound sources in an environment respectively, filtering and amplifying the first audio signals, and transmitting the first audio signals to ears.

Specifically, in practical application, some sound sources far away from the user are filtered by a first audio signal filter in the audio sensing module, then some sound sources near the user (such as conversation sound, whistling sound, broadcast sound, warning sound, and the like of people near the user) are collected by an audio signal collector, and first audio signals corresponding to the sound sources are amplified by an audio signal amplifier and transmitted to ears and then transmitted to the brain through human ears.

S200: changes in brain waves of the human brain for each first audio signal are detected to obtain a second audio signal of interest to the human brain.

In particular, in practical application, the change condition of the brain wave is detected in real time through a detection electrode, a current timer, a brain wave signal amplifier and a brain wave frequency counter in the brain wave detection module, and the reaction condition of the brain wave to each first audio signal is obtained. When the brain waves are detected to have larger fluctuation changes, the brain is shown to be interested in the first audio signals, at the moment, the controller can transfer the second audio signal filter, filter out the audio frequency bands which are not interested by the user, only keep the interested audio signals (namely the second audio signals), and transmit the second audio signals to the audio analysis module.

S300: the second audio signals are sliced and feature extracted, labeled, and the priority level of each second audio signal is determined.

Specifically, after receiving second audio signals, the audio analysis module performs slicing and feature extraction on waveforms of the second audio signals, and marks the signals, so as to cluster the second audio signals respectively by using a corresponding self-learning algorithm; meanwhile, the audio analysis module can judge the priority level of the second audio signals according to the degree of interest of the user, so as to judge which sounds are more important for the user, and send the second audio signals to the audio tracking module.

S400: and respectively carrying out localization tracking on the sound source corresponding to each second audio signal according to the priority level.

The audio tracking module adopts an MEMS six-axis inertial sensor to perform positioning tracking on a sound source which is interested by a user. In the process of positioning and tracking, a sound source interested by a user is used as a fixed reference object, the MEMS six-axis inertial sensor reacts to the physical movement of the user, and the reaction of the linear displacement and the angular speed rotation of the user is converted into an electric signal, so that the function of dynamically positioning and tracking the space of the sound source interested is realized.

According to the dynamic audio perception tracking method provided by the embodiment of the invention, users who wear earphones to listen to music can be helped to monitor sounds in the external environment in real time, and the distance, direction, type and importance degree of the sounds are accurately judged, so that the users are helped to dynamically position and track sound sources interested by the users, the users are helped to accurately judge the contents of the sounds and take corresponding actions, and important information is avoided from missing.

In some embodiments of the present invention, the above-mentioned dynamic audio perception tracking method further includes the following steps: the audio sensing module transmits the collected first audio signal to the audio cache module, and if the frequency of the first audio signal exceeds a threshold value, the first audio signal is transmitted to the audio analysis module. Wherein, the value of the threshold value can be self-defined.

Specifically, after the audio sensing module collects sound sources in the environment, first audio signals corresponding to the sound sources are sent to the audio cache module, so that the audio cache module can judge whether some sounds frequently appear. If the audio buffer module finds that the occurrence frequency of a certain sound is high, the first audio signal corresponding to the sound is sent to the audio analysis module. This is because, when a certain sound occurs frequently, the brain fatigues the sound, and brain waves do not fluctuate significantly; therefore, the sound needs to be identified by the audio buffer module so that the audio tracking module can perform localization tracking on the sound. If the brain wave detection module does not detect brain wave fluctuation and the audio cache module does not detect sound with high frequency, the audio sensing module enters a standby state, and power consumption is saved.

In a third aspect, by adopting the dynamic audio tracking system according to the embodiment of the present invention, the sound in the external environment can be monitored in real time without affecting the condition that the user wears the earphone to listen to music, and the distance, direction, type and importance degree of the sound can be accurately determined, so as to help the user perform dynamic positioning tracking on the sound source that the user is interested in, and help the user perform accurate determination on the content of the sound and take corresponding action, thereby avoiding missing important information.

Fourth aspect, it should be recognized that the steps of the above-described dynamic audio perception tracking method in embodiments of the present invention may be implemented or embodied by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The method may use standard programming techniques. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.

Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.

Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein.

A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.

In the description herein, references to the description of "one embodiment," "a further embodiment," "some specific embodiments," or "some examples," etc., mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.