阅读说明：本技术 用于语音分离的方法和系统 (Method and system for voice separation ) 是由 毕相如 张青山 于 2019-03-07 设计创作，主要内容包括：本公开涉及一种使用滑动窗口的语音分离方法和系统。该方法包括：由至少一个传声器从至少一个用户获取至少一个语音并且将至少一个语音作为语音信号存储在声音记录模块中；通过滑动窗口从声音记录模块提取语音信号并且处理所提取语音信号；以及将所处理语音信号传输到DUET模块进行语音分离。(The present disclosure relates to a voice separation method and system using a sliding window. The method comprises the following steps: acquiring, by at least one microphone, at least one voice from at least one user and storing the at least one voice as a voice signal in a sound recording module; extracting a voice signal from the sound recording module through a sliding window and processing the extracted voice signal; and transmitting the processed voice signals to the DUET module for voice separation.)

1. A method for speech separation, comprising:

acquiring at least one voice from at least one user by at least one microphone and storing the at least one voice as a voice signal in a sound recording module;

extracting the voice signal from the sound recording module through a sliding window and processing the extracted voice signal; and

and transmitting the processed voice signal to a DUET module for voice separation.

2. The method of claim 1, wherein processing the extracted speech signal through the sliding window comprises:

traversing the extracted speech signal to determine a maximum amplitude of the speech signal;

determining a start position of the sliding window, the start position of the sliding window being a position where an amplitude of the speech signal exceeds a predetermined proportion of the maximum amplitude for a first time from a start of the speech signal;

determining an end position of the sliding window, the end position of the sliding window being a position from an end of the speech signal to the beginning of the speech signal where the amplitude of the speech signal exceeds a predetermined proportion of the maximum amplitude for the first time; and

selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

3. The method of claim 2, wherein the predetermined ratio is greater than or equal to 1/4 and less than or equal to 1/2.

4. The method of claim 1, wherein processing the extracted speech signal through the sliding window comprises:

traversing the extracted speech signal to determine an average amplitude of the speech signal;

determining a start position of the sliding window, the start position of the sliding window being a position at which the amplitude of the speech signal first exceeds the average amplitude from the start of the speech signal;

determining an end position of the sliding window, the end position of the sliding window being a position where the amplitude of the speech signal first exceeds the average amplitude from the end of the speech signal to the beginning of the speech signal;

selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

5. A system for speech separation, comprising:

at least one microphone, said at least one microphone acquiring at least one voice from at least one user;

a sound recording module for storing the at least one voice as a voice signal;

a sliding window for extracting the voice signal from the sound recording module and processing the extracted voice signal; and

a DUET module to receive the processed voice signal for voice separation.

6. The system of claim 5, wherein the sliding window is further configured to:

traversing the extracted speech signal to determine a maximum amplitude of the speech signal;

determining a start position of the sliding window, the start position of the sliding window being a position where an amplitude of the speech signal exceeds a predetermined proportion of the maximum amplitude for a first time from a start of the speech signal;

determining an end position of the sliding window, the end position of the sliding window being a position from an end of the speech signal to the beginning of the speech signal where the amplitude of the speech signal exceeds a predetermined proportion of the maximum amplitude for the first time; and is

Selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

7. The system of claim 6, wherein the predetermined ratio is greater than or equal to 1/4 and less than or equal to 1/2.

8. The system of claim 5, wherein the sliding window is further configured to:

traversing the extracted speech signal to determine an average amplitude of the speech signal;

determining a start position of the sliding window, the start position of the sliding window being a position at which the amplitude of the speech signal first exceeds the average amplitude from the start of the speech signal;

determining an end position of the sliding window, the end position of the sliding window being a position where the amplitude of the speech signal first exceeds the average amplitude from the end of the speech signal to the beginning of the speech signal;

selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

9. A computer-readable medium having computer-executable instructions for performing the method of one of claims 1-4.

Technical Field

The present invention relates to a system for speech separation and a method performed in the system, and in particular to a system and method for improving speech separation performance through a sliding window.

Background

In recent years, more and more vehicles have a voice recognition function. However, when more than one person speaks simultaneously in the vehicle, the host of the vehicle will not be able to quickly recognize the sound from the driver from the multiple voices, so that the corresponding operation cannot be accurately and timely performed according to the driver's instruction, and an erroneous operation is easily caused.

Currently, there are mainly two ways to perform speech separation. The first is to create microphone arrays for speech enhancement and the second is to use algorithms for speech separation. Various algorithms for speech separation may include FDICA (frequency domain independent component analysis), DUET (degradation separation estimation technique), or their extended algorithms.

The DUET blind source separation method can use only two hybrids to separate any number of speech sources. The method is effective when the sources are W disjoint orthogonal, i.e. when the support of the windowed fourier transform of the signals in the mix is disjoint. For anechoic mixing of attenuation and delay sources, the method allows for estimating the mixing parameters by clustering the relative attenuation-delay pairs extracted from the ratio of the mixed time-frequency representation. The estimate of the mixing parameters is then used to partition a mixed time-frequency representation to recover the original source.

Fig. 1 shows a conventional voice separation system comprising two microphones, a sound recording module and a DUET module. For example, two microphones are first turned on simultaneously so that both microphones start recording. When two persons start talking, the sound recording module is responsible for receiving and storing the speech signals from the two microphones. In the example shown in fig. 1, the first sound (sound 1) belongs to a first person (person 1) and the second sound (sound 2) belongs to a second person (person 2). The DUET module receives the signals from the sound recording module and then analyzes and separates the signals to recover the original sound source.

In practice, for example, if the time of the segmentation of speech is 4 seconds (such as shown in fig. 2 (a)), the DUET module will directly process the 4 second speech segment. Processing voice data will take a long time due to the complexity of the DUET algorithm. Typically, speech signals are sparse and concentrate a large amount of information in a very short period of time. Most of the time, no speech signal is present in the received signal. However, due to the complexity of the DUET algorithm, the DUET module still waits a certain period of time (such as the entire speech segment, 4s) and takes a longer time to process the received signal.

Accordingly, there is a need to develop an improved speech separation system and method that can quickly perform speech separation to quickly recover the original sound source.

Disclosure of Invention

In one or more illustrative embodiments, a method for speech separation is provided. The method acquires at least one voice from at least one user using at least one microphone and stores the at least one voice as a voice signal in a sound recording module. The method further extracts the speech signal from the sound recording module and processes the extracted speech signal through a sliding window, and transmits the processed speech signal to a DUET module for speech separation.

Preferably, in one embodiment, the method uses a sliding window by: traversing the extracted speech signal to determine a maximum amplitude of the speech signal; determining a start position of the sliding window, the start position of the sliding window being a position where an amplitude of the speech signal exceeds a predetermined proportion of the maximum amplitude for a first time from a start of the speech signal; determining an end position of the sliding window, the end position of the sliding window being a position from the end of the speech signal back to the beginning of the speech signal where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude; and selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

Preferably, in another embodiment, the method uses a sliding window by: traversing the extracted speech signal to determine an average amplitude of the speech signal; determining a start position of the sliding window, the start position of the sliding window being a position where an amplitude of the speech signal exceeds the average amplitude for a first time from a start of the speech signal; determining an end position of the sliding window, the end position of the sliding window being a position where the amplitude of the speech signal first exceeds the average amplitude from the end of the speech signal back to the beginning of the speech signal; selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

In one or more illustrative embodiments, a system for speech separation is provided. A system for speech separation comprising: at least one microphone for acquiring at least one voice from at least one user; a sound recording module to store the at least one voice as a voice signal; a sliding window for extracting the speech signal from the sound recording module and processing the extracted speech signal; and a DUET module for receiving the processed voice signals for voice separation.

Preferably, in one embodiment, the sliding window is configured to: traversing the extracted speech signal to determine a maximum amplitude of the speech signal; determining a start position of the sliding window, the start position of the sliding window being a position where an amplitude of the speech signal exceeds a predetermined proportion of the maximum amplitude for a first time from a start of the speech signal; determining an end position of the sliding window, the end position of the sliding window being a position from the end of the speech signal back to the beginning of the speech signal where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude; selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

Preferably, in another embodiment, the sliding window is configured to: traversing the extracted speech signal to determine an average amplitude of the speech signal; determining a start position of the sliding window, the start position of the sliding window being a position where an amplitude of the speech signal exceeds the average amplitude for a first time from a start of the speech signal; determining an end position of the sliding window, the end position of the sliding window being a position where the amplitude of the speech signal first exceeds the average amplitude from the end of the speech signal back to the beginning of the speech signal; selecting a segment of the speech signal between the start position of the sliding window and the end position of the sliding window as the processed speech signal for speech separation.

A computer-readable medium having computer-executable instructions for performing the foregoing method is provided.

Advantageously, the disclosed voice separation system and method can improve the real-time performance of the DUET by using a sliding window.

The systems, methods, features and advantages will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the present invention.

Drawings

The features, nature, and advantages of the application may be better understood with reference to the drawings and description that follow. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. Moreover, in the figures, like referenced numerals designate corresponding parts throughout the different views.

FIG. 1 is a schematic diagram of a conventional speech separation system.

FIG. 2 shows a schematic diagram of a speech separation system according to an embodiment of the invention.

Fig. 3 schematically shows a sliding window for use in a speech separation system according to an embodiment of the present invention.

Fig. 4 schematically shows a sliding window for use in a speech separation system according to another embodiment of the present invention.

FIG. 5 shows a flow diagram of a speech separation method according to an embodiment of the invention.

Detailed Description

It should be understood that the following description of implementation examples is given for illustrative purposes only and should not be taken in a limiting sense. The division of the examples in the figures by functional blocks, modules or units should not be construed as indicating that these functional blocks, modules or units are necessarily implemented as physically separate units. Functional blocks, modules, or units shown or described may be implemented as individual units, circuits, chips, functions, modules, or circuit elements. One or more of the functional blocks or units may also be implemented in a common circuit, chip, circuit element or unit.

FIG. 2 shows a schematic diagram of a speech separation system according to an embodiment of the invention. The voice separation system may be used in a vehicle and may include at least one microphone, a sound recording module, a sliding window module, and a DUET module. For ease of explanation, fig. 2 shows only two microphones (microphone 1 and microphone 2) and two persons (person 1 and person 2), but those skilled in the art will appreciate that the system may include more microphones. The two microphones may capture at least one voice from at least one user. Fig. 2 shows two persons as an example. For example, the two persons may be a driver and a passenger.

When the system is in operation, for example as shown in fig. 2, two microphones each pick up speech from two people. For example, a first microphone (microphone 1) may collect a first voice (sound 1) from a first person and a second voice (sound 2) from a second person, which are then transmitted to a sound recording module for recording as a voice signal mixing information from two sound sources. Likewise, a second microphone (microphone 1) may collect a first voice (sound 1) from a first person and a second voice (sound 2) from a second person, which are then transmitted to the sound recording module for recording as voice signals comprising information from both sound sources.

The sliding window module may extract a speech signal from the sound recording module and process the extracted speech signal by a sliding window. And then transmitting the processed voice signals to a DUET module for voice separation. Finally, different speech sources can be separated. For example, the processed speech signal may be ultimately separated into a first speech from a first person (sound 1) and a second speech from a second person (sound 2).

The sliding window will be explained with reference to fig. 3 and 4. Fig. 3 schematically shows a sliding window for use in a speech separation system according to an embodiment of the present invention.

For example, the extracted speech signal may last four seconds, as shown in FIG. 3. First, the extracted speech signal is traversed to determine the maximum amplitude of the speech signal. Then, the start position of the sliding window and the end position of the sliding window will be determined. From the beginning of the speech signal, a point is found (such as point X1 as shown in fig. 3). At point X1, the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude. Preferably, the predetermined ratio may be greater than or equal to 1/4 and less than or equal to 1/2. This point X1 is then determined as the starting position of the sliding window. Next, from the end of the speech signal to the beginning of the speech signal, a point is found (such as point X2 shown in fig. 3). At point X2, from the end of the speech signal, the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude. This point X2 is then determined as the end position of the sliding window. The window length of the sliding window may be determined based on the starting position of the sliding window and the ending position of the sliding window, i.e., the window length is equal to X2-X1 (as indicated by X in FIG. 3). Next, a section of the voice signal between the start position and the end position of the sliding window (i.e., a section within the sliding window) is selected as a processed voice signal and sent to the DUET for voice separation.

Fig. 4 schematically shows a sliding window for use in a speech separation system according to another embodiment of the present invention.

For example, fig. 4 shows an extracted speech signal that may also last four seconds. First, an average amplitude of the speech signal is determined by traversing the extracted speech signal. Then, the start position of the sliding window and the end position of the sliding window will be determined. From the beginning of the speech signal, a point is found (such as point X3 as shown in fig. 4). At point X3, the amplitude of the speech signal first exceeds the average amplitude of the speech signal. This point X3 is then determined as the starting position of the sliding window. Next, from the end of the speech signal to the start of the speech signal, a point is found (such as point X4 shown in fig. 4). At point X4, from the end of the speech signal, the amplitude of the speech signal first exceeds the average amplitude. This point X4 is then determined as the end position of the sliding window. The window length of the sliding window may be determined based on the starting position of the sliding window and the ending position of the sliding window, i.e., the window length is equal to X4-X3 (as indicated by X in FIG. 4). Next, a section of the voice signal between the start position and the end position of the sliding window (i.e., a section within the sliding window) is selected as a processed voice signal and sent to the DUET for voice separation.

FIG. 5 shows a flow diagram of a speech separation method according to an embodiment of the invention.

As shown in fig. 5, at least one voice from at least one user is acquired by at least one microphone and then stored as a voice signal in a sound recording module at step 501. At step 502, the speech signal transmitted from the voice recording module is further processed using a sliding window before being sent to the DUET module for speech separation. At step 503, the processed speech signal is transmitted to the DUET module.

Processing using a sliding window at step 502 may include determining a window length of the sliding window and selecting a segment of the speech signal that lies within the window length of the sliding window as the processed speech signal for further speech separation.

According to one embodiment of the present invention, determining the window length of the sliding window may include traversing the extracted speech signal to determine a maximum amplitude of the speech signal. Then, a start position of the sliding window and an end position of the sliding window are determined to obtain a window length of the sliding window. The start position of the sliding window is a position where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude from the start of the speech signal. The end position of the sliding window is a position from the end of the speech signal back to where the amplitude of the starting speech signal of the speech signal for the first time exceeds a predetermined proportion of the maximum amplitude. Preferably, the predetermined ratio may be greater than or equal to 1/4 and less than or equal to 1/2.

According to another embodiment of the present invention, determining the window length of the sliding window may comprise traversing the extracted speech signal to determine an average amplitude of the speech signal. Then, a start position of the sliding window and an end position of the sliding window are determined to obtain a window length of the sliding window. For example, the start position of the sliding window is a position where the amplitude of the speech signal first exceeds the average amplitude from the start of the speech signal. The end position of the sliding window is the position from the end of the speech signal back to the beginning of the speech signal where the amplitude of the speech signal first exceeds the average amplitude.

The voice separation method and system of the present invention introduces a sliding window to preprocess the data before sending the data collected by the microphone to the DUET module for processing. By extracting a relatively concentrated portion of the speech information in the segment of the signal and removing unnecessary portions of the segment signal, the amount of data that the DUET algorithm needs to process is reduced, thereby reducing the runtime of the DUET algorithm and improving the operating efficiency of the overall speech separation system.

The term "module" may be defined to include a plurality of executable modules. A module may include software executable by a processor, hardware, firmware, or some combination thereof. A software module may include instructions stored in memory or another storage device that may be executable by a processor or other processor. A hardware module may include various devices, components, circuits, gates, circuit boards, etc. that may be executed, directed, and/or controlled by a processor for execution.

One or more programs of the program product define functions of the embodiments (including the methods described herein) and can be contained on a variety of computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as compact disk read-only memory (CD-ROM) disks readable by a CD-ROM drive, flash memory, read-only memory (ROM) chips, or any type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory) on which information is alterable to be stored.

The invention has been described above with reference to specific embodiments. However, it will be appreciated by those skilled in the art that various modifications and changes may be made to the specific embodiments without departing from the broader spirit and scope of the invention as set forth in the appended claims.