阅读说明：本技术 声音信号的处理系统、方法及装置 (Sound signal processing system, method and device ) 是由 蔡正辉 苏辉 于 2021-08-02 设计创作，主要内容包括：本说明书实施例提供了一种声音信号的处理系统、方法及装置,声音信号的处理系统包括至少一个信号控制支路,信号控制支路包括：音频处理设备、分别与音频处理设备通信连接的第一麦克风及扬声器；第一麦克风,用于采集待处理的声音信号并发送至音频处理设备；音频处理设备,用于根据待处理的声音信号和当前保存的信号处理参数生成反相声音信号,发送反相声音信号至扬声器；其中,反相声音信号与待处理的声音信号相互抵消后的残留声音信号不大于预设的声音信号阈值；扬声器,用于播放反相声音信号。本说明书实施例,通过扬声器播放的反相声音信号与待处理的声音信号相互抵消,能够不依赖墙体结构的隔音效果,对开放式或半开放区域进行隔音。(The embodiment of the specification provides a system, a method and a device for processing a sound signal, wherein the system for processing the sound signal comprises at least one signal control branch, and the signal control branch comprises: the system comprises an audio processing device, a first microphone and a loudspeaker which are respectively in communication connection with the audio processing device; the first microphone is used for collecting a sound signal to be processed and sending the sound signal to the audio processing equipment; the audio processing equipment is used for generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters and sending the inverse sound signal to the loudspeaker; the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value; and the loudspeaker is used for playing the inverse sound signal. In the embodiment of the present description, the opposite-phase sound signal played by the speaker and the sound signal to be processed cancel each other out, so that the sound insulation can be performed on the open type or semi-open area without depending on the sound insulation effect of the wall structure.)

1. A system for processing sound signals, comprising at least one signal control branch, said signal control branch comprising: the system comprises an audio processing device, a first microphone and a loudspeaker, wherein the first microphone and the loudspeaker are respectively in communication connection with the audio processing device;

the first microphone is used for collecting a sound signal to be processed and sending the sound signal to the audio processing equipment;

the audio processing device is used for generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters, and sending the inverse sound signal to the loudspeaker; the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value;

the loudspeaker is used for playing the reversed phase sound signal.

2. The system of claim 1, wherein the signal control branch further comprises: a second microphone in communicative connection with the audio processing device;

the second microphone is used for collecting the residual sound signal and sending the residual sound signal to the audio processing equipment;

if the audio processing device determines that the received residual sound signal is greater than the sound signal threshold, adjusting the signal processing parameters according to a preset mode to obtain new signal processing parameters; and updating the currently stored signal processing parameters into the new signal processing parameters.

3. The system of claim 2, wherein the audio processing device is specifically configured to:

and gradually adjusting the signal processing parameters according to the residual sound signal, the sound signal collected by the first microphone in a specified time range and the reversed-phase sound signal until the residual sound signal is not greater than the sound signal threshold value.

4. The system of claim 1, wherein the signal processing parameters include filter weights; the audio processing device is specifically configured to:

generating an intermediate sound signal according to the sound signal to be processed and the filter weight;

and carrying out phase inversion processing on the intermediate sound signal to generate the phase-inverted sound signal.

5. The system of claim 2,

the sound emission source of the sound signal to be processed is positioned in the core area of the specified sound field;

the first microphone is arranged in the core area;

the loudspeaker is arranged in the boundary area of the specified sound field; the boundary region surrounds the core region;

the second microphone is arranged in the boundary area.

6. The system of claim 1,

aiming at any signal control branch circuit, a first distance between a sound source of the sound signal to be processed and the first microphone is smaller than a second distance between the sound source and the loudspeaker, and the second distance is smaller than a third distance between the sound source and the second microphone.

7. The system of claim 1, wherein the signal control branch comprises: a first microphone array and a speaker array; the speaker array comprises at least one first speaker and at least one second speaker;

the audio processing device is specifically configured to:

generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters, sending the inverse sound signal to the first loudspeaker, amplifying the sound signal to be processed to generate an amplified sound signal, and sending the amplified sound signal to the second loudspeaker; wherein a residual sound signal after the mutual cancellation of the inverse sound signal, the amplified sound signal and the sound signal to be processed is not greater than a preset sound signal threshold;

the first loudspeaker is used for playing the reversed-phase sound signal;

the second loudspeaker is used for playing the amplified sound signal.

8. The system of claim 2, wherein the first microphone and the second microphone comprise different microphones of a webcam IPC device; the speaker includes a speaker of the IPC device.

9. The processing method of the sound signal is characterized by being applied to a processing system of the sound signal, wherein the processing system of the sound signal comprises at least one signal control branch, and the signal control branch comprises an audio processing device, a first microphone and a loudspeaker which are respectively in communication connection with the audio processing device; the method comprises the following steps:

acquiring a to-be-processed sound signal acquired by the first microphone;

generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters;

sending the reversed-phase sound signal to the loudspeaker so that the loudspeaker plays the reversed-phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

10. The method of claim 9, wherein the signal control branch further comprises a second microphone, the method further comprising:

acquiring the residual sound signal collected by the second microphone;

if the residual sound signal is determined to be larger than the sound signal threshold, adjusting the signal processing parameter according to a preset mode to obtain a new signal processing parameter; and updating the currently stored signal processing parameters into the new signal processing parameters.

11. The method of claim 10, wherein adjusting the signal processing parameters according to a preset manner to obtain new signal processing parameters comprises:

and gradually adjusting the signal processing parameters according to the residual sound signal, the sound signal collected by the first microphone in a specified time range and the reversed-phase sound signal until the residual sound signal is not greater than a preset signal threshold.

12. The method of claim 9, wherein the signal processing parameters include filter weights; generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters, comprising:

generating an intermediate sound signal according to the sound signal to be processed and the filter weight;

and carrying out phase inversion processing on the intermediate sound signal to generate the phase-inverted sound signal.

13. The method of claim 9, wherein generating an inverse sound signal based on the sound signal to be processed and the currently stored signal processing parameters comprises:

generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters; amplifying the sound signal to be processed to generate an amplified sound signal;

the transmitting the inverse sound signal to the speaker includes:

sending the inverted sound signal to the first speaker and sending the amplified sound signal to the second speaker; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal, the amplified sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

14. The processing device of the sound signal is characterized by being applied to a processing system of the sound signal, wherein the processing system of the sound signal comprises at least one signal control branch, and the signal control branch comprises a first microphone and a loudspeaker; the device comprises:

a memory for storing signal processing parameters;

the processor is used for acquiring the sound signal to be processed collected by the first microphone; generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters; sending the reversed-phase sound signal to the loudspeaker so that the loudspeaker plays the reversed-phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

15. An electronic device, comprising: a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; a memory for storing a computer program; a processor for executing a program stored in the memory to implement the steps of the method for processing a sound signal according to any one of claims 9 to 13.

16. A computer-readable storage medium, characterized in that a computer program is stored thereon, which, when being executed by a processor, implements the steps of the method of processing a sound signal according to any one of the preceding claims 9 to 13.

Technical Field

The present disclosure relates to the field of processing of audio signals, and in particular, to a system, a method and an apparatus for processing an audio signal.

Background

When a person is performing an activity in a particular area, a large amount of active speech may be generated. For example, when a plurality of participants meet in an open conference room, there may be voices generated by the plurality of participants speaking, or voices played by an audio/video playing device. In a home scene, for example, when a family member speaks indoors, a loud voice may be generated. The active voice generated in a specific area may interfere with people outside the specific area, and affect the normal work and life of the people. And, for active speech or privacy with privacy requirements, it is not suitable to be heard by people outside the specific area. Therefore, how to insulate sound in a specific area is an urgent problem to be solved.

Disclosure of Invention

An object of the embodiments of the present disclosure is to provide a system, a method and a device for processing sound signals, so as to solve the problem of how to insulate sound in an open or semi-open area.

In order to solve the above technical problem, the embodiments of the present specification are implemented as follows:

in a first aspect, an embodiment of the present specification provides a system for processing a sound signal, including at least one signal control branch, where the signal control branch includes: the system comprises an audio processing device, a first microphone and a loudspeaker, wherein the first microphone and the loudspeaker are respectively in communication connection with the audio processing device;

the first microphone is used for collecting a sound signal to be processed and sending the sound signal to the audio processing equipment;

the audio processing device is used for generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters, and sending the inverse sound signal to the loudspeaker; the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value;

the loudspeaker is used for playing the reversed phase sound signal.

In a second aspect, an embodiment of the present specification provides a processing method for a sound signal, which is applied to a processing system for a sound signal, where the processing system for a sound signal includes at least one signal control branch, where the signal control branch includes an audio processing device, a first microphone and a speaker, and the first microphone and the speaker are respectively connected to the audio processing device in a communication manner; the method comprises the following steps:

acquiring a to-be-processed sound signal acquired by the first microphone;

generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters;

sending the reversed-phase sound signal to the loudspeaker so that the loudspeaker plays the reversed-phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

In a third aspect, an embodiment of the present specification provides a processing apparatus for a sound signal, which is applied to a processing system for a sound signal, where the processing system for a sound signal includes at least one signal control branch, where the signal control branch includes a first microphone and a speaker; the device comprises:

a memory for storing signal processing parameters;

the processor is used for acquiring the sound signal to be processed collected by the first microphone; generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters; sending the reversed-phase sound signal to the loudspeaker so that the loudspeaker plays the reversed-phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

In a fourth aspect, an embodiment of the present specification provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; a memory for storing a computer program; and a processor for executing the program stored in the memory to implement the steps of the method for processing the sound signal according to the second aspect.

In a fifth aspect, the present specification provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the sound signal processing method of the second aspect.

In an embodiment of the present description, a system for processing sound signals comprises at least one signal control branch, the signal control branch comprising: the system comprises an audio processing device, a first microphone and a loudspeaker which are respectively in communication connection with the audio processing device; the first microphone is used for collecting a sound signal to be processed and sending the sound signal to the audio processing equipment; the audio processing equipment is used for generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters and sending the inverse sound signal to the loudspeaker; the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value; and the loudspeaker is used for playing the inverse sound signal. Therefore, the reversed-phase sound signal is generated based on the signal processing parameters and played through the loudspeaker, so that the reversed-phase sound signal can be mutually offset with the sound signal to be processed in a propagation state in the process of propagation. For the open type or semi-open type area, the problem that the interference of the active voice generated in the open type or semi-open type area to the personnel outside the area is caused is avoided; moreover, for the active voice with the privacy requirement, the privacy protection is realized. On the basis of not relying on the wall structure to carry out syllable-dividing promptly, realized effectively giving sound insulation. In addition, the number of the signal control branches can be set according to the sound insulation requirement, so that the sound insulation control system has the characteristics of high designability, high flexibility and the like.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.

Fig. 1 is a first structural diagram of a system for processing an acoustic signal according to an embodiment of the present disclosure;

fig. 2 is a second structural diagram of a system for processing an audio signal according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of a third structure of a system for processing an acoustic signal according to an embodiment of the present disclosure;

FIG. 4 is a schematic view of an open office space provided by an embodiment of the present disclosure;

fig. 5 is a schematic diagram of a fourth structure of a system for processing an acoustic signal according to an embodiment of the present disclosure;

fig. 6 is a schematic diagram of an open conference room provided in an embodiment of the present disclosure;

fig. 7 is a flowchart illustrating a method for processing an audio signal according to an embodiment of the present disclosure;

fig. 8 is a schematic block diagram of an apparatus for processing an audio signal according to an embodiment of the present disclosure;

fig. 9 is a schematic composition diagram of an electronic device provided in an embodiment of the present specification.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In traditional scenes such as official working, meeting and house, adopt wall structure to realize syllable-dividing effect usually, for example, in closed meeting room, utilize the better building material of syllable-dividing effect and construction mode to build the wall structure around the meeting room to the meeting sound that personnel sent in isolated meeting room avoids personnel outside the meeting room to hear the meeting sound that produces in the meeting room.

The open conference room is taken as an example for explanation, the open conference room has a new structure of wall-free office, so that the distances between employees and leaders are reduced, and the conference efficiency can be improved. In addition, compared with the traditional closed meeting room, the open meeting room reduces the required office area and saves the construction cost. However, open meeting room can't give sound insulation between meeting indoor space and meeting outdoor space hardly, lacks wall structure's separation and has leaded to produced sound to propagate to meeting outdoor space in the meeting room, and then leaks privacy, when revealing meeting sound, also causes the office staff outside the meeting room to receive interference such as conference conversation, reduces work efficiency.

Therefore, in the scenes of open office, conference, home and the like, in order to prevent the sound generated in the open or semi-open area from being perceived by the people outside the area and to prevent the people outside the open or semi-open area from being interfered by the sound generated in the area, the embodiment of the specification provides a processing system of the sound signal.

Fig. 1 is a schematic diagram of a first structure of a system for processing an audio signal according to an embodiment of the present disclosure.

Referring to fig. 1, the system for processing sound signals includes at least one signal control branch 102 (only one is shown in fig. 1), and the signal control branch 102 includes: an audio processing device 1024, a first microphone 1022 and a speaker 1026 communicatively coupled to the audio processing device 1024, respectively.

The first microphone 1022 is configured to collect a sound signal to be processed and send the sound signal to the audio processing device 1024;

the audio processing device 1024 is configured to generate an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameter, and send the inverse sound signal to the speaker 1026; the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value;

a speaker 1026 for playing the inverted sound signal.

The first microphone 1022 may be an analog microphone or a digital microphone.

The sound signal to be processed may be a sound signal generated in a designated sound field where there is a need for sound insulation, for example, a voice signal of a conference sound emitted by a participant in an open conference room. The conference sound can be the sound of speaking of a person, the sound of moving an object, or the sound of playing audio software by the electronic device, and the description does not specially limit the type of the sound signal to be processed.

Acoustic field, i.e. the space in which sound waves propagate. The aforementioned designated sound field may be an open or semi-open area where there is a significant sound insulation requirement, such as an open conference room. In addition to open or semi-open areas, the designated sound field may also be a closed area that inherently provides poor sound insulation. For example, activities such as meeting at night in a home environment exist, singing and loud chatting exist in the period, and noise generated by excitation influences the rest of neighbors to a certain extent, so that the neighborhood relationship is reduced. Although the area where the night party is in the home environment may be isolated from the neighboring houses and does not belong to an open or semi-open area, the noise generated by the night party may penetrate through the wall to interfere with the normal working and life of the neighbors, and the original wall structure may not meet the sound insulation requirement. The sound field in the embodiment of the present application may include a core region and a boundary region surrounding the core region, as can be referred to in the following description.

The audio processing device 1024 may be a DSP (Digital Signal processing) module, a processor with a Digital Signal processing function, or a device with a Digital Signal processing function (e.g., a terminal device such as a desktop computer, a server, or the like). The audio processing device 1024 has stored therein signal processing parameters. The signal processing parameters are used to process the sound signals received by the audio processing device 1024.

The audio processing device 1024 may be connected in wireless communication with the first microphone 1022 or may be connected in wired communication. The audio processing device 1024 may be connected in wireless communication with the speaker 1026 or may be connected in wired communication.

Optionally, the signal processing parameters include filter weights; the audio processing device 1024 is specifically configured to: generating an intermediate sound signal according to the sound signal to be processed and the filter weight; the intermediate audio signal is subjected to an inversion process to generate an inverted audio signal.

In general, there is instability in the voice of a person speaking, and the voice greatly fluctuates depending on the content, emotion, and the like. Therefore, no matter the first microphone collects only one person speaking voice, or collects the voice mixed by the voices of a plurality of persons speaking, or collects the voice mixed by the voices of the persons speaking and other voices, the collected voice signal to be processed has instability, the voice signal is difficult to express through a fixed mathematical formula, and further the voice signal to be processed is difficult to directly carry out reverse phase processing.

In this embodiment, the audio processing device 1024 stores signal processing parameters including filter weights h_AF(t) of (d). Filter weight h_AF(t) for processing the sound signal to be processed to obtain an intermediate sound signal, which may be represented by an approximate mathematical expression, and the audio processing device 1024 performs an inverse phase processing on the intermediate sound signal to obtain an inverse sound signal, which implements the mathematical expression of the sound signal.

Assuming that the sound signal to be processed is x (t), x (t) is input to the audio processing apparatus 1024, and the audio processing apparatus 1024 calculates the inverse sound signal b (t) by the following formula.

b(t)＝-h_AF(t)*x(t) (1)

The filter weights h stored in the audio processing device 1024 at the beginning of the start-up of the processing system of the sound signal_AF(t) may be a fixed value that is preset empirically. In the subsequent steps, the filter weight h_AF(t) may be adaptively adjusted through feedback.

And the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value. Considering theoretical values, the processing system of the sound signal expects to completely cancel the inverse sound signal and the sound signal to be processed, that is, the value of the residual sound signal is 0. However, in the actual operation process, the probability of complete cancellation is low, and when the value of the residual sound signal is not greater than the preset sound signal threshold, the human ear can hardly perceive the residual sound signal, so that when the residual sound signal is not greater than the preset sound signal threshold, the processing system of the sound signal can also achieve a good sound cancellation effect, and further achieve effective sound insulation in an open type or semi-open type area.

Optionally, the signal control branch 102 further includes: a second microphone 1028 communicatively connected to the audio processing device 1024; a second microphone 1028, configured to collect a residual sound signal and send the residual sound signal to the audio processing device 1024; the audio processing device 1024, if determining that the received residual sound signal is greater than the sound signal threshold, adjusting the signal processing parameters according to a preset mode to obtain new signal processing parameters; and updating the currently stored signal processing parameters into new signal processing parameters.

The second microphone 1028 can be an analog microphone or a digital microphone other than the first microphone 1022. The first microphone 1022 and the second microphone 1028 may be of the same type or different types.

Optionally, for any signal control branch 102, a first distance between a sound source of the sound signal to be processed and the first microphone 1022 is smaller than a second distance between the sound source and the speaker 1026, and the second distance is smaller than a third distance between the sound source and the second microphone 1028.

The operation of the processing system including the sound signal of the second microphone 1028 can be described in detail with reference to fig. 2 and 3.

Fig. 2 is a schematic diagram of a second structure of a system for processing an audio signal according to an embodiment of the present disclosure.

Referring to fig. 2, in the signal control branch 102, a first distance between the sound source of the sound signal and the first microphone 1022 is smaller than a second distance between the sound source of the sound signal and the speaker 1026, and the second distance is smaller than a third distance between the sound source of the sound signal and the second microphone 1028. Specifically, the sound signal generated by the sound emission source located in the core area is propagated to the first microphone 1022 through the propagation path 1. The first microphone 1022 transmits the collected sound signal to the audio processing device 1024. The audio processing device 1024 generates and sends an inverse sound signal to the speaker 1026. The inverted sound signal played by the speaker 1026 is canceled with the sound signal propagated through the propagation path 2 in the air while propagating toward the second microphone 1028. The second microphone 1028 collects a residual sound signal after the cancellation of the inverted sound signal played by the speaker 1026 and the sound signal propagated through the propagation path 2 in the air, and sends the residual sound signal as an error feedback to the audio processing device 1024.

Due to the different positions of the first microphone 1022 and the second microphone 1028, the sound signal to be processed, which is emitted from the same sound emission source, for example, the sound signal travels a short distance to the first microphone 1022 through the propagation path 1, and the sound signal travels a long distance to the second microphone 1028 through the propagation path 2 in the air, and at this time, the sound signal travels to the second microphone 608 with a part of attenuation. The objective that the embodiments of this specification aim to achieve is to make the residual sound signal collected by the second microphone 1028 as small as possible, so that the opposite-phase sound signal played by the speaker 1026 is ideally the same as the attenuated sound signal collected by the second microphone 1028 in amplitude and opposite in phase, thereby achieving a good sound cancellation effect.

It should be noted that the first microphone 1022, the speaker 1026, and the second microphone 1028 belonging to the same signal control branch 102 may be arranged in a straight line, or may be arranged at a position convenient for installing the apparatus according to the environment of the specified sound field.

Fig. 3 is a schematic diagram of a third structure of a system for processing an audio signal according to an embodiment of the present disclosure.

Referring to fig. 3, the sound signal is propagated to the first microphone 1022. The first microphone 1022 transmits the sound signal to the audio processing device 1024. The audio processing device 1024 has stored therein signal processing parameters. The audio processing device 1024 generates an inverse sound signal from the sound signal and the stored signal processing parameters, and sends the inverse sound signal to the speaker 1026. The speaker 1026 plays the inverted sound signal. The inverted sound signal and the sound signal are mixed and then cancelled out, and the second microphone 1028 collects a residual sound signal obtained by the cancellation of the two, and sends the residual sound signal to the audio processing device 1024, so as to adjust the signal processing parameters stored in the audio processing device 1024.

When the audio processing device 1024 determines that the received residual sound signal is greater than the preset sound signal threshold, it may be understood that the cancellation effect of the inverse sound signal and the sound signal is poor, and at this time, it indicates that the signal processing parameter needs to be adjusted. The audio processing device 1024 updates the stored signal processing parameters in real-time according to the residual sound signal fed back by the second microphone 1028.

It should be noted that although there is a hysteresis in the algorithm for the audio processing device 1024 to adjust the signal processing parameters according to the received residual sound signal fed back by the second microphone 1028, the time consumed for updating the signal processing parameters once is very short, for example, the audio processing device 1024 can update the signal processing parameters once only by several tens of milliseconds. Whereas delays of tens of milliseconds are difficult to capture by the perception capabilities of the human ear. The sound collected within several tens of milliseconds can be generally regarded as a sound signal with a smooth waveform, so even if the signal processing parameters are obtained by the audio processing device 1024 according to feedback calculation of the sound signal before several tens of milliseconds, the signal processing parameters can still be used to generate an inverse sound signal capable of obtaining a good sound cancellation effect.

Optionally, the audio processing device 1024 is specifically configured to: according to the residual sound signal, the sound signal collected by the first microphone 1022 in the designated time range, and the inverse sound signal, the signal processing parameters are adjusted gradually until the residual sound signal is not greater than the preset signal threshold.

The audio processing device 1024 adjusts and processes the signal processing parameters according to a preset mode to obtain new signal processing parameters, including: the audio processing device 1024 adjusts the signal processing parameters successively according to the residual sound signal, the sound signal collected by the first microphone 1022 in the specified time range, and the inverse sound signal until the residual sound signal is not greater than the preset signal threshold.

The audio processing device 1024 stores therein signal processing parameters including filter weights h_AF(t), the filter weights h are updated in the following_AF(t) is an example illustrating how to adjust the signal processing parameters:

updating the filter weight h_AF(t) the following program language can be adopted, and the program language comprises a calculation formula to be circularly processed:

for(k＝-N，k≤L，k++){h_AF(t)＝b(t)-μe(t)x(t-k)}

and k is a loop variable in a loop statement of the program language, N and L are preset time parameters, and a specified time range can be determined according to N and L, so that the sound signal collected by the first microphone in the specified time range is determined. b (t) is the inverse sound signal generated by the audio processing device 1024, and μ is a preset coefficient, which can be preset or adjusted as needed. t is the time argument related to the sound signal to be processed, the inverted sound signal and the filter weights.

According to the residual sound signal, the sound signal collected by the first microphone in the appointed time range and the reversed-phase sound signal, the signal processing parameters are adjusted gradually until the residual sound signal is not larger than the sound signal threshold value, the longer the appointed time range determined by N and L is, the more data representing the received sound signal to be processed is, the more processable information is, the wider the frequency band range of processing can be expanded, and the processing capacity of the algorithm on high-frequency information is improved.

The system for processing a sound signal provided in the embodiments of the present disclosure may include one signal control branch 102, or may include a plurality of signal control branches 102. The more the number of signal control branches 102 included in the processing system of the sound signal is, the denser the arrangement positions of the signal control branches 102 are, the better the sound insulation effect is, but the more the number is, the resource waste may be caused, and the arrangement cost of the processing system of the sound signal is increased. Therefore, the number and arrangement positions of the signal control branches 102 included in the processing system of the sound signal can be determined according to the specified sound field and the signal control range corresponding to each signal control branch 102.

The following may describe, with reference to fig. 4 and 5, the number and arrangement positions of the signal control branches 102 included in the processing system for determining the sound signal according to the specified sound field and the signal control range corresponding to each signal control branch 102.

Fig. 4 is a schematic diagram of an open office space provided in an embodiment of the present disclosure.

Referring to fig. 4, the designated sound field may be a conference area, where the right side of the conference area is a wall, the upper and lower sides are aisles, and the left side is an office area. According to the appointed sound field conference area, signal control branches can be determined to be arranged on the upper side, the lower side and the left side of the conference area, and therefore the sound insulation effect is achieved. The wall on the right side of the conference area has a certain sound insulation function, so that a signal control branch is not required to be arranged on the right side of the conference area.

Fig. 5 is a schematic diagram of a fourth structure of a system for processing an acoustic signal according to an embodiment of the present disclosure.

Referring to fig. 5, the sound field is designated as an open or semi-open area, and signal control branches 102 need to be arranged around the core area of the designated sound field. 6 signal control branches 102 are arranged aiming at the core area and are respectively arranged at the following six positions: right side of core area, right lower side of core area, left upper side of core area, right upper side of core area.

The signal control branch 102 shown in the figure includes a first microphone 1022, a speaker 1026 and a second microphone 1028. The other signal control branches are similar to the devices included in the signal control branch 102, and are not described herein again.

When a side of a given sound field, which is open to the outside of the given sound field when it has no wall structure, is easily propagated toward the side, one or more signal control branches 102 may be arranged at the side in order to avoid sound leakage.

On the side of the specified sound field without the wall structure, the number and the arrangement positions of the signal control branches 102 included in the processing system of the sound signal are determined according to the signal control range corresponding to each signal control branch 102, for example, if the signal control range corresponding to one signal control branch 102 is two meters, one signal control branch 102 is arranged on one side of the specified sound field every two meters.

Optionally, the signal control branch 102 comprises: a first microphone array and a speaker array; the speaker array comprises at least one first speaker and at least one second speaker; the audio processing device 1024 is specifically configured to: generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters, sending the inverse sound signal to a first loudspeaker, amplifying the sound signal to be processed to generate an amplified sound signal, and sending the amplified sound signal to a second loudspeaker; the residual sound signal after the mutual cancellation of the reversed-phase sound signal, the amplified sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value; a first speaker for playing the inverted sound signal; and the second loudspeaker is used for playing the amplified sound signal.

A microphone array refers to a system composed of a certain number of microphones for processing the spatial characteristics of a sound field. A speaker array refers to a system consisting of a certain number of speakers for playing sound signals. The signal control branch may further comprise a second microphone array.

The first microphone 1022 in various embodiments of the present description may be replaced with a first microphone array, the speaker 1026 may be replaced with a speaker array, and the second microphone 1028 may be replaced with a second microphone array. In specific implementation, the microphone array and the loudspeaker array can achieve better sound signal processing effect. The first loudspeaker and the second loudspeaker play different roles in the system, and can be the same type of loudspeaker or different types of loudspeakers from the viewpoint of equipment; the first microphone 1022 and the second microphone 1028 play different roles in the present system, and may be the same type of microphone or different types of microphones from the viewpoint of the device; the first microphone array and the second microphone array are the same, and the description is omitted here.

With the loudspeaker array, the reversed phase sound signal can be played through the first loudspeaker, and the sound signal can be amplified through the second loudspeaker. In a conference scenario, the sound signal may be amplified to enable the participants in the conference room to better hear what the speaking person is saying. In the scene, the residual sound signal after the phase-reversed sound signal, the amplified sound signal and the sound signal to be processed are mutually offset is not larger than a preset sound signal threshold value, namely the amplified sound signal and the sound signal to be processed are mixed and have the same amplitude and opposite phase with the phase-reversed sound signal.

Optionally, the first microphone 1022 and the second microphone 1028 comprise different microphones of a webcam IPC device; the speakers include speakers of the IPC device.

IPC (IP Camera) is a new generation Camera generated by combining a conventional Camera and a network technology. As a new monitoring camera, the system completely gets rid of the constraint of simulation monitoring and has the advantages of good expansibility, strong centralized management capability, low construction cost and the like. Through the distributed layout of a plurality of IPC devices, a certain 'quiet' area can be formed, and the area has the characteristics of privacy protection and low noise and can meet the requirements of offices, families and the like. IPC devices may include microphones, speakers, audio processing devices.

The first microphone may be a microphone of the IPC device, the first microphone array, or may be formed by a plurality of microphones included in at least one set of IPC device. The second microphone is similar to the first microphone and will not be described herein. The loudspeaker can be the loudspeaker of the IPC equipment, and the loudspeaker array can be formed by a plurality of sets of loudspeakers included in the IPC equipment.

The first microphone and the second microphone comprise different microphones of the webcam IPC device, e.g. the IPC device comprises a microphone 1 and a microphone 2, the first microphone may comprise the microphone 1 and the second microphone may comprise the microphone 2.

In the embodiment of the present specification, a sound signal processing system may be configured by combining an independent microphone, an independent speaker, an independent audio processing device, a microphone of the IPC device, a speaker of the IPC device, and the like.

Optionally, the sound emission source of the sound signal to be processed is positioned in the core area of the specified sound field; the first microphone 1022 is disposed in the core region; speakers 1026 are provided in the boundary region of the specified sound field; the boundary area surrounds the core area; the second microphone 1028 is disposed at the boundary region.

Fig. 6 is a schematic diagram of an open conference room provided in an embodiment of the present disclosure. Referring to fig. 6, a core region 602 of a specified sound field is surrounded by a boundary region 604 of the specified sound field.

The first microphone 1022 may be disposed at a core region of a designated sound field, the speaker 1026 may be disposed at a boundary region of the designated sound field, and the second microphone 1028 may be disposed at a boundary region of the designated sound field. Here, the digital signal processor 604, i.e., an audio processing device, may be disposed between the first microphone 1022 and the speaker 1026.

In particular implementations, for example, the designated sound field includes a conference room; the first microphone 1022 is disposed on a conference table in a core area of a conference room; a speaker 1026 is suspended from the ceiling of the border area of the conference room facing a second microphone 1028; a second microphone 1028 is suspended from the ceiling facing the speaker 1026.

When the sound field is designated as a conference room, the first microphone 1022 may be disposed in a core area of the conference room, has a small size and occupies no space, and may be mounted on a conference table for receiving sound signals; a speaker 1026, which can be installed in the border area of the conference room, and can be fixed by hanging on the ceiling, for playing the inverse sound signal; the second microphone 1028 may be disposed in the boundary region, suspended from the ceiling, and configured to receive the residual sound signal generated by the cancellation of the sound signal from the core region and the inverted sound signal played by the speaker 1026, and feed the residual sound signal back to the audio processing device 1024.

In the embodiment of the present specification, the processing system of the sound signal includes at least one signal control branch 102, and the signal control branch 102 includes: an audio processing device 1024, a first microphone 1022 and a speaker 1026 communicatively coupled to the audio processing device 1024, respectively; the first microphone 1022 is configured to collect a sound signal to be processed and send the sound signal to the audio processing device 1024; the audio processing device 1024 is configured to generate an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameter, and send the inverse sound signal to the speaker 1026; the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value; a speaker 1026 for playing the inverted sound signal. Therefore, the reversed-phase sound signal is generated based on the signal processing parameters and played through the loudspeaker, so that the reversed-phase sound signal can be mutually offset with the sound signal to be processed in a propagation state in the process of propagation. For the open type or semi-open type area, the problem that the interference of the active voice generated in the open type or semi-open type area to the personnel outside the area is caused is avoided; moreover, for the active voice with the privacy requirement, the privacy protection is realized. On the basis of not relying on the wall structure to carry out syllable-dividing promptly, realized effectively giving sound insulation. In addition, the number of the signal control branches can be set according to the sound insulation requirement, so that the sound insulation control system has the characteristics of high designability, high flexibility and the like.

Based on the same technical concept, the embodiments of the present specification further provide a processing method of a sound signal, which can be applied to the processing system of the sound signal provided by the foregoing embodiments.

Fig. 7 is a flowchart illustrating a method for processing a sound signal according to an embodiment of the present disclosure, where as shown in fig. 7, the method may include:

s702, acquiring a to-be-processed sound signal collected by a first microphone.

The sound signal to be processed may be a sound signal generated within a given sound field where there is a need for sound insulation, for example, a conference sound emitted by a participant in an open conference room. The conference sound can be the sound of speaking of a person, the sound of moving an object, or the sound of playing audio software by the electronic device, and the description does not specially limit the type of the sound signal to be processed.

S704, according to the sound signal to be processed and the currently stored signal processing parameters, generating an inverse sound signal.

The audio processing apparatus stores therein signal processing parameters for processing the sound signal to be processed to obtain an intermediate sound signal that can be expressed by an approximate mathematical expression, and performing an inversion process on the intermediate sound signal to obtain an inverted sound signal.

Optionally, the signal processing parameters include filter weights; correspondingly, in S704, generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameter includes: generating an intermediate sound signal according to the sound signal to be processed and the filter weight; the intermediate audio signal is subjected to an inversion process to generate an inverted audio signal.

In general, there is instability in the voice of a person speaking, and the voice greatly fluctuates depending on the content, emotion, and the like. Therefore, no matter the first microphone collects only one person speaking voice, or collects the voice mixed by the voices of a plurality of persons speaking, or collects the voice mixed by the voices of the persons speaking and other voices, the collected voice signal to be processed has instability, the voice signal is difficult to express through a fixed mathematical formula, and further the voice signal to be processed is difficult to directly carry out reverse phase processing.

In the embodiment of the present specification, the audio processing device stores signal processing parameters, and the signal processing parameters include filter weights h_AF(t) of (d). Filter weight h_AF(t) processing the sound signal to be processed to obtain an intermediate sound signal, the intermediate sound signal being representable by an approximate mathematical expression, the audio processing device performing an inverse phase processing on the intermediate sound signal to obtain an inverse sound signal, the mathematical expression of the sound signal being achieved.

Optionally, generating an inverse sound signal according to the sound signal to be processed and the currently saved signal processing parameter includes: generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters; amplifying the sound signal to be processed to generate an amplified sound signal; transmitting the inverted sound signal to a speaker, comprising: sending the inverted sound signal to a first speaker and sending the amplified sound signal to a second speaker; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal, the amplified sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

The sound signal to be processed is amplified, which may be according to a preset amplification ratio, to obtain an amplified sound signal. With the loudspeaker array, the reversed phase sound signal can be played through the first loudspeaker, and the sound signal can be amplified through the second loudspeaker.

In a conference scenario, the sound signal may be amplified to enable the participants in the conference room to better hear what the speaking person is saying. In the scene, the residual sound signal after the residual reversed-phase sound signal, the amplified sound signal and the sound signal to be processed are mutually offset is not larger than a preset sound signal threshold value, namely the amplified sound signal and the sound signal to be processed are mixed and have equal amplitude and opposite phase with the reversed-phase sound signal.

S706, sending the reversed phase sound signal to a loudspeaker so that the loudspeaker plays the reversed phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

In a specific implementation, the audio processing device may send the generated inverse sound signal to the speaker, so that the speaker plays the inverse sound signal.

And the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value. Considering theoretical values, the processing system of the sound signal expects to completely cancel the inverse sound signal and the sound signal to be processed, that is, the value of the residual sound signal is 0. However, in the actual operation process, the probability of complete cancellation is very low, and when the value of the residual sound signal is not greater than the preset sound signal threshold value, the human ear can hardly perceive the residual sound signal, and a good sound cancellation effect can be achieved at the moment, so that the sound insulation of an open type or a semi-open type area is realized.

Optionally, the signal control branch further includes a second microphone, and the processing method of the sound signal further includes: acquiring a residual sound signal collected by a second microphone; if the residual sound signal is determined to be larger than the sound signal threshold, adjusting the signal processing parameters according to a preset mode to obtain new signal processing parameters; and updating the currently stored signal processing parameters into new signal processing parameters.

If it is determined that the received residual sound signal is greater than the sound signal threshold, it can be understood that the cancellation effect of the inverse sound signal and the sound signal is poor, and at this time, it indicates that the signal processing parameters need to be adjusted. The signal processing parameters stored in the dsp module 704 are updated in real time according to the residual sound signal fed back from the second microphone.

It should be noted that although there is hysteresis in terms of the algorithm, in which the audio processing device adjusts the filter weights according to the received residual sound signal fed back by the second microphone, the algorithm is very fast in computation speed, for example, updated filter weights can be computed only by tens of milliseconds, and the collected new sound signal to be processed is processed according to the updated filter weights. Whereas delays of tens of milliseconds are difficult to capture by the perception capabilities of the human ear. The sound collected within tens of milliseconds can be generally regarded as a smooth sound signal with small difference, so even if the filter weight is calculated according to the feedback of the sound signal before tens of milliseconds, the filter weight can still be used for generating an anti-phase sound signal with good sound cancellation effect.

Optionally, adjusting the signal processing parameter according to a preset mode to obtain a new signal processing parameter, including: and gradually adjusting the signal processing parameters according to the residual sound signal, the sound signal collected by the first microphone in the appointed time range and the reversed-phase sound signal until the residual sound signal is not greater than a preset signal threshold value.

According to the residual sound signal, the sound signal collected by the first microphone in the appointed time range and the reversed-phase sound signal, the signal processing parameters are adjusted gradually until the residual sound signal is not larger than the sound signal threshold value, the longer the appointed time range is, the more information can be processed, the processing frequency band range can be expanded, and the processing capacity of the algorithm on high-frequency information is improved.

In the processing method of the sound signal provided in the embodiment of the present specification, a sound signal to be processed acquired by a first microphone is acquired; generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters; sending the reversed phase sound signal to a loudspeaker so that the loudspeaker plays the reversed phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value. Therefore, the reversed-phase sound signal is generated based on the signal processing parameters and played through the loudspeaker, so that the reversed-phase sound signal can be mutually offset with the sound signal to be processed in a propagation state in the process of propagation. For the open type or semi-open type area, the problem that the interference of the active voice generated in the open type or semi-open type area to the personnel outside the area is caused is avoided; moreover, for the active voice with the privacy requirement, the privacy protection is realized. On the basis of not relying on the wall structure to carry out syllable-dividing promptly, realized effectively giving sound insulation. In addition, the number of the signal control branches can be set according to the sound insulation requirement, so that the sound insulation control system has the characteristics of high designability, high flexibility and the like.

For the above method embodiment, since it is basically similar to the system embodiment, the description is simple, and the relevant points can be referred to the partial description of the system embodiment.

Based on the same technical concept, an embodiment of the present specification further provides a processing apparatus for a sound signal, and fig. 8 is a schematic block diagram of the processing apparatus for a sound signal provided by the embodiment of the present specification, as shown in fig. 8, the apparatus includes:

a memory 802 for storing signal processing parameters;

the processor 804 is configured to acquire a to-be-processed sound signal collected by the first microphone; generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters; sending the reversed phase sound signal to a loudspeaker so that the loudspeaker plays the reversed phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

Optionally, the processor 804 is further configured to:

acquiring a residual sound signal collected by a second microphone;

if the residual sound signal is determined to be larger than the sound signal threshold, adjusting the signal processing parameters according to a preset mode to obtain new signal processing parameters; and updating the currently stored signal processing parameters into new signal processing parameters.

Optionally, the processor 804 is specifically configured to:

and gradually adjusting the signal processing parameters according to the residual sound signal, the sound signal collected by the first microphone in the appointed time range and the reversed-phase sound signal until the residual sound signal is not greater than a preset signal threshold value.

Optionally, the signal processing parameters include filter weights; the processor 804 is specifically configured to:

generating an intermediate sound signal according to the sound signal to be processed and the filter weight;

the intermediate audio signal is subjected to an inversion process to generate an inverted audio signal.

The processing device for the sound signal provided by the embodiment of the specification acquires the sound signal to be processed collected by the first microphone; generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters; sending the reversed phase sound signal to a loudspeaker so that the loudspeaker plays the reversed phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value. Therefore, the reversed-phase sound signal played by the loudspeaker and the sound signal to be processed are mutually offset, and the open type or semi-open area can be insulated without depending on the sound insulation effect of the wall structure.

In addition, as for the embodiment of the processing device of the sound signal, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment. Further, it should be noted that, of the respective components of the sound signal processing apparatus of the present invention, the components thereof are logically divided according to the functions to be realized, but the present invention is not limited thereto, and the respective components may be newly divided or combined as necessary.

Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, and referring to fig. 9, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required by other services. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the processing device of the sound signal on the logic level. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.

The network interface, the processor and the memory may be interconnected by a bus system. The bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 9, but this does not indicate only one bus or one type of bus.

The memory is used for storing programs. In particular, the program may include program code including computer operating instructions. The memory may include both read-only memory and random access memory, and provides instructions and data to the processor. The Memory may include a Random-Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory.

The processor is used for executing the program stored in the memory and specifically executing the following steps:

acquiring a sound signal to be processed collected by a first microphone;

generating an inverse sound signal according to the sound signal to be processed and the currently stored signal processing parameters;

sending the reversed phase sound signal to a loudspeaker so that the loudspeaker plays the reversed phase sound signal; and the residual sound signal after the mutual cancellation of the reversed-phase sound signal and the sound signal to be processed is not greater than a preset sound signal threshold value.

The method performed by the processing apparatus for processing the sound signal according to the embodiment shown in fig. 8 of the present application may be applied to or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in a hardware decoding processor, or in a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

Based on the same technical concept, the present specification embodiment also provides a computer-readable storage medium storing one or more programs, which when executed by an electronic device including a plurality of application programs, cause the electronic device to perform the processing method of sound signals provided by the foregoing method embodiment.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.