阅读说明：本技术 重放攻击的检测 (Detection of replay attacks ) 是由 J·P·莱索 C·阿朗索 E·C·维克斯 于 2019-07-31 设计创作，主要内容包括：一种检测重放攻击的方法包括：接收表示言语的音频信号；识别存在于所述音频信号的至少一部分中的言语内容；获得关于所述音频信号中识别了言语内容的每个部分的频谱的信息；以及对于所述音频信号中识别了言语内容的每个部分：检索关于所述音频信号的预期频谱的信息；将所述音频信号中识别了言语内容的部分的频谱与相应的预期频谱进行比较；以及如果所述音频信号中识别了言语内容的所述部分的所述频谱与所述相应的预期频谱之间的差异的度量值超出阈值水平,则确定所述音频信号可能是由重放攻击引起的。(A method of detecting replay attacks comprising: receiving an audio signal representing speech; identifying speech content present in at least a portion of the audio signal; obtaining information about a frequency spectrum of each portion of the audio signal in which speech content is identified; and for each portion of the audio signal in which verbal content is identified: retrieving information about an expected spectrum of the audio signal; comparing a frequency spectrum of a portion of the audio signal in which speech content is identified with a corresponding expected frequency spectrum; and determining that the audio signal is likely to be caused by a replay attack if a measure of difference between the spectrum of the portion of the audio signal in which speech content is identified and the corresponding expected spectrum exceeds a threshold level.)

1. A method of detecting a replay attack, the method comprising:

receiving an audio signal representing speech;

identifying speech content present in at least a portion of the audio signal;

obtaining information about a frequency spectrum of each portion of the audio signal in which speech content is identified; and

for each portion of the audio signal in which verbal content is identified:

retrieving information about an expected spectrum of the audio signal;

comparing the frequency spectrum of the portion of the audio signal in which speech content is identified to a corresponding expected frequency spectrum; and

determining that the audio signal is likely to be caused by a playback attack if a measure of a difference between the spectrum of the portion of the audio signal in which speech content is identified and the corresponding expected spectrum exceeds a threshold level.

2. The method of claim 1, comprising:

removing the effects of the channel and/or noise from the received audio signal; and

when obtaining the information about the frequency spectrum of each portion of the audio signal in which speech content is identified, the audio signal is used after removing the influence of the channel and/or noise.

3. The method of claim 1, wherein identifying speech content present in at least a portion of the audio signal comprises identifying at least one test acoustic category.

4. The method of claim 3, wherein the at least one test acoustic category includes one or more specific phonemes.

5. The method of claim 4, wherein the at least one test acoustic category comprises fricatives.

6. The method of claim 5, wherein the at least one test acoustic category comprises sibilants.

7. The method of claim 4, wherein the at least one test acoustic category comprises plosive.

8. The method of claim 3, wherein identifying at least one test acoustic category comprises identifying a location of occurrence of the test acoustic category in known speech content.

9. The method of claim 8, wherein the known verbal content comprises a pass phrase.

10. The method of claim 1, wherein comparing the spectrum of portions of the audio signal in which speech content is identified to the corresponding expected spectrum comprises:

comparing the spectrum of the portion of the audio signal in which speech content is identified with the corresponding expected spectrum in a frequency band in the range of 5kHz to 20 kHz.

11. The method of claim 1, wherein comparing the spectrum of portions of the audio signal in which speech content is identified to the corresponding expected spectrum comprises:

comparing the spectrum of the portion of the audio signal in which speech content is identified with the corresponding expected spectrum in the 20Hz to 200Hz frequency band.

12. The method of claim 1, wherein comparing the spectrum of portions of the audio signal in which speech content is identified to the corresponding expected spectrum comprises:

comparing the frequency spectrum of the portion of the audio signal in which speech content is identified with the corresponding expected frequency spectrum in the ultrasonic frequency band.

13. The method of claim 1, wherein comparing the identified portion of the audio signal with respective retrieved information about corresponding test acoustic categories comprises:

comparing a power level in at least one frequency band of the identified portion of the audio signal with a power level in at least one corresponding frequency band of the expected spectrum of the audio signal.

14. The method of claim 13, wherein a metric of a difference between the identified portion of the audio signal and the respective retrieved information related to the corresponding test acoustic category comprises a power difference greater than 1 dB.

15. The method of claim 1, further comprising:

performing a speaker recognition process on the received audio signal; and

for each test acoustic category, information is retrieved about an expected spectrum of the audio signal of a speaker identified by the speaker identification process.

16. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,

wherein the step of retrieving information about an expected spectrum of the audio signal comprises: identifying a location in the audio signal where at least one particular acoustic category is expected to occur; and is

Wherein the step of comparing the frequency spectrum of the portion of the audio signal in which speech content is identified with the corresponding expected frequency spectrum comprises: comparing the spectrum at the identified location to an expected spectrum at the identified location.

17. The method of claim 16, wherein the particular acoustic category includes at least some fricatives.

18. The method of claims 16 and 17, comprising determining whether the spectrum at the location where a fricative is expected to occur is characteristic of a fricative.

19. The method of claim 16, 17 or 18, wherein the step of identifying a location in the audio signal where at least one particular acoustic category is expected to occur comprises: a location in the predetermined utterance where at least one particular acoustic class occurs is identified.

20. The method of claim 19, wherein the predetermined utterance comprises a passphrase.

21. A system for detecting replay attacks, the system comprising:

an input for receiving an audio signal representing speech; and

a processor, wherein the processor is configured to:

identifying speech content present in at least a portion of the audio signal;

obtaining information about a frequency spectrum of each portion of the audio signal in which speech content is identified;

for each portion of the audio signal in which speech content is identified, retrieving information about an expected frequency spectrum of the audio signal;

comparing the frequency spectrum of the portion of the audio signal in which speech content is identified to a corresponding expected frequency spectrum; and

determining that the audio signal is likely to be caused by a playback attack if a measure of a difference between the spectrum of the portion of the audio signal in which speech content is identified and the corresponding expected spectrum exceeds a threshold level.

22. An apparatus comprising the system of claim 21, wherein the apparatus comprises one of: a smart phone, a tablet or laptop computer, a game console, a home control system, a home entertainment system, a vehicle entertainment system, or a home appliance.

23. A computer program product comprising a tangible computer readable medium storing code for causing a suitable programmed processor to perform the method of claim 1.

24. A method of detecting a replay attack, the method comprising:

receiving an audio signal representing speech;

identifying at least one portion of the audio signal in which the speech consists of fricatives;

obtaining information about the amount of energy present at ultrasonic frequencies during the at least one portion of the audio signal; and

determining whether the audio signal is likely to be caused by a replay attack based on the information about the amount of energy present at ultrasonic frequencies.

25. The method of claim 24, further comprising:

obtaining information about an amount of energy present at ultrasonic frequencies during a second portion of the audio signal where the speech consists of voiced speech; and

determining whether the audio signal is likely to be caused by a replay attack based on the information about the amount of energy present at ultrasonic frequencies and based on the information about the amount of energy present at ultrasonic frequencies during the second portion of the audio signal.

26. The method of claim 25, comprising:

determining whether the audio signal is likely to be caused by a replay attack based on a ratio of the amount of energy present at ultrasonic frequencies during the at least one portion of the audio signal to the amount of energy present at ultrasonic frequencies during the second portion of the audio signal.

27. The method of claim 24, 25 or 26, further comprising:

obtaining information about an amount of energy present at an audible frequency during the at least one portion of the audio signal; and

determining whether the audio signal is likely to be caused by a replay attack based on the information about the amount of energy present at ultrasonic frequencies during the at least one portion of the audio signal and based on the information about the amount of energy present at audible frequencies during the at least one portion of the audio signal.

28. The method of claim 27, comprising:

determining whether the audio signal is likely to be caused by a replay attack based on a ratio of the amount of energy present at ultrasonic frequencies to the amount of energy present at audible frequencies during the at least one portion of the audio signal.

29. The method of any one of claims 24 to 28, comprising:

calculating a first ratio of the amount of energy present at ultrasonic frequencies to the amount of energy present at audible frequencies during the at least one portion of the audio signal;

calculating a second ratio of the amount of energy present at ultrasonic frequencies during a second portion of the audio signal where the speech consists of voiced speech to the amount of energy present at audible frequencies during the second portion of the audio signal; and

calculating a ratio of the first ratio to the second ratio.

30. A system for detecting replay attacks, the system comprising:

an input for receiving an audio signal representing speech; and

a processor, wherein the processor is configured to:

identifying at least one portion of the audio signal in which the speech consists of fricatives;

obtaining information about the amount of energy present at ultrasonic frequencies during the at least one portion of the audio signal; and

determining whether the audio signal is likely to be caused by a replay attack based on the information about the amount of energy present at ultrasonic frequencies.

31. An apparatus comprising the system of claim 30, wherein the apparatus comprises one of: a smart phone, a tablet or laptop computer, a game console, a home control system, a home entertainment system, a vehicle entertainment system, or a home appliance.

32. A computer program product comprising a tangible computer readable medium storing code for causing a suitable programmed processor to perform the method of claim 24.

Technical Field

Embodiments described herein relate to methods and apparatus for detecting replay attacks on a voice biometric system.

Background

Speech biometric recognition systems are being widely used. In such systems, a user trains the system by providing a sample of his speech during the enrollment phase. In subsequent use, the system is able to distinguish registered users from unregistered speakers. Speech biometric recognition systems can in principle be used to control access to a wide range of services and systems.

One way in which malicious parties attempt to defeat a voice biometric system is to obtain a recording of the registered user's speech and play back the recording in an attempt to impersonate the registered user and gain access to services that are intended to be limited to the registered user only.

This is known as a replay attack or as a spoofing attack.

Disclosure of Invention

According to an aspect of the present invention, there is provided a method of detecting a replay attack, the method comprising:

receiving an audio signal representing speech;

identifying speech content present in at least a portion of the audio signal;

obtaining information about a frequency spectrum of each portion of the audio signal in which speech content is recognized;

for each portion of the audio signal in which speech content is identified, retrieving information about an expected frequency spectrum of the audio signal;

comparing the frequency spectrum of the portion of the audio signal in which speech content is identified with a corresponding expected frequency spectrum; and

determining that the audio signal is likely to be caused by a replay attack if a measure of a difference between a spectrum of a portion of the audio signal in which speech content is identified and a corresponding expected spectrum exceeds a threshold level.

The method may further comprise:

removing the effects of the channel and/or noise from the received audio signal; and

when information on a frequency spectrum of each part of an audio signal in which speech content is recognized is obtained, the audio signal is used after removing the influence of a vocal tract and/or noise.

Identifying speech content present in at least a portion of the audio signal may include identifying at least one test acoustic category. The at least one test acoustic category may include one or more specific phonemes. The at least one test acoustic category may comprise fricatives, in particular sibilants; and/or may include a plosive.

Identifying at least one test acoustic category may include identifying a location of occurrence of the test acoustic category in the known speech content. The known verbal content may include a passphrase.

Comparing the frequency spectrum of the portion of the audio signal in which speech content is identified to the corresponding expected frequency spectrum may include:

the frequency spectrum of the portion of the audio signal in which speech content is identified is compared to a corresponding expected frequency spectrum in a frequency band in the range of 5kHz to 20 kHz.

Comparing the frequency spectrum of the portion of the audio signal in which speech content is identified to the corresponding expected frequency spectrum may include:

the frequency spectrum of the portion of the audio signal in which speech content is identified is compared to a corresponding expected frequency spectrum in a frequency band in the range of 20Hz to 200 Hz.

Comparing the frequency spectrum of the portion of the audio signal in which speech content is identified to the corresponding expected frequency spectrum may include:

the frequency spectrum of the portion of the audio signal in which speech content is identified is compared to a corresponding expected frequency spectrum in the ultrasonic frequency band.

Comparing the identified portion of the audio signal with the respective retrieved information about the corresponding test acoustic category may comprise:

the power level in at least one frequency band of the identified portion of the audio signal is compared to the power level in at least one corresponding frequency band of the expected spectrum of the audio signal.

A measure of the difference between the identified portion of the audio signal and the corresponding retrieved information about the corresponding test acoustic category may comprise a power difference of greater than 1 dB.

The method may further comprise:

performing a speaker recognition process on the received audio signal; and

for each test acoustic category, information is retrieved about an expected spectrum of an audio signal of a speaker identified by the speaker identification process.

The step of retrieving information about an expected spectrum of the audio signal may comprise: identifying a location in the audio signal where at least one particular acoustic category is expected to occur; and the step of comparing the frequency spectrum of the portion of the audio signal in which speech content is identified with the corresponding expected frequency spectrum may comprise: comparing the spectrum at the identified location to an expected spectrum at the identified location.

The particular acoustic category may include at least some fricatives.

The method may comprise determining whether a frequency spectrum at the location where a fricative is expected to occur is characteristic of a fricative.

The step of identifying a location in the audio signal where at least one particular acoustic class is expected to occur may comprise: a location in the predetermined utterance where at least one particular acoustic class occurs is identified.

The predetermined utterance may include a passphrase.

According to another aspect of the present invention, there is provided a system for detecting replay attacks, the system comprising:

an input for receiving an audio signal representing speech; and

a processor, wherein the processor is configured to:

identifying speech content present in at least a portion of the audio signal;

obtaining information about a frequency spectrum of each portion of the audio signal in which speech content is recognized;

for each portion of the audio signal in which speech content is identified, retrieving information about an expected frequency spectrum of the audio signal;

comparing the frequency spectrum of the portion of the audio signal in which speech content is identified with a corresponding expected frequency spectrum; and

determining that the audio signal is likely to be caused by a replay attack if a measure of a difference between a spectrum of a portion of the audio signal in which speech content is identified and a corresponding expected spectrum exceeds a threshold level.

There is also provided a device comprising such a system, wherein the device comprises one of: a smart phone, a tablet or laptop computer, a game console, a home control system, a home entertainment system, a vehicle entertainment system, or a home appliance.

According to another aspect of the present invention there is provided a computer program product comprising a tangible computer-readable medium storing code for causing a suitable programmed processor to perform the method of the first aspect.

According to a second aspect, there is provided a method of detecting a replay attack, the method comprising:

receiving an audio signal representing speech;

identifying at least one portion of speech in the audio signal that is comprised of fricatives;

obtaining information about the amount of energy present at ultrasonic frequencies during the at least one portion of the audio signal; and

determining whether the audio signal is likely to be caused by a replay attack based on said information about the amount of energy present at the ultrasonic frequency.

The method may further comprise:

obtaining information about an amount of energy present at ultrasonic frequencies during a second portion of the audio signal where speech consists of voiced speech; and

determining whether the audio signal is likely to be caused by a replay attack based on said information on the amount of energy present at the ultrasonic frequency and based on said information on the amount of energy present at the ultrasonic frequency during the second portion of the audio signal.

The method may comprise:

determining whether the audio signal is likely to be caused by a replay attack based on a ratio of an amount of energy present at the ultrasonic frequency during the at least one portion of the audio signal to an amount of energy present at the ultrasonic frequency during a second portion of the audio signal.

The method may further comprise:

obtaining information about an amount of energy present at an audible frequency during the at least one portion of the audio signal; and

determining whether the audio signal is likely to be caused by a replay attack based on said information on the amount of energy present at the ultrasonic frequency during said at least one portion of the audio signal and based on said information on the amount of energy present at the audible frequency during said at least one portion of the audio signal.

The method may comprise:

determining whether the audio signal is likely to be caused by a replay attack based on a ratio of an amount of energy present at the ultrasonic frequency to an amount of energy present at the audible frequency during the at least one portion of the audio signal.

The method may comprise:

calculating a first ratio of the amount of energy present at the ultrasonic frequency to the amount of energy present at the audible frequency during the at least one portion of the audio signal;

calculating a second ratio of the amount of energy present at the ultrasonic frequency during a second portion of the audio signal where speech consists of voiced speech to the amount of energy present at the audible frequency during the second portion of the audio signal; and

a ratio of the first ratio to the second ratio is calculated.

According to another aspect, there is provided a system for detecting replay attacks, the system comprising:

an input for receiving an audio signal representing speech; and

a processor, wherein the processor is configured to:

identifying at least one portion of speech in the audio signal that is comprised of fricatives;

obtaining information about the amount of energy present at ultrasonic frequencies during the at least one portion of the audio signal; and

determining whether the audio signal is likely to be caused by a replay attack based on said information about the amount of energy present at the ultrasonic frequency.

There is also provided a device comprising such a system, wherein the device comprises one of: a smart phone, a tablet or laptop computer, a game console, a home control system, a home entertainment system, a vehicle entertainment system, or a home appliance.

According to another aspect, there is provided a computer program product comprising a tangible computer-readable medium storing code for causing a suitable programmed processor to perform a method according to the second aspect.

Drawings

For a better understanding of the present invention, and to show how the same may be carried into effect, reference will now be made to the accompanying drawings, in which:

fig. 1 shows a smart phone.

Fig. 2 is a schematic diagram showing the form of a smartphone.

Fig. 3 shows an example of a situation in which a replay attack is being performed.

Fig. 4 is a block diagram illustrating functional blocks in a speech biometric recognition system.

Fig. 5 is a flow chart illustrating a method of detecting a replay attack on a speech biometric recognition system.

Fig. 6 shows a portion of the system of fig. 4.

Fig. 7 to 9 show examples of comparisons between identified portions of an audio signal and corresponding retrieved information about corresponding test acoustic classes in several situations where the audio signal may be caused by a replay attack.

Detailed Description

The following description sets forth example embodiments according to the present disclosure. Other example embodiments and implementations will be apparent to those of ordinary skill in the art. Moreover, those of ordinary skill in the art will recognize that various equivalent techniques may be used in place of or in combination with the embodiments discussed below, and all such equivalents should be considered encompassed by the present disclosure.

The methods described herein may be implemented in a wide range of devices and systems, such as mobile phones, audio players, video players, mobile computing platforms, gaming devices, remote controller devices, toys, machines, or home automation controllers, or home appliances. However, for ease of explanation of one embodiment, an illustrative example will be described in which the implementation occurs in a smartphone.

Fig. 1 shows a smartphone 10 having microphones 12, 12a, and 12b for detecting ambient sounds. In this example, the microphone 12 is of course used for detecting speech of a user holding the smartphone 10, whereas the microphones 12a, 12b are provided on an upper part of the side of the smartphone 10 and are therefore not clearly visible in fig. 1.

The smartphone 10 is but one example of an electronic device in which the methods described herein may be performed. The method may similarly be performed in another portable electronic device, such as a tablet or laptop computer, a game console, a home control system, a home entertainment system, a vehicle entertainment system, or a home appliance.

Fig. 2 is a schematic diagram showing the form of the smartphone 10.

In particular, fig. 2 shows various interconnected components of the smartphone 10. It should be understood that the smartphone 10 will in fact contain many other components, but the following description is sufficient for understanding the present invention.

Thus, fig. 2 shows the microphone 12 mentioned above. In some embodiments, the smartphone 10 is provided with a plurality of microphones 12, 12a, 12b, etc.

Fig. 2 also shows a memory 14, which may in fact be provided as a single component or as a plurality of components. Memory 14 is provided for storing data and program instructions.

Fig. 2 also shows a processor 16, which may in fact also be provided as a single component or as a plurality of components. For example, one component of the processor 16 may be an application processor of the smartphone 10.

Fig. 2 also shows a transceiver 18, which is provided to allow the smartphone 10 to communicate with an external network. For example, the transceiver 18 may include circuitry for establishing an internet connection over a WiFi local area network or over a cellular network.

Fig. 2 also shows an audio processing circuit 20 for performing operations on the audio signal detected by the microphone 12 as required. For example, audio processing circuit 20 may filter audio signals or perform other signal processing operations.

Fig. 2 also shows at least one sensor 22. In an embodiment of the invention, the sensor is a magnetic field sensor for detecting a magnetic field. For example, the sensor 22 may be a Hall effect sensor capable of providing separate measurements of magnetic field strength in three orthogonal directions. Other examples of sensors that may be used may include a gyroscope sensor, an accelerometer, or a software-based sensor operable to determine the orientation of a phone, where such software-based sensor may be compatible with FaceTime, such as that provided by Apple, Inc^TMThe software programs of the system operate in combination.

In this embodiment, the smartphone 10 is provided with voice biometric recognition functionality and control functionality. Thus, the smartphone 10 is able to perform various functions in response to verbal commands from registered users. The biometric function is capable of distinguishing between spoken commands from an enrolled user and the same commands spoken by a different person. Thus, certain embodiments of the present invention relate to the operation of a smartphone or another portable electronic device with some voice operability, such as a tablet or laptop computer, a gaming console, a home control system, a home entertainment system, an in-vehicle entertainment system, a home appliance, etc., in which voice biometric recognition functionality is performed in the device intended to implement spoken commands. Certain other embodiments relate to a system that performs speech biometric recognition functionality on a smartphone or other device that transmits a command to a separate device if the speech biometric recognition functionality is capable of confirming that the speaker is an enrolled user.

In some implementations, while the voice biometric recognition functionality is performed on the smartphone 10 or other device located proximate to the user, the spoken command is transmitted using the transceiver 18 to a remote speech recognition system that determines the meaning of the spoken command. For example, the speech recognition system can be located on one or more remote servers in a cloud computing environment. A signal based on the meaning of the spoken command is then returned to the smartphone 10 or other local device. In other embodiments, a speech recognition system is also provided on the smartphone 10.

One attempt to fool a speech biometric recognition system is to play a recording of the registered user's speech through a so-called replay or spoofing attack.

Fig. 3 shows an example of a situation in which a replay attack is being performed. Thus, in fig. 3, the smartphone 10 is provided with voice biometric recognition functionality. In this example, the smartphone 10 is at least temporarily occupied by an attacker with another smartphone 30. The smartphone 30 has been used to record the voice of a registered user of the smartphone 10. The smartphone 30 is brought close to the microphone inlet 12 of the smartphone 10 and a recording of the registered user's voice is played back. If the voice biometric recognition system cannot detect that the registered user's voice it detects is a recording, the attacker will gain access to one or more services that are intended to be accessible only by the registered user.

It is well known that due to size constraints, smartphones such as smartphone 30 are typically provided with a relatively low quality speaker. Therefore, the recording of the registered user's voice played back through such a speaker does not perfectly match the user's voice, and this fact can be used to identify a replay attack. For example, the speaker may have certain frequency characteristics, and if these frequency characteristics can be detected in the speech signal received by the voice biometric recognition system, the speech signal may be considered to be caused by a replay attack.

For example, at frequencies below a lower threshold frequency, the loudspeaker may suffer from low frequency roll-off, since the bass response is limited by the size of the loudspeaker diaphragm. The lower threshold frequency may be 200 Hz. In another example, at frequencies above the upper threshold frequency, the speaker may suffer from high frequency roll-off. While good quality audio recording equipment, amplifiers, speakers, etc. are designed to exhibit very high fidelity in the audible frequency range (e.g., up to about 20kHz), even high quality audio systems typically exhibit high attenuation at their frequency response above about 20 kHz. Since sound in this frequency range is not perceived by the average person, it is generally not worthwhile for system designers to provide high fidelity recording and reproduction of such sound.

The magnitude of these effects will be determined by the quality of the loudspeaker. For example, in a high quality speaker, the lower threshold frequency and the upper threshold frequency should be such that there is minimal low frequency roll-off or high frequency roll-off within the frequency range that is typically audible to humans. However, size and cost constraints mean that many commercially available speakers, such as those provided in smartphones such as smartphone 30, do suffer to some extent from these effects.

Thus, the speaker may not reproduce the ultrasonic and/or near-ultrasonic frequencies well. In these cases, if the ultrasonic and/or near-ultrasonic frequency components of the detected speech signal do not reach the power level expected for "typical" speech, this may indicate that the detected speech signal was caused by a replay attack.

However, not all speech contains the same level of ultrasound and/or near ultrasound frequency components. For example, some consonant groups, such as fricatives and plosives, contain relatively high levels of ultrasonic and near-ultrasonic frequencies. If the speech to which the detected speech signal relates contains higher than normal amounts of speech in one or more acoustic classes, such as these acoustic classes, that include relatively large components of ultrasonic and/or near-ultrasonic frequencies, then the ultrasonic and/or near-ultrasonic frequencies in the detected speech signal may still be present at power levels similar to those that would be expected for these frequencies in "typical" speech, in the result of a replay attack through the loudspeaker that reduces the level of ultrasonic and/or near-ultrasonic frequency components. Thus, in such a situation, the system may not be able to discern that the speech signal may have been caused by a replay attack.

If the acoustic class of the detected speech signal and the expected frequency characteristics of the acoustic class can be determined, it may be possible to more reliably determine whether the detected signal is likely to be caused by a replay attack using the frequency characteristics of the detected speech signal.

Fig. 4 is a block diagram illustrating functional blocks in a speech biometric recognition system, and fig. 5 is a flowchart illustrating a method of detecting a replay attack on the speech biometric recognition system.

As shown in fig. 4, sound is detected by the microphone 12 (e.g., one of the microphones in the smartphone 10) and passed to an initial processing block 40. The microphone 12 is capable of detecting audible sounds and sounds in the ultrasonic range. As used herein, the term "ultrasound" (and "ultrasonic") refers to sound in the upper portion of the audible frequency range as well as above the audible frequency range. Thus, the term "ultrasound" (and "ultrasonic") refers to sound having a frequency above about 15 kHz.

The initial processing block 40 may, for example, comprise an analog-to-digital converter for converting signals received from an analog microphone into digital form, and may also comprise a buffer for storing the signals. Analog-to-digital conversion involves sampling a received signal at a sampling rate. The sampling rate must be chosen high enough so that any frequency components of interest remain in the digital signal. For example, as described in more detail below, some embodiments of the invention involve looking at the ultrasonic component of the received signal, for example in the region of 20kHz to 30 kHz. As is well known from the nyquist sampling theorem, the sampling rate of a digital signal should be at least twice the highest frequency component of the signal. Therefore, in order to correctly sample a signal containing components at frequencies up to 30kHz, the sampling rate should be at least 60 kHz.

A typical smartphone may be capable of sampling at a sampling rate of up to 192kHz (high enough to provide a 96kHz bandwidth), and thus the sampling rate may be set to any suitable value that accommodates the desired frequency components.

As is conventional, the signal may be divided into frames, for example having a duration of 10ms to 100 ms.

Thus, in step 60 of the method of FIG. 5, an audio signal representing speech is received.

The received audio signal representing speech may then be passed to a spectral extraction block 42. The spectrum extraction block 42 may be configured to obtain a spectrum of the received audio signal. In some examples, the spectrum extraction block 42 may be configured to obtain a power spectrum of the received audio signal, while in some other examples, the spectrum extraction block 42 may be configured to obtain an energy spectrum of the received audio signal.

In some examples, the spectrum extraction block 42 may be configured to perform a fast fourier transform on the received audio signal. The results of the fast fourier transform indicate the power or energy present in the signal at different frequencies.

In another example, the spectral extraction block 42 may be configured to apply several band pass filters to the received audio signal representing speech. Each band pass filter may only allow signals within a particular frequency band of the received audio signal to pass.

The received audio signal representing speech and the spectrum obtained by the spectrum extraction block 42 may then be passed to a channel/noise compensation block 44. As will be appreciated, the received audio signal representing speech may be affected by the vocal tract properties (which refers to any factor change that produces a difference between the user's speech and the speech signal generated by the microphone). The received audio signal representing speech may also be affected by noise. The channel/noise compensation block 44 may be configured to remove the effects of the channel and/or noise from the received audio signal representing speech to obtain a received cleaned audio signal representing speech. An exemplary embodiment of the channel/noise compensation block 44 is described in more detail below.

In particular, figure 6 shows a channel/noise compensation system, which is described in more detail in uk patent application no 1719734.4.

In fig. 6, the signal received on input 70 is passed to a voiced/unvoiced detection block 72, which determines when the received signal represents voiced speech and when it represents unvoiced speech. Thus, the voiced/unvoiced detection block 72 outputs a first component S of the audio signal representing voiced speech_vAnd a second component S representing unvoiced speech_u. More specifically, in some embodiments, a first component S of an audio signal representing voiced speech_vAnd a second component S representing unvoiced speech_uIs the average spectrum of voiced and unvoiced components of speech. Average spectrum refers to the spectrum of speech obtained and averaged over multiple frames.

The signal received on the input 70 is also passed to a speaker recognition block 74 which performs a speech biometric recognition process to identify a speaker from a plurality of registered speakers. The enrollment process in a speaker recognition system typically involves the speaker providing a speech sample, extracting certain features from the speech sample, and using the extracted features to form a model of the speaker's speech. In use, corresponding features are extracted from a speech sample and compared to a previously obtained model to obtain a measure of the likelihood that the speaker is a previously enrolled speaker.

In the system shown in fig. 6, one or more speaker models are stored, for example, in a database 76. Based on the output of the speaker recognition block 74, one or more speaker models are selected. In this embodiment, each speaker model contains separate models of voiced and unvoiced speech of the enrolled users. More specifically, the model of voiced speech and the model of unvoiced speech of the registered user each include amplitude values corresponding to a plurality of frequencies.

Thus, a model of voiced speech may be indicated as M_vWherein M is_vRepresenting vectors including one amplitude value at each of the selected frequencies, and the model of unvoiced speech may be denoted as M_uWherein M is_uRepresenting a vector including one amplitude value at each of the selected frequencies.

As will be appreciated, the received signal containing the user's speech will be affected by the vocal tract properties, meaning that any factors that produce a difference between the user's speech and the speech signal generated by the microphone are altered, and the received signal will also be affected by noise.

Thus, assuming that the vocal tract and noise are constant during the time period in which the received signal is averaged to form the first and second components of the received speech, these first and second components may be expressed as:

sv ═ α Mv + n, and

Su＝αMu+n，

wherein

Alpha represents the frequency spectrum of the multiplicative interference component, referred to herein as the vocal tract, and

n represents the spectrum of an additive interference component, referred to herein as noise.

Thus, with the measurements Sv and Su and the models Mv and Mu, these two equations can thus be solved in a comparison block 78 to obtain the two unknowns α and n.

Accordingly, for the purposes of illustration,

and is

For the sake of completeness, it should be noted that by making spectral measurements at multiple frequencies, these two equations can be effectively solved at each of the frequencies.

Alternatively, by making measurements at f different frequencies, the equations Sv ═ α Mv + n and Su ═ α Mu + n may each be considered as f different equations to be solved for.

In this case, after solving the equation, a low pass filter or statistical filter (such as a Savitsky-Golay filter) may usefully be applied to the result in order to obtain a low pass filtered version of the channel and noise characteristics.

As an alternative example, a least squares method may be used to obtain solutions for 2f different equations.

Thus, information about the channel and/or noise affecting the audio signal is obtained.

While one possible implementation of the channel/noise compensation block 44 has been described above, it will be appreciated that the channel/noise compensation block 44 may be configured to perform any suitable method to remove the effects of the channel and/or noise from the received audio signal.

Additionally, while the system as shown in fig. 4 includes a channel/noise compensation block 44 (and subsequently uses the audio signal after removing the effects of the channel and/or noise, as described below), it will be appreciated that this is optional and that the received audio signal may be used directly without removing the effects of the channel and/or noise from the received audio signal. This may be done, in particular, in situations where the channel is known to be approximately ideal or where the system is known to be in an environment where the background noise level is low.

The effects of the channel and/or noise may then be removed from the input signal, and the output of the channel/noise compensation block 44 may then be passed to an identification block 46. The recognition block 46 may be configured to recognize speech content present in at least a portion of the received audio signal. Generally, this involves not only detecting the presence of human speech, but also meaning identifying specific speech content. In some embodiments, the recognition block 46 is configured to recognize at least one test acoustic category in the received audio signal, wherein an acoustic category is a set of phonemes belonging to the same broad phoneme category, such as fricatives, vowels, and the like. That is, the identification block 46 may be configured to identify portions of the received audio signal that contain one or more particular test acoustic categories.

The at least one test acoustic category may be an acoustic category that includes certain spectral properties. In other words, at least one test acoustic category may be selected to provide a particular power distribution across a particular frequency range. For example, the at least one acoustic category may include relatively high levels of ultrasonic and/or near ultrasonic frequencies.

In some examples, the at least one test acoustic category may include one or more particular phonemes. In some examples, the test acoustic categories may include vowel sets. In other examples, the test acoustic categories may include consonant sets. For example, the at least one test acoustic category may include fricatives, and more particularly, may include sibilants. In another example, the at least one test acoustic category may include plosives. It should be noted that audio signals generated by the vocal tract of humans, especially when fricative and sibilant phonemes are clearly pronounced, contain a lot of energy in the ultrasonic region, above about 20kHz and even above 30 kHz.

The identification block 46 may be configured to identify a portion of the audio signal representing at least one test acoustic category. In some examples, the identification block 46 may be configured to identify a portion of the audio signal representing one of a plurality of pre-selected test acoustic categories. In some examples, the identification block 46 may determine a test acoustic category to which the received audio signal corresponds on a frame-by-frame basis. For example, the recognition block 46 may determine that a first group of received frames corresponds to a phoneme in the fricative category and a second group of received frames corresponds to a phoneme in the plosive category.

The identification of acoustic classes may be performed, for example, by a trained neural network. Alternatively, the recognition of the phonemes contained in the received phrase may be performed by well-known techniques, such as hidden Markov models.

In some examples, the recognition block 46 includes or is associated with an automatic speech recognition block that can recognize speech content present in at least a portion of the received audio signal. After speech content is recognized, the recognition block 46 may then, for example, recognize portions of the received audio signal that contain one or more particular test acoustic categories.

In some examples, the recognition block 46 includes a frame-by-frame estimation of the energy in the near ultrasound frequencies, where the frame with the highest energy is associated with the fricative phonemes, effectively creating a time-map of the occurrence of the fricative phonemes in the phrase. This time map associated with the received audio signal may be aligned with a similar map previously calculated and stored in the database 48. In text-dependent speech biometric recognition, an expected time mapping of phonemes can be calculated during the enrollment phase when a user creates his biometric voice print by speaking a predefined passphrase. In text-independent speech biometric recognition, the mapping may be obtained during an authentication attempt by means of a phonetic decoder (or acoustic class identifier) applied to the received audio.

For example, alignment between the expected time map and the actual time map of the phonemes may be performed using techniques such as Viterbi algorithm or dynamic time warping.

The spectrum of the thus aligned portion of the signal may be compared to the spectrum of the corresponding acoustic class. Further, these alignment techniques may provide a measure of the distance between the time map of the fricatives in the received signal and the expected time map. A distance exceeding a given threshold (meaning that the ultrasonic component of the received signal does not match the high frequency phonetic content of the expected speech) may indicate a spoofing attack.

For example, in some embodiments of the method, when applied in a smartphone, smart speaker, or the like, the smartphone or smart speaker is activated by a registered user speaking a predetermined passphrase, and thus the device is able to recognize an audio signal containing the passphrase. For example, the passphrase may be "Hi helper".

The vowel sounds in the word "Hi" naturally contain higher frequency components than many vowels. Similarly, sibilant consonants contain a relatively large amount of high frequency components. As a result, the recognition block 46 may determine which portions of the audio signal contain the pass phrase, i.e., which frames of the audio signal will contain phonemes with a high proportion of high frequency components. In practice, a time-location map of fricative and/or affricate phonemes within the passphrase may be created. Thus, specific verbal content in these portions of the audio signal may be identified.

For example, if the pass phrase occupies 1 second of the audio signal, a frame in the word "Hi" containing a vowel sound may appear in a portion of the audio signal between, for example, 50 milliseconds and 150 milliseconds of the audio signal. Similarly, an affricate consonant in the word "assistant" may appear in a portion of the audio signal between, for example, 550 milliseconds and 850 milliseconds of the audio signal.

Thus, the frames occurring during these time intervals may be identified.

Thus, in step 62 of the method of fig. 5, speech content present in at least a portion of the audio signal is identified.

In the example as shown in fig. 4, the output of the channel/noise compensation block 44 is passed to an identification block 46. However, it will be appreciated that the received audio signals representing speech may be passed directly to the recognition block 46 without any processing previously performed to remove the effects of the channel and/or noise. This may occur where the channel is known to be approximately ideal, or where the system is known to be in an environment where the background noise level is very low.

In the example as shown in fig. 4, the extracted spectrum is passed to an identification block 46. However, it will be appreciated that the recognition block 46 may receive the received audio signal representing speech in unprocessed form or in a form that is retained in the time domain (rather than in the frequency domain) and recognize speech content present in at least a portion of the audio signal from such a form of the received audio signal (e.g., recognize at least one acoustic category). Such identification may be performed using a suitably trained neural network.

Having identified the speech content, information is obtained about the frequency spectrum of each portion of the audio signal in which the particular speech content was identified. For example, while the spectrum extraction block 42 may be configured to obtain the spectrum of the entire received audio signal, the second spectrum extraction block 47 may be configured to obtain the spectrum of those portions of the received audio signal where particular verbal content of interest is identified. For example, as described above, portions of the signal that represent particular speech content having a high proportion of high frequency components may be considered interesting, and the second spectral extraction block 47 may be configured to obtain the spectrum of frames of the received audio in which that speech content is identified.

The recognition block 46 may transmit an indication of the recognized speech content to the database 48. Database 48 may receive indications of recognized speech content and supply corresponding stored information regarding an expected spectrum of the audio signal. For example, where the related speech content includes one or more particular acoustic categories (e.g., one or more phonemes), the database 48 may be configured to store, for each acoustic category, an expected power spectrum of the audio signal corresponding to that acoustic category. The database 48 may then be configured to retrieve, for each acoustic category, an expected energy spectrum of the audio signal corresponding to that acoustic category.

Thus, in step 66 in the method of fig. 5, for each portion of the audio signal in which verbal content is identified, information is retrieved regarding the expected frequency spectrum of the corresponding portion of the audio signal.

The database 48 may communicate the retrieved information regarding the expected frequency spectrum of the audio signal to the comparison block 50 for each portion of the audio signal in which speech content is identified (that is, for example, for each identified test acoustic category).

The comparison block 50 may also receive the frequency spectrum of the frames of the received audio signal from the second frequency spectrum extraction block 47 (e.g. the or each frequency spectrum of the frames containing at least one test acoustic category) in which the same speech content is identified.

The comparison block 50 may then compare the received identified portion of the audio signal (representing at least one test acoustic category) with the received corresponding retrieved information regarding the corresponding test acoustic category.

In some examples, the comparison block 50 may compare components of the identified portion in the audio signal with respective retrieved information for corresponding test acoustic categories in a frequency band that is within a range of 5kHz to 20kHz, or within a range above 16 kHz. As described above, some speakers may not reproduce ultrasonic and/or near-ultrasonic frequencies well. Thus, received audio signals including test acoustic classes with relatively high levels of ultrasonic and/or near-ultrasonic frequencies may be poorly reproduced by the loudspeaker at these frequencies (e.g., in a frequency band in the range of 5kHz to 20kHz, or in a frequency band above 16 kHz). Thus, the frequency spectrum representing the test acoustic category in the received audio signal, which includes relatively high levels of ultrasonic and/or near ultrasonic frequencies when the received audio signal is caused by a replay attack, may be significantly different from the expected frequency spectrum representing the test acoustic category in the audio signal. Therefore, in order to be able to detect replay attacks using this frequency band, it is preferable to have the test acoustic category include relatively high levels of ultrasonic and/or near ultrasonic frequencies, and suitable acoustic categories may be, for example, fricatives, such as sibilants and plosives.

In some examples, the comparison block 50 may compare components of the identified portion of the audio signal with respective retrieved information for corresponding test acoustic categories in the frequency band of 20Hz to 200 Hz. Some speakers may not reproduce lower frequencies (e.g., frequencies within the 20Hz to 200Hz band) well. Therefore, test acoustic levels that include relatively high levels of these frequencies are likely to be poorly reproduced by the loudspeaker at these frequencies. Thus, the frequency spectrum representing the test acoustic category in the received audio signal may be significantly different from the expected frequency spectrum representing the test acoustic category in the audio signal, wherein the test is caused by a replay attack when the received audio signal is caused by a replay attackThe acoustic classes include relatively high levels of these lower frequency components. Therefore, in order to be able to detect replay attacks using this band, it is preferable to have the test acoustic category include frequencies at relatively high levels in the 20Hz to 200Hz band, and a suitable acoustic category may be, for example, certain vowel sounds. For example, in the international phonetic alphabet<u>Andhigh back circular lip vowels and near high back circular lip vowels are shown to have high levels of these low frequency components, respectively.

High front out-of-round lip vowels, denoted by < i > in the international phonetic alphabet, typically have a relatively low first formant frequency F1 and a relatively high second formant frequency F2, and thus the portion of speech containing such vowel sounds have relatively high levels of low frequencies in the 20Hz to 200Hz band and relatively high levels of high frequencies in the 5kHz to 20kHz band. Thus, if parts of the speech are identified that contain such vowel sounds, it is useful to compare the spectrum of these parts with the expected spectrum in the respective frequency bands in both the 20Hz to 200Hz range and the 5kHz to 20kHz range.

In some examples, the comparison block 50 may compare the frequency spectrum of the identified portion of the audio signal with the retrieved expected frequency spectrum for the corresponding test acoustic category.

In one example, the comparison block 50 may compare the identified portion of the audio signal, which has been obtained from the FFT performed on the audio signal received at the spectrum block 42, with the respective retrieved spectrum for the corresponding test acoustic category. In another example, the comparison block 50 may compare the identified portion of the audio signal, which has been obtained from the band-pass audio signal as generated by the spectrum block 42, with the respective retrieved spectrum relating to the corresponding test acoustic category.

In some examples, the comparison block 50 may compare the power level in at least one frequency band of the identified portion of the audio signal to the power level in at least one corresponding frequency band of the expected spectrum of the audio signal. In this example, the measure of the difference between the identified portion of the audio signal and the respective retrieved information about the corresponding test acoustic category may comprise the difference in power.

As explained above, when compared to the power level of the respective retrieved information regarding the corresponding test acoustic category at a particular frequency (or within a particular frequency range), a difference in the power level of the identified portion of the audio signal at that frequency (or within that frequency range) may indicate that the audio signal is likely to be caused by a replay attack.

In some examples, the comparison block may compare power levels in frequency bands in which it may be assumed that a speaker for a replay attack will have a poor frequency response. The comparison block 50 may compare the power level of the identified portion of the audio signal in a frequency band within the range of 5kHz to 20kHz with the power level of the audio signal in the same frequency band with the expected spectrum of the audio signal in the frequency range of 5kHz to 20 kHz. Additionally or alternatively, the comparison block 50 may compare the power level of the identified portion of the audio signal in a frequency band in the range of 20Hz to 200Hz with the power level of the expected spectrum of the audio signal in the same frequency band in the range of 20Hz to 200 Hz. Additionally or alternatively, the comparison block 50 may compare the power level of the identified portion of the audio signal in a frequency band within the range of 20kHz to 30kHz with the power level of the expected spectrum of the audio signal in the same frequency band within the range of 20kHz to 30kHz frequencies.

Thus, in step 68 of the method of fig. 5, the frequency spectrum of the portion of the audio signal in which speech content is identified is compared to the corresponding expected frequency spectrum.

The comparison as generated by the comparison block 50 may be transmitted to a decision block 52. The decision block 52 may determine whether a metric of the difference between the frequency spectrum of the portion of the audio signal in which speech content is identified and the corresponding expected frequency spectrum exceeds a threshold level. If the measure of difference exceeds the threshold level, decision block 52 may determine that the audio signal is likely to be caused by a replay attack.

Thus, in step 69 of the method of fig. 5, if a measure of the difference between the spectrum of the portion of the audio signal in which speech content is identified and the corresponding expected spectrum exceeds a threshold level, it is determined that the audio signal is likely to be caused by a replay attack.

In some cases, it may be desirable to retrieve information about expected spectra representing particular test acoustic classes in step 66 of the method, where the information is specific to an individual speaker (rather than obtaining spectra representing test acoustic classes spoken by multiple speakers).

Thus, in some implementations, the system may perform a speaker recognition process on the received audio signal. In some examples, the received audio signal representing speech may undergo a speaker recognition process before retrieving information about the expected spectrum of the audio signal from the database 48. The speaker recognition process may be performed before or simultaneously with the first two steps of the method of fig. 5. Speaker recognition methods will be well understood by those skilled in the art.

After identifying the speaker who produced the received audio signal representing speech, the obtained speaker identity may also be communicated to the database 48.

In some examples, the expected spectrum corresponding to each acoustic category stored in database 48 will represent the acoustic category as spoken by a large group of speakers. However, in some examples, the expected spectrum corresponding to each acoustic category stored in database 48 will represent the acoustic category as spoken by a particular individual. The particular individual may have been identified by a speaker recognition process.

Thus, for each test acoustic category, the database 48 may retrieve information about the expected spectrum of the audio signal of the speaker identified by the speaker identification process.

For example, there may be multiple registered users in the system, and each of the multiple registered users may have a model corresponding to their speech stored within database 48. For each registered user, the corresponding model of the user may comprise information about expected spectra of audio signals of a plurality of test acoustic classes for the user.

In the example of two registered users in the system, there may be two expected spectra of information for each test acoustic category stored in database 48, where a first expected spectrum corresponds to a first registered user and a second registered user corresponds to a second registered user. The speaker recognition process may identify which of the two registered users produced the received audio signal representing speech. The identity of the speaker may then be used by the database 48 to retrieve information about the expected spectrum (for each identified test acoustic category in the audio signal) corresponding to the identified speaker. This "speaker-specific" retrieved information may then be passed to the comparison block 50 and used by the comparison block 50 in substantially the same manner as described above. Since the retrieved information about the expected spectrum of the audio signal will be "speaker specific", the comparison between the identified portion of the audio signal and the corresponding retrieved information about the corresponding test acoustic category may be more accurate (since both pieces of information to be compared are speaker specific). Thus, the determination that the audio signal may be caused by a replay attack may thus be more accurate.

Fig. 7, 8 and 9 show examples of comparisons between identified portions of an audio signal and corresponding retrieved information about corresponding test acoustic classes in several situations where the audio signal may be caused by a replay attack. In the following example, the identified portion of the audio signal representing the at least one test acoustic category is a power spectrum, and the respective retrieved information about the corresponding test acoustic category is also a power spectrum.

In fig. 7, 8 and 9, the identified spectrum of the received signal is indicated by a solid line, while the expected spectrum that has been retrieved from the database is indicated by a dashed line.

In FIG. 7, it can be seen that at a particular first frequency (e.g., from f)₁Indicated), the identified spectrum 82 and the expected spectrum 84 have substantially similar power levels (as indicated by P)₁Indicated). As the frequency increases, the expected spectrum and the identified spectrum will continue to have substantially similar power levelsUntil a specific second frequency is reached (e.g. from f)₂Indicated). At this point, the desired spectrum 84 remains at the first power level (as represented by P)₂As indicated) and the power level of the identified spectrum 82 rolls down to a second, different power level (as indicated by P)₃Indicated). Such a roll-off in power level at higher frequencies, as seen in the identified spectrum 82 but not in the expected spectrum for that particular acoustic class, may indicate that the received audio signal representing speech may be caused by a replay attack.

For example, for frequencies above the upper cut-off frequency, if the power level is reduced at a relatively constant rate (such as 6 dB/octave), it may be determined that the received audio signal representing speech may be caused by a replay attack. The upper cutoff frequency may be in the range of 15kHz to 25 kHz. In FIG. 8, it can be seen that at a particular first frequency (e.g., from f)₁Indicated), the identified spectrum 86 and the expected spectrum 88 have significantly different power levels. The identified spectrum 86 is at f₁Lower power level (e.g. from P)₁Indicated) is significantly lower than the expected spectrum 88 at f₁Lower power level (e.g. from P)₃Indicated). As the frequency increases, the power level of the identified spectrum gradually increases until a second particular frequency is reached (as represented by f)₂Indicated). At frequency f₂Next, the power levels of the identified spectrum 86 and the expected spectrum 88 (as represented by P)₂Indicated) are substantially the same. Such a "roll off" of power levels with decreasing frequencies as seen in the identified spectrum but not in the expected spectrum may indicate that the received audio signal representing speech may be caused by a replay attack.

For example, for frequencies below the lower cutoff frequency, if the power level is reduced at a relatively constant rate (such as 6 dB/octave), it may be determined that the received audio signal representing speech may be caused by a replay attack. The lower cut-off frequency may be in the range of 20Hz to 200 Hz.

In fig. 9, it can be seen that at frequencies below a particular first frequency (as indicated by f)₁Indicated), the identified spectrum90 and the desired frequency spectrum 92 are substantially the same power level (as represented by P₁Indicated). As the frequency increases, the power level of the identified spectrum 90 begins to deviate from the power level of the expected spectrum 92. In this example, the power level of the expected spectrum begins to decrease with increasing frequency at a rate that is substantially faster than the power level of the identified spectrum. At a second specific frequency (e.g. from f)₂Indicated), the power level of the identified spectrum 90 is at a power level (as indicated by P) that is greater than the desired spectrum 92₂Indicated) much lower power level (as indicated by P)₃Indicated). Such a difference in absolute power levels at particular frequencies between the identified spectrum and the expected spectrum may indicate that the received audio signal representing speech may be caused by a replay attack. In some examples, a difference in absolute power levels that exceeds a threshold level may indicate a replay attack. For example, the threshold level may be 1 dB. Thus, by comparing signals from a known acoustic class with the expected spectrum specific to that same acoustic class, this allows signals that may be caused by replay attacks to be identified in a more reliable manner.

Thus, as mentioned above, one way to detect the limitations of recording and playback devices such as smart phones and thus detect when a detected speech signal may be caused by a playback attack is to examine the amount of energy present at the ultrasonic frequency. However, the expected amount of energy present at ultrasonic frequencies will depend on the phonemes present in the speech.

Thus, some methods disclosed herein relate to identifying speech content present in at least a portion of an audio signal, and in particular to identifying one or more specific phonemes present in the speech, and then obtaining information about the frequency spectrum of these portions of the audio signal.

One example of such an approach is to measure the average amount of energy present at the ultrasound frequency during each frame containing fricative phonemes (or more generally, during each frame containing unvoiced speech) and compare this average amount to the average amount of energy present at the ultrasound frequency during each frame containing voiced speech.

Therefore, the temperature of the molten metal is controlled,

wherein:

x₁is a measure of the perception of speech presence,

E_u(f) is a measure of the amount of energy present at the ultrasonic frequency during the frames containing the fricative phonemes (or, more generally, during the frames containing the unvoiced speech), and

E_u(v) is a measure of the amount of energy present at the ultrasonic frequency during the frame containing voiced speech.

Thus, x can be calculated₁And compares the value to an appropriate threshold to determine with the required degree of confidence whether the speech signal should be considered as being caused by a replay attack.

Another example of such an approach is to measure the average amount of energy present in a plurality of different ultrasound bands during each frame containing fricative phonemes (or, more generally, during each frame containing unvoiced speech) and compare these average amounts to the amount of energy present in these ultrasound bands during frames containing voiced speech.

Therefore, the temperature of the molten metal is controlled,

wherein:

x₂(b) is a set of values obtained by calculating the following ratios across a plurality of frequency bands,

E_b(f) is a measure of the amount of energy present in one of several ultrasonic frequency bands during the frames containing fricative phonemes (or, more generally, during the frames containing unvoiced speech), and

E_b(v) is a measure of the amount of energy present in one of several ultrasonic frequency bands during a frame containing voiced speech.

Thus, a calculation can be made for each of several ultrasonic frequency bandsAnd may combine these values to form a set of values x₂(b) The set of values may then be used as a measure of the presence.

Machine learning methods may be suitable for determining the calculated value x₂(b) Whether live speech is indicated or whether the speech signal should be considered as being caused by a replay attack.

Another example of such a method is to measure the average amount of energy present at the ultrasound frequency during each frame containing fricative phonemes (or more generally during each frame containing unvoiced speech) and compare this average amount with the amount of energy present in the audible frequency band during the same frame.

Therefore, the temperature of the molten metal is controlled,

wherein:

x₃is a measure of the perception of speech presence,

E_u(f) is a measure of the amount of energy present at the ultrasonic frequency during the frames containing fricative phonemes (or more specifically, during the frames containing sibilant phonemes, or more generally, during the frames containing unvoiced speech), and

E_a(f) is a measure for the amount of energy present in the audible frequency band during the same frame.

Thus, x can be calculated₃And compares the value to an appropriate threshold to determine with the required degree of confidence whether the speech signal should be considered as being caused by a replay attack.

Another example of such a method is to measure the average amount of energy present at the ultrasound frequency during each frame containing fricative phonemes (or more generally during each frame containing unvoiced speech) and compare this average amount with the amount of energy present in the audible frequency band during the same frame, and also measure the amount of energy present at the ultrasound frequency during the frame containing voiced speech and compare this amount with the amount of energy present in the audible frequency band during the same frame, and then take the ratio of the two ratios.

Therefore, the temperature of the molten metal is controlled,

wherein:

x₄is a measure of the perception of speech presence,

E_u(f) is a measure of the amount of energy present at the ultrasonic frequencies during the frames containing the fricative phonemes (or, more generally, during the frames containing the unvoiced speech),

E_a(f) is a measure of the amount of energy present in the audible band during the frames containing the fricative phonemes (or, more generally, during the frames containing the unvoiced speech),

E_u(v) is a measure of the amount of energy present at the ultrasonic frequency during a frame containing voiced speech, and

E_a(v) is a measure of the amount of energy present in the audible frequency band during the frames containing voiced speech.

Thus, x can be calculated₄And can be compared to an appropriate threshold to determine with the required degree of confidence whether the speech signal should be considered as being caused by a replay attack.

Another example of such an approach is to measure the average amount of energy present at the ultrasound frequency during each frame containing fricative phonemes (or more generally, during each frame containing unvoiced speech) and compare this average amount with the amount of energy present in frames that do not contain speech, which can be considered an ultrasound noise floor.

Therefore, the temperature of the molten metal is controlled,

wherein:

x₅is a measure of speech presence，

E_u(f) Is a measure of the amount of energy present at the ultrasonic frequency during the frames containing the fricative phonemes (or, more generally, during the frames containing the unvoiced speech), and

E_u(n) is a measure of the amount of energy present in the non-speech frame.

Thus, x can be calculated₅And compares the value to an appropriate threshold to determine with the required degree of confidence whether the speech signal should be considered as being caused by a replay attack.

Those skilled in the art will recognize that some aspects of the above described apparatus and methods may be embodied as processor control code, for example on a non-volatile carrier medium such as a diskette, CD-ROM or DVD-ROM, programmed memory such as read-only memory (firmware), or a data carrier such as an optical or electrical signal carrier. For many applications, embodiments of the invention will be implemented on a DSP (digital signal processor), an ASIC (application specific integrated circuit), or an FPGA (field programmable gate array). Thus, the code may comprise conventional program code or microcode or, for example, code for setting up or controlling an ASIC or FPGA. The code may also include code for dynamically configuring a reconfigurable device, such as a reprogrammable array of logic gates. Similarly, the code may include code for a hardware description language such as Verilog (TM) or VHDL (very high speed Integrated Circuit hardware description language). As will be appreciated by those skilled in the art, the code may be distributed between a plurality of coupled components in communication with each other. Embodiments may also be implemented using code running on a field (re-) programmable analog array or similar device, where appropriate, to configure analog hardware.

It should be noted that as used herein, the term "module" is applied to refer to a functional unit or block that may be implemented at least in part by dedicated hardware components, such as custom circuits, and/or at least in part by one or more software processors or appropriate code running on a suitable general purpose processor or the like. The modules themselves may include other modules or functional units. A module may be provided by a plurality of components or sub-modules which need not be co-located and may be provided to run on different integrated circuits and/or on different processors.

Embodiments may be implemented in a host device, particularly a portable and/or battery-powered host device such as a mobile computing device, e.g., a laptop or tablet computer, a gaming console, a remote control device, a home automation controller or a home appliance, including a home temperature or lighting control system, a toy, a machine such as a robot, an audio player, a video player, or a mobile phone (e.g., a smartphone).

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim, "a" or "an" does not exclude a plurality, and a single feature or other element may fulfil the functions of several elements recited in the claims. Any reference signs or labels in the claims should not be construed as limiting the scope.