阅读说明：本技术 一种解析智能手机按键信息的方法 (Method for analyzing key information of smart phone ) 是由 蔡俊辉 李千目 邱天 于 2019-05-23 设计创作，主要内容包括：本发明公开了一种解析智能手机按键信息的方法,根据两个麦克风获取的智能手机触摸屏敲击声音、手机的几何形状,以及触摸屏键盘布局,确定所敲击的按键；根据识别的按键序列,解析输入的文本信息。本发明不需要外界麦克风,即可提取输入的文本信息,并且提高了信息解析的准确性。(The invention discloses a method for analyzing key information of a smart phone, which is characterized in that a knocked key is determined according to the knocking sound of a touch screen of the smart phone, the geometric shape of the smart phone and the layout of a keyboard of the touch screen, which are acquired by two microphones; and analyzing the input text information according to the identified key sequence. The invention can extract the input text information without an external microphone, and improves the accuracy of information analysis.)

1. A method for analyzing key information of a smart phone is characterized by comprising the following steps:

determining the knocked keys according to the intelligent mobile phone touch screen knocking sound acquired by the two microphones, the geometric shape of the mobile phone and the touch screen keyboard layout;

and analyzing the input text information according to the identified key sequence.

2. The method for analyzing key information of a smart phone according to claim 1, wherein after obtaining the knocking sound of the touch screen of the smart phone, noise filtering is performed and then key identification is performed.

3. The method as claimed in claim 2, wherein a frequency-domain bandpass filter is used to remove the noise signal corresponding to the spurious frequency, and the cut-off frequencies of the bandpass filter are set to 600-800HZ and 11-13KHZ, respectively.

4. The method for analyzing key information of a smart phone according to claim 1,2 or 3, wherein the time difference of arrival of the audio peaks in the two microphones is calculated according to the captured key tap sound, and the key position is determined by forming a hyperbolic trace according to the characteristic that the tap point has a constant difference distance from the two microphones.

5. The method of parsing key information for a smartphone of claim 1, wherein the input words and sentences are inferred from the recognized key sequence using natural language processing techniques.

Technical Field

The invention relates to a digital signal processing technology, in particular to a method for analyzing key information of a smart phone.

Background

The smartphone is equipped with at least two microphones. These microphones are connected to the body of the handset and are received even though the slightest sounds produced on the surface of the handset, including the touch screen. When a user clicks on the touch screen to input information, a sound is produced that is generally inaudible to the human ear and therefore ignored due to the presence of ambient noise. However, the sound of these taps can be easily captured using the microphone of the smartphone itself. Recent research efforts have been directed to restoring text using sounds emitted by a mechanical keyboard, typically using an external microphone to capture the sounds and then training a classifier to distinguish the sounds emitted by different keys. However, there is little research on extracting typed text using an audio signal captured by a built-in microphone.

Disclosure of Invention

The invention aims to provide a method for analyzing key information of a smart phone.

The technical solution for realizing the patent of the invention is as follows: a system method for analyzing key information of a smart phone comprises the following steps:

determining the knocked keys according to the intelligent mobile phone touch screen knocking sound acquired by the two microphones, the geometric shape of the mobile phone and the touch screen keyboard layout;

and analyzing the input text information according to the identified key sequence.

As a specific implementation mode, after the knocking sound of the touch screen of the smart phone is acquired, noise filtering is firstly carried out, and then key identification is carried out.

As a more specific implementation, a frequency domain band pass filter is used to remove the noise signal corresponding to the spurious frequency, and the cut-off frequency of the band pass filter is set to 600-800Hz and 11-13KHZ, respectively.

According to the specific implementation mode, the arrival time difference of audio peaks in the two microphones is calculated according to the captured key hitting sound, and the key position is determined by forming a hyperbolic curve track through the characteristic that the hitting point has a constant difference distance from the two microphones.

As a specific embodiment, the input words and sentences are inferred from the recognized keystroke sequences using natural language processing techniques.

Compared with the prior art, the invention has the following remarkable advantages: the invention can extract the input text information without an external microphone, and improves the accuracy of information analysis.

Drawings

Fig. 1 is a flowchart of a method for analyzing key information of a smart phone according to the present invention.

Fig. 2 is a schematic diagram of TDoA computational modeling.

Detailed Description

The invention is further illustrated by the following examples in conjunction with the accompanying drawings.

As shown in fig. 1, the method for analyzing key information of a smart phone includes the following steps:

step 1, determining the position of a touch key according to audio signals captured by two microphones of a smart phone by combining the properties of the two signals received by the microphones, the geometric shape of the smart phone and the layout of a touch screen keyboard;

this phase can be divided into two processes, noise filtering and key identification.

(1) Noise filtering

The frequency spectrum of the pure audio relative to the noise signal is analyzed, the approximate frequency range of the mobile phone knocking sound can be determined, the frequency domain band-pass filter is used for removing the noise signal corresponding to the spurious frequency, and the accuracy of extracting the text information can be obviously improved. According to experimental simulation, the cut-off frequencies of the band-pass filters are set to 600-800Hz and 11-13KHz, respectively, and the audio signal recovered by using the band-pass filters in such a range is very close to the original signal.

(2) Key identification method

There is a limited difference in the distance that sound travels to reach the two microphones when the touchscreen key is tapped. And calculating the time difference of arrival (TDoA) of the audio peaks in the two microphones according to the captured key striking sound, forming a hyperbolic trace by combining the characteristic that the striking point has a constant difference distance from the two microphones, and further determining the key position, namely identifying the key.

The correlation of the two signals is then calculated, with the peak being an estimate of their TDoA. If the microphone 1 peak occurs before the microphone 2 peak, then the click must be made on the half of the keyboard closer to 1. The TDoA size is then used to determine the possible touch screen areas where the click occurs, to extract possible characters, and to computationally model them as shown in FIG. 2.

a. Establishing a mobile phone coordinate system;

b. computing

Where Δ s is the sampling offset, f_sFor sampling frequency, the sampling offset is the value that an audio signal needs to be shifted to maximize the correlation between the two signals.

c. Calculating Δ d ═ v_sound*T DoA，Δd＝|d₁-d₂|，d_iIs the distance of the ith microphone from the point of the touchscreen clicked, where i e {1,2 }.

d. The hyperbola determined from TdoA, is represented as follows:

wherein (x)₀,y₀) Is a vertex, x₀＝c,y₀Min (p, q), where p and q are the positions m of the 1 st microphone in the modeled coordinate system₁And the position m of the 2 nd microphone in the modeling coordinate system₂The distance between the mobile phone and the base of the mobile phone,

l represents the length of the longer side of the mobile phone, y represents x, and x is obtained₀±

I.e. where y belongs to [0, w ]]W is the width of the handset;

e. and after the mobile phone is modeled, the coordinates of each letter are determined, the coordinates of each letter in a mobile phone coordinate system are substituted into a hyperbolic equation, and the actual touch key is determined, so that the final key identification is completed.

Step 2, analyzing possibly input words and sentences according to the recognized key sequence by using a Natural Language Processing (NLP) technology; this phase can be done either on the handset or sent to a remote server.

The analysis method can be carried out on a mobile phone or a remote server.