阅读说明：本技术 基于微多普勒和视觉感知融合的非接触测谎系统及方法 (Non-contact lie detection system and method based on micro-Doppler and visual perception fusion ) 是由 孙旭 黄广超 吴鹏 于文卓 于 2020-07-07 设计创作，主要内容包括：本发明公开了一种基于微多普勒和视觉感知融合的非接触测谎系统及方法,系统包括摄像头检测模块、毫米波雷达模块、毫米波雷达转台、测谎判定模块、数据储存模块和输出显示模块；毫米波雷达模块包括毫米波雷达和信号处理单元；摄像头检测模块包括摄像头和图像处理单元；毫米波雷达设置在毫米波雷达转台上。本发明同时获取被试者面部微表情特征和生理特征,通过深度学习进行组合判别,可以提升说谎检测的准确率。本发明使用过程中无需接触被试者,可以随时对被试者进行说谎检测,避免被试者故意抵抗说谎检测。(The invention discloses a non-contact lie detection system and a non-contact lie detection method based on micro-Doppler and visual perception fusion, wherein the system comprises a camera detection module, a millimeter wave radar turntable, a lie detection judgment module, a data storage module and an output display module; the millimeter wave radar module comprises a millimeter wave radar and a signal processing unit; the camera detection module comprises a camera and an image processing unit; the millimeter wave radar is arranged on the millimeter wave radar rotary table. The invention simultaneously obtains the facial micro-expression characteristics and the physiological characteristics of the testee, and the accuracy of lie detection can be improved by carrying out combined judgment through deep learning. The invention can detect lie of the testee at any time without contacting the testee in the using process, thereby preventing the testee from resisting the lie detection intentionally.)

1. A non-contact lie detection system based on micro-Doppler and visual perception fusion is characterized by comprising a camera detection module, a millimeter wave radar turntable, a lie detection judgment module, a data storage module and an output display module; the millimeter wave radar module comprises a millimeter wave radar and a signal processing unit; the camera detection module comprises a camera and an image processing unit; the millimeter wave radar is arranged on the millimeter wave radar rotary table;

the camera is used for collecting the image of the upper half of the body of the testee;

the image processing unit is used for extracting facial micro-expression characteristics according to the upper body image of the testee and acquiring the position of the chest;

the millimeter wave radar rotary table is used for guiding the millimeter wave radar wave beam to point to the chest of the testee;

the millimeter wave radar is used for transmitting and receiving millimeter wave signals and generating radar signals;

the signal processing unit is used for processing the radar signal and acquiring physiological sign parameters of the testee;

the lie detection judging module is used for obtaining the lie-speaking probability of the testee through a deep learning algorithm according to the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module;

the data storage module is used for storing the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module in a local disk in a video mode;

and the output display module is used for displaying the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module on the display screen in a video form.

2. The micro-doppler and visual perception fusion based non-contact lie detection system according to claim 1, wherein the physiological sign parameters include heart rate and respiration.

3. A non-contact lie detection method based on micro Doppler and visual perception fusion is characterized by comprising the following steps:

s1, acquiring the image of the upper half of the body of the testee through a camera;

s2, extracting facial micro-expression characteristics according to the upper half body image of the testee through an image processing unit and acquiring the position of the chest;

s3, based on the position of the chest, guiding the millimeter wave radar wave beam to point to the chest of the testee through the millimeter wave radar rotary table, and transmitting and receiving millimeter wave signals by adopting a millimeter wave radar and generating radar signals;

s4, processing the radar signal through the signal processing unit to obtain physiological sign parameters of the testee;

s5, obtaining the lie probability of the testee through a deep learning algorithm according to the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module through a lie detection judgment module;

s6, storing facial expression images of the testee collected by the camera detection module, physiological sign parameters measured by the millimeter wave radar and lie detection probability obtained by the lie detection determination module in a local disk in a video mode through the data storage module;

s7, displaying the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module on the display screen in a video mode through the output display module.

4. The non-contact lie detection method based on micro-doppler and visual perception fusion according to claim 3, wherein the specific method of the step S2 includes the following sub-steps:

s2-1, removing the noise of the upper half body image of the tested person through an image processing unit;

s2-2, extracting feature points of the image without the noise points by using the Hog feature;

s2-3, acquiring feature points of the face and the chest, and setting the sizes of the face and the chest to obtain the face position and the chest position of the testee;

s2-4, marking the micro-expression key points of the face position, and calculating the distance between the position of the micro-expression key points and the average position of the micro-expression key points;

s2-5, extracting scale-invariant feature transformation features from the micro expression key points to obtain feature vectors;

s2-6, obtaining facial micro-expression features by adopting a least square method based on the distance obtained in the step S2-4 and the feature vector obtained in the step S2-5;

s2-7, repeating the steps S2-1 to S2-6, and finishing the continuous acquisition of the micro-expression characteristics of the face of the testee and the chest positioning.

5. The non-contact lie detection method based on micro-doppler and visual perception fusion according to claim 3, wherein the specific method for guiding the millimeter wave radar beam to point to the chest of the subject through the millimeter wave radar turntable based on the chest position in step S3 is as follows:

acquiring the position of the chest of the testee relative to the camera according to the chest position, and acquiring the relative position of the chest of the testee relative to the millimeter wave radar according to the relative position of the camera relative to the millimeter wave radar; and adjusting the orientation of the millimeter wave radar rotary table according to the relative position of the chest of the testee relative to the millimeter wave radar, so that the millimeter wave radar wave beam points to the chest of the testee.

6. The non-contact lie detection method based on micro-doppler and visual perception fusion according to claim 3, wherein the specific method for transmitting and receiving millimeter wave signals and generating radar signals by using millimeter wave radar in step S3 is as follows:

the millimeter wave radar transmits and receives the returned millimeter wave signals, the received millimeter wave signals are subjected to low-noise amplification, filtering and frequency mixing and then converted into intermediate frequency signals, and the intermediate frequency signals are subjected to down-conversion to obtain zero intermediate frequency I/Q two-path baseband signals, namely radar signals.

7. The non-contact lie detection method based on micro-doppler and visual perception fusion according to claim 6, wherein the specific method of the step S4 is:

performing phase demodulation on the zero intermediate frequency I/Q two paths of baseband signals by adopting an arctangent demodulation method to obtain sign information; the physical sign information with the vibration frequency of 0.8-2.0Hz and the amplitude of 0.1-0.5mm is used as heartbeat data through a band-pass filter, the physical sign information with the vibration frequency of 0.1-0.5Hz and the amplitude of 1-12mm is used as respiration data, and the acquisition of the physiological sign parameters of the testee is completed.

8. The non-contact lie detection method based on micro-doppler and visual perception fusion according to claim 3, wherein the specific method of the step S5 is:

acquiring and taking facial micro-expression characteristics of a human face micro-expression image with a known label and respiratory and heartbeat data corresponding to the image as samples, and training by adopting a deep learning algorithm to obtain a lying expression-physiological sign model; the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module are used as the input of the lying expression-physiological sign model, so that the lying probability of the testee is obtained.

Technical Field

The invention relates to the lie detection field, in particular to a non-contact lie detection system and method based on micro-Doppler and visual perception fusion.

Background

Lying is a special behavior that humans mislead others in the form of false statements, distorted facts, and omissions. Automatic lie detection is an important field for research of various subjects such as electronic science, computer linguistics, psychology and information institutions. Since the human ability to detect lie is almost a random guess, there is a need for a scientific, reliable automated method to detect lie. The automatic lie detection technology can judge whether people lie or not by automatically analyzing behaviors, languages and various physiological indexes of people, plays an important role in criminal case processing, medical treatment, judicial expertise and other professions, and has wide application scenes.

The existing lie detection method generally adopts a wearable sensor to monitor the change of relevant vital sign parameters of a testee, so as to detect whether the testee lies. The disadvantages of this type of detection method are: the testee can know that the testee is being detected, psychological preparation is easy to carry out, and emotional changes can be actively adjusted and overcome, so that the detection result is deviated.

Disclosure of Invention

Aiming at the defects in the prior art, the non-contact lie detection system and method based on the micro-Doppler and visual perception fusion provided by the invention solve the problem that the traditional lie detection method is easy to be deceived by a testee to cause low accuracy.

In order to achieve the purpose of the invention, the invention adopts the technical scheme that:

the non-contact lie detection system based on micro-Doppler and visual perception fusion comprises a camera detection module, a millimeter wave radar turntable, a lie detection judgment module, a data storage module and an output display module; the millimeter wave radar module comprises a millimeter wave radar and a signal processing unit; the camera detection module comprises a camera and an image processing unit; the millimeter wave radar is arranged on the millimeter wave radar rotary table;

the camera is used for collecting the image of the upper half of the body of the testee;

the image processing unit is used for extracting facial micro-expression characteristics according to the upper body image of the testee and acquiring the position of the chest;

the millimeter wave radar rotary table is used for guiding the millimeter wave radar wave beam to point to the chest of the testee;

a millimeter wave radar for transmitting and receiving a millimeter wave signal and generating a radar signal;

the signal processing unit is used for processing the radar signal and acquiring physiological sign parameters of the testee;

the lie detection judging module is used for obtaining the lie-speaking probability of the testee through a deep learning algorithm according to the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module;

the data storage module is used for storing the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module in a local disk in a video mode;

and the output display module is used for displaying the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module on the display screen in a video form.

Further, the physiological sign parameters include heart rate and respiration.

The non-contact lie detection method based on micro-Doppler and visual perception fusion is provided, and comprises the following steps:

s1, acquiring the image of the upper half of the body of the testee through a camera;

s2, extracting facial micro-expression characteristics according to the upper half body image of the testee through an image processing unit and acquiring the position of the chest;

s3, based on the position of the chest, guiding the millimeter wave radar wave beam to point to the chest of the testee through the millimeter wave radar rotary table, and transmitting and receiving millimeter wave signals by adopting a millimeter wave radar and generating radar signals;

s4, processing the radar signal through the signal processing unit to obtain physiological sign parameters of the testee;

s5, obtaining the lie probability of the testee through a deep learning algorithm according to the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module through a lie detection judgment module;

s6, storing facial expression images of the testee collected by the camera detection module, physiological sign parameters measured by the millimeter wave radar and lie detection probability obtained by the lie detection determination module in a local disk in a video mode through the data storage module;

s7, displaying the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module on the display screen in a video mode through the output display module.

Further, the specific method of step S2 includes the following sub-steps:

s2-1, removing the noise of the upper half body image of the tested person through an image processing unit;

s2-2, extracting feature points of the image without the noise points by using the Hog feature;

s2-3, acquiring feature points of the face and the chest, and setting the sizes of the face and the chest to obtain the face position and the chest position of the testee;

s2-4, marking the micro-expression key points of the face position, and calculating the distance between the position of the micro-expression key points and the average position of the micro-expression key points;

s2-5, extracting scale-invariant feature transformation features from the micro expression key points to obtain feature vectors;

s2-6, obtaining facial micro-expression features by adopting a least square method based on the distance obtained in the step S2-4 and the feature vector obtained in the step S2-5;

s2-7, repeating the steps S2-1 to S2-6, and finishing the continuous acquisition of the micro-expression characteristics of the face of the testee and the chest positioning.

Further, based on the chest position in step S3, the specific method for guiding the millimeter wave radar beam to point to the chest of the subject by the millimeter wave radar turntable is as follows:

acquiring the position of the chest of the testee relative to the camera according to the chest position, and acquiring the relative position of the chest of the testee relative to the millimeter wave radar according to the relative position of the camera relative to the millimeter wave radar; and adjusting the orientation of the millimeter wave radar rotary table according to the relative position of the chest of the testee relative to the millimeter wave radar, so that the millimeter wave radar wave beam points to the chest of the testee.

Further, the specific method for transmitting and receiving the millimeter wave signal and generating the radar signal by using the millimeter wave radar in step S3 is as follows:

the millimeter wave radar transmits and receives the returned millimeter wave signals, the received millimeter wave signals are subjected to low-noise amplification, filtering and frequency mixing and then converted into intermediate frequency signals, and the intermediate frequency signals are subjected to down-conversion to obtain zero intermediate frequency I/Q two-path baseband signals, namely radar signals.

Further, the specific method of step S4 is:

performing phase demodulation on the zero intermediate frequency I/Q two paths of baseband signals by adopting an arctangent demodulation method to obtain sign information; the physical sign information with the vibration frequency of 0.8-2.0Hz and the amplitude of 0.1-0.5mm is used as heartbeat data through a band-pass filter, the physical sign information with the vibration frequency of 0.1-0.5Hz and the amplitude of 1-12mm is used as respiration data, and the acquisition of the physiological sign parameters of the testee is completed.

Further, the specific method of step S5 is:

acquiring and taking facial micro-expression characteristics of a human face micro-expression image with a known label and respiratory and heartbeat data corresponding to the image as samples, and training by adopting a deep learning algorithm to obtain a lying expression-physiological sign model; the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module are used as the input of the lying expression-physiological sign model, so that the lying probability of the testee is obtained.

The invention has the beneficial effects that:

1. the invention simultaneously obtains the facial micro-expression characteristics and the physiological characteristics of the testee, and the accuracy of lie detection can be improved by carrying out combined judgment through deep learning.

2. The invention can detect lie of the testee at any time without contacting the testee in the using process, thereby preventing the testee from resisting the lie detection intentionally.

Drawings

FIG. 1 is a schematic flow chart of the present invention.

Detailed Description

The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.

The non-contact lie detection system based on the micro-Doppler and visual perception fusion comprises a camera detection module, a millimeter wave radar turntable, a lie detection judgment module, a data storage module and an output display module; the millimeter wave radar module comprises a millimeter wave radar and a signal processing unit; the camera detection module comprises a camera and an image processing unit; the millimeter wave radar is arranged on the millimeter wave radar rotary table;

the camera is used for collecting the image of the upper half of the body of the testee;

the image processing unit is used for extracting facial micro-expression characteristics according to the upper body image of the testee and acquiring the position of the chest;

the millimeter wave radar rotary table is used for guiding the millimeter wave radar wave beam to point to the chest of the testee;

a millimeter wave radar for transmitting and receiving a millimeter wave signal and generating a radar signal;

the signal processing unit is used for processing the radar signal and acquiring physiological sign parameters of the testee; the physiological sign parameters comprise heart rate and respiration;

the lie detection judging module is used for obtaining the lie-speaking probability of the testee through a deep learning algorithm according to the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module;

the data storage module is used for storing the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module in a local disk in a video mode;

and the output display module is used for displaying the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module on the display screen in a video form.

As shown in fig. 1, the non-contact lie detection method based on micro-doppler and visual perception fusion includes the following steps:

s1, acquiring the image of the upper half of the body of the testee through a camera;

s2, extracting facial micro-expression characteristics according to the upper half body image of the testee through an image processing unit and acquiring the position of the chest;

s3, based on the position of the chest, guiding the millimeter wave radar wave beam to point to the chest of the testee through the millimeter wave radar rotary table, and transmitting and receiving millimeter wave signals by adopting a millimeter wave radar and generating radar signals;

s4, processing the radar signal through the signal processing unit to obtain physiological sign parameters of the testee;

s5, obtaining the lie probability of the testee through a deep learning algorithm according to the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module through a lie detection judgment module;

s6, storing facial expression images of the testee collected by the camera detection module, physiological sign parameters measured by the millimeter wave radar and lie detection probability obtained by the lie detection determination module in a local disk in a video mode through the data storage module;

s7, displaying the facial expression image of the testee collected by the camera detection module, the physiological sign parameters measured by the millimeter wave radar and the lie detection probability obtained by the lie detection judgment module on the display screen in a video mode through the output display module.

The specific method of step S2 includes the following substeps:

s2-1, removing the noise of the upper half body image of the tested person through an image processing unit;

s2-2, extracting feature points of the image without the noise points by using the Hog feature;

s2-3, acquiring feature points of the face and the chest, and setting the sizes of the face and the chest to obtain the face position and the chest position of the testee;

s2-4, marking the micro-expression key points of the face position, and calculating the distance between the position of the micro-expression key points and the average position of the micro-expression key points;

s2-5, extracting scale-invariant feature transformation features from the micro expression key points to obtain feature vectors;

s2-6, obtaining facial micro-expression features by adopting a least square method based on the distance obtained in the step S2-4 and the feature vector obtained in the step S2-5;

s2-7, repeating the steps S2-1 to S2-6, and finishing the continuous acquisition of the micro-expression characteristics of the face of the testee and the chest positioning.

Based on the chest position in step S3, the specific method for guiding the millimeter wave radar beam to point to the chest of the subject by the millimeter wave radar turntable is as follows: acquiring the position of the chest of the testee relative to the camera according to the chest position, and acquiring the relative position of the chest of the testee relative to the millimeter wave radar according to the relative position of the camera relative to the millimeter wave radar; and adjusting the orientation of the millimeter wave radar rotary table according to the relative position of the chest of the testee relative to the millimeter wave radar, so that the millimeter wave radar wave beam points to the chest of the testee.

The specific method for transmitting and receiving the millimeter wave signal and generating the radar signal by using the millimeter wave radar in the step S3 is as follows: the millimeter wave radar transmits and receives the returned millimeter wave signals, the received millimeter wave signals are subjected to low-noise amplification, filtering and frequency mixing and then converted into intermediate frequency signals, and the intermediate frequency signals are subjected to down-conversion to obtain zero intermediate frequency I/Q two-path baseband signals, namely radar signals.

The specific method of step S4 is: performing phase demodulation on the zero intermediate frequency I/Q two paths of baseband signals by adopting an arctangent demodulation method to obtain sign information; the physical sign information with the vibration frequency of 0.8-2.0Hz and the amplitude of 0.1-0.5mm is used as heartbeat data through a band-pass filter, the physical sign information with the vibration frequency of 0.1-0.5Hz and the amplitude of 1-12mm is used as respiration data, and the acquisition of the physiological sign parameters of the testee is completed.

The specific method of step S5 is: acquiring and taking facial micro-expression characteristics of a human face micro-expression image with a known label and respiratory and heartbeat data corresponding to the image as samples, and training by adopting a deep learning algorithm to obtain a lying expression-physiological sign model; the facial micro-expression characteristics of the testee collected by the camera detection module and the physiological sign parameters obtained by the millimeter wave radar module are used as the input of the lying expression-physiological sign model. Expression recognition and lie determination can be implemented in the same artificial neural network by using different functional modules thereof. The lie detection method based on the facial expression and the physiological characteristics and implemented by adopting the deep learning algorithm comprises two stages of expression recognition and lie judgment. The task of the expression recognition stage is to recognize facial expressions based on the feature vector sequence of the human face image features of the human subject; and the lie judging stage is used for comparing the trained lie expression-physiological sign model with the identified facial expression of the testee and the physiological sign parameters of the testee and predicting the lie probability of the testee.

In an embodiment of the invention, the millimeter wave radar adopts an MIMO (multiple input multiple output) array antenna, so that a narrower beam width can be obtained, and the interference of other personnel can be avoided and the signal-to-noise ratio can be improved by utilizing the array antenna and the beam forming method.

In the specific implementation process, when a radio frequency signal of the millimeter wave radar reaches a target and is reflected, frequency modulation can be generated due to the movement of the target, and if the target moves at a speed of v (t) m/s, the frequency of the reflected signal shifts according to the doppler shift:

wherein f is_dIs the Doppler shift, in Hz; f isThe transmission frequency, in Hz; c is the signal propagation speed in m/s, t is the elapsed time in s, and λ is the wavelength of the transmitted signal in m. Assuming that the human target motion is denoted as x (t), the doppler shift of the reflected signal can be described as a phase modulation:

when the thorax of a human body is used as a detection target of a radar, the displacement of the chest wall generates proportional modulation on the radar carrier phase. Through phase demodulation, time-varying phase information proportional to the time-varying displacement of the thoracic cavity is obtained, and then the sign information related to respiration and heartbeat can be detected.

In conclusion, the facial micro-expression characteristic and the physiological characteristic of the testee are obtained at the same time, and the accuracy of lie detection can be improved by performing combined judgment through deep learning. The invention can detect lie of the testee at any time without contacting the testee in the using process, thereby preventing the testee from resisting the lie detection intentionally.