阅读说明：本技术 一种脑机融合的神经信号测谎方法 (Brain-machine fusion neural signal lie detection method ) 是由 潘纲 王翰文 祁玉 余航 王跃明 于 2021-01-08 设计创作，主要内容包括：本发明公开了一种脑机融合的神经信号测谎方法,其利用多种不同类型的刺激构建快速视觉序列呈现的序列,获得被试的脑信号后,有效预处理和降维,最后用自编码器学习分类和提取有效特征,使得本发明能够有效抵御对抗方法,并且有较好的鲁棒性。本发明将快速序列视觉呈现范式应用到了测谎领域,并且通过算法有效提取特征和分类数据,具有较好的抗干扰性和准确率。(The invention discloses a brain-machine fused neural signal lie detection method, which utilizes a plurality of different types of stimuli to construct a sequence presented by a rapid visual sequence, effectively preprocesses and reduces dimension after obtaining a tested brain signal, and finally learns classification and extracts effective characteristics by using a self-encoder, so that the invention can effectively resist an anti-method and has better robustness. The invention applies the rapid sequence visual presentation paradigm to the lie detection field, and effectively extracts the characteristics and classification data through the algorithm, thereby having better anti-interference performance and accuracy.)

1. A brain-machine fused neural signal lie detection method comprises the following steps:

(1) the method comprises the steps that an RSVP stimulation sequence is generated by combining suspicious information and irrelevant information, and the sequence is divided into three types, namely an image sequence formed by combining a single suspicious image, a reference image and a target image with a plurality of random images;

(2) presenting various RSVP stimulation sequences to a testee, acquiring an electroencephalogram signal induced by the testee through sensory stimulation, and labeling the signal;

(3) preprocessing and extracting characteristics of the electroencephalogram signals in sequence to obtain a characteristic matrix of the signals as a group of samples;

(4) constructing a self-encoder neural network with data reconstruction and classification functions, and training the neural network by using a large number of samples to obtain an electroencephalogram lie detection model;

(5) the electroencephalogram signal of the tester is input into the electroencephalogram lie detection model after being preprocessed and feature extracted, and whether the tester hides suspicious information can be judged.

2. The neural signal lie detection method according to claim 1, characterized in that: the suspicious picture contains suspicious objects to be checked, and is used for inducing the electroencephalogram response of the testee and evaluating whether the testee hides the relevant knowledge or not; the control picture is used as the reference control of the suspicious picture, contains other irrelevant objects of the same kind as the suspicious object and is used for comparing the electroencephalogram response of the testee; the target picture is used for the testee to make a specific reaction to the target picture, and the accuracy of the reaction is used for measuring the attention degree of the testee; and randomly selecting the random picture from the ImageNet database for filling the picture sequence.

3. The neural signal lie detection method according to claim 1, characterized in that: in the step (2), the RSVP stimulation sequence is presented to the testee, pictures in the sequence continuously and rapidly appear at the same position of the screen, and the testee is required to perform key reaction after one sequence is finished; if the sequence contains the target picture, the tested person is required to press a key Y to indicate that the person knows; if the sequence contains suspicious pictures, if the fact of the testee is known, but the fact of the testee is unknown, a lie event is formed; if the sequence contains the reference picture, the key of the tested person is pressed to be N, which indicates that the tested person does not know; after the electroencephalogram signal of the testee is collected, if the testee actually lies, the signal is marked as a positive sample signal, and if the testee does not lie, the signal is marked as a negative sample signal.

4. The neural signal lie detection method according to claim 1, characterized in that: the process of preprocessing the electroencephalogram signals in the step (3) is as follows: firstly, selecting signal data of an electroencephalogram signal which is 200 milliseconds before stimulation and is 1 second in total 800 milliseconds after stimulation to perform slice segmentation, and filtering each data segment by adopting a 4-order Butterworth filter, wherein a band-pass frequency band is 0.3-30 Hz; and after filtering, second-order curve fitting is adopted to remove the baseline drift of the data segment and remove the data segment with large eye movement.

5. The neural signal lie detection method according to claim 1, characterized in that: the process of extracting the characteristics of the electroencephalogram signals in the step (3) is as follows: firstly, determining the optimal electrode combination for electroencephalogram detection by utilizing a greedy algorithm in a spatial dimension, namely, sequentially adding electrodes in each round from a single electrode, calculating the lie detection accuracy of the electrode combination obtained by each newly added electrode, selecting the electrode combination with the highest accuracy, repeating a plurality of rounds, stopping when the accuracy of the electrode combination is not increased any more, and recording electroencephalogram signals measured by each electrode channel; and then, smoothing the electroencephalogram signals by adopting windows in a time dimension, namely sequentially sliding the windows on the signals and taking an average value in each window as a data characteristic, thereby establishing a group of characteristic matrixes with the size of m multiplied by n, wherein m is a space dimension, namely the number of electrodes in an electrode combination, and n is a time dimension, namely the number of windows.

6. The neural signal lie detection method according to claim 1, characterized in that: the self-encoder neural network comprises an encoder, a decoder and a classifier, wherein the encoder is used for compressing input data of the neural network into hidden feature vectors and providing the hidden feature vectors to the decoder and the classifier, the decoder is used for carrying out data reconstruction on the hidden feature vectors, and the classifier carries out secondary classification according to the hidden feature vectors and outputs a classification result.

7. The neural signal lie detection method according to claim 6, wherein: the specific process of training the self-encoder neural network in the step (4) is as follows: and inputting the samples into the network one by one to obtain a classification result output by network prediction, calculating a loss function J (theta) between the classification result and a truth label corresponding to the sample, then reversely propagating and updating network parameters by a gradient descent method according to the loss function J (theta), and iterating until the loss function J (theta) is converged.

8. The neural signal lie detection method according to claim 7, wherein: the expression of the loss function J (θ) is as follows:

J(θ)＝J_recon(θ)+J_class(θ)+J_weight(θ)

wherein: j. the design is a square_recon(θ) is a reconstruction term loss function, J_class(θ) is the classification term loss function, J_weight(θ) is the regularization term loss function, N is the number of samples, s_iThe ith sample is corresponding to the characteristic matrix, s, of the EEG signal_i' is derived from s_iAs input, a feature matrix obtained by corresponding reconstruction of an encoder and a decoder, λ corresponding to J_classWeight of (theta), beta corresponding to J_weightWeight of (theta), l_iIs the truth label of the ith sample, l_i' is derived from s_iAs input to the classification result obtained by the encoder and the classifier corresponding prediction, w_kIs the kth parameter in the network, and K is the number of parameters in the network.

Technical Field

The invention belongs to the technical field of electroencephalogram data analysis, and particularly relates to a brain-machine fused neural signal lie detection method.

Background

Lying is a frequent occurrence in human life, but lies involves a great deal of neural activity and is difficult to be accurately recognized. Existing lie detectors are based primarily on measurements of the peripheral nervous system, including cardiac, vascular and cutaneous electrophysiological activity. As a method for directly measuring central nervous signals, electroencephalogram lie detection can directly measure brain responses when a subject lies, and basic electroencephalogram lie detection uses a method of event-related potentials, and the subject responds differently to different types of stimulated brain signals; for suspicious information, an informed subject will have special electroencephalogram responses, while an unknown subject will not produce these responses.

The document Farwell L A, Donchin E.the Truth Will Out: iterative polygraph ("Lie Detection") With Event-Related Brain Potentials [ J ]. Psychologenology, 1991,28(5):531-547] proposes for the first time the construction of a Lie-detecting paradigm using the names of items in a crime scene as stimuli, which is called the GKT paradigm, which contains three types of different stimuli: suspicious, target, and irrelevant stimuli; the suspicious stimulus is a word of an object in a crime scene and can cause an informed brain signal reaction to be tested; the target stimulation needs a specific reaction to be tested, so that the test can focus on attention; irrelevant stimuli are contrasts of suspicious stimuli and are words irrelevant to the crime scene; experiments have shown that, in the case of multiple repetitions, the response of the informed subject to suspected stimuli is significantly greater than that of unrelated stimuli.

Although the GKT paradigm has a relatively good effect in the common lie detection environment, the documents [ Rosenfeld J P, Soskins M, Bosh G, et al. simple, effective counter measures to P300-based tests of detection of connected information [ J ]. Psychophysiology,2004,41(2):205 + 219] propose a very simple and effective method to resist against the GKT lie detection, and the accuracy of lie detection is reduced from 92% to 42% by several simple countermeasures.

To solve the problem of antagonistic action, the document [ Howard, Bowman, Abdulajeed, et al, Breakthroug tasks- (Sub) clinical science Search and EEG discovery Detection on the Fringe of Aware materials [ J ] Journal of Vision,2015] proposes to apply the fast sequence visual presentation paradigm to the lie Detection domain, in which the contents of the stimulus are the name of the tested and the rest of the unrelated names, resulting in a significant difference between the two signals. However, the technology of the document only discusses that the name of the tested object is used as a stimulus, the stimulus is an inherent characteristic of the tested object, and certain question exists about whether task related information, including pictures or words related to crime scenes, can be effective or not.

Therefore, a reliable lie detection method that can resist antagonistic actions and can be used to detect task-related stimuli is of great value.

Disclosure of Invention

In view of the above, the invention provides a brain-machine fused neural signal lie detection method, which constructs a sequence presented by a rapid visual sequence by using various different types of stimuli, effectively preprocesses and reduces dimensions after obtaining a tested brain signal, and finally learns, classifies and extracts effective features by using a self-encoder, so that the invention can effectively resist a countermeasure method and has better robustness.

A brain-machine fused neural signal lie detection method comprises the following steps:

(1) generating an RSVP (rapid serial visual presentation task) stimulation sequence by combining suspicious information and irrelevant information, wherein the sequence is divided into three types, namely a picture sequence formed by combining a single suspicious picture, a reference picture and a target picture with a plurality of random pictures respectively;

(2) presenting various RSVP stimulation sequences to a testee, acquiring an electroencephalogram signal induced by the testee through sensory stimulation, and labeling the signal;

(3) preprocessing and extracting characteristics of the electroencephalogram signals in sequence to obtain a characteristic matrix of the signals as a group of samples;

(4) constructing a self-encoder neural network with data reconstruction and classification functions, and training the neural network by using a large number of samples to obtain an electroencephalogram lie detection model;

(5) the electroencephalogram signal of the tester is input into the electroencephalogram lie detection model after being preprocessed and feature extracted, and whether the tester hides suspicious information can be judged.

Furthermore, the suspicious picture contains suspicious objects (such as crime-making tools of criminals, jewelry stolen by criminals, etc.) to be checked, and is used for inducing the electroencephalogram response of the testee and evaluating whether the testee hides the relevant knowledge; the control picture is used as the reference control of the suspicious picture, contains other irrelevant objects of the same kind as the suspicious object and is used for comparing the electroencephalogram response of the testee; the target picture is used for the testee to make a specific reaction to the target picture, and the accuracy of the reaction is used for measuring the attention degree of the testee; and randomly selecting the random picture from the ImageNet database for filling the picture sequence.

Further, in the step (2), the RSVP stimulation sequence is presented to the subject, the pictures in the sequence continuously and rapidly appear at the same position on the screen, and after one sequence is finished, the subject is required to perform a key reaction; if the sequence contains the target picture, the tested person is required to press a key Y to indicate that the person knows; if the sequence contains suspicious pictures, if the fact of the testee is known, but the fact of the testee is unknown, a lie event is formed; if the sequence contains the reference picture, the key of the tested person is pressed to be N, which indicates that the tested person does not know; after the electroencephalogram signal of the testee is collected, if the testee actually lies, the signal is marked as a positive sample signal, and if the testee does not lie, the signal is marked as a negative sample signal.

Further, the preprocessing process of the electroencephalogram signal in the step (3) is as follows: firstly, selecting signal data of an electroencephalogram signal which is 200 milliseconds before stimulation and is 1 second in total 800 milliseconds after stimulation to perform slice segmentation, and filtering each data segment by adopting a 4-order Butterworth filter, wherein a band-pass frequency band is 0.3-30 Hz; and after filtering, second-order curve fitting is adopted to remove the baseline drift of the data segment and remove the data segment with large electro-oculogram effect (the amplitude is larger than 50 muV).

Further, the process of extracting the features of the electroencephalogram signal in the step (3) is as follows: firstly, determining the optimal electrode combination for electroencephalogram detection by utilizing a greedy algorithm in a spatial dimension, namely, sequentially adding electrodes in each round from a single electrode, calculating the lie detection accuracy of the electrode combination obtained by each newly added electrode, selecting the electrode combination with the highest accuracy, repeating a plurality of rounds, stopping when the accuracy of the electrode combination is not increased any more, and recording electroencephalogram signals measured by each electrode channel; and then, smoothing the electroencephalogram signals by adopting windows in a time dimension, namely sequentially sliding the windows on the signals and taking an average value in each window as a data characteristic, thereby establishing a group of characteristic matrixes with the size of m multiplied by n, wherein m is a space dimension, namely the number of electrodes in an electrode combination, and n is a time dimension, namely the number of windows.

Further, the self-encoder neural network comprises an encoder, a decoder and a classifier, wherein the encoder is used for compressing input data of the neural network into hidden feature vectors and providing the hidden feature vectors to the decoder and the classifier, the decoder is used for carrying out data reconstruction on the hidden feature vectors, and the classifier carries out secondary classification according to the hidden feature vectors and outputs a classification result.

Further, the specific process of training the self-encoder neural network in the step (4) is as follows: and inputting the samples into the network one by one to obtain a classification result output by network prediction, calculating a loss function J (theta) between the classification result and a truth label corresponding to the sample, then reversely propagating and updating network parameters by a gradient descent method according to the loss function J (theta), and iterating until the loss function J (theta) is converged.

Further, the expression of the loss function J (θ) is as follows:

J(θ)＝J_recon(θ)+J_class(θ)+J_weight(θ)

wherein: j. the design is a square_recon(θ) is a reconstruction term loss function, J_class(theta) as a classification term loss function，J_weight(θ) is the regularization term loss function, N is the number of samples, s_iIs a characteristic matrix, s 'of the electroencephalogram signal corresponding to the ith sample'_iIs given by s_iAs input, a feature matrix obtained by corresponding reconstruction of an encoder and a decoder, λ corresponding to J_classWeight of (theta), beta corresponding to J_weightWeight of (theta), l_iIs the truth label, l 'of the ith sample'_iIs given by s_iAs input to the classification result obtained by the encoder and the classifier corresponding prediction, w_kIs the kth parameter in the network, and K is the number of parameters in the network.

The invention applies the rapid sequence visual presentation paradigm to the lie detection field, and effectively extracts the characteristics and classification data through the algorithm, thereby having better anti-interference performance and accuracy. The RSVP-based experimental paradigm of the present invention ensures that the subject is difficult to challenge the experiment during the experiment and forces the subject to focus on the experiment. The self-encoder algorithm adopted by the invention ensures effective feature extraction and higher classification accuracy, and experiments show that the countermeasure method is basically ineffective for the paradigm, and the lie detection algorithm achieves 87% of average accuracy on 10 tested subjects.

Drawings

Fig. 1 is a schematic diagram of an RSVP stimulation sequence employed in the present invention.

FIG. 2 is a schematic diagram of the average brain electrical signal induced and pre-processed by the RSVP stimulation sequence of the present invention.

Fig. 3 is a schematic diagram of the visualized distribution of the features extracted by the method of the present invention.

Detailed Description

In order to more specifically describe the present invention, the following detailed description is provided for the technical solution of the present invention with reference to the accompanying drawings and the specific embodiments.

The invention relates to an electroencephalogram lie detection method based on rapid sequence visual presentation, which comprises the following steps:

(1) and constructing a stimulation sequence according to the information to be detected. The information to be detected is called suspicious information (Probe) and is used for inducing the reaction of the tested object; irrelevant information is selected as Target information (Target) and contrast information (Control), the contrast information is used as a reference of suspicious information, the Target information needs to be tested to make a specific reaction, and the accuracy of the Target information reaction can be used for measuring the concentration degree of the tested Target information; irrelevant information (irrelevant) is randomly selected from the database of ImageNet. Three different stimulation sequences are formed by suspicious information, target information, contrast information and irrelevant information.

The suspect sequence (Probe trim), Target sequence (Target trim) and Control sequence (Control trim) each contain one corresponding stimulus and eleven unrelated stimuli. Pictures in the stimulation sequence continuously and rapidly appear at the same position of the screen, and after one sequence is finished, a key response needs to be tested; if the sequence contains target information, the tested object needs to press a key Y, which represents that the object knows; if the sequence contains suspicious information, the tested person needs to press a key N, which represents that the person does not know, so as to form lie; if the sequence contains control information, the test also needs to press the "N" key, which means it is unknown. During testing, brain signals to be tested are acquired through brain electrical equipment, and suspicious stimuli, target stimuli and irrelevant stimuli are marked in the signals.

One set of stimulation sequences is shown in fig. 1, which shows three sequences. The target stimulus in the target sequence is a banana, which is informed to be tested in the experiment, the other pictures are all random selected irrelevant pictures, and the sequence is ended when the "+" appears, and the user needs to be tested to press the "Y" key to react (position Btn in the figure); suspicious stimulus in the suspicious sequence is a picture of a necklace, which is an object to be tested to simulate stealing before an experiment, and the object to be tested needs to hide information of the object to be stolen in the experiment, so that an 'N' key is pressed to react when a '+' key appears at the end of the sequence; the control information in the control sequence is a picture of other jewelry items, regardless of the simulated theft experiment, and needs to be tested for "N" key response when a "+" appears at the end of the sequence. Each picture in the sequence lasts for 100ms, no interval exists between pictures, and 2 seconds are used for key judgment after the sequence is finished.

(2) Preprocessing the brain signals obtained in the step (1). Segmenting data through the label, and selecting data from 200 milliseconds before the stimulus appears to 800 milliseconds after the stimulus appears, wherein the total time is 1 second, for slicing segmentation. Filtering the data section by using a 4-order Butterworth filter (Butterworth), wherein the band-pass frequency range is 0.3-30 Hz; after filtering, fitting a second-order curve to remove the baseline drift of the data segment; data with large eye electrical effects (amplitude greater than 50 μ V) are removed.

For example, data is acquired with a DSI-24 dry electrode device, the sampling rate is 300Hz, 19 channels are obtained, and the dimension of the obtained data is 300 × 19. The mean values of the collected data segments are shown in fig. 2, and it can be seen that the suspicious stimuli have significant differences in different cases (innocent/guilty) which are unknown by the test; in the case of guilty, the suspected stimulus effectively elicits a greater brain signaling response, whereas in the case of innocent, the suspected stimulus elicits no significant signaling.

(3) And (3) performing time and space dimensionality reduction on the data obtained in the step (2). In the time dimension, data are averaged through a sliding window, the length of the time window is 200/3ms, the step length is 200/3ms, and the average value is taken in each window, so that the characteristics of the data in the time dimension can be effectively extracted, and the interference of noise in the data is reduced. The first window averages the data from point 1 to point 20 over 300Hz, the second window starts from point 21 to point 40, and so on, which reduces the time dimension of the data from 300 to 20.

And selecting the optimal electrode combination by using a greedy algorithm in the spatial dimension, adding electrodes in sequence in each round from a single electrode, calculating the accuracy of the electrode combination obtained by each newly added electrode, selecting the electrode combination with the highest accuracy, repeating the rounds, and stopping when the accuracy of the electrode combination is not increased any more. In the experiment, the algorithm took 8 of the 19 electrodes, reducing the spatial dimension from 19 to 8.

(4) And (4) classifying and reconstructing the data obtained in the step (3) through a self-encoder, and extracting and classifying effective features. Through the step (3), N brain signal segments { s ] can be obtained₁,s₂,…,s_N}∈R^C*TThese signals are from N stimulation pictures t₁,t₂,…,t_NTheir label is { l }₁,l₂,…,l_NIs e.g. 0, 1. C is the number of electrodes screened out, and T is the number of feature points after dimensionality reduction. The algorithm aims to find a mapping function f (-) and satisfy f(s)_i)＝l_i. A conventional self-encoder is divided into two parts, an encoder and a decoder, the encoder compresses s an input signal into a hidden feature vector q, as shown in the following formula:

q＝σ(W₁s+b₁)

wherein: w₁Representing weights in the coding layer, b₁Represents the amount of bias in the coding layer and σ (-) represents the activation function. The decoder will then reconstruct the latent feature vector q into a signal s', as shown in the following equation:

s′＝σ(W₂q+b₂)

wherein: w₂Representing weights in the decoding layer, b₂Indicating the amount of offset in the decoding layer.

However, in the present invention, the self-encoder needs not only to reconstruct data to extract valid features but also to classify the extracted features. Therefore, the classification layer output classification result l' is added after the hidden feature vector q, as shown in the following formula:

l′＝σ(W₃q+b₃)

at the same time, the loss function is adjusted accordingly, and the loss function of the invention is as follows:

J(θ)＝J_recon(θ)+J_class(θ)+J_weight(θ)

wherein: j (θ) is the total penalty function; j. the design is a square_recon(theta) is a loss function of reconstruction, and an algorithm can accurately reconstruct a signal; j. the design is a square_class(theta) is a classified loss function, ensuring that separable features are extracted by an algorithm; j. the design is a square_weightAnd (theta) is loss of regularization, so that overfitting of the algorithm can be prevented, and the robustness of the algorithm is improved. The detailed calculation of three different losses is as follows:

wherein: n is the sample size of the training; s is the input signal, and the dimensionality is the number of channels multiplied by the time; s' is the reconstructed signal, with dimensions the same as s; l is a label corresponding to the sample; l' is a sample label obtained by model prediction; λ is J_class(θ) weight; k is the number of parameters in the model; w represents a parameter in the model; beta is J_weightThe weight of (θ).

The algorithm optimizes the above loss function by a gradient descent method, calculates an optimal model, extracts effective characteristics and classifies data. FIG. 3 shows the classification effect of the algorithm on a piece of data under test visually, the left image is the original data dimension reduction visualization, and the points of the two types are gathered together; the right graph is a hidden feature layer data dimensionality reduction visualization, the two types of data points are completely separated, and the separable features are effectively extracted by the algorithm and the data are compressed.

(5) Dividing the data into a training set and a verification set, selecting parameters lambda and beta in the model according to the accuracy on the verification set, and classifying unknown data through the optimal model to obtain a label so as to confirm whether the tested data is informed of suspicious information. In the experiment, the total 4 days are detected, 270 sequences of data are detected every day, a total of 540 data of the first two days are used as training, the 270 sequences of the third day are used as verification, and after optimal parameters are selected (lambda is 0.5, beta is 1e-6), all data of the first three days are trained and tested on the 270 sequences of the fourth day.

The invention applies the rapid sequence visual presentation paradigm to the lie detection field, and effectively extracts the characteristics and classification data through the algorithm, thereby having better anti-interference performance and accuracy. In fig. 3, the electroencephalogram lie detection method based on rapid sequence visual presentation is used for classifying data, so that the tested informed and unknown states can be better distinguished.

The foregoing description of the embodiments is provided to enable one of ordinary skill in the art to make and use the invention, and it is to be understood that other modifications of the embodiments, and the generic principles defined herein may be applied to other embodiments without the use of inventive faculty, as will be readily apparent to those skilled in the art. Therefore, the present invention is not limited to the above embodiments, and those skilled in the art should make improvements and modifications to the present invention based on the disclosure of the present invention within the protection scope of the present invention.