阅读说明：本技术 一种基于高效多源胶囊网络的跨被试eeg认知状态检测方法 (Cross-tested EEG (electroencephalogram) cognitive state detection method based on efficient multi-source capsule network ) 是由 方欣 戴国骏 赵月 张振炎 吴政轩 金燕萍 吴琪 夏念章 刘洋 曾虹 于 2021-09-30 设计创作，主要内容包括：本发明提出一种基于高效多源胶囊网络的跨被试EEG认知状态检测方法。本发明通过将目标域和多源域的特征分布对齐,以实现域间特征的有效迁移。将EEG数据构建成多通道一维结构,提高了训练效率,同时提高模型性能。其次,引入自表达模块来捕捉样本之间的潜在联系,能够很好地适应不同任务下具有显著个体差异的跨被试EEG数据分析。最后提出了基于动态子胶囊的空间注意力算法来进一步学习EEG数据空间层次上的细粒度特征信息,有效刻画了EEG数据的部分间空间关系和部分-整体层次关系。本发明有效避免了脑认知计算领域脑电信号的个体差异性问题,可适用于任何任务下基于EEG的认知状态识别,具有较强的泛化能力,能够很好的适用于临床诊断和实际应用。(The invention provides a cross-tested EEG cognitive state detection method based on an efficient multi-source capsule network. The invention realizes the effective transfer of the inter-domain characteristics by aligning the characteristic distribution of the target domain and the multi-source domain. EEG data is constructed into a multi-channel one-dimensional structure, so that training efficiency is improved, and model performance is improved. Secondly, the introduction of the self-expression module to capture potential connections between samples can be well adapted to cross-subject EEG data analysis with significant individual differences under different tasks. Finally, a space attention algorithm based on dynamic sub-capsules is provided to further learn fine-grained characteristic information on EEG data space hierarchy, and the partial space relation and the partial-whole hierarchy relation of EEG data are effectively described. The method effectively avoids the problem of individual difference of electroencephalogram signals in the field of brain cognitive computation, is suitable for cognitive state recognition based on EEG under any task, has strong generalization capability, and can be well suitable for clinical diagnosis and practical application.)

1. A cross-tested EEG cognitive state detection method based on an efficient multi-source capsule network is characterized by comprising the following steps:

step 1: data acquisition

The adopted fatigue driving electroencephalogram data set comprises the following data: the electroencephalogram EEG data of a plurality of healthy subjects with good driving experience are tested, each subject fills in an NASA-TLX questionnaire after the test, and two mental states of TAV3 and DROWS are selected for analysis according to the NASA-TLX questionnaire;

step 2: data pre-processing

Aiming at original EEG data, firstly, a band-pass filter is used for eliminating high-frequency noise and power frequency interference signals except for self-generated EEG signals, then an independent component analysis method is used for processing, and finally, power spectral density is used for extracting EEG signal characteristics so as to provide stable signal characteristics for subsequent model construction.

And step 3: a cross-tested EEG cognitive state detection method based on an E3GCAPS algorithm.

2. The method for detecting the cognitive state of the cross-tested EEG based on the efficient multi-source capsule network as claimed in claim 1, wherein the step 3 is implemented as follows:

inputting: firstly, multi-source domain sample data with cognitive state labelWherein N is the total number of source domains, N is the nth source domain,a sample representing the nth source domain,representing a real label corresponding to the nth source domain sample;

second, target domain sample data without cognitive state labelWhereinThe ith sample, | X, representing the target domain_TL is the total number of target domain samples;

(iii) maximum iteration number T, tag class K is {1,2, …, K }, where K is the total number of tag classes;

3-1. taking into account the connections between EEG channels, two-dimensional EEG data is converted into a multi-channel one-dimensional structure, representing each sample of the source and target domains as WhereinChannels represents the number of Channels of the sample, and Features represents the number of Features of each channel of the sample;

3-2, using the point-by-point convolution activated by batch normalization and exponential linear units as a common feature extractor G (-) to extract the common domain invariant features; ith sample in nth source domainThe obtained public domain invariant features areSimplify it intoIth sample in target domainThe obtained public domain invariant features areSimplify it into

3-3, considering the sensitivity of the EEG data outliers, a plurality of self-expression modules are further introduced to acquire potential relation among samples;

3-4. adding a regularization term L to effectively eliminate the dependence on a small amount of data_wcThe self-expression class coefficient C is restrained, so that overfitting is prevented, and the generalization capability is improved; l is_wcIs calculated as follows:

wherein T represents matrix transposition, M represents the number of batch training samples, and alpha represents a hyperparameter;

the total regularization loss is:

wherein the content of the first and second substances,for the regularization term of the nth source domain,regularization terms corresponding to the target domain;

3-5, the domain invariant features output by the common feature extractor are used as input, a plurality of EEGCAPS classifiers are introduced, and various fine-grained features of EEG are effectively extracted, wherein the fine-grained features comprise: extracting domain-specific features from the common domain invariant features, and mapping the domain-specific features to a space of a higher latitude; obtaining capsule layers with vector outputs from an initial capsule layerWherein e^lFor sub-capsules, the latitude of each sub-capsule in the first layer is d^l(ii) a Note that the number of output sub-capsules at this time is p,a jth sub-capsule representing an nth source domain,a jth sub-capsule representing the target domain;

to normalize features, the features are compressed to a range of 0-1 by a squash function, which can be expressed as:

wherein, F_jFor a certain sub-capsule in the current source domain or target domain, | | F_jI means F_jL of₁Norm, | | F_j||²Is represented by F_jL of₂A norm; the output of the jth capsule in the nth source domain isIs marked asThe jth capsule in the target domain is output asIs marked as

3-6, considering the strong spatial correlation of EEG, introducing a dynamic sub-capsule-based spatial attention route, wherein the sub-capsule is routed to the whole of the sub-capsule through the spatial attention route and outputs a cognitive capsule;

3-7, the norm length of the cognitive capsule represents the existence probability of the cognitive capsule; using marginal loss L_capApplying to each cognitive capsule to optimize the overall capsule network module; marginal loss L_capIs calculated as follows:

wherein the content of the first and second substances,is a cognitive capsuleL of₁Norm, if the sample belongs to class k, then Y_k1, otherwise Y_k＝0，m⁺、m^-And λ is a hyper-parameter, max represents the maximum of the two numbers;

3-8, note that the current nth source domain isThe k-th cognitive capsule has a norm length ofThe target domain isThe norm length of the k' cognitive capsule isMeasuring the distance of the predicted distribution of the source domain and the target domain by using the maximum mean difference according to the norm length of each cognitive capsule, aligning the two distributions to obtain the lossThe calculation is as follows:

wherein H is a regenerated nuclear Hilbert space,represents a feature mapping function that maps a domain's particular feature distribution to H;

the total MMD loss was:

3-9. calculate the overall objective optimization function, defined as follows:

L_total＝L_cap+βL_mmd+γL_w#(13)

wherein β and γ are the hyper-parameters of the model;

3-10, repeating the step 3-1 to the step 3-9 until the iteration is carried out for T times;

and (3) outputting: the target domain sample labels correspond to class labels of the integrated results of all classifiers.

3. The method for detecting the cognitive state of the cross-tested EEG based on the efficient multi-source capsule network as claimed in claim 2, wherein the steps 3-3 are implemented as follows:

each self-expression module comprises a full connection layer and a softmax function; using a fully connected layer (FC) as a classifier, mapping the learned feature distribution to a sample label space, and using a softmax function to obtain a class coefficient C from the output of the fully connected layer;

the softmax function is:

wherein, C_iRepresenting samples z from a source or target domain_iThe class coefficient of (a) is determined,denotes the conversion of the FC layer, f_ikDenotes f_iOf (a) belongs to the kth class.

4. The method for detecting the cognitive state of the tested EEG based on the efficient multi-source capsule network according to claim 2 or 3, wherein the steps 3-6 are implemented as follows:

3-6-1, setting the routing times r, initializing the number of layers l to be 0, initializing a prior weight matrix W and a prior probability matrix a, initializing a routing protocol coefficient b, expanding and outputting a category coefficient C in a dimensionality mode, and projecting the sub-capsule into a cognitive capsule:

wherein the content of the first and second substances,represents a sub-capsule T 'in the source domain or the target domain'_jThe l layer of (b) represents C'_jFor the class coefficient of the jth sub-capsule after the dimension expansion,is T'_jAnd cognitive capsule T_kA prior weight matrix at layer l;

3-6-2, obtaining attention coupling coefficient r based on sub-capsule_jk：

Wherein, a_jkIs T'_jAndthe prior probability matrix of (a) is,is the routing protocol coefficient, t_jkIs through learning of T'_jAndspatial attention weights calculated from the spatial relationship therebetween;

3-6-3. obtaining the weighted sum feature s_k：

3-6-4. obtaining output capsule

3-6-5, obtaining routing protocol coefficient

Wherein cos (·) represents a cosine similarity function;

3-6-6. the number of layers l is l +1, repeating the steps 3-6-2 to 3-6-5 until the routing times r are reached, and recording that the k-th cognitive capsule is T_k。

Technical Field

The invention relates to a neuroelectrophysiological signal analysis technology in the field of brain cognitive computation and a multi-source domain adaptation model construction method in the field of unsupervised learning, in particular to a method for detecting a cognitive state by using electroencephalogram signals (EEG) based on a high-efficiency multi-source capsule network. The invention not only can effectively solve the problem of obvious difference of different tested individuals, but also can effectively explain the internal mechanism of the correlation between the EEG characteristics and the cognitive state, and in addition, the training efficiency is effectively improved while the capsule network performance is maintained.

Background

Electroencephalogram (EEG) is a signal which can reflect the cognitive activity of the human brain most and is a key index of a cognitive state detection task. In recent years, EEG-based cognitive state detection methods have received increasing research attention due to their high temporal resolution, non-invasiveness, and low cost, and are widely used for various tasks such as disease diagnosis, driver fatigue state detection, and emotion recognition.

Conventional EEG-based analysis methods typically combine appropriate feature extraction with classifiers to identify different cognitive states for different tasks by extracting and analyzing EEG potential features. Although these methods have high discrimination performance, most of them analyze EEG session data in a test subject or EEG session data obtained by mixing a plurality of test subjects, respectively, and implement cross-session generalization. Due to the features of large individual variability, low signal-to-noise ratio, and non-linearity of EEG signals, the performance of the above methods in cross-subject analysis may be severely degraded, impeding the development of EEG-based cross-subject cognitive state detection. Therefore, it is important to construct a general cross-test EEG cognitive state detection method.

Recently, Deep Learning (DL) has shown great promise in helping to understand complex EEG signals due to its good feature representation capability. However, to obtain better neural network performance, DL requires larger data volume, deeper networks and more complex structures, whereas EEG data is generally limited small sample data and is characterized by large individual variability, and some current DL methods may not be suitable for EEG analysis across the test subject. In particular, with classical Convolutional Neural Networks (CNNs) and their variants, it is also difficult to establish intrinsic mechanisms that explicitly learn or explain the correlation between EEG features and cognitive states, and to analyze why they are not suitable for cross-test EEG data with strong correlation on spatial slices.

The capsule network is an interpretable framework, adopts a new 'vector input vector output' characteristic information transmission strategy, and introduces a dynamic routing algorithm to describe the relationship between parts (vector representation of the bottom-layer characteristic) and the whole (vector representation of the upper-layer characteristic). Under the condition of limited training samples, the dynamic routing mechanism of the capsule network can still obtain better performance. There have been some research efforts to apply capsule networks and variants thereof to EEG-based analysis, such as motor imagery classification, driver alertness estimation, etc.

While existing capsule-based EEG analysis methods have made rapid progress, there are still many challenges to analyzing EEG across subjects: firstly, the original dynamic routing algorithm of the capsule network is sensitive to abnormal values caused by EEG obvious individual difference; second, the dynamic routing process of the capsule network, while able to describe a hierarchical relationship from local to global, is unable to explain the underlying inter-part interaction problem. In particular, for multi-channel EEG analysis, it is difficult to describe the interaction between channels. Theoretically, a strong correlation exists between cognitive states and brain function connection modes, different cognitive states cause differences of activities of different brain function regions, the capsule network sacrifices the training performance to realize the interpretability of the capsule network, and the problem to be solved is how to improve the training efficiency while maintaining the performance of the capsule network.

In addition, relevant documents show that, different from the single-source domain adaptation problem, the multi-source domain transfer learning simultaneously transfers the knowledge of a plurality of source domains to the target domain to assist the learning of the target domain, the data are different from the target domain and different from each other, and the method can effectively solve the problem of individual difference of EEG.

In summary, aiming at the above challenges, the invention will take the cross-tested, interpretable, efficient and multi-source domain as a key starting point to construct the cross-tested EEG cognitive state detection method based on the efficient multi-source capsule network, so as to be more efficiently applied in the field of neurophysiological signal analysis.

Disclosure of Invention

The invention provides a cross-tested EEG cognitive state detection method based on an efficient multi-source capsule network, which fully learns the structural characteristics of a sample according to the highly nonlinear and obvious individual difference conditions of EEG data and the requirements on the accuracy, interpretability and efficiency of the EEG cognitive state detection method, and realizes the effective transfer of inter-domain characteristics by aligning the characteristic distribution of a target domain and a multi-source domain.

The cognitive state detection method based on the EEG comprehensively considers the relevance and the similarity of the cross-test, classifies the cognitive state of the test by carrying out characteristic analysis on the EEG, realizes the detection of the cross-test cognitive state under different experimental tasks, such as the cognitive state (addict and normal control group) of a network game addict, the cognitive state (waking and fatigue) of a driver driving task, the cognitive state (negative, neutral and positive emotions) of the test under different emotional stimulation tasks and the like, and can process different EEG data sets.

The invention is based on fatigue driving electroencephalogram data set, and the technical proposal adopted for overcoming the defects of the prior method is as follows:

step 1: data acquisition

The fatigue driving electroencephalogram data set adopted by the invention is electroencephalogram EEG data of a plurality of healthy tested electroencephalograms with good driving experience, and each tested electroencephalogram is filled with an NASA-TLX questionnaire after the test so as to provide subjective workload perception. According to the NASA-TLX questionnaire, the present invention selected two mental states, TAV3 and DROWS, as analyses.

Step 2: data pre-processing

In order to further filter noise and remove artifacts, the EEG signal processing method aims at original EEG data, firstly, a band-pass filter (1-30Hz) is used for eliminating signals such as high-frequency noise, power frequency interference and the like except for spontaneous EEG signals, then, an Independent Component Analysis (ICA) method is used for processing, and finally, EEG signal features are extracted by using Power Spectral Density (PSD) so as to provide stable signal features for subsequent model construction.

And step 3: cross-tested EEG cognitive state detection method based on E3GCAPS algorithm

Inputting: firstly, multi-source domain sample data with cognitive state labelWhere N is the total number of source domains (i.e., the total number of subjects), N is the nth source domain,a sample representing the nth source domain,representing the true label corresponding to the nth source domain sample.

Second, target domain sample data without cognitive state labelWhereinThe ith sample, | X, representing the target domain_TAnd | is the total number of target domain samples.

And c, the maximum iteration number T, and the label category K is {1,2, …, K }, wherein K is the total number of label categories.

And 3-1, considering the relation among the EEG channels, converting the two-dimensional EEG data into a multi-channel one-dimensional structure, and greatly improving the training efficiency while keeping the model performance. Representing each sample of the source and target domains asWherein Channels represents the number of Channels of the sample and Features represents the number of Features per channel of the sample.

3-2. extracting the common domain invariant features using Batch Normalization (BN) and Exponential Linear Unit (ELU) activated point-by-point convolution as a common feature extractor G (-), and extracting the common domain invariant features. Ith sample in nth source domainThe obtained public domain invariant features areSimplify it intoIth sample in target domainThe obtained public domain invariant features areSimplify it into

3-3. taking into account the sensitivity of the EEG data outliers, a number of self-expression modules were further introduced to obtain potential connections between samples. Each self-expression module comprises a full connection layer and a softmax function; the learned feature distribution is mapped to the sample label space using the fully-connected layer (FC) as a classifier, and the class coefficient C is obtained from the output of this fully-connected layer using the softmax function. The softmax function is:

wherein, C_iRepresenting samples z from a source or target domain_iThe class coefficient of (a) is determined,denotes the conversion of the FC layer, f_ikDenotes f_iOf (a) belongs to the kth class.

3-4. adding a regularization term L to effectively eliminate the dependence on a small amount of data_wcAnd the self-expression class coefficient C is restrained, so that overfitting is prevented, and the generalization capability is improved. L is_wcIs calculated as follows:

where T represents the matrix transpose, M represents the number of training samples in a batch, and α represents the hyperparameter.

The total regularization loss is:

wherein the content of the first and second substances,for the regularization term of the nth source domain,and the regularization term corresponds to the target domain.

3-5, the domain invariant features output by the common feature extractor are used as input, a plurality of EEGCAPS classifiers are introduced, and various fine-grained features of EEG are effectively extracted, wherein the fine-grained features comprise: extracting domain-specific features from the common domain invariant features, and mapping the domain-specific features to a space of a higher latitude; obtaining capsule layers with vector outputs by means of initial capsule layers (Primarycaps)Wherein e^lFor sub-capsules, the latitude of each sub-capsule in the first layer is d^l. Note that the number of output sub-capsules at this time is p,a jth sub-capsule representing an nth source domain,the jth sub-capsule representing the target domain.

To normalize features, the features are compressed to a range of 0-1 by a squash function, which can be expressed as:

wherein, F_jFor a certain sub-capsule in the current source domain or target domain, | | F_jI means F_jL of₁Norm, | | F_j||²Is represented by F_jL of₂And (4) norm. The output of the jth capsule in the nth source domain isIs marked asThe jth capsule in the target domain is output asIs marked as

3-6, considering the strong spatial correlation of EEG, introducing a dynamic sub-capsule based spatial attention route, wherein the sub-capsule is routed to the whole of the sub-capsule through the spatial attention route, and outputting a cognitive capsule, specifically:

setting routing times r, initializing the number of layers l to be 0, initializing a prior weight matrix W and a prior probability matrix a, initializing a routing protocol coefficient b, expanding a category coefficient C dimension to output C', and projecting a sub-capsule into a cognitive capsule:

wherein the content of the first and second substances,represents a sub-capsule T 'in the source domain or the target domain'_jThe l layer of (b) represents C'_jFor the class coefficient of the jth sub-capsule after the dimension expansion,is T'_jAnd cognitive capsule T_kA priori weight matrix at layer l.

② obtaining attention coupling coefficient r based on sub-capsule_jk：

Wherein, a_jkIs T'_jAndthe prior probability matrix of (a) is,is the routing protocol coefficient, t_jkIs through learning of T'_jAndspatial attention weights calculated from the spatial relationship between them.

Obtaining weighted sum characteristic s_k：

Acquiring output capsule

Fifthly, obtaining the routing protocol coefficient

Where cos (·) represents the cosine similarity function.

Sixthly, repeating the step (l) to (v) until reaching the routing times r, and recording the k cognitive capsule as T_k。

3-7, the norm length of the cognitive capsule represents the existence probability of the cognitive capsule. Using marginal loss L_capApplied to each cognitive capsule to optimize the overall capsule network module. Marginal loss L_capIs calculated as follows:

wherein the content of the first and second substances,is a cognitive capsuleL of₁Norm, if the sample belongs to class k, then Y_k1, otherwise Y_k＝0，m⁺、m^-And λ is the hyperparameter, max represents the maximum of the two numbers.

3-8, note that the current nth source domain isThe k-th cognitive capsule has a norm length ofThe target domain isThe norm length of the k' cognitive capsule isMeasuring the distance of the predicted distribution of the source domain and the target domain by using Maximum Mean Difference (MMD) according to the norm length of each cognitive capsule, aligning the two distributions to obtain lossThe calculation is as follows:

wherein H is a regenerated nuclear Hilbert space (RKHS),representing a feature mapping function that maps a domain's particular feature distribution to H.

The total MMD loss was:

3-9. calculate the overall objective optimization function, defined as follows:

L_total＝L_cap+βL_mmd+γL_w#(13)

where β and γ are the hyper-parameters of the model.

3-10, repeating the step 3-1 to the step 3-9 until the iteration is carried out for T times.

And (3) outputting: the target domain sample labels correspond to class labels of the integrated results of all classifiers.

The main contributions of the invention are:

firstly, EEG data is constructed into a multi-channel one-dimensional structure, so that training efficiency is greatly improved, and better model performance is realized. Secondly, the introduction of the self-expression module to capture potential connections between samples can be well adapted to cross-subject EEG data analysis with significant individual differences under different tasks. Finally, a space attention algorithm based on dynamic sub-capsules is provided to further learn fine-grained characteristic information on EEG data space hierarchy, and partial space relation and partial-whole hierarchy relation of EEG data are effectively described.

The method effectively avoids the problem of individual difference of electroencephalogram signals in the field of brain cognitive computation, is suitable for cognitive state recognition based on EEG under any task, has strong generalization capability, and can be well suitable for clinical diagnosis and practical application.

Drawings

FIG. 1 is a block diagram of the present invention.

Detailed Description

The invention is further illustrated with reference to the following figures and examples.

Currently, most capsule-based correlation research results are applied to aspects such as image recognition and target detection, and a capsule network provides a new way for explaining the correlation between EEG and corresponding physical activities. Most of the existing methods use a capsule network to extract multi-level features from multiband EEG data for cognitive state detection, neglect the relation between local capsules, and do not effectively analyze EEG data with significant differences among tested objects based on a capsule framework.

The algorithm provided by the invention mainly comprises the following three aspects: 1) in consideration of interaction among different EEG channels, multi-channel one-dimensional electroencephalogram features are extracted to replace two-dimensional electroencephalogram features which are usually adopted as input, potential feature information among the channels is effectively reserved, and the training process is remarkably accelerated; 2) a self-expression module is introduced, the category coefficient is calculated by considering the potential relation among different samples, the capsule routing module is weighted, the sensitivity of the method to abnormal values is effectively reduced, in addition, the joint probability distribution of batch samples is optimized by adopting a proper regularization algorithm, and overfitting is effectively avoided; 3) an improved dynamic sub-capsule-based spatial attention algorithm (sub-capsules are defined as vector representation of bottom layer features) is provided, strong correlation between cognitive states and brain function connection modes is considered, a spatial attention mechanism is introduced to explore the spatial relationship of multi-channel one-dimensional electroencephalogram data, and further the potential spatial relationship (among parts) and the hierarchical relationship (part-whole) of electroencephalogram features are effectively described.

In essence, the algorithm can effectively avoid the influence of EEG high nonlinearity and obvious individual difference characteristics, can achieve better accuracy and higher efficiency in the cognitive state detection based on the EEG, can better explain the intrinsic mechanism of the correlation between the EEG characteristics and the cognitive state, and has wide application scenes in practice.

In summary, the invention detects the tested cognitive state by taking the individual difference characteristics of the EEG data and the accuracy, interpretability and efficiency of the cognitive detection method as key starting points for the cognitive state detection problem based on the EEG, and the core technology of the invention is mainly to construct an EEG cognitive state detection method for Multi-source domain crossing test based on the Efficient Multi-source Capsule network algorithm (E3 GCAPS). In the invention, each training tested individual is used as a single domain to form a plurality of source domains, a new cross-tested individual is used as a target domain, the cross-tested relevance and similarity are comprehensively considered, and a capsule network and a space attention mechanism are introduced to detect the cross-tested cognitive state. The method fully considers the feature space distribution structure of the data, not only can effectively explain the intrinsic mechanism of the correlation between the EEG features and the cognitive state, but also can greatly improve the model training efficiency, has higher universality, has wide application prospect in the actual brain-computer interaction (BCI), and provides technical support for clinical application.

As shown in fig. 1, the structure diagram of the cross-tested EEG cognitive state detection method (E3GCAPS) based on the efficient multisource capsule network mainly includes the following steps:

step 1: data acquisition

The fatigue driving electroencephalogram data set adopted by the invention is 15 electroencephalogram EEG data of healthy tested subjects with better driving experience, and each tested subject fills in an NASA-TLX questionnaire after the test so as to provide subjective workload perception. According to the NASA-TLX questionnaire, the present invention selected two mental states, TAV3 and DROWS, as analyses.

Step 2: data pre-processing

Taking fatigue driving EEG data as an example, the raw EEG data processing steps are as follows:

2-1, artifact removal: performing artifact removal operation on the acquired original EEG data, firstly performing 1-30Hz band-pass filtering processing, and simultaneously removing power frequency interference and direct current components in the signals; and then removing artifacts in the signal by ICA independent component analysis.

2-2, PSD feature extraction: extracting PSD (phase-sensitive Detector) characteristics of the EEG with the artifacts removed, performing data segmentation on the EEG of each tested individual through a sliding window of 0.5s to obtain 1400 samples, wherein the number of EEG data acquisition channels is 61, extracting a frequency band 4-30Hz (wherein theta is 4-7Hz, alpha is 8-13Hz, and beta is 14-30Hz) related to fatigue driving, splicing the extracted frequency band and the channel dimensions to obtain 61-27 dimensional characteristic vectors, and each tested individual corresponds to 1400-1647 dimensional sample data.

And step 3: cross-tested EEG cognitive state detection method based on E3GCAPS algorithm

Inputting: firstly, a multi-source domain sample data set U with a cognitive state label_sThe method specifically comprises the following steps:

where N is the total number of source domains (i.e., the total number of subjects), N is the nth source domain,a sample representing the nth source domain,the true label corresponding to the nth source domain sample is represented, more specifically, as follows:

whereinThe ith sample representing the nth source domain,the real label, | X, corresponding to the ith sample of the nth source domain is represented_snL is the total number of samples of the nth source domain;

second, target domain sample data X without cognitive state label_TThe method specifically comprises the following steps:

whereinThe ith sample, | X, representing the target domain_TL is the total number of target domain samples;

and c, the maximum iteration number T, and the label category K is {1,2, …, K }, wherein K is the total number of label categories.

And 3-1, considering the relation among the EEG channels, converting the two-dimensional EEG data into a multi-channel one-dimensional structure, and greatly improving the training efficiency while keeping the model performance. Representing each sample of the source and target domains asWherein Channels represents the number of Channels of the sample and Features represents the number of Features per channel of the sample.

3-2. extracting common domain invariant features of the multi-channel one-dimensional data from the plurality of source and target domains using Batch Normalization (BN) and Exponential Linear Unit (ELU) activated point-by-point convolution as a common feature extractor G (-). Ith sample in nth source domainThe obtained public domain invariant features areSimplify it intoIth sample in target domainThe obtained public domain invariant features areSimplify it into

3-3. taking into account the sensitivity of outliers of EEG dataA plurality of self-expression modules are introduced in one step to acquire potential relations among the samples. Each self-expression module comprises a full connection layer and a softmax function; using a fully-connected layer (FC) as a classifier, mapping the learned feature distribution to a sample label space, and using a softmax function to obtain a class coefficient C from the output of the fully-connected layer, wherein: ith sample in nth source domainThe obtained self-expression module output is the category coefficientIth sample in target domainThe obtained self-expression module output is the category coefficientThe softmax function is:

wherein, C_iRepresenting samples z from a source or target domain_iThe class coefficient of (a) is determined,denotes the conversion of the FC layer, f_ikDenotes f_iOf (a) belongs to the kth class.

3-4. adding a regularization term L to effectively eliminate the dependence on a small amount of data_wcAnd the self-expression class coefficient C is restrained, so that overfitting is prevented, and the generalization capability is improved. L is_wcIs calculated as follows:

where T represents the matrix transpose, M represents the number of training samples in a batch, and α represents the hyperparameter.

The total regularization loss is:

wherein the content of the first and second substances,for the regularization term of the nth source domain,and the regularization term corresponds to the target domain.

3-5, the domain invariant features output by the common feature extractor are used as input, a plurality of EEGCAPS classifiers are introduced, and various fine-grained features of EEG are effectively extracted, wherein the fine-grained features comprise:

first, domain-specific features are extracted from common domain-invariant features using a set of convolutional layers, and the domain-specific features are mapped to a higher-latitude space using BN and ELU.

Subsequently, a capsule layer with vector output is obtained by means of initial capsule layers (PrimaryCaps)Wherein e^lFor sub-capsules, the latitude of each sub-capsule in the first layer is d^lThe feature representation at this point is no longer a single neuron, but a vector capsule. Note that the output of the nth source field is p sub-capsules,representing the jth sub-capsule therein, the output of the target domain is p sub-capsules,indicating the jth sub-capsule therein.

Finally, to normalize the features, the features are compressed to a range of 0-1 by a squash function, which can be expressed as:

wherein, F_jFor a certain sub-capsule in the current source domain or target domain, | | F_jI means F_jL of₁Norm, | | F_j||²Is represented by F_jL of₂And (4) norm. The output of the jth capsule in the nth source domain isIs marked asThe jth capsule in the target domain is output asIs marked as

And 3-6, in consideration of strong spatial correlation of EEG, introducing a spatial attention mechanism to constrain dynamic routing coefficients, and routing the sub-capsules to the whole to which the sub-capsules belong through spatial attention routing, and finally outputting the cognitive capsules. The number of cognitive capsules per source domain or target domain is K. The method specifically comprises the following steps:

seventhly, setting routing times r, initializing the number of layers l to 0, initializing a prior weight matrix W and a prior probability matrix a, initializing a routing protocol coefficient b, expanding and outputting a class coefficient C dimension, and projecting the sub-capsules into cognitive capsules:

wherein the content of the first and second substances,represents a sub-capsule T 'in the source domain or the target domain'_jThe l layer of (b) represents C'_jFor pairs after dimension expansionThe class coefficient of the jth sub-capsule,is T'_jAnd a cognitive capsule T_kA priori weight matrix at layer l.

-obtaining an attention coupling coefficient r based on a sub-capsule_jk：

Wherein, a_jkIs T'_jAndthe prior probability matrix of (a) is,is the routing protocol coefficient, t_jkIs through learning of T'_jAndthe spatial attention weight calculated by the spatial relationship between the sub-capsules is obtained by directly learning the characteristics of the sub-capsules through one-dimensional convolution, and the convolution can carry out weight sharing so as to realize a cross-channel (spatial level) interaction strategy without dimension reduction and effectively capture the spatial relationship between the sub-capsules.

Ninthly acquiring weighted sum features s_k：

Capsule for obtaining and outputting in the ear

Obtaining routing protocol coefficients

Wherein cos (·) represents a cosine similarity function, and the similarity is evaluated by calculating the cosine of the included angle between the vectors, so that the spatial difference of the sample characteristics can be effectively measured. The calculation formula is as follows:

where A and B represent two vectors, | A | | | represents L of A₁Norm, | B | | | denotes L of B₁And (4) norm.

Repeating the step (c) until reaching the routing frequency r, and recording the k-th cognitive capsule as T_k。

3-7, the norm length of the vector is used for representing the existence probability of the cognitive capsule. Using marginal loss L_capApplied to each cognitive capsule to optimize the overall capsule network. Marginal loss L_capIs calculated as follows:

wherein the content of the first and second substances,for the k cognitive capsule of the nth source domainL of₁Norm, if the sample belongs to class k, then Y_k1, otherwise Y_k＝0，m⁺、m^-And λ is a hyperparameter and max indicates the maximum value.

3-8, note that the current nth source domain isThe k-th cognitive capsule has a norm length ofThe target domain isThe norm length of the k' cognitive capsule isMeasuring the distance of the predicted distribution of the source domain and the target domain by using Maximum Mean Difference (MMD) according to the norm length of each cognitive capsule, aligning the two distributions to obtain lossThe calculation is as follows:

wherein H is a regenerated nuclear Hilbert space (RKHS),representing a feature mapping function that maps a domain's particular feature distribution to H.

The total MMD loss was:

3-9. calculate the overall objective optimization function, defined as follows:

L_total＝L_cap+βL_mmd+γL_w#(17)

where β and γ are the hyper-parameters of the model.

3-10, repeating the step 3-1 to the step 3-9 until the iteration is carried out for T times.

And (3) outputting: the target domain sample labels correspond to class labels of the integrated results of all classifiers.

The invention is suitable for any cognitive state detection based on EEG, solves the problem of EEG individual difference to a certain extent, explains the internal mechanism of the correlation between EEG characteristics and cognitive states, and has the advantages of high efficiency, strong generalization capability and the like.