Training method and device for electroencephalogram detection model
阅读说明：本技术 一种脑电检测模型的训练方法及装置 (Training method and device for electroencephalogram detection model ) 是由 史佳锋 蒿杰 孙亚强 梁俊 于 2021-09-18 设计创作，主要内容包括：本申请实施例提供一种脑电检测模型的训练方法及装置,涉及脑机接口技术领域,该脑电检测模型的训练方法包括:该方法可以首先采集诱发状态下的脑电信号,以预设频率和预设时长为依据对该脑电信号进行多次采集,得到脑电信号集合。可见,实施这种实施方式,能够获取大量符合特定条件的脑电信号,以便后续脑电检测模型可以对该类脑电信号进行更为精准的模型建立。(The embodiment of the application provides a method and a device for training an electroencephalogram detection model, and relates to the technical field of brain-computer interfaces. Therefore, by implementing the implementation mode, a large number of electroencephalogram signals meeting specific conditions can be obtained, so that a subsequent electroencephalogram detection model can be used for more accurately establishing the electroencephalogram signals.)
1. A method for training an electroencephalogram detection model is characterized by comprising the following steps:
acquiring an electroencephalogram signal set according to a preset frequency and preset time;
preprocessing the electroencephalogram signal set to obtain a preprocessed signal set;
constructing an electroencephalogram data matrix according to the preprocessed signal set;
and training according to the electroencephalogram data matrix and a preset network model to obtain an electroencephalogram detection model.
2. The method for training the electroencephalogram detection model according to claim 1, wherein the step of acquiring the electroencephalogram signal set according to the preset frequency and the preset time comprises the following steps:
acquiring a single electroencephalogram signal in an induced state according to a sampling frequency of 1KHz and a sampling time of 60 seconds;
and collecting the single electroencephalogram signal for multiple times according to preset times to obtain an electroencephalogram signal set.
3. The method for training the electroencephalogram detection model according to claim 1, wherein the step of preprocessing the electroencephalogram signal set to obtain a preprocessed signal set comprises the following steps:
denoising and baseline drift removing are carried out on the electroencephalogram signal set to obtain a primary processing signal set;
separating and extracting the preliminary processing signal set according to a plurality of preset frequency bands to obtain a multi-frequency band signal set;
and carrying out normalization processing on the multi-band signal set to obtain a preprocessed signal set.
4. The method for training the electroencephalogram detection model according to claim 1, wherein the step of constructing the electroencephalogram data matrix according to the preprocessed signal set comprises:
performing matrix construction on the preprocessed signal set according to the number of preset wave band channels and the length of preset signal data to obtain a preprocessed signal matrix;
and performing artifact removal processing and baseline correction processing on the preprocessed signal matrix to obtain an electroencephalogram data matrix.
5. The method for training the electroencephalogram detection model according to claim 1, wherein the preset network model is a convolutional neural network with a single-stage network structure, and the preset network model at least comprises a depth separable convolution module, a point-by-point convolution module and a packet convolution module.
6. The method for training the electroencephalogram detection model according to claim 1, wherein after the electroencephalogram detection model is obtained by training according to the electroencephalogram data matrix and a preset network model, the method further comprises:
acquiring a signal to be detected, and averagely grouping the signal to be detected to obtain a first grouped signal and a second grouped signal;
in the electroencephalogram detection model, performing depth separable convolution on the first packet signal according to a convolution kernel of 1 x 3 to obtain a first convolution result; in the electroencephalogram detection model, performing packet convolution on the first convolution result according to the convolution core of 1 x 1 to obtain a first characteristic;
in the electroencephalogram detection model, performing grouping convolution on the second grouped signals according to the convolution core of 1 x 1 to obtain a second characteristic; in the electroencephalogram detection model, performing depth separable convolution on the second features according to the convolution kernel of 1 x 3 to obtain a second convolution result; in the electroencephalogram detection model, performing grouping convolution on the second convolution result according to the convolution core of 1 x 1 to obtain a third feature;
and performing concat connection and shuffle operation on the first feature and the third feature to obtain a data feature.
7. The method for training the electroencephalogram detection model according to claim 6, wherein after the concat connection and shuffle operation are performed on the first feature and the third feature to obtain the data features, the method further comprises:
performing clustering operation on the data characteristics according to a kmeans algorithm to obtain a clustering result;
carrying out proportion calculation of 7 anchor frames according to the clustering result to obtain an anchor frame setting proportion;
and carrying out characteristic detection on the data characteristics according to the anchor frame setting proportion and the characteristic points in the data characteristics to obtain a quiet state detection result or an active state detection result.
8. The method for training the electroencephalogram detection model according to claim 1, wherein the loss function formula of the preset network model is as follows:
E＝EN+EP；
wherein E isNRepresenting a loss function for calculating the signal class, EPA loss function representing a period for calculating a signal occurrence;
wherein N is the total number of samples, c is the total number of classes, N represents the current sample, k represents the class of the current sample,representing the target class of the network model output,representing the target category of the real label; b denotes the total number of signal regions in the nth sample, j denotes the position of the signal region in the nth sample,indicating the presence of a center containing a true signal, C, in the target framenIndicating the location of the region actually predicted by the nth sample,indicates the marked region position of the nth sample, xnIndicating the actual predicted coordinate position of the nth sample,indicating the coordinate location of the nth sample label.
9. The method for training the electroencephalogram detection model according to claim 1, wherein the electroencephalogram detection model judges the effect of the electroencephalogram detection model by utilizing the accuracy, the precision and the recall ratio, and the calculation formula is as follows:
wherein TP is actually a positive sample and the prediction is also a positive sample;
TN is actually a negative sample, and the prediction is also a negative sample;
FP is actually a negative sample and the prediction is a positive sample;
FN is actually a positive sample and the prediction is a negative sample.
10. The device for training the electroencephalogram detection model is characterized by comprising the following components:
the acquisition unit is used for acquiring an electroencephalogram signal set according to a preset frequency and preset time;
the preprocessing unit is used for preprocessing the electroencephalogram signal set to obtain a preprocessed signal set;
the construction unit is used for constructing an electroencephalogram data matrix according to the preprocessed signal set;
and the training unit is used for training according to the electroencephalogram data matrix and a preset network model to obtain an electroencephalogram detection model.
Technical Field
The application relates to the field of brain-computer interfaces, in particular to a training method and device of an electroencephalogram detection model.
Background
In the field of brain-computer interfaces, signal recognition is currently a very important research direction. The reason is that the electroencephalogram signals are easy to obtain, and meanwhile, compared with information such as facial expressions, the electroencephalogram signals are easy to reflect the internal heart world of people, so that the electroencephalogram signals are a common index for people to perform psychological analysis. At present, electroencephalogram signal analysis is also widely applied in the field of brain-computer interfaces.
However, in practice, even if the same set of equipment is used, the acquired electroencephalogram signals still have large differences due to different testers, so that the analysis difficulty of the electroencephalogram signals is increased to a certain extent, and meanwhile, the general applicability of electroencephalogram signal detection is also reduced.
Disclosure of Invention
The embodiment of the application aims to provide a training method and a training device for an electroencephalogram detection model, which can be better suitable for detection processes of electroencephalograms of different people and different activity degrees, so that the analysis difficulty of the electroencephalograms is reduced, and meanwhile, the general applicability of electroencephalogram detection is improved.
The first aspect of the embodiments of the present application provides a method for training an electroencephalogram detection model, including:
acquiring an electroencephalogram signal set according to a preset frequency and preset time;
preprocessing the electroencephalogram signal set to obtain a preprocessed signal set;
constructing an electroencephalogram data matrix according to the preprocessed signal set;
and training according to the electroencephalogram data matrix and a preset network model to obtain an electroencephalogram detection model.
In the implementation process, the method can firstly collect the electroencephalogram signals in the induction state, and the electroencephalogram signals are collected for multiple times by taking the preset frequency and the preset time as the basis to obtain an electroencephalogram signal set. Therefore, by implementing the implementation mode, a large number of electroencephalogram signals meeting specific conditions can be obtained, so that a subsequent electroencephalogram detection model can be used for more accurately establishing the electroencephalogram signals.
Further, the step of acquiring the electroencephalogram signal set according to the preset frequency and the preset time includes:
acquiring a single electroencephalogram signal in an induced state according to a sampling frequency of 1KHz and a sampling time of 60 seconds;
and collecting the single electroencephalogram signal for multiple times according to preset times to obtain an electroencephalogram signal set.
In the implementation process, the method can determine the sampling frequency as 1KHz and the sampling time as 60s, so that the method can acquire the 60-second electroencephalogram signals of 1KHz in the induced state, and the number of the electroencephalogram signals can be acquired, and 5000 electroencephalogram signals can be acquired. Therefore, by implementing the implementation mode, a large number of electroencephalogram signals which accord with fixed conditions can be acquired, and the subsequent electroencephalogram detection model training is facilitated.
Further, the step of preprocessing the electroencephalogram signal set to obtain a preprocessed signal set includes:
denoising and baseline drift removing are carried out on the electroencephalogram signal set to obtain a primary processing signal set;
separating and extracting the preliminary processing signal set according to a plurality of preset frequency bands to obtain a multi-frequency band signal set;
and carrying out normalization processing on the multi-band signal set to obtain a preprocessed signal set.
In the implementation process, the method can be used for preprocessing the acquired electroencephalogram signals so as to obtain appropriate preprocessed signals, and therefore data can be constrained so as to be convenient for better analysis of the signals.
Further, the step of constructing the electroencephalogram data matrix according to the preprocessed signal set includes:
performing matrix construction on the preprocessed signal set according to the number of preset wave band channels and the length of preset signal data to obtain a preprocessed signal matrix;
and performing artifact removal processing and baseline correction processing on the preprocessed signal matrix to obtain an electroencephalogram data matrix.
In the implementation process, the method can divide the acquired signals into 4 wave bands, and label the acquired preprocessing signal set according to the corresponding time periods of different types of electroencephalogram signals, so that an electroencephalogram data matrix can be obtained.
Further, the preset network model is a convolutional neural network with a single-stage network structure, and the preset network model at least comprises a depth separable convolution module, a point-by-point convolution module and a grouping convolution module.
In the implementation process, the preset network model of the method may be a convolutional neural network with a single-stage network structure, and the preset network model at least includes a depth separable convolution module, a point-by-point convolution module, and a packet convolution module. Therefore, by implementing the implementation mode, the corresponding electroencephalogram detection model can be trained according to the specific network model, and the network structure of the electroencephalogram model is the best choice, so that the most suitable electroencephalogram detection model can be trained by the high-quality network structure at present.
Further, after the electroencephalogram detection model is obtained by training according to the electroencephalogram data matrix and a preset network model, the method further comprises the following steps:
acquiring a signal to be detected, and averagely grouping the signal to be detected to obtain a first grouped signal and a second grouped signal;
in the electroencephalogram detection model, performing depth separable convolution on the first packet signal according to a convolution kernel of 1 x 3 to obtain a first convolution result; in the electroencephalogram detection model, performing packet convolution on the first convolution result according to the convolution core of 1 x 1 to obtain a first characteristic;
in the electroencephalogram detection model, performing grouping convolution on the second grouped signals according to the convolution core of 1 x 1 to obtain a second characteristic; in the electroencephalogram detection model, performing depth separable convolution on the second features according to the convolution kernel of 1 x 3 to obtain a second convolution result; in the electroencephalogram detection model, performing grouping convolution on the second convolution result according to the convolution core of 1 x 1 to obtain a third feature;
and performing concat connection and shuffle operation on the first feature and the third feature to obtain a data feature.
In the implementation process, the method can disturb the channel sequence, so that the parameter quantity and the calculation quantity are reduced on one hand, and the data features can be better extracted on the other hand.
Further, after performing concat connection and shuffle operation on the first feature and the third feature to obtain a data feature, the method further includes:
performing clustering operation on the data characteristics according to a kmeans algorithm to obtain a clustering result;
carrying out proportion calculation of 7 anchor frames according to the clustering result to obtain an anchor frame setting proportion;
and carrying out characteristic detection on the data characteristics according to the anchor frame setting proportion and the characteristic points in the data characteristics to obtain a quiet state detection result or an active state detection result.
In the implementation process, the method can further process the features to obtain data features meeting the requirements, so that the effect of extracting the features in different states in a whole section of signal is realized; meanwhile, the method can also ensure that the model can cover the marking data with different sizes on one hand to ensure the coverage area, and ensure the detection precision while ensuring that the model parameters are not too large on the other hand.
Further, the loss function formula of the preset network model is as follows:
E＝EN+EP；
wherein E isNRepresenting a loss function for calculating the signal class, EPA loss function representing a period for calculating a signal occurrence;
wherein N is the total number of samples, c is the total number of classes, N represents the current sample, k represents the class of the current sample,representing the target class of the network model output,representing the target category of the real label; b denotes the total number of signal regions in the nth sample, j denotes the position of the signal region in the nth sample,indicating the presence of a center containing a true signal, C, in the target framenIndicating the location of the region actually predicted by the nth sample,indicates the marked region position of the nth sample, xnIndicating the actual predicted coordinate position of the nth sample,indicating the coordinate location of the nth sample label.
In the implementation process, the method can complete the design of the loss function by combining the loss functions, and further perform regression training by using a corresponding algorithm.
Further, the electroencephalogram detection model judges the effect of the electroencephalogram detection model by utilizing the accuracy, the precision and the recall rate, and the calculation formula is as follows:
wherein TP is actually a positive sample and the prediction is also a positive sample;
TN is actually a negative sample, and the prediction is also a negative sample;
FP is actually a negative sample and the prediction is a positive sample;
FN is actually a positive sample and the prediction is a negative sample.
In the implementation process, the method can use data in the test set for evaluation, so that the model can well perform on the test set, the generalization error is reduced, and the three evaluation indexes can be required to be as large as possible. Therefore, the implementation of the embodiment is beneficial to more accurately evaluating the quality of the model by using the indexes in the model evaluation stage and adjusting the model according to the quality.
The second aspect of the embodiments of the present application provides a training device for an electroencephalogram detection model, where the training device for the electroencephalogram detection model includes:
the acquisition unit is used for acquiring an electroencephalogram signal set according to a preset frequency and preset time;
the preprocessing unit is used for preprocessing the electroencephalogram signal set to obtain a preprocessed signal set;
the construction unit is used for constructing an electroencephalogram data matrix according to the preprocessed signal set;
and the training unit is used for training according to the electroencephalogram data matrix and a preset network model to obtain an electroencephalogram detection model.
In the implementation process, the device can acquire a large number of electroencephalogram signals meeting specific conditions, so that a subsequent electroencephalogram detection model can be used for more accurately establishing the electroencephalogram signals.
A third aspect of the embodiments of the present application provides an electronic device, including a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the method for training an electroencephalogram detection model according to any one of the first aspect of the embodiments of the present application.
A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores computer program instructions, where the computer program instructions, when read and executed by a processor, perform the method for training an electroencephalogram detection model according to any one of the first aspect of the embodiments of the present application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic flow chart of a training method for an electroencephalogram detection model according to an embodiment of the present application;
fig. 2 is a schematic flow chart of a training method of an electroencephalogram detection model provided in the second embodiment of the present application;
fig. 3 is a schematic structural diagram of a training device for an electroencephalogram detection model provided in the third embodiment of the present application;
fig. 4 is a schematic structural diagram of a training device for an electroencephalogram detection model provided in the fourth embodiment of the present application;
fig. 5 is a schematic structural diagram of a feature extraction module according to a second embodiment of the present disclosure;
fig. 6 is a schematic diagram of a basic structure of an electroencephalogram detection model provided in the second embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Example 1
Referring to fig. 1, fig. 1 is a schematic flow chart of a training method of an electroencephalogram detection model according to an embodiment of the present application. The electroencephalogram detection model training method comprises the following steps:
s101, acquiring an electroencephalogram signal set according to a preset frequency and preset time.
In the embodiment of the application, firstly, electroencephalogram signals in an evoked state are collected, the sampling frequency is a preset frequency, specifically, 1000Hz, and data of a preset time (for example, T seconds) is collected, specifically, T can take 60 seconds, that is, 60 seconds of data, and the electroencephalogram signal set is obtained by repeating the preset times.
In the embodiment of the present application, the preset times may be set to 5000 times, and the like, and the embodiment of the present application is not limited.
S102, preprocessing the electroencephalogram signal set to obtain a preprocessed signal set.
In the embodiment of the application, when the electroencephalogram signals are preprocessed, denoising preprocessing is firstly carried out, and then standardization processing is carried out. The denoising preprocessing includes band-pass filtering (denoising), baseline drift removal, and the like, and the embodiment of the present application is not limited.
In the embodiment of the application, a 5-order butterworth filter is used for separating and extracting signals of different frequency bands (theta, alpha, beta and gamma) of electroencephalograms subjected to denoising preprocessing, wherein the frequency range of theta is 4-8 Hz, the frequency range of alpha is 8-13 Hz, the frequency range of beta is 13-30 Hz, the frequency range of gamma is 30-50 Hz, in order to eliminate the influence of dimensions between data indexes, data needs to be standardized, and a 0-mean value can be specifically usedNormalized by the formulaMu is the mean value of the original data (namely, the electroencephalogram signal set), sigma is the data standard deviation of the electroencephalogram signal set, and the data in the electroencephalogram signal set can be constrained through normalization operation so as to be convenient for better analyzing the signals.
S103, constructing an electroencephalogram data matrix according to the preprocessed signal set.
In the embodiment of the application, the preprocessed signal set includes signal data of 4 wavebands (θ, α, β, γ), which may be regarded as data of 4 channels, and then the acquired data of T seconds is labeled, specifically, a time period corresponding to different types of electroencephalograms is labeled, after this time period is completed, a matrix of C × 1 × M may be obtained, where C denotes the number of channels, and the corresponding electroencephalograms refer to electroencephalograms of different frequency bands, that is, C is 4, and 1 denotes a data dimension, since the acquired electroencephalograms are electroencephalograms, the dimension is 1 dimension, and M denotes a length of data, here is 60000(1 × 60s), and therefore, after the data processing is completed, a data matrix of 4 × 1 × 60000 dimensions may be constructed for subsequent analysis processing.
In the embodiment of the application, the time periods corresponding to different types of electroencephalogram signals are labeled, namely, signals in different time periods are labeled, and corresponding labeling work is mainly performed on the types of the signals in different time periods, namely, what state a tester is in a short time is labeled.
In the embodiment of the application, the independent principal component analysis is carried out after the step to remove the artifact, and the data set is constructed after the baseline correction is carried out.
And S104, training according to the electroencephalogram data matrix and a preset network model to obtain an electroencephalogram detection model.
In the embodiment of the application, after the electroencephalogram detection model is obtained, a preset network model can be used for detecting and processing signals.
In this embodiment of the present application, the preset network model may specifically be a convolutional neural network with a single-stage network structure, and specifically includes a common convolutional layer, a depth separable convolutional module (depthwise module), a point-by-point convolutional module (pointwise module), and a group convolution module (group convolution module), and the embodiment of the present application is not limited thereto.
In the embodiment of the application, the electroencephalogram signals under different states are detected by using strategies such as grid selection frames and anchor frames in the detection stage, and the purpose of detecting the electroencephalogram signals under different states and different types in the whole signal can be achieved.
In the embodiment of the present application, the execution subject of the method may be a computing device such as a computer and a server, and is not limited in this embodiment.
In this embodiment, an execution subject of the method may also be an intelligent device such as a smart phone and a tablet computer, which is not limited in this embodiment.
Therefore, the electroencephalogram detection model training method described in the embodiment can be better adapted to the detection processes of electroencephalogram signals of different people and different activity degrees, so that the analysis difficulty of the electroencephalogram signals is reduced, and meanwhile, the general applicability of electroencephalogram signal detection is improved.
Example 2
Please refer to fig. 2, fig. 2 is a schematic flow chart of a training method of an electroencephalogram detection model according to an embodiment of the present application. As shown in fig. 2, the method for training the electroencephalogram detection model includes:
s201, collecting a single electroencephalogram signal in an inducing state according to a sampling frequency of 1KHz and a sampling time of 60 seconds.
In the embodiment of the application, firstly, electroencephalogram signals in an evoked state are collected, and 1000Hz and 60 seconds of data are collected to obtain a single electroencephalogram signal.
S202, collecting the single electroencephalogram signal for multiple times according to preset times to obtain an electroencephalogram signal set.
In the embodiment of the present application, the preset times may be set to 5000 times, and the like, and the embodiment of the present application is not limited.
In the embodiment of the present application, by implementing the steps S201 to S202, an electroencephalogram signal set can be obtained according to a preset frequency and a preset time.
S203, preprocessing the electroencephalogram signal set to obtain a preprocessed signal set.
As an optional implementation, the step of preprocessing the electroencephalogram signal set to obtain a preprocessed signal set includes:
denoising and baseline drift removing are carried out on the electroencephalogram signal set to obtain a primary processing signal set;
separating and extracting the preliminary processing signal set according to a plurality of preset frequency bands to obtain a multi-frequency band signal set;
and carrying out normalization processing on the multi-band signal set to obtain a preprocessed signal set.
In the above embodiment, when preprocessing the electroencephalogram signal, denoising preprocessing is performed first, and then normalization processing is performed. The denoising preprocessing includes band-pass filtering (denoising), baseline drift removal, and the like, and the embodiment of the present application is not limited.
In the above embodiment, a 5-order butterworth filter is used to separate and extract signals of different frequency bands (θ, α, β, γ) from the electroencephalogram signal after the denoising pre-processing, where the frequency range of θ is 4 to 8Hz, the frequency range of α is 8 to 13Hz, the frequency range of β is 13 to 30Hz, and the frequency range of γ is 30 to 50Hz, in order to eliminate the influence of the dimension between data indexes, the data needs to be normalized, specifically, 0-mean normalization can be used, and the formula isMu is the mean value of the original data (namely, the electroencephalogram signal set), sigma is the data standard deviation of the electroencephalogram signal set, and the data in the electroencephalogram signal set can be constrained through normalization operation so as to be convenient for better analyzing the signals.
After step S203, the following steps are also included:
s204, matrix construction is carried out on the preprocessed signal set according to the number of the preset wave band channels and the preset signal data length, and a preprocessed signal matrix is obtained.
In the embodiment of the application, the preprocessed signal set includes signal data of 4 wavebands (θ, α, β, γ), which may be regarded as data of 4 channels, and then the acquired data of T seconds is labeled, specifically, a time period corresponding to different types of electroencephalograms is labeled, after this time period is completed, a matrix of C × 1 × M may be obtained, where C denotes the number of channels, and the corresponding electroencephalograms refer to electroencephalograms of different frequency bands, that is, C is 4, and 1 denotes a data dimension, since the acquired electroencephalograms are electroencephalograms, the dimension is 1 dimension, and M denotes a length of data, here is 60000(1 × 60s), and therefore, after the data processing is completed, a data matrix of 4 × 1 × 60000 dimensions may be constructed for subsequent analysis processing.
In the embodiment of the application, the time periods corresponding to different types of electroencephalogram signals are labeled, namely, signals in different time periods are labeled, and corresponding labeling work is mainly performed on the types of the signals in different time periods, namely, what state a tester is in a short time is labeled.
S205, artifact removal processing and baseline correction processing are carried out on the preprocessed signal matrix, and an electroencephalogram data matrix is obtained.
In the embodiment of the application, the artifacts are removed by carrying out independent principal component analysis on the preprocessed signal matrix, and simultaneously, the data set is constructed after baseline correction is carried out, so that the electroencephalogram data matrix is obtained.
In the embodiment of the present application, by implementing the steps S204 to S205, an electroencephalogram data matrix can be constructed according to the preprocessed signal set.
And S206, training according to the electroencephalogram data matrix and a preset network model to obtain an electroencephalogram detection model.
In this embodiment of the present application, the preset network model may specifically be a convolutional neural network with a single-stage network structure, and specifically includes a common convolutional layer, a depth separable convolutional module (depthwise module), a point-by-point convolutional module (pointwise module), and a group convolution module (group convolution module), and the embodiment of the present application is not limited thereto.
In the embodiment of the application, the electroencephalogram signals under different states are detected by using strategies such as grid selection frames and anchor frames in the detection stage, and the purpose of detecting the electroencephalogram signals under different states and different types in the whole signal can be achieved.
In the embodiment of the application, an output layer of a network model is preset, and data of each channel is a one-dimensional signal along with time, so that the data essentially still belong to two-dimensional convolution, and then a method of two-dimensional convolution is adopted, wherein three types of convolution, namely depthwise convolution, pointwise convolution and group convolution, are used as a feature extraction module in a feature extraction stage, and a shuffle (random ordering algorithm) is added in a convolution process to disturb operation of the channel for supplement, so that on one hand, parameters and calculated quantity can be reduced, on the other hand, the overall accuracy of the model can be maintained as good as possible, and a basic model structure of the feature extraction module is shown in fig. 5.
In the embodiment of the application, after the electroencephalogram signal to be processed is input into the electroencephalogram detection model, firstly, 64 1 × 3 convolution kernels are used for initially checking data in the electroencephalogram detection model to perform feature extraction, the step length is set to be 1, and padding is set to be 1, so that the dimensionality of the input and output data is kept consistent.
In the embodiment of the application, the electroencephalogram signal to be processed is firstly subjected to feature extraction processing through 64 convolution layers of 1 × 3 convolution kernels to obtain feature extraction data, then, the feature extraction data is inputted into the basic module as shown in fig. 5, and as can be seen from fig. 5, the inputted channel is split by the splitting module, i.e. the input channels are averagely grouped, if the number of the input channels is n, the number of the channels in each group is n/2, one group of the partial convolution branches DW convolution, the DW convolution is a combination form of depthwise convolution and pointwise convolution, the name of the convolution after combination is called depth separable convolution, and then features are further fused through 1-1 group convolution, the group convolution is characterized in that the convolution operation is not performed on all channels, instead, the input channels are divided into G groups, and then the convolution operation is performed on the input channels in the G groups by using 1 × 1 convolution respectively to obtain corresponding outputs. The other branch is to perform low-level feature extraction through 1 × 1 group convolution, then add 1 × 3 DW convolution and 1 × 1 group convolution, and finally add shuffle operation after concat connection of the outputs of the two branches, namely, to disorder the channel sequence.
In the embodiment of the present application, the loss function formula of the preset network model is as follows:
E＝EN+EP；
wherein E isNRepresenting a loss function for calculating the signal class, EPA loss function representing a period for calculating a signal occurrence;
wherein N is the total number of samples, c is the total number of classes, N represents the current sample, k represents the class of the current sample,representing the target class of the network model output,representing the target category of the real label; b denotes the total number of signal regions in the nth sample, j denotes the position of the signal region in the nth sample,indicating the presence of a center containing a true signal, C, in the target framenIndicating the location of the region actually predicted by the nth sample,indicates the marked region position of the nth sample, xnIndicating the actual predicted coordinate position of the nth sample,indicating the coordinate location of the nth sample label.
In the embodiments of the present application, N is 1, 2.
In the embodiment of the present application,specifically representing the target class of the output of the kth class nth sample after being processed by a network model,and specifically representing the target category of the nth sample real annotation of the kth category.
In the embodiment of the present application, B represents the total number of signal regions in the nth sample. In fact, for one sample N of the N samples, the sample includes a strip of electroencephalogram signal, the signal has a plurality of signal regions, each signal region corresponds to a different kind of signal, and the total number of all signal regions in the strip of signal is B. Similarly, for sample n, j represents the position of the signal region in the nth sample, where j is 1, 2, …, B, for example, sample n includes a strip of brain electrical signal, and there are 3 signal regions in total, and j is 1, 2, 3, where j represents the first signal region in the strip of brain electrical signal when j is 1, and similarly represents the second signal region in the strip of brain electrical signal when j is 2, and represents the third signal region in the strip of brain electrical signal when j is 3.
In the embodiment of the present application, the electroencephalogram data matrix may specifically be a matrix of C × 1 × M, where C denotes a number of channels, 1 denotes a dimension of an electroencephalogram signal, and M denotes a length of the electroencephalogram signal. The electroencephalogram data matrix is actually a set of labeled electroencephalogram signal samples. N is actually the total number of electroencephalogram signal samples in the electroencephalogram data matrix.
In the embodiment of the application, in a whole segment of electroencephalogram signal, there are an area without a signal and a signal area with a signal, and in actual detection, the signal area with the signal is identified and detected as the signal area with the signal, so that the signal corresponding to the signal area can be called as a real signal.
In the examples of this application, CnThe actual predicted region position of the nth sample is shown, specifically, the actual predicted region range of the nth sample, and other similar expressions are the same.
In the embodiment of the present application,for calculating the position coincidence degree error corresponding to the nth sample,for calculating the coordinate error corresponding to the nth sample.
In the embodiment of the application, the loss function E for calculating the signal classNIn particular a squared error cost function.
In the embodiment of the present application, since the time when the signal occurs and the type of the signal need to be detected, the loss function needs to perform loss function design on the type of the signal and the time period when the signal occurs, specifically, the preset network model uses two loss functions, including a loss function E for the type of the signalNAnd a loss function E for the period of time during which the signal occursP。
In the embodiment of the present application, two aspects need to be considered with respect to the time period of the signal occurrence, namely, a time error (i.e., a coordinate error), that is, an error between the identified time and the time of the real event occurrence; another aspect is the degree of position overlap (i.e., segment error), i.e., the error between the identified signal segment and the segment of the true signal, which is calculated using two loss functions, since the error includes two factors.
In the embodiment of the application, the loss functions can be designed by combining the loss functions, and the regression training can be performed by using a corresponding algorithm.
In the embodiment of the application, the electroencephalogram detection model judges the effect of the electroencephalogram detection model by utilizing the accuracy, the precision and the recall rate, and the calculation formula is as follows:
wherein TP is actually a positive sample and the prediction is also a positive sample;
TN is actually a negative sample, and the prediction is also a negative sample;
FP is actually a negative sample and the prediction is a positive sample;
FN is actually a positive sample and the prediction is a negative sample.
In the above embodiment, the above formula can be used to determine the electroencephalogram detection model effect by using the indexes such as the accuracy, precision and recall rate of detection. The positive samples refer to samples belonging to a certain category, and the negative samples refer to samples not belonging to a certain category. In the embodiment of the present application, the positive sample is a sample including the electroencephalogram signal, and the negative sample is a sample not including the electroencephalogram signal.
In the above embodiment, when evaluating the electroencephalogram detection model, data in the test set can be used, so that the model can be well represented on the test set, that is, the generalization error is small, and the three evaluation indexes (i.e., accuracy, precision and recall) mentioned above are required to be as good as possible. The indexes are used in the model evaluation stage to accurately evaluate the quality of the model, so that the model is adjusted.
After step S206, the following steps are also included:
and S207, acquiring the signals to be detected, and averagely grouping the signals to be detected to obtain a first grouped signal and a second grouped signal.
S208, performing depth separable convolution on the first packet signal according to the convolution kernel of 1 x 3 in the electroencephalogram detection model to obtain a first convolution result; and in the electroencephalogram detection model, performing packet convolution on the first convolution result according to the convolution kernel of 1 x 1 to obtain a first characteristic.
S209, performing grouping convolution on the second grouped signals according to the convolution kernel of 1 x 1 in the electroencephalogram detection model to obtain a second characteristic; in the electroencephalogram detection model, performing depth separable convolution on the second features according to the convolution kernel 1 x 3 to obtain a second convolution result; and in the electroencephalogram detection model, performing grouping convolution on the second convolution result according to the convolution kernel of 1 x 1 to obtain a third characteristic.
S210, concat connection and shuffle operation are carried out on the first feature and the third feature, and data features are obtained.
And S211, carrying out clustering operation on the data characteristics according to a kmeans algorithm to obtain a clustering result.
S212, calculating the proportion of the 7 anchor frames according to the clustering result to obtain the anchor frame setting proportion.
After step S212, the method further includes the following steps:
and S213, carrying out characteristic detection on the data characteristics according to the anchor frame setting proportion and the characteristic points in the data characteristics to obtain a quiet state detection result or an active state detection result.
In the embodiment of the present application, compared with the existing method that only electroencephalogram signals collected at different periods are classified, the above steps S207 to S213 are implemented, firstly, electroencephalogram signals of different channels are preprocessed, and then, different signal states of the time period in which the signals are collected can be judged through network output, so that the state of a tester can be judged, and thus, the state of a physiological signal can be automatically judged, and the method and the criterion are reliable.
Referring to fig. 6, fig. 6 is a schematic diagram of a basic structure of an electroencephalogram detection model according to an embodiment of the present application. As shown in fig. 6, the data is subjected to feature extraction several times by using a basic module of the preset network model, and the features of the data are already obvious at this time. Since the data is labeled before training, each point on the feature map is first considered as a feature point, called grid, meanwhile, anchor frames (anchors) with different sizes are set for detection by taking each feature point as a center, and the method is mainly used for considering that the states of signals acquired in different time periods are different, the durations of different signals are different, therefore, the extraction of the mark frame is carried out on the signal by using the anchor frames with different sizes, so that the detection range can cover more signals to be detected as far as possible, the input signal of the mark is clustered by using a kmeans algorithm, the clustering result is used as the setting proportion of the anchor frame for calculation, by designing the anchor frames with different sizes, signals can be better processed, and the signal range as much as possible is covered, so that the detection accuracy is improved. Meanwhile, in order to prevent the model from occupying too large resources, clustering 3 types are selected in the arrangement of using a kmeans algorithm and the number of anchor frames, the number of the anchor frames is set to 7, the model can cover the marked data with different sizes through the arrangement of the anchor frames, the coverage range is ensured, and the detection precision is ensured while the model parameters are not too large.
As shown in fig. 6, the input signal (signal length 1 × 60000) is subjected to feature extraction through a convolutional layer (64 convolutional kernels of 1 × 3), and the subsequent structure is the basic module (basic module), which is described above, and the data graph after being processed by the basic module 3, the basic module 4, and the basic module 5 is subjected to detection work on the corresponding signal by using the detection modules, i.e., grid and anchor as described above.
As shown in fig. 6, the classifier uses a cascade classifier (Detector & classifier) including a cascade classifier 1, a cascade classifier 2, and a cascade classifier 3.
As shown in fig. 6, NMS is Non-Maximum Suppression (Non-Maximum Suppression), i.e. suppressing elements that are not Maximum, which can be understood as a local Maximum search. The method is an algorithm for removing non-maxima, and is commonly used for edge detection, object identification and the like in computer vision. Likewise, Fast NMS is also an object detection algorithm.
In the embodiment of the present application, the signals to be recognized in signal recognition are classified into 2 types, which are a state in which a person is at rest and a state in which the person is awake, and these parts can be simply understood as a forward emotion and a backward emotion. Signal localization is the time period during which signals of different classes occur and are present. The final model can be obtained by performing regression training on the two parts, the detection of the electroencephalogram signals is completed, and the final detection can be completed by performing a non-maximum suppression algorithm on the identified detection frame and filtering out non-detection time periods.
Therefore, the electroencephalogram detection model training method described in the embodiment can be better adapted to the detection processes of electroencephalogram signals of different people and different activity degrees, so that the analysis difficulty of the electroencephalogram signals is reduced, and meanwhile, the general applicability of electroencephalogram signal detection is improved.
Example 3
Please refer to fig. 3, fig. 3 is a schematic structural diagram of a training device for an electroencephalogram detection model according to an embodiment of the present application. As shown in fig. 3, the training apparatus for electroencephalogram detection model includes:
an obtaining unit 310, configured to obtain an electroencephalogram signal set according to a preset frequency and a preset time;
the preprocessing unit 320 is used for preprocessing the electroencephalogram signal set to obtain a preprocessed signal set;
the constructing unit 330 is configured to construct an electroencephalogram data matrix according to the preprocessed signal set;
and the training unit 340 is used for training according to the electroencephalogram data matrix and a preset network model to obtain an electroencephalogram detection model.
In the embodiment of the present application, for the explanation of the training device for an electroencephalogram detection model, reference may be made to the description in embodiment 1 or embodiment 2, and details are not repeated in this embodiment.
Therefore, the electroencephalogram detection model training device described in the embodiment can be better adapted to the detection processes of electroencephalogram signals of different people and different activity degrees, so that the analysis difficulty of the electroencephalogram signals is reduced, and meanwhile, the general applicability of electroencephalogram signal detection is improved.
Example 4
Referring to fig. 4, fig. 4 is a schematic structural diagram of a training device for electroencephalogram detection models according to an embodiment of the present application. The electroencephalogram detection model training device shown in fig. 4 is obtained by optimizing the electroencephalogram detection model training device shown in fig. 3. As shown in fig. 4, the obtaining unit 310 includes:
the first acquisition subunit 311 is configured to acquire a single electroencephalogram signal in an evoked state according to a sampling frequency of 1KHz and a sampling duration of 60 seconds;
and the second collecting subunit 312 is configured to collect a single electroencephalogram signal for multiple times according to preset times to obtain an electroencephalogram signal set.
As an alternative embodiment, the preprocessing unit 320 includes:
the processing subunit 321 is configured to perform denoising processing and baseline drift removal processing on the electroencephalogram signal set to obtain a preliminary processed signal set;
a separation and extraction subunit 322, configured to separate and extract the preliminary processed signal set according to a plurality of preset frequency bands, so as to obtain a multi-band signal set;
and the normalization subunit 323 is configured to perform normalization processing on the multi-band signal set to obtain a preprocessed signal set.
As an alternative embodiment, the construction unit 330 includes:
the constructing subunit 331 is configured to perform matrix construction on the preprocessed signal set according to the number of preset band channels and the preset signal data length, so as to obtain a preprocessed signal matrix;
and the removing subunit 332 is configured to perform artifact removal processing and baseline correction processing on the preprocessed signal matrix to obtain an electroencephalogram data matrix.
As an optional implementation manner, the preset network model is a convolutional neural network with a single-stage network structure, and the preset network model at least includes a depth separable convolution module, a point-by-point convolution module, and a packet convolution module.
As an optional implementation, the training device for the electroencephalogram detection model further includes:
the signal acquiring unit 350 is configured to acquire a signal to be detected after the electroencephalogram detection model is obtained by training according to the electroencephalogram data matrix and a preset network model, and averagely group the signal to be detected to obtain a first grouped signal and a second grouped signal;
the model processing unit 360 is used for performing depth separable convolution on the first packet signal according to the convolution kernel of 1 × 3 in the electroencephalogram detection model to obtain a first convolution result; in the electroencephalogram detection model, performing packet convolution on the first convolution result according to the convolution kernel of 1 x 1 to obtain a first characteristic; in the electroencephalogram detection model, performing grouping convolution on the second grouped signals according to the convolution kernel of 1 x 1 to obtain a second characteristic; in the electroencephalogram detection model, performing depth separable convolution on the second features according to the convolution kernel 1 x 3 to obtain a second convolution result; in the electroencephalogram detection model, performing grouping convolution on the second convolution result according to the convolution kernel of 1 x 1 to obtain a third characteristic;
the processing unit 370 is configured to perform concat connection and shuffle operation on the first feature and the third feature to obtain the data feature.
As an optional implementation manner, the clustering unit 380 is configured to perform concat connection and shuffle operation on the first feature and the third feature to obtain a data feature, and then perform clustering operation on the data feature according to a kmeans algorithm to obtain a clustering result;
a detection unit 390, configured to perform proportional calculation on 7 anchor frames according to the clustering result to obtain an anchor frame setting ratio; and carrying out characteristic detection on the data characteristics according to the anchor frame setting proportion and the characteristic points in the data characteristics to obtain a quiet state detection result or an active state detection result.
In the embodiment of the present application, since the time when the signal occurs and the type of the signal need to be detected, the loss function needs to perform loss function design on the type of the signal and the time period when the signal occurs, specifically, the preset network model uses two loss functions, including a loss function for the type of the signal and a loss function for the time period when the signal occurs.
As an optional implementation manner, the loss function formula of the preset network model is as follows:
E＝EN+EP；
wherein E isNRepresenting a loss function for calculating the signal class, EPA loss function representing a period for calculating a signal occurrence;
wherein N is the total number of samples, c is the total number of classes, N represents the current sample, k represents the class of the current sample,representing the target class of the network model output,representing the target category of the real label; b denotes the total number of signal regions in the nth sample, j denotes the position of the signal region in the nth sample,indicating the presence of a center containing a true signal, C, in the target framenIndicating the location of the region actually predicted by the nth sample,indicates the marked region position of the nth sample, xnIndicating the actual predicted coordinate position of the nth sample,indicating the coordinate location of the nth sample label.
In the embodiment of the present application, two aspects need to be considered with respect to the time period of the signal occurrence, namely, a time error (i.e., a coordinate error), that is, an error between the identified time and the time of the real event occurrence; another aspect is the degree of position overlap (i.e., segment error), i.e., the error between the identified signal segment and the segment of the true signal, which is calculated using two loss functions, since the error includes two factors.
In the embodiment of the application, the loss functions can be designed by combining the loss functions, and the regression training can be performed by using a corresponding algorithm.
As an optional implementation, the electroencephalogram detection model judges the effect of the electroencephalogram detection model by using the accuracy, the precision and the recall ratio, and the calculation formula is as follows:
wherein TP is actually a positive sample and the prediction is also a positive sample; TN is actually a negative sample, and the prediction is also a negative sample; FP is actually a negative sample and the prediction is a positive sample; FN is actually a positive sample and the prediction is a negative sample.
In the embodiment of the present application, for the explanation of the training device for an electroencephalogram detection model, reference may be made to the description in embodiment 1 or embodiment 2, and details are not repeated in this embodiment.
Therefore, the electroencephalogram detection model training device described in the embodiment can be better adapted to the detection processes of electroencephalogram signals of different people and different activity degrees, so that the analysis difficulty of the electroencephalogram signals is reduced, and meanwhile, the general applicability of electroencephalogram signal detection is improved.
The embodiment of the application provides an electronic device, which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor runs the computer program to enable the electronic device to execute the electroencephalogram detection model training method in any one of embodiment 1 or embodiment 2 of the application.
The embodiment of the present application provides a computer-readable storage medium, which stores computer program instructions, and when the computer program instructions are read and executed by a processor, the method for training an electroencephalogram detection model in any one of embodiments 1 and 2 of the present application is executed.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.