阅读说明：本技术 一种基于机器学习检测胎儿颅脑异常的方法和系统 (Method and system for detecting fetal craniocerebral abnormality based on machine learning ) 是由 李胜利 李肯立 文华轩 谭光华 于 2019-09-09 设计创作，主要内容包括：本发明公开了一种基于机器学习检测胎儿颅脑异常的方法,具体是在胎儿颅脑的发育过程中,检测颅脑的主要组织结构是否存在异常和异常会导致哪些相关的颅脑疾病。本发明主要通过获取不同孕周系列颅脑标准切面的数据,对数据预处理,训练一个模型来检测颅脑主要组织结构是否存在异常。采用深度卷积网络提取特征,区域生成网络RPN生成候选区域,感兴趣池化层层收集输入的特征图和候选区域,并利用softmax分类器进行分类和回归检测,最后利用检测结果来分析主要的结构是否存在异常。如果未发现存在异常,则判断为正常。本发明旨在利用计算机辅助诊断,不需要医生或者人为的过多参与诊断的情况下,辅助诊断颅脑是否异常。(The invention discloses a method for detecting fetal craniocerebral abnormality based on machine learning, and particularly detects whether the main tissue structure of the craniocerebral is abnormal or not and which related craniocerebral diseases can be caused by the abnormality in the development process of the fetal craniocerebral. The method mainly comprises the steps of obtaining data of a series of craniocerebral standard sections of different gestational weeks, preprocessing the data, and training a model to detect whether the main tissue structure of the craniocerebral is abnormal or not. Extracting features by adopting a deep convolutional network, generating a candidate region by using a region generation network RPN, collecting an input feature map and the candidate region layer by layer in an interested pooling, carrying out classification and regression detection by using a softmax classifier, and finally analyzing whether the main structure is abnormal or not by using a detection result. If no abnormity exists, the system is judged to be normal. The invention aims to utilize computer-aided diagnosis to assist in diagnosing whether the cranium is abnormal or not under the condition that doctors or artificial excessive diagnosis is not needed.)

1. A method for detecting fetal craniocerebral abnormality based on machine learning is characterized by comprising the following steps:

(1) acquiring a fetal craniocerebral standard section data set of different gestational week series;

(2) preprocessing the fetal craniocerebral standard section data set obtained in the step (1) to obtain a preprocessed fetal craniocerebral standard section data set;

(3) inputting the fetal craniocerebral standard section data set preprocessed in the step (2) into a trained craniocerebral tissue detection network to obtain detection results of each key tissue in the fetal craniocerebral.

(4) Judging whether at least one key tissue in the fetal cranium is abnormal according to the detection result of each key tissue in the fetal cranium obtained in the step (3), if not, prompting that the fetal cranium is normal, and ending the process; if there is an abnormality, it is suggested to the fetus which related craniocerebral abnormality may exist and the process is ended.

2. The method according to claim 1, wherein step (2) comprises the sub-steps of:

(2-1) denoising each standard section image in the fetal craniocerebral standard section data set by using a self-adaptive mean filtering method, and cutting the denoised standard section images into uniform sizes to obtain the cut standard section images;

(2-2) denoising each cut standard section image obtained in the step (2-1) by using a self-adaptive median filtering method, sending each standard section image subjected to the denoising treatment to a radiology expert for marking, and obtaining each marked standard section image;

and (2-3) normalizing each marked standard section image obtained in the step (2-2) to obtain a normalized standard section image, wherein all normalized standard section images form a preprocessed fetal craniocerebral standard section data set.

3. The method according to claim 2, wherein the critical tissue in the craniocerebral portion of the standard sectional image comprises, in particular, the posterolateral sulcus, thalamus, choroid plexus, hyaline compartment, third ventricle, midline of the brain, skull, and the like.

4. The method of claim 1, wherein the brain tissue detection network comprises a deep convolutional neural network, an RPN network, an ROI pooling layer, and a classification and regression section connected in series.

5. The method of claim 4, wherein the deep convolutional neural network structure in the craniocerebral tissue detection network is:

the first layer is the input layer, the input being a 512x 3 matrix of pixels.

The second layer is the first convolution layer, which receives a 512x 3 matrix of pixels from the input layer, which uses 32 convolution kernels of size 3 x 3, which has a step size of 1, followed by the RELU activation function and BN, which outputs a matrix of size 512x 32;

the third layer is a second convolution layer, the layer uses 64 convolution kernels with the size of 3 × 3 and the step size is 1, the next convolution kernel is accessed with the RELU activation function and BN, and the output matrix of the layer is 512 × 64;

the fourth layer is a third convolution layer, 64 convolution kernels with the size of 3 × 3 are used, the step size is 1, the Droupt operation, the RELU activation function and the BN are accessed, and the matrix of the output of the layer is 512 × 64;

the fifth layer is a first pooling layer, the size of the pooling window is 2x 2, the step size is 2, and the size of the output matrix is 256 x 64;

the sixth layer is the fourth convolutional layer, which uses 128 convolutional kernels of size 3 x 3, with a step size of 1, and outputs a matrix of 256 x 128;

the seventh layer is a fifth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 2, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 128 × 256;

the eighth layer is the sixth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 1, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 128 × 256;

the ninth layer is a second pooling layer, the size of the pooling windows is 2x 2, the step size is 2, and the size of the output matrix is 64 x 256;

the tenth layer is a seventh convolution layer, which uses 256 convolution kernels with the size of 3 × 3, the step size is 1, the next time the RELU activation function and BN are accessed, and the output matrix is 64 × 256;

the eleventh layer is the eighth convolution layer, which uses 256 convolution kernels with the size of 3 × 3, the step size is 1, the RELU activation function and BN are accessed at the back, and the output matrix is 64 × 256;

the twelfth layer is a ninth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 1, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 64 × 256;

the thirteenth layer is a third pooling layer, the size of the pooling window is 2x 2, the step size is 2, and the size of the output matrix is 32 x 256;

the fourteenth layer is the tenth convolution layer, which uses 256 convolution kernels with size 3 × 3, step size 1, followed by the RELU activation function and BN, and outputs a feature matrix of 32 × 256.

6. The method of claim 4, wherein the RPN network structure in the craniocerebral tissue detection network is:

the first layer is an input layer, and the input of the first layer is the feature matrix with the size of 32 × 256 obtained in the step (3-1);

the second layer is a convolution layer, which uses a convolution kernel of 3 × 3, resulting in a matrix of 32 × 256;

the third layer is a convolution layer of 1 × 1 convolution kernels, the input of which is a 32 × 256 matrix, after convolution, and subsequent activation of a Sigmoid function, a 32 × 18 matrix is obtained, which comprises 32 × 9 results, each of which contains 2 score scores, namely the score scores of the foreground and the background.

The fourth layer is a convolution layer of convolution kernel 1 × 1, the input of the convolution layer is the output of the second layer, and is also a 32 × 256 matrix, after convolution, the convolution is connected with linear function activation, so that a 32 × 36 matrix is obtained, and comprises 32 × 9 results, each result comprises 4 coordinates, the 4 coordinates correspond to a candidate area for anchor point translation and scaling, and each candidate area corresponds to 2 score scores of the output of the third layer;

the fifth layer is an output layer, the input of which is a 32 x 18 matrix output by the third layer and a 32 x 36 matrix output by the fourth layer, then N candidate regions with the highest foreground score are selected from 9 background regions output, then M candidate regions with the highest foreground score are selected from the obtained N candidate regions by a non-maximum method, and the M candidate regions are sent to the ROI pooling layer.

The ROI pooling layer in the brain tissue detection network firstly receives the features with the size of 32 x 256 output by the last layer of the brain tissue detection network and M candidate regions obtained by RPN network screening as input, and then processes the features by adopting a RoiPoolingConv function, so that a plurality of candidate regions are mapped into a feature map with a fixed size, and the feature map with the fixed size is output.

7. The method of claim 4, wherein the classification and regression part of the craniocerebral tissue detection network is structured as:

the first layer is a first full-connection layer, the layer receives the feature map obtained by the ROI pooling layer, the output node is 4096, 4096 feature vectors are obtained, and then a Droupt operation is accessed to prevent overfitting.

The second layer is a second full-connection layer, the layer receives 4096 eigenvectors output by the first full-connection layer, the output node is 4096, 4096 eigenvectors are obtained, and then Droupt operation is accessed to prevent overfitting.

The third layer is a third fully-connected layer, the third layer receives 4096 eigenvectors output by the second fully-connected layer, the output node is 7, and a Softmax activation function is accessed to obtain 7 classification score values.

The fourth layer is a fourth fully-connected layer, and the layer receives 4096 feature vectors output by the second fully-connected layer to obtain 7 x 4-dimensional vectors, wherein 4 dimensions represent four coordinates of the detection frame.

8. The method of claim 1, wherein the craniocerebral tissue detection network is trained by:

(a1) acquiring a standard section image of fetal cranium;

(a2) and (b) performing feature extraction on the data set obtained in the step (a1) by using a deep convolutional neural network to obtain a feature map, extracting a candidate frame on the obtained feature map by using an RPN (resilient packet network), obtaining the feature map and the extracted candidate frame by using an ROI (region of interest) pooling layer, mapping a candidate region corresponding to the candidate frame into the feature map to pool the feature map into a region feature map with a uniform size, sending the region feature map into a full-connection layer to obtain a feature map with a fixed size, inputting the feature map into a classification and regression part, and obtaining classification of key tissues in fetal brains and accurate position coordinates of the key tissues after passing through two branches of the full-connection layer in the classification and regression part.

(a3) And optimizing the craniocerebral tissue detection network by using an Adam algorithm, and performing iterative training on the optimized craniocerebral tissue detection network by using a Keras framework.

9. The method of claim 8,

the learning rate lr in the optimization process is 0.001.

In the iterative training process, the batch size batchsize is 32, 165 batches are total, and the number of iterations is 200.

10. A system for detecting fetal cranial abnormalities based on machine learning, comprising:

the first module is used for acquiring fetal craniocerebral standard section data sets of different gestational week series;

the second module is used for preprocessing the fetal craniocerebral standard section data set acquired by the first module to obtain a preprocessed fetal craniocerebral standard section data set;

and the third module is used for inputting the fetal craniocerebral standard section data set pretreated by the second module into the trained craniocerebral tissue detection network so as to obtain the detection result of each key tissue in the fetal craniocerebral.

The fourth module is used for judging whether at least one key tissue in the fetal cranium is abnormal or not according to the detection result of each key tissue in the fetal cranium obtained by the third module, and if the at least one key tissue in the fetal cranium is abnormal, the fetal cranium is prompted to be normal, and the process is ended; if there is an abnormality, it is suggested to the fetus which related craniocerebral abnormality may exist and the process is ended.

Technical Field

The invention belongs to the technical field of computer-aided diagnosis, and particularly relates to a method and a system for detecting fetal craniocerebral abnormality based on machine learning.

Background

In the whole development process of a fetus, the healthy development of the craniocerebra has an abnormal important significance, and the intelligence of the fetus can be directly influenced by the craniocerebra hypoplasia, and cerebral palsy, mental retardation, epilepsy and the like can be seriously caused. For this reason, prenatal ultrasound is essential for detailed examination of fetal cranium.

However, there are some non-negligible disadvantages of the existing ultrasound for fetal craniocerebral detection: firstly, as the fetal craniocerebral detection process is quite complex, but the prior sonographers with abundant clinical experience and good prenatal examination of fetal craniocerebral abnormality are seriously lack, the working tasks of the experienced sonographers are greatly increased, and the problem of unbalanced high-quality medical resources is caused; secondly, when the brain image is acquired by using the ultrasound, the quality of the ultrasound image acquisition is not high due to various interference factors (such as interference of ultrasound artifacts, unskilled skill of sonographers, etc.), so that it is difficult to accurately analyze the brain development condition of the fetus according to the detection result; thirdly, the prenatal ultrasonic examination of fetal cranium has very high technical requirements for sonographers, and sonographers of different levels may obtain different cranium detection results for the same detection object, thereby affecting the accuracy of the final detection result.

Disclosure of Invention

The invention aims to provide a method and a system for detecting fetal craniocerebral abnormality based on machine learning, and aims to deeply learn large-sample-volume normal and abnormal fetal craniocerebral ultrasonic image data through Artificial Intelligence (AI), solve the technical problems in the existing fetal craniocerebral detection based on ultrasonic imaging.

To achieve the above object, according to one aspect of the present invention, there is provided a method for detecting fetal craniocerebral abnormality based on machine learning, comprising the steps of:

(1) acquiring a fetal craniocerebral standard section data set of different gestational week series;

(2) preprocessing the fetal craniocerebral standard section data set obtained in the step (1) to obtain a preprocessed fetal craniocerebral standard section data set;

(3) inputting the fetal craniocerebral standard section data set preprocessed in the step (2) into a trained craniocerebral tissue detection network to obtain detection results of each key tissue in the fetal craniocerebral.

(4) Judging whether at least one key tissue in the fetal cranium is abnormal according to the detection result of each key tissue in the fetal cranium obtained in the step (3), if not, prompting that the fetal cranium is normal, and ending the process; if there is an abnormality, it is suggested to the fetus which related craniocerebral abnormality may exist and the process is ended.

Preferably, step (2) comprises the sub-steps of:

(2-1) denoising each standard section image in the fetal craniocerebral standard section data set by using a self-adaptive mean filtering method, and cutting the denoised standard section images into uniform sizes to obtain the cut standard section images;

(2-2) denoising each cut standard section image obtained in the step (2-1) by using a self-adaptive median filtering method, sending each standard section image subjected to the denoising treatment to a radiology expert for marking, and obtaining each marked standard section image;

and (2-3) normalizing each marked standard section image obtained in the step (2-2) to obtain a normalized standard section image, wherein all normalized standard section images form a preprocessed fetal craniocerebral standard section data set.

Preferably, the critical tissues in the craniocerebral portion of the standard sectional image include the posterolateral sulcus, thalamus, choroid plexus, transparent compartment, third ventricle, midline brain, skull, and the like, in particular.

Preferably, the cranial tissue detection network comprises a deep convolutional neural network, an RPN network, an ROI pooling layer, and a classification and regression section connected in series.

Preferably, the deep convolutional neural network structure in the craniocerebral tissue detection network is as follows:

the first layer is the input layer, the input being a 512x 3 matrix of pixels.

The second layer is the first convolution layer, which receives a 512x 3 matrix of pixels from the input layer, which uses 32 convolution kernels of size 3 x 3, which has a step size of 1, followed by the RELU activation function and BN, which outputs a matrix of size 512x 32;

the third layer is a second convolution layer, the layer uses 64 convolution kernels with the size of 3 × 3 and the step size is 1, the next convolution kernel is accessed with the RELU activation function and BN, and the output matrix of the layer is 512 × 64;

the fourth layer is a third convolution layer, 64 convolution kernels with the size of 3 × 3 are used, the step size is 1, the Droupt operation, the RELU activation function and the BN are accessed, and the matrix of the output of the layer is 512 × 64;

the fifth layer is a first pooling layer, the size of the pooling window is 2x 2, the step size is 2, and the size of the output matrix is 256 x 64;

the sixth layer is the fourth convolutional layer, which uses 128 convolutional kernels of size 3 x 3, with a step size of 1, and outputs a matrix of 256 x 128;

the seventh layer is a fifth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 2, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 128 × 256;

the eighth layer is the sixth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 1, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 128 × 256;

the ninth layer is a second pooling layer, the size of the pooling windows is 2x 2, the step size is 2, and the size of the output matrix is 64 x 256;

the tenth layer is a seventh convolution layer, which uses 256 convolution kernels with the size of 3 × 3, the step size is 1, the next time the RELU activation function and BN are accessed, and the output matrix is 64 × 256;

the eleventh layer is the eighth convolution layer, which uses 256 convolution kernels with the size of 3 × 3, the step size is 1, the RELU activation function and BN are accessed at the back, and the output matrix is 64 × 256;

the twelfth layer is a ninth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 1, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 64 × 256;

the thirteenth layer is a third pooling layer, the size of the pooling window is 2x 2, the step size is 2, and the size of the output matrix is 32 x 256;

the fourteenth layer is the tenth convolution layer, which uses 256 convolution kernels with size 3 × 3, step size 1, followed by the RELU activation function and BN, and outputs a feature matrix of 32 × 256.

Preferably, the RPN network structure in the craniocerebral tissue detection network is:

the first layer is an input layer, and the input of the first layer is the feature matrix with the size of 32 × 256 obtained in the step (3-1);

the second layer is a convolution layer, which uses a convolution kernel of 3 × 3, resulting in a matrix of 32 × 256;

the third layer is a convolution layer of 1 × 1 convolution kernels, the input of which is a 32 × 256 matrix, after convolution, and subsequent activation of a Sigmoid function, a 32 × 18 matrix is obtained, which comprises 32 × 9 results, each of which contains 2 score scores, namely the score scores of the foreground and the background.

The fourth layer is a convolution layer of convolution kernel 1 × 1, the input of the convolution layer is the output of the second layer, and is also a 32 × 256 matrix, after convolution, the convolution is connected with linear function activation, so that a 32 × 36 matrix is obtained, and comprises 32 × 9 results, each result comprises 4 coordinates, the 4 coordinates correspond to a candidate area for anchor point translation and scaling, and each candidate area corresponds to 2 score scores of the output of the third layer;

the fifth layer is an output layer, the input of which is a 32 x 18 matrix output by the third layer and a 32 x 36 matrix output by the fourth layer, then N candidate regions with the highest foreground score are selected from 9 background regions output, then M candidate regions with the highest foreground score are selected from the obtained N candidate regions by a non-maximum method, and the M candidate regions are sent to the ROI pooling layer.

The ROI pooling layer in the brain tissue detection network firstly receives the features with the size of 32 x 256 output by the last layer of the brain tissue detection network and M candidate regions obtained by RPN network screening as input, and then processes the features by adopting a RoiPoolingConv function, so that a plurality of candidate regions are mapped into a feature map with a fixed size, and the feature map with the fixed size is output.

Preferably, the classification and regression part structure in the craniocerebral tissue detection network is as follows:

the first layer is a first full-connection layer, the layer receives the feature map obtained by the ROI pooling layer, the output node is 4096, 4096 feature vectors are obtained, and then a Droupt operation is accessed to prevent overfitting.

The second layer is a second full-connection layer, the layer receives 4096 eigenvectors output by the first full-connection layer, the output node is 4096, 4096 eigenvectors are obtained, and then Droupt operation is accessed to prevent overfitting.

The third layer is a third fully-connected layer, the third layer receives 4096 eigenvectors output by the second fully-connected layer, the output node is 7, and a Softmax activation function is accessed to obtain 7 classification score values.

The fourth layer is a fourth fully-connected layer, and the layer receives 4096 feature vectors output by the second fully-connected layer to obtain 7 x 4-dimensional vectors, wherein 4 dimensions represent four coordinates of the detection frame.

Preferably, the craniocerebral tissue detection network is trained by the following steps:

(a1) acquiring a standard section image of fetal cranium;

(a2) and (b) performing feature extraction on the data set obtained in the step (a1) by using a deep convolutional neural network to obtain a feature map, extracting a candidate frame on the obtained feature map by using an RPN (resilient packet network), obtaining the feature map and the extracted candidate frame by using an ROI (region of interest) pooling layer, mapping a candidate region corresponding to the candidate frame into the feature map to pool the feature map into a region feature map with a uniform size, sending the region feature map into a full-connection layer to obtain a feature map with a fixed size, inputting the feature map into a classification and regression part, and obtaining classification of key tissues in fetal brains and accurate position coordinates of the key tissues after passing through two branches of the full-connection layer in the classification and regression part.

(a3) And optimizing the craniocerebral tissue detection network by using an Adam algorithm, and performing iterative training on the optimized craniocerebral tissue detection network by using a Keras framework.

Preferably, the learning rate lr in the optimization process is 0.001, and the batch size batchsize is 32 in the iterative training process, which is 165 batches in total, and the number of iterations is 200.

According to another aspect of the present invention, there is provided a system for detecting fetal cranial abnormality based on machine learning, comprising:

the first module is used for acquiring fetal craniocerebral standard section data sets of different gestational week series;

the second module is used for preprocessing the fetal craniocerebral standard section data set acquired by the first module to obtain a preprocessed fetal craniocerebral standard section data set;

and the third module is used for inputting the fetal craniocerebral standard section data set pretreated by the second module into the trained craniocerebral tissue detection network so as to obtain the detection result of each key tissue in the fetal craniocerebral.

The fourth module is used for judging whether at least one key tissue in the fetal cranium is abnormal or not according to the detection result of each key tissue in the fetal cranium obtained by the third module, and if the at least one key tissue in the fetal cranium is abnormal, the fetal cranium is prompted to be normal, and the process is ended; if there is an abnormality, it is suggested to the fetus which related craniocerebral abnormality may exist and the process is ended.

In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:

(1) because the deep convolutional network, the RPN network, the ROI pooling layer and the classification and regression part are used for training the model in a combined training mode, whether key tissues in the cranium are abnormal or not can be intelligently detected without an experienced doctor, the normal and abnormal analysis and judgment can be carried out on the standard cranium section according to the detection result, and the specific fetal cranium development condition can be automatically judged according to the prior expert knowledge, so that the work task of an ultrasonic doctor is greatly reduced, and the condition of high-quality medical resource imbalance is relieved to a certain extent. And fills the gap of the field of detecting fetal craniocerebral development diseases by the current computer-assisted ultrasonic technology;

(2) because the adaptive mean filtering method of the invention carries out denoising processing on the standard section image, the ultrasonic noise interference of the image is reduced, and meanwhile, a deep convolution network is utilized to extract high-order abstract characteristics, thus a computer-aided doctor is adopted to carry out more accurate diagnosis on whether the main tissue structure of the fetal cranium is abnormal;

(3) because the data are marked and detected in the step (2) and the rectangular frame marks of the main cell structures are adopted, the method can detect the cell structure level of the cranium and ensure the high accuracy of the detection result.

Drawings

FIG. 1 is a schematic diagram of the flow of the method for detecting fetal craniocerebral abnormality based on machine learning according to the present invention;

FIG. 2 is a network architecture diagram of a cranial tissue detection network used in the method of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

As shown in fig. 1, the present invention provides a method for detecting fetal craniocerebral abnormality based on machine learning, comprising the following steps:

(1) acquiring a fetal craniocerebral standard section data set of different gestational week series;

specifically, the fetal craniocerebral standard section data set is composed of a plurality of fetal craniocerebral standard section images acquired from ultrasonic equipment of mainstream manufacturers on the market.

The diagnosis of the fetal craniocerebral abnormality by the doctor is realized by the indexes of whether the tissue structure exists in the fetal craniocerebral standard section image, whether the morphology is abnormal, whether the radial line is abnormal and the like, so the step is mainly to collect the fetal craniocerebral standard section data set.

(2) Preprocessing the fetal craniocerebral standard section data set obtained in the step (1) to obtain a preprocessed fetal craniocerebral standard section data set;

the method comprises the following substeps:

(2-1) denoising each standard section image in the fetal craniocerebral standard section data set by using a self-adaptive mean filtering method, and cutting the denoised standard section images into uniform sizes to obtain the cut standard section images;

specifically, in this step, the standard cut plane image size after clipping is 512 × 512 pixels;

(2-2) denoising each cut standard section image obtained in the step (2-1) by using a self-adaptive median filtering method, sending each standard section image subjected to the denoising treatment to a radiology expert for marking, and obtaining each marked standard section image;

specifically, because the speckle noise in the standard tangent plane image can affect the accuracy and reliability of diagnosis, and simultaneously the noise of the source data can directly affect the final diagnosis result, the noise is removed by using a self-adaptive median filtering method in the step;

the prenatal ultrasonic fetus craniocerebral diagnostician manually marks each standard section image after the drying treatment, and specifically marks key tissues in the craniocerebral part of the standard section image by using a rectangular frame, namely, marks lateral sulcus (LS for short), Thalamus (T for short), Choroid plexus (CP for short), transparent compartment (CSP for short), Third ventricle (TV for short), midline Brain (BM for short), Skull (Skull for short) and the like.

And (2-3) normalizing each marked standard section image obtained in the step (2-2) to obtain a normalized standard section image, wherein all normalized standard section images form a preprocessed fetal craniocerebral standard section data set.

Specifically, the normalization process uses the following formula:

the method implements equal scaling of the original image, wherein X_normIs normalized standard section image, X is marked standard section image_max、X_minThe maximum pixel value and the minimum pixel value of the marked standard tangent plane image are respectively. The gray value of the normalized standard section image is in [0,1 ]]In the meantime.

(3) Inputting the fetal craniocerebral standard section data set preprocessed in the step (2) into a trained craniocerebral tissue detection network to obtain detection results of each key tissue in the fetal craniocerebral.

Specifically, the brain tissue detection Network used in this step includes a deep convolutional neural Network (dtn), a Region generation Network (RPN), a Region of interest (ROI) Pooling layer, and a classification and regression part, which are connected in sequence, as shown in fig. 2.

The deep convolution neural network structure in the craniocerebral tissue detection network is as follows:

the first layer is the input layer, the input being a 512x 3 matrix of pixels.

The second layer is the first convolution layer, which receives 512 × 3 pixel matrices from the input layer, the layer uses 32 convolution kernels with size 3 × 3, the layer has step size 1, and then the RELU activation function and batch regularization (BN) are accessed, and the layer has output matrix with size 512 × 32;

the third layer is a second convolution layer, the layer uses 64 convolution kernels with the size of 3 × 3 and the step size is 1, the next convolution kernel is accessed with the RELU activation function and BN, and the output matrix of the layer is 512 × 64;

the fourth layer is a third convolution layer, 64 convolution kernels with the size of 3 × 3 are used, the step size is 1, the Droupt operation, the RELU activation function and the BN are accessed, and the matrix of the output of the layer is 512 × 64;

the fifth layer is a first pooling layer, the size of the pooling window is 2x 2, the step size is 2, and the size of the output matrix is 256 x 64;

the sixth layer is the fourth convolutional layer, which uses 128 convolutional kernels of size 3 x 3, with a step size of 1, and outputs a matrix of 256 x 128;

the seventh layer is a fifth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 2, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 128 × 256;

the eighth layer is the sixth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 1, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 128 × 256;

the ninth layer is a second pooling layer, the size of the pooling windows is 2x 2, the step size is 2, and the size of the output matrix is 64 x 256;

the tenth layer is a seventh convolution layer, which uses 256 convolution kernels with the size of 3 × 3, the step size is 1, the next time the RELU activation function and BN are accessed, and the output matrix is 64 × 256;

the eleventh layer is the eighth convolution layer, which uses 256 convolution kernels with the size of 3 × 3, the step size is 1, the RELU activation function and BN are accessed at the back, and the output matrix is 64 × 256;

the twelfth layer is a ninth convolution layer, which uses 256 convolution kernels with the size of 3 × 3 and the step size of 1, and then is accessed with Droupt operation, RELU activation function and BN, and the output matrix is 64 × 256;

the thirteenth layer is a third pooling layer, the size of the pooling window is 2x 2, the step size is 2, and the size of the output matrix is 32 x 256;

the fourteenth layer is a tenth convolution layer, which uses 256 convolution kernels with the size of 3 × 3, the step length is 1, the next RELU activation function and BN are accessed, and the output feature matrix is 32 × 256;

the RPN network structure in the craniocerebral tissue detection network is as follows:

the first layer is an input layer, and the input of the first layer is the feature matrix with the size of 32 × 256 obtained in the step (3-1);

the second layer is a convolution layer, which uses a convolution kernel of 3 × 3, resulting in a matrix of 32 × 256;

the third layer is a convolution layer of 1 × 1 convolution kernels, the input of which is a 32 × 256 matrix, and after convolution and subsequent activation of a sigmoid function, a 32 × 18 matrix is obtained, which comprises 32 × 9 results, each of which contains 2 score scores, namely the score scores of the foreground and the background.

The fourth layer is a convolution layer of a convolution kernel of 1 × 1, the input of the convolution layer is the output of the second layer, and is also a 32 × 256 matrix, after convolution, the convolution is accessed to a Linear (Linear) function for activation, so that a 32 × 36 matrix is obtained, and comprises 32 × 9 results, each result comprises 4 coordinates, the 4 coordinates correspond to a candidate region for Anchor point (Anchor) translation scaling, and each candidate region corresponds to 2 score scores output by the third layer;

the fifth layer is the output layer, and the inputs are the 32 x 18 matrix of the output of the third layer and the 32 x 36 matrix of the output of the fourth layer, then N candidate regions with the highest foreground score (N ranges from 2000 to 10000, preferably 2000, 2500 or 3000) are selected from the 9 background regions of the output, then M candidate regions with the highest foreground score (M ranges from 200 to 1000, preferably 200 or 300) are selected from the N candidate regions obtained by the non-maximum method, and the M candidate regions (propofol) are sent to the ROI pooling layer.

An ROI (region of interest) pooling layer in the brain tissue detection network firstly receives the features with the size of 32 x 256 output by the last layer of the brain tissue detection network and M candidate regions obtained by RPN (resilient packet network) screening as input, and then processes the features by adopting a RoiPoolingConv function, so that a plurality of candidate regions are mapped into a feature map with a fixed size, and the feature map with the fixed size is output;

the classification and regression part structure in the craniocerebral tissue detection network is as follows:

the first layer is a first full-connection layer, the layer receives the feature map obtained by the ROI pooling layer, the output node is 4096, 4096 feature vectors are obtained, and then a Droupt operation is accessed to prevent overfitting.

The second layer is a second full-connection layer, the layer receives 4096 eigenvectors output by the first full-connection layer, the output node is 4096, 4096 eigenvectors are obtained, and then Droupt operation is accessed to prevent overfitting.

The third layer is a third fully-connected layer, the third layer receives 4096 eigenvectors output by the second fully-connected layer, the output node is 7, and a Softmax activation function is accessed to obtain 7 classification score values.

The fourth layer is a fourth fully-connected layer, and the layer receives 4096 feature vectors output by the second fully-connected layer to obtain 7 x 4-dimensional vectors, wherein 4 dimensions represent four coordinates of the detection frame.

The craniocerebral tissue detection network is obtained by training the following steps:

(a1) acquiring a standard section image of fetal cranium;

specifically, the data set comprised 6600 standard slice images of the fetal cranium at the mid-term of fetal development, which were randomly divided into 3 parts, 80% of which were used for training the model, 10% for testing the model, and 10% for validating the model. The brain tissue detection network was trained using an 80% fetal craniocerebral standard section dataset.

(a2) Performing feature extraction on the data set obtained in the step (a1) by using a deep convolutional neural network to obtain a feature map, extracting candidate frames (including the scores of the candidate frames and the coordinates of the candidate frames) on the obtained feature map by using an RPN (resilient packet network), acquiring the feature map and the extracted candidate frames by using an ROI (region of interest) pooling layer, mapping candidate regions corresponding to the candidate frames into the feature map to pool the feature map into a region feature map with a uniform size, sending the region feature map into a full-connected layer to obtain a feature map with a fixed size, inputting the feature map into a classification and regression part, and obtaining the classification of key tissues in fetal brains and the accurate position coordinates of the key tissues after passing through two branches of the full-connected layer in the classification and regression part.

(a3) And optimizing the craniocerebral tissue detection network by using an Adam algorithm, and performing iterative training on the optimized craniocerebral tissue detection network by using a Keras framework.

Specifically, the learning rate lr in the optimization process is 0.001.

In the iterative training process, the batch size batchsize is 32, 165 batches are total, and the number of iterations is 200.

And (3) training by using a Keras framework to obtain a craniocerebral tissue detection network capable of detecting the mapping relation between the completeness of the fetal craniocerebral tissue structure and common diseases.

(4) Judging whether at least one key tissue in the fetal cranium is abnormal according to the detection result of each key tissue in the fetal cranium obtained in the step (3), if not, prompting that the fetal cranium is normal, and ending the process; if there is an abnormality, it is suggested to the fetus which related craniocerebral abnormality may exist and the process is ended.

For example:

the lack of diaphaneity indicates that the fetus may suffer from isolated diaphaneity, hypoplasia of visual compartment, whole forebrain and other diseases.

The abnormal shape of lateral sulcus indicates that the fetus may have lissencephaly/megacephaly, multiple cerebellar gyrus, fissure malformation, etc.

Abnormal thalamus indicates that the fetus may have lobar or hemilobar holoforebrain, brain stem-pontine dysplasia, stria cerebri-kinetosis, thalamus hemorrhage and the like.

The abnormal choroid plexus indicates that the fetus may have choroid plexus cyst, whole forebrain, intracerebroventricular hemorrhage, choroid plexus papilloma and the like.

Results of Performance testing

We tested three pregnancies of the pregnant women with an Accuracy (Accuracy) and a Recall (Recall) using ultrasound craniocerebral picture data from a 660 midpregnancy birth test, the average Accuracy and Recall being shown in table 1 below.

TABLE 1

	Rate of accuracy	Recall rate
			Outer ditch (LS)	94.96％	87.90％
Thalamus (T)	90.16％	89.20％
			Choroid Plexus (CP)	94.96％	88.90％
Transparent separated Cavity (CSP)	95.96％	89.90％
			Third Ventricle (TV)	90.96％	91.90％
Midline of the Brain (BM)	88.96％	84.90％
			Skull (S)	93.96％	88.90％

As can be seen from table 1 above: the accuracy and the recall rate of the three pregnancy periods are high, and the development diseases possibly suffered by fetal craniocerebral development are intelligently judged according to the missing structures, so that the identification and judgment of the invention are quite ideal on the whole.

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.