阅读说明：本技术 甲状旁腺mibi图像分析系统、计算机设备及存储介质 (Parathyroid MIBI image analysis system, computer device, and storage medium ) 是由 赵婉君 苏安平 魏涛 于 2021-09-27 设计创作，主要内容包括：本发明属于甲状旁腺功能疾病诊断技术领域,具体涉及一种甲状旁腺MIBI图像分析系统、计算机设备及存储介质。本发明的计算机设备用于实现如下甲状旁腺MIBI图像分析的步骤：步骤1,对甲状旁腺的MIBI图像进行预处理后输入深度学习模型；步骤2,深度学习模型进行计算后,输出所述甲状旁腺的MIBI图像的分类结果；其中,所述深度学习模型通过如下方法构建得到：对四种卷积神经网络模型分别建模并融合,构建最终的深度学习模型。本发明还提供了实现上述分析步骤的系统,包括子模型训练模块、子模型融合模块、测试模块、使用模块、降噪模块和增强模块。本发明通过对多种子模型的融合,具有扬长避短的效果,提高了准确性和鲁棒性,具有很好的应用前景。(The invention belongs to the technical field of parathyroid gland functional disease diagnosis, and particularly relates to a parathyroid gland MIBI image analysis system, computer equipment and a storage medium. The computer device of the present invention is used to implement the following parathyroid MIBI image analysis steps: step 1, preprocessing an MIBI image of parathyroid gland and inputting the preprocessed MIBI image into a deep learning model; step 2, after the deep learning model is calculated, outputting a classification result of the MIBI image of the parathyroid gland; the deep learning model is constructed by the following method: and respectively modeling and fusing the four convolutional neural network models to construct a final deep learning model. The invention also provides a system for realizing the analysis steps, which comprises a sub-model training module, a sub-model fusion module, a test module, a use module, a noise reduction module and an enhancement module. The method has the advantages of improving the advantages and avoiding the disadvantages by fusing multiple sub-models, improving the accuracy and robustness and having good application prospect.)

1. A computer device for parathyroid MIBI image analysis, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that: the processor, when executing the program, performs the steps of parathyroid MIBI image analysis as follows:

step 1, preprocessing an MIBI image of parathyroid gland and inputting the preprocessed MIBI image into a deep learning model;

step 2, after the deep learning model is calculated, outputting a classification result of the MIBI image of the parathyroid gland;

the deep learning model is constructed by the following method:

step A, respectively modeling four convolutional neural network models, namely a VGG model, a ResNet model, a DenseNet model and an EfficientNet model, by adopting a multi-head attention mechanism;

and step B, performing neural model fusion by adopting a majority voting mode, and constructing a final deep learning model.

2. The computer apparatus of claim 1, wherein: in step a, the number of layers of the VGG model is 19, the number of layers of the ResNet model is at least one of 18 layers, 50 layers or 152 layers, the number of layers of the DenseNet model is at least one of 169 layers or 264 layers, and the version of the EfficientNet model is at least one of b0, b4 or b 7.

3. The computer apparatus of claim 2, wherein: in the step A, modeling is carried out on a VGG model with 19 layers, a ResNet model with 18 layers, 50 layers and 152 layers, a DenseNet model with 169 layers and 264 layers and EfficientNet models with b0 versions, b4 versions and b7 versions to respectively obtain 9 sub-models;

in step B, the result of performing neural model fusion is:

Result＝k1×VGG19+k2×ResNet18+k3×ResNet50+k4×ResNet152+k5×DenseNet169+k6×DenseNet264+k7×EfficientNetb0+k8×EfficientNetb4+k9×EfficientNetb7；

wherein Result is the Result of classifying parathyroid MIBI images by the final deep learning model, k1, k2, k3, k4, k5, k6, k7, k8 and k9 are coefficients of sub models, which satisfy k1+ k2+ k3+ k4+ k5+ k6+ k7+ k8+ k9 ═ 1, VGG19, ResNet18, ResNet50, ResNet152, densnet 169, densnet 264, efficientnet 0, efficientnet 4 and efficientnet 7 are the results of classifying parathyroid MIBI images by 9 sub models respectively.

4. The computer apparatus of claim 1, wherein: in step 1, the preprocessing includes at least one of noise reduction or super-resolution data enhancement.

5. The computer apparatus of claim 4, wherein: the noise reduction method is to process the image by adopting a self-adaptive median filter.

6. The computer apparatus of claim 4, wherein: the super-resolution data enhancement is realized by adopting a test time sequence enhancement technology.

7. The computer apparatus of claim 1, wherein: and the classification result output by the deep learning model is the probability that the MIBI image belongs to normal parathyroid gland, parathyroid adenoma, parathyroid hyperplasia, parathyroid carcinoma and parathyroid cyst.

8. A computer-readable storage medium characterized by: stored thereon a computer program for carrying out the steps of the parathyroid MIBI image analysis of any of claims 1-7.

9. A system for MIBI image analysis of a parathyroid gland, comprising:

the sub-model training module is used for respectively modeling four convolutional neural network models, namely a VGG model, a ResNet model, a DenseNet model and an EfficientNet model, by adopting a multi-head attention mechanism;

the sub-model fusion module is used for carrying out neural model fusion by adopting a majority voting mode to construct a final deep learning model;

the test module is used for inputting the test set into the deep learning model constructed by the sub-model fusion module and evaluating the accuracy of the model;

and the using module is used for inputting the MIBI images of the parathyroid gland into the deep learning model constructed by the sub-model fusion module and outputting the classification result of the MIBI images of the parathyroid gland.

10. The system of claim 9, wherein the system further comprises:

the noise reduction module is used for carrying out noise reduction pretreatment on the MIBI image of the parathyroid gland;

and the enhancement module is used for carrying out super-resolution data enhancement preprocessing on the MIBI image of the parathyroid gland.

Technical Field

The invention belongs to the technical field of parathyroid gland functional disease diagnosis, and particularly relates to a parathyroid gland MIBI image analysis system, computer equipment and a storage medium.

Background

Parathyroid gland is an important organ for maintaining calcium ion balance of the body, and parathyroid gland related diseases are caused by calcium and phosphorus metabolism disorder due to parathyroid hormone abnormality secreted by parathyroid gland main cells, so that a series of multi-system and multi-organ pathological changes and dysfunction of urinary system, digestive system, nerve system, skin system and the like are caused. Parathyroid disorders are divided into two major categories, hyperparathyroidism and hypofunction, with hyperparathyroidism being the most common. Hyperparathyroidism generally lacks specific clinical manifestations and many changes in imaging due to changes in its structure have become important clinical bases, including parathyroid adenoma, parathyroid hyperplasia, parathyroid carcinoma, and parathyroid cyst. Therefore, accurate localization and assessment of problem foci of the parathyroid gland is critical to the diagnosis and treatment of parathyroid related diseases.

99 mTechnetium-methoxyisobutylisonitrile (99mTc-MIBI, MIBI for short) imaging is the most common and accurate method for parathyroid gland image examination. Currently, the 99mTcMIBI two-phase imaging method is commonly used to localize diseased parathyroid glands clinically. The principle is to use the difference of the clearance rate of the thyroid gland and the parathyroid gland to 99mTc-MIBI, thereby qualitatively and locally diagnosing the hyperparathyroid gland tissue. The MIBI planar imaging is a two-dimensional image and has lower resolution, meanwhile, the parathyroid gland has the characteristics of small volume and low visual identification difference of disease difference, and the identification of the parathyroid gland MIBI imaging has extremely high requirements on the experience and the professional degree of a clinician.

The deep learning neural network algorithm is a more effective image identification method. The neural network algorithm is a nonlinear model for simulating the biological activity of cerebral neurons, and has the advantages of being capable of rapidly identifying linear types, processing mixed and incomplete data, being high in fault tolerance capability, training data popularization capability and the like. Deep learning is a deep artificial neural network. The method is a nonlinear multi-layer feature learning model, which automatically detects and classifies feature information of original data (such as pixels, characters and the like), learns features on a plurality of layers according to a nonlinear modularization mode, and then simplifies and integrates the features step by step to generate a final result. At present, related researches are carried out on a method for diagnosing parathyroid related diseases by utilizing a neural network algorithm, and Chinese patent application CN111062953A, namely a method for identifying parathyroid hyperplasia in an ultrasonic image, provides a method for identifying parathyroid hyperplasia and obtains a training sample comprising the ultrasonic image and a corresponding diagnosis result; wherein, the ultrasonic image is an image obtained by ultrasonic imaging of the hyperparathyroidism affected area of the patient; carrying out data enhancement processing on the training sample by adopting a mixed gamma-CLAHE method; inputting the training sample after the enhancement treatment into a Faster R-CNN network to obtain a trained network model; and inputting the neck ultrasonic image to be recognized into the trained Faster R-CNN model to obtain a recognition result output by the trained model. However, the image data used in this patent application is an ultrasound image, the ultrasound image has extremely low resolution to the parathyroid gland, and the color ultrasound is not conventionally used in clinic as a radiographic examination of the parathyroid gland; meanwhile, the method can only identify the focus of parathyroid hyperplasia, and is not suitable for other types of focuses of parathyroid function diseases.

At present, the mature MIBI imaging artificial intelligence recognition technology can realize the recognition of multiple parathyroid foci at home and abroad.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a parathyroid MIBI image analysis system, computer equipment and a storage medium, and aims to provide a method with high accuracy and good robustness, so that the identification and classification of normal parathyroid gland, parathyroid adenoma, parathyroid hyperplasia, parathyroid carcinoma and parathyroid cyst on a parathyroid MIBI image by using a deep learning model can be realized.

A computer device for parathyroid MIBI image analysis, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to perform the steps of parathyroid MIBI image analysis as follows:

step 1, preprocessing an MIBI image of parathyroid gland and inputting the preprocessed MIBI image into a deep learning model;

step 2, after the deep learning model is calculated, outputting a classification result of the MIBI image of the parathyroid gland;

the deep learning model is constructed by the following method:

step A, respectively modeling four convolutional neural network models, namely a VGG model, a ResNet model, a DenseNet model and an EfficientNet model, by adopting a multi-head attention mechanism;

and step B, performing neural model fusion by adopting a majority voting mode, and constructing a final deep learning model.

Preferably, in step a, the number of layers of the VGG model is 19, the number of layers of the ResNet model is at least one of 18 layers, 50 layers, or 152 layers, the number of layers of the DenseNet model is at least one of 169 layers or 264 layers, and the version of the EfficientNet model is at least one of b0, b4, or b 7.

Preferably, in step a, a VGG model of 19 layers, a ResNet model of 18 layers, 50 layers and 152 layers, a DenseNet model of 169 layers and 264 layers, and EfficientNet models of versions b0, b4 and b7 are modeled to obtain 9 sub-models respectively;

in step B, the result of performing neural model fusion is:

Result＝k1×VGG19+k2×ResNet18+k3×ResNet50+k4×ResNet152+k5×DenseNet169+k6×DenseNet264+k7×EfficientNetb0+k8×EfficientNetb4+k9×EfficientNetb7；

wherein Result is the Result of classifying parathyroid MIBI images by the final deep learning model, k1, k2, k3, k4, k5, k6, k7, k8 and k9 are coefficients of sub models, which satisfy k1+ k2+ k3+ k4+ k5+ k6+ k7+ k8+ k9 ═ 1, VGG19, ResNet18, ResNet50, ResNet152, densnet 169, densnet 264, efficientnet 0, efficientnet 4 and efficientnet 7 are the results of classifying parathyroid MIBI images by 9 sub models respectively.

Preferably, in step 1, the preprocessing includes at least one of noise reduction or super-resolution data enhancement.

Preferably, the noise reduction method is to process the image by using an adaptive median filter.

Preferably, the super-resolution data enhancement is realized by adopting a test timing sequence enhancement technology.

Preferably, the classification result output by the deep learning model is the probability that the MIBI image belongs to normal parathyroid gland, parathyroid adenoma, parathyroid hyperplasia, parathyroid carcinoma and parathyroid cyst.

The present invention also provides a computer-readable storage medium having stored thereon a computer program for carrying out the steps of the above-described parathyroid MIBI image analysis.

The present invention also provides a system for MIBI image analysis of a parathyroid gland, comprising:

the sub-model training module is used for respectively modeling four convolutional neural network models, namely a VGG model, a ResNet model, a DenseNet model and an EfficientNet model, by adopting a multi-head attention mechanism;

the sub-model fusion module is used for carrying out neural model fusion by adopting a majority voting mode to construct a final deep learning model;

the test module is used for inputting the test set into the deep learning model constructed by the sub-model fusion module and evaluating the accuracy of the model;

and the using module is used for inputting the MIBI images of the parathyroid gland into the deep learning model constructed by the sub-model fusion module and outputting the classification result of the MIBI images of the parathyroid gland.

Preferably, the system further comprises:

the noise reduction module is used for carrying out noise reduction pretreatment on the MIBI image of the parathyroid gland;

and the enhancement module is used for carrying out super-resolution data enhancement preprocessing on the MIBI image of the parathyroid gland.

The technical scheme of the invention has the following beneficial effects:

1. the method and the device for identifying the MIBI developed image of the parathyroid gland based on deep learning are used for assisting a doctor in identifying the parathyroid gland and related diseases by imaging, and have the advantage of high accuracy. The accuracy rate of parathyroid MIBI developed image identification reaches 95%, and the method has important significance for clinical auxiliary diagnosis, and especially has important clinical significance for primary screening under the condition that a primary hospital lacks a thyroid ultrasonography department.

2. The invention is based on clinic, highly fit with clinical logic thinking and actual use scenes, and has convenient operation and high practicability.

3. In the preferred scheme, before the image deep learning is carried out, the noise reduction processing is carried out on the MIBI image, the interference of the image quality to a training model is reduced, and the accuracy is improved. And the ATT technology is adopted to perform data enhancement on the training set, so that the richness of the data of the training set is increased. The method is characterized in that the method is trained by using sub models of 9 different levels and versions of four convolutional network models, namely VGG, ResNet, DenseNet and EfficientNet, wherein the models are the latest large deep learning networks with better performance on medical images. The final model combines sub-models of 9 different levels and versions of the four convolutional network models to fuse, so that the advantages and the disadvantages of the sub-models are improved, and the accuracy and the robustness of the training model are improved. When the model is used, the newly input images are subjected to noise reduction processing, so that the degree of adaptation to the model is facilitated, and the accuracy of the system is improved.

Obviously, many modifications, substitutions, and variations are possible in light of the above teachings of the invention, without departing from the basic technical spirit of the invention, as defined by the following claims.

The present invention will be described in further detail with reference to the following examples. This should not be understood as limiting the scope of the above-described subject matter of the present invention to the following examples. All the technologies realized based on the above contents of the present invention belong to the scope of the present invention.

Drawings

FIG. 1 is a schematic flow chart of example 1 of the present invention;

fig. 2 is a schematic structural diagram of a system in embodiment 1 of the present invention.

Detailed Description

It should be noted that, in the embodiment, the algorithm of the steps of data acquisition, transmission, storage, processing, etc. which are not specifically described, as well as the hardware structure, circuit connection, etc. which are not specifically described, can be implemented by the contents disclosed in the prior art.

Example 1 System for Parathyroid MIBI image analysis

The present embodiment provides a system for parathyroid MIBI image analysis, comprising: the device comprises a noise reduction module, an enhancement module, a sub-model training module, a sub-model fusion module, a test module and a use module.

Wherein:

the sub-model training module is used for respectively modeling four convolutional neural network models, namely a VGG model, a ResNet model, a DenseNet model and an EfficientNet model, by adopting a multi-head attention mechanism;

the sub-model fusion module is used for carrying out neural model fusion by adopting a majority voting mode to construct a final deep learning model;

the test module is used for inputting the test set into the deep learning model constructed by the sub-model fusion module and evaluating the accuracy of the model;

the using module is used for inputting the MIBI images of the parathyroid gland into the deep learning model constructed by the sub-model fusion module and outputting the classification result of the MIBI images of the parathyroid gland;

the noise reduction module is used for carrying out noise reduction pretreatment on the MIBI image of the parathyroid gland;

and the enhancement module is used for carrying out super-resolution data enhancement preprocessing on the MIBI image of the parathyroid gland.

The above system may be implemented in a computing and apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the program implementing the parathyroid MIBI image analysis step as shown in fig. 1. The specific process of parathyroid MIBI image analysis is as follows:

1. extracting data: the original data matching, desensitization and extraction are performed by retrospectively collecting postoperative pathological data of patients who have undergone parathyroid gland surgery and parathyroid MIBI imaging pictures before surgery.

2. The method for marking parathyroid MIBI development original pictures comprises the following steps: the image names (classification results) are used as label labels, and the labels are classified into normal parathyroid gland, parathyroid adenoma, parathyroid hyperplasia, parathyroid carcinoma and parathyroid cyst.

3. Noise reduction pretreatment: and carrying out noise reduction processing on the image by adopting an adaptive median filter. It includes two working processes:

and a process A:

A₁＝z_med-z_min

A₂＝z_med-z_max

if A is₁> 0 and A₂If the value is less than 0, the process B is switched to;

otherwise, increasing the image size;

if the window size is less than or equal to S_maxRepeating the process A;

otherwise output z_med；

And a process B:

B₁＝z_xy-z_min

B₂＝z_xy-z_max

if B is present₁> 0 and B₂If < 0, then z is output_xy

Otherwise, output x_med

Wherein: the meaning of the symbols is as follows:

S_xythe image data needing noise reduction processing is obtained;

z_minis S_xyThe minimum gray value;

z_maxis S_xyMedium maximum gray value;

z_medis S_xyMedian of median gray values;

z_xyis the gray value at coordinate (x, y);

S_maxis S_xyThe maximum size allowed.

4. The noise-reduced MIBI developed image is divided into a training image set and a testing image set in a random distribution mode, wherein the training set accounts for 80%, and the testing set accounts for 20%. In this example, 5000 labeled parathyroid MIBI images are provided, 4000 of which are used as training sets and 1000 of which are used as test sets.

5. And (2) performing data enhancement by adopting a test time sequence enhancement (TTA) technology, turning, rotating, cutting, deforming, scaling and the like on the training set image of each model, increasing the training set image quantity, and inputting the training set image quantity into the training model, namely the test time sequence enhancement (TTA) technology.

6. Defining the selected sub-model and the number of layers or version number thereof, in this embodiment, 9 seed models are selected, which are: VGG model at 19 layers, ResNet model at 18, 50 and 152 layers, DenseNet model at 169 and 264 layers, and EfficientNet model at versions b0, b4 and b 7. And respectively utilizing the training sets to train and model the 9 sub-models by adopting a multi-head attention mechanism.

7. Performing neural model fusion by adopting a majority voting mode, setting parameters for each sub-model, and performing neural model fusion as follows:

Result＝k1×VGG19+k2×ResNet18+k3×ResNet50+k4×ResNet152+k5×DenseNet169+k6×DenseNet264+k7×EfficientNetb0+k8×EfficientNetb4+k9×EfficientNetb7；

wherein Result is the Result of classifying parathyroid MIBI images by the final deep learning model (i.e. the probability of predicting that the MIBI images are classified by normal parathyroid gland, parathyroid adenoma, parathyroid hyperplasia, parathyroid carcinoma and parathyroid cyst), k1, k2, k3, k4, k5, k6, k7, k8 and k9 are coefficients of sub-models, which satisfy k1+ k2+ k3+ k4+ k5+ k6+ k7+ k8+ k9 ═ 1, VGG19, ResNet18, ResNet50, ResNet152, densnet 169, densnet 264, efficientb 0, efficientb 4 and efficientnet 7 are the Result of classifying the parathyroid MIBI images by 9 sub-models respectively.

8. And inputting the test set data into the fusion model for internal test, and evaluating the accuracy of the model.

9. And inputting the parathyroid MIBI images to be classified into the final deep learning model after noise reduction preprocessing to obtain classification results of the parathyroid MIBI images to be classified.

In order to further explain the technical scheme of the invention, the beneficial effects of the invention are further explained by experimental examples.

Experimental example 1 investigation of model accuracy

In this experimental example, the same method as that used in example 1 was used to model a deep learning model, and the results of the nine submodels (GG19, ResNet18, ResNet50, ResNet152, DenseNet169, DenseNet264, EfficientNet-b0, EfficientNet-b4, and EfficientNet-b7) obtained and 1 fused deep learning model [ ensemble (tta) ], AUC (area under ROC curve), F1, and k value are shown in the following table:

as can be seen from the data in the table, the accuracy, AUC, F1 and k value of the 9 sub-models are all significantly lower than those of the fused deep learning model. This shows that the fusion of the application to 9 submodels realizes the effect of making good use of the advantages and avoiding the disadvantages of the submodels, thereby improving the accuracy and robustness of the models.

As can be seen from the above examples and experimental examples, the present invention provides a method, system and apparatus for identifying and classifying normal parathyroid gland, parathyroid adenoma, parathyroid hyperplasia, parathyroid carcinoma and parathyroid cyst from parathyroid MIBI images using a deep learning model. According to the invention, the effects of making good use of the advantages and avoiding the disadvantages of different models are realized by means of fusing the four convolutional neural network models, namely the VGG model, the ResNet model, the DenseNet model and the EfficientNet model, so that the deep learning model for analyzing the parathyroid MIBI image has high accuracy and good robustness. Therefore, the parathyroid MIBI image analysis technology has good application prospect.