阅读说明：本技术 训练神经网络的方法、神经网络、成像方法、存储介质、医学图像处理设备及医学检测设备 (Method of training neural network, imaging method, storage medium, medical image processing apparatus, and medical examination apparatus ) 是由 陈鸣之 张欣宇 佘铭钢 张静 于 2020-05-25 设计创作，主要内容包括：本发明涉及一种训练神经网络的方法,包括如下步骤：采集关于受检对象的基准医学图像以作为Ground Truth数据集；获取关于所述受检对象的2D投影图像；以及根据所述2D投影图像形成所述神经网络的输入,并以所述Ground Truth数据集作为目标来训练所述神经网络。(The invention relates to a method for training a neural network, which comprises the following steps: acquiring a reference medical image about a subject as a group Truth data set; acquiring a 2D projection image with respect to the object under examination; and forming an input of the neural network from the 2D projection image, and training the neural network with the group Truth data set as a target.)

1. A method of training a neural network, the method comprising the steps of:

acquiring a reference medical image about a subject as a group Truth data set;

acquiring a 2D projection image with respect to the object under examination; and

forming an input of the neural network from the 2D projection images, and training the neural network with the group Truth data set as a target.

2. The method of claim 1, wherein the step of forming the input to the neural network from the 2D projection images comprises: taking the 2D projection image as an input of the neural network.

3. The method of claim 1, wherein the step of forming the input to the neural network from the 2D projection images comprises: reconstructing based on the 2D projection images to obtain a 3D TOMO image and taking the 3D TOMO image and/or the 2D projection images as input to the neural network.

4. The method according to any one of claims 1-3, wherein the reference medical image comprises one or more of a CT reconstructed image, a CBCT reconstructed image, an MR reconstructed image, an anatomical image of the subject.

5. The method according to any one of claims 1-3, wherein the subject is biological tissue.

6. The method according to any one of claims 1-3, wherein the subject is a pre-fabricated biomimetic tissue.

7. The method according to any one of claims 1-3, wherein the step of acquiring a 2D projection image in respect of the object under examination comprises: and acquiring the 2D projection image by adopting a forward projection algorithm.

8. The method of claim 3, wherein the step of obtaining the 3D TOMO image comprises: reconstructing by an analytical reconstruction algorithm or an iterative reconstruction algorithm to obtain the 3D TOMO image.

9. The method of any one of claims 1-3, wherein the neural network is a U-net or a dual domain network.

10. A neural network, characterized in that the neural network is trained by a method of training a neural network according to any one of claims 1-9.

11. A TOMO imaging method, comprising the steps of:

training a neural network using a method of training a neural network as claimed in any one of claims 1 to 9;

acquiring a 2D projection image with respect to an object under examination;

forming an input of the neural network according to the 2D projection image, and inputting the input into the neural network; and

and outputting the neural network as a 3D TOMO image about the detected object.

12. A TOMO imaging method, comprising the steps of:

acquiring a 2D projection image with respect to an object under examination;

forming an input to a neural network from the 2D projection images and inputting to the neural network of claim 10; and

and outputting the neural network as a 3D TOMO image about the detected object.

13. A computer-readable storage medium having instructions stored therein, which when executed by a processor, cause the processor to perform the method of any one of claims 1-9.

14. A computer-readable storage medium having stored therein instructions, which when executed by a processor, cause the processor to perform the method of claim 11 or 12.

15. A medical image processing apparatus, characterized in that the apparatus comprises:

an input module configured to receive a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image;

a processing module comprising the neural network of claim 10 and configured to process the input data with the neural network to generate a processed 3D TOMO image; and

an output module configured to output the processed 3D TOMO image.

16. A medical image processing apparatus, characterized in that the apparatus comprises:

an input module configured to receive a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image;

the computer-readable storage medium of claim 13;

a processing module comprising a neural network and configured to execute instructions stored in the computer-readable storage medium to train the neural network, and further configured to process the input data with the trained neural network to generate a processed 3D TOMO image; and

an output module configured to output the processed 3D TOMO image.

17. A medical examination apparatus, characterized in that the apparatus comprises:

an imaging module configured to acquire a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image;

a processing module comprising the neural network of claim 10 and configured to process the input data with the neural network to generate a processed 3D TOMO image about the subject; and

an output module configured to output the processed 3D TOMO image.

18. The apparatus of claim 17, wherein the input data is the 2D projection image.

19. The apparatus of claim 17, wherein the imaging module comprises:

a first module configured to acquire a 2D projection image with respect to an object under examination; and

a second module configured to reconstruct by an analytical reconstruction algorithm or an iterative reconstruction algorithm based on the 2D projection images to obtain an original 3D TOMO image for an object under examination, the original 3D TOMO image being the input data.

20. A medical examination apparatus, characterized in that the apparatus comprises:

an imaging module configured to acquire a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image;

the computer-readable storage medium of claim 13;

a processing module comprising a neural network and configured to execute instructions stored in the computer-readable storage medium to train the neural network, and further configured to process the input data with the trained neural network to generate a processed 3D TOMO image; and

an output module configured to output the processed 3D TOMO image.

21. The apparatus of claim 20, wherein the input data is the 2D projection image.

22. The apparatus of claim 20, wherein the imaging module comprises:

a first module configured to acquire a 2D projection image with respect to an object under examination; and

a second module configured to reconstruct by FDK algorithm or SART algorithm based on the 2D projection image to obtain an original 3D TOMO image with respect to an object under examination, the original 3D TOMO image being the input data.

Technical Field

The present invention relates to the field of medical testing, and more particularly, to a method of training a neural network, an imaging method, a storage medium, a medical image processing apparatus, and a medical testing apparatus.

Background

Tomosynthesis (Tomosynthesis), abbreviated as TOMO, is an important medical imaging technique. It can obtain a series of 3D slice images through the reconstruction of a set of 2D projection images, and thus it can contribute to the efficiency and accuracy of diagnosis. TOMO is commonly used in mammograms (mammography) and DR (radiography) systems.

However, unlike CT and CBCT, the range of scannable angles for TOMO is narrow, possibly between 20 ° and 90 ° depending on the system design. The restricted angle may cause artifacts, such as tomographic artifacts; the limited angle also results in relatively poor resolution in the Z-direction (perpendicular to the detector surface). Furthermore, while CT and CBCT can provide better quality 3D images, they are demanding in terms of field, implementation conditions, which limits their applications, for example, some critical patients who are not suitable for metastasis are not suitable for CT or CBCT.

Disclosure of Invention

The present invention aims to provide a mechanism that can generate higher quality medical images, in particular:

according to an aspect of the present invention, there is provided a method of training a neural network, comprising the steps of: acquiring a reference medical image about a subject as a group Truth data set; acquiring a 2D projection image with respect to the object under examination; and forming an input of the neural network from the 2D projection image, and training the neural network with the group Truth data set as a target.

Optionally, according to some embodiments of the invention, the step of forming the input of the neural network from the 2D projection images comprises: taking the 2D projection image as an input of the neural network.

Optionally, according to some embodiments of the invention, the step of forming the input of the neural network from the 2D projection images comprises: reconstructing based on the 2D projection images to obtain a 3D TOMO image and taking the 3D TOMO image and/or the 2D projection images as input to the neural network.

According to some embodiments of the invention, optionally, the reference medical image comprises one or more of a CT reconstructed image, a CBCT reconstructed image, a MR reconstructed image, an anatomical image of the object under examination.

According to some embodiments of the invention, optionally, the subject is a biological tissue.

According to some embodiments of the invention, optionally, the object under examination is a pre-fabricated biomimetic tissue.

According to some embodiments of the invention, optionally, the step of acquiring a 2D projection image of the object under examination comprises: and acquiring the 2D projection image by adopting a forward projection algorithm.

Optionally, according to some embodiments of the present invention, the step of obtaining the 3D TOMO image includes: reconstructing by an analytical reconstruction algorithm or an iterative reconstruction algorithm to obtain the 3D TOMO image.

Optionally, according to some embodiments of the invention, the neural network is a U-net or a dual-domain network (dual-domain network).

According to another aspect of the present invention there is provided a neural network trained by a method of training a neural network as any one of the methods described above.

According to another aspect of the present invention, there is provided a TOMO imaging method including the steps of: training a neural network using any one of the methods of training a neural network described above; acquiring a 2D projection image with respect to an object under examination; forming an input of the neural network according to the 2D projection image, and inputting the input into the neural network; and outputting the neural network as a 3D TOMO image about the subject.

According to another aspect of the present invention, there is provided a TOMO imaging method including the steps of: acquiring a 2D projection image with respect to an object under examination; forming an input of a neural network from the 2D projection image and inputting to any one of the neural networks as described above; and outputting the neural network as a 3D TOMO image about the subject.

According to another aspect of the present invention, there is provided a computer readable storage medium having stored therein instructions which, when executed by a processor, cause the processor to perform any of the methods described above.

According to another aspect of the present invention, there is provided a computer readable storage medium having stored therein instructions which, when executed by a processor, cause the processor to perform any of the methods described above.

According to another aspect of the present invention, there is provided a medical image processing apparatus, the apparatus comprising: an input module configured to receive a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image; a processing module comprising a neural network as any one of above, and configured to process the input data with the neural network to generate a processed 3D TOMO image; and an output module configured to output the processed 3D TOMO image.

According to another aspect of the present invention, there is provided a medical image processing apparatus, the apparatus comprising: an input module configured to receive a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image; any one of the computer-readable storage media described above; a processing module comprising a neural network and configured to execute instructions stored in the computer-readable storage medium to train the neural network, and further configured to process the input data with the trained neural network to generate a processed 3D TOMO image; and an output module configured to output the processed 3D TOMO image.

According to another aspect of the present invention, there is provided a medical examination apparatus, the apparatus comprising: an imaging module configured to acquire a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image; a processing module comprising a neural network as any one of above, and configured to process the input data with the neural network to generate a processed 3D TOMO image about the subject; and an output module configured to output the processed 3D TOMO image.

According to some embodiments of the invention, optionally, the input data is the 2D projection image.

According to some embodiments of the invention, optionally, the imaging module comprises: a first module configured to acquire a 2D projection image with respect to an object under examination; and a second module configured to reconstruct by an analytical reconstruction algorithm or an iterative reconstruction algorithm based on the 2D projection images to obtain an original 3D TOMO image with respect to the object under examination, the original 3D TOMO image being the input data.

According to another aspect of the present invention, there is provided a medical examination apparatus, the apparatus comprising: an imaging module configured to acquire a 2D projection image of an object under examination and to form input data for a neural network from the 2D projection image; any one of the computer-readable storage media described above; a processing module comprising a neural network and configured to execute instructions stored in the computer-readable storage medium to train the neural network, and further configured to process the input data with the trained neural network to generate a processed 3D TOMO image; and an output module configured to output the processed 3D TOMO image.

According to some embodiments of the invention, optionally, the input data is the 2D projection image.

According to some embodiments of the invention, optionally, the imaging module comprises: a first module configured to acquire a 2D projection image with respect to an object under examination; and a second module configured to reconstruct by FDK algorithm or SART algorithm based on the 2D projection image to obtain an original 3D TOMO image with respect to the object under examination, the original 3D TOMO image being the input data.

Drawings

The above and other objects and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which like or similar elements are designated by like reference numerals.

FIG. 1 illustrates a method of training a neural network according to one embodiment of the present invention.

Fig. 2 illustrates a TOMO imaging method according to an embodiment of the present invention.

Fig. 3 illustrates a TOMO imaging method according to an embodiment of the present invention.

Fig. 4 shows a medical image processing device according to an embodiment of the invention.

Fig. 5 shows a medical image processing device according to an embodiment of the invention.

Fig. 6 shows a medical examination apparatus according to an embodiment of the invention.

Fig. 7 shows a medical examination apparatus according to an embodiment of the invention.

Detailed Description

For the purposes of brevity and explanation, the principles of the present invention are described herein with reference primarily to exemplary embodiments thereof. However, those skilled in the art will readily recognize that the same principles are equally applicable to all types of methods of training neural networks, imaging methods, storage media, medical image processing devices, and medical detection devices, and that these same or similar principles may be implemented therein, with any such variations not departing from the true spirit and scope of the present patent application.

According to an aspect of the present invention, a method of training a neural network is provided. FIG. 1 illustrates a method of training a neural network, according to one embodiment of the present invention, as shown, the method includes the steps of: acquiring a reference medical image about a subject as a group Truth data set in step 102; acquiring a 2D projection image of the object under examination in step 104; forming inputs to the neural network from the 2D projection images in step 106; and inputting it as an input to the neural network in step 108, and training the neural network with the group Truth dataset as a target.

In step 102 of the method, a reference medical image is acquired about the subject as a group Truth dataset. The subject herein may be any subject to which the TOMO test is applied, and may be a muscle, a blood vessel, a bone, a joint, or the like, or may be other non-organism. Some examples of the invention do not limit the type of subject, and one of the criteria for selecting a subject is whether a neural network can be trained efficiently or to some extent efficiently. On the other hand, a trained neural network may ultimately be used for, for example, human tissue, and it may be better to select objects that are associated with the ultimate target of action, where "associated" means that they have the same or similar properties in the imaging.

The reference medical image described in some examples of the present invention refers to a medical image that can be referred to as a standard, which is evaluated by a professional, or which is obtained by an imaging means that is superior to TOMO imaging in experience. The quality of TOMO imaging can be improved by training the neural network by taking the data with better imaging effect as the group Truth data set. In some examples, the reference medical image is derived from the same modality, in other examples, the reference medical image may be derived from multiple modalities. If the homologous reference medical image is adopted, the scale of the group Truth data set is possibly small, so that the physical requirements on software, hardware and the like for training the neural network are reduced, and the method is beneficial to real-time training of the neural network and real-time application of the trained neural network. While reference medical images using multiple sources may train neural networks based on the properties of the different sources, better training may be expected in some cases.

In step 104 of the method, a 2D projection image is acquired with respect to the object under examination. Acquiring a 2D projection image (projection image) is a key step in the TOMO imaging, and step 104 of the method may be developed in a manner similar to the method of acquiring a 2D projection image in the existing TOMO imaging step. The acquired 2D projection images are typically represented as limited-angle sinograms (limited-angle sinograms), and the acquired 2D projection images are associated with scannable angles of the TOMO imaging device.

In step 106 of the method, the inputs of the neural network are formed from the 2D projection images. The present invention does not limit the form of the generated input data, and it is sufficient that the input data can participate in neural network training, and the trained neural network can be used for subsequent image generation. In some examples, the input data types involved in the training correspond to the input data types of the neural network in actual use, or belong to the same category, and are not described in detail below. For example, in one example of the invention, the 2D projection image may be used as an input to a neural network. In other examples of the present invention, a reconstruction may be performed based on the 2D projection images to obtain 3D TOMO images, and the 3D TOMO images and/or the 2D projection images may be used as inputs to a neural network. Obtaining a 3D TOMO image is also an important step of TOMO imaging, and according to the above example, step 106 of the method can be developed to follow the method of obtaining a 3D TOMO image in the existing TOMO imaging step. The 3D TOMO images obtained at this time are final finished products in the existing method, and these images have the defects described in the background art. The 3D TOMO image here is present as an intermediate material in the method of the present invention and will be used for the training of neural networks.

In step 108 of the method, the input data generated above (e.g., 2D projection image or 3D TOMO image, which will not be described below) is used as the input of the neural network, and the neural network is trained with the group route data set as the target. Here, we select input data and target output data for the neural network, and training of the neural network can be performed according to the existing means. The invention is not limited to the organization form of the neural network, and those skilled in the art can configure the form of the neural network according to actual needs after reading the invention. Under the condition of reasonable training data and reasonable neural network form, the neural network trained by the method has better generalization capability.

According to some embodiments of the invention, the reference medical image comprises one or more of a CT reconstructed image, a CBCT reconstructed image, an MR reconstructed image, an anatomical image of the object under examination. CT reconstructed images, CBCT reconstructed images, and MR reconstructed images are generally of better quality than TOMO imaging, and thus these reconstructed images can be used as criteria for training neural networks. If the subject is laminar, the subject can also be laminar (e.g., in microns or millimeters), and the laminar layers can be imaged separately and fitted (e.g., by interpolation) to an anatomical image. The anatomical image in the present invention has 3D visibility and can be tomographic again, as in the CT reconstructed image, CBCT reconstructed image, MR reconstructed image, and TOMO reconstructed image, and the "tomographic" layers herein do not necessarily correspond to the "tomographic" layers one-to-one. The anatomical image may more accurately reflect the structure of the object as it is imaged by direct slice.

According to some embodiments of the invention, the subject is a biological tissue. The biological tissue is taken as the object to participate in the neural network training, so that the application occasion of the neural network is better considered, and the generated network has better generalization capability. The biological tissue herein refers to a tissue formed by natural growth. For example, human tissue can be used as the subject. More specifically, breast tissue may be used, for example, as the subject. It should be noted that if the tissue as the subject is a component of the subject, the tissue as the subject does not necessarily need to be separated from the subject to participate in the neural network training.

According to some embodiments of the invention, the object under examination is a pre-fabricated biomimetic tissue. Because of the individual difference in the actual organization, sometimes a standard organization can be prepared to show the commonality of the individuals. For example, the pre-formed thoracic tissue may exhibit ethnic differences, ethnic group differences, age differences (e.g., may be a biomimetic thoracic tissue suitable for asian race pre-forms), such that the neural network of the training process may be more targeted. In some examples, the biomimetic tissue may also be pre-fabricated according to where the tumor is most likely to appear in the thoracic tissue, e.g., the biomimetic tumor may be "placed" or the actual resected tumor may be "placed" at such a location in the biomimetic thoracic tissue. The material of the biomimetic tissue may be a material similar to human tissue in TOMO imaging, e.g., the material of the biomimetic tissue may have an X-ray transparency similar to human tissue. On the other hand, the morphology of the pre-fabricated biomimetic tissue is predetermined, so that the reference medical image can be generated directly without the aid of external means. For example, if the pre-fabricated biomimetic tissue is fabricated by 3D printing, the reference medical image can be directly generated from the engineering drawings.

According to some embodiments of the invention, the step of acquiring a 2D projection image in respect of the object under examination comprises: the 2D projection images are acquired using a forward projection algorithm (forward projection algorithm). In this way, an enhanced 2D image may be obtained.

According to some embodiments of the present invention, the step of obtaining the 3D TOMO image includes: reconstructed by an analytical reconstruction algorithm (e.g., FDK algorithm) or an iterative reconstruction algorithm (e.g., SART algorithm) to obtain a 3D TOMO image. The FDK algorithm is a common 3D reconstruction algorithm, cone beam rays are stacked as fan beam rays with different inclination angles along the direction of a Z axis (perpendicular to a detector), data reconstruction on a central plane belongs to fan beam scanning accurate reconstruction, and for reconstruction of a non-central plane, an approximate reconstruction formula is obtained by correcting a fan beam reconstruction formula. In addition, the SART algorithm is based on the ART algorithm, and unlike the ART algorithm, the SART algorithm is not an algorithm that is updated according to each ray, but an algorithm that is updated simultaneously according to all rays at each projection angle.

According to some embodiments of the invention, the neural network is a U-net or a dual-domain network (e.g., DuDoNet). The U-net has the following advantages: a small amount of data training models are supported, and the segmentation accuracy rate is high and the speed is high. The entire network of U-net has 19 convolution operations, 4 pooling operations, 4 upsampling operations, 4 cropping and copying operations, the generation of which facilitates the study of medical image segmentation. DuDoNet is also a network proposed shortly before that can be used for processing medical images, which has a better artifact-removing effect.

According to another aspect of the present invention, there is provided a neural network trained by a method of training a neural network as any one of the above. The neural network trained via the above training method may be used for processing medical images, more specifically, the neural network herein may be a U-net or a dual domain network such as mentioned above.

According to another aspect of the present invention, there is provided a TOMO imaging method, as shown in fig. 2, including the steps of: training a neural network in step 202 using a method of training a neural network as any one of the above; acquiring a 2D projection image with respect to the object under examination in step 204; forming inputs to a neural network from the 2D projection images in step 206; inputting the input data generated above to a neural network in step 208; and outputting the neural network as a 3D TOMO image with respect to the subject in step 210. In some examples of the invention, the input to form the neural network from the 2D projection images is the 2D projection images themselves in step 206 and input to the neural network in step 208. In other examples of the present invention, a reconstruction may be further performed based on the 2D projection image in step 206 to obtain a process 3D TOMO image, and the process 3D TOMO image is input to the neural network in step 208. The type of input data may correspond to neural network data from when the neural network was trained. In the TOMO imaging method, a neural network to be used by the method is trained, and then the neural network is put into a subsequent processing step. According to this method, it is possible to further process, specifically, to input into the neural network trained in step 202, on the basis of the process 2D projection image or the 3D TOMO image obtained according to the conventional method, so that the quality of the process 3D TOMO image can be improved. The method is suitable for imaging mechanisms which need to train/update the neural network on site.

According to another aspect of the present invention, there is provided a TOMO imaging method, as shown in fig. 2, including the steps of: acquiring a 2D projection image of the object under examination in step 302; forming inputs to a neural network from the 2D projection images in step 304; the input data generated above is input to a neural network (trained) as above in step 306; and outputting the neural network as a 3D TOMO image about the subject in step 308. In some examples of the invention, the input to form the neural network from the 2D projection images is the 2D projection images themselves in step 304 and input to the neural network in step 306. In other examples of the present invention, a reconstruction may be further performed based on the 2D projection image in step 304 to obtain a process 3D TOMO image, and the process 3D TOMO image is input to a neural network in step 306. The type of input data may correspond to neural network data from when the neural network was trained. In the TOMO imaging method, compared to the previous embodiment, it is not necessary to train a neural network to be used in the method. According to this method, it is possible to further process, specifically to invest in a neural network (e.g., U-net or dual domain network) as above, on the basis of the process 2D projection image or 3D TOMO image obtained according to the conventional method, so that the quality of the process 3D TOMO image can be improved. The method is applicable to imaging mechanisms that do not require on-site training/updating of neural networks.

According to another aspect of the present invention, there is provided a computer readable storage medium having stored therein instructions which, when executed by a processor, cause the processor to perform any one of the methods of training a neural network as described above. Furthermore, according to another aspect of the present invention, there is also provided a computer-readable storage medium having stored therein instructions, which when executed by a processor, cause the processor to perform any of the TOMO imaging methods as described above. Computer-readable media, as referred to herein, includes all types of computer storage media, which can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, computer-readable media may include RAM, ROM, EPROM, E²PROM, registers, hard disk, removable disk, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other temporary or non-temporary medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by a general purpose or special purpose computer, or a general purpose or special purpose processor. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks are typically magneticData is copied optically, while discs use lasers to copy data optically. Combinations of the above should also be included within the scope of computer-readable media. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.

According to another aspect of the present invention, a medical image processing apparatus is provided, as shown in fig. 4, the medical image processing apparatus 40 comprises an input module 402, a processing module 404 and an output module 406. Wherein the input module 402 is configured to receive 2D projection images regarding the object under examination and form input data of a neural network from the 2D projection images (e.g. the input data may be the 2D projection images themselves, or may be processed to obtain a 3D TOMO image and serve it as an input to the neural network), the 2D projection images may be generated with existing X-ray imaging devices. The processing module 404 includes a neural network (e.g., a U-net or a dual-domain network) as described above (trained), and the processing module 404 is configured to process the input data using the neural network (e.g., the U-net or the dual-domain network) to generate processed 3D TOMO images that eliminate, at least to some extent, some of the drawbacks of the original 3D TOMO images described in the background. The output module 406 is configured to output the processed 3D TOMO image. Medical image processing devices according to some examples of the invention may work in conjunction with existing TOMO imaging devices, and may be used to upgrade existing TOMO imaging devices without requiring modification of the existing TOMO imaging devices; furthermore, medical image processing devices according to some examples of the invention are suitable for imaging mechanisms that do not require on-site training/updating of neural networks.

According to another aspect of the present invention, a medical image processing device is provided, as shown in fig. 5, the medical image processing device 50 comprises an input module 502, a computer readable storage medium 504 as above, a processing module 506 and an output module 508. Wherein the input module 502 is configured to receive 2D projection images regarding the object under examination and to form input data of a neural network from the 2D projection images (e.g. the input data may be the 2D projection images themselves or may be processed to obtain a 3D TOMO image and serve as input to the neural network), the 2D projection images may be generated with existing X-ray imaging devices. A neural network (e.g., U-net or a dual-domain network) is included in processing module 506, and processing module 506 is configured to execute instructions stored in a computer-readable storage medium to train the neural network, and is further configured to process input data using the trained neural network (e.g., U-net or a dual-domain network) to generate processed 3D TOMO images that eliminate, at least to some extent, some of the drawbacks of the original 3D TOMO images introduced in the background. The output module 508 is configured to output the processed 3D TOMO image. Medical image processing devices according to some examples of the invention may work in conjunction with existing TOMO imaging devices, and may be used to upgrade existing TOMO imaging devices without requiring modification of the existing TOMO imaging devices; furthermore, medical image processing according to some examples of the invention is applicable to imaging mechanisms that require training/updating of neural networks in the field.

According to another aspect of the present invention, a medical examination apparatus is provided, as shown in fig. 6, the medical image processing apparatus 60 comprises an imaging module 602, a processing module 604 and an output module 606. Wherein the imaging module 602 is configured to acquire 2D projection images with respect to the object under examination and to form input data for the neural network from the 2D projection images. Processing module 604 comprises a neural network (e.g., a U-net or a dual-domain network) as described above (trained), and processing module 604 is configured to process the input data using the neural network (e.g., a U-net or a dual-domain network) to generate processed 3D TOMO images about the subject that eliminate, at least to some extent, some of the drawbacks of the original 3D TOMO images introduced in the background. The output module 606 is configured to output the processed 3D TOMO image. Medical examination devices according to some examples of the invention are a completely new upgrade to existing TOMO imaging devices; furthermore, medical examination apparatus according to some examples of the invention are suitable for imaging modalities that do not require on-site training/updating of neural networks.

According to some embodiments of the invention, the input data may be the 2D projection image itself.

According to some embodiments of the invention, imaging module 602 comprises a first module and a second module. Wherein the first module is configured to acquire a 2D projection image with respect to an object under examination; the second module is configured to reconstruct based on the 2D projection images by an analytical reconstruction algorithm (e.g., FDK algorithm) or an iterative reconstruction algorithm (e.g., SART algorithm) to obtain original 3D TOMO images with respect to the object under examination, the original 3D TOMO images being the input data.

According to another aspect of the present invention, a medical examination apparatus is provided, as shown in fig. 7, the medical image processing apparatus 70 comprising an imaging module 702, a computer readable storage medium 704 as above, a processing module 706 and an output module 708. Wherein the imaging module 702 is configured to acquire 2D projection images with respect to the object under examination and to form input data for the neural network from the 2D projection images. Processing module 706 includes a neural network (e.g., a U-net or a dual-domain network), and processing module 706 is configured to execute instructions stored in a computer-readable storage medium to train the neural network, and is further configured to process input data with the trained neural network (e.g., a U-net or a dual-domain network) to generate processed 3D TOMO images that eliminate, at least to some extent, some of the drawbacks of the original 3D TOMO images introduced in the background. The output module 708 is configured to output the processed 3D TOMO image. Medical examination devices according to some examples of the invention are a completely new upgrade to existing TOMO imaging devices; furthermore, medical image processing according to some examples of the invention is applicable to imaging mechanisms that require training/updating of neural networks in the field.

According to some embodiments of the invention, the input data may be the 2D projection image itself.

According to some embodiments of the invention, the imaging module 702 comprises a first module and a second module. Wherein the first module is configured to acquire a 2D projection image of the object under examination by using a forward projection algorithm; and a second module configured to reconstruct, based on the 2D projection images, by an analytical reconstruction algorithm (e.g., FDK algorithm) or an iterative reconstruction algorithm (e.g., SART algorithm) to obtain original 3D TOMO images with respect to the object under examination, the original 3D TOMO images being the input data.

In summary, the neural network training method, the neural network, the imaging method, the storage medium, the medical image processing device and the medical detection device of the present application provide a mechanism for performing AI processing on a TOMO image generated by an existing means, which can effectively improve the quality of TOMO imaging. It should be noted that some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The above examples mainly illustrate the method of training a neural network, the imaging method, the storage medium, the medical image processing apparatus, and the medical examination apparatus of the present invention. Although only a few embodiments of the present invention have been described, those skilled in the art will appreciate that the present invention may be embodied in many other forms without departing from the spirit or scope thereof. Accordingly, the present examples and embodiments are to be considered as illustrative and not restrictive, and various modifications and substitutions may be made therein without departing from the spirit and scope of the present invention as defined by the appended claims.