阅读说明：本技术 一种三维图像的重建方法、装置、设备及存储介质 (Three-dimensional image reconstruction method, device, equipment and storage medium ) 是由 王旭 黄宇翔 于 2020-04-26 设计创作，主要内容包括：本发明实施例公开了一种三维图像的重建方法、装置、设备及存储介质,该方法包括：获取待重建定位像；提取所述待重建定位像中目标区域的待重建目标图像；基于预先训练的网络模型,生成所述待重建目标图像的三维重建图像,其中,所述网络模型包括下采样模块、维度扩展模块和上采样模块,所述维度扩展模块将所述下采样模块和上采样模块进行跨层连接,所述维度扩展模块用于将下采样模块输出的各个通道的二维特征图重组为三维特征图,并将所述三维特征图发送至所述上采样模块。本发明实施例的技术方案,通过预先训练的网络模型,对定位像进行三维重建,提高了重建的精度。(The embodiment of the invention discloses a method, a device, equipment and a storage medium for reconstructing a three-dimensional image, wherein the method comprises the following steps: acquiring a positioning image to be reconstructed; extracting a target image to be reconstructed in a target area in the positioning image to be reconstructed; the method comprises the steps of generating a three-dimensional reconstruction image of a target image to be reconstructed based on a pre-trained network model, wherein the network model comprises a down-sampling module, a dimensionality extension module and an up-sampling module, the down-sampling module and the up-sampling module are connected in a cross-layer mode through the dimensionality extension module, and the dimensionality extension module is used for recombining two-dimensional characteristic graphs of all channels output by the down-sampling module into a three-dimensional characteristic graph and sending the three-dimensional characteristic graph to the up-sampling module. According to the technical scheme of the embodiment of the invention, the positioning image is subjected to three-dimensional reconstruction through the pre-trained network model, so that the reconstruction precision is improved.)

1. A method of reconstructing a three-dimensional image, comprising:

acquiring a positioning image to be reconstructed;

extracting a target image to be reconstructed in a target area in the positioning image to be reconstructed;

the method comprises the steps of generating a three-dimensional reconstruction image of a target image to be reconstructed based on a pre-trained network model, wherein the network model comprises a down-sampling module, a dimensionality extension module and an up-sampling module, the down-sampling module and the up-sampling module are connected in a cross-layer mode through the dimensionality extension module, and the dimensionality extension module is used for recombining two-dimensional characteristic graphs of all channels output by the down-sampling module into a three-dimensional characteristic graph and sending the three-dimensional characteristic graph to the up-sampling module.

2. The method according to claim 1, wherein the network model comprises a lateral network model and a positive network model, wherein the scout image to be reconstructed corresponding to the lateral network model is a lateral scout image, and the scout image to be reconstructed corresponding to the positive network model is a positive scout image.

3. The method of claim 1, further comprising, after acquiring the scout image to be reconstructed:

and identifying the type of the positioning image to be reconstructed, wherein the positioning image to be reconstructed comprises a lateral positioning image or a normal positioning image.

4. The method of claim 3, wherein the identifying the type of scout image to be reconstructed comprises:

determining the type of the positioning image to be reconstructed based on a pre-trained neural network model, or,

identifying the contour information of the to-be-reconstructed positioned image, and determining the type of the to-be-reconstructed positioned image according to the contour information, or,

and acquiring the positioning information of the acquisition equipment of the positioning image to be reconstructed, and determining the type of the positioning image to be reconstructed according to the positioning information.

5. The method of claim 1, wherein the generation of the gold standard for the three-dimensional reconstructed image is:

acquiring a three-dimensional sample image and a corresponding two-dimensional sample image;

determining the center of a target area in the two-dimensional sample image;

determining a three-dimensional center of a gold standard corresponding to the two-dimensional sample image according to the center of a target area in the two-dimensional sample image and the size of the target area;

and intercepting a target area of the three-dimensional sample image based on the three-dimensional center of the gold standard to generate the gold standard of the three-dimensional reconstruction image of the two-dimensional sample image.

6. The method according to claim 1, wherein the extracting the target image to be reconstructed of the target region in the scout image to be reconstructed comprises:

determining the size of a positioning frame corresponding to the target image to be reconstructed and a preset identification algorithm according to the attribute of the target area;

determining the initial position of a positioning frame corresponding to the target image to be reconstructed based on the preset identification algorithm;

and extracting a target image to be reconstructed of the target area in the positioning image to be reconstructed based on the size and the initial position of the positioning frame.

7. The method of claim 1, wherein the downsampling module comprises a first convolution unit and a first residual unit, and wherein the upsampling module comprises a transposed convolution unit and a second residual unit.

8. The method of claim 7, wherein the first convolution unit includes a two-dimensional convolution layer, a batch normalization layer, and an activation layer; the first residual error unit is a two-dimensional residual error module; the transposition convolution unit comprises a three-dimensional transposition convolution layer, a batch normalization layer and an activation layer; the second residual error unit is a three-dimensional residual error module.

9. The method of claim 1, further comprising:

judging whether the three-dimensional reconstruction image is qualified or not according to a discrimination model;

and if so, outputting the three-dimensional reconstruction image.

10. An apparatus for reconstructing a three-dimensional image, comprising:

the positioning image acquisition module is used for acquiring a positioning image to be reconstructed;

the to-be-reconstructed image extraction module is used for extracting a to-be-reconstructed target image of a target area in the to-be-reconstructed positioned image;

the image three-dimensional reconstruction module is used for generating a three-dimensional reconstruction image of the target image to be reconstructed based on a pre-trained network model, wherein the network model comprises a down-sampling module, a dimension expansion module and an up-sampling module, the down-sampling module and the up-sampling module are connected in a cross-layer mode through the dimension expansion module, and the dimension expansion module is used for recombining two-dimensional feature maps of all channels output by the down-sampling module into a three-dimensional feature map and sending the three-dimensional feature map to the up-sampling module.

11. An apparatus for reconstructing a three-dimensional image, the apparatus comprising:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement a method of reconstructing a three-dimensional image as recited in any one of claims 1-9.

12. A storage medium containing computer-executable instructions for performing a method of reconstructing a three-dimensional image according to any one of claims 1-9 when executed by a computer processor.

Technical Field

The embodiment of the invention relates to the technical field of CT image reconstruction, in particular to a method, a device, equipment and a storage medium for reconstructing a three-dimensional image.

Background

Nowadays, CT (Computed Tomography) is widely used for medical diagnosis due to its advantages such as high image resolution and high scanning speed. After the CT scout image is acquired, image reconstruction of the CT scout image, i.e. rough estimation of the size and extent of the human body according to the change of the signal, is often required for dose modulation.

The existing image reconstruction method mainly determines the three-dimensional state of a target part by estimating the width of the part and according to a preset regular shape corresponding to the target part, such as a sphere, an ellipsoid and the like. In the prior art, a preset regular shape is adopted for image reconstruction, the reconstruction precision is low, and the use premise is that the density distribution of a target part is assumed to be approximately uniform, the reconstruction cannot be carried out on the part with larger density difference, and the application range is limited.

Disclosure of Invention

The invention provides a method, a device, equipment and a storage medium for reconstructing a three-dimensional image, which are used for improving the accuracy, efficiency and applicability of image reconstruction.

In a first aspect, an embodiment of the present invention provides a method for reconstructing a three-dimensional image, where the method includes:

acquiring a positioning image to be reconstructed;

extracting a target image to be reconstructed in a target area in the positioning image to be reconstructed;

the method comprises the steps of generating a three-dimensional reconstruction image of a target image to be reconstructed based on a pre-trained network model, wherein the network model comprises a down-sampling module, a dimensionality extension module and an up-sampling module, the dimensionality extension module is used for connecting the corresponding down-sampling module and the corresponding up-sampling module in a cross-layer mode, the dimensionality extension module comprises a channel recombination unit and a second convolution unit, and the channel recombination unit is used for recombining two-dimensional feature maps of all channels output by the down-sampling module into a three-dimensional feature map.

In a second aspect, an embodiment of the present invention further provides an apparatus for reconstructing a three-dimensional image, where the apparatus includes:

the positioning image acquisition module is used for acquiring a positioning image to be reconstructed;

the to-be-reconstructed image extraction module is used for extracting a to-be-reconstructed target image of a target area in the to-be-reconstructed positioned image;

the image three-dimensional reconstruction module is used for generating a three-dimensional reconstruction image of the target image to be reconstructed based on a pre-trained network model, wherein the network model comprises a down-sampling module, a dimension expansion module and an up-sampling module, the down-sampling module and the up-sampling module are connected in a cross-layer mode through the dimension expansion module, and the dimension expansion module is used for recombining two-dimensional feature maps of all channels output by the down-sampling module into a three-dimensional feature map and sending the three-dimensional feature map to the up-sampling module.

In a third aspect, an embodiment of the present invention further provides a three-dimensional image reconstruction apparatus, where the apparatus includes:

one or more processors;

a memory for storing one or more programs;

when the one or more programs are executed by the one or more processors, the one or more processors implement the method for reconstructing a three-dimensional image provided by any of the embodiments of the present invention.

In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the method for reconstructing a three-dimensional image provided in any of the embodiments of the present invention.

According to the technical scheme of the embodiment of the invention, the target area extraction is carried out on the positioning image to be reconstructed, so that the data volume to be processed is reduced, and the reconstruction efficiency is improved; the positioning image is subjected to three-dimensional image reconstruction through the pre-trained network model, so that the three-dimensional reconstruction precision is improved, the imaging part of the positioning image is not limited, and the application range of the three-dimensional reconstruction is improved; and the network model is provided with a dimension extension module and is connected in a cross-layer connection mode, so that the network training process is accelerated, and the reconstruction precision is further improved. The reconstruction method of the three-dimensional image provided by the embodiment of the invention has the advantages of wide reconstruction application range and high precision.

Drawings

Fig. 1 is a flowchart of a method for reconstructing a three-dimensional image according to a first embodiment of the present invention;

FIG. 2A is a flowchart of a method for reconstructing a three-dimensional image according to a second embodiment of the present invention;

FIG. 2B is a schematic diagram of a network model according to a second embodiment of the present invention;

fig. 3 is a schematic structural diagram of a three-dimensional image reconstruction apparatus according to a third embodiment of the present invention;

fig. 4 is a schematic structural diagram of a three-dimensional image reconstruction apparatus according to a fourth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Example one

Fig. 1 is a flowchart of a three-dimensional image reconstruction method according to an embodiment of the present invention, where the present embodiment is applicable to a case of performing three-dimensional reconstruction on a CT scan scout image, and the method may be executed by a three-dimensional image reconstruction apparatus, as shown in fig. 1, the method specifically includes the following steps:

and step 110, obtaining a positioning image to be reconstructed.

The scout image to be reconstructed is a two-dimensional scout image of CT scan, and may be a normal scout image or a lateral scout image, such as a two-dimensional scout image of the head, the chest or the neck.

And 120, extracting a target image to be reconstructed in the target area in the positioning image to be reconstructed.

The target region may be a region including a set portion, such as an eye, a breast, a thyroid, or the like. The target area may be an area set by the user. The target image to be reconstructed refers to an image including a target region.

Specifically, the step is to segment or position the image to be reconstructed and positioned to obtain the target image to be reconstructed including the target region. The target image to be reconstructed can be obtained by performing image segmentation on the positioned image to be reconstructed according to the gray value distribution of the positioned image to be reconstructed and the attribute of the target area.

Further, the corresponding relationship between the type of the target region and the image segmentation algorithm may be pre-established, the type of the target region is determined first, the image segmentation algorithm is determined according to the type of the target region and the corresponding relationship, and the target image to be reconstructed of the target region in the to-be-reconstructed positioned image is extracted according to the image segmentation algorithm.

Optionally, after the target image to be reconstructed is extracted, the target image to be reconstructed may be resampled.

And step 130, generating a three-dimensional reconstruction image of the target image to be reconstructed based on a pre-trained network model.

The network model comprises a down-sampling module, a dimensionality extension module and an up-sampling module, wherein the dimensionality extension module connects the down-sampling module and the up-sampling module in a cross-layer mode, the dimensionality extension module is used for recombining two-dimensional feature maps of all channels output by the down-sampling module into a three-dimensional feature map and sending the three-dimensional feature map to the up-sampling module, and the up-sampling module receives the three-dimensional features and outputs a three-dimensional reconstruction image.

The number of the down-sampling module, the dimension expansion module and the up-sampling module may be multiple, such as 3, 5 or other values. The down-sampling module mainly comprises a convolution layer and an activation function so as to carry out convolution operation on an input target image to be reconstructed. The up-sampling module mainly comprises a transposition convolution layer (a deconvolution layer) and an activation function, and is used for performing transposition convolution operation on the data output by the dimension expansion module and finally outputting a three-dimensional reconstruction image. The activation function may be a Sigmoid function, a Tanh function, a ReLU (Rectified Linear Unit) function, a Swish function, or other functions. The connection mode of the dimension expansion module, the up-sampling module and the down-sampling module is skip connection (cross-layer connection) so as to increase the dimension of the feature diagram and accelerate the convergence speed.

Specifically, the network model has a structure similar to U-Net, and the training process may be:

acquiring two-dimensional sample images including each target area required by training from each historical CT scanning database, wherein the two-dimensional sample images are usually complete positioning images (normal positions or lateral positions), and target area extraction needs to be performed on samples, namely, the operation of step 120 is performed, and the original two-dimensional sample images are subjected to image segmentation to obtain two-dimensional sample images including the target areas so as to form a training set required by network model training; meanwhile, the gold standard of the three-dimensional reconstruction image corresponding to the two-dimensional sample image can be obtained manually or by a preset algorithm to be used as a verification set output by the model. Firstly, initializing each parameter of a network model, inputting data of a training set into the network model for iterative computation, obtaining a loss value through a loss function, and updating each parameter in the network model through back propagation until the loss value of the loss function meets a preset condition. And verifying the trained network model through a verification set, and when the error is smaller than a set value, indicating that the network training is successful.

The loss function may be any existing loss function, such as a Mean Absolute Error (MAE) function, also referred to as an L1 loss function, a Mean Square Error (MSE) function, also referred to as an L2 loss function, a Mean Bias Error (MBE) function, or other loss functions or custom loss functions.

Optionally, the network model includes a lateral network model and a positive network model, where the to-be-reconstructed positioning image corresponding to the lateral network model is a lateral positioning image, and the to-be-reconstructed positioning image corresponding to the positive network model is a positive positioning image.

For different types of positioning images, different network models are adopted for three-dimensional reconstruction, and the reconstruction precision is improved.

According to the technical scheme of the embodiment of the invention, the target area extraction is carried out on the positioning image to be reconstructed, so that the data volume to be processed is reduced, and the reconstruction efficiency is improved; the positioning image is subjected to three-dimensional image reconstruction through the pre-trained network model, so that the three-dimensional reconstruction precision is improved, the imaging part of the positioning image is not limited, and the application range of the three-dimensional reconstruction is improved; and the network model is provided with a dimension extension module and is connected in a cross-layer connection mode, so that the network training process is accelerated, and the reconstruction precision is further improved. The reconstruction method of the three-dimensional image provided by the embodiment of the invention has the advantages of wide reconstruction application range and high precision.

Example two

Fig. 2A is a flowchart of a three-dimensional image reconstruction method according to a second embodiment of the present invention, which is a further refinement and supplement to the first embodiment, and the three-dimensional image reconstruction method according to the present embodiment further includes: and identifying the type of the positioning image to be reconstructed, wherein the positioning image to be reconstructed comprises a lateral positioning image and a normal positioning image.

As shown in fig. 2A, the method for reconstructing a three-dimensional image includes the following steps:

step 210, obtaining a positioning image to be reconstructed;

and step 220, identifying the type of the positioning image to be reconstructed, wherein the positioning image to be reconstructed comprises a lateral positioning image or a normal positioning image.

Specifically, the identification of the type of the to-be-reconstructed positioned image refers to identifying whether the to-be-reconstructed positioned image is a lateral positioned image or a normal positioned image.

Optionally, the identifying the type of the scout image to be reconstructed includes:

determining the type of the to-be-reconstructed positioning image based on a pre-trained neural network model, or identifying contour information of the to-be-reconstructed positioning image, and determining the type of the to-be-reconstructed positioning image according to the contour information, or acquiring positioning information of acquisition equipment of the to-be-reconstructed positioning image, and determining the type of the to-be-reconstructed positioning image according to the positioning information.

Specifically, the neural network model is mainly a network model for performing positive or lateral binary classification, and may be a vgg (visual Geometry group) convolutional neural network model, a ResNet classification network, or another binary neural network model. The contour information refers to the shape and size information of the outer contour of the target area, for example, the contour information of the head can be an ellipse with the circumference of 20-60 cm. The acquisition equipment can be CT imaging equipment, and when the acquisition of the positive positioning image and the acquisition of the lateral positioning image are generally carried out, the positioning information of the equipment is different, and the positioning information of the acquisition equipment is acquired while the to-be-reconstructed positioning image is acquired, so that whether the to-be-reconstructed positioning image is the positive positioning image or the lateral positioning image can be determined according to the positioning information.

And step 230, determining the size of a positioning frame corresponding to the target image to be reconstructed and a preset identification algorithm according to the attribute of the target area.

The positioning frame is used for determining the range of the reconstructed target image. The target area may be the head, chest, neck, etc. The attribute of the target area may be the type of the target area or the contour information of the target area. The size of the positioning frame may be specifically the length and width of the positioning frame, or may be the outline of the positioning frame. The preset recognition algorithm is an algorithm for recognizing the target area.

Specifically, the corresponding relationship between each target region and the preset recognition algorithm thereof may be pre-established, and the preset recognition algorithm may be determined according to the type and the corresponding relationship of the target region.

Further, the specific content of the preset identification algorithm can be determined according to the gray level distribution rule of the target area.

And 240, determining the initial position of the positioning frame corresponding to the target image to be reconstructed based on a preset identification algorithm.

Illustratively, the step of determining the initial position by the preset identification algorithm is as follows:

traversing each line of a target image to be reconstructed from top to bottom, judging whether the gray value corresponding to each pixel of the current line meets a preset gray condition, if so, determining the initial position of the current behavior positioning frame, wherein the preset gray condition is as follows: the number of the gray values of the pixels in the current row within a preset gray threshold range reaches a preset number, wherein the preset gray threshold range is (Tv1, Tv2), and the preset gray threshold range is specifically determined by the gray distribution of the target area. Taking the target area as a header as an example, Tv1 can be-900, -950, -1000 or other values, Tv2 can be-800, -850, -900 or other values, and the preset number can be 80-700.

And 250, extracting a target image to be reconstructed of the target area in the positioning image to be reconstructed based on the size and the initial position of the positioning frame.

Specifically, after the initial position and size of the positioning frame are determined, the position of the positioning frame is determined, so that the region surrounded by the positioning frame is extracted, that is, the target image to be reconstructed of the target region is extracted.

And step 260, generating a three-dimensional reconstruction image of the target image to be reconstructed based on a pre-trained network model.

The network model comprises a down-sampling module, a dimensionality extension module and an up-sampling module, wherein the dimensionality extension module connects the down-sampling module and the up-sampling module in a cross-layer mode, the dimensionality extension module comprises a channel generation unit and a second convolution unit, and the channel generation unit is used for recombining two-dimensional characteristic graphs of all channels output by the down-sampling module into a three-dimensional characteristic graph and sending the three-dimensional characteristic graph to the up-sampling module. The down-sampling module comprises a first convolution unit and a first residual error unit, the up-sampling module comprises a transposition convolution unit and a second residual error unit, the first convolution unit comprises a two-dimensional convolution layer, a batch normalization layer and an activation layer, and the two-dimensional convolution layer stride is 2; the first residual error unit is a two-dimensional residual error module; the second convolution layer comprises a three-dimensional convolution layer, a batch normalization layer and an activation layer; the transposition convolution unit comprises a three-dimensional transposition convolution layer, a batch normalization layer and an activation layer; the second residual error unit is a three-dimensional residual error module.

Specifically, the two-dimensional residual module generates a convolutional layer output f (x) from an input image or a feature map x through two or more two-dimensional convolutional layers, adds the convolutional layer output to the module input image x, and uses the convolutional layer output f (x) as the output of the residual module through an active layer ReLU. The difference between the three-dimensional residual error module and the two-dimensional residual error module is that the dimension of the convolution layer is different, and the convolution layer corresponding to the three-dimensional residual error module is three-dimensional convolution.

Exemplarily, fig. 2B is a schematic structural diagram of a network model according to a second embodiment of the present invention, and as shown In fig. 2B, the network model includes 4 Down-sampling modules (Down), 4 dimension expansion modules (Expand), and 4 Up-sampling modules (Up), where In represents an input of the network model, i.e., the target image to be reconstructed, and Out represents an output of the network model, i.e., the three-dimensional reconstructed image. The Down-sampling module (Down) is composed of a two-dimensional convolution layer, a batch normalization layer, a ReLU activation layer and a two-dimensional residual error module, the dimensionality extension module (expanded) is composed of a channel recombination module, a three-dimensional convolution layer, a batch normalization layer and a ReLU activation layer, and the Up-sampling module (Up) is composed of a three-dimensional transposition convolution layer, a batch normalization layer, a ReLU activation layer and a three-dimensional residual error module.

Specifically, the network model takes the two-dimensional sample image extracted from the target area as input and takes the gold standard of the three-dimensional image as output to carry out network parameter training.

Optionally, the generation process of the gold standard of the three-dimensional reconstructed image is as follows:

acquiring a three-dimensional sample image and a corresponding two-dimensional sample image; determining the center of a target area in a two-dimensional sample image in the two-dimensional sample image; determining a three-dimensional center of a gold standard corresponding to the three-dimensional sample image according to the center of the target area and the size of the target area; and intercepting the target area of the three-dimensional sample image based on the three-dimensional center of the gold standard and the size of the target area to generate the gold standard of the three-dimensional reconstruction image of the two-dimensional sample image, so as to verify the network model according to the gold standard.

Specifically, after the three-dimensional center of the gold standard is determined, the segmentation markers of the three-dimensional sample image are moved according to the position of the three-dimensional center of the gold standard, so as to obtain the gold standard with the three-dimensional center as the center.

Specifically, the two-dimensional sample image may be an image obtained by extracting historical scout image data of a CT scan through the target region. Specifically, the center of the target area may be determined according to the initial position of the positioning frame and the attribute of the target area.

Specifically, the size of the target area comprises the gravity center or the depth of the target area, a projection graph of the two-dimensional sample image is generated based on a projection algorithm, the gravity center of the projection graph is calculated according to the center of the target area, the depth of the target area is determined according to the gravity center of the projection graph, and the gravity center of the target area is determined according to the center of the target area and the depth. And then, according to the center and the depth of the target area, determining the three-dimensional center of the gold standard corresponding to the dimensional sample image.

Specifically, after the center of the target region is determined, the center of gravity of the target region is calculated within a preset range from top to bottom in the axial direction of the center position, and coordinate values in the depth direction of the target region are recorded, so that the depth of the target region is obtained, and the gold standard of the three-dimensional image of the two-dimensional sample image is generated based on the center, the center of gravity and the depth of the target region.

Further, after obtaining the gold standard of the three-dimensional image, the gold standard of the three-dimensional image may be resampled.

Optionally, the method for reconstructing a three-dimensional image further includes:

judging whether the three-dimensional reconstruction image is qualified or not according to a discrimination model; and if so, outputting the three-dimensional reconstruction image.

Specifically, the generated network model is used as a generator, and a discriminator (also called a discrimination model) corresponding to the generator is constructed, so as to form a generation countermeasure network, so as to improve the quality of output, i.e. improve the precision of three-dimensional reconstruction.

Optionally, the discriminant model may be a 3D pitchgan discriminator, where the 3D pitchgan discriminator is also referred to as a markov discriminator.

According to the technical scheme of the embodiment of the invention, the three-dimensional reconstruction is carried out by automatically identifying the type of the positioning image and setting the corresponding network model according to different types, so that the accuracy of the three-dimensional reconstruction is improved; the gold standard for network verification is determined based on the gravity center positioning method, so that the gold standard is aligned with the input positioning image, and the accuracy of network verification is improved; meanwhile, the batch normalization layer and the residual error layer introduced by the network model improve the depth and convergence speed of the network, thereby improving the efficiency of three-dimensional reconstruction; meanwhile, a generation countermeasure network is introduced, a network model is taken as a generator, and a corresponding discriminator is established to discriminate the output three-dimensional reconstruction image, so that the quality and the precision of the output three-dimensional reconstruction image are further improved.

EXAMPLE III

Fig. 3 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional image according to a third embodiment of the present invention, as shown in fig. 3, the apparatus includes: a positioning image obtaining module 310, an image to be reconstructed extracting module 320 and an image three-dimensional reconstruction module 330.

The positioning image obtaining module 310 is configured to obtain a positioning image to be reconstructed; an image to be reconstructed extracting module 320, configured to extract a target image to be reconstructed in a target region in the to-be-reconstructed positioned image; the image three-dimensional reconstruction module 330 is configured to generate a three-dimensional reconstruction image of the target image to be reconstructed based on a pre-trained network model, where the network model includes a down-sampling module, a dimension expansion module and an up-sampling module, the dimension expansion module connects the down-sampling module and the up-sampling module across layers, and the dimension expansion module is configured to recombine the two-dimensional feature maps of each channel output by the down-sampling module into a three-dimensional feature map and send the three-dimensional feature map to the up-sampling module.

According to the technical scheme of the embodiment of the invention, the target area extraction is carried out on the positioning image to be reconstructed, so that the data volume to be processed is reduced, and the reconstruction efficiency is improved; the positioning image is subjected to three-dimensional image reconstruction through the pre-trained network model, so that the three-dimensional reconstruction precision is improved, the imaging part of the positioning image is not limited, and the application range of the three-dimensional reconstruction is improved; and the network model is provided with a dimension extension module and is connected in a cross-layer connection mode, so that the network training process is accelerated, and the reconstruction precision is further improved. The reconstruction method of the three-dimensional image provided by the embodiment of the invention has the advantages of wide reconstruction application range and high precision.

Optionally, the network model includes a lateral network model and a positive network model, where the to-be-reconstructed positioning image corresponding to the lateral network model is a lateral positioning image, and the to-be-reconstructed positioning image corresponding to the positive network model is a positive positioning image.

Optionally, the apparatus for reconstructing a three-dimensional image further includes:

and the type identification module is used for identifying the type of the positioning image to be reconstructed.

The positioning image to be reconstructed comprises a lateral positioning image and a normal positioning image.

Optionally, the type identification module is specifically configured to:

determining the type of the to-be-reconstructed positioning image based on a pre-trained neural network model, or identifying contour information of the to-be-reconstructed positioning image, and determining the type of the to-be-reconstructed positioning image according to the contour information, or acquiring positioning information of acquisition equipment of the to-be-reconstructed positioning image, and determining the type of the to-be-reconstructed positioning image according to the positioning information.

Optionally, the generation process of the gold standard of the three-dimensional reconstructed image is as follows:

acquiring a three-dimensional sample image and a corresponding two-dimensional sample image; determining the center of a target area in the two-dimensional sample image; determining a three-dimensional center of a gold standard corresponding to the two-dimensional sample image according to the center of a target area in the two-dimensional sample image and the size of the target area; and intercepting a target area of the three-dimensional sample image based on the three-dimensional center of the gold standard to generate the gold standard of the three-dimensional reconstruction image of the two-dimensional sample image, and verifying a network model according to the gold standard.

Optionally, the to-be-reconstructed image extracting module 320 is specifically configured to:

determining the size of a positioning frame corresponding to the target image to be reconstructed and a preset identification algorithm according to the attribute of the target area; determining the initial position of a positioning frame corresponding to the target image to be reconstructed based on a preset identification algorithm; and extracting a target image to be reconstructed of the target area in the positioning image to be reconstructed based on the size and the initial position of the positioning frame.

Optionally, the down-sampling module includes a first convolution unit and a first residual unit, the up-sampling module includes a transposed convolution unit and a second residual unit, and the first convolution unit includes a two-dimensional convolution layer, a batch normalization layer and an activation layer; the first residual error unit is a two-dimensional residual error module; the transposition convolution unit comprises a three-dimensional transposition convolution layer, a batch normalization layer and an activation layer; the second residual error unit is a three-dimensional residual error module.

Optionally, the apparatus for reconstructing a three-dimensional image further includes:

the output judging module is used for judging whether the three-dimensional reconstruction image is qualified or not according to a judging model; and if so, outputting the three-dimensional reconstruction image.

The device for reconstructing the three-dimensional image provided by the embodiment of the invention can execute the method for reconstructing the three-dimensional image provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example four

Fig. 4 is a schematic structural diagram of an apparatus for reconstructing a three-dimensional image according to a fourth embodiment of the present invention, as shown in fig. 4, the apparatus includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the device processors 410 may be one or more, and one processor 410 is taken as an example in fig. 4; the processor 410, the memory 420, the input device 430 and the output device 440 in the apparatus may be connected by a bus or other means, for example, in fig. 4.

The memory 420 serves as a computer-readable storage medium, and may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the three-dimensional image reconstruction method in the embodiment of the present invention (for example, the positioning image acquisition module 310, the to-be-reconstructed image extraction module 320, and the image three-dimensional reconstruction module 330 in the three-dimensional image reconstruction apparatus). The processor 410 executes various functional applications of the apparatus and data processing, i.e., implements the above-described three-dimensional image reconstruction method, by executing software programs, instructions, and modules stored in the memory 420.

The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 420 may further include memory located remotely from the processor 410, which may be connected to the device/terminal/server via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input means 430 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the apparatus. The output device 440 may include a display device such as a display screen.

EXAMPLE five

An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for reconstructing a three-dimensional image, the method including:

acquiring a positioning image to be reconstructed;

extracting a target image to be reconstructed in a target area in the positioning image to be reconstructed;

the method comprises the steps of generating a three-dimensional reconstruction image of a target image to be reconstructed based on a pre-trained network model, wherein the network model comprises a down-sampling module, a dimensionality extension module and an up-sampling module, the down-sampling module and the up-sampling module are connected in a cross-layer mode through the dimensionality extension module, and the dimensionality extension module is used for recombining two-dimensional characteristic graphs of all channels output by the down-sampling module into a three-dimensional characteristic graph and sending the three-dimensional characteristic graph to the up-sampling module.

Of course, the storage medium containing the computer-executable instructions provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the method for reconstructing a three-dimensional image provided by any embodiments of the present invention.

From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.

It should be noted that, in the embodiment of the three-dimensional image reconstruction apparatus, the units and modules included in the embodiment are only divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.