阅读说明：本技术 基于可变形图谱的医学影像配准网络训练数据集扩充方法 (Medical image registration network training data set expansion method based on deformable atlas ) 是由 韩璐 朱闻韬 饶璠 张铎 祁二钊 于 2021-09-09 设计创作，主要内容包括：本发明公开了一种基于可变形图谱的医学影像配准网络训练数据集扩充方法。该方法首先在尽可能避免引入不符合形态学变形的前提下,将输入的训练数据配准到不同形态的图谱上。随后,评估配准后训练数据和图谱的差异,修正图谱的参数向不引入不合理变形的方向微调。接着,在最大化配准精度的条件下,将输入的训练数据配准到不同形态的修正后的图谱上,恢复出训练数据第一次配准时未变形完全的细节。最后,计算相关性度量参数,剔除不满足预期的变形数据,从而生成与输入训练数据集形态不同且反应人体形态真实变化规律的扩充数据集。本发明可以有效地扩充医学影像配准网络的训练数据集,从而降低构建数据集的时间成本、人力成本和费用成本。(The invention discloses a medical image registration network training data set expansion method based on a deformable atlas. The method firstly registers input training data to atlases of different forms on the premise of avoiding introducing non-conforming morphological deformation as much as possible. Subsequently, the differences between the registered training data and the atlas are evaluated, and the parameters of the corrected atlas are fine-tuned in a direction that does not introduce unreasonable deformation. Then, under the condition of maximizing the registration accuracy, registering the input training data to the corrected atlas in different forms, and recovering the details which are not completely deformed when the training data is registered for the first time. And finally, calculating a correlation measurement parameter, and eliminating deformation data which do not meet expectations, thereby generating an expansion data set which has a shape different from that of the input training data set and reflects the real change rule of the human body shape. The invention can effectively expand the training data set of the medical image registration network, thereby reducing the time cost, the labor cost and the expense cost for constructing the data set.)

1. A medical image registration network training data set expansion method based on a deformable atlas is characterized by comprising the following steps:

the method comprises the following steps: adjusting morphological characteristic control parameters of the atlas model to generate a deformation atlas different from the shape of the input training data image;

step two: under the condition of ensuring the registration deformation smoothness, registering the training data images on deformation maps with different forms one by one to obtain registered training data images;

step three: correcting parameters of the deformation map according to the corresponding registered training data images and the results of shape fitting and/or gray level fitting of the deformation map, and outputting the corrected deformation map;

step four: under the condition of ensuring the registration precision, registering the training data images on the corresponding modified deformation maps one by one to obtain an expansion data set;

step five: and calculating the correlation measurement parameters of the corresponding expansion data set images and the training data images one by one, and screening out the expansion data set which has different forms from the training data set and reflects the real change rule of the human body form.

2. The method for expanding a medical image registration network training data set based on a deformable atlas as claimed in claim 1, wherein in the first step, the deformation equation of the atlas model is expressed as:

X = X ₀ +∑ _k a _k V _k

wherein X represents the mesh vertex coordinates of the deformation map,X ₀the coordinates of the vertices of the mesh representing the basic form of the atlas,V _k is shown askThe shape feature vector corresponding to each feature,a _k controlling parameters for corresponding morphological features;

the method for generating the deformation map with the shape different from the shape of the input training data image by adjusting the shape characteristic parameters of the map model comprises the following steps:

registering the maps corresponding to the basic forms in the map model on the input training data images one by one, and calculating and fitting the form characteristic control parameters of the registered mapsa _k Then, further witha _k Randomly generating new morphological feature control parameters for reference using random numbersa’ _k And generating a deformation map with a different form from the input training data.

3. The method for expanding a medical image registration network training data set based on a deformable atlas as claimed in claim 1, wherein in the second step and the fourth step, the registration tool is one or two of trained registration network, SimpleITK, ANTs and Elastix.

4. The method as claimed in claim 3, wherein the loss function of the trained registration network training is a loss functionWhereinf、mRespectively representing a fixed image and a floating image at the time of registration,ϕin order to be a field of deformation,m ◦ ϕfor the floating images after the registration to be the same,L _sim for error loss of the registered floating image from the original floating image,L _smooth and for the loss of the registration deformation smoothness, lambda is a regularization coefficient, and the registration deformation smoothness and the registration accuracy are adjusted by controlling the regularization coefficient of a loss function in training, wherein the smaller the lambda is, the higher the registration accuracy is, and the lower the registration deformation smoothness is.

5. The medical image registration network training data set expansion method based on deformable atlas as claimed in claim 4, wherein the condition for ensuring the smoothness of registration deformation is: the lambda value is 0.01-0.1, and the condition for ensuring the registration precision is as follows: the lambda value is 0.001-0.01.

6. The medical image registration network training data set expansion method based on deformable atlas of claim 1, wherein in the third step, the shape fitting specifically is:

calculating the displacement difference between the two corresponding point clouds in the registered training data image obtained in the step two and the deformed atlas generated in the step one, and approximating the displacement difference between the two corresponding point clouds by using each shape feature vector in the atlas model; correcting the deformed atlas generated in the first step according to the numerical component change of each shape feature vector obtained by solving;

the gray fitting specifically comprises: calculating the gray difference between the two registered training data images obtained in the step two and the two images of the deformed atlas generated in the step one, and fitting the gray change between the two images by utilizing each gray feature vector in the atlas model; and correcting the gray scale of the deformed atlas generated in the step one according to the numerical component change of each gray scale feature vector obtained by solving.

7. The method for expanding a medical image registration network training data set based on a deformable atlas as claimed in claim 1, wherein the correlation metric parameters in the fifth step are correlation and/or mutual information parameters.

8. The method for expanding a medical image registration network training data set based on a deformable atlas of claim 1, wherein the training data image is a CT image, an MRI image, an ultrasound image, a PET image, or a SPECT image.

Technical Field

The invention belongs to the field of image data expansion, and particularly relates to a medical image registration network training data set expansion method based on a deformable atlas.

Background

In recent years, deep learning has become more widely used in medical image analysis research. The medical image registration method based on deep learning achieves good effects and has great advantages and potentials. Compared with the traditional registration method, the registration method based on deep learning can shorten the operation time magnitude on the premise of obtaining consistent or higher registration precision. However, the lack of a training data set limits the advantages of the deep learning approach in many cases. Therefore, a method for effectively constructing a deep learning training data set is urgently needed at present, and the method has a good practical significance for deepening the application of deep learning in medical image analysis research.

With the advent of the big data age, characteristic parameters of map deformation can be learned from a large number of real human body shape data samples by adopting a statistical modeling method, so that the deformable map can represent individual shape difference information by adjusting the characteristic parameters. The personalized three-dimensional deformable atlas technology is increasingly applied to the medical field, and provides valuable reference data for doctor-patient communication, analog simulation and clinical diagnosis and treatment.

The deformable atlas technique takes an average human anatomy model as a basic form, and realizes the deformation of the model by applying deformation vectors to the basic form, wherein the deformation equation is as follows: x =X ₀ +∑ _k a _k V _k . Wherein the content of the first and second substances,X ₀the coordinates of the vertices of the mesh representing the basic form of the atlas are described asX ₀ = [x ₁,y ₁,z ₁,…, x _i ,y _i ,z _i ,…x _n ,y _n ,z _n ] ^T ，(x _i ,y _i ,z _i ) First, theiCoordinates of each vertex; x represents the grid vertex coordinates after the atlas is deformed;V _k is shown askThe shape feature vector corresponding to each feature is generated by the statistics of a large amount of real human body shape dataV _k = [v _k,1(x), v _k,1(y), v _k,1(z),…, v _{k, i} (x), v _{k, i} (y), v _k,i (z),…,v _{k, n} (x), v _{k, n} (y), v _k,n (z),] ^T ，( v _{k, i} (x), v _{k, i} (y), v _k,i (z) Is a firstkIn a feature directed toiDisplacement vectors of the vertices;a _k the parameters are controlled for the corresponding morphological characteristics,a _k the value of (a) determines the deformation result of the map.

Disclosure of Invention

The invention provides a medical image registration network training data set expansion method based on a deformable atlas, aiming at the problem of the shortage of the existing deep learning registration network training data set. The method utilizes the characteristic that a deformable atlas really reflects the individual difference of the human body morphology, firstly, an input training data set is registered to atlases of different individual morphologies, then, morphological characteristic parameters of the atlases are finely adjusted, the atlases are corrected in a direction which does not conform to the morphological deformation, finally, the input training data is registered to atlases of different modified individual morphologies, correlation measurement parameters are calculated, and an expansion data set which is different from the training data set in morphology, can represent the individual morphological difference and conforms to the morphological rule is output after screening. The invention utilizes the smooth constraint of the registration deformation and the characteristics obtained by the statistics of the deformable atlas to constrain the unreasonable deformation, thereby generating the expansion training data with different forms under the condition of ensuring the change rule of the forms.

The technical scheme of the invention is as follows:

a medical image registration network training data set expansion method based on a deformable atlas comprises the following steps:

the method comprises the following steps: generating a deformation map with a shape different from that of the input training data image by manually adjusting or automatically adjusting morphological characteristic control parameters of the map model through an algorithm;

step two: under the condition of ensuring the registration deformation smoothness, registering the training data images on deformation maps with different forms one by one to obtain registered training data images;

step three: correcting parameters of the deformation map according to the corresponding registered training data images and the results of shape fitting and/or gray level fitting of the deformation map, finely adjusting in a direction without unreasonable deformation, and outputting the corrected deformation map;

step four: under the condition of ensuring the registration accuracy, registering the training data images on the corresponding modified deformation maps one by one, and recovering the details which are not completely deformed when the training data in the step one is registered for the first time to obtain an expansion data set;

step five: and calculating the correlation measurement parameters of the corresponding expansion data set images and the training data images one by one, and screening out the expansion data set which has different forms from the training data set and reflects the real change rule of the human body form.

Further, in the first step, the deformation equation of the atlas model is expressed as:

X = X ₀ +∑ _k a _k V _k

wherein X represents the mesh vertex coordinates of the deformation map,X ₀the coordinates of the vertices of the mesh representing the basic form of the atlas,V _k is shown askThe shape feature vector corresponding to each feature,a _k is in a corresponding shapeA state characteristic control parameter;

the different forms of the atlas are realized by adjusting the form characteristic control parameters of the atlas according to the deformation equation of the atlas under the basic form of default characteristic parameters, or are adjusted according to an algorithm for automatically adjusting the form characteristic of the atlas, and the method specifically comprises the following steps:

registering the maps corresponding to the basic forms in the map model on the input training data images one by one, and calculating and fitting the form characteristic control parameters of the registered mapsa _k Then, further witha _k Randomly generating new morphological feature control parameters for reference using random numbersa’ _k And generating a deformation map with a different form from the input training data.

Further, in the second step and the fourth step, the registration tool is one or two of a trained registration network, simpletick, ANTs and Elastix.

Further, the loss function during the training of the trained registration network is specifically the loss function during the training of the trained registration networkWhereinf、mRespectively representing a fixed image and a floating image at the time of registration,ϕin order to be a field of deformation,m ◦ ϕfor the floating images after the registration to be the same,L _sim for error loss of the registered floating image from the original floating image,L _smooth and for the loss of the registration deformation smoothness, lambda is a regularization coefficient, and the registration deformation smoothness and the registration accuracy are adjusted by controlling the regularization coefficient of a loss function in training, wherein the smaller the lambda is, the higher the registration accuracy is, and the lower the registration deformation smoothness is. In the second step, the registration is nonlinear registration, and the regularization factor is large enough to suppress deformation which does not conform to morphology as much as possible under the condition of certain registration accuracy. In the fourth step, the registration is nonlinear registration, the regularization factor is small enough to maximize the registration accuracy, and the non-deformed complete detail information in the second step of registration is recovered. Therefore, the expansion data set can be changed as much as possible on the premise of ensuring reasonable deformation in the fourth stepCan reflect individual morphological differences.

Further, the condition for ensuring the registration deformation smoothness is as follows: the lambda value is 0.01-0.1, and the condition for ensuring the registration precision is as follows: the lambda value is 0.001-0.01.

Further, in the third step, the shape fitting specifically includes:

calculating the displacement difference between the two corresponding point clouds in the registered training data image obtained in the step two and the deformed atlas generated in the step one, and approximating the displacement difference between the two corresponding point clouds by using each shape feature vector in the atlas model; correcting the deformed atlas generated in the first step according to the numerical component change of each shape feature vector obtained by solving;

the gray fitting specifically comprises: calculating the gray difference between the two registered training data images obtained in the step two and the two images of the deformed atlas generated in the step one, and fitting the gray change between the two images by utilizing each gray feature vector in the atlas model; and correcting the gray scale of the deformed atlas generated in the step one according to the numerical component change of each gray scale feature vector obtained by solving.

By correcting the morphological characteristic parameters of the atlas and combining the maximization of the registration precision in the fourth step, the individual morphological differences are truly reflected as far as possible on the premise of ensuring reasonable deformation of the expansion data set.

Further, the correlation metric parameter in the fifth step is a correlation and/or mutual information parameter.

Further, the training data image is a CT image, an MRI image, an ultrasound image, a PET image, or a SPECT image.

Furthermore, the human organs to which the method is applied are not limited, and the method can be applied to organs such as lung, liver and the like.

The invention has the beneficial effects that:

the invention provides a method for constructing a medical image registration network training data set by utilizing the characteristic that a deformable map truly reflects the individual difference of human body morphology. The method can reflect individual form difference as real as possible on the premise of ensuring reasonable deformation of the expansion data set.

The invention can greatly reduce the time cost, the labor cost and the expense cost for constructing the medical image registration network data set, and helps the deep learning method to exert the advantages on medical image analysis.

Drawings

FIG. 1 is a flow chart of a medical image registration network training data set expansion method based on a deformable atlas of the present invention;

FIG. 2 is a schematic diagram of the generation of different individual morphological maps by adjusting morphological feature control parameters of the map model;

fig. 3 is a schematic network structure diagram of an unsupervised non-linear medical image registration network.

Detailed Description

The invention provides a medical image registration network training data set expansion method based on a deformable atlas, which is characterized in that limited training samples for deep learning are expanded by a registration method by means of the advantage of representing individual form difference information of the deformable atlas, and an expansion data set with a form different from that of an input training data set is output after correction and screening. The invention can effectively expand the training data set of the medical image registration network, thereby reducing the time cost, the labor cost and the expense cost for constructing the data set.

The present invention will be described in further detail below with reference to examples and the accompanying drawings.

Example 1

Fig. 1 is a flowchart of a method for expanding a network training data set for registration of medical images of lungs based on a deformable atlas, which specifically includes the following steps:

the method comprises the following steps: the morphological feature control parameters of the manual adjustment atlas model generate a deformation atlas of a different morphology than the input training data image, as shown in fig. 2. Wherein, the construction of the deformable atlas model of the lung follows the following steps:

a. segmenting lungs of CT image data of a large number of healthy subjects, and drawing a segmentation curved surface by using an isosurface extraction algorithm;

and registering the anatomical template obtained by a professional three-dimensional model website to all CT image data samples drawn with the segmentation curved surfaces by adopting a point cloud registration algorithm, so that the curved surface vertexes correspond to the same anatomical positions in different samples, thereby obtaining the corresponding relation of the anatomical structures among the samples.

b. And performing generalized analysis on the point clouds of all the samples after registration, namely performing normalization processing on the spatial position, the direction and the size.

c. And (3) carrying out principal component analysis on the normalized sample, and extracting a deformation vector, wherein the method specifically comprises the following steps:

the normalized sample set is represented asWhere M is the number of samples, and the ith sample is represented by a column vector of the set of vertex coordinatesN is the number of vertex coordinates per sample, the base shape. Covariance matrix based on XPerforming eigenvector decomposition on C, and selecting the front with the largest eigenvaluekFeature vector corresponding to each featureV _k As an estimate of the deformable atlas shape.

d. Deformable atlas techniques rely on averaging human anatomic models by altering the morphologya _k And realizing the deformation of the model, wherein the deformation equation is as follows: x =X ₀ +∑ _k a _k V _k 。

Step two: under the condition of ensuring the registration deformation smoothness, the training data images are registered on deformation maps with different forms one by one, so that the registered training data images which conform to the morphological deformation as much as possible are obtained. In this embodiment, a medical image registration network using unsupervised nonlinearitiesThe registration is realized, the network structure is UNet, and the specific structure is shown in fig. 3. The input of the network is a double-channel three-dimensional image (f, m) formed by splicing a fixed image f and a floating image m (which respectively correspond to a deformation map and a training data image), and the encoding and decoding are carried out by utilizing a three-dimensional convolution operation with a convolution kernel size of 3 and a step length of 2. Each rectangular box in the figure represents a 3D convolution, the change in spatial resolution compared to the input is noted below. The arrows in the decoding phase represent the features connecting the encoding and decoding phases as inputs to the layer; the output is the deformation field and the registered floating image. Using collected normalized CT image data to carry out training, wherein the loss function isWherein, in the step (A),f、mrespectively representing a fixed image and a floating image at the time of registration,ϕin order to be a field of deformation,m ◦ ϕfor the registered floating images, λ is the regularization coefficient,L _sim for error loss of the registered floating image from the original floating image,L _smooth for registration deformation smoothness loss, the following are expressed respectively:

，

,

，，

where p is a certain voxel in the image space omega,table showing fixed images f in relation to pThe function is shown as being a function of,a representation function representing the registered floating image with respect to p, | x | is a norm,p _x ,p _y ,p _z respectively representing the index positions of p in the three directions of x, y and z. u denotes the deformation function of the fixed image f and the registered floating image with respect to p.

The regularization coefficient needs to be large enough to ensure smoothness of registration distortion, and avoid distortion that does not conform to morphology as much as possible, preferably set to 0.01 to 0.1.

Step three: and correcting the map through shape fitting and gray fitting. And (3) calculating the displacement difference between the registered training data image and the two corresponding point clouds in the deformed atlas generated in the step one when the shape is fitted. And (4) the registered training data image obtained in the step (II) is consistent with the curved surface vertex of the deformed atlas generated in the step (I), and the point clouds of the two point clouds have the same number sequence and correspond to the same anatomical position. Thus, the core of shape fitting is the use of individual shape feature vectors in the atlas model, i.e., theV _k Approximating the displacement difference between the registered training data image and the corresponding two point clouds in the deformed atlas generated in step one, and solving equation AY = b by using a singular value, wherein A is the shape feature vectorV _k B is a position difference obtained by difference of two point clouds, Y is a numerical value component of each shape characteristic vector to be solved, and the morphological characteristic control parameters of the atlas are corrected according to Y so as to be subjected to direction fine adjustment without introducing unreasonable deformation; the gray fitting is similar to the shape fitting, and the difference is that the gray variation between two images is fitted by utilizing each gray feature vector in the atlas model, the equation AY = b is solved through singular values, at the moment, A is a matrix formed by the gray feature vectors, b is the gray difference obtained by the difference of the two images, Y is the numerical component of the deformation vector of each gray feature to be solved, and the unreasonable gray variation of the deformed atlas generated in the first step is corrected according to Y.

Step four: under the condition of ensuring the registration accuracy, the training data images are non-linearly registered to the corresponding modified deformation atlas one by one through a registration network shown in fig. 3 to obtain an extended data set. To maximize the accuracy of registration, the regularization coefficients during the training of the registration network need to be small enough, preferably set to 0.001 to 0.01, so that the details of the first registration of the step-one training data image that are not completely deformed can be recovered. The arrow in fig. 1 illustrates the details of incomplete deformation caused by insufficient registration accuracy due to smooth constraint of deformation during the first registration, and after the atlas is corrected, an image which is different from the input training data set in shape and reflects the real shape of a human body is obtained through the second registration criterion.

Step five: the correlation measurement parameter NCC normalization correlation coefficient of the expansion data set image and the training data image is calculated one by one,wherein, in the step (A),f、mrespectively representing a fixed image and a floating image at the time of registration,ϕin order to be a field of deformation,m ◦ ϕfor the floating images after the registration to be the same,andmean gray values representing the fixed image and the registered floating image, respectively:，and p is a certain voxel in the image space omega. And eliminating an expansion data set which is too identical to the input training data set according to the NCC, and screening out the expansion data set which is different from the input training data set in form and reflects the real change rule of the human body form.

Example 2

As another preferred scheme, the invention may also adopt a specific algorithm to automatically adjust morphological characteristic parameters of the atlas to generate a deformed atlas with a different shape from the input training data, and the registration may be implemented by using a traditional medical image registration method and by using toolkits such as simpletik, ANTs, Elastix, and the like, specifically as follows:

the method comprises the following steps: registering the maps corresponding to the basic forms in the map model on the input training data images one by one, and calculating and fitting the form characteristic control parameters of the registered mapsa _k Then, further witha _k Randomly generating new morphological feature control parameters for reference using random numbersa’ _k And generating a deformation map with a different form from the input training data.

Step two: and registering the training data images to deformation maps with different forms one by using an Elastix tool kit to obtain the registered training data images. The Elastix toolkit uses a command line program Elastix for registration, and the most basic registration operation command is as follows: the method comprises the following steps of (1) defining the contents of measurement, optimization, parameters and the like of registration. The Penalty terms in the parameter file txt parameter file can be modified to adjust the smoothness and registration accuracy of the registration deformation according to the official instruction manual of Elastix, with the reference configuration as follows:

(Metric "AnySimilarityMetric" "TransformBendingEnergyPenalty")

(Metric0Weight 1.0)

(Metric1Weight <weight>)

wherein, the larger the weight is, the higher the smooth degree of the deformation is; the smaller the weight, the lower the smoothness of the deformation.

Step three: and mapping the vertex of the atlas to the registered training data obtained in the second step by using transformix of an Elastix toolkit in combination with the registered training data and the deformed atlas generated in the first step, wherein the corresponding calling command is transformix-def inputPoint. And then, fine tuning is carried out on the parameters of the corrected map through shape fitting and gray fitting in the direction without introducing unreasonable deformation, and the corrected deformation map is output.

Step four: under the condition of maximizing the registration accuracy, registering the training data images on the corresponding modified deformation maps one by one, and recovering the details which are not completely deformed when the training data is firstly registered in the first step to obtain an expanded data set.

Step five: and calculating the correlation measurement parameters of the corresponding expansion data set images and the training data images one by one, and screening out the expansion data set which has different forms from the training data set and reflects the real change rule of the human body form.

It should be noted that, although the example is described using a CT image, the present invention is not limited to the CT image, and may include an ultrasound image, an MRI image, a PET image, and a SPECT image. Meanwhile, the applicable human organs are not limited, and the preparation method can be applied to organs such as lung, liver and the like.

The above examples are merely for clarity of explanation and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should all embodiments be exhaustive. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.