阅读说明：本技术 一种利用光学体表运动信号合成实时图像的系统 (System for synthesizing real-time image by using optical body surface motion signal ) 是由 张艺宝 黄宇亮 李晨光 吴昊 刘宏嘉 于 2021-02-07 设计创作，主要内容包括：本发明公开了一种利用光学体表运动信号合成实时图像的系统,包含：获取单元,获取实时体表信号；图像转化单元,根据体表信号与4D医学图像显示的内部解剖结构信息的映射关系,通过获取体表信号,得到对应的时间的4D医学图像显示的内部解剖结构信息。本发明的有益效果：呼吸运动会增加放疗过程中射线的脱靶风险,引起肿瘤复发转移和正常器官损伤,利用光学体表运动信号合成实时图像的系统可以在治疗过程中实时显示患者内部解剖结构信息,降低放疗过程中射线的脱靶风险,同时具有兼容性好、成本低、高效、无创、安全、可靠等优点。(The invention discloses a system for synthesizing real-time images by utilizing optical body surface motion signals, which comprises: the acquisition unit is used for acquiring real-time body surface signals; and the image conversion unit is used for obtaining the internal anatomical structure information displayed by the 4D medical image at the corresponding time by acquiring the body surface signal according to the mapping relation between the body surface signal and the internal anatomical structure information displayed by the 4D medical image. The invention has the beneficial effects that: the respiratory motion can increase the miss-target risk of rays in the radiotherapy process, cause tumor recurrence and metastasis and normal organ injury, the system for synthesizing the real-time image by utilizing the optical body surface motion signal can display the internal anatomical structure information of a patient in real time in the treatment process, reduce the miss-target risk of rays in the radiotherapy process, and simultaneously has the advantages of good compatibility, low cost, high efficiency, no wound, safety, reliability and the like.)

1. A system for synthesizing a real-time image using optical body surface motion signals, comprising:

the acquisition unit is used for acquiring real-time body surface signals; and the image conversion unit is used for obtaining the internal anatomical structure information displayed by the 4D medical image corresponding to the time by acquiring the body surface signal according to the mapping relation between the body surface signal and the internal anatomical structure information displayed by the 4D medical image.

2. The system for synthesizing real-time images using optical body surface motion signals according to claim 1, wherein the 4D medical images comprise directly captured 4D images or reconstructed synthesized 4D images.

3. The system of claim 1, further comprising a 4D image reconstruction and synthesis unit for acquiring CBCT before treatment and simultaneously acquiring synchronous body surface data, dividing each angular projection of CBCT into different time phases, associating the synchronous optical body surface data with a respiratory time phase, reconstructing each time phase projection of CBCT based on sparse projection data, and associating the body surface data with an in vivo anatomical structure image of the same respiratory time phase.

4. The system for synthesizing real-time images from optical body surface motion signals according to claim 3, wherein the 4D image reconstructing and synthesizing unit reconstructs the 4D images by the following specific process: (1) generation of a prior respiratory motion model: registering each time phase of the four-dimensional CT with the four-dimensional CT at a certain moment as a reference, and decomposing the obtained deformation field into a weighted sum of main components; (2) sequencing the body surface data: a method for obtaining a respiratory time phase inferred from the instantaneous body surface by segmenting a body surface contour from each time phase of the four-dimensional CT and associating the body surface contour with the respiratory time phase; (3) reconstructing sparse projection data based on CBCT: according to optical body surface data synchronously acquired in the CBCT scanning process, sequencing CBCT two-dimensional projections according to respiration; calculating the weight of each principal component in the prior respiratory motion model according to the fact that the two-dimensional digital reconstructed image calculated after the CBCT deformation is approximately consistent with the actually acquired CBCT two-dimensional projection; and determining a deformation field according to the weight, and obtaining a reconstructed and synthesized 4D image according to the deformation field and the reference image.

5. The system of claim 3 or 4, wherein the real-time optical body surface signals are collected during the treatment and correlated with the acquired body surface data before the treatment, and a 4D image of the in-vivo anatomical structure is further obtained by mapping the body surface data to the in-vivo anatomical structure image at the same respiratory phase.

6. The system for synthesizing real-time images using optical body surface motion signals according to claim 1, further comprising a 4D image reconstruction synthesizing unit for reconstructing a boundary of the anatomical structure in the body corresponding to a change in the body surface and for inferring a deformation of the anatomical structure in the body from the deformation of the body surface.

7. The system of claim 6, wherein the relationship between the body surface deformation and the body internal deformation is mined from historical data by the following steps: collecting a four-dimensional CT image of a patient, segmenting a body surface contour, taking the body surface deformation or the four-dimensional CT image or the body surface contour as the input of a model, taking the CT of a certain time phase as a reference image, and registering the reference image to the CT of other time phases to obtain the model which is output as a time sequence of a structural deformation field of the in-vivo anatomy; during treatment, real-time optical body surface data is input into the model to predict the deformation field of the in-vivo anatomical structure, and the image at the moment can be obtained by acting on the reference image.

8. The system for synthesizing real-time images using optical body surface motion signals according to any one of claims 1 to 7, wherein during the treatment, a two-dimensional projection image of the patient is obtained by means of X-ray imaging, and a corresponding respiratory phase is determined according to the synchronously collected optical body surfaces, and a four-dimensional image is predicted, so as to generate digital reconstruction projection images of the same phase and the same angle; and (4) reconstructing the projection image, and comparing the projection image with a two-dimensional projection image of the patient obtained by means of X-ray imaging to verify the accuracy of the method.

9. The system for synthesizing real-time images using optical body surface motion signals according to any one of claims 3 to 5, wherein during the treatment, a two-dimensional projection image of the patient is obtained by means of X-ray imaging, the acquired projection image is used as correction data, the subsequent results are improved and optimized, the respiratory phase is determined by the synchronously acquired optical body surface, and then the 4D image is reconstructed by the 4D image reconstruction and synthesis unit, and at this time, the deformation field of the anatomical structure in the body should satisfy the condition that the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image is close to the real acquired two-dimensional projection image, and if the deviation is large, the deformation field satisfying the condition is calculated, and then the model is retrained.

10. The system of claim 6 or 7, wherein during the treatment, a two-dimensional projection of the patient is obtained by means of X-ray imaging, the acquired projection is used as correction data, the subsequent results are improved and optimized, the respiratory phase is determined by the synchronously acquired optical body surface, and the deformation field of the anatomical structure in the body is such that the two-dimensional digital reconstruction projection calculated by applying the deformation field to the reference image is approximately consistent with the two-dimensional projection acquired actually, the weight of the deformation field satisfying the condition is calculated, and then the model is retrained.

Technical Field

The invention relates to the technical field of medical equipment, in particular to a system for synthesizing a real-time image by utilizing optical body surface motion signals.

Background

Respiratory motion can increase the risk of off-target of radiation during radiotherapy, causing tumor recurrence and metastasis and normal organ damage. In the prior art, the precision of a 1D body surface motion substitution signal is limited, the internal structure cannot be visualized in real time, and anatomical changes between 4D planned CT and actual treatment cannot be reflected by the 1D respiratory signal. X-ray fluoroscopy, 2D anatomical information superposition, poor image quality, imaging at an angle orthogonal to a therapeutic ray, a monitoring blind area with the same-direction movement of the imaging ray exists, a large amount of radiation dose is accumulated in fluoroscopy, and radiation injury and secondary carcinogenic risk are increased. 3DCBCT or 4DCBCT, pre-treatment imaging, does not allow real-time visual navigation during treatment. Optical body surface, internal structures cannot be visualized in real time.

A real-time system capable of displaying information about the internal anatomy of a patient during treatment is a technical problem to be solved.

Disclosure of Invention

The invention aims to solve the technical problem that the information of the internal anatomical structure of a patient cannot be displayed in real time in the treatment process in the prior art.

In order to solve the technical problems, the invention adopts the technical scheme that: a system for synthesizing a real-time image using optical body surface motion signals, comprising: the acquisition unit is used for acquiring real-time body surface signals; and the image conversion unit is used for obtaining the internal anatomical structure information displayed by the 4D medical image corresponding to the time by acquiring the body surface signal according to the mapping relation between the body surface signal and the internal anatomical structure information displayed by the 4D medical image.

Preferably, the 4D medical image includes a 4D image obtained by direct photographing, or a reconstructed composite 4D image.

Preferably, the device further comprises a 4D image reconstruction and synthesis unit, wherein the device acquires CBCT before treatment and simultaneously acquires synchronous body surface data, divides each angle projection of the CBCT into different time phases, associates the synchronous optical body surface data with a respiratory time phase, reconstructs the projections of the CBCT at each time phase based on sparse projection data, and associates the body surface data with an in-vivo anatomical structure image of the same respiratory time phase.

Preferably, the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) generation of a prior respiratory motion model: registering each time phase of the four-dimensional CT with the four-dimensional CT at a certain moment as a reference, and decomposing the obtained deformation field into a weighted sum of main components; (2) sequencing the body surface data: a method for obtaining a respiratory time phase inferred from the instantaneous body surface by segmenting a body surface contour from each time phase of the four-dimensional CT and associating the body surface contour with the respiratory time phase; (3) reconstructing sparse projection data based on CBCT: according to optical body surface data synchronously acquired in the CBCT scanning process, sequencing CBCT two-dimensional projections according to respiration; calculating the weight of each principal component in the prior respiratory motion model according to the fact that the two-dimensional digital reconstructed image calculated after the CBCT deformation is approximately consistent with the actually acquired CBCT two-dimensional projection; and determining a deformation field according to the weight, and obtaining a reconstructed and synthesized 4D image according to the deformation field and the reference image.

Preferably, the real-time optical body surface signals are collected during treatment, the real-time optical body surface signals are associated with the obtained body surface data before treatment, and then the 4D image of the in-vivo anatomical structure is further obtained through the mapping relation between the body surface data and the in-vivo anatomical structure image at the same breathing time.

Preferably, the system further comprises a 4D image reconstruction and synthesis unit, wherein the boundary of the in-vivo anatomical structure corresponds to the change of the body surface, and the deformation of the in-vivo anatomical structure is deduced from the deformation of the body surface.

Preferably, the relation between the body surface deformation and the body internal deformation is mined from historical data, and the specific process is as follows: collecting a four-dimensional CT image of a patient, segmenting a body surface contour, taking the body surface deformation or the four-dimensional CT image or the body surface contour as the input of a model, taking the CT of a certain time phase as a reference image, and registering the reference image to the CT of other time phases to obtain the model which is output as a time sequence of a structural deformation field of the in-vivo anatomy; during treatment, real-time optical body surface data is input into the model to predict the deformation field of the in-vivo anatomical structure, and the image at the moment can be obtained by acting on the reference image.

Preferably, in the treatment process, a two-dimensional projection image of the patient is obtained in an X-ray imaging mode, a corresponding respiratory time phase is determined according to the synchronously collected optical body surface, and a four-dimensional image is predicted, so that digital reconstruction projection images of the same time phase and the same angle are generated; and (4) reconstructing the projection image, and comparing the projection image with a two-dimensional projection image of the patient obtained by means of X-ray imaging to verify the accuracy of the method.

Preferably, in the treatment process, a two-dimensional projection image of the patient is obtained in an X-ray imaging mode, the acquired projection image is used as correction data, the follow-up result is improved and optimized, the respiratory time phase is determined through the synchronously acquired optical body surface, and then the 4D image is reconstructed through the 4D image reconstruction and synthesis unit, at this time, the deformation field of the in-vivo anatomical structure is required to meet the condition that the two-dimensional digital reconstruction projection calculated after the deformation field acts on the reference image is approximately consistent with the actually acquired two-dimensional projection image, if the deviation is large, the deformation field meeting the condition is calculated, and then the model is retrained.

Preferably, in the treatment process, a two-dimensional projection image of the patient is obtained in an X-ray imaging mode, the acquired projection image is used as correction data, the follow-up result is improved and optimized, the respiratory time phase is determined through the synchronously acquired optical body surface, the deformation field of the in-vivo anatomical structure is required to meet the condition that the two-dimensional digital reconstruction projection calculated after the deformation field acts on the reference image is approximately consistent with the actually acquired two-dimensional projection image, the weight of the deformation field meeting the condition is calculated, and then the model is retrained.

Has the advantages that:

the system for synthesizing the real-time image by using the optical body surface motion signal can display the internal anatomical structure information of the patient in real time in the treatment process, reduces the miss risk of rays in the radiotherapy process, and has the advantages of good compatibility, low cost, high efficiency, no wound, safety, reliability and the like.

Drawings

FIG. 1 is a schematic diagram of a system for synthesizing real-time images using optical body surface motion signals in accordance with the present invention;

FIG. 2 is a schematic diagram of a system for synthesizing real-time images using optical body surface motion signals in accordance with the present invention;

FIG. 3 is a schematic representation of the deformation of a body surface according to the present invention;

FIG. 4 is a schematic representation of predicted principal component weights of the present invention;

FIG. 5 is a diagram illustrating the loss function during the model training process of the present invention.

Detailed Description

The following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings, will make the advantages and features of the invention easier to understand by those skilled in the art, and thus will clearly and clearly define the scope of the invention.

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. It should be apparent that the drawings in the following description are only examples or embodiments of the present invention, and technical features of various embodiments can be combined with each other to form a practical solution for achieving the purpose of the present invention, and a person skilled in the art can apply the present invention to other similar situations according to the drawings without creative efforts. Unless otherwise apparent from the context, or otherwise indicated, like reference numbers in the figures refer to the same structure or operation.

It should be understood that "system" and "unit" as used herein is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, other words may be substituted by other expressions if they accomplish the same purpose. Also, "system" and "unit" may be implemented by software or hardware, and may be a name of a portion having the function, which is physical or virtual.

Flow charts are used in the present invention to illustrate the operations performed by a system according to embodiments of the present invention. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, the various steps may be processed in reverse order or simultaneously. Meanwhile, other operations may be added to the processes, or a certain step or several steps of operations may be removed from the processes. The technical solutions in the embodiments can be combined with each other to achieve the purpose of the present invention.

The first embodiment is as follows: as shown in fig. 1, the technical scheme adopted by the invention is as follows: a system for synthesizing a real-time image using optical body surface motion signals, comprising: the acquisition unit is used for acquiring real-time body surface signals; and the image conversion unit is used for obtaining the internal anatomical structure information displayed by the 4D medical image corresponding to the time by acquiring the body surface signal according to the mapping relation between the body surface signal and the internal anatomical structure information displayed by the 4D medical image.

The 4D (Four-Dimensional) medical image includes 4DCT (Four-Dimensional Computed tomogry Four-Dimensional X-ray Computed Tomography), 4DMR (Four-Dimensional nuclear magnetic resonance), 4DPET (Four-Dimensional positron emission Computed Tomography), 4DCBCT (Four-Dimensional Cone beam Computed tomogry Four-Dimensional X-ray Computed Tomography), and the like.

The body surface signals comprise optical body surface 4D signals or body surface data extracted from 4DCBCT based on other modes, such as obtaining motion signals through body surface reflecting blocks and internal markers such as diaphragm motion, implanted markers and the like.

The system for synthesizing the real-time image by utilizing the optical body surface motion signal can display the internal anatomical structure information of the patient in real time in the treatment process, and reduce the miss risk of rays in the radiotherapy process.

Example two: as shown in FIG. 2, in the system for synthesizing real-time images by using optical body surface motion signals based on the first embodiment, the 4D medical image comprises a 4D image obtained by direct shooting, or a synthesized 4D image is reconstructed.

The directly capturing the acquired 4D image specifically includes directly capturing a 4D image before radiotherapy, such as 4DCT, 4DMR, 4DPET, 4DCBCT, and the like.

Before radiotherapy, the mapping relation between the optical body surface data and the directly shot 4D image is utilized, or the mapping relation between the directly shot 4DCBCT body surface data and the directly shot 4DCBCT image extracted based on other modes (such as body surface reflecting blocks, and in-vivo markers such as diaphragm movement and implantation markers) is used for being associated with the optical body surface signal data obtained in the radiotherapy process, and mapping and displaying from the body surface to the in-vivo structure are realized.

And reconstructing the synthesized 4D image, further comprising a 4D image reconstruction and synthesis unit, and acquiring a CBCT image before treatment including the CBCT image not limited to positioning. The CBCT images used for positioning are taken before each day of treatment.

And meanwhile, synchronous body surface data are acquired, each angle projection of the CBCT is divided into different time phases, a deformation field of an anatomical structure is obtained by utilizing a prior respiratory motion model obtained in 4DCT, the synchronous optical body surface data are associated with the respiratory time phases, reconstruction based on sparse projection data is carried out on each time phase projection of the CBCT, and the body surface data are associated with an in-vivo anatomical structure image of the same respiratory time phase.

Further, the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) generation of a prior respiratory motion model: registering each time phase of the four-dimensional CT with the four-dimensional CT at a certain moment as a reference, and decomposing the obtained deformation field (deformation vector field) into a weighted sum of principal components;

and (4) correlating instantaneous kV two-dimensional projection original data of the 3D-CBCT with a motion signal of the 4D-CT. The problem that the number of instantaneous kV two-dimensional projections is insufficient is solved by using the 4D-CT priori anatomical structure of the patient, and meanwhile anatomical information of the treatment day or the day closest to the treatment day reflected by the kV two-dimensional projections is reserved. The specific method comprises the following steps:

first, with a certain phase of 4D-CT as a reference (I0), the new image can be represented as:

I(I,j,k)＝F(I0,D)＝I0(i+Dx(I,j,k),j+Dy(I,j,k),k+Dz(I,j,k))

wherein (I, j, k) is the voxel position and D is the new image I relative to I₀Dx, Dy, Dz are the components of the deformation field D in the x, y, z directions, respectively. Calculating the phase of each 4D-CT and I₀The average deformation of the deformation is calculatedAnd principal component analysis is carried out to obtain the first three principal components D of the deformation₁、D₂And D₃I is then relative to I₀The deformation field D of (a) can be simplified as:

w1, W2, and W3 are weights of the respective principal components. Searching for a kV two-dimensional projection (P) of a certain time phase, the 4D image of this time phase is then relative to I₀Should satisfy DRR (F (I)₀,D))＝P

Where DRR represents a digitally reconstructed projection of I. The weight of each deformation principal component can be obtained by solving the above formula through gradient descent, and the estimation of the 4D image is obtained. And subsequently, the deformation field can be further finely adjusted by adopting a B-spline substrate, so that the I and the simultaneous phase kV two-dimensional projection have better consistency.

(2) Sequencing the body surface data: the method comprises the steps of dividing a body surface contour from each time phase of four-dimensional CT, calculating the difference and gradient between the body surfaces of other time phases and a reference body surface by taking the body surface at a certain moment (including but not limited to end-expiratory or end-inspiratory) as the reference body surface, and correlating the difference and gradient statistical average value of the body surfaces of the same time phase with the respiratory time phase.

(3) Reconstructing sparse projection data based on CBCT: according to optical body surface data synchronously acquired in the CBCT scanning process, sequencing CBCT two-dimensional projections according to respiration; calculating the weight of each principal component in the prior respiratory motion model according to the fact that the two-dimensional digital reconstructed image calculated after the CBCT deformation is approximately consistent with the actually acquired CBCT two-dimensional projection; and determining a deformation field D according to the weight, and obtaining a reconstructed and synthesized 4D image according to the deformation field D and the reference image.

Sparse projections refer to a small number of projections. The reconstruction is poor due to the small number of projections belonging to one phase. The optimal weight of each principal component in the prior breathing motion model can be calculated by adopting an optimization algorithm, such as gradient reduction and the like, so that the two-dimensional digital reconstructed image calculated after the CBCT deformation is approximately consistent with the actually acquired CBCT two-dimensional projection; approaching agreement means that the error is within a set tolerable threshold, which may be adjusted.

The CBCT image obtained according to the deformation field and the reference image represents the current anatomical structure of the patient, so that the anatomical structure is not distorted during treatment of the patient, and the quality of the reconstructed image is improved. The method needs a short period, and the 4D image can be reconstructed and synthesized only by one period.

Further, in the application stage, real-time optical body surface signals are collected during treatment and are associated with the obtained body surface data before treatment, and a 4D image of the in-vivo anatomical structure is further obtained through the mapping relation between the body surface data and the in-vivo anatomical structure image in the same breathing phase. Namely, real-time optical body surface signals are collected during treatment to be associated and mapped with the reconstructed body surface data of the patient treatment 4D image, and the corresponding internal dynamic anatomical structure information is displayed, so that virtual perspective is realized.

Furthermore, in the treatment process, a two-dimensional projection image of a patient can be obtained in an X-ray imaging mode, a corresponding respiratory time phase is determined according to a synchronously collected optical body surface, and a four-dimensional image is predicted by adopting the 4D image reconstruction and synthesis method, so that digital reconstruction projection images at the same time phase and the same angle are generated; the reconstructed projection image is compared with a two-dimensional projection image of the patient obtained by means of X-ray imaging, and the accuracy of the method is verified.

Further, in the treatment process, a two-dimensional projection image of the patient is obtained in an X-ray imaging mode, the collected projection image is used as correction data, the follow-up result is improved and optimized, the respiratory time phase is determined through the synchronously collected optical body surface, the 4D image is reconstructed through the 4D image reconstruction and synthesis unit to obtain a synthesized image of the patient in the day in the specific process of the 4D image of the patient in the day, at the moment, the deformation field of the in-vivo anatomical structure is required to meet the condition that the two-dimensional digital reconstruction projection calculated after the deformation field is acted on the reference image is approximately consistent with the two-dimensional projection image which is really collected, if the deviation is large, the deformation field meeting the condition is calculated, and then the model is retrained.

Example three: as shown in fig. 2, based on the first embodiment, the system for synthesizing a real-time image by using optical body surface motion signals further includes a 4D image reconstruction synthesizing unit, wherein a Deformation (DVF) of an in-vivo anatomical structure corresponds to a change of a boundary of the in-vivo anatomical structure, the boundary of the in-vivo anatomical structure corresponds to a change of a body surface, and an in-vivo corresponding deformation field is deduced from the deformation of the body surface.

Furthermore, the relation between the body surface deformation and the body internal deformation is mined from historical data, and complex physical modeling is avoided. The specific training process is as follows: the method comprises the steps of collecting historical four-dimensional CT images of a patient, segmenting a body surface contour, taking body surface deformation or the four-dimensional CT images or the body surface contour as the input of a model, taking CT of a certain time phase as a reference image, and registering the reference image to CT of other time phases to obtain the model which is output as a time sequence of an in-vivo anatomical structure variation field, namely the in-vivo anatomical structure variation field is the prediction target of the model. The model can adopt a classical convolution neural network and a function with similar function; during treatment, real-time optical body surface data is input into the model to predict the deformation field of the in-vivo anatomical structure, and the four-dimensional image at the moment can be obtained by acting on the reference image.

The deformation field has more parameters, and the dimension reduction is beneficial to the rapid training of the model, for example, PCA (principal component analysis), ICA (independent component analysis) and IsoMap methods are adopted to decompose the deformation field into the weighted sum of principal components, including but not limited to using a priori respiratory motion model, the prediction target of the model is converted into the weights of different principal components, and the deformation can be obtained by calculating the weights.

And (3) a model application stage: optical body surface data of a patient in the treatment process are collected, converted into a form associated with body surface data segmented in a CT image in a training stage, input into a model, a deformation field of an anatomical structure in a body is predicted, and the image at the moment can be obtained by acting on a reference image. The model is verified by experimental data, the body surface deformation is shown in figure 3, an example of the predicted principal component weight is shown in figure 4, and a loss function in the model training process is shown in figure 5.

And (3) testing the effect of the model on ten samples according to the verification result of the model, and obtaining a better result with statistical difference before model prediction by taking the DICE (Daiss similarity coefficient) similarity coefficient of the lung, the DICE similarity coefficient of the target area and the distance of the tumor center point as evaluation indexes.

Furthermore, in the treatment process, a two-dimensional projection image of a patient can be obtained in an X-ray imaging mode, a corresponding respiratory time phase is determined according to the synchronously collected optical body surface, and a four-dimensional image is predicted through the model, so that digital reconstruction projection images with the same time phase and the same angle are generated; and (4) reconstructing the projection image, and comparing the projection image with a two-dimensional projection image of the patient obtained by means of X-ray imaging to verify the accuracy of the method.

Furthermore, in the treatment process, a two-dimensional projection image of the patient is obtained in an X-ray imaging mode, the collected projection image is used as correction data, the follow-up result is improved and optimized, the respiratory time phase is determined through the synchronously collected optical body surface, the deformation field of the internal anatomical structure is obtained through the model, the two-dimensional digital reconstruction projection calculated after the deformation field acts on the reference image is approximately consistent with the actually collected two-dimensional projection image, the weight of the deformation field meeting the condition is calculated, and then the model is retrained. Approaching agreement means that the error is within a set tolerable threshold, which may be adjusted.

Example four: a method for synthesizing real-time images by using optical body surface motion signals corresponds to the first to the third phases of the above embodiments, and the explanation part is specifically referred to above. Acquiring a real-time body surface signal; and obtaining the internal anatomical structure information displayed by the corresponding time 4D medical image by acquiring the body surface signal according to the mapping relation between the body surface signal and the internal anatomical structure information displayed by the 4D medical image.

Further, the 4D medical image includes a 4D image acquired by direct photographing, or a reconstructed synthesized 4D image.

Further, acquiring CBCT before treatment and simultaneously acquiring synchronous body surface data, dividing each angle projection of the CBCT into different time phases, associating the synchronous optical body surface data with a respiratory time phase, reconstructing each time phase projection of the CBCT based on sparse projection data, and associating the body surface data with an in-vivo anatomical structure image of the same respiratory time phase.

Further, the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) generation of a prior respiratory motion model: registering each time phase of the four-dimensional CT with the four-dimensional CT at a certain moment as a reference, and decomposing the obtained deformation field into a weighted sum of main components; (2) sequencing the body surface data: a method for obtaining a respiratory time phase inferred from the instantaneous body surface by segmenting a body surface contour from each time phase of the four-dimensional CT and associating the body surface contour with the respiratory time phase; (3) reconstructing sparse projection data based on CBCT: according to optical body surface data synchronously acquired in the CBCT scanning process, sequencing CBCT two-dimensional projections according to respiration; calculating the weight of each principal component in the prior respiratory motion model according to the fact that the two-dimensional digital reconstructed image calculated after the CBCT deformation is approximately consistent with the actually acquired CBCT two-dimensional projection; and determining a deformation field according to the weight, and obtaining a reconstructed and synthesized 4D image according to the deformation field and the reference image.

Furthermore, real-time optical body surface signals are collected during treatment and are associated with the obtained body surface data before treatment, and then 4D images of the in-vivo anatomical structures are further obtained through the mapping relation between the body surface data and the in-vivo anatomical structure images at the same breathing time phase.

Further, in the treatment process, a two-dimensional projection image of a patient is obtained in an X-ray imaging mode, the collected projection image is used as correction data, the follow-up result is improved and optimized, the respiratory time phase is determined through the synchronously collected optical body surface, then a 4D image is reconstructed through a 4D image reconstruction synthesis unit, at the moment, the deformation field of the in-vivo anatomical structure is required to meet the condition that the two-dimensional digital reconstruction projection calculated after the deformation field acts on the reference image is approximately consistent with the actually collected two-dimensional projection image, if the deviation is large, the deformation field meeting the condition is calculated, and then the model is retrained.

Further, the boundary of the in-vivo anatomical structure corresponds to a change in the body surface, and the deformation of the in-vivo anatomical structure is inferred from the deformation of the body surface.

Further, the relation between the body surface deformation and the body internal deformation is mined from the historical data, and the specific process is as follows: collecting a four-dimensional CT image of a patient, segmenting a body surface contour, taking the body surface deformation or the four-dimensional CT image or the body surface contour as the input of a model, taking the CT of a certain time phase as a reference image, and registering the reference image to the CT of other time phases to obtain the model which is output as a time sequence of a structural deformation field of the in-vivo anatomy; during treatment, real-time optical body surface data is input into the model to predict the deformation field of the in-vivo anatomical structure, and the image at the moment can be obtained by acting on the reference image.

Further, in the treatment process, a two-dimensional projection image of the patient is obtained in an X-ray imaging mode, the acquired projection image is used as correction data, the follow-up result is improved and optimized, the respiratory time phase is determined through the synchronously acquired optical body surface, at the time, the deformation field of the internal anatomical structure is required to meet the condition that the two-dimensional digital reconstruction projection calculated after the deformation field acts on the reference image is approximately consistent with the actually acquired two-dimensional projection image, the weight of the deformation field meeting the condition is calculated, and then the model is retrained.

Furthermore, in the treatment process, a two-dimensional projection image of a patient is obtained in an X-ray imaging mode, a corresponding respiratory time phase is determined according to the synchronously collected optical body surface, and a four-dimensional image is predicted, so that digital reconstruction projection images of the same time phase and the same angle are generated; and (4) reconstructing the projection image, and comparing the projection image with a two-dimensional projection image of the patient obtained by means of X-ray imaging to verify the accuracy of the method.

It is to be noted that different embodiments may produce different advantages, and in different embodiments, any one or combination of the above advantages may be produced, or any other advantages may be obtained.

Having thus described the basic concept, it will be apparent to those skilled in the art that the foregoing detailed disclosure is to be regarded as illustrative only and not as limiting.

Additionally, the order in which the elements and sequences of the process are described, the use of letters or other designations herein is not intended to limit the order of the processes and methods of the invention unless otherwise indicated by the claims. While various presently contemplated embodiments of the invention have been discussed in the foregoing disclosure by way of example, it should be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements that are within the spirit and scope of the embodiments of the invention.

Similarly, it should be noted that in the preceding description of embodiments of the invention, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the embodiments. This method of disclosure, however, is not intended to suggest that the claimed subject matter requires more features than are expressly recited in the claims.

Finally, it should be understood that the embodiments described herein are merely illustrative of the principles of embodiments of the present invention. Other variations are possible within the scope of the invention. Thus, by way of example, and not limitation, alternative configurations of embodiments of the present invention can be viewed as being consistent with the teachings of the present invention. Accordingly, the embodiments of the invention are not limited to only those explicitly described and depicted.