阅读说明：本技术 确定和显示心脏消融球囊的3d地点和朝向 (Determining and displaying 3D location and orientation of cardiac ablation balloon ) 是由 J·萨拉 S·库哈特 于 2018-04-16 设计创作，主要内容包括：一种使用单平面荧光检查法在预定义3D空间内的心脏区域中的心脏消融球囊进行3D可视化的方法,该方法包括：(1)将球囊放置、充气和定位到该区域中,球囊具有不透射线的标记和中央导管部分；(2)从第一角度捕获第一视角图像；(3)从不同角度捕获第二视角图像；(4)选择在两个图像的心脏-呼吸相位之间具有最小差异的第一视角图像和第二视角图像；(5)在两个图像中识别标记；(6)在两个图像中放置朝向标记,其中中央导管部分与投影的球囊图像在距标记最远处相交；(7)分别关联两个图像中的标记和朝向标记；(8)确定3D球囊地点和朝向；以及(9)将3D球囊模型插入到预定义的空间中以生成3D可视化。(A method for 3D visualization of a cardiac ablation balloon in a region of the heart within a predefined 3D space using single plane fluoroscopy, the method comprising: (1) placing, inflating and positioning a balloon into the area, the balloon having radiopaque markers and a central catheter portion; (2) capturing a first perspective image from a first angle; (3) capturing a second perspective image from a different angle; (4) selecting a first view image and a second view image having a minimum difference between the cardiac-respiratory phases of the two images; (5) identifying a marker in both images; (6) placing an orientation marker in both images, wherein the central catheter portion intersects the projected balloon image furthest from the marker; (7) respectively associating the markers and the orientation markers in the two images; (8) determining a 3D balloon location and orientation; and (9) inserting the 3D balloon model into the predefined space to generate a 3D visualization.)

1. A method for generating and displaying a 3D visualization of a cardiac ablation balloon in a region of a living heart within a predefined 3D space, the method using a single plane fluoroscopic image and comprising:

■ placing, inflating and positioning a balloon into the area, the balloon having radiopaque site markers and a central catheter portion;

■ capturing a string of digitized 2D images of the area from a first perspective of a fluoroscope positioned at a first angle;

■ capturing a string of digitized 2D images of the region from the fluoroscope positioned at a second angle different from the first angle;

■ selecting a first perspective image and a second perspective image from the string such that a difference between measurements of the cardiac-respiratory phase of the selected first perspective image and the second perspective image is minimal;

■ identifying location markers in each of the two selected images;

■ placing first and second orientation markers in the selected first and second perspective images, respectively, wherein the central catheter portion intersects the projected image of the inflated balloon at a point furthest from the location marker;

■ associating the location marker and the second orientation marker in the selected second perspective image with the location marker and the first orientation marker in the selected first perspective image;

■ determining the 3D location and orientation of the balloon in the region using the selected first and second perspective images;

■ inserting the 3D balloon model into a predefined space to generate a 3D visualization based on the determined location and orientation, and

■ display the 3D visualization on a display device,

so that the user can visualize where within the region cardiac ablation was applied after removing the balloon from the site where the ablation occurred.

2. The method of claim 1, wherein the cardiac ablation balloon ablates cardiac tissue using optical energy.

3. The method of claim 1, wherein the balloon ablates cardiac tissue using radiofrequency energy.

4. The method of claim 1, wherein the balloon ablates cardiac tissue using focused ultrasound energy.

5. The method of claim 1, wherein the balloon is a cryoballoon that ablates heart tissue using refrigeration.

6. The method of claim 1, wherein the step of displaying comprises displaying the projected image of the 3D visualization on a 2D fluoroscopic image of the area.

7. The method of claim 1, wherein the step of displaying comprises displaying the 3D visualization in a 3D rotatable perspective format.

8. The method of claim 1, wherein selecting the first view image and the second view image comprises determining a cardiac phase and a respiratory phase for each captured first view image and second view image.

9. The method of claim 8, wherein selecting the first perspective image and the second perspective image comprises:

■ identifying candidate images in the first and second image strings that satisfy the cardiac phase criterion and the respiratory phase criterion, and

■ the first perspective image and the second perspective image are selected from the candidate images using a similarity criterion based on the cardiac phase and the respiratory phase of the candidate images.

10. The method of claim 8, wherein the cardiac phase of each image is estimated using an R-wave detector to identify R-waves and measure R-wave spacing.

11. The method of claim 10, wherein selecting the first perspective image and the second perspective image comprises:

■ identifying candidate images in the first and second image strings that satisfy the cardiac phase criterion and the respiratory phase criterion, and

■ the first perspective image and the second perspective image are selected from the candidate images using a similarity criterion based on the cardiac phase and the respiratory phase of the candidate images.

12. The method of claim 11, wherein the estimate of the cardiac phase of the image is a percentage of time spaced along the R-wave at which the image was captured.

13. The method of claim 12, wherein the cardiac phase criterion is met if the estimated cardiac phase of the image is between 30% and 80%.

14. The method of claim 8, wherein the breathing phase of each image in the series of images is estimated by:

■ determining the expiration/inspiration range based on the location of the radiopaque object in the series of images, and

■ determine the percentage of the expiration/inspiration range along the location of the radiopaque object in the image.

15. The method of claim 14, wherein the radiopaque object is a site marker.

16. The method of claim 15, wherein selecting the first perspective image and the second perspective image comprises:

■ identifying candidate images in the first and second image strings that satisfy the cardiac phase criterion and the respiratory phase criterion, and

■ the first perspective image and the second perspective image are selected from the candidate images using a similarity criterion based on the cardiac phase and the respiratory phase of the candidate images.

17. The method of claim 16, wherein the respiratory phase criterion is met when the respiratory phase of the image is between 0% and 20% of maximum exhalation.

18. The method of claim 9, wherein the step of selecting further comprises:

■ for each pair of candidate first perspective images I_iAnd candidate second perspective image I_jCalculating an image I_iAnd I_jAnd image I_iAnd I_jThe sum of the absolute values of the differences between the respiratory phases of (a); and

■ selects the smallest pair of first perspective image and second perspective image.

19. The method of claim 18, wherein the cardiac phase difference and the respiratory phase difference are given relative weights prior to summing.

20. The method of claim 1, wherein all steps except the placing, inflating and positioning steps occur during cardiac ablation.

21. The method of claim 1, wherein determining the 3D location and orientation of the cardiac ablation balloon comprises determining a 3D location of location markers and final orientation markers from the selected first view image and second view image using a back projection calculation.

22. The method of claim 21, wherein the fluoroscope includes a detector defining a detector plane and an X-ray source defining a source point, and determining the 3D location and orientation of the cardiac ablation balloon further comprises:

■ generates a first plane containing three points defined by:

■ a location mark and a first orientation mark of the first view image in the detector plane, and

■ source points;

■, a second plane is generated that contains three points, which are defined by:

■ in the detector plane, a location marker and a second orientation marker of the second view angle image, and

■ source points;

■ determining the intersection of the first plane and the second plane;

■ from the 3D location of the location marker on the intersection line, and

■ determining the orientation of the balloon from the 3D location of the determined final orientation marker.

23. A method for generating and displaying a 3D visualization of a cardiac ablation balloon in a region of a living heart within a predefined 3D space, the method using a single plane fluoroscopic image and comprising:

■ placing, inflating and positioning a balloon having radiopaque site markers and a radiopaque central catheter portion;

■ capturing a first perspective digitized 2D image of the region from a first fluoroscope positioned at a first angle;

■ capturing a second perspective digitized 2D image of the area from a second fluoroscope positioned at a second angle different from the first angle;

■ identifying location markers in each image;

■ placing first and second orientation markers in the first and second perspective images, respectively, wherein the central catheter portion intersects the projected image of the inflated balloon at a point furthest from the location marker;

■ associating the location marker and the second orientation marker in the second perspective image with the location marker and the first orientation marker in the first perspective image;

■ determining the 3D location and orientation of the balloon in the region using the selected first and second perspective images;

■ inserting the 3D balloon model into a predefined space to generate a 3D visualization based on the determined location and orientation, and

■ display the 3D visualization on a display device,

so that the user can visualize where within the region cardiac ablation was applied after removing the balloon from the site where the ablation occurred.

24. The method of claim 23, wherein the first fluoroscope and the second fluoroscope are the same fluoroscope, and the second perspective image is captured after the first perspective image is captured.

25. The method of claim 24, wherein:

■ capturing the first perspective image includes capturing a first image string and selecting the first perspective image from the first image string, and

■ capturing the second perspective image includes capturing a second image string and selecting the second perspective image from the second image string.

26. The method of claim 25, further comprising determining a cardiac phase and a respiratory phase for each captured first view image and second view image.

27. The method of claim 26, wherein selecting the first perspective image and the second perspective image comprises:

■ identifying candidate images in the first and second image strings that satisfy the cardiac phase criterion and the respiratory phase criterion, and

■ the first perspective image and the second perspective image are selected from the candidate images using a similarity criterion based on the cardiac phase and the respiratory phase of the candidate images.

28. The method of claim 23, wherein determining the 3D location and orientation of the cardiac ablation balloon comprises determining a 3D location of location markers and final orientation markers from the selected first view and second view images using a back projection calculation.

29. The method of claim 28, wherein the fluoroscope includes a detector defining a detector plane and an X-ray source defining a source point, and determining the 3D location and orientation of the cardiac ablation balloon further comprises:

■ generates a first plane containing three points defined by:

■ a location mark and a first orientation mark of the first view image in the detector plane, and

■ source points;

■, a second plane is generated that contains three points, which are defined by:

■ in the detector plane, a location marker and a second orientation marker of the second view angle image, and

■ source points;

■ determining the intersection of the first plane and the second plane;

■ from the 3D location of the location marker on the intersection line, and

■ determining the orientation of the balloon from the 3D location of the determined final orientation marker.

30. A method for generating and displaying a 3D visualization of a cardiac ablation balloon in an area of a living heart within a predefined 3D space, the balloon having radiopaque site markers and a radiopaque central catheter portion, the balloon having been placed, inflated and positioned in the area, the method using a single plane fluoroscopic image and comprising:

■ capturing a string of digitized 2D images of the area from a first perspective of a fluoroscope positioned at a first angle;

■ capturing a string of digitized 2D images of the region from a fluoroscope positioned at a second angle different from the first angle;

■ selecting a first perspective image and a second perspective image from the string such that a difference between measurements of the cardiac-respiratory phase of the selected first perspective image and the second perspective image is minimal;

■ identifying location markers in each of the two selected images;

■ placing first and second orientation markers in the selected first and second perspective images, respectively, wherein the central catheter portion intersects the projected image of the inflated balloon at a point furthest from the location marker;

■ associating the location marker and the second orientation marker in the selected second perspective image with the location marker and the first orientation marker in the selected first perspective image;

■ determining the 3D location and orientation of the balloon in the region using the selected first and second perspective images;

■ inserting the 3D balloon model into a predefined space to generate a 3D visualization based on the determined location and orientation, and

■ display the 3D visualization on a display device,

so that the user can visualize where within the region cardiac ablation was applied after removing the balloon from the site where the ablation occurred.

31. The method of claim 30, wherein determining the 3D location and orientation of the cardiac ablation balloon comprises determining a 3D location of location markers and final orientation markers from the selected first view image and second view image using a back projection calculation.

32. The method of claim 31, wherein the fluoroscope includes a detector defining a detector plane and an X-ray source defining a source point, and determining the 3D location and orientation of the cardiac ablation balloon further comprises:

■ generates a first plane containing three points defined by:

■ a location mark and a first orientation mark of the first view image in the detector plane, and

■ source points;

■, a second plane is generated that contains three points, which are defined by:

■ in the detector plane, a location marker and a second orientation marker of the second view angle image, and

■ source points;

■ determining the intersection of the first plane and the second plane;

■ from the 3D location of the location marker on the intersection line, and

■ determining the orientation of the balloon from the 3D location of the determined final orientation marker.

Technical Field

The present invention relates generally to the field of medical fluoroscopy and more specifically to the field of cardiac ablation using balloon catheters within a living heart.

Background

In recent years, large area ablation of cardiac tissue using balloons has been developed to replace point-by-point ablation procedures. Several types of cardiac ablation balloon catheters have been introduced. Including cryoballoons, which use freezing (sometimes referred to as cryoenergy) to ablate tissue, radiofrequency thermal balloons, which use radiofrequency energy to ablate, ultrasonic balloons, which deliver focused ultrasound energy to tissue, and laser balloons, which use light energy as the means of ablation.

The use of cardiac ablation balloon catheters for treating patients with atrial fibrillation has become an important medical procedure, and it is estimated that in 2016, there are more than 80000 such procedures worldwide. This common tachyarrhythmia (atrial fibrillation) is often triggered by ectopic foci in and around the pulmonary veins. Prior to such treatment using a cardiac ablation balloon, ablation is performed using a point-by-point ablation strategy to electrically isolate the pulmonary veins.

A major drawback of using cardiac ablation techniques is that the electrocardiograph performing such a procedure has no good way to visualize where ablation is applied after it has occurred. Since ablation is typically performed at more than one site of the heart (e.g., there are four pulmonary veins), it is important and very useful for the electrophysiologist to be able to visually reference the geometry of the entire procedure as it is performed. The present invention is a method of providing such capability to a physician both during and after a procedure (by virtue of stored records).

Some of the techniques used in the inventive methods disclosed herein relate to method steps applicable to a method of rapidly generating a 3D map of cardiac parameters in a live cardiac region using a single plane fluoroscopic image, as disclosed in co-pending U.S. patent application entitled "Rapid 3D Medical ParameterMapping" filed on 13.4.2017, application serial No.15/487,245 (referred to herein as Sra et al).

Object of the Invention

It is an object of the present invention to provide a method by which a cardiologist can visualize in three dimensions, after removing a cardiac ablation balloon, where tissue in a living heart has been ablated using the ablation balloon.

It is another object of the present invention to provide such visualization using only single plane fluoroscopic images to provide data from which a visualization is generated.

It is another object of the invention to provide such visualization in a manner that does not increase the length of time of the cardiac ablation procedure.

It is a further object of the invention to provide such visualization in a form that can be stored for later use.

It is a further object of the inventive method to provide a convenient and useful visual display approach for cardiologists, including an approach in which cardiologists can interact with a display device to enhance the insight provided.

These and other objects of the present invention will be apparent from the following description and the accompanying drawings.

Disclosure of Invention

The invention is a method for 3D visualization of a cardiac ablation balloon in a region of a living heart within a predefined 3D space using single plane fluoroscopy images. The method comprises the following steps: (1) placing, inflating and positioning a balloon into the area, the balloon having radiopaque site markers and a central catheter portion; (2) capturing a string of first perspective digitized 2D images of the area from a fluoroscope positioned at a first angle; (3) capturing a string of digitized 2D images of a second perspective of the area from a fluoroscope positioned at a second angle different from the first angle; (4) selecting images of the first and second view angles from the series such that a difference between measurements of the cardiac-respiratory phase of the selected images of the first and second view angles is minimal; (5) identifying a location marker in each of the two selected images; (6) placing first and second orientation markers in the selected first and second perspective images, respectively, wherein the central catheter portion intersects the projected image of the inflatable balloon at a point furthest from the location marker; (7) associating the location marker and the second orientation marker in the selected second perspective image with the location marker and the first orientation marker in the selected first perspective image; (8) determining a 3D location and orientation of the balloon in the region using the selected first and second perspective images; (9) inserting the 3D balloon model into a predefined space to generate a 3D visualization based on the determined location and orientation; and (10) displaying the 3D visualization on a display device so that a user can visualize where within the region cardiac ablation was applied after the balloon is removed from the location where ablation occurred. In some embodiments of the present methods, the cardiac ablation balloon ablates cardiac tissue using optical energy, in some embodiments the balloon ablates cardiac tissue using radiofrequency energy, in some embodiments the balloon ablates cardiac tissue using focused ultrasound energy, and in some embodiments the balloon ablates cardiac tissue using cryoablation.

In some preferred embodiments, the displaying step comprises displaying the projection image of the 3D visualization on a 2D fluoroscopic image of the area, and in some embodiments, the displaying step comprises displaying the 3D visualization in a 3D rotatable perspective format.

In some preferred embodiments, selecting the first perspective image and the second perspective image comprises determining a cardiac phase and a respiratory phase for each captured first perspective image and second perspective image. In some of these embodiments, selecting the first perspective image and the second perspective image comprises: (a) identifying candidate images in the first and second image strings that satisfy a cardiac phase criterion and a respiratory phase criterion, and (b) selecting a first perspective image and a second perspective image from the candidate images using a similarity criterion based on the cardiac phase and the respiratory phase of the candidate images.

In some highly preferred embodiments, an R-wave detector is used to identify R-waves and measure the R-wave spacing to estimate the cardiac phase of each image. In some of these embodiments, the estimate of the cardiac phase of the image is a percentage of time spaced along the R-wave at which the image was captured, and in some embodiments, the cardiac phase criterion is satisfied if the estimated cardiac phase of the image is between 30% and 80%.

In some highly preferred embodiments, the respiratory phase of each image in the series of images is estimated by: (a) determining an expiration/inspiration range from the location of the radiopaque object in the image string, and (b) determining a percentage of the expiration/inspiration range along the location of the radiopaque object in the image. In some of these embodiments, the radiopaque object is a location marker, and in some embodiments, the respiratory phase criterion is met when the respiratory phase of the image is between 0% and 20% of maximum exhalation.

In some highly preferred embodiments of the method of the present invention, the selecting step further comprises: (a) for each pair of candidate first view images I_iAnd candidate second perspective image I_jCalculating an image I_iAnd I_jThe absolute value of the difference between the cardiac phases and image I_iAnd I_jAnd (b) selecting a pair of first view and second view images having the smallest sum. In some of these embodiments, the cardiac phase difference and the respiratory phase difference are given relative weights prior to summing.

In a highly preferred embodiment of the inventive method, in addition to the placement, inflation and positioning steps occurring during cardiac ablation, a 3D visualization of the cardiac ablation balloon in a region of the living heart within a predefined 3D space is generated and displayed using single plane fluoroscopy images.

In some highly preferred embodiments, determining the 3D location and orientation of the cardiac ablation balloon comprises determining the 3D location of the location markers and final orientation markers from the selected first and second perspective images using a back projection calculation. In some of these embodiments, the fluoroscope includes a detector defining a detector plane and an X-ray source defining a source point, and determining the 3D location and orientation of the cardiac ablation balloon further comprises: (a) generating a first plane containing three points defined by a first orientation marker and a location marker of a first perspective image in the detector plane and a source point; (b) generating a second plane containing three points defined by a second orientation marker and a location marker of a second perspective image in the detector plane and the source point; (c) determining an intersection of the first and second planes; (d) determining a location of the balloon from the 3D location of the location marker on the intersection line; and (e) determining the orientation of the balloon according to the determined 3D location of the final orientation marker.

In another aspect of the invention, the method comprises: (a) placing, inflating and positioning a balloon having a radiopaque site marker and a radiopaque central catheter portion;

(b) capturing a first perspective digitized 2D image of the area from a first fluoroscope positioned at a first angle; (c) capturing a second perspective digitized 2D image of the area from a second fluoroscope positioned at a second angle different from the first angle; (d) identifying a location marker in each image; (e) placing first and second orientation markers in the first view and second view images, respectively, wherein the central catheter portion intersects the projected image of the inflated balloon at a point furthest from the location marker; (f) associating the location marker and the second orientation marker in the second perspective image with the location marker and the first orientation marker in the first perspective image; (g) determining a 3D location and orientation of the balloon in the region using the selected first and second perspective images; (h) inserting a 3D balloon model into a predefined space to generate a 3D visualization based on the determined location and orientation; and (i) displaying the 3D visualization on a display device, whereby a user can visualize where within the region cardiac ablation was applied after the balloon is removed from the location where ablation occurred. In some of these embodiments, the first and second fluoroscopes are the same fluoroscope, and the second perspective image is captured after the first perspective image is captured.

In yet another aspect, the invention is a method for generating and displaying 3D visualizations of a cardiac ablation balloon in an area of a living heart within a predefined 3D space, the balloon having radiopaque site markers and a radiopaque central catheter portion, the balloon having been placed, inflated, and positioned in the area. The method uses a single plane fluoroscopic image and comprises the steps of: (i) capturing a string of first perspective digitized 2D images of the area from a fluoroscope positioned at a first angle; (ii) capturing a string of digitized 2D images of a second perspective of the area from a fluoroscope positioned at a second angle different from the first angle; (iii) selecting a first view image and a second view image from the series such that a difference between measurements of cardiac-respiratory phases of the selected first view image and the second view image is minimized; (iv) identifying a location marker in each of the two selected images; (v) placing first and second orientation markers in the selected first view and second view images, respectively, wherein the central catheter portion intersects the projected image of the inflatable balloon at a point furthest from the location marker; (vi) associating the location marker and the second orientation marker in the selected second perspective image with the location marker and the first orientation marker in the selected first perspective image; (vii) determining a 3D location and orientation of the balloon in the region using the selected first and second perspective images; (viii) inserting the 3D balloon model into a predefined space to generate a 3D visualization based on the determined location and orientation; and (ix) displaying the 3D visualization on a display device, whereby a user can visualize the location within the region where the cardiac ablation was applied after the balloon is moved away from the location where the ablation occurred.

The terms "image" and "frame" are used interchangeably herein and, unless otherwise specified, refer to a collection of digitized data captured from a conventional fluoroscope. An image or frame is a two-dimensional array of pixels (picture elements) each having an associated image intensity value.

The terms "X-ray" and "fluoroscope" are used interchangeably herein.

As used herein, the term "string of images" refers to a collection of sequential fluoroscopic images captured over a period of time, the frequency of which is typically determined by the frame rate setting of the fluoroscope.

The terms "location" and "position" may be used interchangeably herein to refer to 3D coordinates of an object, such as a radiopaque marker.

As used herein, the term "expiration/inspiration range" refers to the distance between the extreme 2D positions of a radiopaque object as it moves from image to image in an image sequence.

As used herein, the term "cardiac-respiratory phase" refers to the phase of a combination of cardiac and respiratory motion. Thus, as used herein, minimizing the difference between the cardiac-respiratory phases of the two images may also include minimizing a combination of measurements of both cardiac and respiratory phases.

The terms "method steps," "method elements," and "functional elements" or other similar terms may be used interchangeably herein to refer to portions of the methods of the present invention.

As used herein, the term "3D balloon model" refers to a three-dimensional computer image of a cardiac ablation balloon that includes shape and dimensional information corresponding to the actual cardiac ablation balloon device. The shape and dimensional information may be customizable such that a "3D balloon model" is adapted to represent more than one specific cardiac ablation balloon device. The cardiologist can also adjust the color, opacity, and shading of the 3D model to enhance the visualization effect.

Drawings

The present invention uses two X-ray images from different angles (i.e., View1 and View 2). In the drawings, the numbering convention used herein is to label such two View icons as N-1 and N-2 to indicate that the views are related to View1 and View2, respectively, when there are corresponding views for the two views.

Fig. 1 is a pictorial representation of an exemplary conventional X-ray machine (fluoroscope). The exemplary machine shown in fig. 1 is a GE Innova2100 system.

Fig. 2 illustrates an exemplary set of axes defining 3D coordinates of a protocol fluoroscopy suite. Each element in the kit has a position that can be described by coordinates in this coordinate system. Indicating the positive direction of each axis.

Fig. 3A to 3D are illustrations of a cardiac ablation balloon (in this case, a cryoballoon) placed in a living heart. (fig. 3A-3D approved for use by Medtronic inc. of Minneapolis, Minnesota.) fig. 3A shows an uninflated cryoballoon in the left atrium of the heart.

Fig. 3B shows the inflated cryoballoon prior to placement for a cryoablation procedure.

Fig. 3C shows the inflated cryoballoon at the sinus of the pulmonary vein for the location of cryoablation.

Fig. 3D shows the cryoballoon uninflated after ablation.

Figures 4-1 and 4-2 are representative X-ray images of the patient's chest taken with a cardiac ablation balloon (in this case a cryoballoon) at a location on the patient in the AP (anterior/posterior) and LAO (left anterior oblique) positions, respectively. Each of the two images is one image in the image string from a first angle (View 1) and one image in the image string from a second angle (View 2), respectively.

Fig. 5A is a schematic block diagram illustrating an embodiment of the inventive method for generating and displaying a model of a heart ablation balloon in a region of a living heart using single plane fluoroscopic images.

Fig. 5B is a schematic block diagram illustrating an alternative embodiment of the step of determining the 3D location and orientation of a cardiac ablation balloon in the inventive method of fig. 5A.

Fig. 5C is a schematic block diagram illustrating a second alternative embodiment of the step of determining the 3D location and orientation of a cardiac ablation balloon in the inventive method of fig. 5A.

FIG. 6 is a digitized signal S (t) from the R-wave detector_i) An exemplary time diagram of (a). This signal is used to derive cardiac phase information for each of the View1 and View2 images.

FIGS. 7-1 and 7-2 are graphs of exemplary y-position data for a cardiac ablation balloon (for use as location marker 71 in the images of FIGS. 4-1 and 4-2) for thirty (30) frames of the View1 string and thirty (30) frames of the View2 string, respectively. Note that FIGS. 7-1 and 7-2 are paired with FIGS. 8-1 and 8-2, respectively, and are thus on different pages, as are FIGS. 8-1 and 8-2.

FIGS. 8-1 and 8-2 are graphs of the y-position data of FIGS. 7-1 and 7-2, respectively, that have been smoothed and interpolated to generate a respiratory phase estimate for each image.

FIGS. 9-1 and 10-1 are graphs of respiratory and cardiac phases for thirty View1 frames and thirty View2 frames, respectively. The values of both cardiac phase and respiratory phase have been normalized to the 0-1 scale. Note that FIGS. 9-1 and 9-2 are paired with FIGS. 10-1 and 10-2, respectively, and are thus on different pages, as are FIGS. 10-1 and 10-2.

FIGS. 9-2 and 10-2 are graphs of respiratory and cardiac phases of View1 and View2 frames, respectively. In each such plot, frames satisfying the cardiac phase criterion are plotted, as well as frames satisfying the respiratory phase criterion, FIG. 9-2 for the View1 image and FIG. 10-2 for the View2 image. Such frames illustrate the determination of a set of candidate View1 and View2 frames for final selection as a pair of images from which to determine the 3D location of the cardiac ablation balloon using a back-projection calculation.

FIG. 11 is a schematic block diagram illustrating an embodiment of a method of selecting the best View1 and View2 frames from a set of candidate View1 and View2 frames.

Fig. 12A is a 3D perspective view of a 3D model of a cryoballoon in a region of a living heart as determined in examples presented herein.

Fig. 12B is a representative X-ray image of an overlay of the 3D model with the cryoballoon of fig. 12A. The opacity of the overlay is less than 100% to enhance visualization. The X-ray portion of FIG. 12B is the same as the "View 1" image of FIG. 4-1.

Fig. 13A is a 3D perspective view of four 3D models of cryoballoons that have applied ablation one after the other at four locations in a living heart. A first ablation is applied at the location indicated in fig. 12A. Fig. 13A shows a front/rear view.

Fig. 13B is a second 3D perspective view of the four 3D balloon models of fig. 13A. Fig. 13B shows a left lateral view.

Fig. 13C is a third 3D perspective view of the four 3D balloon models of fig. 13A. Fig. 13B shows a right lateral view.

Fig. 13D is a fourth 3D perspective view of the four 3D balloon models of fig. 13A. Fig. 13B shows a top view.

Fig. 14A is the same front/rear view as fig. 13A, and is disposed adjacent to fig. 14B for convenience.

FIG. 14B is a representative X-ray image with an overlay of the 3D fluoroscopic image of FIG. 14A placed thereon. The opacity of the overlay image is 100%. The X-ray image in FIG. 14B is slightly different from the X-ray image of FIG. 4-1; after all four ablation locations have been applied and after the stage of the fluoroscopic system has been translated to the right, X-ray images are taken.

Fig. 15A and 15B are the same as fig. 14A and 14B, respectively, except that the opacity of the 3D fluoroscopic image of fig. 15A has been reduced to enhance the visualization effect.

Detailed Description

FIG. 1 illustrates an exemplary conventional fluoroscopic system 10 for acquiring 2D fluoroscopic image data. The imaging process for conventional fluoroscopy involves an X-ray source 11, which X-ray source 11 sends an X-ray beam through a patient (not shown) on a table 12. An X-ray detector 13, which may be a flat panel detector or an image intensifier/camera assembly, receives X-rays transmitted through the patient and converts the X-ray energy into an image.

An X-ray source 11 and an X-ray detector 13 are mounted at opposite ends of the C-arm 8. The detector 13 may perform conversion using an X-ray detection layer that either emits light or releases electrons when excited by X-rays, and suitably using a photo-electron conversion layer (e.g., a photodiode or an electron collection layer) in which a charge signal proportional to the X-ray signal intensity in each picture element (pixel) is collected. The analog to digital conversion then produces a digital image. Whichever type of X-ray detector 13 is employed, the resulting digital image will then be processed, possibly stored and displayed on a screen 14. The control panel is shown at 15. The image may then be displayed on the computer display 14.

FIG. 2 illustrates an exemplary coordinate system for the fluoroscopic system 10. These three axes are shown in solid lines in fig. 2. A z-axis is defined from the X-ray source 11 to the center of the X-ray detector 13, wherein the X-ray beam is perpendicular and perpendicular to the table 12(AP position-front/back position). N (z)⁺) Oriented from the patient's chest (anterior)Section) definition, z^-As the back (posterior) of the patient. The X-ray table 12 defines an X-axis and a y-axis. The y-axis is parallel to the worktable and the positive direction (y)⁺) Towards the patient's head (above). The x-axis is perpendicular to both the y-axis and the z-axis, and the positive direction (x)⁺) Located on the left side of the patient. The intersection of the axes is located at the origin O (0,0,0) of the 3D space defined by the axes x, y and z. The control panel 15 is configured to translate the patient along all three axes (three translational degrees of freedom) as defined above.

As shown in fig. 1, the fluoroscope system 10 is also configured to rotate about three axes 7a, 8a, 9a (indicated by dotted lines) as another means of allowing the desired positioning of the patient in the field of view of the fluoroscope system 10 and providing sufficient space for medical personnel to perform the desired procedure. In the fluoroscopic system 10, the origin O is also the rotation center of these three rotational degrees of freedom, that is, the isocenter (the rotation center of the X-ray beam center ray) of the fluoroscopic system 10. The fluoroscope system 10 comprises a base 7 rotatable on the floor about an axis 7a, a C-shaped arm 8 rotatable about an axis 8a and an L-shaped arm 9 rotatable about an axis 9 a. Arrows 7r, 8r and 9r indicate the movements that these three rotational degrees of freedom may produce.

Note that the three axes x, y, z defining the coordinate system within the fluoroscopic system 10 do not have to be identical to the axes 7a, 8a, 9a, since rotation about these axes changes the relative positions of these axes with respect to the axes x, y, z. Of course, the coordinate systems are relative, and other coordinate systems may be used; the exemplary set of axes described above is not intended to be limiting. Moreover, not all fluoroscopic systems are configured with all translational and rotational degrees of freedom described in the exemplary fluoroscopic system 10, and this set of degrees of freedom is not intended to be limiting.

Fig. 3A to 3D are illustrations of a cardiac ablation balloon 20 in a region of a living heart. In this case, the cardiac ablation balloon 20 is a cryoballoon. In this illustrated sequence, a cryoballoon 20 (also referred to as 20u for uninflated and 20i for inflated) is placed, inflated and positioned in the left atrium 21 of the living heart for performing a cryoablation procedure. The cryoballoon 20 is a cardiac instrument that also includes a radiopaque central catheter 20c and location markers (not visible in the illustrations of fig. 3A-3D).

Fig. 3A shows the uninflated cryoballoon 20u in the left atrium 21. The central catheter 20c includes one end which is a looped end 20r shown in fig. 3A-3D in a pulmonary vein 23.

Fig. 3B shows the inflated cryoballoon 20i at the sinus 25 (entrance) of the pulmonary vein 23 prior to positioning for a cryoablation procedure. (sinus 25 is indicated by two examples of reference numeral 25.)

Fig. 3C shows the inflated cryoballoon 20i for cryoablation in a position at the sinus 25 of the pulmonary vein 23. The difference in shading in the pulmonary vein 23 to the right of the cryoballoon 20i illustrates that prior to cryoablation, the fluorescent contrast agent 27 is released from the catheter to verify that the cryoballoon 20i completely occludes the pulmonary vein 23 at the sinus 25. After such verification, the heart tissue is ablated where it comes into contact with the cryoballoon 20i, thereby forming a circumferential lesion at the desired location in the heart.

Fig. 3D shows the uninflated cryoballoon 20u after the ablation procedure. The looped end 20r of the catheter includes a plurality of electrodes that are used as mapping catheters after ablation to verify the effectiveness of the cryoablation procedure.

The method of the present invention involves the use of one or more programmable computers to perform image processing, signal processing, and other computational steps involved. Furthermore, a means of sensing heart rhythm (such as an R-wave detector with associated electrodes) may be required to supply a signal from which the cardiac phase of the monoplanar fluoroscopic image can be derived.

Figures 4-1 and 4-2 are representative X-ray images of a patient's chest using a cardiac ablation balloon 70 (in this case a cryoballoon 70 (as part of a cardiac catheter 72)) in place in the patient at the AP and LAO 20 (20 ° left), respectively. Each of the two images is one image of the image string from the first angle (View 1) and one image of the image string from the second angle (View 2), respectively. In fact, the image pairs shown in fig. 4-1 and 4-2 are the images selected as the best pairs among the example data for selection as described in method steps 31 to 51 in method embodiment 30 shown in fig. 5A described later.

The cryoballoon 70 includes radiopaque site markers 71 and a radiopaque central catheter portion 73, as indicated in fig. 4-1 and 4-2. These figures also illustrate two orientation markers 75-1 and 75-2, one in each of the View1 and View2 images, respectively, that are digitally placed at the point of intersection of the central catheter portion 73 and the image of the cryoballoon 70 that is farthest from the location marker 71 at which such intersection occurs. Orientation marks 75-1 and 75-2 are also referred to herein as first and second orientation marks, respectively. The placement of the orientation markers 75-1 and 75-2 may occur through manual interaction of a user with a computer system on which the steps of the method have been programmed using a computer pointing device. The 2D coordinates of the orientation markers 75-1 and 75-2 are then digitally captured by the computer system; the two-dimensional coordinates are located in the detector plane 13.

In this example, the location marker 71 is a radiopaque object near but not at the distal end of the cryoballoon 70. In the example given, the location marker 71 is about 5mm inward from the distal end. Other cardiac ablation balloons may have different detailed structures, but radiopaque objects having a known dimensional relationship to the cardiac ablation balloon must be available as site markers for application of the inventive methods presented herein.

As can be appreciated from fig. 4-1 and 4-2, the freezing balloon 70 is more opaque than the surrounding portions of the X-ray images, so the visibility of the freezing balloon 70 in these X-ray images is very limited except for the location markers 71 and the central catheter portion 73, but the desired points of intersection can usually be found. This is due to the fact that: in the inflated state, the cryoballoon 70 contains a gas that is more radio-opaque than the blood that the gas displaces, thereby enabling the cardiologist to place the orientation markers 75-1 and 75-2.

Fig. 4-1 and 4-2 also show a radiopaque ring 74 at the end of the shaft of the catheter 72. Since the distance between the ring 74 and the balloon 70 can vary, orientation markers 75-1 and 75-2 need to be placed in the View1 and View2 images. The location of the ring 74 does not provide accurate information about the location of the cryoballoon 70. Fig. 4-1 and 4-2 also show that the central conduit portion 73 is straight. The central catheter portion 73 is the only rigid portion of the catheter 72, and the distance between the location marker 71 along the central catheter portion 73 and the opposite end of the cryoballoon 70 is a known distance and does not necessarily extend to the loop 74. Thus, this information (location marker 71 and location of orientation markers 75-1 and 75-2 within the View1 and View2 images) is sufficient to determine the 3D location and orientation of the cryoballoon 70 from the View1 and View2 images.

Fig. 4-1 and 4-2 also show the coronary sinus catheter CSC and the mapping catheter MC (as well as at least one other cardiac catheter). The mapping catheter MC is similar to what is referred to as the looped end 20r (and mapping catheter) in the illustrations of fig. 3A-3D. In the X-ray images of fig. 4-1 and 4-2, the mapping catheter MC is not circular (i.e., in a single plane), but the electrodes of the mapping catheter MC are generally oriented in a helical manner. The invention disclosed in the above-mentioned co-pending application Sra et al may be used in conjunction with the present invention to create a map of cardiac parameters as desired, as necessary in parallel with determining the 3D location and orientation of the cardiac ablation balloon 70.

Fig. 5A is a schematic block diagram illustrating an embodiment 30 of the inventive method for generating and displaying a model of a cardiac ablation balloon in a region of a living heart using single plane fluoroscopic images. Method embodiment 30 uses single plane fluoroscopic images taken from two different angles (View1 and View 2) to enable determination of the 3D location of the cardiac ablation balloon 70 within predetermined coordinates as shown in fig. 2.

The View1 and View2 images may be captured sequentially (with a single fluoroscope set at a first angle and then sequentially set at a second angle), or simultaneously (using both the first and second fluoroscopes). In example 30, a single fluoroscope is first used to capture a string of View1 images in method step 31, followed by a string of View2 images (at a second angle different from the first angle) in method step 33. (in the following example, the frame rate of the fluoroscope is 7.5 frames/second.) the time period of the train should be long enough to incorporate at least one complete breathing cycle.

In method step 35, a cardiac voltage signal may be captured, from which an R-wave interval may be determined in method step 41. The functional elements 37 and 39 use the R-wave data from step 41 to determine the cardiac phase for each of the View1 image (step 37) and the View2 image (step 39). In the method of the invention, cardiac phase and respiratory phase information are used to select the best View1 and View2 images for 3D location determination. Since the motion of the patient during the cardiac protocol is mainly caused by cardiac and respiratory activity, in order to use sequential View1 and View2 images for the computation, ideally image data taken at the same time instant should be employed, so that selecting the best or optimal View1 and View2 images involves finding the pair of images that has the smallest combination of the differences in the two motion phases. Thus, method steps 37 and 39 determine cardiac phase information for each View1 and View2 image, respectively.

FIG. 6 is a digitized signal S (t) from the R-wave detector_i) Exemplary time graph 77. Signal S (t)_i) For deriving cardiac phase information for each of the View1 and View2 images. R wave interval 79 is from signal S (t)_i) The time period between adjacent R-waves of the inner QRS complex (cardiac cycle length). X-ray frames are captured in turn, each occurring at a time relative to the R-wave interval 79. Then, based on the position in time within the R-wave interval 79, a value of the cardiac phase is assigned to each View1 and View2 image. As mentioned above, it is beneficial to determine the location of the 3D cardiac ablation balloon using a pair of View1 and View2 images taken during minimal cardiac and respiratory motion. As part of this determination in method step 51, cardiac phase criteria 80c (frames having cardiac phases between 30% and 80% of the R-wave interval 79 as shown in FIG. 6) are frames 80c (0.3 cardiac phase 0.8) that satisfy such cardiac phase criteria. This 30% -80% value of cardiac phase standard 80c is not limiting. Values outside this range may also be used.

Method steps 43 and 45 (View1 and View2, respectively) include identifying the location markers 71 as the source of displacement information from which respiratory phase information can be determined. Since the motion of the object in the y-direction (generally parallel to the spine of the patient) in the image series is primarily a result of respiratory motion, the y-coordinate of the object in the image series (series) can be used to estimate the respiratory phase. In the example described below, the minimum y-position value is closest to full exhalation.

It should be noted that in example 30, the most obvious choice of y-position objects mentioned in method steps 43 (for View 1) and 45 (for View 2) is the radiopaque location markers 71 of the cryoballoon 70 (see fig. 4-1 and 4-2), but another radiopaque object that moves in the y-direction due to respiration can be used for such y-position measurement. The use of location markers 71 is not intended to be limiting.

The y-coordinate of the location marker 71 (also referred to as y-position object 71 in this example) is the y-coordinate of the geometric center of the y-position object 71, and such determination is well known to those skilled in image processing. However, the use of geometric centers for this determination is not intended to be limiting.

The initial identification of the y-position object 71 may be done manually on a computer display within the first image in each of the View1 and View2 image strings. The motion of the y-position object 71 is then determined within each image of the series in order to determine the respiratory phase information for each image in the series. As in this example, the y-position object 71 may be the same object in each of the View1 and View2 image strings, but this is not necessarily so, as all that is required is that the y-position in each string is indicative of the respiratory movement of the object within the string. In embodiment 30, the fact that the y-position objects in the two strings are identical is not intended to be limiting.

Method steps 47 and 49 include determining the breathing phase of each image in the View1 and View2 series, respectively. One embodiment of such a determination is illustrated in detail in fig. 7-1 through 10-2.

The functional element 51 includes method steps by which an optimal View1 image and an optimal View2 image are selected to minimize the effects of cardiac and respiratory motion in subsequent calculations of the 3D location and orientation of the cardiac ablation balloon 70. One embodiment of method step 51 is shown in FIG. 11. As described above, the breathing phase of the View1 and View2 images is determined from the frame-to-frame variation in the y-position of the location marker 71 (y-position marker 71) in method steps 47 and 49, respectively. FIGS. 7-1 and 7-2 are graphs of exemplary y-position data for the y-position marker 71 in thirty View1 (data points along line 81) and thirty View2 (data points along line 83), respectively. Given the nature of such data, the respiratory phase is estimated, and FIGS. 8-1 and 8-2 are plots of the y-location data of FIGS. 7-1 and 7-2, respectively, that have been smoothed (points 81a and 83a, respectively) and interpolated (lines 81i and 83i, respectively) to generate estimates of respiratory phase for the View1 and View2 images.

Several alternative methods are possible for such smoothing and interpolation. In this example, each of the View1 and View2 frames occurs during some portion of five different R-wave intervals, and each point 81a and 83a is computed by averaging the y-position of the frame within each R-wave interval and averaging the corresponding frame number to generate a highly smooth representation of the respiratory phase across the set of View1 and View2 frames. Curves 81i and 83i are generated by computing a cubic spline fit to these point sets 81a and 83a, respectively, to produce an estimate of the respiratory phase of each image.

FIGS. 9-1 and 9-2 are graphs presenting the respiratory phase and cardiac phase of thirty View1 frames and thirty View2 frames, respectively. The values of both cardiac phase and respiratory phase have been normalized to the 0-1 scale. In fig. 9-1, 9-2, 10-1, and 10-2, the cardiac phase values for the frames are shown with small square markers and the respiratory phase values are shown with small circular markers. The solid and dotted lines are shown only for ease of viewing.

In FIGS. 9-1 and 9-2, each dot line group labeled 85(View 1) and 87(View 2) represents the cardiac phase of a frame occurring within a particular R-wave interval 79.

FIG. 10-1 presents a diagram of View1 frames 85s satisfying a cardiac phase criterion 80c and frames 81s satisfying a respiratory phase criterion 80 r. FIG. 12-2 presents a diagram of View2 frames 87s satisfying a cardiac phase criterion 80c and frames 83s satisfying a respiratory phase criterion 80 r. In this example, the respiratory phase criterion is such that the respiratory phase of frames meeting the criterion is between 0% and 20% of maximum exhalation (respiratory phase ≦ 0.2). Thus, FIGS. 10-1 and 10-2 illustrate cardiac and respiratory phases for a subset of the frames illustrated in FIGS. 9-1 and 9-2.

Therefore, the final selection of the best View1 and View2 images is simplified to selecting from the View1 and View2 images that satisfy both cardiac phase and respiratory phase criteria. These include a View1 image with cardiac and respiratory phase values falling within four regions 89, and a View2 image with cardiac and respiratory phase values falling within three regions 91. In this example, candidate View1 image I_iAre frames 1, 18, 22-25, and 29-30, and candidate View2 image I_jAre frames 4, 9 and 30.

FIG. 11 is a schematic block diagram of an embodiment 51o illustrating the final selection of the best View1 and View2 frames from the set of candidate View1 frames within region 89 and candidate View2 frames within region 91. As indicated in FIG. 11, in this example, there is N₁A View1 frame I_i(N ₁8; index i ═ 1 to 8) and N₂Individual view frame (N)₂4; index j ═ 1 to 4).

In fig. 11, method steps 93, 95, 97 and 99 represent the following facts: the calculation in method step 51o is to use View1 frame I shown in FIGS. 10-1(View 1) and 10-2(View 2)_iAnd View2 frame I_jCardiac phase and respiratory phase values. In method step 101, N is calculated₁A View1 frame I_iAnd N₂A View2 frame I_jThe absolute value of the difference between the cardiac phases of all possible pairs of; presence of N₁·N₂Such pairs and absolute differences. Similarly, in method step 103, N of the breathing phase is calculated₁·N₂The absolute difference. In functional element 105, N is₁·N₂Multiplying each of the cardiac phase difference values by a cardiac weight W_CAnd in a similar manner, in method step 107, N is added₁·N₂Is multiplied by the breathing weight W_R. (in the specific examples shown in FIGS. 7-1 to 10-2, W is used_C1 and W_RA value of 1. )

In a method step 109, N is added₁·N₂Cardiac phase difference sum N₁·N₂Corresponding pairs of respiratory phase differences are added to generate N₁·N₂The set of values, in method step 111, selects the minimum value in this set as the "best" or "matching" pair of View1 and View2 frames. The weighted sum formed for each pair of frames in method step 109 is one possible measure of the similarity of the View1 and View2 frames in each pair of frames, and the similarity criterion is that this measure is minimized.

Similarity can be considered as the inverse of this measure, since a smaller value of this measure indicates a higher inter-frame similarity. In other words, N calculated in method step 109₁·N₂The minimum of the sums of the values represents the maximum similarity (minimum combined phase difference) between the candidate frame pairs. As a result of the method step 51o of FIG. 11, View1 frame number 29 and View2 frame number 9 are selected as the best or matching frame pair. In FIG. 10-1, the View1 frame 29 is labeled with the reference numerals 81o (cardiac phase) and 85o (respiratory phase), respectively. In FIG. 10-2, frame 9 of View2 is labeled with numbers 83o (cardiac phase) and 87o (respiratory phase).

Referring again to fig. 5A and as seen above, the upper portion of the inventive method of example 30 results in the selection of an optimal (View1, View 2) image pair for determining the 3D location and orientation of the cardiac ablation balloon 70, as shown in fig. 4-1 and 4-2. After the best (View1, View 2) image pair is selected in method step 51, a first (or View 1) orientation marker is placed in the selected View1 image and a second (or View 2) orientation marker is placed in the selected View2 image, each marker being located at a point at which the intersection of the images of the projection surfaces of the balloon 70 and the central catheter portion 73 is furthest from the location marker 71, in method step 53. The user may manually input the orientation markers 75-1 and 75-2 using the display 14 (or other computer display) and a computer input device (not shown) such as a mouse to position the orientation markers 75-1 and 75-2 at the desired intersection point in the selected View1 and View2 images.

Prior to determining the 3D location and orientation of the cryoballoon 70, the images of the location markers 71 in the View1 and View2 images are correlated to each other, as are the first and second orientation markers 75-1 and 75-2, in method element 55. Details of these associations are further described with respect to fig. 5B and 5C. The image of the location mark 71 represents the same physical object of the cryoballoon 70. However, the first and second orientation markers 75-1 and 75-2 are described herein as two distinct points, as they are actually placed in the selected View1 and View2 images, respectively, as will be seen below.