阅读说明：本技术 用于目标的照明和成像的系统和方法 (System and method for illumination and imaging of an object ) 是由 F·A·摩尔 A·W·兰普雷克特 L·M·奥特西 P·R·韦斯特维克 M·N·A·B·祖尔卡 于 2016-11-10 设计创作，主要内容包括：本发明涉及一种与成像设备一起使用的垂帘,该垂帘包括：包住成像设备的防护材料；限定防护材料中的开口的垂帘窗框；防护材料中的开口中的垂帘透镜；以及与垂帘窗框集成以将垂帘透镜固定到成像设备的窗框的接口。本发明还涉及一种用来对目标成像的处理器,该处理器用来在一个时段内：接通激发光源来生成用来照亮目标的激发脉冲；接通白光源来生成用来照亮目标的白脉冲以使得该白脉冲不会与激发脉冲重叠并且在该时段内至少生成白脉冲两次；在激发脉冲期间使图像传感器曝光达荧光曝光时间；在至少一个白脉冲期间使图像传感器曝光达可见光曝光时间；检测来自图像传感器的输出；补偿环境光；以及输出结果产生的图像。(The present invention relates to a drapery for use with an imaging device, the drapery comprising: a protective material encasing the image forming apparatus; a drapery window frame defining an opening in the protective material; a drape lens in an opening in the protective material; and an interface integrated with the drapery window frame to secure the drapery lens to the window frame of the imaging device. The invention also relates to a processor for imaging an object, the processor being arranged to, during a time period: switching on an excitation light source to generate an excitation pulse for illuminating the target; turning on a white light source to generate a white pulse for illuminating the target such that the white pulse does not overlap with the excitation pulse and generates the white pulse at least twice within the period; exposing the image sensor for a fluorescence exposure time during the excitation pulse; exposing the image sensor for a visible light exposure time during the at least one white pulse; detecting an output from an image sensor; compensating for ambient light; and outputting the resulting image.)

1. A drapery for use with an imaging device, the drapery comprising:

a protective material encasing the image forming apparatus;

a drapery window frame defining an opening in the protective material;

a drape lens in an opening in the protective material; and

an interface integrated with a drapery window frame to secure the drapery lens to a window frame of an imaging device.

2. The drapery of claim 1, wherein the drapery window frame is insertable into a window frame of an imaging device.

3. The drapery of claim 1 or 2, wherein the interface comprises two clamps symmetrically integrated on respective opposite sides of the drapery window frame.

4. The drapery of claim 3, wherein the two clamps are on the top and bottom of the drapery window frame.

5. A processor for imaging an object, the processor being operable to, during a time period:

switching on an excitation light source to generate an excitation pulse for illuminating the target;

turning on a white light source to generate a white pulse for illuminating the target such that the white pulse does not overlap with the excitation pulse and generates the white pulse at least twice within the period;

exposing the image sensor for a fluorescence exposure time during the excitation pulse;

exposing the image sensor for a visible light exposure time during the at least one white pulse;

detecting an output from an image sensor;

compensating for ambient light; and

and outputting the resulting image.

6. The processor of claim 5, wherein to compensate for ambient light, the processor is to:

exposing a first set of sensor pixel rows of an image sensor for a portion of a fluorescence exposure time for the first set of sensor pixel rows; and

a second set of sensor pixel rows of the image sensor is exposed for all fluorescence exposure times.

7. The processor of claim 6, wherein the portion may be 1/2.

8. The processor of claim 7, wherein the processor is configured to determine the fluorescence signal F using the following equation:

F = 2*Exp2 – Exp1，

where Exp1 is the signal output during this portion of the fluorescence exposure time and Exp2 is the signal output during all fluorescence exposure times.

9. The processor of claim 6, wherein the portion of the exposure time is equal to a width of the excitation pulse.

10. The processor according to claim 5, wherein the visible light exposure time is longer than a width of at least one white pulse.

11. The processor according to claim 5, wherein the visible light exposure time is for a white pulse within the period.

12. The processor of claim 5, wherein the visible light exposure time is for two white pulses within the period.

13. The processor of claim 5, wherein to compensate for ambient light, the processor is to:

the image sensor is exposed for a background exposure time when the target is not illuminated at least once during the period.

14. A method for imaging an object over a period of time, the method comprising:

generating an excitation pulse for illuminating the target;

generating a white pulse to illuminate the target such that the white pulse does not overlap the excitation pulse and the white pulse is generated at least twice within the period;

exposing the image sensor for a fluorescence exposure time during the excitation pulse;

exposing the image sensor for a visible light exposure time during the at least one white pulse;

detecting an output from an image sensor;

compensating for ambient light; and

and outputting the resulting image.

15. The method of claim 14, wherein compensating for ambient light comprises:

exposing a first set of sensor pixel rows of an image sensor for a portion of a fluorescence exposure time; and

a second set of sensor pixel rows of the image sensor is exposed for all fluorescence exposure times.

16. The method of claim 14, wherein compensating for ambient light comprises:

the image sensor is exposed for a background exposure time when the target is not illuminated at least once during the period.

17. The method of any of claims 14 to 16, wherein generating an excitation pulse comprises providing uniform, anamorphic illumination to a target.

18. The method of claim 17, wherein providing uniform, anamorphic illumination to the target comprises overlapping illumination from at least two illumination openings.

19. The method of any of claims 14 to 18, wherein exposing the image sensor for a fluorescence exposure time comprises providing a wavelength dependent aperture upstream of the image sensor during exposure, the wavelength dependent aperture to block visible light outside the central region.

20. The method of claim 19, wherein the wavelength dependent aperture is configured to produce substantially similar depth of field at the sensor plane for both visible light and fluorescence excited by the excitation pulse.

21. A method of displaying fluorescence intensity in an image, the method comprising:

displaying a target reticle covering a region of the image;

calculating the normalized fluorescence intensity in the target reticle; and

the normalized fluorescence intensity is displayed in a display region associated with the target.

22. The method of claim 21, wherein the display area is projected onto a target.

23. The method according to claim 21 or 22, wherein the normalized fluorescence intensity comprises a single numerical value.

24. The method according to any one of claims 21 to 23, wherein the normalized fluorescence intensity comprises a historical map of normalized fluorescence intensities.

Technical Field

The present disclosure relates generally to the field of illumination and imaging. More particularly, the present disclosure relates to illumination and imaging of target materials.

Background

Illumination is an important component of imaging systems, such as broadband imaging systems with self-contained illumination, for example. In many applications of imaging systems, such as in medical imaging, it can be a challenge to achieve uniform full-field illumination of the imaging field of view and also to provide sufficient illumination intensity to produce a sufficiently strong imaging signal. Adherence to matching the illumination profile to the imaging field of view is one way to conserve illumination power, and multiple illumination openings can be used to provide uniform illumination across the field of view. Conventional illumination projections in imaging systems may feature anamorphic projections that match the imaging field of view, but typically feature only a single illumination opening and are not configured for adjacent working distances. A single opening illumination system can cause a large number of shadow areas to obscure vision when illuminating complex topographies such as, for example, human anatomy or other biological materials. Existing designs for field surgical imaging and illumination devices may utilize multiple illumination openings to minimize shadow areas (such as annular light surrounding the imaging optics), but these designs waste excessive illumination outside the field of view and fail to achieve uniform illumination of the field of view over a range of working distances.

Disclosure of Invention

One or more embodiments are directed to an illumination module for use in an imaging system having an imaging field of view for imaging a target, the illumination module comprising: a first illumination opening to output a first light beam having a first illumination distribution at a target to illuminate the target; and a second illumination opening to output a second light beam having a second illumination distribution at the target to illuminate the target. The second illumination distribution may be substantially similar to the first illumination distribution at the target, the second illumination opening being spaced apart from the first illumination opening, the first and second illumination distributions being simultaneously provided to the target and overlapping at the target, wherein the illumination from the first and second openings is matched to the same aspect ratio and field coverage as the imaging field of view.

The light from the first and second illumination openings may overlap respectively to provide uniform illumination over the target field of view.

The illumination module may include a steering driver to steer (steer) the first and second illumination openings simultaneously through different fields of view.

Each of the first and second illumination openings may include a lens module having at least one fixed lens, a steerable housing in communication with a steering driver, and at least one lens mounted in the steerable housing.

The lighting module may include a housing that houses the first and second lighting openings and the steering driver.

The housing may be a hand-held housing and may include a control surface that includes an activation device to control the steering driver.

Each of the first and second illumination distributions may be a rectangular illumination distribution.

Each of the first and second illumination openings may include a lens module having two pairs of cylindrical lenses.

The first and second illumination apertures may be symmetrically offset from a long dimension centerline of the rectangular illumination distribution.

One or more embodiments are directed to an imaging device having an imaging field of view, the imaging device comprising: a first illumination opening to output first light having a first illumination distribution at a target to illuminate the target; a second illumination opening to output second light having a second illumination distribution at the target to illuminate the target, the second illumination distribution at the target being substantially similar to the first illumination distribution, the second illumination opening being spaced apart from the first illumination opening, the first and second illumination distributions being simultaneously provided to the target and overlapping at the target, wherein the illumination from the first and second openings is matched to the same aspect ratio and field coverage as the imaging field of view; and a sensor to detect light from the target.

The imaging device may include a housing that houses the first and second illumination openings, and the sensor.

The imaging device may include a steering driver to simultaneously steer the first and second illumination openings through different fields of view.

The imaging device may include an imaging element to focus light onto the sensor, wherein the steering driver is to move the imaging element in synchronization with steering of the first and second illumination openings.

The steering drive may be in a housing and the housing may include a control surface including an activation device to control the steering drive.

The housing may have a hand-held housing with a form factor that allows one-handed control of the control surface and illumination of the target from multiple orientations.

The imaging device may include an illumination source for outputting light to the first and second illumination openings, the illumination source being external to the housing.

The illumination source may output visible light and/or excitation light to the first and second illumination openings.

The sensor may be a single sensor for detecting light from the target produced by illumination by visible light and excitation light.

The imaging device may include a wavelength dependent aperture upstream of the sensor to block visible light outside the central region.

The imaging device may include a video processor box outside the housing.

The illumination source may be integrated with the video processor box.

One or more embodiments are directed to a method of inspecting an object, the method including simultaneously illuminating the object with a first light output having a first illumination distribution at the object and with a second light output having a second illumination distribution at the object, the second illumination distribution being substantially similar to the first illumination distribution, the first and second illumination distributions overlapping at the object, wherein the illumination on the object is matched to the same aspect ratio and field coverage as the imaging field of view.

The method may comprise simultaneously steering the first and second light outputs through different fields of view.

The method may include receiving light from the target and focusing the light onto the sensor using an imaging element, moving the imaging element in synchronism with the simultaneous turning of the first and second light outputs.

One or more embodiments are directed to a drapery for use with an imaging device, the drapery comprising: a drape frame defining an opening in the drape material, a drape lens in the opening in the drape material; and an interface integrated with the drapery window frame to secure the drapery lens to the window frame of the imaging device.

The drape may be insertable into a window frame of the imaging device.

The interface may include two clamps symmetrically integrated on respective opposite sides of the drapery window frame.

The two clamps are on the top and bottom of the drapery window frame.

One or more embodiments are directed to a processor to image a target, the processor to turn on an excitation light source for a period to generate an excitation pulse to illuminate the target, turn on a white light source to generate a white pulse to illuminate the target such that the white pulse does not overlap the excitation pulse and generate the white pulse at least twice within the period, expose an image sensor for a fluorescence exposure time during the excitation pulse, expose the image sensor for a visible exposure time during at least one white pulse, detect an output from the image sensor, compensate for ambient light, and output a resulting image.

To compensate for ambient light, the processor may expose a first set of sensor pixel rows of the image sensor for a portion of a fluorescence exposure time for the first set of sensor pixel rows; and exposing a second set of sensor pixel rows of the image sensor for all fluorescence exposure times, the first and second sets being used to detect at least one color different from the other set.

The portion may be 1/2.

The processor may determine the fluorescence signal F using the following equation:

F = 2*Exp2 – Exp1，

where Exp1 is the signal output during this portion of the fluorescence exposure time and Exp2 is the signal output during all fluorescence exposure times.

This portion of the exposure time may be equal to the width of the excitation pulse.

The visible light exposure time may be longer than the width of the at least one white pulse.

The visible light exposure time may be for a white pulse within the period.

The visible light exposure time may be for two white pulses within the period.

To compensate for ambient light, the processor may expose the image sensor for a background exposure time when the target is not illuminated at least once within the period.

One or more embodiments are directed to a method for imaging a target over a period of time, the method including generating an excitation pulse to illuminate the target, generating a white pulse to illuminate the target such that the white pulse does not overlap with the excitation pulse and generating the white pulse at least twice within the period of time, exposing an image sensor for a fluorescence exposure time during the excitation pulse, exposing the image sensor for a visible exposure time during at least one white pulse, detecting an output from the image sensor, compensating for ambient light, and outputting a resulting image.

Compensating for the ambient light may include exposing a first set of sensor pixel rows of the image sensor for a portion of the fluorescence exposure time and exposing a second set of sensor pixel rows of the image sensor for all of the fluorescence exposure time, the first and second sets to detect at least one color different from the other set.

Compensating for the ambient light may include exposing the image sensor for a background exposure time when the target is not illuminated at least once within the period.

Generating the excitation pulse may include providing uniform, anamorphic illumination to the target.

Providing uniform, anamorphic illumination to the target includes overlapping illumination from at least two illumination apertures.

One or more embodiments are directed to a method of displaying fluorescence intensity in an image, the method including displaying a target reticle (particle) covering an area of the image, calculating a normalized fluorescence intensity within the target reticle, and displaying the normalized fluorescence intensity in a display area associated with the target.

The display area may be projected onto the target.

The normalized fluorescence intensity may include a single numerical value and/or a historical map of the normalized fluorescence intensity.

One or more embodiments are directed to a kit comprising: an illumination module comprising at least two illumination openings spaced apart from each other, providing the first and second illumination distributions simultaneously to the target and the first and second illumination distributions overlapping at the target; and an imaging module including a sensor to detect light from the target.

The kit may include a housing to enclose the illumination module and the imaging module.

One or more embodiments are directed to an imaging agent, such as, for example, a fluorescence imaging agent for use in imaging devices and methods as described herein. In one or more embodiments, the use can include blood flow imaging, tissue perfusion imaging, lymph imaging, or a combination thereof, which can occur during an invasive surgical procedure, a minimally invasive surgical procedure, a non-invasive surgical procedure, or a combination thereof. The fluorescence imaging agent can be included in the kits described herein. The fluorescence imaging agent may include ICG alone or in combination with other imaging agents.

In one or more embodiments, the invasive surgical procedure can include a cardiac-related surgical procedure or a reconstructive surgical procedure. The cardiac-related surgical procedure may include a cardiac Coronary Artery Bypass Graft (CABG) procedure, which may be extracorporeal and/or non-extracorporeal.

In one or more embodiments, the minimally invasive or non-invasive surgical procedure may include a wound care procedure.

In one or more embodiments, the lymph imaging may include identification of lymph nodes, lymph node drainage, lymph visualization, or a combination thereof. The lymph imaging may be related to the female reproductive system.

Drawings

Features will become apparent to those of ordinary skill in the art by describing in detail exemplary embodiments with reference to the attached drawings, wherein:

FIG. 1 illustrates a schematic diagram of a system for illumination and imaging, according to one embodiment;

fig. 2 illustrates a schematic diagram of a lighting module according to an embodiment;

fig. 3A and 3B illustrate schematic side and plan views, respectively, of an exemplary lens module in a steerable housing according to one embodiment;

FIG. 4A illustrates a schematic diagram of a linkage for synchronized focusing of an imaging system and steering of an illumination system, according to an embodiment;

FIGS. 4B and 4C illustrate bottom and top views, respectively, of a linkage for synchronized focusing of an imaging system and steering of an illumination system, according to an embodiment;

FIGS. 5A and 5B illustrate bottom views of the linkage at the far and near working distances, respectively, according to one embodiment;

FIGS. 6A and 6B illustrate perspective top and bottom views of an illumination and imaging system according to one embodiment;

FIG. 7 illustrates a housing according to one embodiment;

FIGS. 8A and 8B illustrate perspective views of different exemplary locations in which the system may be used;

FIG. 9 illustrates a drapery for use with the system according to one embodiment;

10A-10C illustrate illumination distributions for different illumination configurations;

FIG. 11A illustrates a timing diagram for visible and excitation illumination according to one embodiment;

FIG. 11B illustrates a timing diagram for visible and excitation illumination according to one embodiment;

FIG. 11C illustrates a timing diagram for visible and excitation illumination according to one embodiment;

FIG. 11D illustrates a timing diagram for visible and excitation illumination according to one embodiment;

12A-12C illustrate a pixel layout and interpolation scheme according to one embodiment;

13A-13C illustrate graphs of one embodiment of display method output when a target reticle is placed over regions of non-normalized fluorescence intensity, high relative normalized fluorescence intensity, and medium relative normalized fluorescence intensity, respectively;

FIG. 13D illustrates a graph of one embodiment of a display method output including a time history of signal plots of normalized fluorescence intensity values on a display;

FIG. 14 illustrates a recorded image of an anatomic fluorescence imaging phantom characterized by one embodiment of a display method output displaying normalized fluorescence intensity;

FIG. 15 illustrates an exemplary light source for an exemplary illumination source of the system for illumination shown in FIG. 1; and

FIG. 16 illustrates an exemplary imaging module of the fluorescence imaging system of FIG. 1, including a camera module.

Detailed Description

Example embodiments will now be described more fully hereinafter with reference to the accompanying drawings; however, they may be embodied in different forms and should not be construed as limiting the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey exemplary implementations to those skilled in the art. Various devices, systems, methods, processors, kits, and imaging agents are described herein. While at least two variations of the devices, systems, methods, processors, kits, and imaging agents are described, other variations may include aspects of the devices, systems, methods, processors, kits, and imaging agents described herein combined in any suitable manner with combinations of all or some of the described aspects.

In general, corresponding or analogous reference numbers (when possible) will be used throughout the drawings to refer to the same or corresponding parts.

Spatially relative terms, such as "below," "lower," "upper," and the like, used herein to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures may be used for ease of description. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be oriented "above" the other elements or features. Thus, the exemplary term "below" can encompass both an orientation of above and below. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

FIG. 1 illustrates a schematic diagram of an illumination and imaging system 10 according to one embodiment. As can be seen therein, the system 10 may include an illumination module 11, an imaging module 13, and a video processor/illuminator (VPI) 14. The VPI 14 may include an illumination source 15 to provide illumination to the illumination module 11 and a processor assembly 16 to send control signals and receive data regarding light detected by the imaging module 13 from the target 12 illuminated by the light output by the illumination module 11. The illumination source 15 may output light of different wavelength regions (e.g., white (RGB) light, excitation light) to cause fluorescence in the object 12, etc., depending on the characteristics to be examined and the material of the object 12. Light of different wavelength bands may be output by the illumination source simultaneously or sequentially. The illumination and imaging system 10 may be used, for example, to facilitate a surgical procedure. The target 12 may be a morphologically complex target, such as a biological material including tissue, anatomical structures, other objects having contours and shapes that cause shadows when illuminated, and so forth. The VPI 14 may record, process, display, etc., the resulting image and associated information.

Fig. 2 illustrates a schematic perspective view of the lighting module 11 of fig. 1 according to an embodiment. As can be seen therein, the illumination module 11 may comprise at least two illumination openings directing illumination from an illumination source 23 (which may be comprised in the VPI box 14) to, for example, a rectangular target field 24. Each illumination opening provides illumination on target field 24 such that the light substantially or completely overlaps, for example, at target material 12 (shown in fig. 1). More than two illumination openings may be used. The illumination distributions are substantially similar and substantially overlap at the target 12 to provide uniform illumination of the target 12. The use of the at least two illumination openings reduces shadow effects due to anatomical morphology and helps provide uniform illumination over the target field 24. Directing illumination from the illumination module 11 to the rectangular target field 24 allows matching the area of illumination to the rectangular imaging field of view, which helps provide uniform illumination and enhances the efficiency of the illumination module by reducing extraneous illumination. Matching the illumination field to the imaging field of view also provides a useful indication of the location and extent of the anatomical region currently being imaged.

In some embodiments, light pipes may be used to achieve mixing of the illumination light to produce a uniform illumination profile. The mixing of the illumination light through the light pipe may remove the effect of the structure of the light source on the illumination profile, which may otherwise adversely affect the uniformity of the illumination profile. For example, using a light pipe to mix the illumination light output from a fiber optic light guide may remove an image of the structure of the individual optical fibers from the illumination profile. In some embodiments, a rectangular light pipe may be used to conserve illumination power while matching the illumination profile to the rectangular imaging field of view. In some embodiments, light pipe materials having high refractive indices for both visible and near infrared light (such as optical glass material N-SF 11) may be used for high efficiency illumination power delivery.

According to some embodiments, a rectangular light pipe having an aspect ratio that matches the aspect ratio of the imaging field of view (e.g., both aspect ratios are 16: 9) may be used in conjunction with rotationally symmetric illumination optics.

According to some embodiments, a rectangular light pipe having a different aspect ratio than the imaging field of view (e.g., a square light pipe having an aspect ratio of 16:9 imaging field of view) may be used in conjunction with a cylindrical illumination optic. Cylindrical optical elements may be used to individually follow one or both dimensions of a rectangular illumination profile used to match the aspect ratio of the imaging field of view.

Depending on the desired system requirements and illumination uniformity for the working distance range, various methods for matching the illumination to the rectangular imaging field of view may be used. For example, applications with high requirements on working distance range and illumination uniformity may require the use of illumination optics with dynamic steering in order to adequately match the illumination to the imaging field of view, while applications with lower requirements may be used with fixed illumination optics in order to match the illumination to the field of view.

In some embodiments, one or more illumination optics elements may be rotated by a driver to steer the illumination.

In some embodiments, one or more illumination optics may be translated perpendicular to the imaging optical axis by a driver to steer the illumination.

In some embodiments, one or more illumination optical elements may be configured to provide some distortion in the illumination profile in order to account for distortion inherent to the accompanying imaging system.

In some embodiments, the fixed position and orientation of the illumination optics may be utilized to achieve uniform illumination of the imaging field of view over a specified range of working distances. The offset distance of the illumination optics from the imaging optical axis, along with the orientation of the illumination optics, can be configured to optimize the matching of the illumination profile to the imaging field of view at working distances within a specified range of working distances while also maintaining substantial matching of the illumination profile to the imaging field of view at other working distances within the specified range.

As illustrated in fig. 2, each illumination opening may comprise a lens module 20, a connection cable 22 connected to an illumination light source 23, and a light pipe 21 adapting the high numerical aperture of the connection cable 22 to the lower numerical aperture of the lens module 20. The lens module 20 may be steerable, as described in detail below. In some cases, acceptable performance may be achieved without steering. In other words, only an illumination module providing illumination with a rectangular form factor matching the field of view of the imaging system (e.g. providing uniform illumination within the imaging field of view) using at least two illumination openings (in which each opening produces an illumination gradient such that the illumination flux in the object plane and at each point in the illumination field are reasonably the same) and an imaging device with an illumination module may be sufficient.

Fig. 3A and 3B illustrate side and plan views, respectively, of the lens module 20. The lens module 20 may include a lens mounted in a steerable lens housing 30. As used herein, a lens is any optical element having optical power, whether implemented by a refractive element or a diffractive element. For ease of illustration, other elements not important for understanding, such as a cover enclosing the lens module (see fig. 2), are not shown.

In the particular example shown herein, the lenses may include a pair of horizontal-axis cylindrical lenses 31-32 and a pair of vertical-axis cylindrical lenses 33-34. Also shown is a prism element 35 which can align the illumination light with the intended off-axis. In particular, according to an embodiment, the prism element 35 corrects the angle introduced by the light guide 21 to increase the device compactness. The mounting design for each lens element 31-35 may allow tuning of the magnification and focus of the illumination optics. According to this embodiment, the steerable lens housing 30 encloses and steers three cylindrical lenses 31, 33, 34 and a prism lens element 35 (e.g., collectively referred to as a group). This example of the lens is merely illustrative, and the lens in the lens module 20 may be modified as appropriate.

In this particular embodiment, the base portion of the steerable shell 30 is latched about pivot point 36 to a fixed chassis frame 90 (see fig. 6A) and mechanical linkage 40 (see fig. 4A-4C), described in detail below, respectively, for example using pin 46 (see fig. 6B) inserted into shell aperture 37, while the lens 32 is rigidly connected to the chassis 90, i.e., not to the shell 30 (see fig. 6B).

Fig. 4A illustrates a schematic diagram showing the directions of motion provided by the various components of the linkage 40. The linkage 40 may include a drive cam 41, illumination cams 45a, 45b (one for each illumination opening), and an imaging cam 43. The drive cam 41 receives input from a user (see fig. 6) and translates it into synchronous motion of the lens modules 20a, 20B attached to the corresponding illumination cams 45A, 45B via respective housings 30 and pins 46, and the imaging lens 51 and imaging sensor 52 (see fig. 5A and 5B) attached to the imaging cam 43 via a post-pin cam. The imaging lens 51 is shown here as a single field lens, but additional and/or alternative lenses for focusing light from the target 20 onto the imaging sensor 52 may be employed. Each opening has its own associated illumination cam 45A and 45B, shown here as the left and right sides of the input window for receiving light from the target 12.

In particular, translation of the drive cam 41 may translate the imaging cam 43 along the x-axis (which in turn may cause the imaging cam 43 to translate the imaging lens 51 and imaging sensor 52 along the z-axis), as well as the illumination cams 45a, 45b (which in turn simultaneously steer the corresponding lens modules 20a, 20b about the respective pivot points 36), such that steering of the lens modules 20a, 20b is performed synchronously with positional adjustment of the imaging lens 51 and imaging sensor 52 to ensure proper focusing of light from the target on the sensor 52. Alternatively, the imaging cam 43 may merely translate the imaging lens 51 or any other combination of imaging optical elements along the z-axis in order to ensure proper focusing of light from the target on the sensor 52.

Fig. 4B illustrates a bottom view and fig. 4C illustrates a top view of the linkage 40 according to one embodiment. The drive cam 41 may comprise two drive portions 41a and 41b, and a third drive portion 41c (if steering is included), all of which are shown here as being rigidly attached to form a rigid drive cam 41. Similarly, the imaging cam 43 may include two imaging portions 43a and 43 b. The drive cam 41 receives input from the user via the first drive portion 41a (via the control surface 62) and cam-translates the imaging cam 43 via the pins in the drive portion 41b, causing the imaging cam portion 43a to translate the sensor 52 and the imaging cam portion 43b to translate the imaging lens 51. If steering is included in the linkage, the third driving portion 41c simultaneously steers (rotates) the lens modules 20a, 20B using the pins 46 (see fig. 6B) associated with each of the illumination cam portions 45a and 45B by translating the illumination cam portions 45a and 45B. The pin 46 may be inserted through a through slot 49 in each of the illumination cams 45a, 45b and a corresponding housing hole 37 in the lens modules 20a, 20 b. The driving portion 41c simultaneously steers the lens modules 20a, 20b so that both still illuminate the same field of view as each other at the target field of view of the target 12.

Fig. 5A and 5B illustrate bottom views of the linkage in combination with lens modules 20a, 20B, imaging field lens 51 and sensor 52 at the far and near working distances, respectively, according to one embodiment. As can be seen therein, the linkage 40 synchronizes the steering of the illumination source with the focusing of the imaging system at two sample working distance illumination steering settings. Fig. 5A-5B illustrate the positions of lens modules 20a, 20B (rotating about pivot point 37) and lens 51 and sensor 52 (translating along optical axis 55 of the imaging system and along the x-axis) at two focus positions caused by user input.

As illustrated in fig. 5A and 5B, each part moving in axial direction within the linkage 40 may be guided by two fixed rolling elements 47 and one spring-loaded rolling element 48 in order to reduce or minimize friction during movement. The linkage 40 may also include a drive cam input connection point 42.

Fig. 6A and 6B illustrate a perspective top view and a perspective bottom top view of device 10 according to one embodiment. In fig. 6A and 6B, the illumination module 11 and the imaging module 13 are mounted on a chassis 90, with the top portions thereof removed for clarity. Also, a focus actuation mechanism 70 is illustrated that translates motion from user input to motion of the drive cam 41 via the drive cam input connection point 42.

As can be seen in fig. 6A, the optical axis 55 of the imaging module 13 runs through the center of the imaging module, wherein the lens modules 20a, 20b are arranged symmetrically with respect to the imaging optical axis 55. Light to be imaged from the target 12 travels along the optical axis 55 so as to be incident on the lens 51 and the sensor 52. A wavelength dependent aperture 53 comprising a smaller central aperture allowing transmission of all visible and fluorescent light (e.g. Near Infrared (NIR) light) and a surrounding larger aperture blocking transmission of visible light but allowing transmission of fluorescent light may be provided upstream of the lens 51.

Referring to fig. 6B and 4A-4B, pin 46 connects lens module 20 via housing hole 37 in housing 30, slot 49 of linkage 40. Also, a pivot point pin 44 connects the lens module 20 to the chassis 90.

Fig. 7 illustrates one embodiment of an ergonomic housing 60 enclosing the illumination module 11 and the imaging module 13. The ergonomic housing 60 is designed to be held in different modes of use/configurations, such as a pistol grip (fig. 8A) for forward imaging in a scanning imaging orientation and a vertical orientation grip (fig. 8B) for use when imaging downward in an overhead imaging orientation. As can be seen in fig. 7, the housing 60 includes a control surface 62, a gripping detail 64, a window frame 68 and a nose clip 66. The ergonomic housing 60 may be connected to the VPI box 14 via a light guide cable 67 (through which light is provided to the illumination openings 20a, 20 b) and a data cable 65 (which carries power, sensor data) and any other (non-optical) connection.

The control surface 62 includes focus buttons 63a (decreasing working distance) and 63b (increasing working distance) that control the linkage 40. Other buttons on the control surface 62 may be programmable and may be used for various other functions (e.g., excitation laser power on/off, display mode selection, white light imaging white balance, saving screenshots, etc.). As an alternative to or in addition to the focus button, a proximity sensor may be provided on the housing and may be employed to automatically adjust the linkage 40.

As can be seen in fig. 8A, when the housing 60 is held with the imaging window facing forward, the thumb rests on the control surface 62, while the other fingers on the operator's hand wrap loosely around the bottom of the grip detail 64. As can be seen in fig. 8B, when the housing 60 is held with the imaging window facing down, the grip detail 64 is between the thumb and forefinger and the fingers wrapped around to access the control buttons or activate the control surface 62. The grip detail 64 is shaped to provide partial support for the weight of the device held in a vertical orientation on top of the wrist so that the housing 60 can hang loosely and does not require a tight grip of the housing 60. Thus, the housing 60 can be manipulated by a single hand in multiple orientations. In various other embodiments, the housing 60 may be supported on a support (e.g., a movable support).

The window frame 68 (see also fig. 9) defines various windows for the housing 60. In other words, the window frame 68 defines windows 68a and 68b corresponding to the two lens modules 20a and 20b, and a window 68c (which serves as an input window for light from a target to be incident on the sensor 52).

As illustrated in fig. 9, the housing 60 may be used in conjunction with a drapery 80. The drape 80 may be a surgical drape suitable for use during a surgical procedure. The drapery includes drapery material 81, a drapery lens 82, a drapery window frame 83 around the drapery lens, and an interlock interface 84 integrated with the drapery window frame 83. The drape material 81 is used to enclose the device in the housing 60 and cover anything else as desired. The drapery window frame 83 may follow the shape of the housing nose clip 66 so that the drapery window frame 83 may be inserted therein without obstructing the windows 68 a-68 c. The drapery 80 is designed to minimize reflection and imaging ghosts by ensuring that the drapery lens 82 is flush with the imaging and illumination window frame 68, for example, within 0.5 millimeters. The drape 80 may use an interlocking interface 84, which interlocking interface 84 may fit into a recess on the inner surface of the housing nose clip 66 to be secured flush therewith.

One or more interlock interfaces 84 may be used on the inner or outer surface of the housing nose clip 66 to ensure a safe and tight fit of the drapery lens 82 against the window frame 68. In the particular embodiment shown, two interfaces 84 for engagement by the inner surface of the housing nose clip 66 are used (here one on top of the drapery window frame 83 and one on the bottom of the drapery window frame 83).

Fig. 10A-10C illustrate typical illumination distributions (fills) with respect to a rectangular imaging field of view (profile) at working distances of 10cm (left column), 15cm (middle column), and 20cm (right column) for an illumination ring (fig. 10A), a pair of fixed anamorphic projection illumination sources (fig. 10B), and a pair of steered anamorphic projection illumination sources (fig. 10C) according to one embodiment. Fig. 10A illustrates the use of illuminated split rings to minimize shadows but not match the illumination to the imaging field of view and may not provide uniform illumination (e.g., distribution as a function of distance) at all working distances. Fig. 10B illustrates anamorphic projections from two illumination sources that are fixed (using, for example, an illumination lens arrangement featuring a cylindrical lens or an engineered diffuser), so they are well collimated to achieve uniform illumination that matches the imaging field of view at a fixed working distance (e.g., 15 centimeters), but they are not as uniform or well matched at other distances (whether smaller or larger). As noted above, such lighting is often acceptable by itself. Fig. 10C illustrates the ability to better maintain uniform illumination and constrain the illumination to the field of view by diverting the illumination when changing the working distance (and imaging focus), according to one embodiment.

As noted above, the illumination used may include both white light and fluorescence excitation illumination, e.g., from a laser, to excite Near Infrared (NIR) light from the target. However, ambient light may interact with light from the target.

Fig. 11A illustrates timing diagrams for white light (RGB) and fluorescence excitation (laser) illumination, and visible light (VIS) and NIR Fluorescence (FL) imaging sensor exposures configured to allow ambient room light to be subtracted from the fluorescence signal with a single sensor. As used herein, a white pulse will indicate that white light (RGB) is illuminating the target and an excitation pulse will indicate that laser light is illuminating the target.

Exposures of even (Exp 1) and odd (Exp 2) sensor pixel rows are shown interleaved with different exposure times to facilitate estimated isolation of ambient room light signal components. Such an interleaved exposure readout mode is provided on some imaging sensors, such as the 'high dynamic range interleaved readout' mode on the cmos CMV2000 sensor.

Pulsing the white light illumination at 80Hz brings the frequency of the strobe light above what the human eye can perceive, or this may trigger a seizure. The visible image exposure may be longer (e.g., twice) than the RGB illumination to ensure overlap between the exposure frame rate at 60H and the 80Hz RGB illumination pulses. Due to the much greater intensity of the RGB illumination pulses and signals from the target 12, additional ambient light captured during the visible light exposure may be ignored.

By setting the NIR fluorescence image exposure time, Exp1 and Exp2 acquire the field and quarter frame periods, respectively, while running the excitation laser only in the last quarter of every three frames, the even line (Exp 1) records the field of ambient room light in addition to the quarter frame of NIR fluorescence, while the odd line (Exp 2) records the quarter frame of ambient room light plus the quarter frame of NIR fluorescence. Performing these fractional exposures within each visible or NIR fluorescent frame minimizes motion artifacts that would otherwise be caused by inserting additional exposure frames in the frame sequence for the purpose of ambient room light subtraction.

With such an acquisition design, an estimate of the ambient room light contribution to the image signal can be isolated by subtracting the Exp2 sensor line of the NIR fluorescence image from the Exp1 sensor line (interpolated to match the Exp2 pixel locations), yielding an estimate of the quarter frame of the ambient room light signal. The estimate of the quarter frame of the ambient room light signal may then be subtracted from the Exp2 sensor row of the NIR fluorescence image to produce an estimate of the NIR fluorescence signal with the quarter frame of ambient room light removed. Control of the illumination and exposure may be performed by the VPI box 14.

In one embodiment, the above chamber light subtraction method may be varied to accommodate the use of a bayer pattern color sensor. Fig. 12A illustrates a bayer pattern arrangement of color sensor pixels, where even and odd sensor rows have different filter arrangements (e.g., no red pixels in the even sensor rows and no blue pixels in the odd sensor rows), so ambient light recorded on the even rows will not be able to estimate well where it reaches the odd rows in the same period. However, each row does include a green pixel signal that is also sensitive to NIR fluorescence. Using only green pixels and performing two-dimensional interpolation from green pixel signals to other pixel locations may yield estimates of ambient light signal components and thus also NIR fluorescence or visible light components for NIR and visible light images, respectively.

To calculate the NIR signal value at a given location, Exp1 (even lines) and Exp2 (odd lines) green pixel values near that location are calculated, where one or both of these values need to be interpolated. Fig. 12B shows an example in which the best estimate of Exp1 (even rows) green values at red pixel locations is the average of the immediately above and below green values, while the best estimate of Exp2 (odd rows) green values is the average of the immediately left and right green values.

The following mathematical example is used to illustrate one embodiment of the ambient room light subtraction method. If a = ambient light incident in a quarter frame period and F = fluorescence incident in a quarter frame period, then:

Exp 1 = 2A + F

Exp 2 = A + F

solving for F yields:

F = 2*Exp 2 – Exp 1。

in the particular example illustrated in fig. 11A, the period for sensing is three frames, the white light pulse and the excitation pulse have the same duration or width but different frequencies, visible light is sensed during two frames (e.g., the first two frames) and fluorescence is sensed during one frame (e.g., the third or last frame) for two different exposure times. As shown therein, the visible light exposure time may be twice the duration of the white light pulse, the first fluorescence exposure time may be equal to the duration of the excitation pulse, and the second fluorescence exposure time may be a longer (e.g., twice) pulse than the excitation pulse. Further, the visible light exposure may have a different frequency than the white light pulses, e.g., the visible light exposure does not occur with each white light pulse, while the fluorescence exposure may have the same frequency as the excitation pulses.

Alternative timing and exposure maps are discussed below in which a sensor having rows that are all active for a common exposure duration may be used while a single sensor is used to compensate for ambient light. For example, the background light may be detected directly by the sensor when the target is not illuminated. Other variations on pulsing, exposing, and sensing may be apparent to those skilled in the art.

Fig. 11B illustrates an alternative timing diagram for white light (RGB) and fluorescence excitation (laser) illumination, and visible light (VIS) and NIR Fluorescence (FL) imaging sensor exposures configured to allow subtraction from ambient room light of a fluorescence signal with a single sensor. The exposures to visible light and to fluorescence are shown in sequence along with the exposure used to capture the Background (BG) image signal due to ambient light. The white light illumination may be pulsed at 80Hz as described above. The fluorescence excitation illumination may be pulsed at 20Hz and the pulse duration or width may be increased (e.g., up to twice the white light pulse duration) to achieve a longer corresponding fluorescence exposure. If an imaging sensor with a global shutter is used, each sensor exposure must terminate at the end of the imaging frame with a readout period. The exposure to capture the ambient light background image signal may be performed at the end portion of the frame in the absence of any pulsed white light or excitation light. As shown in the example in fig. 11B, where video is acquired at a frame rate of 60Hz, a white light illumination pulse width of quarter frame duration may be used when the end of the white light illumination pulse is aligned with the end of a frame, along with the quarter frame duration visible light exposure that occurs in each frame.

The scaled image signals recorded during one or more background exposures may be subtracted from each fluorescence exposure image to remove the contribution of ambient light from the fluorescence image. For example, the image signal from a quarter frame duration background exposure may be scaled up by a factor of two and subtracted from the subsequent image signal from a half frame duration fluorescence exposure. As another example, both a quarter frame duration background exposure image signal before a half frame duration fluorescence exposure image signal, and a second quarter frame background image signal after fluorescence exposure may be subtracted from the fluorescence image signal. The scaling of the image signal from the first and second background exposures may comprise interpolation of pixel values from the first exposure time point and the second exposure time point in order to estimate a pixel value corresponding to the intermediate time point.

The use of an imaging sensor with a high speed readout that enables higher video frame acquisition rates may allow additional exposure periods to be allocated within the illumination and exposure timing scheme for a given white light pulse frequency. For example, maintaining 80Hz white light illumination pulses as above and using a sensor with a higher video frame acquisition rate (such as 120 Hz) may allow additional white light, ambient background, or fluorescence to be exposed for a given period of time, as compared to when using a slower video frame acquisition rate (such as 60 Hz).

In the particular example illustrated in fig. 11B, the period for sensing is three frames, the excitation pulse has twice the width of the white light pulse, visible light is sensed during one frame (e.g., the first frame), background light is sensed during one frame (e.g., the second frame), and fluorescence is sensed during one frame (e.g., the third or last frame). Here, the visible light exposure time may be equal to the duration of the white light pulse, the background exposure time may be equal to the duration of the white light pulse, and the fluorescence exposure time may be equal to the duration of the excitation pulse. Further, the visible light exposure may have a different frequency than the white light pulses, e.g., the visible light exposure does not occur with each white light pulse, while the fluorescence exposure may have the same frequency as the excitation pulses. Finally, the background exposure may occur only once within the period.

Fig. 11C illustrates an alternative timing diagram for white light (RGB) and fluorescence excitation (laser) illumination, and visible light (VIS) and NIR Fluorescence (FL) imaging sensor exposures configured to allow ambient room light subtraction from a fluorescence signal with a single sensor at a 120Hz video frame acquisition rate. A white light pulse frequency of 80Hz is used and when the end of the white light illumination pulse aligns with the end of the frame, a half-frame duration white light illumination pulse width may be used, along with the half-frame duration visible light exposure that occurs in each frame. The fluorescence excitation illumination is shown pulsed at 40Hz in case of pulse duration of one frame to achieve a higher frequency corresponding to the fluorescence exposure. The exposure to capture the ambient light background image signal may be performed at the end portion of the frame in the absence of any pulsed white light or excitation light, such as the exposure of half-frame duration that occurs in the frame between a fluorescence exposure and a subsequent white light exposure, as shown in this example embodiment.

In the particular example illustrated in fig. 11C, the period for sensing is three frames, the width of the excitation pulse is twice the width of the white light pulse, visible light is sensed during one frame (e.g., the second frame), background light is sensed during one frame (e.g., the first frame), and fluorescence is sensed during one frame (e.g., the third or last frame). Here, the visible light exposure time may be equal to the duration of the white light pulse, the background exposure time may be equal to the duration of the white light pulse, and the fluorescence exposure time may be equal to the duration of the excitation pulse. Further, the visible light exposure may have a different frequency than the white light pulses, e.g., the visible light exposure does not occur with each white light pulse, while the fluorescence exposure may have the same frequency as the excitation pulses. Finally, the background exposure may occur only once within the period.

Depending on the intensity of the fluorescence excitation light used, there may be safety considerations limiting the duration and frequency of the excitation light pulses. One method for reducing the intensity of the applied excitation light is to reduce the duration of the excitation light pulse and the corresponding fluorescence exposure. Additionally or alternatively, the frequency of the excitation light pulses (and the corresponding fluorescence exposures) may be reduced, and instead the readout period that would otherwise be available for fluorescence exposures may be used for background exposures to improve the measurement of ambient light.

Fig. 11D illustrates an alternative timing diagram for white light (RGB) and fluorescence excitation (laser) illumination, and visible light (VIS) and NIR Fluorescence (FL) imaging sensor exposures configured to allow ambient room light subtraction from a fluorescence signal with a single sensor at a 120Hz video frame acquisition rate. A white light pulse frequency of 80Hz is used and when the end of the white light illumination pulse aligns with the end of the frame, a half-frame duration white light illumination pulse width may be used, along with the half-frame duration visible light exposure that occurs in each frame. The fluorescence excitation illumination pulsed at 20Hz is shown in the case of a pulse duration of one frame. The exposure to capture the ambient light background image signal may be performed at the end portion of the frame in the absence of any pulsed white light or excitation light, such as a background exposure of half-frame duration occurring in the frame between the fluorescence exposure and the successive first white light exposure, and both a first background exposure and a second background exposure of half-frame duration occurring in the frame between the first white light exposure and the successive second white light exposure, as shown in this example embodiment.

In the particular example illustrated in fig. 11D, the period for sensing is six frames, the excitation pulse has twice the width of the white light pulse, the visible light is sensed during two frames (e.g., the second and fifth frames), the background light is sensed during three frames (e.g., the first, third, and fourth frames), and the fluorescence is sensed during one frame (e.g., the sixth or last frame). Here, the visible light exposure time may be equal to the duration of the white light pulse, the background exposure time may be equal to the duration of the white light pulse or twice thereof, and the fluorescence exposure time may be equal to the duration of the excitation pulse. Further, the visible light exposure may have a different frequency than the white light pulses, e.g., the visible light exposure does not occur with each white light pulse (e.g., only occurs twice within the time period), while the fluorescent light exposure may have the same frequency as the excitation pulses. Finally, the background exposure may occur three times over a period of total duration equal to four times the duration of the white light pulse.

To improve the performance of such ambient room light compensation methods as described above, a wavelength dependent aperture (e.g., element 55 in fig. 6A) may be used that includes a smaller central aperture that allows transmission of all visible and NIR light and a surrounding larger aperture that blocks visible light but allows transmission of NIR light. The use of such a wavelength dependent aperture allows a larger proportion of NIR signal to be collected relative to visible light signal, which improves the performance of image signal subtraction for estimating and removing ambient room light components. The wavelength dependent aperture may also be characterized by a third larger aperture around the other smaller apertures, which blocks both visible and NIR light. As one example, the wavelength dependent apertures may include membrane apertures in which a film of material (e.g., a plastic or glass film) that blocks transmission of visible light but allows transmission of NIR light has a central opening (e.g., a hole) that allows transmission of both visible and NIR light. Such a membrane aperture may comprise a material that prevents transmission of visible light by reflection and/or a material that prevents transmission of visible light by absorption. As another example, the wavelength dependent apertures may include dichroic apertures formed by mask film deposition on a single substrate, with a film that allows transmission of visible and NIR light deposited on the smaller central aperture, and a second film that blocks transmission of visible light but allows transmission of NIR light deposited on the surrounding larger aperture. The corresponding aperture sizes of the smaller central aperture of the wavelength dependent aperture and the surrounding larger aperture may be set so as to make the depth of field appear substantially similar for visible light and for NIR light when imaged by the imaging system. One or more wavelength dependent filters may be placed in different locations throughout the device, in which case the rejection of visible light and the passage of NIR signals may be optimized. For example, such a wavelength dependent filter may be positioned just in front of the lens 51. As another example, one or more wavelength dependent filters may be placed in the pupil plane of the imaging lens.

This may be useful, for example, to facilitate comparison of fluorescence signals of different regions, to display a target reticle around a region within an imaging field of view, and to calculate and display normalized fluorescence intensity within the region. Normalization of the measured fluorescence intensity values may allow meaningful comparison of multiple images and corresponding values. To correct for variations in measured fluorescence intensity with working distance (e.g., the distance of the imaging system to the imaged anatomy), the normalized fluorescence intensity value can be based on the ratio between the measured fluorescence intensity value and the light value reflected within the target reticle area.

A numerical representation of the normalized fluorescence intensity values within the target reticle field may be displayed on or near the image frame to facilitate comparing values when targeting the target reticle at different locations on the imaged anatomy. For example, the numerical representation may be an average of the normalized fluorescence intensity values for all image pixels in the target reticle area.

Additionally or alternatively, a time history map of the numerical representation of normalized fluorescence intensity values within the target reticle field may be displayed on or near the image frame to facilitate comparing values when targeting a target reticle at different locations on the imaged anatomy or at the same location over a series of time points. Such a temporal history map may further assist the user in assessing the fluorescence profile in the imaged tissue surface by scanning across the anatomical region of interest and viewing the relative normalized fluorescence intensity profile map.

FIG. 13A illustrates a diagram of sample display output from one embodiment of a display method in which a target reticle 125 is positioned within an area of the imaged anatomy 120 that is free of fluorescence intensity 122 and a numerical representation of fluorescence intensity 126 is displayed near the target reticle 125. FIG. 13B illustrates a plot of another sample display output in which the target reticle 125 is positioned over an area of high relative normalized fluorescence intensity 124 and a corresponding numerical representation 126 of the relatively high fluorescence intensity is shown. FIG. 13C illustrates a plot of another sample display output in which the target reticle 125 is positioned over an area of medium relative normalized fluorescence intensity 124 and a corresponding numerical representation 126 of the relative medium fluorescence intensity is shown. FIG. 13D illustrates a plot of sample display output in which the target reticle 125 is positioned over an area of medium relative normalized fluorescence intensity 124 and shows a time history plot 128 of a numerical representation of normalized fluorescence intensity consistent with sequential imaging of areas of zero, high, and medium relative normalized fluorescence intensity. Alternatively or additionally, to display the numerical representation and/or the historical map on the target, a display area associated with the target reticle (e.g., on the device itself or some other display) may display this information.

Figure 14 illustrates a recorded image of an anatomic fluorescence imaging phantom featuring one embodiment of a display method output displaying normalized fluorescence intensity. In particular, the target 110 is illuminated with excitation light and a target reticle 115 is placed over the region of fluorescence intensity 112 according to one embodiment. The numerical representation of the target reticle 115 is displayed in a field 116 associated with the target reticle 115. A time history plot 118 of numerical representations of normalized fluorescence intensity attributed to imaging of different locations of the reticle 115 can be displayed.

Such a display method may be useful for a variety of fluorescence imaging systems, including endoscopic or laparoscopic fluorescence imaging systems, open field fluorescence imaging systems, or combinations thereof. Such normalization and display of fluorescence intensity values may allow for useful quantitative comparison of relative fluorescence intensity between image data from various time points within an imaging session. Such normalization and display of fluorescence intensity values may further allow for useful quantitative comparisons of relative fluorescence intensity between image data from different imaging sessions, by combination with appropriate standardized fluorescence agent management and imaging protocols, and standardized calibration of the imaging device.

Examples are given.

Fluoroscopic medical imaging system for acquisition of image data。

In some embodiments, the system for illumination and imaging of a patient may be used with or as a component of a medical imaging system, such as, for example, a fluorescence medical imaging system for acquiring fluorescence medical image data. One example of such a fluorescence medical imaging system is a fluorescence imaging system 10 schematically illustrated in fig. 1. In this embodiment, the fluorescence imaging system 10 is configured to acquire a time series of fluorescence signal intensity data (e.g., images, video) that captures the transport of the fluorescence imaging agent through the tissue.

The fluorescence imaging system 10 (FIG. 1) includes: an illumination source 15 and an illumination module 11 for illuminating tissue of a patient so as to induce fluorescence emissions from a fluorescence imaging agent 17 in the tissue of the patient (e.g., in blood); an imaging module 13 configured to acquire a time series of fluorescence images from the fluorescence emissions; and a processor component 16 configured to utilize a time series of fluorescence images (fluorescence signal intensity data) acquired according to various embodiments described herein.

In various embodiments, the illumination source 15 (FIG. 1) includes, for example, a light source 200 (FIG. 15) comprising a fluorescence excitation source configured to generate excitation light having an appropriate intensity and an appropriate wavelength for exciting the fluorescent imaging agent 17. Light source 200 in fig. 15 includes a laser diode 202 (which may include, for example, one or more fiber coupled laser diodes) configured to provide excitation light for exciting fluorescent imaging agent 17 (not shown). Examples of other sources of excitation light that may be used in various embodiments include one or more LEDs, arc lamps, or other illumination techniques of sufficient intensity and appropriate wavelength to excite fluorescent imaging agents 17 in tissue (e.g., in blood). For example, excitation of the fluorescent imaging agent 17 in blood may be performed using one or more 793nm conduction-cooled single-rod fiber-coupled Laser Diode modules from DILAS Diode Laser, ltd, germany, where the fluorescent imaging agent 17 is a fluorescent dye with near-infrared excitation characteristics.

In various embodiments, the light output from the light source 200 in fig. 15 may be projected through an optical element (e.g., one or more optical elements) to shape and direct the output used to illuminate the tissue region of interest. The shaping optics may consist of one or more lenses, light guides and/or diffractive elements in order to ensure a flat field over substantially the entire field of view of the imaging module 13. In particular embodiments, the fluorescence excitation source is selected to emit at a wavelength near the absorption maximum of the fluorescence imaging agent 17 (e.g., ICG). For example, referring to the embodiment of light source 200 in fig. 15, the output 204 from laser diode 202 passes through one or more focusing lenses 206 and then through a homogeneous light pipe 208 (such as, for example, a light pipe commonly available from Newport corporation, usa). Finally, the light is passed through an optical diffractive element 214 (e.g., one or more optical diffusers), such as, for example, a ground glass diffractive element also available from Newport corporation, usa. The laser diode 202 itself is powered by, for example, a high current laser driver such as those available from luminea Power limited, usa. The laser may optionally be operated in a pulsed mode during the image acquisition process. In this embodiment, an optical sensor, such as a solid state photodiode 212, is incorporated in the light source 200 and samples the illumination intensity produced by the light source 200 via reflections scattered or diffuse reflections from various optical elements. In various embodiments, additional illumination sources may be used to provide guidance when aligning and positioning the modules within the region of interest. In various embodiments, at least one of the components of the light source 200 depicted in fig. 15 may be a component that includes the illumination source 15 and/or includes the illumination module 11.

Referring back to FIG. 1, in various embodiments, imaging module 13 may be, for example, a component of fluorescence imaging system 10 configured to acquire a time series (e.g., video) of fluorescence images from fluorescence emissions from fluorescence imaging agent 17. Referring to fig. 16, one exemplary embodiment of imaging module 13 is shown including camera module 250. As shown in fig. 16, the camera module 250 acquires an image of the fluorescence emissions 252 from a fluorescence imaging agent 17 in tissue (not shown), such as blood, by using a system of imaging optics (e.g., a front element 254, a rejection filter 256, a dichroic element 260, and a back element 262) to collect and focus the fluorescence emissions on an image sensor assembly 264 that includes at least one 2D solid-state image sensor. Rejection filter 256 may be, for example, a notch filter to reject one wavelength band corresponding to excitation light. The dichroic element 260 may be, for example, a dichroic mirror to selectively pass a subset of the incoming light wavelength spectrum and redirect the remaining wavelengths out of the optical path for rejection or toward a separate image sensor. The solid-state image sensor may be a Charge Coupled Device (CCD), CMOS sensor, CID, or similar 2D sensor technology. The charge resulting from the optical signal transduced by the image sensor assembly 264 is converted to an electrical video signal (which includes both digital and analog video signals) by appropriate readout and amplification electronics in the camera module 250.

According to some embodiments, an excitation wavelength of about 800nm +/-10 nm and an emission wavelength > 820nm is used along with NIR compatible optics for ICG fluorescence imaging. One skilled in the art will appreciate that other excitation and emission wavelengths may be used for other imaging agents.

Referring back to fig. 1, in various embodiments, the processor assembly 16 includes, for example,

a processor module (not shown) configured to perform various processing operations, including executing instructions stored on a computer-readable medium, wherein the instructions cause one or more of the systems described herein to perform the methods and techniques described herein, and

a data storage module (not shown) to record and store data from the operations and, in some embodiments, to store instructions executable by the processor module to perform the methods and techniques disclosed herein.

In various embodiments, the processor module comprises any computer or computing device, such as, for example, a tablet, laptop, desktop, networked computer, or dedicated stand-alone microprocessor. Inputs are obtained, for example, from the image sensor 264 of the camera module 250 shown in fig. 16, from solid state photodiodes in the light source 200 in fig. 15, and from any external control hardware, such as foot switches or remote controls. The output is provided to a laser diode driver, and an optical alignment aid. In various embodiments, the processor component 16 (fig. 1) may have a data storage module with the capability to save a time series of input data (e.g., image data) to a tangible, non-transitory computer readable medium, such as, for example, an internal memory (e.g., a hard disk or flash memory) in order to enable the recording and processing of the data. In various embodiments, the processor module may have an internal clock to ensure control of the various elements and to ensure proper timing of the illumination and sensor shutters. In various other embodiments, the processor module may also provide graphical displays of user inputs and outputs. The fluoroscopic imaging system may optionally be configured with a video display (not shown) to display images being acquired or played back after recording, or to further visualize data as generated at various stages of the above-described method.

In operation and with continued reference to the exemplary embodiment in fig. 1, 15 and 16, the patient is in position for imaging, in which case the anatomical region of interest of the patient is located beneath both the illumination module 11 and the imaging module 13, such that a substantially uniform illumination field is produced across substantially the entire region of interest. In various embodiments, prior to administering the fluorescent imaging agent 17 to the patient, an image of the region of interest may be acquired for background subtraction purposes. To do so, for example, an operator of the fluoroscopic imaging system 10 in fig. 1 may initiate a time sequence of fluoroscopic images (e.g., video) by depressing a remote switch or foot control or via a keyboard (not shown) connected to the processor assembly 16. Accordingly, the illumination source 15 is turned on and the processor assembly 16 begins recording the fluorescence image data provided by the image acquisition assembly 13. Instead of the pulse patterns discussed above, it will be appreciated that in some embodiments, the illumination source 15 may include a transmission source that continues to be turned on during the image acquisition sequence. When operating in the pulsed mode of the embodiment, the image sensor 264 in the camera module 250 (fig. 16) is synchronized to collect the fluorescent emissions following the laser pulses generated by the diode laser 202 in the light source 200 (fig. 15). In this way, the maximum fluorescence emission intensity is recorded and the signal-to-noise ratio is optimized. In this embodiment, the fluorescence imaging agent 17 is administered to the patient and the fluorescence imaging agent 17 is delivered to the region of interest via arterial flow. For example, the acquisition of the time series of fluorescence images is initiated shortly after administration of the fluorescence imaging agent 17, and the time series of fluorescence images throughout the entrance of the fluorescence imaging agent 17 is acquired from substantially the entire region of interest. The fluorescent light emitted from the region of interest is collected by the collection optics of camera module 250. The residual ambient and reflected excitation light is attenuated by subsequent optics in camera module 250 (e.g., optics 256 in fig. 16, which may be a filter) so that the fluorescent emissions can be acquired by image sensor assembly 264 with minimal light interference from other sources.

In various embodiments, the processor is in communication with or a component of the imaging system. Program code or other computer readable instructions according to various embodiments may be written and/or stored in any suitable programming language and delivered to a processor in various forms including, for example, but not limited to, information permanently stored on a non-writable storage medium (e.g., read-only memory device such as a ROM or CD-ROM disk), information alterably stored on a writable storage medium (e.g., hard disk drive), information conveyed to the processor via a transitory medium such as a signal, information conveyed to the processor via a communication medium such as a local area network, a public network such as the internet, or any type of medium suitable for storing electronic instructions. In various embodiments, the tangible, non-transitory computer-readable medium includes all computer-readable media. In some embodiments, only computer readable instructions for performing one or more of the methods or techniques discussed herein may be stored on the non-transitory computer readable medium.

In some embodiments, the illumination and imaging system may be a component of a medical imaging system that acquires medical image data, such as the fluoroscopic medical imaging system 10. In embodiments where the illumination and imaging system is a component of an imaging system, such as the fluoroscopic imaging system described above, the light source, illumination module, imaging module and processor of the medical imaging system may function as the camera assembly and processor of the illumination and imaging system. Those skilled in the art will recognize that imaging systems other than fluorescence imaging systems may be employed for use with illumination and/or imaging systems such as those described herein (depending on the type of imaging being performed).

Exemplary imaging Agents for use in generating image data。

According to some embodiments, in a fluorescence medical imaging application, the imaging agent is a fluorescence imaging agent (such as, for example, an indocyanine green (ICG) dye). When ICG is administered to a patient, it binds to blood proteins and circulates with the blood in the tissues. A fluorescent imaging agent (e.g., ICG) may be administered to a patient as a bolus injection (e.g., into a vein or artery) at a concentration suitable for imaging, such that the bolus circulates in the vasculature and through the microvasculature. In other embodiments where multiple fluorescent imaging agents are used, such agents may be administered simultaneously, for example in a single bolus, or sequentially in separate boluses. In some embodiments, the fluorescent imaging agent may be administered through a catheter. In certain embodiments, the fluorescent imaging agent may be administered less than one hour before performing the measurement of the intensity of the signal caused by the fluorescent imaging agent. For example, the patient may be administered a fluorescent imaging agent for less than 30 minutes prior to the measurement. In still other embodiments, the fluorescent imaging agent may be administered for at least 30 seconds before the measurement is performed. In still other embodiments, the fluorescent imaging agent may be administered simultaneously with the measurement being performed.

According to some embodiments, the fluorescent imaging agent may be administered at various concentrations to achieve a desired circulating concentration in the blood. For example, in embodiments where the fluorescence imaging agent is ICG, the fluorescence imaging agent may be administered at a concentration of about 2.5 mg/mL in order to achieve a circulating concentration in the blood of about 5 μ Μ to about 10 μ Μ. In various embodiments, the upper concentration limit for administration of the fluorescent imaging agent is the concentration at which the fluorescent imaging agent becomes clinically toxic in circulating blood, and the lower concentration limit is an instrument limit for acquiring signal intensity data caused by the fluorescent imaging agent as the blood circulates in order to detect the fluorescent imaging agent. In various other embodiments, the upper concentration limit for administration of the fluorescent imaging agent is the concentration at which the fluorescent imaging agent becomes self-quenching. For example, the circulating concentration of ICG may range from about 2 μ Μ to about 10 mM. Thus, in one aspect, the method includes the steps of administration of an imaging agent (e.g., a fluorescent imaging agent) to the patient and acquisition of signal intensity data (e.g., video) prior to processing the signal intensity data in accordance with various embodiments. In another aspect, the method excludes any step of administering an imaging agent to the patient.

According to some embodiments, a suitable fluorescence imaging agent for use in a fluorescence imaging application used to generate fluorescence image data is an imaging agent that can circulate with blood (e.g., a fluorescent dye that can circulate with, for example, components of blood, such as lipoproteins or serum plasma in blood) and carry vasculature of tissue (i.e., large blood vessels and microvasculature) and produce signal intensity therefrom when exposed to suitable light energy (e.g., excitation light energy or absorption light energy). In various embodiments, the fluorescence imaging agent comprises a fluorescent dye, an analog thereof, a derivative thereof, or a combination of these. Fluorescent dyes include any non-toxic fluorescent dye. In certain embodiments, the fluorescent dye optimally fluoresces in the near infrared spectrum. In certain embodiments, the fluorescent dye is or includes a tricarbocyanine dye. In certain embodiments, the fluorescent dye is or includes indocyanine green (ICG), methylene blue, or a combination thereof. In other embodiments, the fluorescent dye is or includes fluorescein isothiocyanate, rhodamine, phycoerythrin, phycocyanin, allophycocyanin, o-phthaldehyde, fluorescamine, rose bengal, trypan blue, fluorogold, or a combination thereof that can be excited using an excitation light wavelength suitable for each dye. In some embodiments, analogs or derivatives of fluorescent dyes may be used. For example, a fluorescent dye analog or derivative includes a fluorescent dye that has been chemically modified but still retains its ability to produce fluorescence when exposed to light energy of an appropriate wavelength.

In various embodiments, the fluorescence imaging agent can be provided as a lyophilized powder, solid, or liquid. In certain embodiments, the fluorescence imaging agent may be provided in a vial (e.g., a sterile bottle) that may allow reconstitution to the appropriate concentration by administering a sterile fluid with a sterile syringe. Reconstitution can be performed using any suitable carrier or diluent. For example, a fluorescence imaging agent can be reconstituted with an aqueous diluent immediately prior to administration. In various embodiments, any diluent or carrier that holds the fluorescent imaging agent in solution may be used. As one example, ICG may be reconstructed using water. In some embodiments, once the fluorescence imaging agent is reconstituted, it may be mixed with additional diluents and carriers. In some embodiments, the fluorescent imaging agent may be conjugated to another molecule (such as a protein, peptide, amino acid, synthetic polymer, or sugar), for example, to enhance solubility, stability, imaging properties, or a combination thereof. Additional buffers may optionally be added, including Tris, HC1, NaOH, phosphate buffer, and/or HEPES.

One skilled in the art will recognize that while a fluorescent imaging agent is described in detail above, other imaging agents may be used depending on the optical imaging modality in conjunction with the systems, methods, and techniques described herein.

In some embodiments, the fluorescence imaging agents used in conjunction with the methods and systems described herein may be used for blood flow imaging, tissue perfusion imaging, lymph imaging, or combinations thereof, which may be performed during invasive surgical procedures, minimally invasive surgical procedures, non-invasive surgical procedures, or combinations thereof. Examples of invasive surgical procedures that may involve blood flow and tissue perfusion include cardiac-related surgical procedures (e.g., CABG in extracorporeal or non-extracorporeal circulation) or reconstructive surgical procedures. Examples of such non-invasive or minimally invasive procedures include wound (e.g., chronic wounds such as, for example, pressure sores) treatment and/or management. Examples of lymph imaging include identification of one or more lymph nodes, lymph node drainage, lymph visualization, or a combination thereof. In some variations, such lymph imaging may be related to the female reproductive system (e.g., uterus, cervix, vulva). In some embodiments, the fluorescence imaging agent may be administered in sufficient concentration and in an appropriate manner to achieve lymphatic imaging.

Tissue perfusion involves the microcirculation flow of blood per unit volume of tissue, where oxygen and nutrients are provided to, and waste products are removed from, the capillary beds of the tissue being perfused. Tissue perfusion is a phenomenon related to, but distinct from, blood flow in a blood vessel. Quantitative blood flow through a blood vessel can be expressed in terms of a defined flow (i.e., volume/time) or a defined velocity (i.e., distance/time). Tissue blood perfusion defines the movement of blood through microvasculature (such as arterioles, capillaries, or venules) within a volume of tissue. Quantitative tissue blood perfusion can be expressed in terms of blood flow through a tissue volume (i.e., blood volume/time/tissue volume (or tissue mass)). Perfusion is associated with a nutritive blood vessel (e.g., a microvessel called a capillary) that includes blood vessels associated with the exchange of metabolites between blood and tissue, rather than larger diameter non-nutritive blood vessels. In some embodiments, quantification of the target tissue may include calculating or determining parameters or quantities related to the target tissue, such as rate, size volume, time, distance/time, and/or volume/time, and/or changes in quantity as related to any one or more of the foregoing parameters or quantities. However, blood movement through the capillaries of an individual can be highly unstable (due primarily to vascular motion) compared to blood movement through larger diameter vessels, where spontaneous oscillations in vascular tone appear as pulsations in red blood cell movement.

In summary and by way of review, one or more embodiments may accommodate varying working distances while providing a flat illumination field and matching the illumination field to a target imaging field, thus allowing accurate quantitative imaging applications. An imaging element that focuses light from a target onto the sensor may be moved in synchronization with the steering of the illumination field. Additionally or alternatively, drapes may be used that ensure a tight fit between the drapery lens and the window frame of the device. Additionally or alternatively, one or more embodiments may allow for the use of a single sensor to subtract ambient light from the light to be imaged, as well as the controlled timing of illumination and exposure or detection. Additionally or alternatively, one or more embodiments may allow for the display of normalized fluorescence intensity measured within a target reticle region of an image frame.

In contrast, when the illumination and imaging device does not conform to the illumination to the target imaging field of view or provides a flat (i.e., uniform or substantially uniform) illumination field, the illumination and image quality may be affected. Non-uniform illumination fields can cause distracting and inaccurate imaging artifacts, especially for handheld imaging devices and when used at varying working distances, while excessive light outside the imaging field of view can reduce device efficiency and can be distracting to the user when positioning the device.

The methods and processes described herein may be performed by code or instructions to be executed by a computer, processor, manager or controller, or in hardware or other circuitry. Because the algorithms that form the basis of the methods (or the operations of a computer, processor, or controller) are described in detail, the code or instructions for carrying out the operations of the method embodiments may transform the computer, processor, or controller into a special purpose processor for carrying out the methods described herein.

Moreover, another embodiment may include a computer-readable medium (e.g., a non-transitory computer-readable medium) for storing the code or instructions described above. The computer-readable medium may be a volatile or non-volatile memory or other storage device (which may be removably or fixedly coupled to a computer, processor) or controller (which is used to execute code or instructions for performing the method embodiments described herein).

Example embodiments have been disclosed herein, and although specific terms are employed, they are used and are to be interpreted in a generic and descriptive sense only and not for purposes of limitation. In some instances, features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics and/or elements described in connection with other embodiments, unless expressly indicated otherwise, as would be apparent to one of ordinary skill in the art to which this application is filed. It will therefore be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as set forth below.

While the disclosure has been illustrated and described in connection with various embodiments shown and described in detail, it is not intended to be limited to the details shown, since various modifications and structural changes may be made therein without departing in any way from the scope of the disclosure. Various modifications in the form, arrangement of parts, steps, details and order of operation of the illustrated embodiments, as well as other embodiments of the disclosure, may be made in any way without departing from the scope of the disclosure, and will be apparent to those skilled in the art upon reference to this description. It is, therefore, intended that the appended claims cover such modifications and embodiments as fall within the true scope of the disclosure. For purposes of clarity and brevity, the features are described herein as part of the same or separate embodiments, however, it will be recognized that the scope of the present disclosure includes embodiments having combinations of all or some of the features described. The phrase "and not limited to" is to be construed as being mandatory unless expressly specified otherwise, for the terms "for example" and "such as" and grammatical equivalents thereof. As used herein, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.