Single-frame exposure rapid three-dimensional fluorescence imaging system and method based on DMD
阅读说明：本技术 基于dmd的单帧曝光快速三维荧光成像系统及方法 (Single-frame exposure rapid three-dimensional fluorescence imaging system and method based on DMD ) 是由 陈冲 李辉 金鑫 于 2020-05-21 设计创作，主要内容包括：本发明公开了一种基于DMD的单帧曝光快速三维荧光成像系统及方法,系统包括DMD双侧照明模块,用于产生两束不同波长的光并分别从不同的方向入射至DMD,生成互补的条纹光；双色荧光激发模块,用于接收互补的条纹光,激发样品上的双色荧光,生成携带双色荧光的条纹光；双色分画幅成像模块,用于将不同波长的携带双色荧光的条纹光成像至相机靶面的不同位置,形成双色图像；重构模块,用于实现去除离焦信息。本发明通过单次曝光、单次采集即可以实现结构光照明三维层切成像；此外,采用DMD投影,提高投影速度和投影条纹的精度,进而提高了三维成像的速度和精度。(The invention discloses a single-frame exposure rapid three-dimensional fluorescence imaging system and method based on a DMD (digital micromirror device), wherein the system comprises a DMD bilateral lighting module and a digital micromirror device) imaging module, wherein the DMD bilateral lighting module is used for generating two beams of light with different wavelengths and respectively irradiating the two beams of light to the DMD from different directions to generate complementary stripe light; the double-color fluorescence excitation module is used for receiving the complementary stripe light, exciting the double-color fluorescence on the sample and generating the stripe light carrying the double-color fluorescence; the double-color framing imaging module is used for imaging the stripe light carrying the double-color fluorescence with different wavelengths to different positions of a camera target surface to form a double-color image; and the reconstruction module is used for removing the out-of-focus information. The invention can realize the structured light illumination three-dimensional slice imaging through single exposure and single acquisition; in addition, the DMD projection is adopted, so that the projection speed and the projection fringe precision are improved, and the three-dimensional imaging speed and the three-dimensional imaging precision are further improved.)
1. The single-frame exposure rapid three-dimensional fluorescence imaging system based on the DMD is characterized by comprising a DMD bilateral lighting module, a bicolor fluorescence excitation module, a bicolor frame division imaging module and a reconstruction module;
the DMD bilateral illumination module is used for generating two beams of light with different wavelengths and respectively irradiating the two beams of light to the digital micromirror device DMD from different directions to generate complementary stripe light;
the two-color fluorescence excitation module is used for receiving the complementary stripe light, exciting two-color fluorescence on the sample and generating stripe light carrying the two-color fluorescence;
the double-color frame-dividing imaging module is used for imaging the stripe light carrying double-color fluorescence with different wavelengths to different positions of a camera target surface to form a double-color image;
and the reconstruction module is used for removing the out-of-focus information.
2. The DMD based single frame exposure fast three dimensional fluorescence imaging system of claim 1, wherein the DMD double side illumination module comprises: the device comprises a first laser coupling unit (1), a second laser coupling unit (5), a first Kohler illumination lens group (2), a second Kohler illumination lens group (4) and a DMD (3); the first laser coupling unit (1) comprises a first light source and a first optical fiber, and emergent light of the first light source is coupled into the first optical fiber; the second laser coupling unit (5) comprises a second light source and a second optical fiber, and emergent light of the second light source is coupled into the second optical fiber; the wavelengths of the first light source and the second light source are different; the first laser coupling unit (1) and the second laser coupling unit (5) are respectively positioned on two sides of the axis of the DMD (3); emergent light of the first laser coupling unit (1) and the emergent light of the second laser coupling unit (5) are respectively incident to the DMD (3) through the first Kohler lighting lens group (2) and the second Kohler lighting lens group (4), angles of two beams of light incident to the DMD (3) are adjusted, reflected light passing through the DMD (3) is perpendicular to a target surface of the DMD (3), and complementary stripe light is formed.
3. The DMD based single frame exposure fast three dimensional fluorescence imaging system of claim 2, wherein the first light source or the second light source is a laser light source or a mercury lamp or an LED.
4. The DMD-based single-frame exposure fast three-dimensional fluorescence imaging system according to claim 1, characterized in that the two-color fluorescence excitation module comprises a first cylindrical mirror (6), a first reflecting mirror (7), a first dichroic mirror (8), a second reflecting mirror (9), a liquid zoom lens (10), a first relay lens group, a first objective lens (13), and a sample (14) arranged in sequence along an optical axis; the first relay lens group comprises a first relay lens (11) and a second relay lens (12); carrying out double-color fluorescence labeling on the same site of the sample (14); the complementary stripe light is reflected by the first cylindrical mirror (6) and the first reflecting mirror (7), the first dichroic mirror (8) is reflected to the second reflecting mirror (9), and then the complementary stripe light is excited by the liquid zoom lens (10), the first relay lens group and the first objective lens (13) to generate stripe light carrying bicolor fluorescence with different wavelengths.
5. The DMD-based, single-frame-exposure, fast three-dimensional, fluorescence imaging system according to claim 1, 2, or 4, characterized in that the two-color burst imaging module comprises a second relay lens group, a second dichroic mirror (17), a first color filter (18), a second color filter (22), a third mirror (19), a fourth mirror (23), a second barrel mirror (20), a third barrel mirror (24), a third dichroic mirror (21), and a camera (25) arranged along the optical axis; the second relay lens group comprises a fourth cylindrical mirror (15) and a third relay lens (16); the fringe light primary path carrying the bicolor fluorescence and having different wavelengths returns to enter a second relay lens group after being reflected by a first objective lens (13), a first relay lens group, a liquid zoom lens (10), a second reflecting mirror (9) and transmitted by a first dichroic mirror (8) in sequence; emergent light of the second relay lens group is reflected by the second dichroic mirror (17) and then enters the second color filter (22) and the fourth reflecting mirror (23), and is transmitted by the second dichroic mirror (17) and then enters the first color filter (18) and the third reflecting mirror (19); reflected light of the third reflector (19) is transmitted by the second barrel mirror (20) and the third dichroic mirror (21) and then imaged to a target surface of the camera (25), reflected light of the fourth reflector (23) is reflected by the third barrel mirror (24) and the third dichroic mirror (21) and then imaged to the target surface of the camera (25), and a double-color image is formed; the cut-off wavelengths of the first color filter (18) and the second color filter (22) correspond to the wavelengths of the first light source and the second light source, respectively.
6. The DMD-based, single-frame-exposure, fast three-dimensional, fluorescence imaging system according to claim 5, characterized in that the third mirror (19), the third dichroic mirror (21) and the fourth mirror (23) are adjustable in angle and position for adjusting the position of the bi-color image on the camera (25) target surface; the third reflector (19) is used for adjusting the position of one path of image after monochromatic imaging on the target surface of the camera (25), the third dichroic mirror (21) and the fourth reflector (23) form a double-reflection system, and the position of the other path of image after monochromatic imaging on the target surface of the camera (25) is adjusted together.
7. The DMD-based single-frame exposure fast three-dimensional fluorescence imaging system according to claim 5, characterized in that the DMD (3), the camera (25), the liquid zoom lens (10), the first light source, the second light source are all controlled by FPGA; the FPGA can adjust the intensity of the light emitted by the first light source and the second light source; the rising edge signal output by the FPGA triggers the DMD (3) and the camera (25) to work simultaneously, the DMD (3) starts to display stripes, and the camera (25) starts to expose; when the camera (25) completes one frame of exposure, the FPGA sends an analog voltage signal to the liquid zoom lens (10), and the analog voltage signal is used for changing the light curvature of the liquid zoom lens (10) and realizing three-dimensional slicing scanning of the sample (14).
8. The DMD based single frame exposure fast three dimensional fluorescence imaging system of claim 1, wherein the reconstruction module comprises:
the image preprocessing unit is used for segmenting the image acquired by the camera and segmenting two strip images with complementary phases by positioning; then, image registration is carried out according to the positions of the stripes of the two images, and non-registered pixels at the edges of the images, which are introduced by errors of an optical system, are removed;
the deconvolution unit is used for respectively carrying out Richardson-Lucy deconvolution processing on the two strip images with complementary phases for a plurality of times;
the normalization processing unit is used for respectively carrying out intensity normalization processing on the two strip images with complementary phases after deconvolution processing;
the wide-field image generation unit is used for summing and averaging two strip images with complementary phases after intensity normalization processing to obtain a wide-field image;
the local contrast extraction unit is used for extracting the local contrast of the stripes aiming at a certain stripe image to form a local contrast matrix;
a low-frequency information extraction unit, which is used for multiplying the local contrast matrix and the wide field image, and then extracting the low-frequency information of the in-focus information from the wide field image by using low-pass filtering;
a high-frequency information extraction unit for extracting high-frequency information of in-focus information from the wide-field image by using high-pass filtering;
and the defocusing image generation unit is used for summing the low-frequency information and the high-frequency information to obtain a defocusing image.
9. The imaging method of the DMD based single frame exposure fast three-dimensional fluorescence imaging system according to any of claims 1 to 8, comprising the steps of:
the initial starting step of the system: the first light source and the second light source in the first laser coupling unit and the second laser coupling unit start to emit light, and the intensity of the light emitted by the first light source and the second light source can be adjusted through the FPGA; the FPGA outputs a rising edge signal to trigger the DMD and the camera to work;
and (3) complementary stripe light generation step: emergent light of the first light source and emergent light of the second light source are respectively coupled into the first optical fiber and the second optical fiber, the emergent light of the first laser coupling unit and the emergent light of the second laser coupling unit are respectively incident to the DMD from two sides of the DMD through the first Kohler lighting lens group and the second Kohler lighting lens group, and the reflected light passing through the DMD is vertical to the target surface of the DMD to form complementary stripe light;
a two-color fluorescence excitation step: the complementary stripe light is reflected to the second reflecting mirror through the first cylindrical mirror, the first reflecting mirror and the first dichroic mirror in sequence, and then is excited by the sample through the liquid zoom lens, the first relay lens group and the first objective lens to generate stripe light with different wavelengths carrying bicolor fluorescence;
two-color frame splitting imaging: the fringe light primary path with different wavelengths carrying the bicolor fluorescence returns, and enters the second relay lens group after being reflected by the first objective lens, the first relay lens group, the liquid zoom lens, the second reflector and the first dichroic mirror in sequence and then transmitted; emergent light of the second relay lens group is reflected by the second dichroic mirror and then enters the second color filter and the fourth reflecting mirror, and is transmitted by the second dichroic mirror and then enters the first color filter and the third reflecting mirror; reflected light of the third reflector is transmitted through the second barrel mirror and the third dichroic mirror and then imaged to the camera target surface, reflected light of the fourth reflector is reflected through the third barrel mirror and the third dichroic mirror and then imaged to the camera target surface, and a double-color image is formed;
and (3) realizing defocusing imaging: and removing out-of-focus information processing is carried out on the two strip images with complementary phases acquired by the camera to obtain an out-of-focus removed image.
10. The DMD based single frame exposure fast three dimensional fluorescence imaging method of claim 9, further comprising: when the camera finishes one-frame exposure, the FPGA sends an analog voltage signal to the liquid zoom lens, the light curvature of the liquid zoom lens is changed, and three-dimensional slicing scanning of the sample is realized.
Technical Field
The invention belongs to the field of fluorescence imaging, and particularly relates to a single-frame exposure rapid three-dimensional fluorescence imaging system and method based on a DMD (digital micromirror device).
Background
Three-dimensional fluorescence imaging of living biological samples is an important tool commonly used in biological research. The invention discloses a series of three-dimensional fluorescence imaging methods, which comprise a confocal fluorescence microscope, a two-photon microscope, a light sheet microscope, a structure illumination microscope and the like. Both confocal microscopes and two-photon microscopes are point scanning microscopes, and the imaging speed is slow and not suitable for in vivo imaging. The sample of the light sheet microscope is complex to clamp and is not easy to obtain in a common laboratory. The structured light fluorescence microscope is a wide-field imaging, has high imaging speed and is easy to reform on the traditional fluorescence microscope.
The method for realizing three-dimensional fluorescence imaging of the existing structured light illumination microscope comprises the following steps: the paper "Method of illuminating by using structured light in a coherent microscope" proposes a Method for removing defocus information from a sample by using structured light illumination, acquires three different phase sample structure fringe images, and reconstructs a slice image with defocus information removed by using a homodyne detection Method, wherein the slice effect can be equivalent to that of a confocal microscope. The article "Optically sectional fluorescence with hybrid-illumination imaging through a flexible fiber bundle" proposes a method of acquiring two images, a uniformly illuminated wide-field image (uniform illumination image) and a structure illuminated image (structured illumination image). The structured illumination can be speckle (optical section visual imaging with specific illumination high level micro-optics) or stripe (optical section fluorescence with high-frequency thin-fiber bundle), which is achieved by fusing complementary high-frequency I in focushp(x, y) and low frequency Ilp(x, y) information obtaining a slice image from which defocus information is removed; because the high-frequency information is basically in-focus information, the high-frequency information is directly extracted from the wide-field image through high-pass filtering; since the defocus signal always has a low frequency and a low local contrast, it can be separated from the low frequency component of the wide-field image using the local contrast informationAnd (6) discharging.
It can be seen from the above that, in the existing structured light illumination microscope technology, a sliced image can be reconstructed only by acquiring at least two images, and when three-dimensional imaging is performed, separate imaging is required layer by layer, so that a large number of images need to be acquired, and the three-dimensional imaging speed is slow.
Disclosure of Invention
The invention aims to overcome the problems in the prior art, improve the three-dimensional imaging speed and provide a single-frame exposure rapid three-dimensional fluorescence imaging system based on a DMD (digital micromirror device).
The technical solution for realizing the purpose of the invention is as follows: the single-frame exposure rapid three-dimensional fluorescence imaging system based on the DMD comprises a DMD bilateral lighting module, a bicolor fluorescence excitation module, a bicolor frame-splitting imaging module and a reconstruction module;
the DMD bilateral illumination module is used for generating two beams of light with different wavelengths and respectively irradiating the two beams of light to the DMD from different directions to generate complementary stripe light;
the two-color fluorescence excitation module is used for receiving the complementary stripe light, exciting two-color fluorescence on the sample and generating stripe light carrying the two-color fluorescence;
the double-color frame-dividing imaging module is used for imaging the stripe light carrying double-color fluorescence with different wavelengths to different positions of a camera target surface to form a double-color image;
and the reconstruction module is used for removing the out-of-focus information.
Further, the DMD double-side illumination module includes: the device comprises a first laser coupling unit, a second laser coupling unit, a first Kohler illumination lens group, a second Kohler illumination lens group and a DMD; the first laser coupling unit comprises a first light source and a first optical fiber, and emergent light of the first light source is coupled into the first optical fiber; the second laser coupling unit comprises a second light source and a second optical fiber, and emergent light of the second light source is coupled into the second optical fiber; the wavelengths of the first light source and the second light source are different; the first laser coupling unit and the second laser coupling unit are respectively positioned on two sides of the axis of the DMD; emergent light of the first laser coupling unit and the emergent light of the second laser coupling unit are respectively incident to the DMD through the first Kohler lighting lens group and the second Kohler lighting lens group, the angle of the two beams of light incident to the DMD is adjusted, the reflected light passing through the DMD is perpendicular to the target surface of the DMD, and complementary stripe light is formed.
Further, the first light source or the second light source employs a laser light source or a mercury lamp or an LED.
Further, the double-color fluorescence excitation module comprises a first cylindrical lens, a first reflecting mirror, a first dichroic mirror, a second reflecting mirror, a liquid zoom lens, a first relay lens group, a first objective lens and a sample which are sequentially arranged along an optical axis; the first relay lens group comprises a first relay lens and a second relay lens; carrying out double-color fluorescence labeling on the same site of the sample; the complementary stripe light is reflected by the first cylindrical mirror, the first reflecting mirror and the first dichroic mirror to the second reflecting mirror, and then is excited by the liquid zoom lens, the first relay lens group and the first objective lens to generate stripe light with different wavelengths carrying bicolor fluorescence.
Further, the bicolor split-frame imaging module comprises a second relay lens group, a second dichroic mirror, a first color filter, a second color filter, a third reflecting mirror, a fourth reflecting mirror, a second barrel mirror, a third dichroic mirror and a camera which are arranged along the optical axis; the second relay lens group comprises a fourth cylindrical mirror and a third relay lens; the fringe light primary path with different wavelengths carrying the bicolor fluorescence returns, and enters the second relay lens group after being reflected by the first objective lens, the first relay lens group, the liquid zoom lens, the second reflector and the first dichroic mirror in sequence and then transmitted; emergent light of the second relay lens group is reflected by the second dichroic mirror and then enters the second color filter and the fourth reflecting mirror, and is transmitted by the second dichroic mirror and then enters the first color filter and the third reflecting mirror; reflected light of the third reflector is transmitted through the second barrel mirror and the third dichroic mirror and then imaged to the camera target surface, reflected light of the fourth reflector is reflected through the third barrel mirror and the third dichroic mirror and then imaged to the camera target surface, and a double-color image is formed; the cut-off wavelengths of the first color filter and the second color filter respectively correspond to the wavelengths of the first light source and the second light source.
Further, the angles and the positions of the third reflector, the third dichroic mirror and the fourth reflector are adjustable, and the angles and the positions of the third reflector, the third dichroic mirror and the fourth reflector are used for adjusting the position of the bi-color image on the target surface of the camera; the third reflector is used for adjusting the position of one path of image after monochromatic imaging on the camera target surface, the third dichroic mirror and the fourth reflector form a double-reflection system, and the position of the other path of image after monochromatic imaging on the camera target surface is adjusted together.
Furthermore, the DMD, the camera, the liquid zoom lens, the first light source and the second light source are all controlled by the FPGA; the FPGA can adjust the intensity of the light emitted by the first light source and the second light source; the rising edge signal output by the FPGA triggers the DMD and the camera to work simultaneously, the DMD starts to display stripes, and the camera starts to expose; when the camera finishes one-frame exposure, the FPGA sends an analog voltage signal to the liquid zoom lens, and the analog voltage signal is used for changing the light curvature of the liquid zoom lens and realizing three-dimensional slicing scanning of the sample.
Further, the reconstruction module comprises:
the image preprocessing unit is used for segmenting the image acquired by the camera and segmenting two strip images with complementary phases by positioning; then, image registration is carried out according to the positions of the stripes of the two images, and non-registered pixels at the edges of the images, which are introduced by errors of an optical system, are removed;
the deconvolution unit is used for respectively carrying out Richardson-Lucy deconvolution processing on the two strip images with complementary phases for a plurality of times;
the normalization processing unit is used for respectively carrying out intensity normalization processing on the two strip images with complementary phases after deconvolution processing;
the wide-field image generation unit is used for summing and averaging two strip images with complementary phases after intensity normalization processing to obtain a wide-field image;
the local contrast extraction unit is used for extracting the local contrast of the stripes aiming at a certain stripe image to form a local contrast matrix;
a low-frequency information extraction unit, which is used for multiplying the local contrast matrix and the wide field image, and then extracting the low-frequency information of the in-focus information from the wide field image by using low-pass filtering;
a high-frequency information extraction unit for extracting high-frequency information of in-focus information from the wide-field image by using high-pass filtering;
and the defocusing image generation unit is used for summing the low-frequency information and the high-frequency information to obtain a defocusing image.
An imaging method of a single-frame exposure rapid three-dimensional fluorescence imaging system based on a DMD (digital micromirror device), comprising the following steps of:
the initial starting step of the system: the first light source and the second light source in the first laser coupling unit and the second laser coupling unit start to emit light, and the intensity of the light emitted by the first light source and the second light source can be adjusted through the FPGA; the FPGA outputs a rising edge signal to trigger the DMD and the camera to work;
and (3) complementary stripe light generation step: emergent light of the first light source and emergent light of the second light source are respectively coupled into the first optical fiber and the second optical fiber, the emergent light of the first laser coupling unit and the emergent light of the second laser coupling unit are respectively incident to the DMD from two sides of the DMD through the first Kohler lighting lens group and the second Kohler lighting lens group, and the reflected light passing through the DMD is vertical to the target surface of the DMD to form complementary stripe light;
a two-color fluorescence excitation step: the complementary stripe light is reflected to the second reflecting mirror through the first cylindrical mirror, the first reflecting mirror and the first dichroic mirror in sequence, and then is excited by the sample through the liquid zoom lens, the first relay lens group and the first objective lens to generate stripe light with different wavelengths carrying bicolor fluorescence;
two-color frame splitting imaging: the fringe light primary path with different wavelengths carrying the bicolor fluorescence returns, and enters the second relay lens group after being reflected by the first objective lens, the first relay lens group, the liquid zoom lens, the second reflector and the first dichroic mirror in sequence and then transmitted; emergent light of the second relay lens group is reflected by the second dichroic mirror and then enters the second color filter and the fourth reflecting mirror, and is transmitted by the second dichroic mirror and then enters the first color filter and the third reflecting mirror; reflected light of the third reflector is transmitted through the second barrel mirror and the third dichroic mirror and then imaged to the camera target surface, reflected light of the fourth reflector is reflected through the third barrel mirror and the third dichroic mirror and then imaged to the camera target surface, and a double-color image is formed;
and a defocus removing imaging step: and removing out-of-focus information processing is carried out on the two strip images with complementary phases acquired by the camera to obtain an out-of-focus removed image.
Further, the method further comprises: when the camera finishes one-frame exposure, the FPGA sends an analog voltage signal to the liquid zoom lens, the light curvature of the liquid zoom lens is changed, and three-dimensional slicing scanning of the sample is realized.
Compared with the prior art, the invention has the following remarkable advantages: 1) a DMD chip is adopted as a spatial light modulator, the forward and reverse turning characteristics of the spatial light modulator are fully utilized, and a bicolor complementary fringe light field can be modulated under the condition that the DMD turns once through a bicolor lighting module; 2) through the frame-dividing imaging module, the two-color fluorescent images are projected at different positions of the target surface of the camera, the target surface of the camera is fully utilized, two complementary fluorescent stripe images can be acquired simultaneously through one-time exposure, the image acquisition speed of the system is improved by two times, and the imaging speed of the three-dimensional layer-cutting system is greatly improved; 3) the liquid zoom lens is adopted to realize rapid axial scanning, and compared with the traditional mode of realizing axial scanning by mechanically moving a sample, the scanning speed can be improved by more than two times, so that the imaging speed of the three-dimensional slicing system is greatly improved; 4) the relay lens group is arranged between the objective lens and the liquid zoom lens to realize mutual conjugation of the pupil of the objective lens and the pupil of the liquid zoom lens, and the design can ensure that the magnification of the system is kept unchanged in the process of realizing axial scanning by changing the optical curvature of the liquid zoom lens; 5) in the double-side illumination module, a Kohler illumination mode is adopted to realize uniform illumination; 6) the design of a frame dividing module consisting of a dichroic mirror, a dichroic filter and a double reflecting mirror is adopted, so that the convenience of system adjustment can be improved, and the confocal adjustment of double-color stripes and the adjustment of the positions of double-color fluorescence images can be realized more easily; 7) in the reconstruction algorithm, the R-L deconvolution algorithm is adopted to perform repeated iteration deconvolution on the original fringe image, and the method obtains higher fringe contrast, so that the calculation of the local contrast matrix of the image is more accurate, and the in-focus low-frequency information is further more accurately extracted; 8) according to the positions of the stripes of the two images, image registration is carried out in an image preprocessing stage, the edge position of the image is removed as a registered pixel in an algorithm mode, and the artifact of the edge of the reconstructed image caused by image misregistration is reduced.
The present invention is described in further detail below with reference to the attached drawing figures.
Drawings
FIG. 1 is a block diagram of a single frame exposure fast three-dimensional fluorescence imaging system based on DMD according to an embodiment.
FIG. 2 is a light path diagram of a DMD-based single-frame exposure fast three-dimensional fluorescence imaging system in one embodiment.
Fig. 3 is a schematic diagram of complementary stripe light formed by DMD in an embodiment, in which (a) is a schematic diagram of an illumination area of DMD and (b) is an enlarged diagram of the illumination area.
Fig. 4 is a schematic diagram of an embodiment of a relay, where (a) is a schematic diagram of an illumination relay and (b) is a schematic diagram of an imaging relay.
Fig. 5 is a spectral plot of a dichroic mirror in one embodiment.
FIG. 6 is a timing diagram for FPGA control in one embodiment.
FIG. 7 is a flow diagram of an algorithm for removing defocus information in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, in combination with fig. 1, a DMD-based single-frame exposure fast three-dimensional fluorescence imaging system is provided, the system comprising a DMD double-side illumination module, a two-color fluorescence excitation module, a two-color framing imaging module, and a reconstruction module;
the DMD bilateral illumination module is used for generating two beams of light with different wavelengths and respectively irradiating the two beams of light to the DMD from different directions to generate complementary stripe light;
the two-color fluorescence excitation module is used for receiving the complementary stripe light, exciting two-color fluorescence on the sample and generating stripe light carrying the two-color fluorescence;
the double-color frame-dividing imaging module is used for imaging the stripe light carrying double-color fluorescence with different wavelengths to different positions of a camera target surface to form a double-color image;
and the reconstruction module is used for removing the out-of-focus information.
The structured light illumination slicing method adopted by the invention is a method combining a light path and an algorithm. The DMD is used to generate the structural stripe to project on the sample surface, and since the structural stripe can only modulate the sample near the focal surface, that is, only the fluorescence signal near the focal surface is striped, and the defocus information is not modulated, the part of modulated signal can be demodulated out to be the in-focus signal through an algorithm. And three-dimensional slicing is to scan samples layer by layer and then reconstruct the samples by an algorithm, and finally, the images are overlapped to form a three-dimensional image. According to the invention, the algorithm to be adopted for slicing only needs two images to complete reconstruction, so that the function of turning the micro-mirror of the DMD in two directions is ingeniously utilized, two paths of laser generate two stripes with different colors from illumination on two sides and project the stripes to the sample surface to excite the fluorescence of the sample, then two stripe images carrying sample information are imaged to different positions of the camera target surface through the dichroic mirror, the color filter and the like of the framing module, two images are extracted, and reconstruction is carried out.
Further, in one embodiment, in combination with fig. 2, the DMD double-side illumination module includes: a first laser coupling unit 1, a second laser coupling unit 5, a first kohler
Further, in one embodiment, the first light source or the second light source is a laser light source or a mercury lamp or an LED.
Here, in order to improve monochromaticity and collimation of the light source, a laser light source is preferably used.
Preferably, in one embodiment, 488nm illumination light source and 561nm illumination light source are respectively adopted as the first light source and the second light source. The 488nm illumination light source can adopt an OBIS semiconductor laser (OBIS 488 LS), the maximum output power of the laser is 150mW, and the output power of the fiber end is 60mW after the multimode fiber coupling; the 561nm illumination light source can adopt an OBIS semiconductor laser (OBIS 561 LS), the maximum laser output power is 150mW, and the output power of the fiber end is 63mW after the multimode fiber coupling.
Here, the kohler illumination lens group is used to make the DMD illumination area more uniform, the illumination being shown in fig. 3 a.
Preferably, in one embodiment, the DMD employs a DMD chip of Ti DLP4100 with microlenses having three states, positive 12 degrees, zero degrees, and negative 12 degrees flip. Two adjacent rows of the DMD are inverted by positive 12 degrees and inverted by negative 12 degrees respectively (the inversion from 0 degree counterclockwise in the reset state by 12 degrees is positive, and the inversion from clockwise to negative is defined), so that 488nm light is reflected by the micromirror with negative inversion from one side, and a stripe pattern with a phase of 0 is formed. Similarly, 561nm light illuminated from the other side will form a complementary fringe pattern with a phase π, as shown in FIG. 3 b. The angle between the two incident beams is 24 degrees.
Further, in one embodiment, the bicolor fluorescence excitation module includes a first cylindrical mirror 6, a first reflecting mirror 7, a first dichroic mirror 8, a second reflecting mirror 9, a
Here, a liquid zoom lens is used, and changing the load current of the liquid zoom lens changes the optical curvature of the lens, thereby realizing rapid Z-axis scanning. Compared with the existing Z-axis scanning method of moving a sample or an objective lens through a piezoelectric displacement table, the scanning speed is higher.
Illustratively, the liquid zoom lens is selected from EL-10-30, and a control box of the liquid zoom lens adopts TR-CL180, and the change of the focusing position is controlled by loading analog voltage on the control box.
Here, in order to prevent the magnification of the imaging system from changing together when the liquid zoom lens light curvature changes, a pair of relay lenses are employed, and a 4f system is configured such that the pupil of the objective lens and the pupil of the liquid zoom lens remain conjugate as shown in fig. 4 a.
Illustratively, a pair of relay lenses of f 100mm is employed.
Further, in one of the embodiments, the two-color split frame imaging module includes a second relay lens group, a second dichroic mirror 17, a first color filter 18, a second color filter 22, a third reflecting mirror 19, a fourth reflecting mirror 23, a second barrel mirror 20, a third barrel mirror 24, a third dichroic mirror 21, and a camera 25, which are disposed along the optical axis; the second relay lens group comprises a fourth cylindrical mirror 15 and a third relay lens 16; the fringe light primary path with different wavelengths carrying the bicolor fluorescence returns, and enters the second relay lens group after being reflected by the first objective lens 13, the first relay lens group, the liquid zoom lens 10, the second reflecting mirror 9 and the first dichroic mirror 8 in sequence; the emergent light of the second relay lens group is reflected by the second dichroic mirror 17 and then enters the second color filter 22 and the fourth reflecting mirror 23, and is transmitted by the second dichroic mirror 17 and then enters the first color filter 18 and the third reflecting mirror 19; reflected light of the third reflecting mirror 19 is transmitted by the second barrel mirror 20 and the third dichroic mirror 21 and then imaged to a target surface of a camera 25, reflected light of the fourth reflecting mirror 23 is reflected by the third barrel mirror 24 and the third dichroic mirror 21 and then imaged to the target surface of the camera 25, and a double-color image is formed; the cut-off wavelengths of the first color filter 18 and the second color filter 22 correspond to the wavelengths of the first light source and the second light source, respectively.
Here, since one factor that limits the imaging speed in the current prior art is that the reading speed of the camera is not fast enough, the invention designs a two-color frame division imaging module, which can make full use of the target surface of the camera. The pixels of the commonly used camera are generally 2048 × 2048, but 512 × 512 pixel points are commonly used in the actual biological imaging, and the camera is read out in rows, so that the number of rows of the camera determines the reading time of the camera, three quarters of the pixels are not used, and the target surface of the camera can be well utilized by the framing imaging method.
Here, since the second relay lens group needs to be connected with a framing module behind, but the focal length of the fourth barrel mirror is relatively short, it is necessary to design an imaging relay module as shown in fig. 4b to constitute a 4f system.
Here, the first dichroic mirror 8 plays a role of transmitting the two-color illumination light and reflecting the two-color fluorescence, and is selected according to the wavelength of the light source. For the 488nm illumination light source and the 561nm illumination light source, the dichroscope is multi-channel at 488nm and 561nm, and the spectrum curve is shown in fig. 5.
Preferably, the camera is an sCMOS camera, has the characteristics of high speed, high quantum efficiency and the like, and is suitable for detecting weak light such as fluorescence and the like.
Further, in one embodiment, the angles and positions of the third mirror 19, the third dichroic mirror 21 and the fourth mirror 23 are adjustable, so as to adjust the position of the bi-color image on the target surface of the camera 25; the third reflector 19 is used for adjusting the position of one path of monochromatic imaged image on the target surface of the camera 25, the third dichroic mirror 21 and the fourth reflector 23 form a double-reflection system, and the position of the other path of monochromatic imaged image on the target surface of the camera 25 is adjusted together.
Here, the position of the double-color image on the target surface of the camera can be adjusted in a self-adaptive manner according to the actually adopted camera and the actual imaging requirement, and the flexibility of the system is improved.
Here, in the actual use process, the angle and position of the third mirror 19 are generally adjusted to realize the adjustment of one image position, and this process can be completed by a single mirror. Then the adjustment of the position of another image needs to be matched with the adjusted image position, and the adoption of the third dichroic mirror 21 and the fourth reflecting mirror 23 to form a double reflecting system is easier to adjust.
Further, in one embodiment, the DMD 3, the camera 25, the
Further, in one embodiment, the reconstruction module comprises:
the image preprocessing unit is used for segmenting the image acquired by the camera and segmenting two strip images with complementary phases by positioning; then, image registration is carried out according to the positions of the stripes of the two images, and non-registered pixels at the edges of the images, which are introduced by errors of an optical system, are removed;
the deconvolution unit is used for respectively carrying out Richardson-Lucy deconvolution processing on the two strip images with complementary phases for a plurality of times;
here, by Richardson-Lucy deconvolution processing, better fringe contrast and better image quality can be obtained, improving the accuracy of the post-slicing algorithm.
Here, for the 488nm illumination source and the 561nm illumination source selected above, exemplarily, in the deconvolution algorithm, the PSF (optical transfer function) used is the real PSF of the system, and the 488nm laser-excited system PSF and the 561nm laser-excited PSF are measured using the multi-wavelength-excited 40nm fluorescence bead, respectively. And then calling an R-L deconvolution tool box of matlab to deconvolute the two fringe images respectively.
Here, the Richardson-Lucy deconvolution processing is performed several times, and preferably 5 times.
The normalization processing unit is used for respectively carrying out intensity normalization processing on the two strip images with complementary phases after deconvolution processing;
here, since the fluorescence quantum efficiency of the sample for different excitation lights is different, the overall intensity of the two images is different, and therefore, the intensity normalization processing needs to be performed on the two images.
The wide-field image generation unit is used for summing and averaging two strip images with complementary phases after intensity normalization processing to obtain a wide-field image;
the local contrast extraction unit is used for extracting the local contrast of the stripes aiming at a certain stripe image to form a local contrast matrix;
a low-frequency information extraction unit, which is used for multiplying the local contrast matrix and the wide field image, and then extracting the low-frequency information of the in-focus information from the wide field image by using low-pass filtering;
here, the local contrast may represent the content of the low frequency information at the location, since the more the low frequency information, the lower the local contrast; while the fringes modulate only the in-focus information, the local contrast of the fringes is correlated with the low frequency part of the in-focus information.
A high-frequency information extraction unit for extracting high-frequency information of in-focus information from the wide-field image by using high-pass filtering;
here, the high frequency information represents detail information of the image.
And the defocusing image generation unit is used for summing the low-frequency information and the high-frequency information to obtain a defocusing image.
In one embodiment, in conjunction with fig. 2, there is provided a DMD based single frame exposure fast three-dimensional fluorescence imaging method, comprising the steps of:
the initial starting step of the system: the first light source and the second light source in the first laser coupling unit and the second laser coupling unit start to emit light, and the intensity of the light emitted by the first light source and the second light source can be adjusted through the FPGA; the FPGA outputs a rising edge signal to trigger the DMD and the camera to work;
and (3) complementary stripe light generation step: emergent light of the first light source and emergent light of the second light source are respectively coupled into the first optical fiber and the second optical fiber, the emergent light of the first laser coupling unit and the emergent light of the second laser coupling unit are respectively incident to the DMD from two sides of the DMD through the first Kohler lighting lens group and the second Kohler lighting lens group, and the reflected light passing through the DMD is vertical to the target surface of the DMD to form complementary stripe light;
a two-color fluorescence excitation step: the complementary stripe light is reflected to the second reflecting mirror through the first cylindrical mirror, the first reflecting mirror and the first dichroic mirror in sequence, and then is excited by the sample through the liquid zoom lens, the first relay lens group and the first objective lens to generate stripe light with different wavelengths carrying bicolor fluorescence;
two-color frame splitting imaging: the fringe light primary path with different wavelengths carrying the bicolor fluorescence returns, and enters the second relay lens group after being reflected by the first objective lens, the first relay lens group, the liquid zoom lens, the second reflector and the first dichroic mirror in sequence and then transmitted; emergent light of the second relay lens group is reflected by the second dichroic mirror and then enters the second color filter and the fourth reflecting mirror, and is transmitted by the second dichroic mirror and then enters the first color filter and the third reflecting mirror; reflected light of the third reflector is transmitted through the second barrel mirror and the third dichroic mirror and then imaged to the camera target surface, reflected light of the fourth reflector is reflected through the third barrel mirror and the third dichroic mirror and then imaged to the camera target surface, and a double-color image is formed;
with reference to fig. 7, the defocus imaging step is implemented: two phase-complementary fringe images acquired by a camera are processed by removing out-of-focus information to obtain out-of-focus removed images, and the specific process comprises the following steps:
step 1, image preprocessing, including segmenting an image acquired by a camera, and segmenting two strip images with complementary phases by positioning; then, image registration is carried out according to the positions of the stripes of the two images, and non-registered pixels at the edges of the images, which are introduced by errors of an optical system, are removed;
step 3, respectively carrying out intensity normalization processing on the two strip images with complementary phases after deconvolution processing;
step 4, summing and averaging two strip images with complementary phases after intensity normalization processing to obtain a wide field image;
step 5, extracting the local contrast of the stripes aiming at a certain stripe image to form a local contrast matrix;
step 6, multiplying the local contrast matrix by the wide field image, and then extracting low-frequency information of in-focus information from the wide field image by utilizing low-pass filtering;
step 7, extracting high-frequency information of in-focus information from the wide-field image by using high-pass filtering;
and 8, summing the low-frequency information and the high-frequency information to obtain a defocused image.
Further, in one embodiment, the method further comprises: when the camera finishes one-frame exposure, the FPGA sends an analog voltage signal to the liquid zoom lens, the light curvature of the liquid zoom lens is changed, and three-dimensional slicing scanning of the sample is realized.
For specific limitations of the DMD based single-frame exposure fast three-dimensional fluorescence imaging method, reference may be made to the above limitations of the DMD based single-frame exposure fast three-dimensional fluorescence imaging system, and details are not repeated here.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
- 上一篇：一种医用注射器针头装配设备
- 下一篇：一种同步大视场测光测谱终端系统及其应用