阅读说明：本技术 一种基于led阵列的光谱显微成像装置及其实现方法 (Spectral microscopic imaging device based on LED array and implementation method thereof ) 是由 颜成钢 王逸铮 吕彬彬 孙垚棋 张继勇 李宗鹏 于 2021-06-30 设计创作，主要内容包括：本发明公开了一种基于LED阵列的光谱显微成像装置及其实现方法,光谱显微成像装置包括依次设置的15×15单光源窄光谱红光光源LED阵列,载物台,显微物镜,视场光阑,4F中继透镜,阿米西棱镜,带通滤波器,微透镜阵列和CCD阵列工业相机；所述的4F中继透镜共有两组,分别设置在视场光阑和阿米西棱镜之间以及带通滤波器和微透镜阵列之间；本发明装置通过光路设计,达到15×15单光源窄光谱红光光源LED阵列中单一波长的LED灯逐一点亮225次即可同时获得观测样本多路单个连续光谱通道,可以实时获得观察样本单个光谱图像视频信息,无时间延时,无计算耗时；本发明装置采用了阿米西棱镜,其体积较小并且可以使物镜和目镜位于一条直线上,且阿米西棱镜不会受限于全反射的临界角,能接受较大角度的入射光。(The invention discloses a spectral microimaging device based on an LED array and an implementation method thereof, wherein the spectral microimaging device comprises a 15 multiplied by 15 single-light-source narrow-spectrum red-light-source LED array, an objective table, a microobjective, a field diaphragm, a 4F relay lens, an Amisy prism, a band-pass filter, a microlens array and a CCD array industrial camera which are sequentially arranged; the 4F relay lenses are divided into two groups and are respectively arranged between the field diaphragm and the Amisy prism and between the band-pass filter and the micro-lens array; the device disclosed by the invention has the advantages that through the light path design, the LED lamps with a single wavelength in the 15X 15 single-light-source narrow-spectrum red-light-source LED array are lightened for 225 times one by one, so that multiple paths of single continuous spectrum channels of an observation sample can be obtained simultaneously, the video information of a single spectrum image of the observation sample can be obtained in real time, and no time delay or calculation time is consumed; the device adopts the Amisy prism, the size of the device is small, the objective lens and the ocular lens can be positioned on the same straight line, and the Amisy prism is not limited by the critical angle of total reflection and can accept incident light with larger angle.)

1. The spectral microscopic imaging device based on the LED array is characterized by comprising a 15 multiplied by 15 single-light-source narrow-spectrum red-light-source LED array (1), an objective table (2), a microscope objective (3), a field diaphragm (4), a 4F relay lens (5), an Amisy prism (6), a band-pass filter (7), a microlens array (8) and a CCD array industrial camera (9) which are sequentially arranged; the 4F relay lenses (5) are divided into two groups and are respectively arranged between the field diaphragm (4) and the Amisy prism (6) and between the band-pass filter (7) and the micro-lens array (8);

2. the spectral microimaging device based on the LED array according to claim 1, characterized in that the imaging lens of the microscope objective (3) is used for acquiring two-dimensional image information of the sample on the objective table (2), imaging the two-dimensional image information on the plane where the field stop (4) is located, and then relaying the two-dimensional image information to the surface of the Amisy prism (6) through the first set of 4F relay lenses (5); the Amisy prism (6) cuts the image along the central line and exchanges the left part and the right part, the band-pass filter (7) enables the spectral band to be recorded in the +1 level with the highest brightness of the Amisy prism (6) to pass through independently, and the band and light rays on other grating levels are shielded; at the moment, light after grating dispersion is converged on the plane where the micro-lens array (8) is located again through the second group of 4F relay lenses (5), light with different wavelengths is focused at the position behind the micro-lens array (8) and at the distance F from the focal length of the micro-lens, the continuous spectrum is linearly expanded along the grating dispersion direction, and the expanded image is imaged on the CCD array industrial camera (9) again; the numerical aperture of the whole optical path system needs to be matched in front and at the back, namely the numerical aperture of the light projected onto the micro lens array (8) and the numerical aperture of the micro lens array (8) cannot exceed a set threshold value and are as close as possible to avoid image overlapping confusion.

3. The method for realizing the spectral microscopic imaging device based on the LED array according to the claim 1 or 2, is characterized by comprising the following steps:

the method comprises the following steps: when an object is irradiated by the 15 x 15 single-light-source narrow-spectrum red-light-source LED array (1), 225 LED lamps which are the same in interval and arranged into a 15 x 15 square array emit light rays with specific monochromatic wavelengths from different angles to successively illuminate an observed object on the objective table (2) for 255 times, and an imaging lens of a microscope objective (3) images a real image of the observed object on a plane where a field diaphragm (4) is located and is mapped on the surface of an Amisy prism (6) through a first group of 4F relay lenses (5);

step two: the real image of an observed object mapped on the surface of the Amisy prism (6) is subjected to dispersion, and a band-pass filter (7) enables spectral wave bands L1 to Ln to be recorded in the +1 level with the highest brightness of the Amisy prism (6) to pass through independently and to be converged on a micro lens array (8) again through a second group of 4F relay lenses (5);

step three: because the real image of the observation object mapped on the surface of the Amisy prism (6) has a dispersion angle, light with different wavelengths is converged on the microlens array (8) again, the real image has different exit angles, the dispersion occurs along one dimension on the microlens focal plane (10), and the dispersed real image of the observation object is imaged on a pixel array of a CCD array industrial camera (9);

step four: each microlens in the microlens array (8) corresponds to a sub-pixel (11) area in a pixel array of the CCD array industrial camera (9), the size of the sub-pixel (11) is N multiplied by N pixels, wherein N is an odd number, and 3<N<13; and the emergent light passing through the micro-lens is projected to a row of pixels of which the sub-pixels (11) are positioned in the middle; at this time, the pixels at the corresponding positions in the middle row in the sub-pixels (11) are recombined in a manner that the ith pixel in the (N +1)/2 th row in the sub-pixel (11) corresponding to each microlens is combined into the ith image Ai according to the microlens position sequence, wherein i is 1 and 2 … … N, and the observed object on the objective table (2) at the lambda position can be obtained_iSpectral image A corresponding to wavelength_iWherein:

λ_i＝L1+(i-0.5)×(Ln-L1)/N；

step five: automatically intercepting the pictures of 15 multiplied by 15LED lamps after the LED lamps are lightened by a CCD array camera by utilizing a pre-programmed MATLAB image interception function script to generate 225 pictures, and renaming and sequencing the pictures so as to facilitate the next image analysis and processing; the picture pixel generated by the camera is C × D;

step six:

converting a time domain coordinate of each LED lamp into a frequency domain coordinate;

intercepting the spectrum information in the corresponding sub-aperture on the high-resolution spectrum of the object by using the pupil function obtained previously;

thirdly, updating the amplitude information of the target light field by using the low-resolution-intensity image recorded under the corresponding inclined plane wave;

fourthly, further normalization processing is carried out on the pupil function of the picture;

updating the spectrum information and the pupil function in the corresponding sub-aperture in the high-resolution spectrum of the object by updating the spectrum distribution difference of the target light field before and after updating;

continuously iterating and reconstructing the information of the picture;

seventhly, calculating error parameters;

and generating 225 new pictures by the obtained KxK image information and the reconstructed images, and further obtaining the information of the images.

4. The method for realizing the spectral microscopic imaging device based on the LED array according to claim 3, characterized in that the six detailed steps are as follows:

225 (225-15 × 15) original images are input, and the images are numberedK is the number of groups, (m, n) represents the image produced by the LEDs in the rows and columns,is an image original frequency spectrum function; calculating to obtain a pupil function P (u, v) | exp [ i |. 2 π | (u, v, x) of the optical path system_t,y_t)](ii) a Where (u, v) represents frequency domain coordinates, | P (u, v) | is the amplitude of the pupil function, (x)_t,y_t) Is a spatial coordinate at a location in the field of view;

(1) performing coordinate conversion on the LED array; let the central point LED lamp coordinate be (x)₀,y₀) The unit coordinates of the LED lamps in the mth row and the nth column are LEDs_m,n＝(x_m,n，y_m,n) (ii) a Let lambda be the wavelength of incident light, and h be the distance between the LED lamp at the central point of the LED array and the central point of the sample, then the LED_m,nCorresponding inclined planeThe spatial frequency of the wave is (u)_m,n,v_m,n)；

(2) At the LED_m,nUnder illumination of a single lamp of the array, the previously determined pupil function P (u, v) is used to achieve a high-resolution spectral object^kUp-clipping the spectral information in the corresponding sub-apertureAnd generating corresponding low-resolution complex amplitude function on the image surface through inverse Fourier transformReferred to as the target light field;

(3) keeping the phase information of the target light field unchanged, using low-resolution intensity images recorded under corresponding oblique plane wavesUpdating amplitude information of a target light field

(4) The device is a microscopic imaging device with a circular aperture, and can thus be represented as a circular low-pass filter with a cut-off frequency u_c＝NA_objλ, wherein NA_objλ is the wavelength of the incident light, which is the numerical aperture of the device; carrying out further normalization processing on the pupil function P (u, v) of the picture, wherein the processing formula is as follows;

(5) fourier transform for obtaining frequency spectrum of updated target light field And through the difference of the frequency spectrum distribution of the target light field before and after updatingTo update the spectral information and pupil function in the corresponding sub-aperture in the object high-resolution spectrum:

wherein is the complex conjugate operator;

(6) performing phase recovery on the images by using the target light field obtained in the step (3), performing iteration in a spiral line mode from the upper left corner of the central lighting, and expanding the cut-off frequency by using the target light field of each image; at the overlapping position, carrying out two-removing processing on the phase information of the overlapping area of the overlapping part of the low-pass channel;

(7) repeating the steps (2) to (5) H times to update the frequency spectrum components corresponding to other illumination angles; when all the illumination angles are updated once, an iteration process is completed; after H times, until the reconstruction algorithm converges, so as to obtain the optimal solution of the complex amplitude of the object under high resolution; taking the error value E of the amplitude of the object after each iteration as a termination condition of the algorithm loop, taking j as the iteration frequency, and jumping out of the loop to finish the reconstruction process if the iteration frequency is less than a set threshold value;

(8) reconstructing the target light field converged in the step (7) to obtain a target light intensity map, wherein the reconstructed map is a super-resolution map of the Kth spectral band; k multiplied by K image information is obtained from the super-resolution image of the Kth spectral band and recombined into a new 225 images, and further image information is obtained.

5. The method of claim 4, wherein H is 2-3.

Technical Field

The invention relates to the field of spectral imaging, in particular to a rapid spectral microscopic imaging device.

Background

Compared with the traditional imaging technology, the spectral imaging is used for shooting a two-dimensional image of a sample and simultaneously recording the one-dimensional spectral information and the two-dimensional spatial information. The spectrum imaging technology can improve and increase the richness of the recorded information, and is favorable for simple and convenient further analysis and processing in the later period. In the initial application stage of the spectral imaging technology, the traditional experimental method is used for acquiring spectral information, namely two-dimensional spatial information and one-dimensional spectral information at corresponding wavelengths are recorded through a narrow-band filter. The method has the advantages of high precision and easy realization. But at the same time it has the disadvantage that the system can only acquire a limited number of spectral channel information of the sample and the acquired spectral information is not consistent. Moreover, the method cannot record the spectral information of the sample on different spectral channels at the same time, so that only the spectral imaging of the sample in a static scene can be realized.

The rapid spectral microscopic imaging technology can realize the information acquisition of a plurality of continuous single spectral channels, so that the acquired spectral data is richer and more accurate. Therefore, the rapid spectral microscopic imaging technology can effectively solve the problems that the spectrum channels in the early spectral imaging technology are less in disorder and the aliasing acquisition of the dynamic scene images cannot be processed.

Disclosure of Invention

Aiming at the defects in the prior art, the invention provides a spectral microscopic imaging device based on an LED array and an implementation method thereof. The method can continuously record a plurality of single spectral channel information and can be used for recording spectral microscopic imaging of dynamic and static scenes.

The utility model provides a spectral microimaging device based on LED array, includes 15 x 15 single light source narrow spectrum red light source LED array (1), objective table (2), microobjective (3), field of view diaphragm (4), 4F relay lens (5), amici prism (6), band-pass filter (7), microlens array (8) and CCD array industry camera (9) that set gradually. The 4F relay lenses (5) are divided into two groups and are respectively arranged between the field diaphragm (4) and the Amisy prism (6) and between the band-pass filter (7) and the micro-lens array (8);

an imaging lens of the microscope objective (3) is used for acquiring two-dimensional image information of a sample on the objective table (2), imaging the two-dimensional image information on a plane where the field diaphragm (4) is located, and relaying the two-dimensional image information to the surface of the Amisy prism (6) through the first group of 4F relay lenses (5). The Amisy prism (6) cuts the image along the central line and exchanges the left part and the right part, and the band-pass filter (7) enables the spectral band to be recorded in the +1 stage with the highest brightness of the Amisy prism (6) to pass through independently and shields the band and light rays on other grating stages. At the moment, light after grating dispersion is converged on the plane where the micro-lens array (8) is located again through the second group of 4F relay lenses (5), light with different wavelengths is focused at the position behind the micro-lens array (8) and at the distance F from the focal length of the micro-lens, the continuous spectrum is linearly expanded along the grating dispersion direction, and the expanded image is imaged on the CCD array industrial camera (9) again. The numerical aperture of the whole optical path system needs to be matched in front and at the back, namely the numerical aperture of the light projected onto the micro lens array (8) and the numerical aperture of the micro lens array (8) cannot exceed a set threshold value and are as close as possible to avoid image overlapping confusion.

By adopting the structure, because the micro-lens array performs sampling segmentation on the imaging in the visual field, the imaging of different spectrum channels can be focused in different pixel coordinates, and the pixels at corresponding positions in the sub-pixels are selected to be recombined to obtain corresponding spectrum information.

The invention has the following beneficial effects:

the device disclosed by the invention has the advantages that through the light path design, the LED lamps with a single wavelength in the 15X 15 single-light-source narrow-spectrum red-light-source LED array are lightened for 225 times one by one, so that multiple paths of single continuous spectrum channels of an observation sample can be obtained simultaneously, the video information of a single spectrum image of the observation sample can be obtained in real time, and no time delay or calculation time is consumed; the device adopts the Amisy prism, the size of the device is small, the objective lens and the ocular lens can be positioned on the same straight line, and the Amisy prism is not limited by the critical angle of total reflection and can accept incident light with larger angle.

Drawings

FIG. 1 is a schematic diagram of an FPM reconstruction according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a spectral microscopic imaging device according to an embodiment of the invention.

Reference numerals: the system comprises a 15X 15 single-light-source narrow-spectrum red light source LED array 1, an objective table 2, a microscope objective 3, a field diaphragm 4, a 4F relay lens 5, an Amisy prism 6, a band-pass filter 7, a micro-lens array 8, a CCD array industrial camera 9, a micro-lens focal plane 10 and sub-pixels 11.

Detailed Description

The invention is further described with reference to the following figures and examples.

The invention discloses a spectral microscopic imaging device based on an LED array, which is characterized in that LED lamps with the same interval in the 15 x 15LED array are continuously lightened for 225 times from different positions to realize pixel recovery of high-frequency signals, and further a plurality of continuous spectral information of a biological sample is acquired, and the spectral microscopic imaging device comprises the following steps:

as shown in fig. 2, a spectral microimaging device based on an LED array includes: the system comprises a 15X 15 single-light-source narrow-spectrum red-light-source LED array 1, an objective table 2, a microscope objective 3, a field diaphragm 4, a 4F relay lens 5, an Amisy prism 6, a band-pass filter 7, a micro-lens array 8 and a CCD array industrial camera 9 which are sequentially arranged from left to right; the 4F relay lenses 5 are divided into two groups and respectively arranged between the field diaphragm 4 and the Amisy prism 6 and between the band-pass filter 7 and the micro-lens array 8;

225 LED lamps which are arranged in a 15 x 15 square array and have the same interval in the 15 x 15 single-light-source narrow-spectrum red-light-source LED array 1 emit single-spectrum light from different positions, and illuminate an observation object on the object stage 2 for 255 times continuously. The imaging lens of the microscope objective 3 images the real image of the observed object on the plane where the field diaphragm 4 is located, and the real image is mapped on the surface of the Amisy prism 6 through the first group of 4F relay lenses 5.

The real image of the observation object mapped on the surface of the amici prism 6 is dispersed, and the band-pass filter 7 passes the spectral band to be recorded in the +1 level with the highest brightness of the amici prism 6 independently and is converged on the micro-lens array 8 again through the 4F relay lens 5. Then, dispersion occurs along one dimension on the focal plane 10 of the microlens, and a real image of the observed object after dispersion is imaged on a pixel array of the CCD array industrial camera 9.

An implementation method of a spectral microscopic imaging device based on an LED array comprises the following steps:

the method comprises the following steps: when the 15 × 15 single-light-source narrow-spectrum red light source LED array 1 irradiates an object, 225 LED lamps with the same interval and arranged in a 15 × 15 square array emit specific monochromatic wavelength light from different angles to illuminate an object to be observed on the stage 2 255 times in succession, and an imaging lens of the microscope objective 3 images a real image of the object to be observed on a plane where the field stop 4 is located, and the real image is mapped on the surface of the amici prism 6 through the first group of 4F relay lenses 5.

Step two: the real image of the observed object mapped on the surface of the amici prism 6 is dispersed, and the band-pass filter 7 passes through the spectral bands L1 to Ln to be recorded in the +1 level with the highest brightness of the amici prism 6 alone and converges again on the microlens array 8 through the second group of 4F relay lenses 5.

Step three: since the real image of the observation object mapped on the surface of the amici prism 6 has a dispersion angle, light with different wavelengths is converged on the microlens array 8 again, the real image has a different exit angle, the dispersion occurs along one dimension on the microlens focal plane 10, and the dispersed real image of the observation object is imaged on the pixel array of the CCD array industrial camera 9.

Step four: each microlens in the microlens array 8 corresponds to a sub-pixel 11 region in the pixel array of the CCD array industrial camera 9, the size of the sub-pixel 11 is N multiplied by N pixels, wherein N is an odd number, and 3<N<13. And the outgoing light passing through the microlens will be projected onto a row of pixels in the middle of the sub-pixels 11. At this time, the middle row of the sub-pixels 11 is mappedThe position pixels are recombined in a manner that the ith pixel in the (N +1)/2 th row in the sub-pixel 11 corresponding to each microlens is combined into the ith image Ai according to the microlens position sequence, wherein i is 1 and 2 … … N, so that the observed object on the object stage 2 on the lambda is obtained_iSpectral image A corresponding to wavelength_iWherein;

λ_i＝L1+(i-0.5)×(Ln-L1)/N。

step five: and automatically intercepting the pictures of 15 multiplied by 15LED lamps after the LED lamps are lightened by the CCD array camera by utilizing a pre-programmed MATLAB image interception function script to generate 225 pictures, and renaming and sequencing the pictures so as to facilitate the next image analysis and processing. The picture pixel generated by the camera is C × D (C, D value is the number of horizontal and vertical pixels of the obtained image, and the specific parameters of the corresponding camera can be inquired).

Step six:

converting a time domain coordinate of each LED lamp into a frequency domain coordinate;

intercepting the spectrum information in the corresponding sub-aperture on the high-resolution spectrum of the object by using the pupil function obtained previously;

thirdly, updating the amplitude information of the target light field by using the low-resolution-intensity image recorded under the corresponding inclined plane wave;

fourthly, further normalization processing is carried out on the pupil function of the picture;

updating the spectrum information and the pupil function in the corresponding sub-aperture in the high-resolution spectrum of the object by updating the spectrum distribution difference of the target light field before and after updating;

continuously iterating and reconstructing the information of the picture;

seventhly, calculating error parameters;

and generating 225 new pictures by the obtained KxK image information and the reconstructed images, and further obtaining the information of the images. The detailed steps are as follows:

225 (225-15 × 15) original images are input, and the images are numberedK is the number of groups, (m, n) denotes the numberThe image produced by the row and column LEDs,is the original spectrum function of the image. Calculating to obtain a pupil function P (u, v) | exp [ i |. 2 π | (u, v, x) of the optical path system_t,y_t)]. Where (u, v) represents frequency domain coordinates, | P (u, v) | is the amplitude of the pupil function, (x)_t,y_t) Is a spatial coordinate at a location in the field of view;

(1) and carrying out coordinate conversion on the LED array. Let the central point LED lamp coordinate be (x)₀,y₀) The unit coordinates of the LED lamps in the mth row and the nth column are LEDs_m,n＝(x_m,n，y_m,n). Let lambda be the wavelength of incident light, and h be the distance between the LED lamp at the central point of the LED array and the central point of the sample, then the LED_m,nThe corresponding tilted plane wave has a spatial frequency of (u)_m,n,v_m,n)；

(2) At the LED_m,nUnder illumination of a single lamp of the array, the previously determined pupil function P (u, v) is used to achieve a high-resolution spectral object^kUp-clipping the spectral information in the corresponding sub-apertureAnd generating corresponding low-resolution complex amplitude function on the image surface through inverse Fourier transformReferred to as the target light field;

(3) keeping the phase information of the target light field unchanged, using low-resolution intensity images recorded under corresponding oblique plane wavesUpdating amplitude information of a target light field

(4) The device is a microscopic imaging device with a circular aperture, and can thus be represented as a circular low-pass filter with a cut-off frequency u_c＝NA_objλ, wherein NA_objλ is the wavelength of the incoming light, which is the numerical aperture of the device. Carrying out further normalization processing on the pupil function P (u, v) of the picture, wherein the processing formula is as follows;

(5) fourier transform for obtaining frequency spectrum of updated target light field And through the difference of the frequency spectrum distribution of the target light field before and after updatingTo update the spectral information and pupil function in the corresponding sub-aperture in the object high-resolution spectrum:

where is the complex conjugate operator.

(6) And (4) performing phase recovery on the images by using the target light field obtained in the step (3), performing iteration in a spiral line mode from the upper left corner of the central lighting, and expanding the cut-off frequency by using the target light field of each image. The specific iterative method of the spiral is shown in fig. 1. At the overlapping position, carrying out two-removing processing on the phase information of the overlapping area of the overlapping part of the low-pass channel;

(7) repeating the steps (2) to (5) H times to update the spectrum components corresponding to other illumination angles. When all illumination angles are updated once, an iterative process is completed. And after H times, until the reconstruction algorithm converges, thereby obtaining the optimal solution of the complex amplitude of the object under high resolution. In general, the error value E of the amplitude of the object after each iteration is used as a termination condition of the algorithm loop, j is the iteration number, and if the iteration number is smaller than a set threshold (usually 1%), the loop is skipped to complete the reconstruction process.

Further, the value of H is 2-3.

(8) And (5) reconstructing the target light field converged in the step (7) to obtain a target light intensity map, wherein the reconstructed map is a super-resolution map of the Kth spectral band. K multiplied by K image information is obtained from the super-resolution image of the Kth spectral band and recombined into a new 225 images, and further image information is obtained.