阅读说明：本技术 基于傅里叶叠层显微成像的led照明优化方法及装置 (LED illumination optimization method and device based on Fourier laminated microscopic imaging ) 是由 赵巨峰 毛海锋 崔光茫 张钰 林君 臧月 于 2019-09-05 设计创作，主要内容包括：本发明公开了一种基于傅里叶叠层显微成像的LED照明优化方法及装置。装置包括计算机以及疏密排列的菱形LED阵列、被测样品、显微物镜、成像透镜、相机。同时提出了一种用于傅里叶叠层显微成像的LED照明优化方法,包括步骤一：基于LED,孔径和样本之间的关系,在傅里叶域中确定频谱范围；步骤二：使用图像质量评估算法来提取任意子LED和全LED之间的差异表达式；步骤三：分析差异表达式,确定照明角度优化方案。本发明能有效解决现有的傅里叶叠层显微成像技术下样品图像采集效率低、重建图像分辨率低的技术问题。(The invention discloses an LED illumination optimization method and device based on Fourier laminated microscopic imaging. The device comprises a computer, a diamond LED array arranged in a sparse and dense mode, a tested sample, a microscope objective, an imaging lens and a camera. Meanwhile, an LED illumination optimization method for Fourier laminated microscopic imaging is provided, and comprises the following steps: determining a spectral range in the fourier domain based on a relationship between the LED, the aperture and the sample; step two: extracting a difference expression between any sub-LED and the full LED using an image quality evaluation algorithm; step three: and analyzing the difference expression to determine an illumination angle optimization scheme. The invention can effectively solve the technical problems of low sample image acquisition efficiency and low reconstructed image resolution under the existing Fourier laminated microscopic imaging technology.)

1. An LED illumination optimization method based on Fourier laminated microscopic imaging is characterized by comprising the following steps:

Step one, determining a frequency spectrum range in a Fourier domain based on a relation among an LED, an aperture and a sample;

Step two, extracting a difference expression between any sub LED and the whole LED by using an image quality evaluation algorithm;

And step three, analyzing the difference expression and determining an illumination angle optimization scheme.

2. The method for LED illumination optimization based on Fourier stacked microscopy imaging as defined in claim 1, wherein: the first step is as follows:

Let the interval between adjacent LED units be D mm, the distance between the LED plane and the sample be h mm, the objective lens parameters be M x (magnification), NA (numerical aperture), the working wavelength be lambda, and the field angle theta₀and the numerical aperture satisfies the following formula:

NA＝sinθ₀

wherein the central LED unit is incident perpendicularly through the sample to the objective lens with a cut-off frequency in the fourier plane of:

Let the incident angle of any sub-LED be phi, the relationship between the LED, the aperture and the sample satisfies the following formula:

a and b are the number difference between the current LED unit and the LED unit from the center along the x and y directions respectively, and r is the corresponding distance;

The interval between the edge LED and the center LED satisfies L ═ r_max/D]，[A]Representing the largest integer less than or equal to a, the effective LED array distribution is (2L +1) × (2L + 1).

3. The method for optimizing LED illumination based on Fourier stacked microscopy as defined in claim 2, wherein: the second step is as follows:

The spectra of adjacent LEDs overlap, making the effective LED unit spectrum F₀the spectrum of the LED at coordinate (i, j) is F_i,jLet the difference function beAdopting two full reference image quality evaluation algorithms of peak signal-to-noise ratio (PSNR) and Structural Similarity (SSIM), and intensity map f₀corresponding to spectrum F₀fres corresponds to residual spectrum (F)₀-F_ij)；

SSIM＝[l(x,y)][c(x,y)][s(x,y)]

wherein mu_x、σ_xAnd mu_y、σ_yMean and variance of x and y, σ, respectively_x,yIs the covariance of x and y, C₁＝C₂＝C₃＝0.01；

And (3) the coordinates (i, j) move on the whole image pixel by pixel to finally obtain a differential matrix of the PSNR and a differential matrix of the SSIM, namely the differential matrix is a difference expression between any sub-LED and the full LED, and the importance contribution level of all LEDs at a single position to the image reconstruction quality is determined through the two differential matrices.

4. the LED illumination optimization method based on Fourier stacked microscopy imaging as claimed in claim 3, wherein: the third step is as follows:

According to the importance contribution level of the LED at the single position obtained in the step two to the image reconstruction effect, when the most important LED range is selected, a main illumination area is extracted according to the shape of a diamond, wherein the main illumination area can be divided into a bright field imaging area and a dark field imaging area; the bright field imaging requires the illumination aperture to be within the numerical aperture of the microscope, and since fourier stack imaging requires a certain aperture overlap ratio in the process of spectrum scanning, the distribution of LEDs in the bright field imaging range can be determined by the following formula:

the distance between the adjacent LEDs in the bright field imaging range is reduced to half of the distance between the adjacent LEDs in the dark field imaging range, and the acquisition quantity of low-resolution images under bright field imaging is increased;

The illumination mode of the diamond LED array is as follows: the LED array lights the most central LED unit firstly, n is 0, n is a first control command sent by a computer, and then when n is 1, the LED array lights the outer circle of the LED units one by one according to a rule; by analogy, when n is greater than 1, the lighting range of the LED array is that the number of LED breviscapines along the diagonal line in the horizontal direction is 2n +3, the number of LED breviscapines along the diagonal line in the vertical direction is 2n +1, and finally the whole lighting range is in a diamond shape.

5. the utility model provides a little device of Fourier stromatolite based on LED array illumination optimization, includes camera (1) that sets gradually from top to bottom along the light path, imaging lens (2), microscope objective (3), sample 4, objective table 5, sparsely arranged rhombus LED array (6) and computer (7), its characterized in that: the sparsely arranged rhombic LED arrays (6) sequentially light the light emitting units to provide illumination light at different angles for the tested sample 4; a sample (4) to be measured is placed on an objective table (5), is positioned between a diamond LED array (6) and a microscope objective (3) which are arranged sparsely, and is positioned on a focal plane of the microscope objective (3); the microscope objective (3) and the imaging lens (2) jointly form a 4f system; the illumination light beams are emitted from the rhombic LED array (6), are scattered by the tested sample (4), are imaged on the back focal plane of the imaging lens (2) through the 4f system, and are recorded by the camera (1); the lighting sequence of each sub-LED and the image capture of the camera are controlled by external computer programming.

Technical Field

The invention belongs to the technical field of optical microscopic imaging, relates to a Fourier laminated microscopic imaging technology, and particularly relates to a Fourier laminated microscopic imaging-based LED illumination optimization method and device.

Background

Fourier imaging is a method of balancing high resolution and wide field of view in microscopic imaging, which is a recently developed imaging technique. In fourier microscopy, a sample is illuminated with different oblique angles of incidence and corresponding low resolution images are acquired. By using a Fourier imaging restoration method, researchers can iteratively synthesize original images to generate high-resolution images. Typically, fourier imaging switches between the spatial and fourier domains. The acquired images are intensity signals, which are intensity constrained in the spatial domain. The support constrains restricted pupil function from the objective lens. The pupil function constraint translates over the fourier frequency space as the illumination angle changes. In this process, too much time is spent acquiring too many low resolution images. Initially, illumination is performed using a periodic LED array, and this uniform illumination sampling scheme results in a large number of raw images being acquired, such that acquisition and iterative reconstruction take a large amount of time, and the acquired low resolution images under a large number of dark-field imaging make the reconstructed image resolution modest.

Disclosure of Invention

The invention provides an LED illumination optimization method and device based on Fourier laminated microscopic imaging, and aims to solve the technical problems that the efficiency of acquiring a sample image and the quality resolution of a reconstructed image are low in the prior art.

the invention provides an LED illumination optimization method based on Fourier laminated microscopic imaging, which is characterized by comprising the following steps of:

Step one, determining a frequency spectrum range in a Fourier domain based on the relation among an LED, an aperture and a sample.

And step two, extracting a difference expression between any sub LED and the full LED by using an image quality evaluation algorithm.

And step three, analyzing the difference expression and determining an illumination angle optimization scheme.

Determining spectral range in Fourier domain based on relationship between LED, aperture and sample

Let the spacing between adjacent LED units be Dmm, the distance between the LED plane and the sample be h mm, Mx (magnification), NA (numerical aperture), the operating wavelength be λ, and the field angle θ₀and the numerical aperture satisfies the following formula:

NA＝sinθ₀；

Wherein the central LED unit is incident perpendicularly through the sample to the objective lens with a cut-off frequency in the fourier plane of:

Let the incident angle of any sub-LED be φ (angle α between incident light and X-axis, angle β between incident light and Y-axis), the relationship between LED, aperture and sample satisfies the following equation:

a and b are the number difference between the current LED unit and the central LED unit in x and y directions, respectively, and r is the corresponding distance. The interval between the edge LED and the center LED satisfies L ═ r_max/D]。[A]Representing the largest integer less than or equal to a, the effective LED array distribution is (2L +1) × (2L + 1).

spectral range in the Fourier domain is determined by the angle of incidence φ of the edge LED₀(the angle between the light and the X-axis of the LED array max (α), the angle between the light and the Y-axis of the LED array max (β)). For a synthetic aperture, the cutoff frequency values along the X-axis and the Y-axis satisfy the following formula:

Where max (α) is max (β) as an index for selecting the LED array according to the present invention.

Extracting a difference expression between an arbitrary sub-LED and a full LED using an image quality evaluation algorithm

The spectra of adjacent LEDs overlap. The fourier space domain changes due to the difference in the angles of incidence, and the overlap region changes when the position of the LED changes. To select the minimum number of LEDs to achieve fast low resolution image acquisition efficiency, the difference of each LED needs to be found. Therefore, the present invention analyzes the redundancy of the overlapping spectra and extracts a differential expression between any LED and the full spectrum. Firstly, the effective LED unit spectrum is F₀The spectrum of the LED at coordinate (i, j) is F_i,jLet the difference function benamely PSNR (peak signal-to-noise ratio) and SSIM (structural similarity). Intensity map f₀Corresponding to spectrum F₀Fres corresponds to residual spectrum (F)₀-F_ij) The PSNR and SSIM evaluation algorithms satisfy the following formula:

SSIM＝[l(x,y)][c(x,y)][s(x,y)]

Wherein mu_x、σ_xand mu_y、σ_yThe mean and variance of x and y, respectively. Sigma_x,yis the covariance of x and y, C₁＝C₂＝C₃＝0.01。

Multiple sets of experimental data show that the larger the PSNR and SSIM results are, the closer x and y are. Thus when f is_resAnd f₀Instead of x and y, we can get a fully reconstructed image and no f_ijthe difference between the information images. And (4) moving ij on the whole image pixel by pixel to finally obtain a differential matrix of the PSNR and a differential matrix of the SSIM, wherein the differential matrix is a difference expression between any sub-LED and the whole LED, and the importance contribution level of all LEDs at a single position to the image reconstruction quality can be determined through the two differential matrices.

analyzing the difference expression to determine the illumination angle optimization scheme

According to the importance contribution level of the LED with the single position to the image reconstruction effect realized in the step two, when the most important LED range is selected, the main illumination area can be extracted according to the shape of a diamond, wherein the main illumination area can be divided into a bright field imaging area and a dark field imaging area. Bright field imaging requires the illumination aperture to be within the numerical aperture of a microscope, and since fourier stack imaging requires a certain aperture overlap ratio (greater than 60%) in the spectral scanning process, the distribution of LEDs in the bright field imaging range is determined by the following formula:

The invention proposes to reduce the spacing (1/2Dmm) of adjacent LEDs in the bright field imaging range to half the spacing (Dmm) of adjacent LEDs in the dark field imaging range to improve the number of low resolution image acquisitions for bright field imaging. Meanwhile, the illumination mode of the diamond LED array is as follows: the LED array lights the most central LED unit first, n is 0(n is the first control command sent by the computer), and then when n is 1, the LED array lights the outer circle of the LED units one by one according to a rule. By analogy, when n >1, the number of the LED lamps of the diagonal line along the horizontal direction in the illumination range of the LED array is 2n +3, the number of the LED lamps of the diagonal line along the vertical direction is 2n +1, and finally the whole illumination range is in a diamond shape.

The invention also provides a Fourier laminated microscopic device based on LED array illumination optimization, which comprises the following components in parts by weight: the device comprises a camera, an imaging lens, a microscope objective, a sample, an objective table, a diamond LED array and a computer which are arranged sparsely from top to bottom along a light path. The LED arrays are sequentially lightened to provide illumination light at different angles for the sample to be measured; the sample to be measured is arranged between the LED array and the microscope objective, and the sample to be measured is positioned on the focal plane of the microscope objective; the microscope objective and the imaging lens jointly form a 4f system, the illumination light beam is emitted from the LED array, is scattered by a tested sample, then is imaged on the back focal plane of the imaging lens through the 4f system, and is recorded by a camera; the microscope objective and the imaging lens are both arranged on a microscope, and the objective table of the microscope is used for fixing a sample. The lighting sequence of each sub-LED and the image capture of the camera are controlled by external computer programming.

compared with the prior art, the invention has the following advantages and effects:

(1) the acquisition of low-resolution images is effectively reduced, and the image reconstruction efficiency is improved.

(2) the acquisition of bright field imaging images is effectively increased, and the image resolution is improved.

Drawings

FIG. 1 is a schematic view of the apparatus of the present invention;

FIG. 2 is a sparsely populated diamond LED array;

The reconstructed high resolution image in the embodiment of fig. 3 (using 57 LED illuminations in total);

FIG. 4 shows a reconstructed high resolution image under conventional LED array illumination (using 225 LED illuminations in total);

The reference numerals in the drawings mean: the method comprises the following steps of 1-camera, 2-imaging lens, 3-microscope objective, 4-sample, 5-objective table, 6-diamond LED array, 7-computer, 8-bright field imaging range and 9-dark field imaging range.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The following describes the practice of the present invention in conjunction with the accompanying drawings.

As shown in FIG. 1, the Fourier laminated microscope device based on LED array illumination optimization of the invention comprises a camera 1, an imaging lens 2, a microscope objective 3, a sample 4, an objective table 5, a diamond LED array 6 arranged sparsely and a computer 7 which are arranged along an optical path from top to bottom in sequence. The diamond LED arrays 6 which are arranged sparsely sequentially light the light emitting units to provide illumination light at different angles for the sample 4 to be measured; the tested sample 4 is arranged on the objective table 5, is positioned between the diamond LED array 6 and the microscope objective 3 which are arranged sparsely, and is positioned on the focal plane of the microscope objective 3; the microscope objective 3 and the imaging lens 2 together form a 4f system. The illumination light beams are emitted from the rhombic LED array 6, are scattered by the tested sample 4, then are imaged on the back focal plane of the imaging lens 2 through the 4f system, and are recorded by the camera 1; the distance from the sparsely arranged diamond-shaped LED array 6 to the sample 4 adopted in the embodiment is 90mm, the illumination light wavelength is 630nm, the numerical aperture of the objective lens is 0.1, and the size of an imaging pixel is 2.25 um.

As shown in fig. 2, the illumination light source used in the present embodiment is a diamond LED array 6 arranged sparsely, wherein the LED array interval in the bright field imaging range 8 is set to 2mm, and the LED array interval in the dark field imaging range 9 is set to 4 mm.

image acquisition: the computer 7 controls the LED array to illuminate while controlling the camera to capture a low resolution image. Firstly, the computer controls the LED array to light the most central LED unit, n is 0(n is a first control command sent by the computer), and then the LED array lights the outer circle of the LED units one by one according to a rule when n is 1 according to a constraint condition that adjacent sub-LEDs must be overlapped. By analogy, when n >1, the number of the LED lamps along the diagonal line in the horizontal direction is 2n +3, the number of the LED lamps along the diagonal line in the vertical direction is 2n +1, and finally the whole illumination range is a rhombus as shown in fig. 2. In the present embodiment, a total of 57 low resolution images are acquired.

image reconstruction: reconstructing the acquired low-resolution image by the computer, and reconstructing by using a classical GS phase recovery algorithm to obtain a high-resolution and wide-field microscopic image, as shown in FIG. 3; by contrast, fig. 4 is a high resolution image reconstructed under conventional LED array (225) illumination. Under the condition of keeping other imaging system parameters unchanged, the quality of the reconstructed high-resolution image in the embodiment is superior to that of the reconstructed high-resolution image under the illumination of the traditional LED array by comparing two high-resolution images, and the Fourier microscope system only adopts 57 images but is far smaller than that of the traditional 225 images, so that the overlapping rate and the data redundancy of low-frequency regions in a frequency spectrum can be effectively increased by using the sparsely arranged diamond LED array for illumination, and the image reconstruction efficiency and the image reconstruction quality can be further improved.