阅读说明：本技术 基于多图像面积关系的显微成像系统自动对焦方法及装置 (Automatic focusing method and device of microscopic imaging system based on multi-image area relation ) 是由 史金飞 何睿清 张健 郝飞 于 2021-05-24 设计创作，主要内容包括：本发明公开了一种基于多图像面积关系的显微成像系统自动对焦方法,包括以下过程：将两束激光光束投射到样品表面,并接收样品表面反射的两束激光光束形成的两个面积不同的光斑图像,获得显微物镜与样品之间的距离与两个光斑面积之间对应关系；调整显微物镜与样品之间的距离,由不同距离与光斑面积之间映射关系形成面积查找表；获取待测环境中光斑图像,根据面积查找表获得此光斑面积对应的显微物镜与样品之间距离；根据此距离计算得到离焦距离,基于此离焦距离移动显微物镜相对样品位置,从而实现显微成像系统的自动对焦。本发明采用两条光束投影到测量物体表面,利用两者之间的相互关系来考察聚焦情况,提高检测精度。(The invention discloses a microscopic imaging system automatic focusing method based on a multi-image area relation, which comprises the following processes: projecting the two laser beams to the surface of a sample, and receiving two spot images with different areas formed by the two laser beams reflected by the surface of the sample to obtain the corresponding relation between the distance between the microscope objective and the sample and the areas of the two spots; adjusting the distance between the microscope objective and the sample, and forming an area lookup table by the mapping relation between different distances and the areas of the light spots; acquiring a light spot image in an environment to be detected, and acquiring the distance between the microscope objective and the sample corresponding to the area of the light spot according to an area lookup table; and calculating the distance from the focal point according to the distance, and moving the position of the microscope objective relative to the sample based on the distance from the focal point, thereby realizing the automatic focusing of the microscope imaging system. The invention adopts two light beams to project on the surface of a measuring object, and utilizes the mutual relation between the two light beams to investigate the focusing condition, thereby improving the detection precision.)

1. A microscopic imaging system automatic focusing method based on multi-image area relation is characterized by comprising the following processes:

based on a microscopic imaging system, projecting two laser beams to the surface of a sample through a microscopic objective lens, receiving two light spot images formed by the two laser beams reflected by the surface of the sample, and calculating the areas of the two light spots to obtain the corresponding relation between the distance between the microscopic objective lens and the sample and the areas of the two light spots in the microscopic imaging system;

calculating to obtain two corresponding light spot areas by adjusting the distance between the microscope objective and the sample, and forming an area lookup table by the mapping relation between different distances and the light spot areas;

acquiring a light spot image in an environment to be detected, and acquiring the distance between the microscope objective and the sample corresponding to the area of the light spot according to an area lookup table;

and calculating the distance from the focal point according to the distance, and moving the position of the microscope objective relative to the sample based on the distance from the focal point, thereby realizing the automatic focusing of the microscope imaging system.

2. The method as claimed in claim 1, wherein the microscopic imaging system comprises: the device comprises a first laser, a second laser, a first beam splitter prism, a second beam splitter prism, a third beam splitter prism, a microscope objective, a lens, a fourth beam splitter prism, a first band-pass filter, a second band-pass filter, a first area array detector, a second area array detector, a field lens and a third area array detector;

the optical axes of a first laser beam emitted by the first laser and a second laser beam emitted by the second laser are mutually vertical and have different wavelengths; after the first laser beam and the second laser beam are combined through the first beam splitter prism, the optical axes of the first laser beam and the second laser beam are superposed; after beam combination, the first laser beam and the second laser beam irradiate the third beam splitter prism through the second beam splitter prism, and after the first laser beam and the second laser beam are reflected by the third beam splitter prism, the first laser beam and the second laser beam enter the microscope objective and are projected onto the surface of a sample;

the first laser beam and the second laser beam are reflected by the surface of a sample, collected by a microscope objective, reflected to a lens through a third beam splitter prism and a second beam splitter prism, converged through the lens and then reach a fourth beam splitter prism, and divided into a transmission part and a reflection part through the fourth beam splitter prism, wherein the two parts comprise the first laser beam and the second laser beam; the reflected part irradiates the first band-pass filter, the first laser beam is filtered, and the second laser beam is received by the first area array detector; the transmission part irradiates the second band-pass filter, the second laser beam is absorbed, and the first laser beam is received by the second area array detector; acquiring two light spot images with different areas on the first area array detector and the second area array detector;

the microscope is composed of a microscope objective, a field lens and an area array detector and is used for displaying images of the sample.

3. The method of claim 1, wherein the calculation of the spot area in the spot image comprises:

the image of the light spot is marked as I₁，I₁Is denoted as I₁(x,y)；

Setting a threshold value T, and₁performing the operation of formula 1 on each pixel in the image, extracting a light spot area in the image, and obtaining a processed image recorded as I'₁，I′₁Is recorded as I 'per pixel size'₁(x,y)；

To l'₁Summing pixel values in the image to obtain a light spot image I₁Medium spot area S₁。

4. The method as claimed in claim 1, wherein the distance between the microscope objective and the sample is adjusted, the corresponding two light spot areas are obtained by calculation, and the area lookup table is formed by the mapping relationship between different distances and light spot areas, and the specific process is as follows:

1) the microscope objective can move in the vertical direction relative to the sample, and the single movement distance is delta_d,Δ_dShould be the depth of field of the microscope;

2) assuming that the microscope objective is at a vertical distance d from the sample, the maximum and minimum values of the variable distance d are labeled d_maxAnd d_min；

3) Using delta_dDividing d to form N depth positions; wherein the nth depth position is marked as d⁽ⁿ⁾N ranges from 1 to N, d⁽ⁿ⁾Is shown in equation 2:

d⁽ⁿ⁾＝d_min+(n-1)Δ_d (2)

4) moving the microscope objective to each d⁽ⁿ⁾Recording the corresponding spot image I₁And I₂Area of medium spot, markAndand forming an area lookup table according to the mapping relation between the distance and the area of the light spot.

5. The method as claimed in claim 4, wherein the obtaining of the light spot image in the environment to be measured and the obtaining of the distance between the microscope objective lens and the sample corresponding to the light spot area according to the area lookup table comprises:

1) in actual measurement, for an environment to be measured with unknown distance, firstly, a light spot image I is acquired₁And I₂Are respectively marked as S_1DAnd S_2D；

2) Will S_1DAnd all in area look-up tableCompared with the area S in the look-up table_1DTwo values of closest magnitude;

a specific comparison method is to compare S_1DAndmake a difference, as shown in equation 3, all of which are in the history tableObtain NIs marked asWill be provided withThe superscripts of the smallest two values in the set are respectively marked as n₁And n₂；

3) Will S_2DAnd n₁And n₂Corresponding in positionMaking a difference as shown in formula 4 to obtain a difference valueAndgetAnddepth d corresponding to the superscript of the medium or small numerical value⁽ⁿ⁾The distance d between the microscope objective and the sample, which is the first estimate, is denoted d_step1；

4) Will S_2DAnd all in the lookup tableMake a difference, as shown in equation 5, go through allObtain NIs marked asWill be provided withThe superscripts of the smallest two values in are respectively marked as n'₁And n'₂；

5) Will S_1DAnd n'₁And n'₂Corresponding in positionMaking a difference as shown in formula 6 to obtainAndgetAnddepth d corresponding to the superscript of the medium or small numerical value⁽ⁿ⁾The distance between the microscope objective and the sample, denoted d, as a second estimate_step2；

6) If the distance between the microscope objective and the sample of the first estimation and the second estimation satisfies the following condition: | d_step1-d_step2|≤σ_TWherein σ is_TIs a preset distance error threshold; then the two are matched by cross check, which shows that the obtained distance is correct, and at this moment, d is arbitrarily selected_step1Or d_step2One value of (a) is taken as the distance between the present microscope objective and the sample, marked as d_c。

6. The method of claim 5, further comprising:

if the distance between the microscope objective and the sample does not satisfy | d_step1-d_step2|≤σ_TIf the condition is satisfied, the estimated distance is wrong; in this case, the microscope objective must be moved by 2 Δ in a direction away from the sample_dAnd repeating the above processes until the estimated distance meets the condition.

7. The method of claim 1, wherein the moving the position of the microscope objective relative to the sample based on the defocus distance comprises:

when defocus distance Δ_defocusWhen the angle is larger than 0, the microscope objective lens is moved to the direction close to the sample by delta_defocus|；

When defocus distance Δ_defocusWhen the value is less than 0, the microscope objective lens is moved to the direction far away from the sample by the delta_defocus|；

When defocus distance Δ_defocusWhen the current state is 0, it indicates that the current state is in focus.

8. A microscopic imaging defocusing state acquisition device based on a multi-image area relation is characterized by comprising:

the light spot acquisition module is used for projecting two laser beams to the surface of a sample through a microscope objective based on a microscope imaging system, receiving two light spot images with different areas formed by the two laser beams reflected by the surface of the sample, calculating the areas of the two light spots and obtaining the corresponding relation between the distance between the microscope objective and the sample and the areas of the two light spots in the microscope imaging system;

the lookup table generation module is used for calculating to obtain two corresponding light spot areas by adjusting the distance between the microscope objective and the sample, and an area lookup table is formed by the mapping relation between different distances and the light spot areas;

the distance estimation module is used for acquiring a light spot image in the environment to be measured and acquiring the distance between the microscope objective and the sample corresponding to the light spot area according to the area lookup table;

and the focusing adjustment module is used for calculating the focusing distance according to the distance and moving the position of the microscope objective relative to the sample based on the focusing distance so as to realize automatic focusing of the microscope imaging system.

Technical Field

The invention belongs to the field of semiconductor detection, and particularly relates to an automatic focusing method of a microscopic imaging system based on a multi-image area relation, and further relates to an automatic focusing device of the microscopic imaging system based on the multi-image area relation.

Background

The automatic focusing of a microscope (i.e. automatically acquiring a focused image) is widely applied in the fields of industrial detection, such as semiconductors, liquid crystal screens, and the like. Conventional focus judgment methods, such as image definition function, distance measurement, etc., have no way to be used in large scale in industrial production due to slow speed (definition function) and high cost (distance measurement). In the prior art, the problems of speed and cost are solved by adopting light beam active projection, forming an image on the surface of a measured object and using the information of the image as a judgment basis of focusing conditions. However, the single image is susceptible to external factors, such as background light, the surface of the measurement object, and the installation of the measurement object, which results in the reduction of the measurement accuracy.

Disclosure of Invention

The invention aims to overcome the defects in the prior art, provides an automatic focusing method of a microscopic imaging system based on a multi-image area relation, and solves the technical problems of high depth estimation error rate and low focusing efficiency caused by single information source in the traditional method.

In order to solve the technical problems, the technical scheme of the invention is as follows.

In a first aspect, the present invention provides an automatic focusing method for a microscopic imaging system based on a multi-image area relationship, which comprises the following steps:

based on a microscopic imaging system, projecting two laser beams to the surface of a sample through a microscopic objective lens, receiving two light spot images with different areas formed by the two laser beams reflected by the surface of the sample, calculating the areas of the two light spots, and obtaining the corresponding relation between the distance between the microscopic objective lens and the sample and the areas of the two light spots in the microscopic imaging system;

adjusting the distance between the microscope objective and the sample, calculating to obtain two corresponding light spot areas, and forming an area lookup table by the mapping relation between different distances and the light spot areas;

acquiring a light spot image in an environment to be detected, and acquiring the distance between the microscope objective and the sample corresponding to the area of the light spot according to an area lookup table;

and calculating the distance from the focal point according to the distance, and moving the position of the microscope objective relative to the sample based on the distance from the focal point, thereby realizing the automatic focusing of the microscope imaging system.

Optionally, the microscopic imaging system comprises: the device comprises a first laser, a second laser, a first beam splitter prism, a second beam splitter prism, a third beam splitter prism, a microscope objective, a lens, a fourth beam splitter prism, a first band-pass filter, a second band-pass filter, a first area array detector, a second area array detector, a field lens and a third area array detector;

the optical axes of a first laser beam emitted by the first laser and a second laser beam emitted by the second laser are mutually vertical and have different wavelengths; after the first laser beam and the second laser beam are combined through the first beam splitter prism, the optical axes of the first laser beam and the second laser beam are superposed; after beam combination, the first laser beam and the second laser beam irradiate the third beam splitter prism through the second beam splitter prism, and after the first laser beam and the second laser beam are reflected by the third beam splitter prism, the first laser beam and the second laser beam enter the microscope objective and are projected onto the surface of a sample;

the first laser beam and the second laser beam are reflected by the surface of a sample, collected by a microscope objective, reflected to a lens through a third beam splitter prism and a second beam splitter prism, converged through the lens and then reach a fourth beam splitter prism, and divided into a transmission part and a reflection part through the fourth beam splitter prism, wherein the two parts comprise the first laser beam and the second laser beam; the reflected part irradiates the first band-pass filter, the first laser beam is filtered, and the second laser beam is received by the first area array detector; the transmission part irradiates the second band-pass filter, the second laser beam is absorbed, and the first laser beam is received by the second area array detector; acquiring two light spot images with different areas on the first area array detector and the second area array detector;

the microscope is composed of a microscope objective, a field lens and an area array detector and is used for displaying images of the sample.

Optionally, the calculation process of the area of the light spot in the light spot image is as follows:

the image of the light spot is recorded asI₁，I₁Is denoted as I₁(x,y)；

Setting a threshold value T, and₁performing the operation of formula 1 on each pixel in the image, extracting a light spot area in the image, and obtaining a processed image recorded as I'₁，I′₁Is recorded as I 'per pixel size'₁(x,y)；

To l'₁Summing pixel values in the image to obtain I₁Medium spot area S₁。

Optionally, the distance between the microscope objective and the sample is adjusted, two corresponding light spot areas are obtained through calculation, an area lookup table is formed by mapping relationships between different distances and the light spot areas, and the specific process is as follows:

1) the microscope objective can move in the vertical direction relative to the sample, and the single movement distance is delta_d,Δ_dShould be the depth of field of the microscope;

2) assuming that the microscope objective is at a vertical distance d from the sample, the maximum and minimum values of the variable distance d are labeled d_maxAnd d_min；

3) Using delta_dDividing d to form N depth positions; wherein the nth depth position is marked as d⁽ⁿ⁾N ranges from 1 to N, d⁽ⁿ⁾Is shown in equation 2:

d⁽ⁿ⁾＝d_min+(n-1)Δ_d (2)

4) moving the microscope objective to each d⁽ⁿ⁾Recording the corresponding spot image I₁And I₂Area of medium spot, markAndfrom the mapping relation between the distance and the area of the light spotAn area lookup table is formed.

Optionally, the obtaining of the light spot image in the environment to be measured and the obtaining of the distance between the microscope objective and the sample corresponding to the light spot area according to the area lookup table include:

1) in actual measurement, for an environment to be measured with unknown distance, firstly, the areas of light spots are obtained and are respectively marked as S_1DAnd S_2D；

2) Will S_1DAnd all in area look-up tableCompared with the area S in the look-up table_1DTwo values of closest magnitude;

a specific comparison method is to compare S_1DAndmake a difference, as shown in equation 3, all of which are in the history tableObtain NIs marked asWill be provided withThe superscripts of the smallest two values in the set are respectively marked as n₁And n₂；

3) Will S_2DAnd n₁And n₂Corresponding in positionMaking a difference as shown in formula 4 to obtain a difference valueAndgetAnddepth d corresponding to the superscript of the medium or small numerical value⁽ⁿ⁾The distance d between the microscope objective and the sample, which is the first estimate, is denoted d_step1；

4) Will S_2DAnd all in the lookup tableMake a difference, as shown in equation 5, go through allObtain NIs marked asWill be provided withThe superscripts of the smallest two values in are respectively marked as n'₁And n'₂；

5) Will S_1DAnd n'₁And n'₂Corresponding in positionMaking a difference as shown in formula 6 to obtainAndgetAnddepth d corresponding to the superscript of the medium or small numerical value⁽ⁿ⁾The distance between the microscope objective and the sample, denoted d, as a second estimate_step2；

6) If the distance between the microscope objective and the sample of the first estimation and the second estimation satisfies the following condition: | d_step1-d_step2|≤σ_TWherein σ is_TIs a preset distance error threshold; then the two are matched by cross check, which shows that the obtained distance is correct, and at this moment, d is arbitrarily selected_step1Or d_step2One value of (a) is taken as the distance between the present microscope objective and the sample, marked as d_c。

Optionally, the method further includes:

if the distance between the microscope objective and the sample is not satisfied

|d_step1-d_step2|≤σ_TIf the condition is satisfied, the estimated distance is wrong; in this case, the microscope objective must be moved by 2 Δ in a direction away from the sample_dAnd repeating the above processes until the estimated distance meets the condition.

Optionally, the moving the microscope objective relative to the sample position based on the defocus distance includes:

when defocus distance Δ_defocusWhen the angle is larger than 0, the microscope objective lens is moved to the direction close to the sample by delta_defocus|；

When defocus distance Δ_defocusWhen the value is less than 0, the microscope objective lens is moved to the direction far away from the sample by the delta_defocus|；

When defocus distance Δ_defocusWhen the current state is 0, it indicates that the current state is in focus.

In a second aspect, the invention provides a multi-image area relationship-based microscopic imaging defocus state acquisition apparatus, including:

the light spot acquisition module is used for projecting two laser beams to the surface of a sample through a microscope objective based on a microscope imaging system, receiving two light spot images formed by the two laser beams reflected by the surface of the sample, calculating the areas of the two light spots and obtaining the corresponding relation between the distance between the microscope objective and the sample and the areas of the two light spots in the microscope imaging system;

the lookup table generating module is used for calculating to obtain two corresponding light spot areas by adjusting the distance between the microscope objective and the sample, and forming an area lookup table according to the mapping relation between different distances and the light spot areas;

the distance estimation module is used for acquiring a light spot image in the environment to be measured and acquiring the distance between the microscope objective and the sample corresponding to the light spot area according to the area lookup table;

and the focusing adjustment module is used for calculating the focusing distance according to the distance and moving the position of the microscope objective relative to the sample based on the focusing distance so as to realize automatic focusing of the microscope imaging system.

Compared with the prior art, the invention has the following beneficial effects: the invention adopts two light beams to project on the surface of the measured object, and because the projection information of the same measured surface is investigated, the influence is the same, the focusing condition is investigated by utilizing the mutual relation between the two, the influence of the factors can be avoided, and the detection precision is improved.

Drawings

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2 is a schematic diagram of the emission light path;

FIG. 3 is a schematic diagram of a receive optical path;

FIG. 4 is a schematic diagram of two light sources for target detection;

FIG. 5 is a schematic diagram of a light source for target detection.

Reference numerals: 1. the device comprises a first laser, a second laser, a third laser, a fourth laser, a fifth laser, a sixth laser, a fifth, a sixth laser, a fifth, a sixth laser, a sample 9, a lens, a sample 10, a lens, 11, a fourth beam fourth, a 12, a band-pass filter 13, a band-pass filter, a field lens, a 14, a plane array detector 15, a plane array detector, a 16, a field lens, a plane array detector.

Detailed Description

The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.

Example 1

The invention conception of the invention is as follows: two laser beams with different wavelengths are projected on the surface of a measuring object, the projection information of the same measuring surface is inspected, the focusing condition is inspected by utilizing the correlation between the two laser beams, the influence of light path transmission factors can be avoided, and the detection precision is improved.

The invention discloses an automatic focusing method of a microscopic imaging system based on a multi-image area relation, which is shown in figure 1 and comprises the following processes:

and step S1, two laser beams with different wavelengths are projected to the surface of the sample through the microscope objective, and two spot images with different areas formed by the laser beams reflected by the surface of the sample are received for later detection.

Structure of the microscopic imaging system referring to fig. 2, a laser beam is projected onto a surface of a sample to obtain projection information of the sample. The emission process of the laser beam is shown in fig. 2, and the process is as follows: the first laser 1 and the second laser 2 are arranged as shown in the figure, namely: the optical axes of the laser emitted by the two lasers are vertical to each other. A laser beam I3 (shown as a solid) emitted by the laser I1Line representation) having a wavelength λ₁A second laser beam 4 (shown by a dotted line) emitted by the second laser 2 and having a wavelength λ₂The optical axes of the first laser beam 3 and the second laser beam 4 are perpendicular to each other, and the wavelengths of the two beams are different. After the first laser beam 3 and the second laser beam 4 irradiate the first beam splitting prism 5, the first laser beam 3 is reflected by the first beam splitting prism 5, and the second laser beam 4 directly penetrates through the first beam splitting prism 5. After the beam is combined by the first beam splitter prism 5, the optical axes of the first laser beam 3 and the second laser beam 4 are combined. After beam combination, the first laser beam 3 and the second laser beam 4 pass through the second beam splitting prism 6 and irradiate onto the third beam splitting prism 7, the included angles between the third beam splitting prism 7 and the first laser beam 3 and between the second laser beam 4 and the main optical axis are both 45 degrees. After being reflected by the beam splitting prism three 7, the first laser beam 3 and the second laser beam 4 enter the microscope objective 8 and then are projected on the surface of the sample 9.

The procedure of receiving the reflected laser beam from the sample through the receiving optical path is as shown in fig. 3: the first laser beam 3 and the second laser beam 4 are reflected by the surface of the sample 9, collected by the microscope objective 8, reflected to the lens 10 through the third beam splitter prism 7 and the second beam splitter prism 6, and converged by the lens 10 to reach the fourth beam splitter prism 11. The first laser beam 3 and the second laser beam 4 are divided into a transmission part and a reflection part after passing through the fourth beam splitting prism 11, and the two parts respectively comprise the first laser beam 3 and the second laser beam 4. Since the two light beams use different wavelengths, the two lights can be separated by the filter. The reflected part irradiates on the first band-pass filter 12, and the first band-pass filter 12 is characterized in that: blocking wavelength of lambda₁Of light having a passing wavelength of λ₂Of the light source. Therefore, the first laser beam 3 is filtered out, and the second laser beam 4 is received by the first area array detector 14. The transmission part irradiates on the second band-pass filter 13, and the second band-pass filter 13 is characterized in that: blocking wavelength of lambda₂Of light having a passing wavelength of λ₁So that the second laser beam 4 is absorbed and the first laser beam 3 is received by the second area array detector 15. And two spot images with different areas are obtained on the first area array detector 14 and the second area array detector 15.

In addition, the microscope objective 8, the field lens 16, and the area array detector three 17 constitute an infinity corrected microscope, simply referred to as a microscope. The microscope shows an image of the sample.

And step S2, calculating to obtain two corresponding light spot areas by adjusting the distance between the microscope objective and the sample, and forming an area lookup table by the mapping relation between different distances and the light spot areas.

The present invention uses two light sources to detect the target sample 9, and the principle is shown in fig. 4. Fig. 4 illustrates the relationship between the spot size (area) on the observation surface and the light source when one laser light source projects the target sample to be detected, wherein the light emitted from the light source has a certain divergence angle, and the half divergence angle is denoted as θ. Where the source point may represent either laser one 1 or laser two 2. The light emitted from the light source point irradiates on the microscope objective 8, and is converged at the other side of the microscope objective 8 to form an image point.

To explain the principles of the present invention, the following model was established. The microscope objective 8 can be regarded here as an equivalent lens with a focal length f. This equivalent lens is not numbered to distinguish other lenses in the inventive arrangements. The distance from the light source point (the light source point can refer to a laser I1 or a laser II 2) to the object side main surface of the equivalent lens is u (u > f), and the distance from the image point to the image side main surface of the equivalent lens is v. On the equivalent lens, the perpendicular distance from the optical axis of the incident light beam farthest from the optical axis at the exit position is recorded as h. As shown in fig. 4, the sample 9 is placed on the other side of the equivalent lens, where the sample surface is an observation surface, the spot radius on the observation surface is denoted by r, and the distance from the observation surface to the image-side main surface in the optical axis direction is denoted by d. Through the object-image relationship and the similarity relationship, an equation set can be simultaneously established to obtain the relationship between r and u, theta, d and f, thereby illustrating the basic principle of the method.

The system of equations is:

it should be noted that all variables in the set of equations are positive. The above equations can be combinedObviously, when the positions of the observation plane and the microscope objective 8 are fixed, d and f are both fixed, then r is proportional to u and theta under the condition that the light emitted by the light source does not exceed the diameter of the lens, namely the size of the spot area formed by the reflected light beam on the surface of the sample is positively correlated with u and theta.

The method uses two light sources for projection, and the sizes of the obtained light spots are different by adjusting the vertical distance between the two light sources and the microscope objective 8 and the respective divergence angles of the two light sources. At a certain observation position (i.e. at the timing d in fig. 4), two light spots with different areas can be obtained by using the first area array detector 14 and the second area array detector 15, respectively, that is, one observation position corresponds to a unique pair of area combinations. Then by constructing an area look-up table, a correspondence of d to the two areas is established. When actually observing, the distance d between the sample 9 and the microscope objective 8 can be determined by comparing the areas of the two actually photographed spots only with the areas in the look-up table.

In addition, it should be noted that if a single light source is used for projection, two positions with the same spot area may appear by placing the observation surfaces on both sides of the image point of the single light source, as shown in fig. 5. In FIG. 5, if viewing surface 1, viewing surface 2 is at a distance a from the image plane₁，a₂Equal, then it is apparent that the radius r of the spot on the viewing surface 1 is equal₁Radius r of the light spot on the observation plane 2₂Similarly, the areas of the light spots on the two observation surfaces are equal, so that the position of the current observation surface cannot be determined accordingly. Thus introducing two light sources with different object distances for projection.

The above is the basic principle of the present invention, and the following first calculates the area of two light spots, and the specific process is as follows:

1) the image of the light spot detected by the first area array detector 14 is marked as I₁，I₁Is denoted as I₁(x, y). The light spot image detected by the second area array detector 15 is marked as I₂。I₂Is denoted as I₂(x,y)。

2) Setting a threshold T (which may be based on illumination intensity, phase)Mechanical property and the like) and I is set₁Performing the operation of formula 1 on each pixel in the image, extracting a light spot area in the image, and obtaining a processed image recorded as I'₁，I′₁Is recorded as I 'per pixel size'₁(x, y). In the same way for I₂Is processed to obtain I'₂。

This step extracts the image I₁，I₂I.e. the spot area, with a significant brightness. Then treated of formula (1) to give I'₁,I′₂Represents I₁，I₂Spot position in (2).

3) Are respectively to I'₁And l'₂Summing pixel values in the image to obtain I₁And I₂Medium spot area S₁And S₂。

And calculating the corresponding areas of the two light spots according to the distance between the current microscope objective and the sample to obtain the mapping relation between the distance and the areas of the light spots.

And then adjusting the position of the microscope objective 8 to obtain the mapping relation between different distances and the area of the light spot, and generating an area lookup table. The specific process is as follows:

1) the microscope objective 8 is connected with a motor, so that the microscope objective 8 can move in the vertical direction relative to the sample 9. Let the single movement distance of the motor be Delta_d,Δ_dShould be the depth of field of the microscope.

2) Let the microscope objective 8 be at a vertical distance d from the sample 9. The maximum and minimum values of the distance d are denoted by d_maxAnd d_min。

3) Using delta_dAnd d is divided to form N depth positions. Wherein the nth depth position is marked as d⁽ⁿ⁾N ranges from 1 to N, d⁽ⁿ⁾The expression of (c) is shown in equation 2.

d⁽ⁿ⁾＝d_min+(n-1)Δ_d (2)

4) Moving the microscope objective 8 to each d⁽ⁿ⁾Record corresponding I₁And I₂Area of medium spot, markAndan area lookup table is formed by mapping the distance and the spot area, as shown in table 1.

Table 1 area lookup table

When making the area look-up table, the microscope objective 8 is moved to the in-focus position, i.e. d ═ v.

And step S3, acquiring a light spot image in the microscopic imaging system in the environment to be measured, acquiring a distance corresponding to the area of the light spot according to the area lookup table, and calculating an actual defocusing distance according to the distance so as to move the microscopic objective to realize automatic focusing.

1) In actual measurement, the environment to be measured with unknown distance is obtained. Firstly, obtaining light spot images of a first area array detector 14 and a second area array detector 15, obtaining the light spot areas thereof through a formula 1, and respectively recording the areas as S_1DAnd S_2D；

2) Will S_1DAll of the area look-up tables (as shown in Table 1)Compared with the area S in the look-up table_1DThe two values with the closest magnitude.

A specific comparison method is to compare S_1DAndmake a difference (as shown in equation 3), all of them in the history tableObtain NIs marked asWill be provided withThe superscripts of the smallest two values in the set are respectively marked as n₁And n₂。

The minimum difference indicates the area of the shot light spot and the serial number n in the lookup table₁And n₂Corresponding to the closest spot area, then n₁And n₂One of the distances that corresponds is the distance between the microscope objective 8 and the sample 9. For screening, the following 3) steps were performed.

3) Will S_2DAnd n₁And n₂Corresponding in positionMaking a difference, wherein the formula is shown as formula 4, and obtaining a difference valueAndgetAnddepth d corresponding to the superscript of the medium or small numerical value⁽ⁿ⁾The distance d between the microscope objective 8 and the sample 9, which is the first estimate, is denoted d_step1。

For example, ifThenThe distance between microscope objective 8 and sample 9, denoted d, is estimated for the first time_step1。

The above 2) -3) is carried out by using S_1DAnd all ofComparison, reuse of S_2DAnd (5) screening. To ensure the correctness of the acquisition depth, S is carried out in the steps 4) -5)_2DAnd all ofMaking a comparison, and reusing S_1DAnd (5) screening. Finally, the correctness of the estimated depth is judged by comparing the depths respectively obtained from 2) -3) and 4) -5).

4) Will S_2DAnd all in the lookup tableMaking a difference, the formula is shown as formula 5, and traversing all theObtain NIs marked asWill be provided withThe superscripts of the smallest two values in are respectively marked as n'₁And n'₂。

5) Will S_1DAnd n'₁And n'₂Corresponding in positionMaking a difference, the formula is shown as formula 6, and obtainingAndgetAnddepth d corresponding to the superscript of the medium or small numerical value⁽ⁿ⁾The distance between the microscope objective 8 and the sample 9, which is a second estimate, is denoted d_step2。

For example, ifThenThe distance between the microscope objective 8 and the sample 9 for the second evaluation is d_step2。

6) If the distance between the microscope objective 8 and the sample 9 for the first and second estimations satisfies the following condition: | d_step1-d_step2|≤σ_TIf the two are matched, the distance is correct, and d is selected_step1Or d_step2One value of (a) is taken as the distance between the present microscope objective 8 and the sample 9, marked d_c. Wherein σ_TIs a preset distance error threshold.

If the above condition is not satisfied, it indicates that the estimated distance is erroneous. In this case, the microscope objective 8 must be moved by 2 Δ in a direction away from the sample 9_dRepeating the steps 1-5, and judging until the estimated distance satisfies | d_step1-d_step2|≤σ_TThis condition is set.

In this step, move by 2 Δ_dThe reason for this is to adjust the position of the microscope objective 8 so that the images captured by the first area array detector 14 and the second area array detector 15 after the movement are significantly different from the current position image. Finally, the distance between the microscope objective 8 and the sample 9 is marked as d_c。

7) At this time, the defocus distance is: delta_defocus＝d_c-v。

8) And adjusting the position of the microscope objective relative to the sample according to the defocusing distance to realize automatic focusing of the microscope objective.

When delta_defocusWhen the angle is larger than 0, the motor is controlled to move the microscope objective 8 to the direction close to the sample 9 by delta_defocusL, |; when delta_defocusWhen the absolute value is less than 0, the motor is controlled to move the micro objective 8 to the direction far away from the sample 9 by delta_defocusL, |; when delta_defocusWhen the current state is 0, it indicates that the current state is in focus.

After the microscopic imaging system is automatically in focus, a clear image of sample 9 can be observed with the microscope.

The invention

Example 2

Based on the same inventive concept as the method of the embodiment 1, the device for acquiring the defocused state of the microscopic imaging based on the multi-image area relationship comprises the following steps:

the light spot acquisition module is used for projecting two laser beams to the surface of a sample through a microscope objective based on a microscope imaging system, receiving two light spot images formed by the two laser beams reflected by the surface of the sample, calculating the areas of the two light spots and obtaining the corresponding relation between the distance between the microscope objective and the sample and the areas of the two light spots in the microscope imaging system;

the lookup table generation module is used for adjusting the distance between the microscope objective and the sample, calculating to obtain the corresponding two light spot areas, and forming an area lookup table by the mapping relation between different distances and the light spot areas;

the distance estimation module is used for acquiring a light spot image in the environment to be measured and acquiring the distance between the microscope objective and the sample corresponding to the light spot area according to the area lookup table;

and the focusing adjustment module is used for calculating the focusing distance according to the distance and moving the position of the microscope objective relative to the sample based on the focusing distance so as to realize automatic focusing of the microscope imaging system.

The concrete implementation schemes of each module in the device of the invention refer to the procedures of each step in the method of the embodiment 1.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.