阅读说明：本技术 一种成像椭圆偏振仪的焦平面自动聚焦方法 (Focal plane automatic focusing method of imaging ellipsometer ) 是由 郭海燕 吴锜 于 2021-10-21 设计创作，主要内容包括：本发明属于光学设备聚焦技术领域,属于成像椭圆偏振仪技术领域的光学设备自动聚焦相关技术,涉及一种通过分析焦平面附近图像边缘探测进行成像椭圆偏振仪的焦平面自动聚焦的方法,具体步骤包括：图像采集、边缘检测、确定精准焦平面位置、开发板选取、聚焦应用；与现有技术相比,可以更加快速有效地确定成像椭圆偏振仪的焦平面,对半透明、微小生物样品测量节省了聚焦时间；不需要选择阈值,非常容易使用；测量的偏振光与物质光学相互作用得到椭偏图像数据焦平面变化比同时CCD取图同样处理方法得到的数据灵敏度更高；其成像椭偏图焦平面确定的灵敏度高,整体设计原理科学可靠,实现工艺步骤简单,自动聚焦效果好,应用价值高,应用环境友好。(The invention belongs to the technical field of focusing of optical equipment, belongs to the related technology of automatic focusing of optical equipment in the technical field of imaging ellipsometers, and relates to a method for automatically focusing a focal plane of an imaging ellipsometer by analyzing edge detection of an image near the focal plane, which comprises the following specific steps: image acquisition, edge detection, accurate focal plane position determination, development board selection and focusing application; compared with the prior art, the focal plane of the imaging ellipsometer can be determined more quickly and effectively, and the focusing time is saved for the measurement of semitransparent and tiny biological samples; the threshold does not need to be selected, and the method is very easy to use; the measured polarized light and the material are optically interacted to obtain the focal plane change of the ellipsometric image data, and the data sensitivity is higher than that obtained by the same processing method of CCD image taking; the imaging ellipsometry focal plane has high determined sensitivity, scientific and reliable overall design principle, simple implementation process steps, good automatic focusing effect, high application value and environment-friendly application.)

1. A focal plane automatic focusing method of an imaging ellipsometer is characterized by comprising the following steps: the method comprises the following specific steps:

(1) image acquisition: changing the focusing depth of an objective lens of the imaging ellipsometer by moving a sample carrier, and acquiring an ellipsometric image point by point on the full stroke of the focusing depth of the objective lens;

(2) edge detection: carrying out edge detection on each elliptic polarization image Ip or Is of the focusing depth acquired in the step (1);

(3) determining the position of the accurate focal plane: selecting the clearest position of the transverse edge and the longitudinal edge obtained by detection, and determining the clearest position as the accurate focal plane position of the measurement sample, thereby realizing the focal plane automatic focusing of the imaging ellipsometer;

(4) selecting a development board: the development board is an Arduino, STM32 or Field Programmable Gate Array (FPGA) type development board; the development board controls the sample carrier to enable the focusing depth of the objective lens to travel through the whole stroke including the focal plane, and the full stroke takes the Ip or Is image of the multi-point imaging ellipsometer.

(5) Focusing application: the sample carrying platform is moved by a motor controlled by a development board, so that the objective lens is focused to a focal plane position, and imaging ellipsometry is further performed to realize application.

2. The method of claim 1, wherein the method comprises: the specific implementation process of the control of the auxiliary focusing of the imaging ellipsometer comprises the following steps:

(1) reading in all focusing depth image data obtained by lifting or lowering a sample carrying platform from an ellipsometer;

(2) then obtaining the position of a precise focal plane by using an automatic focusing method; transmitting the obtained accurate focal plane position to development board control software for controlling the focusing depth of the objective lens;

(3) the development board controls the moving of the focusing depth of the objective lens to the corresponding focal plane position, and the automatic focusing of the whole imaging ellipsometer is realized.

3. The method of claim 1, wherein the method comprises: an apparatus for performing autofocus includes a memory and a processor, the memory storing a computer program that, when executed by the processor, performs the steps of: firstly, a processor controls a driving motor to move a focus plane of an objective lens near the position of a focus plane determined by CCD coarse focusing through moving a sample carrier, and an Ip or Is image of the imaging ellipsometer Is acquired at multiple points in the full stroke; observing a clear edge detection diagram corresponding to an expected sample to be measured through the change along with the focusing depth, namely obtaining the accurate focal plane position of the sample measurement; wherein, the development board be Arduino or STM32 or can process image data fast integratedly can also control the FPGA development board of drive hardware simultaneously.

The technical field is as follows:

the invention belongs to the technical field of focusing of optical equipment, belongs to the related technology of automatic focusing of optical equipment in the technical field of imaging ellipsometers, and relates to a method for automatically focusing a focal plane of an imaging ellipsometer by analyzing edge detection of an image near the focal plane, and the method is applied to the related technical field.

Background art:

the imaging ellipsometer (ellipsometry) technology is a technology for realizing imaging by elliptical polarized light by adding a CCD camera on the basis of a traditional ellipsometer, and is a high-precision measurement method combining high resolution in the thickness direction of a common ellipsometer and high resolution in a sample plane. The imaging ellipsometer technology can provide real-time dynamic video and images of a measured sample besides providing related information such as thickness and refractive index of a thin film like a general ellipsometer, and is attracting more and more interests of other subjects such as biomedical researchers, for example, the researchers find that the imaging ellipsometer technology can be used in combination with Surface Plasmon Resonance (SPR), and can realize detection of biochips and biosensors. These interdisciplinary fields of research have created new research hotspots for ellipsometry techniques as well as challenges for such techniques, such as thin film measurements and microscopic imaging on unstable liquid surfaces.

On one hand, the imaging ellipsometer can be used for measuring the bacterial biological cells and other biological samples in submicron scale without damage and dyeing marks; on the other hand, in biological applications, some living cells are not stable enough in the environment and may sometimes die quickly, and it is more common that the characteristics of living cells change during observation. Autofocus techniques become particularly important in time-limited situations, since rapid and accurate positioning to a location of interest can be ensured. However, in practical applications, many of the autofocus methods available for general topography imaging (e.g., general cameras and holograms) are no longer suitable for autofocusing imaging ellipsometers due to the more complex polarized light-material interactions. In addition, the working mode of the imaging ellipsometer is different from that of a common optical imaging system, the plane of the objective lens of the imaging ellipsometer has an included angle with the imaging lens, and the CCD is parallel to the plane of the objective lens, so that the imaging principle of the imaging ellipsometer is more complex, and the difficulty of focusing and positioning when a tiny and semitransparent biological sample is measured is increased.

The auto-focusing method of the conventional camera is classified into an active type focusing method and a passive type focusing method (Chen CY, Hwang RC, Chen YJ. A passive auto-focus camera control system. appl Soft Compat. 2010; 10(1): 296-. The active type focusing method uses an infrared or ultrasonic signal to transmit to a target, and measures an object distance according to an arrival time of a reflected signal to perform focusing. Passive focusing methods are implemented by image analysis, for example: determining the focal plane by analyzing the Image spectra of the hologram (Liebling M, Unser M. auto focus for digital Fresnel halogen by use of J Opt SocAmA-Opt Image Sci Vis. 2004; 21(12): 2424-30); gray scale gradient analysis of the bubbles and particles within the CCD image to determine whether they are in the focal plane (Broder D, Sommerfeld M. planar image gradient for the analysis of the hydrophilic in bubbly flows. Meas Sci technol.2007; 18(8): 2513-28); performing high-frequency component or edge analysis and gradient edge detection on a scanning electron microscope picture to determine the position of a focal plane (Groen FCA, Young IT, Ligthhart G.A complexity of differential focus functions for use in auto focus algorithms. cytometry. 1985; 6(2): 81-91.); or discrete wavelet transform to determine whether the image is sharp (Chen CY, Hwang RC, Chen YJ. Apassive auto-focus camera control system. appl Soft Comp. 2010; 10(1): 296-.

The traditional method for detecting the focal plane is generally based on the difference of image definition at different focusing depths and the sharpness of the edge of the image, such as Zhang nationality peak; shixu; depression; aging; sea, vibration and Kun; bear seed; old Ruiyun; an adult of Qin; xiaolian ball; the invention relates to an automatic focusing method of a laser confocal microscope (P) invented by Jiakatang in 2019, with the patent number of CN201911009133.4, which uses CCD to collect the reflected light on the surface (the surface for reflecting laser) of a sample slide in real time, converts the collected image into a gray value two-dimensional matrix, and obtains the position of a focal plane by measuring the maximum position of the gray value of the picture; john B pertmann; march.c.pettman; wadeim persky; system, apparatus and method for automatic microscope focusing by nanoelectronic imaging limited, invented in 2019, danis Y-saru hough [ P ], patent No.: CN201980029638.9 determines the focus with sharpness values determined from images taken by offset focus cameras by using multiple focus cameras positioned on different sides of the image conjugate plane.

Ellipsometers are not based on simple optical principles and obtain information about a sample by measuring the amplitude and phase of polarized light reflected from the sample to reveal more subtle features, in contrast toThe thickness variation is required to be widely applied in the fields of semiconductor industry and the like. Imaging ellipsometers based on ellipsometers differ from conventional cameras, microscopes, holograms, and ellipsometric images may also have completely different characteristics depending on the depth of focus, for example, Cancer cells (Chen Y-D, Hsu HY, Khaleel MI, Chang Y-C, Wu C-H, Wu H-C.study of Biological Reaction in Cancer Cell with Spectroscopic Imaging Ellippage. in: Verma P, Egner A, edition, Nanoimaging and microbiological Imaging Iv. Processings of SPma.99252016), and Streptococcus mutans cells (Khaleel MI, Chen Y-D, Chien C-H, Chang Y-C.Micropic Imaging of microscope-sample of sample-Cell of Journal of culture of John 27116: 27116.

Other cameras that collect topographical images (including holographic photographs) are relatively easy to autofocus, for example, when using an autofocus evaluation function method, a set of full-stroke depth-of-focus images that include the focal plane are collected once and analyzed to obtain a reliable focal plane position. This work of determining the hologram focal plane is for example the following: langhanenberg P, KemperB, Dirksen D, von Bally G.automatic in digital pharmacological phase control microprocessor on pure phase objects for live cell imaging. apple optics.2008; 47(19) D176. While for imaging ellipsometers the focal plane position is sought with the autofocus evaluation function (Chen S, Meng Y, Jin G. study on the autofocus evaluation function in the imaging system. acta optical Sinica.2005; 25(7):923-9), they found that the Laplace function and the Sobel evaluation function are more suitable for silicon wafer fibrin chips and multichannel protein chips, but a multiframe averaging method of more than 9 frames is required for suppressing noise interference. This indicates on the one hand that imaging ellipsometry is not the same mechanism as general imaging, and on the other hand it can be seen that in case of need for fast measurement, multiple passes through the full depth of focus including the focal plane are measured, and then averaging such a method of fixation is not fast enough. Image intensity gradient sharpness function method (Meng YH, Chen S, Jin G. an auto-focusing method for imaging ellipsometry system. in: Arwin H, Belk U, Schubert M, editors. Physics Status C-Current Topics in Solid State Physics, Vol 5, No. 5 Physics Status Solidi C-Current Topics M Solid State Physics.5. Weinheim: Wiley-V C H Verlag Gmbh; 2008.p.1046-9) was also tried for imaging ellipsometer autofocusing. The method determines the focal plane by continuously moving the imaging sensor along the direction of the optical axis to search the sharpness of the maximum brightness gradient, but the method completely depends on the brightness gradient to determine the focal plane involves less information of a sample on one hand and is also easily influenced by the measuring environment.

In conclusion, due to the interaction between polarized light and a substance, the imaging ellipsometer can reveal more details and has a good application prospect, but no good technology exists for the situations of biological samples such as tiny, semitransparent and living cells needing to quickly and accurately determine a focal plane.

Another relevant physical background knowledge is: the ellipsometry measurement parameter has a ratio (r) to the amplitude of the p-wave and s-wave representing the reflection of the sample_p/r_s) The relevant parameters are:

wherein I_p＝r_p ²And I_s＝r_s ²Reflecting the p-wave (r) for the sample_p) And s wave (r)_s) The strength of (2). The amplitude ratio (Ψ) and phase difference (Δ) of the p-and s-waves reflected back from the sample is related to the complex Fresnel reflection coefficient (χ) of the sample (Yoshizawa T. principles and applications. handbook of optical metrology. xiii. boca Raton: CRC Press; 2009. p.730; Bubert H, Jenett H. surface and thin film analysis: principles, analysis, applications: applications. Weinheim: Wiley-VCH Verlag GmbH; 2002):

in conclusion, the imaging ellipsometer is expected to be a very useful device for researching tiny and semitransparent biological samples, but the automatic focusing method of the imaging ellipsometer still has a plurality of spaces needing to be improved, and the research on the focal plane automatic focusing method and process of the imaging ellipsometer has wide theoretical significance and application value.

The invention content is as follows:

the invention aims to overcome the defects in the prior art, and provides an automatic focusing method for a focal plane of an imaging ellipsometer, so as to be beneficial to the application of the method in the related technical field.

In order to achieve the above object, the method for automatically focusing the focal plane of the imaging ellipsometer according to the present invention comprises:

(1) image acquisition: changing the focusing depth of an objective lens of the imaging ellipsometer by moving a sample carrier, and acquiring an ellipsometric image point by point on the full stroke of the focusing depth of the objective lens;

(2) edge detection: ellipsometry images of all the focusing depths acquired in the step (1)I_pOr I_sCarrying out edge detection;

(3) determining the position of the accurate focal plane: selecting the clearest position of the transverse edge and the longitudinal edge obtained by detection, and determining the clearest position as the accurate focal plane position of the measurement sample, thereby realizing the focal plane automatic focusing of the imaging ellipsometer;

(4) the development board is an Arduino, STM32 or Field Programmable Gate Array (FPGA) type development board; the development board controls the sample carrier to make the focusing depth of the objective lens go through the whole travel of the focal plane, and the whole travel takes the I of the multi-point imaging ellipsometer_pOr I_sAnd (4) an image.

(5) Focusing application: the sample carrying platform is moved by a motor controlled by a development board, so that the objective lens is focused to a focal plane position, and imaging ellipsometry is further performed to realize application.

The invention provides a control method of auxiliary focusing of an imaging ellipsometer, which comprises the following concrete implementation processes:

(1) reading in all focusing depth image data obtained by lifting or lowering a sample carrying platform from an ellipsometer;

(2) then obtaining the position of a precise focal plane by using an automatic focusing method;

(3) and the obtained accurate focal plane position is transmitted to development board control software for controlling the focusing depth of the objective lens, and the development board controls the focusing depth of the objective lens to be moved to the corresponding focal plane position, so that the automatic focusing of the whole imaging ellipsometer is realized.

The device for realizing automatic focusing comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the following steps: firstly, a processor controls a driving motor to move a sample carrier to enable an objective lens focusing plane to do full-stroke movement near a position of a CCD coarse focusing determined focal plane, and the full-stroke multipoint acquisition imaging ellipsometer I_pOr I_sAn image; observing a clear edge detection diagram corresponding to an expected sample to be measured through the change along with the focusing depth, namely obtaining the accurate focal plane position of the sample measurement; wherein the development board is Arduino or STM32 or can integrally and rapidly process the graphLike data, the FPGA development board can also control the driving hardware.

Compared with the prior art, the invention has the advantages and positive effects that: the focal plane of the imaging ellipsometer can be determined more quickly and effectively, and the focusing time is saved for measuring semitransparent and tiny biological samples; different from evaluation function methods such as Laplace operators and the like, the method does not need to select a threshold value and is very easy to use; the method for determining the imaging ellipsometry focal plane is simple and convenient, the overall design principle is scientific and reliable, the implementation process steps are simple, the automatic focusing effect is good, the application value is high, and the application environment is friendly.

Description of the drawings:

FIG. 1 is a block diagram illustrating the flow of auto-focusing of the focal plane of an imaging ellipsometer according to the present invention.

Fig. 2 is a schematic flow chart of the application of the focal plane auto-focusing of the imaging ellipsometer according to the present invention.

Fig. 3 is a block diagram of the general structure control principle of the device according to the present invention.

FIG. 4 shows a 19-objective depth-of-focus ellipsometer I for imaging 4 Streptococcus mutans on gold film according to the invention_pThe first 15 of the images; the distance of sequentially moving the objective lens for the 15 focusing positions is as follows: 0, +200nm, +1.8 μm, +800nm, +200nm, +600nm, +200nm, +2 μm, +800nm, +200nm, +600nm, +200nm, +2.6 μm, +200nm, +200 nm. The set of measurement focal planes is at position 8, and only the first 15I are selected_pAnd displaying the implementation result in an image.

FIG. 5(a) shows the initial positions of 4 Streptococcus mutans on the gold film, corresponding to the first depth of focus (focus position 1) I in FIG. 4_pAnd detecting horizontal and longitudinal edges of the image to obtain a result map.

FIG. 5(b, c, d) shows the selected focal plane positions (5(c)) and the focal depths (corresponding to 7, 8, 9 in FIG. 4) I of 4 Streptococcus mutans on the gold membrane_pAnd detecting horizontal and longitudinal edges of the image to obtain a result map.

FIG. 5(e) is a graph of the results of horizontal and longitudinal edge detection at the 15 th focal depth (corresponding to the last graph in FIG. 4) for 4 S.mutans on the gold film; a dynamic demonstration video can be provided when the detailed 19-position pointwise edge detection of 4 streptococcus mutans on the gold film is needed.

FIG. 6(a) is a graph showing I of the initial depth of focus (focal position 1) of 2 Streptococcus mutans on a gold film according to the invention_pThe horizontal and vertical edge detection results are shown.

FIG. 6(b, c, d, e) shows the 2 S.mutans selection focal points on the gold membrane (6(c), focal depth position 11) and two focal depths before and after the focal plane (focal positions 10, 11, 12, 13) I_pHorizontal and vertical edge detection results of the graph.

FIG. 6(f) is the I of the last depth of focus (focus position 29) of 2 S.mutans on the gold film_pAnd detecting horizontal and longitudinal edges of the image to obtain a result map.

The specific implementation mode is as follows:

the invention is described in detail below with reference to the figures and examples.

Example 1:

the method for automatically focusing the focal plane of the imaging ellipsometer provided by the embodiment comprises the following steps of:

s101: roughly focusing by the CCD to preliminarily position the approximate position of a focal plane;

s102: adjusting an objective lens carrier of the imaging ellipsometer to enable the focusing depth of the objective lens carrier to travel through a full stroke including a CCD coarse focusing position;

s103: ellipsometric images (I) collected for each focus position for the full stroke_pOr I_s) Transverse and longitudinal edge detection is performed.

Example 2:

the focal plane auto-focusing method of imaging ellipsometer provided in this example uses 4 ellipsometric images of Streptococcus mutans on gold film, and detailed conditions for sample and imaging ellipsometer image collection are described in (Khaleel MI, ChenY-D, Chien C-H, ChangY-C.Micropic imaging ellipsometry of sub-micron-scale bacterial cells, radiological Journal of Pharmaceutical research, 2017; 16(11): 2713-25); the focusing depth adjustment of the objective lens is realized by moving the position of the sample stage, 19 focusing position images are collected in the full stroke, and the specific steps of accurately focusing by using the images and using an imaging ellipsometer image automatic focusing method comprise:

the method comprises the following steps that firstly, the approximate position of a focal plane is determined by CCD coarse focusing, and the focusing depth of an objective lens is moved to the position above or below the focal plane by controlling a driving motor through Arduino, STM32 or other development boards;

in the second step, the depth of focus of the objective lens is controlled to move through the whole travel of the focal plane through Arduino, STM32 or other development boards, and the I of the full-travel multipoint imaging ellipsometer_pOr I_sImages, see the first 15 ellipsoids I shown in FIG. 4_pA drawing;

thirdly, detecting the transverse and longitudinal edges of the ellipsometric image obtained by the imaging ellipsometer objective lens focusing depth full stroke, wherein the detection result is shown in reference to fig. 5(a, b, c, d, e) or providing video demonstration; it can be seen that the lateral and longitudinal edges of fig. 5(c) (fine focal plane, position 8) are most clearly defined (lateral detection of each cell clearly shows the upper and lower edges, and longitudinal detection clearly shows the left and right edges), while 5(b) and 5(d) are blurred to a different extent than the lateral and/or longitudinal edges of fig. 5 (c); the further away (whether up or down) from the focal plane (position 8) its lateral and longitudinal edges are less pronounced; FIG. 5(d) is the 15 th depth of focus position (corresponding to the last image in FIG. 4); detailed 19 position pointwise edge detection, additional video presentations are visible as needed. Thus determining the eighth position as the precise focal plane position; (in addition, the executive program of the dynamic video process can be integrated into the electromechanical control of the ellipsometer to realize the fast and accurate focusing and positioning)

And fourthly, controlling the moving sample carrying platform through the development board to enable the objective lens to be focused to the focal plane position, realizing the automatic focusing of the imaging ellipsometer quickly and accurately, and further developing related applications by utilizing imaging ellipsometry on the basis.

Example 3:

the method for automatically focusing the focal plane of the imaging ellipsometer provided in this example uses the ellipsometric images of 2 streptococcus mutans on the gold film, and the detailed conditions for collecting the sample and the imaging ellipsometer images are shown in (khalel MI, ChenY-D, Chien C-H, ChangY-C(iii) a scientific imaging instrumentation of sub-scale bacterial cells, a Pharmaceutical Journal of Pharmaceutical research, 2017; 16(11) 2713-25); the focusing depth of the objective lens is adjusted by moving the position of the sample stage, and the total stroke is used for collecting I of 29 focusing depth positions_pThe ellipsometry images are utilized, and the specific steps of realizing accurate focusing by using an imaging ellipsometer image automatic focusing method comprise:

the method comprises the following steps that firstly, the approximate position of a focal plane is determined by CCD coarse focusing, and the focusing depth of an objective lens is moved to the position above or below the focal plane by controlling a driving motor through Arduino, STM32 or other development boards;

in the second step, the depth of focus of the objective lens is controlled by Arduino, STM32 or other development board to move through the entire stroke including the focal plane, the I of the full-stroke multi-point imaging ellipsometer_pOr I_sAn image;

thirdly, carrying out transverse and longitudinal edge detection on the ellipsometric image of the type obtained by the imaging ellipsometer objective lens focusing depth full stroke; the detection results are shown in FIG. 6(a, b, c, d, e, f); it can be seen that the I of FIG. 6(c) (precision focal plane, the set measuring the 11 th depth of focus position)_pThe lateral and longitudinal edges of the plot are most clearly defined (lateral detection of each cell clearly shows the top and bottom edges, and longitudinal detection clearly shows the left and right edges), while 6(b) and 6(d, e) are blurred to a different extent than the lateral and/or longitudinal edges of FIG. 6 (c); the further away (whether up or down) the focal plane (positions 10,12,13,29) the transverse and longitudinal edges are less pronounced.

And fourthly, controlling the moving sample carrying platform through the development board to enable the objective lens to be focused to the focal plane position, so that the rapid and accurate automatic focusing of the imaging ellipsometer is realized, and on the basis, the imaging ellipsometer can be further utilized to carry out related research and application.

The imaging ellipsometer related to the embodiment is provided with a CCD camera, an objective lens and the like, and auxiliary equipment comprises a development board and a motor; the upper computer controls the development board to give a control instruction through software, the control instruction comprises a corresponding picture reading instruction and is connected with the CCD camera through a wire, the development board is connected with the motor through a wire, and the motor is connected with the sample carrying platform to achieve the purpose of controlling the focusing depth of the objective lens; the method related to the embodiment can also be realized by adopting equipment for realizing integration of Field Programmable Gate Array (FPGA) image processing and hardware control.