阅读说明：本技术 用于光热反射显微热成像的自动对焦方法及控制装置 (Automatic focusing method and control device for photothermal reflection microscopic thermal imaging ) 是由 刘岩 乔玉娥 邹学峰 丁立强 吴爱华 赵英伟 李锁印 于 2021-07-14 设计创作，主要内容包括：本发明提供一种用于光热反射显微热成像的自动对焦方法及控制装置。该方法包括：获取照明光强在预设最大离焦范围内单调变化的光热反射显微热成像装置采集的被测件位于待对焦位置时的采集图像,及被测件位于对焦位置时的参考图像；根据采集图像计算得到第一总强度值,并根据参考图像计算得到第二总强度值；比较第一总强度值和第二总强度值的大小,基于比较结果和照明光强在预设最大离焦范围内单调变化的趋势,确定被测件的离焦方向；获取被测件的离焦深度；离焦方向和离焦深度用于对被测件进行对焦。本发明能够在提高对焦效率的同时,避免人工对焦一致性不够的问题,保证多帧采集图像的对焦稳定性,提高基于光热反射进行测温时测量结果的准确性。(The invention provides an automatic focusing method and a control device for photothermal reflection microscopic thermal imaging. The method comprises the following steps: acquiring a collected image of a measured piece when the measured piece is positioned at a position to be focused and a reference image of the measured piece when the measured piece is positioned at the focusing position, wherein the collected image is collected by a photothermal reflection microscopic thermal imaging device with the illumination intensity monotonously changing within a preset maximum defocusing range; calculating to obtain a first total intensity value according to the collected image, and calculating to obtain a second total intensity value according to the reference image; comparing the first total intensity value with the second total intensity value, and determining the defocusing direction of the measured piece based on the comparison result and the monotonous change trend of the illumination light intensity in the preset maximum defocusing range; acquiring the defocus depth of a measured piece; the out-of-focus direction and the out-of-focus depth are used for focusing the measured piece. The method can improve the focusing efficiency, avoid the problem of insufficient manual focusing consistency, ensure the focusing stability of multi-frame collected images, and improve the accuracy of the measurement result during temperature measurement based on photothermal reflection.)

1. An auto-focusing method for photothermal reflection microscopy thermography, comprising:

acquiring an acquired image of a measured piece which is acquired by a photothermal reflection microscopic thermal imaging device and is positioned at a position to be focused; the illumination intensity of the photothermal reflection microscopic thermal imaging device monotonously changes within a preset maximum defocusing range;

calculating to obtain a first total intensity value of the collected image according to the collected image, and calculating to obtain a second total intensity value of the reference image according to the reference image; the reference image is an image of the measured piece which is acquired by the photothermal reflection micro thermal imaging device and is located at a focusing position;

comparing the first total intensity value with the second total intensity value, and determining the defocusing direction of the measured piece based on the comparison result and the monotonous change trend of the illumination intensity in a preset maximum defocusing range;

acquiring the defocus depth of the measured piece; wherein, the out-of-focus direction and the out-of-focus depth are used for focusing the measured piece.

2. The auto-focusing method for photothermal reflection microscopy thermal imaging according to claim 1, wherein the determining the defocus direction of the measured object based on the comparison result and the tendency of monotonic variation of the illumination intensity in a preset maximum defocus range comprises:

when the trend that the illumination light intensity monotonically changes in the preset maximum defocusing range is that the illumination light intensity monotonically increases along with the increase of the object distance in the preset maximum defocusing range, if the first total intensity value is greater than the second total intensity value, determining the defocusing direction of the measured piece as the direction of the increase of the object distance; if the first total intensity value is smaller than the second total intensity value, determining the defocusing direction of the measured piece as the direction of reducing the object distance;

when the monotonous change trend of the illumination light intensity in the preset maximum defocusing range is that the illumination light intensity monotonously decreases along with the increase of the object distance in the preset maximum defocusing range, if the first total intensity value is larger than the second total intensity value, determining the defocusing direction of the measured piece as the direction of the decrease of the object distance; and if the first total intensity value is smaller than the second total intensity value, determining that the defocusing direction of the measured piece is the direction of increasing the object distance.

3. The auto-focusing method for photothermal reflection microscopy thermography according to claim 1 or 2, wherein said calculating a first total intensity value of said captured image from said captured image and a second total intensity value of said reference image from said reference image comprises:

according toCalculating to obtain a first total intensity value of the collected image according toCalculating to obtain a second total intensity value of the reference image;

wherein, I_cIs the first total intensity value, c (x, y) is the gray value of the (x, y) pixel point in the collected image, I_rAnd r (x, y) is the gray value of the (x, y) pixel point in the reference image, and is the second total intensity value.

4. The auto-focusing method for photothermal reflection microscopy thermography according to claim 1 or 2, wherein acquiring the defocus depth of the measured piece comprises:

calculating to obtain a first Fourier transform of the collected image according to the collected image, and calculating to obtain a second Fourier transform of the reference image according to the reference image;

determining a fitted diameter of a point spread function of an optical subsystem in the photothermal reflection micro thermography arrangement from the first Fourier transform and the second Fourier transform;

and calculating the defocusing depth of the measured piece according to the fitting diameter and the imaging parameters of an optical subsystem in the photothermal reflection micro thermal imaging device.

5. The auto-focusing method for photothermal reflection microscopy thermography according to claim 4, wherein said determining a fitted diameter of a point spread function of an optical subsystem in said photothermal reflection microscopy thermography device from said first Fourier transform and said second Fourier transform comprises:

according toOrCalculating to obtain a point spread function of an optical subsystem in the photothermal reflection micro thermal imaging device;

determining a fitting diameter of the point spread function according to the point spread function;

wherein p (x, y) is the point spread function of the optical subsystem in the photothermal reflectance micro thermography apparatus, R (u, v) is the first Fourier transform, C (u, v) is the second Fourier transform,is an inverse fourier transform.

6. A control apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of the preceding claims 1 to 5 when executing the computer program.

7. The photothermal reflection microscopic thermal imaging device is characterized by comprising an illumination light path system, an imaging light path system and a camera;

and the distance of at least a preset maximum defocusing range is reserved at the position of the light source imaging corresponding to the maximum illumination light intensity in the illumination light path system, which is far away from the ideal focusing position.

8. The photothermal reflection microthermal imaging apparatus according to claim 7, wherein said illumination optical path system is illuminated in a critical illumination mode,

the light source imaging position corresponding to the maximum illumination intensity in the critical illumination is above or below the image focal plane, and the distance between the light source imaging position corresponding to the maximum illumination intensity in the critical illumination and the image focal plane is greater than the preset maximum defocus range;

or the illumination mode of the illumination optical path system is Kohler illumination,

the illumination light in the kohler illumination is diffused or converged outside the preset maximum defocusing range.

9. A photothermal reflection microscopic thermal imaging system comprising the control device according to claim 6, the photothermal reflection microscopic thermal imaging device according to claim 7 or 8, and a displacement stage;

the control device is respectively and electrically connected with the photothermal reflection micro thermal imaging device and the displacement table;

the photothermal reflection microscopic thermal imaging device is used for collecting a collected image of the measured piece when the measured piece is positioned at a position to be focused and collecting a reference image of the measured piece when the measured piece is positioned at the focusing position;

the displacement table is used for placing a measured piece and moving the measured piece according to the defocusing direction and the defocusing depth so as to focus the measured piece.

10. The photothermal reflection microthermal imaging system of claim 9 further comprising: a temperature control table; the photothermal reflection microscopic thermal imaging device comprises an optical platform and an optical subsystem; the optical subsystem comprises an illumination light path system, an imaging light path system and a camera;

the temperature control table is positioned on the displacement table and is electrically connected with the control device; the optical subsystem and the displacement table are respectively positioned on the optical platform;

the optical subsystem is used for collecting a collected image of the measured piece when the measured piece is positioned at a position to be focused and collecting a reference image of the measured piece when the measured piece is positioned at a focusing position; the optical platform is used for providing support for the optical subsystem and the displacement table.

11. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5 above.

Technical Field

The invention relates to the technical field of microscopic temperature imaging, in particular to an automatic focusing method and a control device for photothermal reflection microscopic thermal imaging.

Background

The photothermal reflection temperature measurement technology is a non-contact temperature measurement technology, and is based on the photothermal reflection phenomenon, which is basically characterized in that the reflectivity of an object changes along with the temperature change of the object.

When temperature measurement is performed based on photothermal reflection, in order to realize microscopic thermal imaging with high spatial resolution, a photothermal reflection microscopic imaging device is generally constructed based on a high-performance optical microscope. The detection light is provided by an illumination light path system of the optical microscope, microscopic imaging is recorded by a high-performance camera, and the output camera reading is used as a measured value.

However, in the temperature measurement process, in order to ensure the measurement accuracy, the camera reading at the reference temperature and the camera reading at the temperature to be measured generally need to be averaged by multiple frames of images. This requires that the data on each pixel of the camera and the spatial position of the surface of the measured object have a stable correspondence relationship during the whole measurement process, and if the correspondence relationship is disturbed, the accuracy of the temperature measurement result will be affected. However, the temperature changes for several times during the test process, and the thermal expansion of the corresponding tested piece can cause the defocusing of the collected image to cause the image blurring, so that the multiple focusing is required. However, the existing manual focusing method has poor repeatability and insufficient focusing consistency, and further influences the focusing stability of multi-frame collected images, thereby introducing extra measurement errors and influencing the accuracy of measurement results.

Disclosure of Invention

The embodiment of the invention provides an automatic focusing method and a control device for photothermal reflection microscopic thermal imaging, and aims to solve the problems that the existing manual focusing method is not enough in consistency, influences the focusing stability of multi-frame collected images and causes inaccurate measuring results.

In a first aspect, an embodiment of the present invention provides an auto-focusing method for photothermal reflection microscopy thermal imaging, including:

acquiring an acquired image of a measured piece which is acquired by a photothermal reflection microscopic thermal imaging device and is positioned at a position to be focused; the illumination intensity of the photothermal reflection microscopic thermal imaging device monotonously changes within a preset maximum defocusing range;

calculating to obtain a first total intensity value of the collected image according to the collected image, and calculating to obtain a second total intensity value of the reference image according to the reference image; the reference image is an image of the measured piece which is acquired by the photothermal reflection micro thermal imaging device and is located at a focusing position;

comparing the first total intensity value with the second total intensity value, and determining the defocusing direction of the measured piece based on the comparison result and the monotonous change trend of the illumination intensity in a preset maximum defocusing range;

acquiring the defocus depth of the measured piece; wherein, the out-of-focus direction and the out-of-focus depth are used for focusing the measured piece.

In a possible implementation manner, the determining the defocus direction of the measured object based on the comparison result and a tendency of monotonic change of the illumination intensity in a preset maximum defocus range includes:

when the trend that the illumination light intensity monotonically changes in the preset maximum defocusing range is that the illumination light intensity monotonically increases along with the increase of the object distance in the preset maximum defocusing range, if the first total intensity value is greater than the second total intensity value, determining the defocusing direction of the measured piece as the direction of the increase of the object distance; if the first total intensity value is smaller than the second total intensity value, determining the defocusing direction of the measured piece as the direction of reducing the object distance;

when the monotonous change trend of the illumination light intensity in the preset maximum defocusing range is that the illumination light intensity monotonously decreases along with the increase of the object distance in the preset maximum defocusing range, if the first total intensity value is larger than the second total intensity value, determining the defocusing direction of the measured piece as the direction of the decrease of the object distance; and if the first total intensity value is smaller than the second total intensity value, determining that the defocusing direction of the measured piece is the direction of increasing the object distance.

In a possible implementation manner, the calculating to obtain a first total intensity value of the collected image according to the collected image and obtaining a second total intensity value of the reference image according to the reference image includes:

according toCalculating to obtain a first total intensity value of the collected image according toCalculating to obtain a second total intensity value of the reference image;

wherein, I_cIs the first total intensity value, c (x, y) is the gray value of the (x, y) pixel point in the collected image, I_rAnd r (x, y) is the gray value of the (x, y) pixel point in the reference image, and is the second total intensity value.

In one possible implementation, obtaining the defocus depth of the measured object includes:

calculating to obtain a first Fourier transform of the collected image according to the collected image, and calculating to obtain a second Fourier transform of the reference image according to the reference image;

determining a fitted diameter of a point spread function of an optical subsystem in the photothermal reflection micro thermography arrangement from the first Fourier transform and the second Fourier transform;

and calculating the defocusing depth of the measured piece according to the fitting diameter and the imaging parameters of an optical subsystem in the photothermal reflection micro thermal imaging device.

In one possible implementation, the determining a fitted diameter of a point spread function of an optical subsystem in the photothermal reflection micro thermography device according to the first fourier transform and the second fourier transform comprises:

according toOrCalculating to obtain a point spread function of an optical subsystem in the photothermal reflection micro thermal imaging device;

determining a fitting diameter of the point spread function according to the point spread function;

wherein p (x, y) is the point spread function of the optical subsystem in the photothermal reflectance micro thermography apparatus, R (u, v) is the first Fourier transform, C (u, v) is the second Fourier transform,is an inverse fourier transform.

In a second aspect, an embodiment of the present invention provides a control apparatus, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the method according to the first aspect or any possible implementation manner of the first aspect.

In a third aspect, an embodiment of the present invention provides a photothermal reflection micro thermal imaging apparatus, including an illumination optical path system, an imaging optical path system, and a camera;

and the distance of at least a preset maximum defocusing range is reserved at the position of the light source imaging corresponding to the maximum illumination light intensity in the illumination light path system, which is far away from the ideal focusing position.

In a possible implementation manner, the illumination mode of the illumination light path system is critical illumination,

the light source imaging position corresponding to the maximum illumination intensity in the critical illumination is above or below the image focal plane, and the distance between the light source imaging position corresponding to the maximum illumination intensity in the critical illumination and the image focal plane is greater than the preset maximum defocus range;

or the illumination mode of the illumination optical path system is Kohler illumination,

the illumination light in the kohler illumination is diffused or converged outside the preset maximum defocusing range.

In a fourth aspect, an embodiment of the present invention provides a photothermal reflection micro thermal imaging system, including the control device described in the second aspect, the photothermal reflection micro thermal imaging device described in the third aspect or any possible implementation manner of the third aspect, and a displacement stage;

the control device is respectively and electrically connected with the photothermal reflection micro thermal imaging device and the displacement table;

the photothermal reflection microscopic thermal imaging device is used for collecting a collected image of the measured piece when the measured piece is positioned at a position to be focused and collecting a reference image of the measured piece when the measured piece is positioned at the focusing position;

the displacement table is used for placing a measured piece and moving the measured piece according to the defocusing direction and the defocusing depth so as to focus the measured piece.

In one possible implementation, the photothermal reflection microscopic thermal imaging system further includes: a temperature control table; the photothermal reflection microscopic thermal imaging device comprises an optical platform and an optical subsystem; the optical subsystem comprises an illumination light path system, an imaging light path system and a camera;

the temperature control table is positioned on the displacement table and is electrically connected with the control device; the optical subsystem and the displacement table are respectively positioned on the optical platform;

the optical subsystem is used for collecting a collected image of the measured piece when the measured piece is positioned at a position to be focused and collecting a reference image of the measured piece when the measured piece is positioned at a focusing position; the optical platform is used for providing support for the optical subsystem and the displacement table.

In a fifth aspect, the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the method according to the first aspect or any one of the possible implementation manners of the first aspect.

The embodiment of the invention provides an automatic focusing method and a control device for photothermal reflection microscopic thermal imaging. And comparing the first total intensity value with the second total intensity value, and automatically determining the defocusing direction of the detected piece relative to the focusing position at the position to be focused based on the comparison result and the monotonous change trend of the illumination intensity in the preset maximum defocusing range. And simultaneously acquiring the out-of-focus depth of the measured piece, and further focusing the measured piece according to the out-of-focus direction and the out-of-focus depth. According to the method and the device, the defocusing direction of the detected piece relative to the focusing position at the position to be focused can be automatically determined according to the comparison result of the first total intensity value and the second total intensity value based on the monotonous change trend of the illumination intensity in the preset maximum defocusing range. On the one hand, the focusing efficiency can be improved, on the other hand, even if the temperature changes for multiple times, the reference image of the measured piece in the focusing position and the collected image of the measured piece in the position to be focused are focused, so that the problem of insufficient manual focusing consistency can be avoided, the focusing stability of multi-frame collected images is ensured, the error caused by defocusing in the collecting process is reduced, and the accuracy of the measurement result is improved when the temperature measurement is carried out based on photothermal reflection.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

FIG. 1 is a diagram of an application scenario of an auto-focusing method for photothermal reflection microscopy thermal imaging according to an embodiment of the present invention;

FIG. 2 is a flowchart of an implementation of an auto-focusing method for photothermal reflection microscopy thermography according to an embodiment of the present invention;

FIG. 3 is a flowchart of an implementation of acquiring a defocus depth of a measured object according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an autofocus device for photothermal reflection microscopy thermography according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a control device according to an embodiment of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

In order to make the objects, technical solutions and advantages of the present invention more apparent, the following description is made by way of specific embodiments with reference to the accompanying drawings.

In the prior art, the change in reflectance with temperature can be considered linear, and therefore can be characterized by a coefficient of rate of change, commonly referred to in the literature as the coefficient of photothermal reflection (thermo-reflection coefficient) or the coefficient of photothermal reflection Calibration (thermo-reflection Calibration coefficient), using C_TRIs defined as:

in the formula, R is a reference reflectivity, Δ R is a reflectivity variation, and Δ T is a temperature variation.

For most metal and semiconductor materials, C_TRIs usually in the range of (10)^-2～10^-5)K^-1And, depending on the material, the wavelength of the incident light, and the incident angle, if the surface of the object to be measured has a multilayer structure, the material composition of each layer and the interference of light among the multilayer materials also directly affect C_TRThe measurement of (A) is usually carried out by selecting an appropriate measurement wavelength for each sample (type or model) of the object to be measured, and determining C_TRCommonly referred to as C_TRCalibration (C)_TRcalibrmation) and using the determined C_TRTemperature measurements are taken.

At C_TRIn known cases, the temperature can be calculated by measuring the change in reflectance of the piece under test according to the following formula:

in the formula, T_xTo be measured for temperature, T₀For reference temperature, R_xIs the reflectivity at the temperature to be measured, R₀Is the reflectance at the reference temperature.

Since it is actually the rate of change of the reflectivity that is of interestTherefore, a beam of detection light (incident light) can be projected to the surface of a measured piece, and then the temperature measurement can be realized by measuring the change rate of the intensity of the reflected light, which is also the mainstream realization mode of the current photothermal reflection temperature measurement technology. The rate of change of reflectivity in the formula for calculating temperature can be equivalent to the rate of change of the detector reading, assuming the detected light intensity is constant, i.e. the formula for calculating temperature changes to:

in the formula, c_xFor the reading of the detector at the temperature to be measured, c₀Is the detector reading at the reference temperature.

In order to achieve high spatial resolution microthermography, photothermal reflection microthermography devices are typically constructed based on high performance optical microscopes. The detection light is provided by an illumination light path system of the optical microscope, microscopic imaging is recorded by a high-performance camera, and the output camera reading is used as a measured value c.

Due to C_TRLow magnitude, in order to guarantee measurement accuracy, at acquisition c₀And c_xUsually, a mean value of multiple frames of images is required, and the total number of measured frames is recorded as N, then:

from the above principle, it can be known that, in the whole measurement process, data on each pixel of the camera and the spatial position of the surface of the measured object need to have a stable corresponding relationship, and if the corresponding relationship is interfered, the accuracy of the temperature measurement result is affected. The temperature changes for several times in the test process, the image blurring caused by defocusing can be caused by the thermal expansion of the corresponding tested piece, multiple times of focusing is needed, the focusing consistency is good enough to ensure the stability and consistency of the corresponding relation, and otherwise, additional errors can be introduced.

However, the existing manual focusing method has poor repeatability and insufficient focusing consistency, thereby affecting the focusing stability of multi-frame collected images and the accuracy of measurement results.

In order to solve the above problems, an embodiment of the present invention provides an auto-focusing method for photothermal reflection micro thermal imaging, and fig. 1 is an application scenario diagram of the auto-focusing method for photothermal reflection micro thermal imaging according to the embodiment of the present invention. The method may be applied but is not limited to the application scenario.

The automatic focusing method for the photothermal reflection microscopic thermal imaging is combined with the photothermal reflection microscopic thermal imaging device to form the photothermal reflection microscopic thermal imaging system. As shown in fig. 1, the photothermal reflection micro thermal imaging system includes a photothermal reflection micro thermal imaging device composed of a control device 10, an optical platform 21 and an optical subsystem 22, a displacement stage 30, and the like.

The photo-thermal reflection micro thermal imaging device is used for collecting a collected image of a measured piece located at a position to be focused and collecting a reference image of the measured piece located at the focusing position. The photothermal reflection micro thermal imaging device and the displacement table 30 are electrically connected with the control device 10, the control device 10 obtains the collected image and the reference image collected by the photothermal reflection micro thermal imaging device, and the defocusing direction and the defocusing depth of the measured piece are obtained after the processing process of the automatic focusing method for photothermal reflection micro thermal imaging of the embodiment of the invention is executed. The displacement stage 30 is used for placing the object to be measured and moving the object to be measured according to the defocus direction and the defocus depth obtained by the control device 10 to focus the object to be measured.

Wherein the displacement stage 30 and the optical subsystem 22 may be respectively located on the optical platform 21, and the optical platform 21 is used to provide support for the optical subsystem 22 and the displacement stage 30.

Optionally, the displacement stage 30 may be a 3-axis nanometer displacement stage, so as to compensate for displacement of the measured object caused by temperature variation and other factors in the temperature testing process, for example, displacement of the measured object caused by defocusing.

Optionally, the photothermal reflection microscopic thermal imaging system may further include a temperature control table 40, wherein the temperature control table 40 is located on the displacement table 30, the temperature control table 40 may also be electrically connected to the control device 10, and the temperature control table 30 is used for placing the measured piece, so as to control the temperature environment of the measured piece. Illustratively, the temperature controlled stage 40 can be a programmed cold-hot stage.

According to the photothermal reflection micro thermal imaging system provided by the embodiment of the invention, the photothermal reflection micro thermal imaging device is a photothermal reflection micro thermal imaging device with the following illumination light intensity monotonously changing in the preset maximum defocusing range, and the control device is a device for processing according to the following automatic focusing method for photothermal reflection micro thermal imaging, so that the defocusing direction of the measured piece relative to the focusing position in the to-be-focused position can be automatically determined based on the monotonous changing trend of the illumination light intensity in the preset maximum defocusing range and the processing of the control device, and focusing is performed according to the defocusing direction and the defocusing depth. The focusing efficiency can be improved, when the temperature changes for many times, the reference image of the measured piece at the focusing position and the collected image of the measured piece at the position to be focused are focused for many times, the problem that manual focusing consistency is not enough is avoided, the focusing stability of multi-frame collected images is guaranteed, errors caused by defocusing in the collecting process are reduced, and the accuracy of the measuring result is improved when temperature measurement is carried out based on photothermal reflection.

With reference to fig. 1, the photothermal reflection microthermal imaging apparatus for acquiring an acquired image of a measured object located at a position to be focused and acquiring a reference image of the measured object located at the position to be focused can be divided into an illumination optical path system (e.g., an apparatus for realizing that an illumination beam emitted by an LED in fig. 1 reaches a space where the measured object is located), an imaging optical path system (e.g., an apparatus for realizing that a reflected beam emitted by the measured object in fig. 1 reaches a camera) and a camera according to functions.

In order to facilitate the auto-focusing method for photothermal reflection microscopy thermal imaging provided by the embodiment of the invention to determine the defocus direction of the measured piece within the preset maximum defocus range, it is required that the light source imaging position corresponding to the maximum illumination intensity in the illumination light path system is far away from the ideal focusing position by at least the distance of the preset maximum defocus range.

Optionally, the illumination mode of the illumination optical path system is critical illumination, the light source imaging position corresponding to the maximum illumination intensity in the critical illumination is above or below the image-side focal plane, and the distance between the light source imaging position corresponding to the maximum illumination intensity in the critical illumination and the image-side focal plane is greater than the preset maximum defocus range. The photothermal reflection micro thermal imaging device is used for obtaining the distance of at least a preset maximum defocusing range of the light source imaging position far away from the ideal focusing position corresponding to the maximum illumination light intensity in the illumination light path system. Or adjusting the original photothermal reflection microthermal imaging device of the light source imaging position corresponding to the maximum illumination intensity in the critical illumination on the image-side focal plane to obtain the photothermal reflection microthermal imaging device of which the light source imaging position corresponding to the maximum illumination intensity in the critical illumination is above or below the image-side focal plane after adjustment, and the distance between the light source imaging position corresponding to the maximum illumination intensity in the critical illumination and the image-side focal plane is greater than the preset maximum defocusing range.

Optionally, the illumination mode of the illumination optical path system is kohler illumination, and illumination light in the kohler illumination is diffused or converged outside a preset maximum defocus range. The photothermal reflection micro thermal imaging device is used for obtaining the distance of at least a preset maximum defocusing range of the light source imaging position far away from the ideal focusing position corresponding to the maximum illumination light intensity in the illumination light path system. Or adjusting the parallel illuminating light in the Kohler illumination to obtain the photo-thermal reflection micro-thermal imaging device, wherein the illuminating light in the Kohler illumination is diffused or converged outside the preset maximum defocusing range after adjustment.

Referring to fig. 2, it shows a flow chart of implementing the auto-focusing method for photothermal reflection microscopy thermal imaging provided by the embodiment of the present invention, which is detailed as follows:

in step 201, a captured image of a measured object captured by a photothermal reflection micro thermal imaging device when the measured object is located at a position to be focused is obtained.

The photothermal reflection micro thermal imaging device is the photothermal reflection micro thermal imaging device provided in the above embodiment, wherein the illumination light intensity monotonically changes within a preset maximum defocusing range.

In step 202, a first total intensity value of the collected image is calculated according to the collected image, and a second total intensity value of the reference image is calculated according to the reference image.

The reference image is an image of a measured piece which is collected by the photothermal reflection micro thermal imaging device and is located at a focusing position.

The detected piece can be located at a focusing position through manual focusing or an existing automatic focusing method, and then the corresponding reference image is collected by the photothermal reflection microthermal imaging device with the illumination intensity monotonously changing within the preset maximum defocusing range. And when focusing and focus following are needed in the subsequent testing process, processing the acquired image by taking the reference image as a reference.

Optionally, the obtaining a first total intensity value of the collected image by calculation according to the collected image, and obtaining a second total intensity value of the reference image by calculation according to the reference image may include:

according toComputingObtaining a first total intensity value of the collected image according toAnd calculating to obtain a second total intensity value of the reference image.

Wherein, I_cAnd c (x, y) is a gray value of a pixel point (x, y) in the acquired image and is used for representing the acquired image. I is_rAnd r (x, y) is the gray value of the (x, y) pixel point in the reference image and is used for representing the reference image.

In step 203, the first total intensity value and the second total intensity value are compared, and the defocus direction of the measured object is determined based on the comparison result and the monotonous change trend of the illumination intensity in the preset maximum defocus range.

Optionally, determining the defocus direction of the measured object based on the comparison result and the monotonous variation trend of the illumination intensity in the preset maximum defocus range may include:

when the trend that the illumination light intensity changes monotonously in the preset maximum defocusing range is that the illumination light intensity increases monotonously along with the increase of the object distance in the preset maximum defocusing range, if the first total intensity value I_cGreater than the second total intensity value I_rDetermining the defocusing direction of the measured piece as the direction of increasing the object distance; if the first total intensity value I_cLess than the second total intensity value I_rAnd determining the defocusing direction of the measured piece as the direction of the object distance reduction.

When the monotonous change trend of the illumination light intensity in the preset maximum defocusing range is that the illumination light intensity monotonously decreases along with the increase of the object distance in the preset maximum defocusing range, if the first total intensity value I_cGreater than the second total intensity value I_rDetermining the defocusing direction of the measured piece as the direction of the object distance reduction; if the first total intensity value I_cLess than the second total intensity value I_rAnd determining the defocusing direction of the measured piece as the direction of increasing the object distance.

In this embodiment, since the illumination light intensity of the photothermal reflection microthermal imaging device changes monotonically within the preset maximum defocus range (i.e., the illumination light intensity increases or decreases monotonically with the increase of the object distance within the preset maximum defocus range), and the reference image is the image when the measured object is located at the in-focus position, it can be determined whether the collected image is shifted in the direction in which the object distance increases or in the direction in which the object distance decreases relative to the reference image according to the magnitude of the first total intensity value of the collected image relative to the second total intensity value of the reference image, so as to determine the defocus direction of the measured object relative to the in-focus position when the measured object is located at the to-be-focused position.

In step 204, the defocus depth of the measured object is obtained. The defocusing direction and the defocusing depth are used for focusing the measured piece.

When the measured piece is focused, besides the defocusing direction, the defocusing depth also needs to be determined.

Optionally, referring to fig. 3, it shows a flowchart of implementing obtaining the defocus depth of the measured object according to the embodiment of the present invention, which is detailed as follows:

in step 301, a first fourier transform of the captured image is computed from the captured image and a second fourier transform of the reference image is computed from the reference image.

Wherein, according toCalculating to obtain a first Fourier transform C (u, v) of the acquired imageA second fourier transform R (u, v) of the reference image is computed. Wherein the content of the first and second substances,is a fourier transform.

In step 302, a fitted diameter of a point spread function of an optical subsystem in a photothermal reflectance micro thermography arrangement is determined from the first fourier transform and the second fourier transform.

Optionally, determining a fitted diameter of a point spread function of an optical subsystem in the photothermal reflectance micro thermography arrangement from the first fourier transform and the second fourier transform may comprise:

according toOrCalculating to obtain a point spread function of an optical subsystem in the photothermal reflection micro thermal imaging device; and determining the fitting diameter of the point spread function according to the point spread function.

Wherein p (x, y) is the point spread function of the optical subsystem in the photothermal reflectance micro thermography apparatus, R (u, v) is the first Fourier transform, C (u, v) is the second Fourier transform,is an inverse fourier transform.

Where only defocus effects are considered, the point spread function p (x, y) should be in the approximate airy disk form. Depending on the precision, the fitting diameter of the point spread function p (x, y) in the approximate airy disk form can be obtained by directly counting the number of pixels (i.e., counting from the origin to two opposite directions until the values of p (x, y) in both opposite directions are 0). Or solving according to an analytical expression of the airy disk according to discrete points on the point spread function p (x, y), determining unknown parameters in the analytical expression of the airy disk, and further determining the fitting diameter of the point spread function p (x, y) according to the analytical expression of the airy disk after the unknown parameters are determined.

In step 303, the defocus depth of the measured object is calculated according to the fitting diameter and the imaging parameters of the optical subsystem in the photothermal reflection micro thermal imaging device.

Wherein the imaging parameters of the optical subsystem in the photothermal reflection micro thermal imaging apparatus may comprise one or more of: camera pixel size parameters, magnification parameters, and objective lens aperture angle parameters.

Optionally, calculating the defocus depth of the measured object according to the fitting diameter and the imaging parameters of the optical subsystem in the photothermal reflection micro thermal imaging apparatus, and may include:

according toAnd calculating to obtain the out-of-focus depth of the measured piece.

Wherein d is the fitting diameter, a is a camera pixel size parameter, m is a magnification parameter, theta is one half of an objective lens aperture angle parameter, and the numerical aperture in the air is n.a. ═ sin theta.

After the defocusing direction and the defocusing depth are obtained, the vertical direction of a 3-axis nanometer displacement table in the photothermal reflection microthermal imaging system can be operated through a proportional integral derivative control algorithm so as to carry out closed-loop focusing.

When continuous focusing is needed to follow focus, each collected image can be processed according to the automatic focusing method for photothermal reflection microscopic thermal imaging in the embodiment, so that focus following is realized.

According to the embodiment of the invention, the photo-thermal reflection micro-thermal imaging device with the illumination intensity monotonously changing in the preset maximum defocusing range is used for collecting the collected image of the measured piece at the position to be focused and collecting the reference image of the measured piece at the focusing position, so that the first total intensity value of the collected image can be obtained according to the collected image, and the second total intensity value of the reference image can be obtained according to the reference image. And comparing the first total intensity value with the second total intensity value, and automatically determining the defocusing direction of the detected piece relative to the focusing position at the position to be focused based on the comparison result and the monotonous change trend of the illumination intensity in the preset maximum defocusing range. And simultaneously acquiring the out-of-focus depth of the measured piece, and further focusing the measured piece according to the out-of-focus direction and the out-of-focus depth. According to the method and the device, the defocusing direction of the detected piece relative to the focusing position at the position to be focused can be automatically determined according to the comparison result of the first total intensity value and the second total intensity value based on the monotonous change trend of the illumination intensity in the preset maximum defocusing range. On the one hand, the focusing efficiency can be improved, on the other hand, even if the temperature changes for multiple times, the reference image of the measured piece in the focusing position and the collected image of the measured piece in the position to be focused are focused, so that the problem of insufficient manual focusing consistency can be avoided, the focusing stability of multi-frame collected images is ensured, the error caused by defocusing in the collecting process is reduced, and the accuracy of the measurement result is improved when the temperature measurement is carried out based on photothermal reflection.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

The following are embodiments of the apparatus of the invention, reference being made to the corresponding method embodiments described above for details which are not described in detail therein.

Fig. 4 shows a schematic structural diagram of an auto-focusing device for photothermal reflection microscopy thermal imaging provided by an embodiment of the present invention, and for convenience of illustration, only the parts related to the embodiment of the present invention are shown, and the details are as follows:

as shown in fig. 4, the autofocus device 4 for photothermal reflection microscopic thermal imaging includes: a first obtaining module 41, a calculating module 42, a comparing module 43 and a second obtaining module 44.

The first acquisition module 41 is configured to acquire an acquired image of a measured object, acquired by the photothermal reflection microscopic thermal imaging apparatus, at a position to be focused; the illumination intensity of the photothermal reflection microscopic thermal imaging device monotonously changes within a preset maximum defocusing range;

the calculation module 42 is configured to calculate a first total intensity value of the collected image according to the collected image, and calculate a second total intensity value of the reference image according to the reference image; the reference image is an image of the measured piece which is acquired by the photothermal reflection micro thermal imaging device and is located at a focusing position;

the comparison module 43 is configured to compare the first total intensity value with the second total intensity value, and determine the defocus direction of the measured object based on the comparison result and the trend of monotonic change of the illumination intensity in a preset maximum defocus range;

a second obtaining module 44, configured to obtain a defocus depth of the measured object; wherein, the out-of-focus direction and the out-of-focus depth are used for focusing the measured piece.

According to the embodiment of the invention, the photo-thermal reflection micro-thermal imaging device with the illumination intensity monotonously changing in the preset maximum defocusing range is used for collecting the collected image of the measured piece at the position to be focused and collecting the reference image of the measured piece at the focusing position, so that the first total intensity value of the collected image can be obtained according to the collected image, and the second total intensity value of the reference image can be obtained according to the reference image. And comparing the first total intensity value with the second total intensity value, and automatically determining the defocusing direction of the detected piece relative to the focusing position at the position to be focused based on the comparison result and the monotonous change trend of the illumination intensity in the preset maximum defocusing range. And simultaneously acquiring the out-of-focus depth of the measured piece, and further focusing the measured piece according to the out-of-focus direction and the out-of-focus depth. According to the method and the device, the defocusing direction of the detected piece relative to the focusing position at the position to be focused can be automatically determined according to the comparison result of the first total intensity value and the second total intensity value based on the monotonous change trend of the illumination intensity in the preset maximum defocusing range. On the one hand, the focusing efficiency can be improved, on the other hand, even if the temperature changes for multiple times, the reference image of the measured piece in the focusing position and the collected image of the measured piece in the position to be focused are focused, so that the problem of insufficient manual focusing consistency can be avoided, the focusing stability of multi-frame collected images is ensured, the error caused by defocusing in the collecting process is reduced, and the accuracy of the measurement result is improved when the temperature measurement is carried out based on photothermal reflection.

In a possible implementation manner, the comparing module 43 may be configured to, when the trend that the illumination light intensity changes monotonically within the preset maximum defocus range is that the illumination light intensity increases monotonically with the increase of the object distance within the preset maximum defocus range, determine, if the first total intensity value is greater than the second total intensity value, that the defocus direction of the measured object is the direction in which the object distance increases; if the first total intensity value is smaller than the second total intensity value, determining the defocusing direction of the measured piece as the direction of reducing the object distance;

when the monotonous change trend of the illumination light intensity in the preset maximum defocusing range is that the illumination light intensity monotonously decreases along with the increase of the object distance in the preset maximum defocusing range, if the first total intensity value is larger than the second total intensity value, determining the defocusing direction of the measured piece as the direction of the decrease of the object distance; and if the first total intensity value is smaller than the second total intensity value, determining that the defocusing direction of the measured piece is the direction of increasing the object distance.

In one possible implementation, the calculation module 42 may be configured to calculate the data according toCalculating to obtain a first total intensity value of the collected image according toCalculating to obtain a second total intensity value of the reference image;

wherein, I_cIs the first total intensity value, c (x, y) is the gray value of the (x, y) pixel point in the collected image, I_rAnd r (x, y) is the gray value of the (x, y) pixel point in the reference image, and is the second total intensity value.

In a possible implementation manner, the second obtaining module 44 may be configured to calculate a first fourier transform of the acquired image according to the acquired image, and calculate a second fourier transform of the reference image according to the reference image;

determining a fitted diameter of a point spread function of an optical subsystem in the photothermal reflection micro thermography arrangement from the first Fourier transform and the second Fourier transform;

and calculating the defocusing depth of the measured piece according to the fitting diameter and the imaging parameters of an optical subsystem in the photothermal reflection micro thermal imaging device.

In one possible implementation, the second obtaining module 44 may be configured to obtain the data according toOrCalculating to obtain a point spread function of an optical subsystem in the photothermal reflection micro thermal imaging device;

determining a fitting diameter of the point spread function according to the point spread function;

wherein p (x, y) is the point spread function of the optical subsystem in the photothermal reflectance micro thermography apparatus, R (u, v) is the first Fourier transform, C (u, v) is the second Fourier transform,is an inverse fourier transform.

Fig. 5 is a schematic diagram of a control device according to an embodiment of the present invention. As shown in fig. 5, the control device 5 of this embodiment includes: a processor 50, a memory 51 and a computer program 52 stored in said memory 51 and executable on said processor 50. The processor 50 executes the computer program 52 to implement the steps of the above-mentioned embodiments of the autofocus method for photothermal reflection microscopy thermography, such as steps 201 to 204 shown in fig. 2 or steps 301 to 303 shown in fig. 3. Alternatively, the processor 50, when executing the computer program 52, implements the functions of the modules in the above-described device embodiments, such as the functions of the modules 41 to 44 shown in fig. 4.

Illustratively, the computer program 52 may be partitioned into one or more modules/units that are stored in the memory 51 and executed by the processor 50 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 52 in the control device 5. For example, the computer program 52 may be divided into the modules 41 to 44 shown in fig. 4.

The control device 5 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The control device 5 may include, but is not limited to, a processor 50 and a memory 51. It will be understood by those skilled in the art that fig. 5 is only an example of the control device 5, and does not constitute a limitation to the control device 5, and may include more or less components than those shown, or combine some components, or different components, for example, the control device may also include input-output devices, network access devices, buses, etc.

The Processor 50 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 51 may be an internal storage unit of the control device 5, such as a hard disk or a memory of the control device 5. The memory 51 may also be an external storage device of the control apparatus 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the control apparatus 5. Further, the memory 51 may also include both an internal storage unit of the control apparatus 5 and an external storage device. The memory 51 is used for storing the computer program and other programs and data required by the control device. The memory 51 may also be used to temporarily store data that has been output or is to be output.

The embodiment of the invention also provides a photothermal reflection microscopic thermal imaging device, which comprises an illumination light path system, an imaging light path system and a camera;

and the distance of the light source imaging position corresponding to the maximum illumination light intensity in the illumination light path system is far away from the ideal focusing position by at least presetting the distance of the maximum defocusing range.

Optionally, the illumination mode of the illumination optical path system is critical illumination, the light source imaging position corresponding to the maximum illumination intensity in the critical illumination is above or below the image-side focal plane, and the distance between the light source imaging position corresponding to the maximum illumination intensity in the critical illumination and the image-side focal plane is greater than the preset maximum defocus range.

Optionally, the illumination mode of the illumination optical path system is kohler illumination, and illumination light in the kohler illumination is diffused or converged outside the preset maximum defocus range.

The photothermal reflection microscopic thermal imaging device provided by the embodiment of the invention can ensure that the light source imaging position corresponding to the maximum illumination light intensity in the illumination light path system is far away from the ideal focusing position by at least presetting the distance of the maximum defocusing range. The automatic focusing method for photothermal reflection microscopic thermal imaging provided by the embodiment can determine the defocusing direction of the measured piece within the preset maximum defocusing range.

The embodiment of the present invention further provides a photothermal reflection micro thermal imaging system, referring to fig. 1, the system includes a control device 10, a photothermal reflection micro thermal imaging device, and a displacement stage 30;

the control device 10 is electrically connected with the photothermal reflection micro thermal imaging device and the displacement table 30 respectively.

The photothermal reflection microscopic thermal imaging device is used for collecting a collected image of the measured piece when the measured piece is located at a position to be focused and collecting a reference image of the measured piece when the measured piece is located at the focusing position.

The displacement table 30 is used for placing the measured object and moving the measured object according to the defocusing direction and the defocusing depth so as to focus the measured object.

Optionally, the photothermal reflection microscopic thermal imaging system further comprises: a temperature control stage 40; the photothermal reflection microscopic thermal imaging device comprises an optical platform 21 and an optical subsystem 22; the optical subsystem 22 includes, among other things, an illumination optical path system, an imaging optical path system, and a camera.

The temperature control table 40 is positioned on the displacement table 30, and the temperature control table 40 is electrically connected with the control device 10; the optical subsystem 22 and the displacement table 30 are respectively positioned on the optical platform 21; the temperature control table 40 is used for placing a tested piece, and the optical subsystem 22 is used for collecting a collected image of the tested piece when the tested piece is located at a position to be focused and collecting a reference image of the tested piece when the tested piece is located at a focusing position; the optical bench 21 is used to provide support for the optical subsystem 22 and the displacement table 30.

According to the photothermal reflection micro thermal imaging system provided by the embodiment of the invention, the photothermal reflection micro thermal imaging device is the photothermal reflection micro thermal imaging device with the illumination light intensity monotonously changing in the preset maximum defocusing range, and the control device is the device for processing according to the automatic focusing method for photothermal reflection micro thermal imaging, so that the defocusing direction of the measured piece relative to the focusing position in the position to be focused can be automatically determined based on the monotonous changing trend of the illumination light intensity in the preset maximum defocusing range and the processing of the control device, and focusing is performed according to the defocusing direction and the defocusing depth. The focusing efficiency can be improved, when the temperature changes for many times, the reference image of the measured piece at the focusing position and the collected image of the measured piece at the position to be focused are focused for many times, the problem that manual focusing consistency is not enough is avoided, the focusing stability of multi-frame collected images is guaranteed, errors caused by defocusing in the collecting process are reduced, and the accuracy of the measuring result is improved when temperature measurement is carried out based on photothermal reflection.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/control apparatus and method may be implemented in other ways. For example, the above-described apparatus/control apparatus embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer readable storage medium and can be executed by a processor to realize the steps of the embodiments of the autofocus method for photothermal reflection microscopy thermal imaging described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media which may not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.