阅读说明：本技术 异源图像的优化配准方法及系统 (Optimal registration method and system for different-source images ) 是由 李培华 刘玉莉 鲁兴平 钱名思 季常刚 徐晨 章盛 于 2021-10-15 设计创作，主要内容包括：本发明提供一种异源图像的优化配准方法及系统,属于图像配准技术领域。所述方法包括：采集待配准的红外图像信息和对应的可见光图像信息；根据预设预处理规则对所述红外图像信息和所述可见光图像信息进行预处理；根据预设特征提取算法将预处理后的红外图像和预处理后的可见光图像处理为对应的红外图像特征描述符序列和可见光图像特征描述符序列；根据预设特征匹配规则进行所述红外图像特征描述符序列和所述可见光图像特征描述符序列的特征匹配,获得投影变换矩阵；输出所述投影变换矩阵,完成配准。本发明方案相对于传统SIFT算法、传统SURF算法,具备图像配准准确率更高的优势。(The invention provides an optimal registration method and system for a heterogeneous image, and belongs to the technical field of image registration. The method comprises the following steps: acquiring infrared image information to be registered and corresponding visible light image information; preprocessing the infrared image information and the visible light image information according to a preset preprocessing rule; processing the preprocessed infrared image and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm; performing characteristic matching of the infrared image characteristic descriptor sequence and the visible light image characteristic descriptor sequence according to a preset characteristic matching rule to obtain a projection transformation matrix; and outputting the projective transformation matrix to finish registration. Compared with the traditional SIFT algorithm and the traditional SURF algorithm, the scheme of the invention has the advantage of higher image registration accuracy.)

1. An optimized registration method of a heterogeneous image, which is applied to registration between an infrared image and a visible light image, and is characterized in that the method comprises the following steps:

acquiring infrared image information to be registered and corresponding visible light image information;

preprocessing the infrared image information and the visible light image information according to a preset preprocessing rule;

processing the preprocessed infrared image and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm;

performing characteristic matching of the infrared image characteristic descriptor sequence and the visible light image characteristic descriptor sequence according to a preset characteristic matching rule to obtain a projection transformation matrix;

and outputting the projective transformation matrix to finish registration.

2. The method of claim 1, wherein the pre-set pre-processing rules comprise:

smoothing, calculating image gradient information, suppressing non-maximum value of image gradient amplitude, detecting and connecting edge contours.

3. The method of claim 1, wherein the preset feature extraction algorithm is designed based on the KAZE algorithm.

4. The method according to claim 3, wherein the processing the pre-processed infrared image and the pre-processed visible light image into a corresponding infrared image feature descriptor sequence and visible light image feature descriptor sequence according to a preset feature extraction algorithm comprises:

constructing a nonlinear scale space by using a nonlinear diffusion equation; wherein the nonlinear diffusion equation is:

wherein L represents the brightness of the image;

t represents a scale parameter;

div represents the divergence;

represents a gradient;

c (x, y, t) represents the conduction function at coordinate (x, y);

representing the gradient of the original image L after Gaussian filtering on a delta scale;

for the preprocessed infrared image and the preprocessed visible light image, extracting image characteristic points of the infrared image and image characteristic points of the visible light image in the nonlinear scale space by calculating Hessian matrix values of pixel points;

respectively calculating the main directions of the image characteristic points of the infrared image and the image characteristic points of the visible light image;

respectively aiming at the infrared image and the visible light image, selecting a rectangular area with a preset size by taking an image characteristic point as a center, dividing sub-areas with a preset number, and performing weighted calculation on first-order differentials of pixel points in the sub-areas by using a Gaussian core to obtain descriptor vectors of the sub-areas;

and performing weighted calculation and normalization processing on the descriptor vectors of all the sub-regions by using a Gaussian window to obtain an infrared image feature descriptor sequence and a visible light image feature descriptor sequence.

5. The method as claimed in claim 4, wherein the extracting of the image feature points of the infrared image and the image feature points of the visible light image in the nonlinear scale space by calculating the Hessian matrix value of the pixel point comprises:

respectively aiming at the infrared image and the visible light image, comparing the Hessian matrix value of each pixel point with the Hessian matrix values of all the pixel points of the current layer and the two adjacent layers of the current layer;

and if the Hessian matrix value of the current pixel point is greater than or less than the Hessian matrix values of all the pixel points, determining that the Hessian matrix value extreme point of the corresponding pixel point is the image feature point.

6. The method according to claim 4, wherein the calculating of the main directions of the image feature points of the infrared image and the visible light image respectively comprises:

selecting a circular area with a preset radius by taking the image characteristic point as a central point, and calculating first-order differential of pixel points in the area;

rotating the sector area with a preset opening angle and step length around the central point, and accumulating the first order differential in the rotating process;

and screening out the direction corresponding to the maximum accumulated value in the accumulation process as the main direction.

7. The method according to claim 1, wherein the performing feature matching of the infrared image feature descriptor sequence and the visible light image feature descriptor sequence according to a preset feature matching rule to obtain a projective transformation matrix comprises:

according to the infrared image feature descriptor sequence and the visible light image feature descriptor sequence, performing feature point matching on the infrared image to the visible light image by using a KNN algorithm to obtain a first matching image feature point pair set;

according to the infrared image feature descriptor sequence and the visible light image feature descriptor sequence, performing feature point matching on the visible light image to the infrared image by using a FLANN algorithm to obtain a second matching image feature point pair set;

and comparing the first matching image characteristic point pair set with the second matching image characteristic point pair set, screening out the same matching image characteristic point pairs, and integrating the same matching image characteristic point pairs as a final matching image characteristic point pair set to serve as the projection transformation matrix.

8. The method according to claim 7, wherein the using the KNN algorithm to perform feature point matching on the infrared image to the visible light image to obtain a first matching image feature point pair set comprises:

taking any image feature point of the infrared image feature descriptor sequence as an object, traversing all image feature points in the visible light image feature descriptor sequence by using an Euclidean distance definitional mode, screening a definition value minimum value and a definition value secondary minimum value, and taking the definition value minimum value and the definition value secondary minimum value as a nearest neighbor image feature point and a next nearest neighbor image feature point respectively;

judging whether the nearest neighbor image feature point and the next nearest neighbor image feature point meet a preset relationship, and if the nearest neighbor image feature point and the next nearest neighbor image feature point meet the preset relationship, judging that the nearest neighbor image feature point is a matched image feature point of image feature points in a preselected infrared image feature descriptor sequence; wherein the content of the first and second substances,

the preset relation is as follows:

wherein the content of the first and second substances,the image feature points are nearest neighbor image feature points;

d(U_i,V_2j) The image feature points are the next nearest neighbor image feature points;

T_h1is a preset relationship threshold;

traversing all image feature points in the infrared image feature descriptor sequence to obtain matched image feature points corresponding to all the image feature points;

and integrating each image characteristic point in the infrared image characteristic descriptor sequence and the corresponding matched image characteristic point, wherein each corresponding relation is used as a matched image characteristic point pair set, and the first characteristic matched image characteristic point pair set is obtained.

9. An optimized registration system for heterogeneous images for registration between infrared images and visible light images, the system comprising:

the acquisition unit is used for acquiring infrared image information to be registered and corresponding visible light image information;

a processing unit to:

preprocessing the infrared image information and the visible light image information according to a preset preprocessing rule;

processing the preprocessed infrared image and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm;

performing characteristic matching of the infrared image characteristic descriptor sequence and the visible light image characteristic descriptor sequence according to a preset characteristic matching rule to obtain a projection transformation matrix;

and the output unit is used for outputting the projection transformation matrix to finish the registration.

10. A computer readable storage medium having stored thereon instructions which, when executed on a computer, cause the computer to perform the method for optimized registration of a heterologous image according to any of claims 1-8.

Technical Field

The invention relates to the technical field of image registration, in particular to an optimal registration method and an optimal registration system for a heterogeneous image.

Background

The registration between the infrared image and the visible light image is widely applied in the fields of binocular vision, body temperature detection, fire rescue and the like, and the registration algorithm is one of hot techniques of the current computer vision field research. Because the infrared light and the visible light are respectively in different wave bands of the spectrum and have inherent characteristics, the infrared image is determined by the emissivity and the temperature of the surface of an object, and the visible light image is determined by the inverse rate and the shadow of the surface of the object, so that the imaging principle between the infrared image and the visible light image has difference, and then, as the infrared sensor and the visible light sensor cannot be in the same position and view angle when shooting the same region at the same time, the infrared sensor and the visible light sensor have certain view angle difference, the image has geometric distortion, and the estimation of a registration model between the infrared image and the visible light image is difficult, and the two factors are main reasons for low registration accuracy between the infrared image and the visible light image. The existing registration algorithm generally has the problem of low registration accuracy, and therefore, a new optimal registration method for a heterogeneous image needs to be created.

Disclosure of Invention

The embodiment of the invention aims to provide an optimal registration method and system for a heterogeneous image, so as to at least solve the problem that the registration accuracy of the existing registration algorithm is low.

In order to achieve the above object, a first aspect of the present invention provides an optimized registration method for a heterogeneous image, which is applied to registration between an infrared image and a visible light image, and the method includes: acquiring infrared image information to be registered and corresponding visible light image information; preprocessing the infrared image information and the visible light image information according to a preset preprocessing rule; processing the preprocessed infrared image and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm; performing characteristic matching of the infrared image characteristic descriptor sequence and the visible light image characteristic descriptor sequence according to a preset characteristic matching rule to obtain a projection transformation matrix; and outputting the projective transformation matrix to finish registration.

Optionally, the preset preprocessing rule includes: smoothing, calculating image gradient information, suppressing non-maximum value of image gradient amplitude, detecting and connecting edge contours.

Optionally, the preset feature extraction algorithm is designed based on a KAZE algorithm.

Optionally, the processing the preprocessed infrared image and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm includes: constructing a nonlinear scale space by using a nonlinear diffusion equation; wherein the nonlinear diffusion equation is:

wherein L represents the brightness of the image; t represents a scale parameter; div represents the divergence;represents a gradient; c (x, y, t) represents the conduction function at coordinate (x, y);representing the original image L passing Gaussian in the delta scaleA filtered gradient; for the preprocessed infrared image and the preprocessed visible light image, extracting image characteristic points of the infrared image and image characteristic points of the visible light image in the nonlinear scale space by calculating Hessian matrix values of pixel points; respectively calculating the main directions of the image characteristic points of the infrared image and the image characteristic points of the visible light image; respectively aiming at the infrared image and the visible light image, selecting a rectangular area with a preset size by taking an image characteristic point as a center, dividing sub-areas with a preset number, and performing weighted calculation on first-order differentials of pixel points in the sub-areas by using a Gaussian core to obtain descriptor vectors of the sub-areas; and performing weighted calculation and normalization processing on the descriptor vectors of all the sub-regions by using a Gaussian window to obtain an infrared image feature descriptor sequence and a visible light image feature descriptor sequence.

Optionally, the extracting image feature points of the infrared image and image feature points of the visible light image by calculating a Hessian matrix value of a pixel point in the nonlinear scale space includes: respectively aiming at the infrared image and the visible light image, comparing the Hessian matrix value of each pixel point with the Hessian matrix values of all the pixel points of the current layer and the two adjacent layers of the current layer; and if the Hessian matrix value of the current pixel point is greater than or less than the Hessian matrix values of all the pixel points, determining that the Hessian matrix value extreme point of the corresponding pixel point is the image feature point.

Optionally, the calculating the main directions of the image feature points of the infrared image and the image feature points of the visible light image respectively includes: selecting a circular area with a preset radius by taking the image characteristic point as a central point, and calculating first-order differential of pixel points in the area; rotating the sector area with a preset opening angle and step length around the central point, and accumulating the first order differential in the rotating process; and screening out the direction corresponding to the maximum accumulated value in the accumulation process as the main direction.

Optionally, the performing feature matching of the infrared image feature descriptor sequence and the visible light image feature descriptor sequence according to a preset feature matching rule to obtain a projection transformation matrix includes: according to the infrared image feature descriptor sequence and the visible light image feature descriptor sequence, performing feature point matching on the infrared image to the visible light image by using a KNN algorithm to obtain a first matching image feature point pair set; according to the infrared image feature descriptor sequence and the visible light image feature descriptor sequence, performing feature point matching on the visible light image to the infrared image by using a FLANN algorithm to obtain a second matching image feature point pair set; and comparing the first matching image characteristic point pair set with the second matching image characteristic point pair set, screening out the same matching image characteristic point pairs, and integrating the same matching image characteristic point pairs as a final matching image characteristic point pair set to serve as the projection transformation matrix.

Optionally, the performing feature point matching on the infrared image to the visible light image by using the KNN algorithm to obtain a first matching image feature point pair set includes: taking any image feature point of the infrared image feature descriptor sequence as an object, traversing all image feature points in the visible light image feature descriptor sequence by using an Euclidean distance definitional mode, screening a definition value minimum value and a definition value secondary minimum value, and taking the definition value minimum value and the definition value secondary minimum value as a nearest neighbor image feature point and a next nearest neighbor image feature point respectively; judging whether the nearest neighbor image feature point and the next nearest neighbor image feature point meet a preset relationship, and if the nearest neighbor image feature point and the next nearest neighbor image feature point meet the preset relationship, judging that the nearest neighbor image feature point is a matched image feature point of image feature points in a preselected infrared image feature descriptor sequence; wherein the preset relationship is as follows:

wherein d (U)_i,V_1j) The image feature points are nearest neighbor image feature points; d (U)_i,V_2j) The image feature points are the next nearest neighbor image feature points; t is_h1Is a preset relationship threshold; traversing all image feature points in the infrared image feature descriptor sequence to obtain matched image feature points corresponding to all the image feature points; integrating each image feature point in the infrared image feature descriptor sequence and the corresponding matching image feature point, wherein each corresponding relation is used as oneAnd matching the image characteristic point pair set to obtain the first characteristic matching image characteristic point pair set.

A second aspect of the present invention provides an optimized registration system for a heterogeneous image for registration between an infrared image and a visible image, the system comprising: the acquisition unit is used for acquiring infrared image information to be registered and corresponding visible light image information; a processing unit to: preprocessing the infrared image information and the visible light image information according to a preset preprocessing rule; processing the preprocessed infrared image and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm; performing characteristic matching of the infrared image characteristic descriptor sequence and the visible light image characteristic descriptor sequence according to a preset characteristic matching rule to obtain a projection transformation matrix; and the output unit is used for outputting the projection transformation matrix to finish the registration.

In another aspect, the present invention provides a computer-readable storage medium having stored thereon instructions, which when executed on a computer, cause the computer to perform the above-mentioned method for optimized registration of heterogeneous images.

According to the technical scheme, the image feature point matching method comprises three stages of image preprocessing, image feature point extraction and descriptor sequence generation and image feature point matching. And extracting and comparing the characteristic points of the infrared image and the visible light image, and registering the infrared image and the visible light image according to the matching relation of the characteristic points. Compared with the traditional SIFT algorithm and the traditional SURF algorithm, the scheme of the invention has the advantage of higher image registration accuracy.

Additional features and advantages of embodiments of the invention will be set forth in the detailed description which follows.

Drawings

The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the embodiments of the invention without limiting the embodiments of the invention. In the drawings:

FIG. 1 is a flowchart illustrating steps of a method for optimized registration of heterogeneous images according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating the steps of obtaining a feature descriptor sequence according to an embodiment of the present invention;

FIG. 3 is a flowchart of the steps provided by one embodiment of the present invention to obtain a final matching image feature point pair set;

fig. 4 is a system configuration diagram of an optimized registration system for a heterogeneous image according to an embodiment of the present invention.

Description of the reference numerals

10-an acquisition unit; 20-a processing unit; 30-output unit.

Detailed Description

The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.

Fig. 4 is a system configuration diagram of an optimized registration system for a heterogeneous image according to an embodiment of the present invention. As shown in fig. 4, the embodiment of the present invention provides an optimized registration system for heterogeneous images, the system comprising: the acquisition unit is used for acquiring infrared image information to be registered and corresponding visible light image information; a processing unit to: preprocessing the infrared image information and the visible light image information according to a preset preprocessing rule; processing the preprocessed infrared image and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm; performing characteristic matching of the infrared image characteristic descriptor sequence and the visible light image characteristic descriptor sequence according to a preset characteristic matching rule to obtain a projection transformation matrix; and the output unit is used for outputting the projection transformation matrix to finish the registration.

Fig. 1 is a flowchart of a method for optimizing registration of a heterogeneous image according to an embodiment of the present invention. As shown in fig. 1, an embodiment of the present invention provides a method for optimized registration of heterogeneous images, the method including:

step S10: and acquiring infrared image information to be registered and corresponding visible light image information.

Specifically, since the infrared sensor and the visible light sensor cannot be in the same position and viewing angle when shooting the same region at the same time, a certain viewing angle difference exists between the infrared sensor and the visible light sensor, and thus geometric distortion of the image is caused. The optimal registration method of the heterogeneous images provided by the invention aims at the registration between the infrared images and the visible light images under the condition of image distortion. Aiming at the same scene, the infrared image and the visible light image have certain difference, and the difference correction and the corresponding registration between the infrared image and the visible light image are carried out, which is the technical problem to be solved by the scheme of the invention. Before registration, an acquisition unit is required to acquire an infrared image to be registered and a corresponding visible light image, the corresponding relation of the infrared image to be registered and the corresponding visible light image is image information in the same scene, a certain difference can exist between shooting time and shooting angle, but the image information of two completely different scenes has no registration value. The acquisition unit is connected with the infrared image acquisition device and the visible light image acquisition device or is connected with a storage device which stores the infrared image and the visible light image, the infrared image and/or the visible light image are correspondingly acquired through the connected devices, and then the acquired infrared image and the acquired visible light image are sent to the processing unit for the subsequent registration of the processing unit.

Step S20: and preprocessing the infrared image information and the visible light image information according to a preset preprocessing rule.

Specifically, the acquired infrared image and the acquired visible light image are original images, and the original images have much interference information, and if the original images are directly used for subsequent registration operation, the results are greatly influenced. And the assumed configuration algorithm needs the adaptive input image, so that not only can the result interference be effectively avoided, but also the subsequent registration effect can be improved. Therefore, after the original images of the infrared image and the visible light image are obtained, the original images are preprocessed based on the preset preprocessing rule, the original images are processed into the adaptive input images of the subsequent algorithm, and the processing efficiency is improved on the premise of ensuring the accuracy of the processing result. Preferably, the preset preprocessing rule includes smoothing processing, calculating image gradient information, suppressing non-maximum value of image gradient amplitude, detecting and connecting edge contour. Smoothing the infrared image and the visible light image by using a Gaussian filter; then, calculating the amplitude and the direction of the image gradient by using the finite difference of the first-order partial derivatives; carrying out non-maximum suppression on the image gradient amplitude; finally, the edge contour is detected and connected by a double-threshold method. And obtaining an edge contour image, and taking the edge contour image as an input image of a subsequent processing algorithm.

Step S30: and processing the preprocessed infrared image information and the preprocessed visible light image into a corresponding infrared image feature descriptor sequence and a corresponding visible light image feature descriptor sequence according to a preset feature extraction algorithm.

Specifically, the preset feature extraction algorithm of the scheme of the invention is designed based on a KAZE algorithm, the principle is that an optimized KAZE algorithm is used for extracting image feature points at a descriptor stage and generating a 32-dimensional image feature point descriptor sequence, the optimized KAZE algorithm is used for extracting the image feature points at the image feature point extraction stage by using an extreme value principle, if only a maximum value or a minimum value is used, the registration effect between homologous images is remarkable, but the registration effect between an infrared image and a visible light image is poor, the extracted image feature points are fewer, and the subsequent image feature point matching is not facilitated, so the image feature points are extracted by using the extreme value principle. Specifically, as shown in fig. 2, the method includes the following steps:

step S301: and constructing a nonlinear scale space.

Specifically, the optimized KAZE algorithm takes the change of image brightness on different scales as the divergence of a certain flow function, and utilizes a nonlinear diffusion equation to perform diffusion filtering on the image to construct a nonlinear scale space; wherein the nonlinear diffusion equation is:

wherein L represents the brightness of the image; t represents a scale parameter; div represents the divergence;represents a gradient; c (x, y, t) represents a transfer function at coordinates (x, y) that depends on the local image difference structure and the gradient that reduces the local edge spread of the image;representing the gradient of the original image L after Gaussian filtering on a delta scale; wherein, the g function formula is as follows:

where k is the contrast factor, the value of which is the gradient imageThe values on the 70% percentile of the histogram. After implicit difference is carried out on the nonlinear diffusion equation, an Additive Operator Splitting Algorithm (AOS) is adopted to construct a linear scale space, and a solution formula of the equation is obtained;

wherein I represents a unit matrix, ti represents evolution time, Al represents a three-diagonal matrix with dominant diagonal, and Li represents the I-th layer image brightness of the nonlinear scale space.

Step S302: and extracting image feature points.

Specifically, image feature points of the infrared image information and the visible light image are extracted by calculating a Hessian matrix value of a pixel point in the nonlinear scale space. The Hessian matrix value of each pixel point and the Hessian moment of all the pixel points of the current layer and the two adjacent layers of the current layerComparing the array values; and if the Hessian matrix value of the current pixel point is larger than or smaller than the Hessian matrix values of all the pixel points, judging the extreme value point of the Hessian matrix value of the corresponding pixel point as the image characteristic point. For example, the Hessian matrix value of each pixel point is compared with the delta of the current layer i, the upper layer i +1 and the lower layer i-1_i×δ_iThe Hessian matrix values of 26 pixel points in the rectangular window are compared, if the Hessian matrix value of the pixel point is larger than or smaller than the Hessian matrix value of the 26 pixel points, the Hessian matrix value of the pixel point is an extreme value point, and the extreme value point is an image feature point.

Step S303: and calculating the main direction of the image characteristic point.

Specifically, in order to obtain a descriptor sequence of the rotation invariant image feature points, the main direction of the image feature points needs to be calculated, and the calculation method is to select the radius of 6 δ by taking the image feature points as the central points_iThe first order differential L is calculated for the pixel points in the area_x、L_yThen, the sector area with the opening angle of 60 degrees and the step length of 0.15rad rotates around the image characteristic point, and the first order differential L is accumulated in the rotating process_x、L_yWherein the direction to which the accumulated value maximally corresponds is the main direction.

Step S304: a plurality of sub-regions are divided and descriptor vectors for the sub-regions are obtained.

Specifically, the size of the selected image feature point is 24 delta by taking the image feature point as a center_i×24δ_iAnd is divided into 8 sub-areas with equal size, and the adjacent area has 4 delta_iUsing Gaussian kernel to perform weighted calculation on first-order differential of pixel points in the sub-region, thereby obtaining a descriptor vector d_νDescriptor vector d_νThe calculation formula is as follows:

d_ν＝(∑L_x,∑L_y,∑|L_x|,∑|L_y|)

wherein d is_νRepresenting a descriptor vector, L_xRepresenting the first differential, L, of the image in the x-direction_yRepresents the first differential of the image in the y-direction, | L_xI represents the first order of the image in the x directionAbsolute value of differential, | L_y| represents the absolute value of the first order differential of the image in the y direction.

Step S305: a sequence of image feature descriptors is obtained.

Specifically, a gaussian window is used for carrying out weighting calculation and normalization processing on descriptor vectors of all sub-regions, and an infrared image feature descriptor sequence and a visible light image feature descriptor sequence are obtained. For example, 8 subregion descriptor vectors d using a Gaussian window_νAnd further carrying out weighting calculation and normalization processing to obtain a 32-dimensional image feature point descriptor sequence.

Step S40: and performing characteristic matching on the infrared image characteristic descriptor sequence and the visible light image characteristic descriptor sequence according to a preset characteristic matching rule to obtain a projection transformation matrix.

Specifically, the optimized image registration method uses an optimized distance similarity measurement method combining a KNN algorithm and a FLANN algorithm in a matching stage, and comprises the following three steps of 1) using the KNN algorithm to match feature points of an infrared image to a visible light image to obtain a first matched image feature point pair set; 2) performing characteristic point matching on the visible light image to the infrared image by using a FLANN algorithm to obtain a second matching image characteristic point pair set; 3) and comparing the first matching characteristic point pair set with the second matching characteristic point pair set to obtain a final matching image characteristic point pair set. Specifically, as shown in fig. 3, the method includes the following steps:

step S401: a first set of matching pairs of feature points is obtained.

Specifically, after the 32-dimensional image feature point descriptor sequence parameter in step S305 is obtained, the image feature point descriptor sequence parameter in the infrared image is set to U_i(x₁,x₂,x₃,……,x₃₀,x₃₁,x₃₂) The sequence parameter of the image feature point descriptor in the visible light image is V_j(y₁,y₂,y₃,……,y₃₀,y₃₁，y₃₂). Assuming that the image feature point set in the infrared image is U and the image feature point set in the visible light image is V, then taking the image feature point U in the image feature point set U_iAnd traversing all the image feature points in the image feature point set V by using the Euclidean distance definition mode as an object. Wherein, the Euclidean distance is defined as:

wherein x is_mElements representing parameters of a sequence of image feature point descriptors in an infrared image, y_mAnd elements representing image feature point descriptor sequence parameters in the visible light image. To obtain d (U)_i,V_j) Minimum value of d (U)_i,V_1j) And the next minimum value d (U)_i,V_2j) I.e. nearest neighbor image feature points V_1jAnd the next nearest neighbor image feature point V_2jIf d (U)_i,V_j) Minimum value of d (U)_i,V_1j) And the next minimum value d (U)_i,V_2j) Satisfy the Euclidean distance comparison formula. Wherein, the Euclidean distance comparison formula is as follows:

wherein d (U)_i,V_1j) The image feature points are nearest neighbor image feature points; d (U)_i,V_2j) The image feature points are the next nearest neighbor image feature points; t is_h1The threshold value is preferably set to 0.80 for the preset relationship. The image feature points V in the image feature point set V_1jFor the image feature points U in the image feature point set U_iTraversing all the image feature points in the image feature point set U to obtain a first matched image feature point pair set.

Step S402: a second set of matched pairs of feature points is obtained.

Specifically, with an image feature point set V in the visible light image as an object, all image feature points in an image feature point set U are traversed to obtain d (V)_j,U_i) Minimum value of d_min(V_j,U_i) I.e. nearest neighbor image feature points U_iTaking all objects in the image feature point set V in the visible light image to obtain d (V)_j,U_i) Minimum value of d_min(V_j,U_i) Set of (K { d) }_min(V_j,U_i) H, computing the set K { d }_min(V_j,U_i) Minimum value in (f) is d_MIN(V_j,U_i) (ii) a Wherein the minimum value d_MIN(V_j,U_i) The calculation formula of (A) is as follows:

d_min(V_j,U_i)<(2.0*d_MIN(V_j,U_i))＝(2.0*MIN(K{d_min(V_j,U_i)}))

wherein d is_MIN(V_j,U_i) Denotes all d_min(V_j,U_i) MIN represents the minimum value of the computation set, K { } represents the set, d represents the minimum value of the computation set_min(V_j,U_i) Representing image features V in a set V of image feature points_jAnd nearest neighbor image feature points U in the image feature point set U_iThe distance similarity metric of (2). If d of the matched pair of image feature points_min(V_j,U_i) If the minimum value of (a) satisfies the preset relationship, the minimum value of (b) is included in the second matching image feature point pair set. Wherein, the preset relationship is as follows:

d_min(V_j,U_i)<(2.0*d_MIN(V_j,U_i))

wherein d is_min(V_j,U_i) Representing image feature points V in a set of image feature points V_jAnd nearest neighbor image feature points U in the image feature point set U_iA distance similarity measure of d_MIN(V_j,U_i) Denotes all d_min(V_j,U_i) The minimum value of the set of (a).

Step S403: and obtaining a final matching image feature point pair set.

Specifically, the first matching image feature point pair set and the second matching image feature point pair set are compared, the same matching image feature point pairs are screened out, and the matching image feature point pairs are integrated to be used as a final matching image feature point pair set and used as a projection transformation matrix.

Step S50: and outputting the projective transformation matrix to finish registration.

Specifically, the final matching image feature point pair set is output as an input of subsequent image processing, and the subsequent image processing is performed. Through the front-end registration of the scheme of the invention, the projective transformation matrix of the infrared image and the visible light image is obtained, and the subsequent processing algorithm is matched based on the projective transformation matrix, so that the accuracy and the efficiency can be greatly improved.

In the embodiment of the invention, an optimized image registration method is provided for the problem of low registration accuracy between the infrared image and the visible light image. The optimized image registration method is processed in three stages of an image preprocessing stage, an image feature point extraction and descriptor sequence generation stage and an image feature point matching stage, and has the advantage of higher image registration accuracy compared with the traditional SIFT algorithm and the traditional SURF algorithm.

In which the number of the first and second groups is reduced,

the touch control sensor is fixed on the touch control screen to be detected and used for collecting touch signals, and a plurality of touch control sensors are uniformly distributed on the touch control screen to be detected. In one possible implementation manner, in order to compare and optimize the operating efficiency of the image registration method, the simulation experiment selects seven statistical parameters, namely, the number of feature points, the matching number of feature points, the correct matching number of feature points, the registration accuracy of feature points, the image descriptor stage consumption time, the image feature point matching stage consumption time, and the total image registration consumption time, as evaluation indexes, wherein the calculation formula of the registration accuracy of image feature points is as follows:

wherein, P_mImage registration accuracy, M, representing an optimized image registration method_{Novel registration method}Number of image feature point matches, N, representing an optimized image registration method_{Novel registration method}The number of correct matches of image feature points representing an optimized image registration method. The evaluation index statistical data are shown in table 1. Tong (Chinese character of 'tong')The comparative analysis of the result graph and the evaluation index statistical data shows that: 1) the registration effect of the traditional SIFT algorithm and the traditional SURF algorithm on the heterogeneous images is not ideal, and a large number of mismatching pairs occur; 2) the average registration accuracy of the optimized image registration method is 91.62%, but the time consumption is long, and the method has obvious superiority in application scenes such as image splicing and the like, which have high registration accuracy requirements but do not pursue registration real-time performance.

TABLE 1SIFT Algorithm, SURF Algorithm, and statistical data comparison table for optimized image registration method

Embodiments of the present invention also provide a computer-readable storage medium having instructions stored thereon, which when executed on a computer, cause the computer to perform the above-mentioned method for optimized registration of heterogeneous images.

Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be implemented by a program, which is stored in a storage medium and includes several instructions to enable a single chip, a chip, or a processor (processor) to execute all or part of the steps in the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

While the embodiments of the present invention have been described in detail with reference to the accompanying drawings, the embodiments of the present invention are not limited to the details of the above embodiments, and various simple modifications can be made to the technical solution of the embodiments of the present invention within the technical idea of the embodiments of the present invention, and the simple modifications are within the scope of the embodiments of the present invention. It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, the embodiments of the present invention will not be described separately for the various possible combinations.

In addition, any combination of the various embodiments of the present invention is also possible, and the same should be considered as disclosed in the embodiments of the present invention as long as it does not depart from the spirit of the embodiments of the present invention.