阅读说明：本技术 一种无人机非制冷型热像仪测温结果漂移去除方法 (Method for removing drift of temperature measurement result of non-refrigeration thermal imager of unmanned aerial vehicle ) 是由 周纪 王子卫 孟令宣 马晋 丁利荣 张旭 于 2021-08-26 设计创作，主要内容包括：本发明公开了一种无人机非制冷型热像仪测温结果漂移去除方法,属于无人机热红外遥感技术领域。本发明包括：提取目标范围的采集图像并从中选取一景图像作为参考图像,计算所提取图像的DN值频数分布直方图,求得其“代表DN值”,以及计算各图像与参考图像的“代表DN值”之差；从图像中取出该差值以获取初步去除温度漂移之后的热红外图像,并保存为指定格式；利用拼图软件拼接各景校正后的图像并进行波段运算,获取飞行目标区域的热红外亮温图像,结合地面仪器的亮温观测数据和辐射传输模拟,可实现温度漂移的完全去除。本发明克服了传统方法野外布设多个参考温度板进行温度漂移粗校正的缺陷,极大地提高了人员野外作业效率。(The invention discloses a method for removing drift of temperature measurement results of a non-refrigeration thermal imager of an unmanned aerial vehicle, and belongs to the technical field of thermal infrared remote sensing of the unmanned aerial vehicle. The invention comprises the following steps: extracting the collected images in the target range, selecting a scene image as a reference image, calculating a DN value frequency distribution histogram of the extracted images, obtaining a representative DN value of the histogram, and calculating the difference between the representative DN values of the images and the reference image; taking out the difference value from the image to obtain a thermal infrared image after the temperature drift is preliminarily removed, and storing the thermal infrared image in a specified format; splicing the images after each scene is corrected by using jigsaw software, carrying out band operation to obtain a thermal infrared bright temperature image of a flight target area, and combining bright temperature observation data and radiation transmission simulation of a ground instrument to realize complete removal of temperature drift. The invention overcomes the defect that a plurality of reference temperature plates are arranged in the field to carry out rough temperature drift correction in the traditional method, and greatly improves the field operation efficiency of personnel.)

1. A method for removing drift of temperature measurement results of an unmanned aerial vehicle non-refrigeration thermal imager is characterized by comprising the following steps:

step 1: extracting an acquisition image sequence of a thermal infrared image of the unmanned aerial vehicle in a flight target area;

step 2: carrying out primary correction processing on the acquired image sequence:

acquiring a DN value frequency distribution histogram of each image in the acquired image sequence according to a designated DN value threshold; searching an interval corresponding to a bar graph with the highest DN value frequency distribution histogram, and taking the average DN value of all pixels in the interval as a 'representative DN value' of each image;

selecting a reference image from the extracted collected image sequence, calculating the difference value of 'representing DN value' between each image and the reference image for all non-reference images of the collected image sequence, and taking the difference value as the temperature drift value between each image and the reference image; quantizing the temperature drift value between the image and the reference image by DN value;

for all non-reference images, subtracting the temperature drift value from the DN value matrix of each image to obtain a corrected DN value matrix of each image, and directly taking the DN value matrix of the reference image as the corrected DN value matrix;

extracting exchangeable image file format data of the collected image sequence, and obtaining an image sequence after primary correction processing based on the DN value matrix after each image is corrected;

and step 3: splicing the preliminarily corrected images based on jigsaw software, carrying out band operation processing, and converting the corrected DN value into a bright temperature value to obtain a complete thermal infrared bright temperature image of the unmanned aerial vehicle in a flight target area;

and 4, step 4: after ground actual measurement temperature data are utilized and the flight height of the unmanned aerial vehicle is simulated, secondary correction processing is carried out on the spliced complete thermal infrared image to remove the temperature drift value of the reference image, and therefore the complete thermal infrared bright temperature image of the flight target area after the temperature drift is fully removed is obtained.

2. The method of claim 1 wherein in step 2, the DN value threshold is a DN value that characterizes 0.5 °.

3. The method according to claim 1, wherein in step 2, the reference image is the image containing the most target task objects.

4. A method according to any one of claims 1 to 3, wherein in step 3, the band calculation process is: t is_ba.DN + b, where T_bAnd a and b represent conversion coefficients of DN values and the brightness temperatures provided by a thermal imager manufacturer, a represents a conversion gain value, and b represents a conversion offset.

Technical Field

The invention relates to the field of thermal infrared remote sensing of unmanned aerial vehicles, in particular to a method for removing drift of temperature measurement results of a non-refrigeration thermal imager of an unmanned aerial vehicle.

Background

Unmanned aerial vehicle thermal infrared remote sensing is an important technical means for acquiring earth surface temperature data with high time and high spatial resolution. The high-precision ground surface temperature data can provide powerful data support for the application fields of ground surface evapotranspiration, crop water stress monitoring, crop yield estimation and the like.

Due to the limitation of the conditions such as the load capacity, energy consumption, operation cost and the like of the unmanned aerial vehicle, various unmanned aerial vehicle thermal imagers widely used at present are basically non-refrigeration thermal imagers, namely, the bodies of the thermal imagers cannot maintain a stable temperature in the flight process; but a stable instrument temperature is important to ultimately obtain an accurate surface temperature. However, in an actual field operation task, the thermal imager of the unmanned aerial vehicle is often influenced by factors such as wind, illumination, ambient temperature and the like in flight, so that the temperature of the body of the thermal imager of the unmanned aerial vehicle is changed; even if most thermal imagers have an automatic correction function, the correction capability is limited, temperature measurement result drift caused by external condition change cannot be well eliminated (abnormal brightness change of spliced temperature images can be caused visually), and the obtained brightness and temperature data have large errors, so that accurate earth surface temperature cannot be obtained, and further application of the data is restricted.

Disclosure of Invention

The invention aims to: aiming at the problem of drift of the temperature measurement result of the existing non-refrigeration thermal imager of the unmanned aerial vehicle, a drift removal method is provided, so that reliable bright temperature data can be obtained.

The invention provides a method for removing drift of temperature measurement results of an unmanned aerial vehicle non-refrigeration thermal imager, which comprises the following steps:

step 1: extracting an acquisition image sequence of a thermal infrared image of the unmanned aerial vehicle in a flight target area;

step 2: carrying out primary correction processing on the acquired image sequence:

acquiring a DN (digital number) value frequency distribution histogram of each image in an acquired image sequence according to a specified DN (digital value, namely pixel value) threshold; searching an interval corresponding to a bar graph with the highest DN value frequency distribution histogram, and taking the average DN value of all pixels in the interval (namely the pixels corresponding to the interval end points in the interval and the interval end points in the interval) as a 'representative DN value' of each image;

selecting a reference image from the extracted collected image sequence, calculating the difference value of 'representing DN value' between each image and the reference image for all non-reference images of the collected image sequence, and taking the difference value as the temperature drift value between each image and the reference image; quantizing the temperature drift value between the image and the reference image by DN value;

for all non-reference images, subtracting the temperature drift value from the DN value matrix of each image to obtain a corrected DN value matrix of each image, and directly taking the DN value matrix of the reference image as the corrected DN value matrix, thereby normalizing the temperature drift level of each image to the level of the reference image so that each image and the reference image have the same temperature drift level;

extracting EXIF (exchangeable image file format) data of the collected image sequence, and obtaining an image sequence after primary correction processing based on DN value matrixes after correction of all images, namely obtaining a thermal infrared image sequence after primary correction, wherein EXIF data of an original shooting device is reserved;

and step 3: splicing the images after the preliminary correction based on jigsaw software, carrying out band operation processing, converting the DN value after the correction into a bright temperature value, and obtaining a complete thermal infrared bright temperature image of the flying target area of the unmanned aerial vehicle in the flying target area;

and 4, step 4: after ground actual measurement temperature data are utilized and the flight height of the unmanned aerial vehicle is simulated, secondary correction processing is carried out on the spliced complete thermal infrared image to remove the temperature drift value of the reference image, and therefore the complete thermal infrared bright temperature image of the flight target area after the temperature drift is fully removed is obtained.

Further, in step 3, the band operation processing is as follows: t is_ba.DN + b, where T_bA matrix representing the bright temperature of the image, a and b representing the conversion coefficient of the DN value and the bright temperature provided by the thermal imager manufacturer, a representing the gain value (gain), and b representing the offset (offset).

The technical scheme provided by the invention at least has the following beneficial effects:

the invention can solve the problems of abnormal brightness and temperature and unnatural image brightness change caused by the drift of the temperature measurement result of the non-refrigeration thermal imager of the unmanned aerial vehicle, thereby providing guarantee for further application of thermal infrared data of the unmanned aerial vehicle.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flow diagram of a method for removing drift of a temperature measurement result of a non-refrigeration thermal imager for an unmanned aerial vehicle according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a part of adjacent thermal infrared images, which are obtained by performing SIFT (Scale-inverse Feature Transform) Feature matching on the part of adjacent thermal infrared images, searching for corresponding Feature points, and performing continuous linear fitting to correct a result, (a) - (d) show adjacent image pairs obtained at different times;

fig. 3 shows SIFT feature point matching connection results of a partial adjacent image pair provided in the embodiment of the present invention, where (a) - (b) represent adjacent image pairs acquired at different times;

fig. 4 is a comparison result of continuous linear relative normalization of thermal infrared data of a flight zone of a full-station flight target area in 13 days of 7 and 2020 in accordance with an embodiment of the present invention, wherein (a) is a graph of a stitching result of an unprocessed thermal infrared image; (b) the method is a splicing result graph after continuous linear relative normalization processing;

fig. 5 is a schematic diagram of results of finding corresponding feature points and performing linear fitting on part of adjacent thermal infrared images through SIFT feature matching, where the adjacent thermal infrared images are provided by the embodiment of the present invention, and (a) - (d) represent adjacent image pairs obtained at different times;

fig. 6 is a scatter plot line graph of a first-order coefficient obtained by SIFT feature matching and finding corresponding feature points and performing linear fitting on an adjacent image pair in one flight strip according to the embodiment of the present invention;

fig. 7 is a comparison result of continuous noise-removing relative normalization of thermal infrared data of a certain flight zone of a full-station flight target area in 13 days of 2020 and 7 months, provided by an embodiment of the present invention, wherein (a) is a graph of a stitching result of thermal infrared images without processing; (b) the method comprises the steps of obtaining a splicing result graph after continuous additive noise removal relative normalization processing;

fig. 8 is a DN value frequency distribution histogram of a partial thermal infrared image according to an embodiment of the present invention, where (a) - (d) represent thermal infrared images acquired at different times;

fig. 9 is a schematic diagram of a part of thermal infrared images "representing DN values" provided by an embodiment of the present invention, wherein (a) - (d) represent thermal infrared images acquired at different times;

fig. 10 is a comparison graph of the effect of removing the temperature drift of the thermal infrared data of the complete flight target area in the full station in 13 days of 2020 and 7 months, wherein (a) is a graph of the stitching result of the thermal infrared images without being processed; (b) the splicing result graph after the temperature drift is removed is obtained;

fig. 11 is a comparison graph of the effect of removing the thermal infrared data temperature drift of the complete flight target region of the 14-Rizhai Zi substation in 2020, in which (a) is a graph of the stitching result of unprocessed thermal infrared images; (b) the splicing result graph after the temperature drift is removed is obtained;

fig. 12 is a comparison graph of the effect of removing the thermal infrared data of the complete flight target area of the wetland station in 14 days 7 and 14 months 2020 in the embodiment of the present invention, wherein (a) is a graph of the stitching result of an unprocessed thermal infrared image; (b) and (4) a splicing result graph after the temperature drift is removed.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Referring to fig. 1, the method for removing temperature drift of observed data of an unmanned aerial vehicle non-refrigeration thermal imager provided by the embodiment of the invention comprises seven parts: removing redundant images, selecting reference images, calculating DN value frequency distribution histogram, calculating 'representing DN value', generating new thermal infrared images, splicing images and calculating wave bands, and removing the temperature drift value of the reference images.

Currently, temperature drift of observation data of an unmanned aerial vehicle non-refrigeration thermal imager usually causes thermal infrared images acquired at different moments to show abnormal light and shade changes; the embodiment of the invention determines the change mode of the temperature drift by analyzing the homonymy points between every two adjacent thermal infrared images, as shown in formula (1). In this embodiment, in order to simplify the processing, when selecting the reference image, the 1 st image is directly used as the selected reference image, but may be another image. As shown in fig. 2 (in fig. 2, the diagonal line corresponding to "y ═ x" is located above the "fitting function line"), it is found that there is a significant linear relationship between DN values corresponding to the same-name points of adjacent thermal infrared images, and therefore, the formula (1) can be expressed as the formula (2).

In the formula, i represents an image number,representing a DN value matrix of the ith thermal infrared image after temperature drift correction; f. of_i() Representing a correction function between the ith thermal infrared image and the (i-1) th thermal infrared image;representing a DN value matrix of the i-1 st thermal infrared image after temperature drift correction; DN₁A matrix of DN values representing the 1 st picture (i.e., the selected reference picture).

In the formula, k_iA first-order coefficient representing a linear correction function between the ith thermal infrared image and the (i-1) th thermal infrared image; b_iAnd a constant term representing a linear correction function between the ith thermal infrared image and the (i-1) th thermal infrared image.

As shown in fig. 3, the same-name points between the overlapping areas of adjacent thermal infrared images are searched through SIFT feature matching, and the matched same-name points are connected through line segments, so that the matching accuracy of SIFT on the thermal infrared images is very high, and obvious matching errors are few; therefore, the result of finding the same-name point using this method is reliable.

As shown in fig. 4, a result of continuous linear relative normalization (correction) of thermal infrared data of a certain flight zone of a full-station flight target area in 13 days of 7 and 7 months in 2020 is shown (i.e., correction is performed by using the formula (2)); according to the result, the spliced images have larger deviation if the acquired images are directly and continuously processed; the present invention will further derive an improvement.

As shown in fig. 5, it is shown that a Scale-invariant feature transform (SIFT-invariant feature transform) feature matching is used to find the homologous points between the overlapped regions of adjacent thermal infrared images that are not subjected to continuous linear normalization, and in fig. 5, the oblique line corresponding to "y ═ x" is located below the "fitting function line"; and linear fitting is performed on the relationship of DN values (using formula (3)), and a first term of the linear fitting is found to be tens of points close to 1; as shown in fig. 6, a first-order coefficient obtained by finding corresponding feature points through SIFT feature matching and performing linear fitting on adjacent image pairs in one flight zone is counted, and is drawn into a scatter-point line graph, and the average value of the scatter-point line graph is 0.9985, the standard deviation of the scatter-point line graph is 0.0152, and the scatter-point line graph is actually distributed near 1. Therefore, the present invention further treats temperature drift as additive noise, and the linear relationship can thus be converted into a constant difference relationship, as shown in equation (4).

DN_m＝k_m·DN_n+b_m (3)

In the formula, DN_m、DN_nRepresenting DN value matrixes of any two adjacent thermal infrared images; k is a radical of_mRepresenting a linear fitting primary term coefficient of the DN values of the homonymous points between the two; b_mConstant terms representing the fitting results.

DN_m＝DN_n+b_m (4)

As shown in fig. 7, for thermal infrared data of a certain flight zone in a full-station flight target area in 7/13/2020, it can be found that due to continuous accumulation of fitting errors, the spliced image also has a phenomenon of an excessively large head-to-tail light and shade difference, by regarding temperature drift as a result of relative normalization after additive noise (i.e., performing correction in the manner of formula (4)). Therefore, by combining the characteristics of the flight operation of the unmanned aerial vehicle, the invention controls the error transmission by calculating the DN value frequency distribution histogram to obtain the 'representative DN value', thereby realizing the removal of the temperature drift, and the method comprises the following specific steps:

and step S1, extracting an acquired image sequence (redundant image elimination) of the unmanned aerial vehicle in the flight target area.

Due to the fact that the battery endurance time of the unmanned aerial vehicle is limited, the unmanned aerial vehicle returns to a flying starting point after flying for a certain time to replace the battery, a thermal imager can shoot a large number of useless pictures on the way of going and on the ground, and the pictures can affect the execution efficiency of the method and the quality of image splicing. Such redundant images need to be excluded by using jpg format images recorded simultaneously by the thermal imager before processing. That is, for an acquired image sequence which completes a certain acquisition task, irrelevant images (an image shot by the unmanned aerial vehicle on a path from a flying point to a starting point of a flight task and an image shot by the unmanned aerial vehicle on a path from the flying point to the flying task and returning the flight task to the flying point) are removed, and a required image sequence is reserved.

Step S2, selecting a reference image: the acquisition time of each scene image shot by the unmanned aerial vehicle is different, so that the selection of the reference image (reference image) relates to the corresponding moment of the final spliced image; if there is no specific requirement on the acquisition time of the final stitched image, it is not necessary to select an image at a specific time as a reference image, and an image containing the most target task objects is usually selected. For example, in the case where the target task is a field crop, it is recommended to select, as a reference image, an image including a large number of main features (such as farmlands, bare lands, and the like) in the flight target area.

Step S3, calculating DN value frequency distribution histogram: setting the representative ground object temperature fluctuation range of the flight target area as the interval of the histogram; in the embodiment of the invention, the default 20DN value (corresponding to 0.5 ℃) is realized, and in the actual implementation process, the interval can be increased appropriately according to the actual situation of the flight area. And calculating a DN value frequency distribution histogram of the reference image according to the selected interval.

Step S4, "representing DN value" calculation: the interval with the most distributed pixel number in the reference image DN value frequency distribution histogram is used as a reference interval, and the lower limit value isThe upper limit isCalculating DN value in interval in reference imageAverage value of all pixels in the image, as shown in equation (5):

in the formula (I), the compound is shown in the specification,"representing DN value" for a reference picture; n is₀Indicating that the reference image is located in a sectionThe number of pixels in the pixel array; x is an accumulation variable.

According to the way of steps S3 and S4, frequency distribution histograms of DN values of the remaining images to be processed are sequentially drawn, and "representative DN value" thereof is calculated, as shown in formula (7):

in the formula (I), the compound is shown in the specification,"representing DN value" representing ith picture;representing the lower limit of the interval with the most pixel number distribution in the ith image frequency distribution histogram;representing the upper limit of the interval with the most pixel number distribution in the ith image frequency distribution histogram; n is_iIndicates that it is located in a sectionThe number of pixels in the pixel array. FIG. 8 illustrates a DN value frequency distribution histogram for portions of an exemplary image taken at different times; fig. 9 shows the "representative DN value" corresponding to the determination of the result of fig. 8.

Step S5, generating a new thermal infrared image: let the reference image itself have a temperature drift value delta₀The temperature drift values (here quantified by DN values) of the remaining pictures relative to the reference picture can be calculated by equation (8). Further, the temperature drift level of each image can be normalized to the same level as the reference image by equation (9):

in the formula, delta_iRepresenting the ith image relative to a reference mapTemperature drift values of the image (quantified in DN values).

In the formula, DN_iA matrix of raw DN values representing the ith picture.

After each image is subjected to calculation processing of formula (8) and formula (9), the image is output to an image of a specified format (such as tiff format) (original EXIF information needs to be reserved), and the temperature drift removal of the original data is primarily completed.

Step S6, image stitching and band computing: and (3) splicing the monoscopic thermal infrared images (including the reference image) acquired in the step S5 by using image splicing software (such as Pix4D), so that a complete thermal infrared image of the flight target area can be acquired. The DN value image can be further converted into a bright temperature image by equation (10):

T_b＝a·DN+b (10)

in the formula, T_bA luminance temperature matrix representing an image, the elements of the matrix having units of; and a and b represent the conversion coefficient of the DN value and the brightness temperature provided by the thermal imager manufacturer.

Step S7, removing the reference image temperature drift value: according to the steps, the spliced hot infrared bright temperature image has a temperature drift constant delta₀This can be corrected by the observation of surface instruments such as a SI-111 infrared radiometer. Before correction, simple radiative transfer simulation can be performed through a MODTRAN model (a radiative transfer model), and brightness temperature data recorded by a ground instrument at a reference image acquisition moment (acquisition time is recorded in image EXIF data) is simulated to brightness temperature data at the flying height of the unmanned aerial vehicle, as shown in formula (11):

in the formula (I), the compound is shown in the specification,the brightness temperature value of the ground observation instrument at the reference image acquisition moment is simulated to a corresponding value at the flying height of the unmanned aerial vehicle, and the unit is; g represents the mapping relation between the earth surface brightness temperature constructed by the MODTRAN model and the brightness temperature at the flying height of the unmanned aerial vehicle;and the brightness temperature value recorded by the ground observation instrument at the reference image acquisition moment is represented and has the unit of ℃.

The temperature drift value δ included in the bright temperature image acquired in step S6 can be calculated by the formula (12)₀And then, the flying area brightness temperature image after the temperature drift is completely removed can be obtained through a formula (13).

In the formula (I), the compound is shown in the specification,and the brightness temperature value of the pixel on the unmanned aerial vehicle image corresponding to the GPS position of the ground observation instrument is represented.

T_b-change＝T_b-origin-δ₀ (13)

In the formula, T_b-changeRepresenting a thermal infrared bright temperature image matrix of the flight target area after the temperature drift is completely removed, wherein the unit is; t is_b-originThe original thermal infrared bright temperature image matrix obtained in step S6 is represented in units of ℃.

As shown in fig. 10-12, in the embodiment of the present invention, the thermal infrared images of 3 flight target regions (i.e., the large full station, the village substation, and the wetland station) in different geographic locations are subjected to the temperature drift removal processing, and it can be found that the images of the entire regions have uneven light and dark patch distribution before the temperature drift removal; after the method disclosed by the invention is used for removing the temperature drift, the quality of the lighting temperature images of 3 stations is obviously improved, unreasonable light and shade changes can hardly be seen visually, and the distribution range of the lighting temperature values is more consistent with the actual situation of a flight target area.

In the method for removing drift of temperature measurement results of the non-refrigeration thermal imager for the unmanned aerial vehicle, provided by the embodiment of the invention, a scene image in a flight area of the unmanned aerial vehicle is selected as a reference image, and a DN value frequency distribution histogram is calculated to obtain a 'representative DN value'; then, the remaining images in the flight area after the redundant images are removed are also subjected to DN value frequency distribution histogram and the corresponding 'representative DN values', and the difference between the DN value frequency distribution histogram and the corresponding 'representative DN values' relative to the reference images is obtained; meanwhile, subtracting the obtained difference value from the images to obtain thermal infrared images after the temperature drift is removed preliminarily, and storing the thermal infrared images as new images with specified formats; finally, splicing the images after each scene is corrected by using jigsaw software and carrying out band operation, so that a thermal infrared bright temperature image of a flight target area can be obtained, and complete removal of temperature drift can be realized by combining bright temperature observation data and radiation transmission simulation of a ground instrument. According to the method, from the perspective of image post-processing, few ground observation data are utilized, the drift of the temperature measurement result of the non-refrigeration thermal imager of the unmanned aerial vehicle is removed, and the precision and consistency of bright temperature data are improved; in addition, the invention is based on the implementation mode of post-processing, and overcomes the defect that a plurality of reference temperature plates are arranged in the field to carry out rough temperature drift correction in the traditional method, so that the method greatly improves the field operation efficiency of personnel while ensuring the high-precision temperature drift removal effect.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

What has been described above are merely some embodiments of the present invention. It will be apparent to those skilled in the art that various changes and modifications can be made without departing from the inventive concept thereof, and these changes and modifications can be made without departing from the spirit and scope of the invention.