阅读说明：本技术 一种双光融合无人机的图像采集系统 (Double-light fusion unmanned aerial vehicle's image acquisition system ) 是由 王洪兴 宋天明 刘维仁 金凯欣 李鑫 刘尧 于 2021-07-12 设计创作，主要内容包括：本发明公开了一种双光融合无人机的图像采集系统,属于无人机图像采集领域。上述系统包括：数据采集模块、智能数据处理模块和实时数据传输模块,数据采集模块用于可见光图像和红外光图像的采集,并将采集的图像传输至智能数据处理模块进行图像处理和融合,实时数据传输模块用于接收智能数据处理模块融合后的图像,并实时由机载的图传天空端经过特定频段的通讯链路发送至地面端。本发明的利用拓扑自适应snake算法模型对可见光和红外光图像进行处理,并能够实现双光图像快速融合、实时显示、低延时快速传输等,可以有效解决森林火点不易找寻、人员搜救容易遗漏等的问题,可以更加精准的相应作业。(The invention discloses an image acquisition system of a double-light fusion unmanned aerial vehicle, and belongs to the field of unmanned aerial vehicle image acquisition. The above-mentioned system includes: the system comprises a data acquisition module, an intelligent data processing module and a real-time data transmission module, wherein the data acquisition module is used for acquiring visible light images and infrared light images and transmitting the acquired images to the intelligent data processing module for image processing and fusion, and the real-time data transmission module is used for receiving the images fused by the intelligent data processing module and transmitting the images to a ground end through a communication link of a specific frequency band by an airborne image transmission sky end in real time. The method utilizes the topology self-adaptive snake algorithm model to process the visible light and infrared light images, can realize quick fusion, real-time display, low-delay quick transmission and the like of double light images, can effectively solve the problems that forest fire points are not easy to find, personnel search and rescue are easy to miss and the like, and can perform more accurate corresponding operation.)

1. The utility model provides a two optical fusion unmanned aerial vehicle's image acquisition system which characterized in that includes: the system comprises a data acquisition module, an intelligent data processing module and a real-time data transmission module;

the data acquisition module is used for acquiring visible light images and infrared light images and transmitting the acquired images to the intelligent data processing module;

the intelligent data processing module is used for storing and processing the collected visible light image and infrared light image and comprises a storage unit, a data analysis unit, an intelligent algorithm unit and a data fusion unit;

the storage unit is used for receiving and storing the data transmitted by the data acquisition module; the data analysis unit is used for carrying out data analysis on the received image and transmitting the analyzed data to the intelligent algorithm unit; the intelligent algorithm unit is used for carrying out topology self-adaptive snake algorithm model processing on the analyzed data; the data fusion unit is used for registering and fusing the processed visible light and infrared light images and transmitting the fused images to the real-time data transmission module;

the real-time data transmission module comprises an image transmission sky end, a communication link and an image transmission ground end and is used for receiving the image fused by the intelligent data processing module and sending the image transmission sky end carried by the aircraft to the ground end through the communication link of a specific frequency band in real time.

2. The image acquisition system of the dual light fusion unmanned aerial vehicle of claim 1, wherein the data acquisition module comprises a data acquisition card, a dual-lens stabilization tripod head, and a visible light lens and an infrared light lens mounted on the dual-lens stabilization tripod head; the visible light lens and the infrared light lens shoot images and then transmit the images to the data acquisition card for storage, and the data acquisition card synchronously transmits the received data to the intelligent data processing module.

3. The image acquisition system of a dual optical fusion unmanned aerial vehicle of claim 2, wherein the intelligent algorithm unit is configured to perform topology adaptive snake algorithm model processing on the analyzed data, and the processing comprises:

step 1: acquiring an initial region of interest in the visible light image based on a topology self-adaptive snake model;

step 2: performing expansion processing on the initial region of interest through a balloon topology self-adaptive snake external force model to obtain a region of interest;

and step 3: extracting an infrared light signal sub-image I corresponding to the region of interest from the infrared light image_s；

And 4, step 4: for the infrared light signal sub-image I_sExtracting effective data to obtain effective data subimage I_t；

And 5: for valid data subimage I_tFiltering the small-area signal to obtain a filtered subimage I_p。

4. The image acquisition system of the dual optical fusion unmanned aerial vehicle of claim 3, wherein the step 1 is specifically: and setting an infrared image heat threshold value through a human-computer interaction interface, and calibrating an artificial neural network within a threshold value range to automatically select an initial region of interest.

5. The image acquisition system of the dual optical fusion unmanned aerial vehicle of claim 4, wherein in the step 2, the balloon topology adaptive snake external force model is as follows:

wherein k is₁，k₂Represents a constant, k₁，k₂Are all real numbers, and are all real numbers,representing a unit external normal vector, Eext ═ DeltaI (x, y) & gtnon-conducting²And I (x, y) denotes the signal strength at the (x, y) coordinate.

6. The image acquisition system of the dual-light fusion unmanned aerial vehicle of claim 5, wherein the step 4 is specifically:

using a predetermined threshold value for an infrared light signal subimage I_sFiltering out the clutter signal to obtain effective data subimage I_t，

Two threshold values I are preset₀And I₁，I₀<I₁Using the two thresholds to filter out the infrared light signal subimage I_sThe intensity of the mid-infrared light signal is less than I₀Or greater than I₁I.e.:

in the infrared light signal sub-image I_sIf the signal intensity at (x, y) coordinate I (x, y) satisfies I (x, y)<I₀Or I (x, y)>I₁If I (x, y) is 0, the signal is filtered.

7. The image acquisition system of a dual optical fusion unmanned aerial vehicle of claim 6, wherein in the signal filtering process, if there are 5 or more coordinates without signals in the δ -field range of a certain pixel point, the signal on the pixel point is filtered.

8. The image acquisition system of the dual-light fusion unmanned aerial vehicle of claim 6, wherein the step 5 is specifically:

first, a threshold S is set₀In the effective data subimage I_tFor a region where a signal is concentrated, the area of the region is smaller than S₀Then the signals of all points in the region are filtered out.

9. The image acquisition system of a dual light fusion unmanned aerial vehicle of claim 3, wherein the data fusion unit filters the sub-imagesI_pAnd superposing the two images to a visible light image to obtain a Cerenkov registration image and outputting the Cerenkov registration image to obtain a double-light fusion image.

10. The image acquisition system of a dual light fusion unmanned aerial vehicle of claim 9, wherein the filtering sub-image I_pIn the process of superimposing on the white light image, sub-image I is filtered_pIs uniformly fixed on [ L, I_max]Is colored on a color scale of (1), wherein L is 256, I_maxIs the maximum signal value of the optical image.

Technical Field

The invention relates to the technical field of unmanned aerial vehicle systems, in particular to an image acquisition system of a double-light fusion unmanned aerial vehicle.

Background

In recent years, with the continuous development of unmanned aerial vehicle technology, various types of unmanned aerial vehicles are widely applied to industries, so that the production and development efficiency of the industries is improved, and the consumption of manpower and material resources is saved. The forestry fire control unmanned aerial vehicle's application has mainly been solved and has been patrolled and protected the early discovery that can't take place forest fire in the remote area that can't take care of on the ground to and accurate the grasp and the timely understanding of the on-the-spot various dynamic information of major forest fire, also can solve the aircraft and patrol and protect unable night navigation, smog causes visibility to reduce unable flight scheduling problem. As a powerful supplement to the existing forestry monitoring means, the unmanned aerial vehicle shows incomparable superiority of other means, and is bound to be widely applied in aspects of monitoring, preventing, putting out a fire, evaluating after a disaster and the like of forestry fire.

The existing unmanned aerial vehicle monitoring mode generally adopts a relatively extensive working mode, and the unmanned aerial vehicle can only display a single visible light image or infrared light image in the image and video acquisition process and cannot well combine the two. Especially in the links such as forest fire point searching, personnel search and rescue, often the omission appears, has certain influence to the work of putting out a fire and rescue work. In order to guarantee the inspection quality of the unmanned aerial vehicle, the image acquisition system of the unmanned aerial vehicle needs to be further optimized.

Related patents in the prior art also disclose image acquisition systems based on visible light and infrared light double-light fusion, wherein different methods are adopted to process images and then perform image fusion, but the data processing method is complex, and phenomena such as misjudgment can also exist.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides an image acquisition system of a double-light fusion unmanned aerial vehicle, the image acquisition system utilizes a specific algorithm to process the image, and then visible light and infrared light images are fused, the system has the advantages of real-time display, low delay and quick transmission and the like, can effectively solve the problems that forest fire points are not easy to find, personnel search and rescue are easy to omit and the like, and can perform more accurate corresponding operation.

In order to solve the technical problems, the invention provides the following technical scheme:

the invention provides an image acquisition system of a double-light fusion unmanned aerial vehicle, which comprises: the system comprises a data acquisition module, an intelligent data processing module and a real-time data transmission module.

The data acquisition module is used for acquiring visible light images and infrared light images and transmitting the acquired images to the intelligent data processing module;

the intelligent data processing module is used for storing and processing the collected visible light image and infrared light image and comprises a storage unit, a data analysis unit, an intelligent algorithm unit and a data fusion unit;

the storage unit is used for receiving and storing the data transmitted by the data acquisition module; the data analysis unit is used for carrying out data analysis on the received image and transmitting the analyzed data to the intelligent algorithm unit; the intelligent algorithm unit is used for carrying out topology self-adaptive snake algorithm model processing on the analyzed data; the data fusion unit is used for registering and fusing the processed visible light and infrared light images and transmitting the fused images to the real-time data transmission module;

the real-time data transmission module comprises an image transmission sky end, a communication link and an image transmission ground end and is used for receiving the image fused by the intelligent data processing module and sending the image transmission sky end carried by the aircraft to the ground end through the communication link of a specific frequency band in real time.

Furthermore, the data acquisition module comprises a data acquisition card, a double-lens stability-increasing cradle head, and a visible light lens and an infrared light lens which are arranged on the double-lens stability-increasing cradle head; the visible light lens and the infrared light lens shoot images and then transmit the images to the data acquisition card for storage, and the data acquisition card synchronously transmits the received data to the intelligent data processing module.

Further, the intelligent algorithm unit is configured to perform topology adaptive snake algorithm model processing on the analyzed data, and includes:

step 1: acquiring an initial region of interest in the visible light image based on a topology self-adaptive snake model;

step 2: performing expansion processing on the initial region of interest through a balloon topology self-adaptive snake external force model to obtain a region of interest;

and step 3: extracting an infrared light signal sub-image I corresponding to the region of interest from the infrared light image_s；

And 4, step 4: for the infrared light signal sub-image I_sExtracting effective data to obtain effective data subimage I_t；

And 5: for valid data subimage I_tFiltering the small-area signal to obtain a filtered subimage I_p。

Further, the step 1 specifically comprises: and setting an infrared image heat threshold value through a human-computer interaction interface, and calibrating an artificial neural network within a threshold value range to automatically select an initial region of interest.

Further, in step 2, the balloon topology adaptive snake external force model is as follows:

wherein k is₁，k₂Represents a constant, k₁，k₂Are all real numbers, and are all real numbers,representing a unit external normal vector, Eext ═ DeltaI (x, y) & gtnon-conducting²And I (x, y) denotes the signal strength at the (x, y) coordinate.

Further, the step 4 specifically includes:

using a predetermined threshold value for an infrared light signal subimage I_sFiltering out the clutter signal to obtain effective data subimage I_t，

Two threshold values I are preset₀And I₁，I₀<I₁Using the two thresholds to filter out the infrared light signal subimage I_sThe intensity of the mid-infrared light signal is less than I₀Or greater than I₁I.e.:

in the infrared light signal sub-image I_sIf the signal intensity at (x, y) coordinate I (x, y) satisfies I (x, y)<I₀Or I (x, y)>I₁If I (x, y) is 0, the signal is filtered.

Preferably, in the signal filtering process, if 5 or more coordinates in the range of the δ field of a certain pixel point have no signal, the signal on the pixel point is filtered.

Further, the step 5 specifically includes:

first, a threshold S is set₀In the effective data subimage I_tFor a region where a signal is concentrated, the area of the region is smaller than S₀Then the signals of all points in the region are filtered out.

Further, the data fusion unit filters the subimage I_pAnd superposing the two images to a visible light image to obtain a Cerenkov registration image and outputting the Cerenkov registration image to obtain a double-light fusion image.

Preferably, the filtering of the subimage I_pIn the course of superimposing on the visible image, sub-image I is filtered_pIs uniformly fixed on [ L, I_max]Is colored on a color scale of (1), wherein L is 256, I_maxIs the maximum signal value of the optical image.

The basic idea of the Snakes model is simple, some control points forming a certain shape are used as templates (contour lines), and the templates are matched with local features of the image through the elastic deformation of the templates to achieve harmony, namely, a certain energy function is minimized, and the image segmentation is completed. And further analyzing the template to realize the understanding and recognition of the image. In brief, the snake model is a deformable parameter curve and a corresponding energy function, a minimized energy objective function is taken as a target, the deformation of the parameter curve is controlled, and a closed curve with minimum energy is a target contour. The Snakes model is constructed to reconcile the contradiction between the upper knowledge and the underlying image features. All image features are local, whether intensity, gradient, corner, texture, or optical flow. Locality means that the feature of a point on an image depends only on the neighborhood where the point is located, and is not related to the shape of an object. But the recognition of the object is mainly derived from its outer contour. How to fuse the two together effectively is the strength of the Snakes model.

Compared with the prior art, the invention has the following beneficial effects:

according to the image acquisition system of the double-light fusion unmanned aerial vehicle, the unmanned aerial vehicle reaches the data acquisition stage after the unmanned aerial vehicle reaches the designated operation position, and the visible light image and the infrared light image are acquired simultaneously after the shooting position is determined. Visible light images are collected through the visible light lens, infrared light images are collected through the infrared light lens, and the two lenses are fixed through the double-lens stability-increasing cradle head, so that the positions of the lenses can be fixed, the processing of the images is facilitated, the stability of the lenses can be improved, and the quality of the collected images is improved. According to different operation requirements, the airplane flies to a corresponding position and carries out image acquisition, and the acquired image is transmitted to the intelligent data processing module through the data acquisition card after the image is acquired. A storage unit to store an image to be processed; processing the collected visible light image and infrared light image; analyzing the two images, and extracting feature points; matching images through intelligent algorithm model processing; and finally, carrying out image fusion and sending the fused image to a real-time data transmission module. The processed fused images are rapidly transmitted in real time, airborne low-delay high-definition images are transmitted to the sky end through a communication link of a specific frequency band and are transmitted to the ground end, and the ground end receives the processed images and can display the processed images in real time so as to carry out subsequent fire rescue and personnel search and rescue.

According to the method, a topological self-adaptive snake algorithm model is utilized to select the region of interest of the visible light image, then the corresponding position of the infrared light image is extracted, the disordered signal filtering and the small-area signal filtering are carried out on the infrared light image, the filtered sub-image is fused with the visible light image, and then the image transmission is carried out, so that the rapid fusion of the two-light image is realized. The method utilizes a specific algorithm to process the image, then performs visible light and infrared light image fusion, has the advantages of real-time display, low-delay and quick transmission and the like, can effectively solve the problems that forest fire points are not easy to find, personnel search and rescue are easy to omit and the like, and can perform corresponding operation more accurately.

Drawings

Fig. 1 is a schematic diagram of an image acquisition system of a dual-light fusion unmanned aerial vehicle according to the present invention;

FIG. 2 is a schematic diagram of an intelligent medicine dispensing module in the image acquisition system of the dual optical fusion unmanned aerial vehicle of the invention;

fig. 3 is a schematic diagram of an accurate spraying module in the image acquisition system of the dual optical fusion unmanned aerial vehicle of the present invention;

fig. 4 is a schematic diagram of an intelligent monitoring module in the image acquisition system of the dual optical fusion unmanned aerial vehicle of the invention;

the arrow direction in the figure indicates the signal transfer direction.

Detailed Description

In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following detailed description is given with reference to specific embodiments.

In the present invention, the materials and reagents used are not specifically described, and are commercially available.

The invention provides an image acquisition system of a double-light fusion unmanned aerial vehicle, which is shown in figures 1-4 and comprises: the system comprises a data acquisition module, an intelligent data processing module and a real-time data transmission module.

The data acquisition module is used for acquiring visible light images and infrared light images and transmitting the acquired images to the intelligent data processing module;

the intelligent data processing module is used for storing and processing the collected visible light image and infrared light image and comprises a storage unit, a data analysis unit, an intelligent algorithm unit and a data fusion unit;

the storage unit is used for receiving and storing the data transmitted by the data acquisition module; the data analysis unit is used for carrying out data analysis on the received image and transmitting the analyzed data to the intelligent algorithm unit; the intelligent algorithm unit is used for carrying out topology self-adaptive snake algorithm model processing on the analyzed data; the data fusion unit is used for registering and fusing the processed visible light and infrared light images and transmitting the fused images to the real-time data transmission module;

the real-time data transmission module comprises an image transmission sky end, a communication link and an image transmission ground end and is used for receiving the image fused by the intelligent data processing module and sending the image transmission sky end carried by the aircraft to the ground end through the communication link of a specific frequency band in real time.

In one embodiment of the invention, the data acquisition module and the intelligent data processing module are connected by a communication line, and the intelligent data processing module and the real-time data transmission module are connected by a DuPont line. The real-time data transmission module and the intelligent data processing module are both installed on the flight control panel, and the data acquisition module is a detachable external device.

Further, the data acquisition module comprises a data acquisition card, a double-lens stability-increasing cradle head, and a visible light lens and an infrared light lens which are arranged on the double-lens stability-increasing cradle head; the visible light lens and the infrared light lens shoot images and then transmit the images to the data acquisition card for storage, and the data acquisition card synchronously transmits the received data to the intelligent data processing module.

In the invention, the unmanned aerial vehicle simultaneously acquires the visible light image and the infrared light image after determining the shooting position in the data acquisition stage after reaching the designated operation position. Visible light images are collected through the visible light lens, infrared light images are collected through the infrared light lens, and the two lenses are fixed through the double-lens stability-increasing cradle head, so that the positions of the lenses can be fixed, the processing of the images is facilitated, the stability of the lenses can be improved, and the quality of the collected images is improved. According to different operation requirements, the airplane flies to a corresponding position and carries out image acquisition, and the acquired image is transmitted to the intelligent data processing module through the data acquisition card after the image is acquired. A storage unit to store an image to be processed; processing the collected visible light image and infrared light image; analyzing the two images, and extracting feature points; matching images through intelligent algorithm model processing; and finally, carrying out image fusion and sending the fused image to a real-time data transmission module. The processed fused images are rapidly transmitted in real time, airborne low-delay high-definition images are transmitted to the sky end through a communication link of a specific frequency band and are transmitted to the ground end, and the ground end receives the processed images and can display the processed images in real time so as to carry out subsequent fire rescue and personnel search and rescue.

Further, the intelligent algorithm unit is configured to perform topology adaptive snake algorithm model processing on the analyzed data, and includes:

step 1: acquiring an initial region of interest in the visible light image based on a topology self-adaptive snake model: setting an infrared image heat threshold value through a human-computer interaction interface, and calibrating an artificial neural network within a threshold value range to automatically select an initial region of interest;

step 2: performing expansion processing on the initial region of interest through a balloon topology self-adaptive snake external force model to obtain a region of interest; the balloon topology self-adaptive snake external force model is as follows:

wherein k is₁，k₂Represents a constant, k₁，k₂Are all real numbers, and are all real numbers,representing a unit external normal vector, Eext ═ DeltaI (x, y) & gtnon-conducting²I (x, y) represents the signal strength at (x, y) coordinates;

and step 3: extracting an infrared light signal sub-image I corresponding to the region of interest from the infrared light image_s；

And 4, step 4: for the infrared light signal sub-image I_sExtracting effective data to obtain effective data subimage I_t；

Using a predetermined threshold value for an infrared light signal subimage I_sFiltering to obtain effective data subimage I_tFor example, two thresholds I may be preset₀And I₁，I₀<I₁Using the two thresholds to filter out the infrared light signal subimage I_sThe intensity of the mid-infrared light signal is less than I₀Or greater than I₁I.e.:

in the infrared light signal sub-image Is, if the signal intensity I (x, y) at the (x, y) coordinate satisfies I (x, y) < I0, or I (x, y) > I1, let I (x, y) > 0.

For the valid data subimage I_tFiltering out a disordered signal;

in the step, if 5 or more pixels in the delta field range of a certain pixel point have no signals, the signals on the pixel point are filtered;

and 5: for valid data subimage I_tFiltering the small-area signal to obtain a filtered subimage I_p: first, a threshold S is set₀In the effective data subimage I_tFor a region where a signal is concentrated, if the area of the region is smaller than S₀Then the signals of all points in the region are filtered out.

Further, the data fusion unit will filter the subimage I_pAnd superposing the two images to a visible light image to obtain a Cerenkov registration image and outputting the Cerenkov registration image to obtain a double-light fusion image.

Preferably, sub-picture I will be filtered_pIn the course of superimposing on the visible image, sub-image I is filtered_pIs uniformly fixed on [ L, I_max]Is colored on a color scale of (1), wherein L is 256, I_maxIs the maximum signal value of the optical image.

In conclusion, the method utilizes the topology self-adaptive snake algorithm model to process the visible light image and the infrared light image, can realize quick fusion, real-time display, low-delay quick transmission and the like of the double light images, can effectively solve the problems that forest fire points are not easy to find, personnel are easy to miss during search and rescue, and can perform corresponding operation more accurately.

The foregoing is a preferred embodiment of the present invention, and it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and are intended to be within the scope of the invention.