阅读说明：本技术 全景成像直升机辅助降落装置 (Auxiliary landing device of panoramic imaging helicopter ) 是由 纪明 何樱 陶忠 高强 李良福 桑蔚 张鑫诚 安学智 章文娟 谢欢 于 2019-11-27 设计创作，主要内容包括：本发明公开了一种基于全景成像的直升机辅助降落装置,属于光电侦察、导航领域。该装置由双通道红外成像仪和全景环带成像镜头组合的成像系统、控制单元,信息处理单元组成。成像系统安装在直升机机腹下方,其中的红外成像仪和环带镜头分别提供直升机下方的中心视场图像和环带全景图像。成像系统获取的图像,由电子箱中的信息处理单元接收控制单元的控制指令,进行目标距离解算和环带图像展开。同时发送中心视场图像和展开的全景图像至机上综显系统。该系统作为一种全新的辅助降落装置,能够向飞行员同时呈现机腹以下360°范围内的全景图像并具备深度信息,能显著提升直升机降落的安全性。(The invention discloses a helicopter landing assisting device based on panoramic imaging, and belongs to the field of photoelectric reconnaissance and navigation. The device consists of an imaging system formed by combining a dual-channel infrared imager and a panoramic annular imaging lens, a control unit and an information processing unit. The imaging system is arranged below the belly of the helicopter, and the infrared imager and the annular lens respectively provide a central view field image and an annular panoramic image below the helicopter. The image acquired by the imaging system is received by the information processing unit in the electronic box to carry out target distance calculation and annulus image expansion. And simultaneously sending the central view field image and the unfolded panoramic image to an integrated display system on the machine. As a brand-new auxiliary landing device, the system can simultaneously present panoramic images within a range of 360 degrees below the belly to a pilot, has depth information, and can remarkably improve the landing safety of the helicopter.)

1. The utility model provides a supplementary landing device of panoramic imaging helicopter which characterized in that: the system comprises an imaging system, a control unit and an information processing unit, wherein the imaging system consists of a two-channel near-infrared imager and a panoramic annular lens;

the imaging system can simultaneously provide a central view field image and a panoramic annular image below the helicopter, wherein the two-channel near-infrared imager acquires the central view field image below the helicopter, and the panoramic annular lens acquires the panoramic annular image in the 360-degree range of the lower direction of the helicopter;

the information processing unit collects two images of two thermal infrared imagers in a double-channel near infrared imager in the same scene, identifies the same characteristic point, calculates the distance value of a selected target through characteristic matching operation and according to a stereo parallax method, superposes the distance information on a central view field image, and transmits the central view field image to a comprehensive display and control device on a helicopter for display;

the information processing unit collects images imaged by the two panoramic annular lenses, and calculates a distance value of a target point by a triangulation method aiming at the target which simultaneously appears in the two panoramic annular lens imaged images; the information processing unit also selects an image imaged by a certain panoramic annular lens according to a control command of the control unit, the image is linearly transformed and expanded into a rectangular panoramic image through a pixel matrix, a distance value of a target point is superposed on the rectangular panoramic image, and then the rectangular panoramic image is transmitted to the comprehensive display control equipment on the helicopter for display;

the control unit is used for selecting an observation target, sending a control command and adjusting the stereoscopic observation effect.

2. A panoramic imaging helicopter supplemental landing device, as defined in claim 1, wherein: the panoramic annular lens adopts an annular three-dimensional imaging system based on a panoramic annular lens, and the annular three-dimensional imaging system based on the panoramic annular lens comprises two groups of coaxially arranged panoramic annular lenses.

3. A panoramic imaging helicopter supplemental landing device, as defined in claim 2, wherein: the two thermal infrared imagers are installed on two sides of the panoramic annular lens and are jointly fixed below the belly.

Technical Field

The invention belongs to the field of photoelectric reconnaissance and navigation, and mainly relates to an auxiliary landing device of a panoramic imaging helicopter.

Background

The landing action of a helicopter in a complex environment has been a difficult flight operation. The air transportation material and wounded rescue of the helicopter often need to be implemented in bad weather and mountain areas, the complex terrain and weather often limit the mission flight and efficiency of the helicopter, one important factor is that advanced landing equipment is lacked, and a driver cannot complete the landing action of the helicopter under the condition of not accurately sensing the environment. Statistics and analysis of the accidents worldwide in 2013 show that the number of the accidents occurring in the approach/landing stage is 204, and accounts for 44% of the total number of the accidents.

When the helicopter lands, a driver needs to observe the surrounding environment firstly, for example, in a mountainous area, some trees can damage the rotor wing possibly, and the helicopter is damaged. In the battle, whether there is enemy threat around needs to be observed. Secondly, the terrain below needs to be observed, the helicopter can be overturned due to the seriously uneven ground, and different landforms have different influences on the airflow lift model when the helicopter lands. The helicopter landing aid system utilizes the characteristic that the annular belt imaging has a 360-degree field angle, can observe the terrain and the landform below and the surrounding environment when the helicopter lands, and simultaneously combines the terrain state accurately calibrated by a stereoscopic vision technology to provide an accurate landing judgment basis for a driver.

The existing helicopter landing assisting related patents have no landing assisting device or method based on panoramic imaging. The patent related to the safe landing system of the falling helicopter solves the problem that the helicopter cannot be rescued in case of air accident through a parachute, a deceleration jet, an inflatable rubber tube and the like which are arranged on the helicopter. There is also a patent of a ship-borne helicopter emergency lighting landing assisting method, and belongs to the technical field of lighting. The guide for the landing of the helicopter is realized by the centralized control of lamplight when the helicopter lands at night. The patent of a helicopter landing assisting system and a landing assisting method based on wireless ultraviolet light belongs to a communication system, and the helicopter landing is assisted by utilizing an ultraviolet light communication technology. The auxiliary landing device or method in the existing patent can not meet the requirement of the armed helicopter for landing in strange places. The device that this patent put forward satisfies this demand betterly.

Disclosure of Invention

The invention provides a panoramic imaging helicopter landing aid device, which aims to solve the problem that the full-view landform and external stereoscopic vision information below a belly cannot be observed when a helicopter lands, so that the landing safety is influenced, and the defect that the full-view sensing below the belly is lacked when an existing armed helicopter lands is solved.

According to the binocular stereo vision principle, the helicopter landing assistant system device based on panoramic imaging is designed by adopting a panoramic annular stereo imaging technology, an infrared night vision system technology and a dual-sensor image matching processing technology, so that a real and definite panoramic image below the belly in the landing process is provided for a driver.

The technical scheme of the invention is as follows:

the utility model provides a supplementary landing device of panorama formation of image helicopter which characterized in that: the system comprises an imaging system, a control unit and an information processing unit, wherein the imaging system consists of a two-channel near-infrared imager and a panoramic annular lens;

the imaging system can simultaneously provide a central view field image and a panoramic annular image below the helicopter, wherein the two-channel near-infrared imager acquires the central view field image below the helicopter, and the panoramic annular lens acquires the panoramic annular image in the 360-degree range of the lower direction of the helicopter;

the information processing unit collects two images of two thermal infrared imagers in a double-channel near infrared imager in the same scene, identifies the same characteristic point, calculates the distance value of a selected target through characteristic matching operation and according to a stereo parallax method, superposes the distance information on a central view field image, and transmits the central view field image to a comprehensive display and control device on a helicopter for display;

the information processing unit collects images imaged by the two panoramic annular lenses, and calculates a distance value of a target point by a triangulation method aiming at the target which simultaneously appears in the two panoramic annular lens imaged images; the information processing unit also selects an image imaged by a certain panoramic annular lens according to a control command of the control unit, the image is linearly transformed and expanded into a rectangular panoramic image through a pixel matrix, a distance value of a target point is superposed on the rectangular panoramic image, and then the rectangular panoramic image is transmitted to the comprehensive display control equipment on the helicopter for display;

the control unit is used for selecting an observation target, sending a control command and adjusting the stereoscopic observation effect.

Further preferred scheme, the supplementary landing device of panorama formation of image helicopter, its characterized in that: the panoramic annular lens adopts an annular three-dimensional imaging system based on a panoramic annular lens, and the annular three-dimensional imaging system based on the panoramic annular lens comprises two groups of coaxially arranged panoramic annular lenses.

Further preferred scheme, the supplementary landing device of panorama formation of image helicopter, its characterized in that: the two thermal infrared imagers are installed on two sides of the panoramic annular lens and are jointly fixed below the belly.

Advantageous effects

The whole technical effect of the invention is shown in the following aspects.

The invention realizes the purpose of providing the full-scene type annular image information and the lower central view field image information below the helicopter through related technical means. Compared with the prior art, the panoramic image information which is formed by the central view field and the annular panoramic view field and is true and definite in external scene can be provided for the driver.

The method can extract the nuances of two images of the same scene observed by two cameras at the same time, identify the same characteristic points, calculate the distance value of a selected target through characteristic matching operation and according to a stereo parallax method, and superimpose distance information on a central view field scene below the abdomen; the depth information of an object in the image can be extracted by adopting a triangulation method for the image acquired by the panoramic annular lens, and the extracted depth information is superposed on the image scene of the panoramic annular below the belly; the panoramic annular image can be expanded into a rectangular panoramic image through linear transformation of a pixel matrix; compared with the prior art, the method can provide the driver with the environment information of all directions below the belly.

Drawings

Fig. 1 is a schematic view of a panoramic imaging system in the apparatus of the present invention.

Fig. 2 is a schematic diagram of a single PAL configuration and imaging light path in the apparatus of the present invention.

FIG. 3 is a schematic diagram of imaging of a panoramic stereo zone imaging system in the apparatus of the present invention.

FIG. 4 is a schematic diagram of a triangulation method for extracting stereo information by a panoramic annular imaging system

Fig. 5 is an image plane schematic diagram of a panoramic annular stereo imaging system.

FIG. 6 is a diagram of the detection field of view of a panoramic annular imaging system.

FIG. 7 is a schematic diagram of detector position relationships for a panoramic annular imaging system.

FIG. 8 is a schematic view of a display interface layout of a panoramic imaging system.

FIG. 9 is a schematic diagram of the coordinate correspondence of the unfolded rectangular image of the panoramic imaging annular image.

FIG. 10 is a schematic diagram of a helicopter landing aid system.

Detailed Description

The invention is described below with reference to specific examples:

the auxiliary landing device of the panoramic imaging helicopter provided by the invention comprises an imaging system, an information processing unit, a control unit and a display unit, wherein the imaging system is composed of a double-channel thermal infrared imager and a panoramic annular lens. The two-channel thermal infrared imager of the imaging system detects a central view field below the belly of the helicopter, and the panoramic annular lens detects a view field in a 360-range direction below the belly of the helicopter, as shown in fig. 2, and the two are complementary to each other and can detect all spaces below the whole helicopter. The central field-of-view left and right detectors and the annular field-of-view detector are integrated on a mounting platform of a small photoelectric nacelle and are mounted below the belly of the helicopter. The left detector and the right detector of the central view field have slight difference on the image detected by the same central view field at the lower part through accurate calibration and calibration, and the difference is related to parameters such as the length of a base line, the focal length, the resolution ratio and the like of the dual-channel infrared detector. The information processing unit extracts the slight difference, extracts the distance information of the target in the image, superimposes the distance information on the central view field scene, and sends the central view field image to the display unit. The annular visual field detector comprises two groups of Panoramic Annular Lenses (PAL) which are coaxially arranged and imaged on the same sensor, and the imaging circle of the PAL is two internal and external connected rings. The object is imaged in the two rings through the two PAL units respectively and sent to the information processing unit of the electronic box, the information processing unit compares the position difference of the same target simultaneously appearing in the two rings through a triangulation method, the distance information of the target is extracted, meanwhile, the information processing unit also linearly transforms and expands a certain panoramic annular image into a rectangular panoramic image through a pixel matrix, and then the distance information extracted before is superposed to a panoramic scene and sent to the display unit. The control unit receives a user instruction and sends the user instruction to the information processing unit to perform various image processing.

Description of panoramic sensor

(1) Description of optical System

The optical ring-belt imaging technology applies a novel cylindrical surface/plane projection imaging principle, namely, image information is shot from a cylindrical surface and projected to a plane system to display the image information, replaces the plane/plane projection principle of the conventional optical system, and projects a cylindrical view field (α area in the figure) in a range of 360 degrees around an optical axis of the optical system into an annular area on a two-dimensional plane as shown in fig. 4, so that panoramic real-time imaging of 360-degree ring-belt space is realized conveniently.

The optical path principle of the panoramic annular stereo imaging is shown in figures 3 and 4The imaging optical path of the single PAL lens is as shown in FIG. 3, 1, 3 are reflective surfaces, 2, 4 are refractive surfaces, the imaging can be based on planar cylindrical projection method, as shown in FIG. 2, the scenery with 360 ° horizontal optical axis is projected to a two-dimensional plane, A point and B point are imaged on CCD through 2 times of refraction and reflection, as A 'point and B' point in FIG. 1, the imaging point position is determined by angle of view α, β is a blind area, the object point with the same vertical angle of view as the optical axis is imaged on CCD to form a concentric circle with the same radius after passing through PAL system, if the three-dimensional object point connecting line is parallel to the optical axis, the images of these points on the image surface are on the same radial extension line, all the object points finally form a ring belt image on CCD, the horizontal angle of view is 360 °, and the vertical angle range is (- α -₂,α₁). Since the horizontal field of view is 360, the angles of view discussed below are all vertical angles of view.

The optical path principle of the panoramic system with the double PAL lens is shown in fig. 4. PAL (phase alternating line)_upAnd RL_upIs the upper PAL part and its subsequent lens group; PAL (phase alternating line)_downAnd RL_downIs the lower PAL part and its subsequent lens group, the upper PAL system utilizes the blind area of the lower PAL system to image, the imaging view angle of the upper PAL is (- β)_up，α_up) The imaging field angle of the lower PAL is (- β)_do_wn，α_down). It follows that stereo information can only be extracted from the area where two PAL units overlap. Two rays, one at the angle of incidence θ, from point P in FIG. 3_downEntering the PAL system below, and imaging at P after two refractions and reflections₂One point and the other point at incident angle of view theta_upAfter the light is refracted and reflected twice by the upper PAL system, the image is formed on P by the blind area part of the lower PAL system₁The different angles of view differ at the location of the CCD image.

Imaging surface as shown in fig. 5, the outer zone is the imaging area of the lower PAL, the inner zone is the imaging area of the upper PAL, and the central circular area is the blind area. Phi' is the azimuth angle of the object point P, its magnitude being related to the initially defined 0 deg. azimuth. P1 and P2 are both images of point P, their radii are r_upAnd r_down。r_up-max、r_up-min、r_down-maxAnd r_down-minThe two PAL units are imaged separately to the zone border. The two PAL units are coaxial with the center of the sensor, the 2 image points of the same object point on the inner and outer ring belts are in the same radial ray direction, but are respectively positioned at both sides of the optical axis, such as the space object point P, the image P passing through the two PAL systems₁And P₂The connecting line passes through the center of the circle.

(2) Detector structure composition

The detector comprises two central view field detectors and an annular view field detector, and the two view fields need to completely cover the space below the airplane when being superposed, as shown in fig. 6; the detector comprises a left central view field detector, a right central view field detector and an annular view field detector, the detector is arranged on a base, a schematic diagram of the installation position relationship is shown in figure 7, wherein D1 is the length of a base line of a lens of a panoramic annular, D2 is the length of two base lines of infrared lenses, the two base lines need to meet the requirement of fine adjustment in structural design, the length of the base line is an important factor for restricting the resolution of a system, the longer base line length is helpful for improving the resolution of the system, but the size of the detector is larger, the detector serving as airborne equipment has certain limitation on the size of the detector, and the design can be adjusted according to specific parameters.

(II) information processing Unit description

The information processing unit comprises the following steps of:

(1) extracting three-dimensional information of the two-channel near-infrared image;

the double-channel near-infrared detector comprises a left detector and a right detector, and can simultaneously acquire a left image and a right image with a small difference corresponding to the same scene, and the acquisition algorithm framework of the image three-dimensional information comprises: (1) extracting characteristic points of the left image by using a Harris algorithm; (2) corresponding to each feature point of the left image, extracting corresponding epipolar lines of the right image according to an epipolar constraint method; (3) aiming at each feature point of a left image, constructing an SIFT feature descriptor, constructing a feature description vector by utilizing a direction statistical histogram of gradients in a feature point neighborhood image window by the descriptor, searching a central point of a sub-image by using the minimum Euclidean distance as a matching point by calculating Euclidean distances between the feature vector of a certain feature point and a plurality of search element sub-image feature vectors of a right image in an epipolar direction, obtaining a registration point, then solving the parallax of the two image points on an imaging surface according to the registration point pair, and calculating a target distance value in the image according to the following formula:

wherein f is the focal length of the detector and is a known quantity, b is the base lines of the two infrared detectors and is a known quantity, d is the parallax of the same target point on the two detectors, and the parallax can be easily calculated according to the pixel position, the focal length and the resolution of the registration point.

(2) The panoramic annular image is unfolded into a panoramic rectangular image;

in order to facilitate a driver to observe the full-scene landform below the belly, the panoramic annular image needs to be expanded into a panoramic rectangular image, and the expansion process is actually linear transformation of a pixel matrix. The unfolding process can be considered as that the ring belt image is broken from one position and then is converted into a rectangular image through stretching and bending operations, the fan-shaped part of the original ring belt image is converted into the rectangular part of the rectangular image, and the radial pixels of the ring belt image in one direction with the circle as the center correspond to a column of longitudinal pixels of the rectangular image. The two images are represented in the form of a pixel matrix, and the rectangular image can be regarded as a panoramic annular image obtained through linear transformation.

To illustrate the correspondence, a coordinate system as shown in fig. 9 is established. P (x, y) is an arbitrary pixel in the rectangular image, P '(x', y ') is a pixel in the corresponding annular region, and assuming polar coordinates as P' (r, a), an annular center point coordinate o (x, y)_c,y_c) The following correspondence relationship exists:

x′＝x_c+rcosa

y′＝y_c+rsina

let the starting point position A' of the ring belt image development (r)₀,α₀) With a (x) of a rectangle₀,y₀) Correspondingly, then there are

Δx＝x-x₀

Δy＝y-y₀

Is derived from

r＝r₀+Δr＝r₀+Δy＝r₀+(y-y₀)

a＝a₀+Δa＝a₀+Δx＝a₀+(x-x₀)

Combined upper type, have

x′＝x_c+[r₀+(y-y₀)]*cos[a₀+(x-x₀)]

y′＝y_c+[r₀+(y-y₀)]*cos[a₀+(x-x₀)]

The coordinate conversion relationship from the ring-belt image point to the rectangular image point can be completed according to the formula, and the color values of the ring-belt image point and the rectangular image point are equal to each other:

g(x,y)＝f(x′,y′)。

(3) and extracting the stereo information of the panoramic annular image.

To simplify the imaging system, the single-point imaging is based on FIG. 5, the Z-axis is the optical axis, the distance from the object point P to the optical axis is defined as S, α is the upper incident angle, β is the lower incident angle, O, O' are the centers of the entrance pupils of the upper and lower PAL rays, respectively, d is the center distance of the entrance pupils of the two PAL units, α and β are the distances from the object point to the centers of the upper and lower entrance pupils, respectively (labeled as r, respectively)₁And r₂) Determined, the P-point location may be determined by:

α is a known quantity, which can be obtained by searching and calculating according to an optical design calibration parameter table, the parameter table reflects the relationship between the radius of the image point on the annular image plane image and the incident angle, the radius of the image point on the image plane image can be very easily obtained by calculating the pixel position, and then the parameter table is searched to conveniently obtain the value of the incident angle α.

d(α，β)＝d₀+Δd(α)-Δd(β)

Δ d (α) is the upper PAL entrance pupil movement amount, Δ d (β) is the lower entrance pupil movement amount, d₀The distance between the centers of the entrance pupils at the minimum field of view of the upper and lower PALs is a known quantity.

After S is obtained, the position P (X, Y, Z) of the P point can be calculated according to the following formula,

wherein gamma is the included angle between the image point and the X axis and is a known quantity.

Wherein, O_z' is the Z-axis distance of the optical center of the lower PAL in object space, a known quantity.

Therefore, two corresponding image points of a certain target point in space in the two annular images are searched in the panoramic stereo image, the position of the corresponding point on the image plane and the entrance pupil distance are obtained, and the distance value Z of the target point can be obtained.

Description of the (III) display units

The display unit simultaneously provides the central view field image and the expanded panoramic rectangular image information and other key mechanism images on the helicopter for the user, all space images below the helicopter can be simultaneously displayed on one display screen, the central view field image and the panoramic view field ring image can be arranged on the display screen in different modes, and fig. 7 shows an alternative embodiment: and simultaneously displaying a central view field picture and a peripheral direction 360-degree range picture below the belly on the on-board comprehensive display platform and images of other on-board shutdown components.

Description of the control units

The control unit provides the pilot with software operation and control interfaces optional for the panoramic imaging system, including but not limited to the following: the system comprises an annular panoramic image unfolding command interface, a tracking target command interface, a target distance calculating command interface and the like.