阅读说明：本技术 一种基于rbf网络的无人机视觉导航定位方法 (Unmanned aerial vehicle visual navigation positioning method based on RBF network ) 是由 贾海涛 吴婕 于 2019-09-23 设计创作，主要内容包括：本发明公开了一种基于RBF网络的无人机视觉导航定位方法。本发明的方案为：在GNSS信号未丢失时,通过相机采集图像,并从相机中提取图像帧,对每张图像检测特征点,提取描述子,保留图像的特征点信息；重复对提取的各图像帧的描述子的处理,将描述子信息与定位信息存储到视觉数据库中；在GNSS信号丢失情况下,提取相机拍摄图像,同样进行描述子提取,并利用视觉数据库信息,训练RBF网络分类器：然后根据RBF网络分类器对生成的描述子进行邻域搜索,估计最优匹配位置并基于其记录的定位信息得到当前定位信息。本发明在GNSS信号丢失情况下,基于其所构建的视觉数据库能实现对无人机的定位导航处理,且视觉数据库只存储图像的特征点描述子信息,内存占用空间小。(The invention discloses an unmanned aerial vehicle visual navigation positioning method based on a RBF network. The scheme of the invention is as follows: when the GNSS signal is not lost, acquiring an image through a camera, extracting an image frame from the camera, detecting a feature point of each image, extracting a descriptor, and keeping the feature point information of the image; repeating the processing of the descriptors of the extracted image frames, and storing descriptor information and positioning information into a visual database; under the condition that GNSS signals are lost, images shot by a camera are extracted, descriptor extraction is also carried out, and the RBF network classifier is trained by utilizing visual database information: and then performing neighborhood search on the generated descriptors according to the RBF network classifier, estimating the optimal matching position and obtaining the current positioning information based on the positioning information recorded by the optimal matching position. According to the invention, under the condition that GNSS signals are lost, positioning and navigation processing of the unmanned aerial vehicle can be realized based on the visual database constructed by the GNSS signals, and the visual database only stores characteristic point descriptor information of images, so that the occupied space of a memory is small.)

1. An unmanned aerial vehicle visual navigation positioning method based on an RBF network is characterized by comprising the following steps:

step S1: setting an RBF neural network for matching the feature point descriptors of the image, and training the neural network;

the RBF neural network comprises an input layer, a hidden layer and an output layer, wherein a transfer function of the hidden layer adopts a radial basis function;

the training samples are: in the navigation process of the unmanned aerial vehicle, images collected by an airborne camera are used; the feature vectors of the training samples are: feature point descriptors of the image obtained through ORB feature point detection processing;

step S2: constructing a visual database of the unmanned aerial vehicle during navigation:

in the navigation process of the unmanned aerial vehicle, images are collected through an airborne camera, ORB feature point detection processing is carried out on the collected images, descriptors of all feature points are extracted, and feature point descriptors of the current images are obtained; storing the feature point descriptors of the image and positioning information during image acquisition into a visual database;

step S3: unmanned aerial vehicle vision navigation positioning based on visual database:

based on a fixed interval period, extracting an image acquired by an airborne camera to serve as an image to be matched;

carrying out ORB feature point detection processing on the image to be matched, and extracting a descriptor of each feature point to obtain a feature point descriptor of the image to be matched;

inputting the feature point descriptor of the image to be matched into a trained RBF neural network, and performing neighborhood search to obtain the optimal matching feature point descriptor of the image to be matched in a visual database;

and obtaining the current visual navigation positioning result of the unmanned aerial vehicle based on the positioning information recorded in the database by the optimal matching feature point descriptor.

2. The method of claim 1, wherein step S3 further comprises: detecting whether the similarity between the optimal matching feature point descriptor and the feature point descriptor of the image to be matched is smaller than a preset similarity threshold value or not; if so, obtaining a current visual navigation positioning result of the unmanned aerial vehicle based on the positioning information recorded in the database by the optimal matching feature point descriptor; otherwise, the navigation is continued based on the recently obtained visual navigation positioning result.

3. The method of claim 1, wherein a weight between an input layer to a hidden layer of the RBF neural network is fixed to 1.

4. The method of claim 3, wherein a plurality of basis function centers of the radial basis functions are determined using a k-means clustering algorithm when training the RBF neural network.

The variance σ of the radial basis function is set as:

the weight W between the hidden layer and the output layer weight is:wherein x is_qFeature vector, x, representing the q-th input sample_iRepresenting the ith basis function center.

5. The method of claim 1, wherein the positioning information comprises pose information and position information of the drone.

6. The method according to claim 1, wherein in step S3, the interval for extracting the images collected by the onboard camera is: extracted once every ten frames.

Technical Field

The invention belongs to the technical field of unmanned aerial vehicle navigation positioning, and particularly relates to an unmanned aerial vehicle visual navigation positioning method based on an RBF (radial Basis function) network.

Background

The unmanned aerial vehicle comprehensive positioning system plays a crucial role in stability and integrity. The most common solution for positioning involves combining the Global Navigation Satellite System (GNSS) and the Inertial Navigation System (INS) within a multi-sensor fusion framework. In this case, GNSS is used as a compact and economical method to constrain the unbounded errors generated by the INS sensors during the positioning process. In fact, however, the INS integrates time in an iterative process of continuously acquiring data from a plurality of sensors to obtain an approximate drone position, and in this process, measurement errors generated by the sensors are rapidly accumulated and increase without limitation. Therefore, most drones use an Extended Kalman Filter (EKF) framework to fuse data from INS and GNSS, which combines the short-term accuracy of the inertial navigation system with the long-term accuracy of the global navigation satellite system, thereby effectively suppressing the positioning error. Therefore, the global navigation satellite system is widely used for various drones.

Despite the advantages of the global navigation satellite system, it has proven unreliable in many recorded cases. Outdoor scenes such as urban canyons, forests, jungles, and rainy regions also prove to be vulnerable to both intentional attacks and unintentional environmental disturbances. In addition to this, drones using global navigation satellite systems have proven to be vulnerable to signal spoofing on a number of occasions, and such attacks are now becoming a reality. A disadvantage of using global navigation satellite systems in drone navigation is the radio communication necessary to acquire positioning data, which radio communication systems are generally prone to usability problems, interference and signal changes. The root cause of using GNSS/INS fusion is to rely on global information obtained from GNSS to solve the local positioning problem. In order to solve these problems, suitable navigation sensors and new navigation algorithms are introduced to solve the problem of navigation and positioning of the drone when subjected to wireless communication interference and short-term or long-term GNSS/INS failures.

One popular method of reliably determining drone position in an outdoor environment where gnss rejection/gnss degradation is to use monocular 2D cameras in combination with vision-based techniques. These techniques fall into two categories: one is a technique using a priori knowledge of the environment and a technique that does not use a priori knowledge of the environment. In the field of visual navigation using a priori knowledge, map-based navigation techniques seem to be advanced, which match images taken by drones with previously flown high resolution landmark satellite images or landmark images, and the limitations of this solution include the need for a large database of geographic images, database access by network-connected onboard devices, and another important limitation is the need to know the starting point or predefined boundaries in advance. Therefore, the map-based solution has serious limitations, which prevent its application in practical scenes. The second category of vision-based techniques does not have this limitation because they do not require prior knowledge of the environment. This class of solutions includes visual measurement and simultaneous localization and mapping (SLAM), among others. In visual measurements, the motion of the drone is estimated by tracking features or pixels between successive images obtained from a monocular camera. However, even the most advanced monocular vision measurements are affected over time because the current location estimate is based on the previous location estimate, resulting in an accumulation of errors. With respect to visual measurements, SLAM solves the localization problem while building an environmental map. The map building requires multiple steps such as tracking, repositioning and loop closure, and this solution is always accompanied by heavy computation and memory usage.

Disclosure of Invention

The invention aims to: aiming at the existing problems, the RBF network-based visual navigation positioning method for the unmanned aerial vehicle is provided, and by acquiring the ground image feature descriptors in the navigation process of the unmanned aerial vehicle and using the RBF network classifier trained by the feature description sub data set to perform neighborhood search on the feature point descriptors of the currently acquired images, the optimal matching position of the current images is obtained, so that more accurate positioning information of the location of the unmanned aerial vehicle is estimated.

The invention discloses an unmanned aerial vehicle visual navigation positioning method based on an RBF network, which comprises the following steps:

step S1: setting an RBF neural network for matching the feature point descriptors of the image, and training the neural network;

wherein, the training sample is: in the navigation process of the unmanned aerial vehicle, images collected by an airborne camera are used; the feature vectors of the training samples are: feature point descriptors of the image obtained through ORB feature point detection processing;

step S2: constructing a visual database of the unmanned aerial vehicle during navigation:

in the navigation process of the unmanned aerial vehicle, images are collected through an airborne camera, ORB feature point detection processing is carried out on the collected images, descriptors of all feature points are extracted, and feature point descriptors of the current images are obtained; storing the feature point descriptors of the image and positioning information during image acquisition into a visual database;

step S3: unmanned aerial vehicle vision navigation positioning based on visual database:

based on a fixed interval period, extracting an image acquired by an airborne camera to serve as an image to be matched;

carrying out ORB feature point detection processing on the image to be matched, and extracting a descriptor of each feature point to obtain a feature point descriptor of the image to be matched;

inputting the feature point descriptor of the image to be matched into a trained RBF neural network, and performing neighborhood search to obtain the optimal matching feature point descriptor of the image to be matched in a visual database;

and obtaining the current visual navigation positioning result of the unmanned aerial vehicle based on the positioning information recorded in the database by the optimal matching feature point descriptor.

Further, step S3 further includes: detecting whether the similarity between the optimal matching feature point descriptor and the feature point descriptor of the image to be matched is smaller than a preset similarity threshold value or not; if so, obtaining a current visual navigation positioning result of the unmanned aerial vehicle based on the positioning information recorded in the database by the optimal matching feature point descriptor; otherwise, the navigation is continued based on the recently obtained visual navigation positioning result.

In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:

(1) the visual database only stores the feature point descriptor information of the image, so that the occupied space of a memory is reduced;

(2) under the condition of no reference image library, the visual database can be directly accessed to match images shot by the unmanned aerial vehicle;

(3) and (4) realizing feature descriptor neighborhood search based on the RBF network obtained by training, obtaining the best matching position, and estimating positioning information.

Drawings

FIG. 1 is a visual positioning overall system framework;

FIG. 2 is a flow chart of ORB feature point detection;

FIG. 3 is a schematic diagram of rough feature point extraction during ORB feature point extraction;

FIG. 4 is a schematic diagram of an RBF network architecture;

fig. 5 is a schematic diagram of a matching positioning process of the RBF network.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings.

According to the unmanned aerial vehicle navigation positioning method based on vision, the ground image feature descriptors of the current area where the unmanned aerial vehicle is located are collected, the feature point descriptors of the collected images are subjected to neighborhood search by using the RBF network classifier trained by the feature description sub data set, and the optimal matching position of the images is obtained, so that more accurate positioning information of the location of the unmanned aerial vehicle is estimated.

Referring to fig. 1, the method for positioning navigation of unmanned aerial vehicle based on vision mainly comprises two parts: firstly, data acquisition during outbound voyage and secondly positioning estimation during return voyage;

in the data acquisition part, acquiring images through a camera, extracting image frames from the camera, detecting characteristic points of each image, extracting a descriptor, discarding image data in the process, and keeping the characteristic point information of the image; repeating the processing of the descriptors of the extracted image frames, and storing descriptor information and positioning information into a visual database;

during positioning estimation processing, under the condition that GNSS signals are lost, images shot by a camera are extracted, descriptor extraction is also carried out, and an RBF network classifier is trained by using visual database information: then performing neighborhood search on the generated descriptors according to an RBF network classifier, and estimating an optimal matching position; and finally, estimating the positioning information of the current image according to the positioning information stored in the visual database at the optimal matching position.

The method comprises the following concrete implementation steps:

(1) and (6) collecting data.

The method comprises the steps of collecting images from an onboard camera, carrying out ORB (optimized Brief) feature point detection on each frame of image, extracting descriptors of each feature point, and then creating and storing a database entry, wherein the database entry consists of a feature point descriptor set extracted previously and corresponding positioning information. The positioning information is composed of attitude information and position information provided by an airborne equipment application program of the unmanned aerial vehicle, and the format or the property of the information is highly dependent on the specific application program.

(2) And (5) feature extraction.

ORB feature point detection uses the FAST (features from obtained segment test) algorithm to detect feature points on each level of the scale pyramid. And detecting a circle of pixel values around the candidate characteristic point based on the gray value of the image around the characteristic point, and if the gray value difference between enough pixel points in the area around the candidate point and the gray value of the candidate point is large enough (namely the gray value difference is larger than a preset threshold), considering the candidate point as a characteristic point.

The method comprises the following specific steps:

1) and detecting ORB characteristic points.

Referring to fig. 2, when detecting ORB feature points, first, FAST corner detection is performed on an input image; then, calculating a Harris corner response value from the selected FAST characteristic points by using a Harris corner measurement method; then according to the result of the angular point response value sorting, picking out N characteristic points with the maximum response values; then, calculating the direction of the ORB characteristic point by adopting a gray centroid method, and adopting BRIEF as a characteristic point description method; finally, each feature point generates 256-bit binary point pairs.

That is, the ORB features detect FAST feature points by using a FAST feature point detection method, and then calculate Harris corner response values from the selected FAST feature points by using a Harris corner measurement method, and pick the top N feature points with the largest response values.

Among them, of FAST characteristic pointsCorner response function f_CRFDefined as:

wherein epsilon_dAnd the threshold value is I (x), the pixel value of a pixel point in the neighborhood of the point to be measured, and I (p), the pixel value of the current point to be measured.

The sum of the corner response function values of the point to be measured and all the corresponding surrounding points is denoted as N, and when N is greater than the set threshold value, the point to be measured is the FAST feature point, and the threshold value is usually 12.

The specific processing flow for ORB feature point extraction is as follows:

the first step is as follows: and (5) roughly extracting the feature points. Selecting a point in the image as p, taking p as the center of circle and 3 pixels as radius, detecting the pixel values of the corresponding points with the position numbers of 1, 5, 9 and 13 on the circumference (as shown in fig. 3, one of the points includes 16 positions, when rough extraction is performed, four points on the circumference in the four directions of the upper, lower, left and right of the center of circle p are detected), and if the pixel value of at least 3 points in the 4 points is greater than or less than the pixel value of the p point, then the p point is considered as a feature point.

The second step is that: and removing local dense points. And (4) calculating by adopting a non-maximum inhibition algorithm, reserving the characteristic points at the maximum position, and deleting the rest characteristic points.

The third step: the scale invariance of the feature points. And establishing a pyramid to realize multi-scale invariance of the feature points. A scale factor scale (e.g. 1.2) and the pyramid level levels nlevels (e.g. 8 levels) are set. The original image is down-sampled into n levels of images according to the scale factor, and the relation between each level of down-sampled image I' and the original image I is as follows:

I’＝I/scale^k(k＝1,2,…,8)

the fourth step: rotational invariance of feature points. And calculating the direction of the characteristic point by adopting a gray scale centroid method, wherein the moment in the radius r range of the characteristic point is the centroid, and the vector formed between the characteristic point and the centroid is the direction of the characteristic point.

The vector angle theta of the feature point and the centroid C is the main direction of the feature point:

θ＝arctan(C_x,C_y)

wherein (C)_x,C_y) Representing the coordinates of the centroid C.

2) And generating a characteristic point descriptor.

ORB features use descriptor BRIEF as feature point description method. The BRIEF descriptor is composed of a binary string of length n, where n is 256 in this embodiment. The formula for calculating the binary value τ (p: x, y) of a certain bit in the descriptor is as follows:

wherein p (x) and p (y) are the respective gray levels of two points in a pair of points, and n is a feature descriptor f consisting of the pair of points_n(p) can be expressed as:

f_n(p)＝∑_1≤i≤n2^i-1τ(p:x,y)

constructing affine transformation matrix R_θMaking the descriptor rotationally invariant, a rotation-corrected version S of the generator matrix S is obtained_θ：

S_θ＝R_θS

Wherein the generator matrix S is n point pairs (x)_i,y_i) I is 1, 2n,

theta is the principal direction of the feature point.

Finally obtained feature point descriptor g_n(p,θ)＝f_n(p)|x_i,y_i∈S_θAnd 256-bit descriptors of the feature points are formed.

(3) And matching positioning based on the RBF neural network.

When the GNSS/INS signal of the unmanned aerial vehicle is unavailable, the system prompts the unmanned aerial vehicle to return to the home. And matching the image descriptor extracted in the return process with the descriptor previously inserted into the database by utilizing the unmanned aerial vehicle motion information stored in the characteristic database to obtain the positioning information. The matching positioning system based on the RBF neural network consists of network pattern training and pattern positioning. The concrete mode is as follows:

1) a training mode is set.

And setting a training mode, learning the training samples and providing a classification decision.

The RBF network only contains one hidden layer, the distance between an input value and a central vector is taken as an independent variable of a function, and a radial basis function is taken as an activation function. The local approximation approach can simplify the computational complexity, since for one input X, only some neurons respond, and others are approximately 0, and w of the response adjusts the parameter.

Referring to FIG. 4, the RBF neural network is composed of an input layer, a hidden layer and an output layer, wherein

An input layer, the transformation from the input space to the hidden layer space being nonlinear;

a hidden layer, neurons using radial basis functions as activation functions, the hidden layer to output layer spatial transformation being linear;

the output layer adopts neurons of linear functions and is a linear combination of the output of the neurons of the hidden layer;

the RBF network adopts RBF as the 'base' of the hidden unit to form a hidden layer space, and an input vector is directly mapped to the hidden space. After the center point is determined, the mapping relationship can be determined. The mapping from input to output of the network is nonlinear, the network output is linear for adjustable parameters, and the connection weight can be directly solved by a linear equation set, so that the learning speed is greatly increased, and the local minimum problem is avoided.

In the specific embodiment, the weight from the input layer to the hidden layer of the RBF neural network is fixed to 1, the transfer function of the hidden layer unit adopts a radial basis function, and the hidden layer neuron uses the layer weight vector w_iAnd the input vector X_iVector distance and deviation b between_iAfter multiplication, the input is the neuron activation function. Taking the radial basis function as a gaussian function, the output of the neuron is:

where x represents the input data, i.e. the input vector, x_iσ is a function width parameter, centered on the basis function, used to determine the input vector to each radial basis layer neuron.

2) And (4) RBF neural network learning.

The RBF network has three parameters to learn: center x of basis function_iAnd the variance σ and the weight w between the hidden layer and the output layer.

i. Determining a basis function center x_i。

Feature descriptor vectors of images acquired by a camera are used for generating a feature database, and a k-means clustering algorithm is adopted to determine the center x of a kernel function_iRandomly selecting I different samples from the training samples as initial center x_i(0) Random input of training sample X_kDetermining which center the training sample is closest to, finding the center that satisfies:

i(X_k)＝argmin||X_k-x_i(n)||

wherein I1, 2_i(n) denotes the ith center of the radial basis function at the nth iteration, and the iteration step number n is set to 0. The basis function center is adjusted by the following formula:

where γ is the learning step size, 0< γ < 1.

Namely, the center of the basis function is continuously updated by iteration training, and the updating formula is as follows: x is the number of_i(n+1)＝x_i(n)+γ[X_k(n)- x_i(n)]When the change of the processing result of the latest two iteration updates does not exceed the preset threshold, the update is stopped (learning is finished), and x is considered to be_i(n +1) is approximately equal to x_i(n) taking the updated basis function center of the last time as a final iteration training output result x_i(I ═ 1,2, …, I). Otherwise, n is n + 1.

Determining the variance σ of the basis function.

After the RBF neural network center is determined, the width is expressed as:

wherein M is the number of hidden layer units, d_maxIs the maximum distance between the selected centers.

Determining a hidden-layer-to-output-layer weight w.

The connection weight of the hidden layer to the output layer unit is calculated by adopting a least square method, namely

In the formula, g_qiWeight, X, representing the center of the vector and basis function of the qth input sample_qIs a vector of the qth input sample, q 1, 2.

3) And matching and positioning.

In view of the time sequence of images shot by the unmanned aerial vehicle, during return voyage, images shot by the camera can be extracted at intervals of fixed frames and features are extracted, for example, every ten frames are extracted to extract features, feature descriptor vectors are generated, neighborhood search is performed by using a trained RBF network classifier, and an optimal matching position is obtained, namely, an optimal matching result of the currently extracted feature descriptors and the feature descriptors of the shot images in the process of departure (from a departure point to a destination voyage) stored in a database is obtained based on the trained RBF network classifier, whether the similarity between the currently extracted descriptors and the optimal matching result does not exceed a preset similarity threshold value is detected, and if yes, the position of the currently optimal matching result is used as a current position estimation result of the unmanned aerial vehicle during return voyage, and positioning information is obtained.

Furthermore, the error compensation of the navigation system can be carried out on the obtained position estimation result to obtain the positioning information. If the similarity between the position and the optimal matching position is lower than a predefined similarity threshold value, the position can be defined as unknown, the ground image of the area where the unmanned aerial vehicle is located is continuously acquired, and navigation is further performed according to the speed and posture information of the unmanned aerial vehicle and the positioning result obtained last time.

The error formula of the navigation system is as follows:

wherein, the symbol

Indicating the position estimation result (the position of the current best match result),

the final position estimation result (or position estimation result) representing the latest previous n times

) J ═ 1, 2.. times, n, n are preset values. That is, the average standard error of the obtained positioning results of the latest times is used as the current compensation amount, and the error compensation is performed on the current position estimation result to obtain the current final position estimation result

Referring to fig. 5, the RBF network matching location processing procedure of the present invention is:

firstly, randomly sampling feature descriptor information stored in a visual database; and training the two-level system data of the feature descriptors obtained by sampling (training RBF network): setting a training mode, and determining an RBF center by adopting K-means clustering; determining the RBF width d according to the obtained center; determining a connection weight from a hidden layer to an output layer by adopting a least square method according to the RBF center and the width, and finally determining an RBF network structure;

and obtaining an RBF network structure according to training, performing neighborhood matching on the feature point descriptors generated by the acquired images during return flight, judging the optimal matching position, and finally performing positioning estimation on the current image during return flight according to the stored positioning information.

In summary, the present invention starts with the acquisition of images by the camera during the data collection process, performs feature point detection on the images by using the ORB feature point extraction technique, and extracts descriptors for each keypoint. A database entry is created and stored, the database entry consisting of the previously extracted descriptor and the positioning information. Wherein the positioning information comprises attitude information and position information of the unmanned aerial vehicle. While the parameters to be solved in the RBF network are mainly 3: including the center of the basis function, the variance, and the weight from the hidden layer to the output layer. The method adopts a self-organizing selection center learning method, and solves the center and variance of the underlying layer basis function by using an unsupervised learning process in the first step; and in the second step, a supervised learning process is used, and finally, a weight value between the hidden layer and the output layer is directly obtained by using a least square method. In order to reduce the similarity of adjacent images, one image is extracted at intervals of fixed frames, then key points are detected, and descriptors are extracted for each key point by using the same characteristic point extraction method as the data collection process. And (4) obtaining the closest distance between the current image and the descriptor previously inserted into the database by using an RBF network, and finding the optimal matching position. And estimating the positioning information of the current image according to the optimal matching position.

While the invention has been described with reference to specific embodiments, any feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise; all of the disclosed features, or all of the method or process steps, may be combined in any combination, except mutually exclusive features and/or steps.