阅读说明：本技术 一种基于fpga的实时遥感影像地理参考方法 (Real-time remote sensing image geographical reference method based on FPGA ) 是由 刘德全 周国清 张荣庭 舒磊 王凡 周详 黄景金 于 2019-08-21 设计创作，主要内容包括：本发明公开一种基于FPGA的实时遥感影像地理参考方法,包括如下步骤：S1、选定地理参考影像坐标构建控制点对坐标关系模型；S2、从控制点对坐标关系模型中选取相应的坐标参数实现待校正影像坐标(x<Sub>g</Sub>,y<Sub>g</Sub>)转换；S3、判断待校正影像坐标(x<Sub>g</Sub>,y<Sub>g</Sub>)是否为正数,如果满足进入下一步；否则结束；S4、判断待校正影像坐标(x<Sub>g</Sub>,y<Sub>g</Sub>)是否在选定地理参考图像范围,如果满足进入下一步；否则结束；S5、对待校正影像坐标(x<Sub>g</Sub>,y<Sub>g</Sub>)通过最邻近插值算法获得灰度值重采样输出具有地理坐标的遥感影像；该方法基于FPGA作为硬件实现,设计硬件语言为Verilog HDL语言,算法代码仿真有效,并实现了地理参考影像的实时输出。(The invention discloses a real-time remote sensing image geographical reference method based on an FPGA (field programmable gate array), which comprises the following steps of: s1, selecting the geographic reference image coordinates to construct a control point pair coordinate relation model; s2, selecting corresponding coordinate parameters from the control point pair coordinate relation model to realize the image coordinate (x) to be corrected g ,y g ) Converting; s3, determining the coordinates (x) of the image to be corrected g ,y g ) If the number is positive, entering the next step if the number is positive; otherwise, ending; s4, determining the coordinates (x) of the image to be corrected g ,y g ) Whether the geographic reference image is in the selected geographic reference image range or not, and if yes, entering the next step; otherwise, ending; s5, image coordinate (x) to be corrected g ,y g ) Obtaining a gray value resampling through a nearest neighbor interpolation algorithm and outputting a remote sensing image with geographic coordinates; the method is realized by taking FPGA as hardware, designing the hardware language into Verilog HDL language, realizing effective algorithm code simulation and realizing geographic referenceAnd outputting the image in real time.)

1. A real-time remote sensing image geographical reference method based on FPGA is characterized by comprising the following steps:

s1, selecting the geographic reference image coordinates to construct a control point pair coordinate relation model; namely:

Δ_a＝(A^TA)^-1(A^TL_x)

Δ_b＝(A^TA)^-1(A^TL_y)；

s2, selecting corresponding coordinate parameters from the control point pair coordinate relation model to realize the image coordinate (x) to be corrected_g,y_g) Converting;

s3, determining the coordinates (x) of the image to be corrected_g,y_g) If the number is positive, entering the next step if the number is positive; otherwise, ending;

s4, determining the coordinates (x) of the image to be corrected_g,y_g) Whether the geographic reference image is in the selected geographic reference image range or not, and if yes, entering the next step; otherwise, ending;

s5, image coordinate (x) to be corrected_g,y_g) By passingAnd obtaining a gray value by the nearest adjacent interpolation algorithm, resampling and outputting the remote sensing image with the geographic coordinates.

2. The method according to claim 1, wherein the control point in the step S1 is used for establishing a coordinate relation model:

1.1, establishing a relation between the coordinates (X, Y) of the geographic reference image and the coordinates (X, Y) of the earth through a quadratic polynomial;

1.2, selecting control points to establish a coordinate matrix model corresponding to the coordinates (X, Y) of the geographic reference image and the coordinates (X, Y) of the earth;

(A^TPA)Δ_a＝A^TPL_x

wherein A is a control point geodetic coordinate matrix, P is a control point weight matrix, and 1, delta is generally taken_aIs a_iCoefficient matrix, L_xThe image coordinate to be corrected is the control point.

3. The method for georeferencing remote sensing images in real time based on FPGA of claim 1, wherein said step S2 is implemented by implementing coordinates (x) of images to be corrected_g,y_g) And (3) conversion process:

2.1, outputting the selected image area through the coordinates of four corner points of the image to be corrected by the following formula:

X_min＝min(X_ul,X_ur,X_lr,X_ll)

X_max＝max(X_ul,X_ur,X_lr,X_ll)

Y_min＝min(Y_ul,Y_ur,Y_lr,Y_ll)

Y_max＝max(Y_ul,Y_ur,Y_lr,Y_ll)

2.2, converting the scanning coordinates of the selected image area;

(1) the row (M) and column (N) coordinate ranges of the selected image area are calculated by the following formula:

M＝(X_max-X_min)/X_GSD+1

N＝(Y_max-Y_min)/Y_GSD+1

wherein, X_GSDAnd Y_GSDRespectively is a ground sampling interval;

(2) and obtaining the geodetic coordinates of the geographic reference image according to the scanning coordinates according to the following formula:

X_g＝X_min+X_GSD(x'-1)

Y_g＝Y_max-Y_GSD(y'-1)

where X 'and y' are the scan coordinates (X '═ 1,2,3, …, M; y' ═ 1,2,3, …, N), (X)_g,Y_g) Is the corresponding geodetic coordinate.

Technical Field

The invention relates to the field of remote sensing image processing, in particular to a real-time remote sensing image geographical reference method based on an FPGA.

Background

With the continuous progress of the technology, remote sensing image data are more and more widely applied to natural disaster and positioning systems, which need to provide geographical coordinates quickly and in real time, however, the traditional remote sensing image data do not have geographical coordinates, and the traditional processing system cannot meet the requirement of real-time performance, which is mainly shown in that: (1) the primary remote sensing image processed and distributed by the ground receiving station generally has certain geometric distortion due to the influence of various factors, mainly represented by displacement, rotation, scaling, affine, bending and higher-order bending, or represented by squeezing, stretching, twisting or offsetting of an image element relative to the actual position of the ground. (2) The general remote sensing images have no geographic coordinates. Therefore, high-precision geometric correction is one of the problems that must be solved before remote sensing images are put into use, and it is of great practical significance to research various geometric correction theories and develop related geometric correction algorithms.

When image data obtained by photogrammetry, remote sensing, map scanning and the like or point cloud data obtained by scanning of a laser scanner are applied to an imaging or geographic information system, the image data needs to be matched with real geographic coordinates, namely, the image is corrected to the earth, the image is flattened like a map, and the process with geodetic coordinates is called geographic reference and is also called geographic coordinate calibration. The geographic reference is one of important means for geometric correction, and the process should comprise three processes of selecting a proper mathematical variant model, coordinate transformation and pixel gray value resampling.

The geographic reference is divided into indirect geographic reference and direct geographic reference, and the indirect geographic reference is the matching of geographic coordinates realized by using ground control points; and the direct geographic reference is to directly complete the calculation of geographic coordinates by using inertial navigation on a satellite and a global satellite navigation system. However, for general users, relevant parameters of the satellite are not disclosed, so that the design is mainly based on the implementation of an indirect geographical reference algorithm of the control point.

The general processing process of the remote sensing image is to receive downloaded data through a ground receiving station, then preprocess the image to form remote sensing data of different levels, and then process the remote sensing data according to different user requirements and send the remote sensing data to a user.

Disclosure of Invention

Aiming at the problems in the prior art, the application provides a geographic reference algorithm of a real-time remote sensing image based on an FPGA (field programmable gate array). A quadratic polynomial equation is utilized to establish the relation between a control point and an original image coordinate, then geographic coordinate matching is carried out point by point, and finally a nearest neighbor interpolation method is utilized to carry out gray value resampling so as to realize the remote sensing image production with geographic coordinates; the method has real-time performance and low power consumption.

In order to solve the problems in the prior art, the invention adopts the following technical scheme:

the invention discloses a real-time remote sensing image geographical reference method based on an FPGA (field programmable gate array), which comprises the following steps of:

s1, selecting the geographic reference image coordinates to construct a control point pair coordinate relation model;

s2, selecting corresponding coordinate parameters from the control point pair coordinate relation model to realize the image coordinate (x) to be corrected_g,y_g) Converting;

s3, determining the coordinates (x) of the image to be corrected_g,y_g) If the number is positive, entering the next step if the number is positive; otherwise, ending;

s4, determining the coordinates (x) of the image to be corrected_g,y_g) Whether the geographic reference image is in the selected geographic reference image range or not, and if yes, entering the next step; otherwise, ending;

s5, image coordinate (x) to be corrected_g,y_g) And obtaining a gray value through a nearest neighbor interpolation algorithm, resampling the gray value and outputting a remote sensing image with geographic coordinates.

The process of establishing the coordinate relation model by the control point in the step S1:

1.1, establishing a relation between the coordinates (X, Y) of the geographic reference image and the coordinates (X, Y) of the earth through a quadratic polynomial equation;

1.2, selecting control points to establish a coordinate matrix model corresponding to the coordinates (X, Y) of the geographic reference image and the coordinates (X, Y) of the earth;

(A^TPA)Δ_a＝A^TPL_x

wherein A is a control point geodetic coordinate matrix, P is a control point weight matrix, and 1, delta is generally taken_aIs a_iCoefficient matrix, L_xThe image coordinate to be corrected is the control point.

The step S2 realizes the coordinate (x) of the image to be corrected_g,y_g) And (3) conversion process:

2.1, outputting the selected image area through the coordinates of four corner points of the image to be corrected by the following formula:

X_min＝min(X_ul,X_ur,X_lr,X_ll)

X_max＝max(X_ul,X_ur,X_lr,X_ll)

Y_min＝min(Y_ul,Y_ur,Y_lr,Y_ll)

Y_max＝max(Y_ul,Y_ur,Y_lr,Y_ll)

2.2, converting the scanning coordinates of the selected image area;

(1) the row (M) and column (N) coordinate ranges of the selected image area are calculated by the following formula:

M＝(X_max-X_min)/X_GSD+1

N＝(Y_max-Y_min)/Y_GSD+1

wherein, X_GSDAnd Y_GSDRespectively is a ground sampling interval;

(2) and obtaining the geodetic coordinates of the geographic reference image according to the scanning coordinates according to the following formula:

X_g＝X_min+X_GSD(x'-1)

Y_g＝Y_max-Y_GSD(y'-1)

where X 'and y' are the scan coordinates (X '═ 1,2,3, …, M; y' ═ 1,2,3, …, N), (X)_g,Y_g) Is the corresponding geodetic coordinate.

Advantageous effects

The invention is realized by taking FPGA as hardware, the hardware language is designed into Verilog HDL language, and the algorithm code simulation is effective. The design scheme has the characteristics of high running speed and low power consumption, and can realize the real-time output of the geographic reference of the remote sensing image.

Drawings

FIG. 1 is a block diagram of a real-time geographic reference structure of a remote sensing image of a quadratic polynomial based on an FPGA;

FIG. 2 is a block diagram of a quadratic polynomial equation model solution structure;

FIG. 3 is a block diagram of the structure of the original image coordinates and the output image coordinates;

FIG. 4 is a flow chart of a coordinate transformation and nearest neighbor interpolation algorithm;

FIG. 5 is a diagram of a nearest neighbor interpolation structure;

FIG. 6 is a schematic circuit diagram of a nearest neighbor interpolation algorithm;

FIG. 7 is a power consumption simulation diagram of the FPGA;

FIG. 8 is an original SPOT image;

FIG. 9 SPOT images with geographic references;

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The first step is as follows: second order polynomial coefficient solution

Firstly, a coordinate conversion equation between an image to be corrected and a reference image is established, at present, a strict imaging model, a polynomial model and a rational function model are mainly used.

A block diagram of a structure for realizing real-time geographic reference of a remote sensing image based on a second-order polynomial of an FPGA is shown in fig. 1, and mainly includes three main parts, namely establishment of a second-order polynomial model, coordinate conversion and nearest neighbor interpolation, which are respectively described in detail below.

The quadratic polynomial equation is shown in formula 1:

wherein, (X, Y) is the coordinates of the image to be georeferenced, (X, Y) is the geodetic coordinates of the corresponding homonymous point, a_iAnd b_i(i ═ 0,1, …, 5) are coefficients of a quadratic polynomial, totaling 12 unknowns. Theoretically, the unknown equation coefficients can be solved by only 6 point pairs, but it is necessary to ensure that the 6 points should be uniformly distributed on the image, which is generally achieved greatly, and therefore, in order to ensure that the longitude of the whole geographical reference process is high, 10 control point pairs are selected. Let 10 control point pair coordinates be expressed as: (x)₁,y₁,X₁,Y₁),(x₂,y₂,X₂,Y₂),…,(x₁₀,y₁₀,X₁₀,Y₁₀) Due to a_iAnd b_iIs completely the same as the coefficient solving process of (a), in this case_iThe coefficient solution is illustrated. The control point pairs are substituted into formula 1 and converted into a matrix form as shown in formula 2:

according to the principle of least squares, equation (2) can be written as equation (3):

(A^TPA)Δ_a＝A^TPL_x (3)

wherein A is a control point geodetic coordinate matrix and P isThe weight matrix of the control points, typically taken to be 1, Δ_aIs a_iCoefficient matrix, L_xThe coordinates of the image to be corrected for the control points are respectively shown as follows.

Δ_a＝[a₀ a₁ a₂ a₃ a₄ a₅]^T，L_x＝[x₁ x₂ x₃ x₄ x₅ …x₁₀]^T。

Then, a in the formula (3)_iThe coefficient can be obtained by equation (4).

Δ_a＝(A^TA)^-1(A^TL_x) (4)

By the same token,. DELTA._b＝(A^TA)^-1(A^TL_y) The value of (c).

The block diagram of the quadratic polynomial coefficient solving structure is shown in fig. 2. Coefficient of time a₀，…,a₅And b₁，…,b₅After the calculation, the equation (1) becomes a binary quadratic equation of the corresponding equation of the reference image coordinates (X, Y) and the geodetic coordinates (X, Y). A set of unique influence coordinates (X, Y) can be obtained every time a pair of geodetic coordinates (X, Y) is given, and coordinate mapping is realized, and the specific mapping process is as follows.

The second step is that: converting coordinates;

after the quadratic polynomial coefficient is solved, the relation of coordinates between the reference image and the homonymous point of the image to be corrected can be established according to a polynomial equation, then coordinate conversion is carried out pixel by pixel, and finally the image coordinate is converted into the maximum and maximum, so that the original image has both image coordinates and geodetic coordinates.

(1) Region of output image

In order to realize coordinate transformation, firstly, the output range of the corrected image is estimated, and the coordinates of four corner points of the image to be corrected are set as follows: (x)_ul,y_ul),(x_ur,y_ur),(x_lr,y_lr) And (x)_ll,y_ll) (wherein, the first letter: l represents lower, lower; u, represents up, top. The second letter: r represents right, right; l stands for left, and four sets of coordinates (X) can be obtained in the formula (1)_ul,Y_ul), (X_ur,Y_ur),(X_lr,Y_lr) And (X)_ll,Y_ll) Respectively selecting the minimum and maximum coordinates in the coordinates as the range coordinates of the corrected output image, namely:

the positional relationship of the coordinates is shown in fig. 3, in which 3(a) is a coordinate diagram of four vertices of the original image, and 3(b) is the output range coordinates of the corrected image.

(2) Conversion of output image and scanning coordinate

The coordinate ranges of the row number (M) and the column number (N) of the corrected image are obtained by the formula (6):

in the formula, X_GSDAnd Y_GSDRespectively ground sampling interval.

The geodetic coordinates of the geodetic reference image can be determined from the scanning coordinates by equation (7).

Where X 'and y' are the scan coordinates (X '═ 1,2,3, …, M; y' ═ 1,2,3, …, N), (X)_g,Y_g) Is the corresponding geodetic coordinate. The geodetic coordinates (X) to be obtained_g,Y_g) The coordinates (x) of the corresponding image to be corrected can be determined by substituting the formula (1)_g,y_g) The next step is to interpolate the geodetic coordinates (X) by interpolation_g,Y_g) And the coordinates (x) of the image to be corrected_g,y_g) A one-to-one correspondence relationship is established,and endowing the image to be corrected with the corresponding geodetic coordinates with the maximum gray value.

The flow chart of parallel coordinate transformation and nearest neighbor interpolation based on FPGA is shown in FIG. 4.

The third step: nearest neighbor interpolation.

In the coordinate conversion by the quadratic polynomial equation, sometimes the calculated maximum value may not exactly fall on the grid of the image to be corrected, and in order to ensure that the calculated coordinate falls on the grid line, the point which is not on the grid line must be approximately moved to the grid nearest to the point, and this process is completed by interpolation operation, and the nearest neighbor interpolation algorithm is adopted in this document. Therefore, the second and third steps can be combined and completed simultaneously.

The coordinate conversion relation between the image coordinate and the geodetic coordinate is established through a quadratic polynomial equation, the solving of the quadratic polynomial coefficient can be completed by 6 pairs of coordinate points distributed uniformly theoretically by utilizing the known ground control point data pair solving power quadratic polynomial coefficient, but the 6 accurate coordinate points cannot be found practically, so that the actually adopted coordinate point pairs are all larger than the theoretical point logarithm, and the method is realized by adopting 10 pairs of coordinate points in order to compromise the limited resources of the FPGA and the accuracy of data solving.

The coordinates (x) of the image to be corrected calculated by the equation (7)_g,y_g) Possibly not an integer, and in this case Interpolation is required, and Nearest Neighbor Interpolation (NNI) is used in this patent. The NNI algorithm principle is shown in FIG. 5. The coordinates of the image to be corrected are (i + u, j + v) (where i, j is an integer part and u, v represents a fractional part), and the 4 pixel coordinates nearest to the coordinates are (i, j), (i +1, j), (i, j +1) and (i +1, j +1), respectively. Then, the NNI algorithm can be represented by equation (8).

Wherein I (I + u, j + v), I (I, j), I (I +1, j), I (I, j +1) and I (I +1, j +1) represent gray values of the respective coordinates (I, j), (I +1, j), (I, j +1) and (I +1, j + 1).

The nearest neighbor interpolation circuit diagram based on the FPGA is shown in fig. 6.

Experiment:

the method comprises the steps of using Xilinx Vivado2014 environment to design hardware language under a Windows system, comparing time sequence simulation with MATLAB generation correction image data, wherein simulation data are consistent, an original SPOT (TM) remote sensing graph is shown in figure 8, a corrected remote sensing graph with geographic reference information is shown in figure 9, and FPGA resource consumption of correction model solving, coordinate conversion and nearest neighbor interpolation algorithm is shown in table 1. Fig. 7 is a power consumption simulation diagram of the system. In conclusion, the system realizes the real-time low-power-consumption geographic reference algorithm of the second-order polynomial remote sensing image.

TABLE 1(a) coordinate transformation resource consumption List