阅读说明：本技术 嵌入矢量红线数据的自然资源实时视频监测方法及系统 (Real-time video monitoring method and system for natural resources embedded with vector red line data ) 是由 邵振峰 李从敏 于 2020-05-21 设计创作，主要内容包括：一种嵌入矢量红线数据的自然资源实时视频监测方法及系统,包括获取自然资源矢量红线数据和对应的高分辨率遥感影像,获取摄像头初始画面图像；将矢量红线数据叠加到高分辨率遥感影像上,借助遥感影像语义信息,在视频图像和矢量红线数据上识别同名点,计算几何映射关系；将矢量红线数据映射到实时视频,获得视频监控区域；通过获得在尺度和空间均匀分布的特征点,对监控视频相邻帧图像进行匹配获得同名点；根据同名点得到图像间几何变换关系,将前一帧图像上的监控区域范围映射到后一帧图像,实现图像帧之间监控区域关联；根据几何关系,在监控视频上进行面积量测,并支持基于确定的视频区域,进行后续相关目标运动分析,实现区域自动预警报警。(A natural resource real-time video monitoring method and system with embedded vector red line data comprises the steps of obtaining the vector red line data of natural resources and a corresponding high-resolution remote sensing image, and obtaining an initial picture image of a camera; superposing the vector red line data to the high-resolution remote sensing image, identifying homonymous points on the video image and the vector red line data by means of semantic information of the remote sensing image, and calculating a geometric mapping relation; mapping the vector red line data to a real-time video to obtain a video monitoring area; matching adjacent frame images of the monitored video to obtain homonymous points by obtaining characteristic points uniformly distributed in scale and space; obtaining a geometric transformation relation between the images according to the same name points, and mapping a monitoring area range on the previous frame of image to the next frame of image to realize the association of the monitoring areas between the image frames; according to the geometric relation, area measurement is carried out on the monitoring video, and the subsequent relevant target motion analysis is carried out based on the determined video area, so that automatic area early warning and alarming are realized.)

1. A real-time video monitoring method for natural resources embedded with vector red line data is characterized by comprising the following steps:

step 1, acquiring natural resource vector red line data and a corresponding high-resolution remote sensing image of a certain area with the same projection coordinate system, and simultaneously acquiring an initial picture image of a PTZ camera;

step 2, superposing the vector red line data to the high-resolution remote sensing image, identifying homonymy points on the video image and the vector red line data respectively by means of semantic information of the remote sensing image, and then calculating a geometric mapping relation between the vector red line data and the monitoring video by adopting a least square method;

step 3, mapping the vector red line data to a real-time video according to the mapping relation calculated in the step 2 to obtain a video monitoring area;

step 4, matching adjacent frame images of the monitoring video by obtaining feature points uniformly distributed in scale and space, so as to obtain homonymous points on the two images;

step 5, obtaining a geometric transformation relation between the two images according to the homonymy points obtained in the step 4, and mapping the monitoring area range on the previous frame of image to the next frame of image according to the relation to realize the association of the monitoring area between the monitoring video image frames;

and 6, according to the geometric relationship established in the step, carrying out area measurement on the monitoring video, and supporting the subsequent relevant target motion analysis based on the determined video area, thereby realizing the automatic early warning and alarming of the specific area.

2. The real-time video monitoring method for natural resources embedded with vector red line data according to claim 1, characterized in that: in step 2, modeling the geometric mapping relation between the monitoring video image and the remote sensing image into projection transformation, and using a homography matrix H₀The method comprises the steps that at least 4 non-collinear same-name points are identified on two images, and the mutual mapping relation between a monitoring video image and a remote sensing image is quickly established;

corresponding homonymous points q (X, Y) on the remote sensing image are set to be corresponding to the monitoring video image I₀At any point p (x, y) above, the equation is satisfied as follows,

wherein the homography matrix H₀The expression of (a) is as follows,

3. the real-time video monitoring method for natural resources embedded with vector red line data according to claim 1, characterized in that: in step 3, plane vector data { P is set₁,P₂,...,P_nFor any one of the faces P_iThe coordinates are expressed as { (X)_i1,Y_i1),(X_i2,Y_i2),...,(X_im,Y_im)}，

For arbitrary vertex (X)_ij,Y_ij) I 1,2, n, j 1,2, m is the number of planes and m is the number of points, and the coordinates in the surveillance video are (x) according to the geometric relationship established in step 2_ij',y_ij'), calculated as follows,

4. the real-time video monitoring method for natural resources embedded with vector red line data according to claim 1, characterized in that: in the step 4, the feature points uniformly distributed in the scale and space are obtained by determining the number of the feature points extracted from each layer of pyramid image according to the scale coefficient in the aspect of scale; in the aspect of images, performing uniform grid division on the images, and determining the quantity of feature points extracted from each grid; then, feature points are screened according to the uniqueness and density of the feature points.

5. The method of claim 4, wherein the real-time video monitoring method for natural resources embedded with vector red line data comprises: in the aspect of scale, the number of the feature points extracted from the pyramid image of each layer is determined according to the scale coefficient, the implementation mode is as follows,

suppose that the total number of feature points extracted from an image is N_totalAnd the weight of each layer of image in each group of pyramid images is W (o, l), so that the number N of feature points extracted from each layer of image is N_scaleThe expression of (o, l) is,

N_scale(o,l)＝N_total×W(o,l)

wherein, o is the mark number of the Gaussian pyramid, and l is the mark number of the image layer;

the weight of each scale image is defined as follows,

wherein, SC_olThe scale coefficient, SC, of the image of the l layer in the o group of Gaussian pyramids₁₁Scale coefficient, r, representing the level 1 image in the set 1 Gaussian pyramid₀Is an intermediate parameter; SC (Single chip computer)_olThe expression is as follows:

wherein σ₀As an initial scale parameter, N_oNumber of pyramid images generated, N_sThe number of layers of each group of pyramids, K is a constant,

6. the method according to claim 5, wherein the real-time video monitoring method for natural resources embedded with vector red line data comprises: in the aspect of the image, the image is uniformly meshed, the number of the characteristic points extracted from each mesh is determined, the realization mode is as follows,

the quantity of the feature points extracted from each layer of image of the pyramid is assumed to be N_scale(o, l) mapping the layerImage uniform division into n²Number of feature points to be extracted in each grid, kth grid, N _ grid_kThe way of calculating (a) is as follows,

N_grid_k＝N_scale(o,l)×f_k

wherein f is_kAs grid weight coefficients, from the entropy E of the information in the grid area_kThe number N of the characteristic points_kAnd the contrast mean C of the feature points_kDetermining that the expression is:

wherein, w_e、w_nAnd w_cAre respectively information entropy E_kThe number N of the characteristic points_kContrast mean value C of feature points_kWeight coefficient of (d), w_e+w_n+w_c＝1。

7. The method of claim 6, wherein the real-time video monitoring method for natural resources embedded with vector red line data comprises: in step 4, the selected N _ grid is screened out in each grid by adopting the following strategy_kThe number of the characteristic points is one,

step a, calculating the contrast of each feature point, sorting the feature points from high to low according to the contrast value, and then selecting the top 3 × N _ grid_kThe characteristic points are used as pre-selected characteristic points;

b, accurately determining the positions and the scales of the feature points by adopting an SIFT method, and removing edge response points in the feature points obtained in the step a;

step C, calculating the information entropy of the feature points in the neighborhood with the feature points as the center, and ordering the feature points from large to small according to the information entropy to obtain an initial feature point set C_int；

Step d, sequentially taking point sets C_intRespectively calculating Euclidean distance between the point and the rest characteristic points, if the minimum distance is still larger than the corresponding threshold value, keeping the point to a target point set C_setFrom (1) to (C)_setNumber of middle feature points and N _ grid_kUntil equal.

8. A real-time video monitoring method for natural resources embedded with vector red line data according to claim 1,2, 3, 4, 5, 6 or 7, characterized in that: in step 6, for the polygon on the Mth frame image of the current monitoring video, the area measurement result is obtained by adopting the following method,

firstly, determining the geometric mapping relation between the current video frame image M and GIS data, and for any point P on the video frame image M_M(x_M,y_M) The homography matrix with the previous frame image is H_M-1Inversion of homography H_M-1 ^-1Then corresponds to the coordinate P on the M-1 frame image_M-1(x_M-1,y_M-1)＝H_M-1 ^-1P_M(x_M,y_M) And so on, calculating the coordinate P in the projection coordinate system_M(X_M,Y_M)＝H₀ ^-1H₁ ^-1...H_M-1 ^-1P(x_M,y_M)；

Then, according to the relation, the polygon coordinates on the video frame image are subjected to inverse calculation to obtain the coordinates corresponding to the projection coordinate system;

and finally, calculating the area of a polygon by using the cross product of the vectors to obtain an area measurement result.

9. A real-time video monitoring system of natural resources of embedding vector red line data which characterized in that: a real-time video monitoring method of natural resources for embedding vector red line data according to claims 1-8.

Technical Field

The invention belongs to the technical field of image processing, and relates to a natural resource real-time video monitoring method and system with embedded vector red line data.

Background

Ecological red lines, permanent basic farmlands and urban boundaries are three bottom lines which need strict defense in ecological environment construction. At present, the phenomena of damaged fertile farmlands, disorder and excessive cutting of forest resources, non-construction of urban development areas and the like are frequent, the problem that national natural resources are abused is increasingly prominent, and how to support accurate, real-time and quick efficient monitoring of the natural resources through technical means is an important and urgent matter.

With the rapid development of sensor technology, video compression processing technology and network communication technology, remote video monitoring of these natural resources has become possible by using modern technological means. If the technology suitable for practical application exists, relevant users can find illegal behaviors in the field of the homeland resources in time by looking up the monitoring camera, and corresponding places are treated immediately. However, in the existing mode, a large amount of manual troubleshooting real-time monitoring systems are still required, and automatic monitoring of natural resources in a designated area cannot be achieved. How to monitor the designated area range in the video field by using the image processing technology to realize the automatic early warning and alarm of the area is the fundamental way to really realize the video monitoring of natural resources. Therefore, the invention provides a natural resource real-time video monitoring method and system with embedded vector red line data.

The common monitoring camera can only monitor a fixed position, has limited visual field and is not suitable for monitoring natural resources in the type. The PTZ (Pan-Tilt-Zoom, security monitoring equipment) camera with the Pan-Tilt control function and capable of adjusting the camera to rotate, translate and stretch is more suitable for the scene. The definition of the natural resource distribution area is basically stored in the form of a vector file, and how to embed GIS graphic data in the form of vectors into a real-time PTZ monitoring video image is a problem worthy of research. Based on the area, the area can be analyzed, such as area measurement, and the behavior of people can be detected and prejudged, so that automatic early warning and alarming of illegal activities can be realized. Here the face mainly comprises two parts: (1) mapping the vector data into a video picture; (2) when the camera is rotated, the vector data needs to be remapped, so that the relative position of the vector data in different video pictures is kept unchanged.

Vector data is a graph and visually displays geometric structure information, attribute information of the vector data is stored in a text form, and the geometric relation between a vector structure and a monitoring video image is difficult to directly establish, so that the establishment of the mapping relation between the vector structure and the monitoring video image needs to be assisted by a high-resolution remote sensing image. At present, in the research of the fusion of the GIS data and the surveillance video, two methods can be roughly classified: homography matrix based methods and camera view intersection with DEM based methods. The former is to solve the geometric transformation relation between the monitoring video and the remote sensing image through homonymous feature points, generally described by a homography matrix, and is suitable for a plane or an approximate plane area. The latter is to blend the monitoring video into the 3DGIS, and map the video image into the 3DGIS based on the real world coordinate system, so as to realize the unification of the two. The method needs to acquire the position and attitude information of the camera during imaging and a high-precision DSM (digital Surface model), and is suitable for images of small scenes. For a common monitoring camera, the position parameters and the attitude parameters during photographing can be acquired, but for a PTZ camera, the acquired parameter information is not accurate when the camera is controlled by a holder. In addition, large scale fine DSMs are expensive to acquire.

A video may be understood as a collection of a series of images. When the camera rotates to obtain the monitoring picture, the camera can acquire images from different visual angles, and certain geometric deformation exists among the images due to the problem of the acquisition angle. At present, a method based on feature matching is widely applied to the calculation of geometric relationships between images, wherein the SIFT algorithm is widely concerned due to the image invariance kept in the aspects of rotation and scale, but when the geometric distortion is large, a satisfactory result is difficult to obtain. On the basis, ASIFT (affinity-SIFT) is provided, and because a series of geometric deformation simulation is carried out on an original image before SIFT matching, the ASIFT can better solve the matching problem of geometric distortion between images, and a certain number of homonymous points can be obtained even if the viewing angle is greatly different. However, these feature-based matching methods do not take into account the distribution problem of the feature points when obtaining the matching points. The number and distribution of the homonymous points are two determining factors affecting the accuracy of the geometric transformation matrix.

How to conveniently and accurately acquire the area of a specific area in the visual field of the camera is also an urgent problem to be solved. At present, the methods for measuring the area from the video mainly include: the method comprises the steps of measuring on the spot, calculating based on an estimation method of a calibration template and calculating according to a simple proportional relation among image pixel coordinates, a camera focal length and vertical height, has certain limitation, and is not suitable for PTZ monitoring videos with a holder function.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a natural resource real-time video monitoring method and system with embedded vector red line data, which can greatly improve the monitoring efficiency of natural resources.

In order to achieve the above object, the technical solution of the present invention provides a real-time video monitoring method for natural resources embedded with vector red line data, comprising the following steps:

step 1, acquiring natural resource vector red line data and a corresponding high-resolution remote sensing image of a certain area with the same projection coordinate system, and simultaneously acquiring an initial picture image of a PTZ camera;

step 2, superposing the vector red line data to the high-resolution remote sensing image, identifying homonymy points on the video image and the vector red line data respectively by means of semantic information of the remote sensing image, and then calculating a geometric mapping relation between the vector red line data and the monitoring video by adopting a least square method;

step 3, mapping the vector red line data to a real-time video according to the mapping relation calculated in the step 2 to obtain a video monitoring area;

step 4, matching adjacent frame images of the monitoring video by obtaining feature points uniformly distributed in scale and space, so as to obtain homonymous points on the two images;

step 5, obtaining a geometric transformation relation between the two images according to the homonymy points obtained in the step 4, and mapping the monitoring area range on the previous frame of image to the next frame of image according to the relation to realize the association of the monitoring area between the monitoring video image frames;

and 6, according to the geometric relationship established in the step, carrying out area measurement on the monitoring video, and supporting the subsequent relevant target motion analysis based on the determined video area, thereby realizing the automatic early warning and alarming of the specific area.

In step 2, the geometric mapping relation between the monitoring video image and the remote sensing image is modeled into projection transformation, and a homography matrix H is used₀The method comprises the steps that at least 4 non-collinear same-name points are identified on two images, and the mutual mapping relation between a monitoring video image and a remote sensing image is quickly established;

corresponding homonymous points q (X, Y) on the remote sensing image are set to be corresponding to the monitoring video image I₀At any point p (x, y) above, the equation is satisfied as follows,

wherein the homography matrix H₀The expression of (a) is as follows,

in step 3, plane vector data { P } is provided₁,P₂,...,P_nFor any one of the faces P_iThe coordinates are expressed as { (X)_i1,Y_i1),(X_i2,Y_i2),...,(X_im,Y_im)}，

For arbitrary vertex (X)_ij,Y_ij) I 1,2, n, j 1,2, m is the number of planes and m is the number of points, and the coordinates in the surveillance video are (x) according to the geometric relationship established in step 2_ij',y_ij'), calculated as follows,

in step 4, the feature points uniformly distributed in scale and space are obtained by determining the number of feature points extracted from each layer of pyramid image according to the scale coefficient in the aspect of scale; in the aspect of images, performing uniform grid division on the images, and determining the quantity of feature points extracted from each grid; then, feature points are screened according to the uniqueness and density of the feature points.

And, in terms of scale, the number of the feature points extracted from the pyramid image of each layer is determined according to the scale coefficient, which is implemented as follows,

suppose that the total number of feature points extracted from an image is N_totalAnd the weight of each layer of image in each group of pyramid images is W (o, l), so that the number N of feature points extracted from each layer of image is N_scaleThe expression of (o, l) is,

N_scale(o,l)＝N_total×W(o,l)

wherein, o is the mark number of the Gaussian pyramid, and l is the mark number of the image layer;

the weight of each scale image is defined as follows,

wherein, SC_olThe scale coefficient, SC, of the image of the l layer in the o group of Gaussian pyramids₁₁Scale coefficient, r, representing the level 1 image in the set 1 Gaussian pyramid₀Is an intermediate parameter; SC (Single chip computer)_olThe expression is as follows:

wherein σ₀As an initial scale parameter, N_oNumber of pyramid images generated, N_sThe number of layers of each group of pyramids, K is a constant,

o＝1,2,3,...,N_o，l＝1,2,3,...,N_s。

in the aspect of the image, the image is uniformly meshed, the number of the feature points extracted in each mesh is determined, and the method is realized as follows,

the quantity of the feature points extracted from each layer of image of the pyramid is assumed to be N_scale(o, l) uniformly dividing the layer image into n²Number of feature points to be extracted in each grid, kth grid, N _ grid_kThe way of calculating (a) is as follows,

N_grid_k＝N_scale(o,l)×f_k

wherein f is_kAs grid weight coefficients, from the entropy E of the information in the grid area_kThe number N of the characteristic points_kAnd the contrast mean C of the feature points_kDetermining that the expression is:

wherein, w_e、w_nAnd w_cAre respectively information entropy E_kThe number N of the characteristic points_kContrast mean value C of feature points_kWeight coefficient of (d), w_e+w_n+w_c＝1。

In step 4, the selected N _ grid is screened out in each grid by adopting the following strategy_kThe number of the characteristic points is one,

step a, calculating the contrast of each feature point, sorting the feature points from high to low according to the contrast value, and then selecting the top 3 × N _ grid_kThe characteristic points are used as pre-selected characteristic points;

b, accurately determining the positions and the scales of the feature points by adopting an SIFT method, and removing edge response points in the feature points obtained in the step a;

step C, calculating the information entropy of the feature points in the neighborhood with the feature points as the center, and ordering the feature points from large to small according to the information entropy to obtain an initial feature point set C_int；

Step d, sequentially taking point sets C_intRespectively calculating Euclidean distance between the point and the rest characteristic points, if the minimum distance is still larger than the corresponding threshold value, keeping the point to a target point set C_setFrom (1) to (C)_setNumber of middle feature points and N _ grid_kUntil equal.

Furthermore, in step 6, the area measurement result is obtained for the polygon in the Mth frame image of the current monitoring video in the following manner,

firstly, determining the geometric mapping relation between the current video frame image M and GIS data, and for any point P on the video frame image M_M(x_M,y_M) The homography matrix with the previous frame image is H_M-1Inversion of homography H_M-1 ^-1Then corresponds to the coordinate P on the M-1 frame image_M-1(x_M-1,y_M-1)＝H_M-1 ^-1P_M(x_M,y_M) And so on, calculating the coordinate P in the projection coordinate system_M(X_M,Y_M)＝H₀ ^-1H₁ ^-1...H_M-1 ^-1P(x_M,y_M)；

Then, according to the relation, the polygon coordinates on the video frame image are subjected to inverse calculation to obtain the coordinates corresponding to the projection coordinate system;

and finally, calculating the area of a polygon by using the cross product of the vectors to obtain an area measurement result.

The invention also provides a natural resource real-time video monitoring system embedded with the vector red line data, which is used for realizing the natural resource real-time video monitoring method embedded with the vector red line data.

According to the method, the natural resource vector data are embedded into the real-time monitoring video, the geometric mapping relation between the 2D GIS data and the PTZ monitoring video is established, the area measurement can be directly carried out on the monitoring video, and on the basis of the area, a foundation can be provided for automatic early warning and alarming of a specific area in the subsequent video field, so that the monitoring efficiency of natural resources is greatly improved.

Compared with the prior art, the technical scheme provided by the invention has the beneficial effects that:

(1) the invention provides a geographic vector data and real-time video fusion scheme based on semi-automatic mapping;

(2) the invention provides a multi-view image matching method based on an improved ASIFT matching mode;

(3) the invention provides a PTZ monitoring video area extraction method based on 2DGIS data.

Drawings

FIG. 1 is a flow chart of an embodiment of the present invention.

Detailed Description

In order to facilitate the understanding and implementation of the present invention for those of ordinary skill in the art, the present invention is further described in detail with reference to the accompanying drawings and examples, it is to be understood that the embodiments described herein are merely illustrative and explanatory of the present invention and are not restrictive thereof.

Referring to fig. 1, a method and a system for real-time video monitoring of natural resources embedded with vector red line data provided by the embodiment of the present invention include the following steps:

(1) obtaining natural resource vector red line data and corresponding high-resolution remote sensing images of a certain area with the same projection coordinate system, and simultaneously obtaining an initial picture image I of a PTZ camera₀；