阅读说明：本技术 一种视频跟踪系统及方法 (Video tracking system and method ) 是由 肖红 鄢冬斌 于 2019-10-10 设计创作，主要内容包括：本发明公开了一种视频跟踪系统,图像输入支路接收图像采集设备采集的图像信号并发送至FPGA,FPGA将图像信号处理后发送储存单元储存为缓存图像；FPGA调取缓存图像并对缓存图像中的光斑进行检测后对光斑定位；FPGA根据光斑定位结果得出方位和俯仰调整数据,并将方位和俯仰调整数据通过串口通信支路发送至方位俯仰调整机构,方位俯仰调整机构根据方位和俯仰调整数据对图像采集设备的方位和俯仰调整使得图像采集设备持续捕捉光斑。本发明还公开了一种视频跟踪方法。本发明一旦检测到反射光斑后,视频跟踪系统发送指令控制转台停止转动,之后计算反射光斑位置,输出方位、俯仰调整参数至转台,最终使反射光斑位于图像正中心。(The invention discloses a video tracking system.A image input branch receives an image signal acquired by image acquisition equipment and sends the image signal to an FPGA (field programmable gate array), and the FPGA processes the image signal and then sends the image signal to a storage unit to be stored as a cache image; the FPGA calls the cache image, detects the light spots in the cache image and positions the light spots; the FPGA obtains the azimuth and pitching adjustment data according to the light spot positioning result, and sends the azimuth and pitching adjustment data to the azimuth pitching adjustment mechanism through the serial port communication branch, and the azimuth pitching adjustment mechanism adjusts the azimuth and pitching of the image acquisition equipment according to the azimuth and pitching adjustment data so that the image acquisition equipment continuously captures the light spots. The invention also discloses a video tracking method. Once the invention detects the reflected light spot, the video tracking system sends an instruction to control the rotary table to stop rotating, then calculates the position of the reflected light spot, outputs the azimuth and pitch adjustment parameters to the rotary table, and finally makes the reflected light spot be positioned in the center of the image.)

1. A video tracking system is characterized by comprising an FPGA, a storage unit, an image input branch and a serial port communication branch;

the image input branch receives an image signal acquired by the image acquisition equipment and sends the image signal to the FPGA, and the FPGA processes the image signal and then sends the image signal to the storage unit to store the image signal as a cache image;

the FPGA calls the cache image, detects the light spots in the cache image and positions the light spots;

the FPGA obtains azimuth and pitching adjustment data according to the light spot positioning result, and sends the azimuth and pitching adjustment data to the azimuth and pitching adjustment mechanism through the serial port communication branch, and the azimuth and pitching adjustment mechanism adjusts the azimuth and pitching of the image acquisition equipment according to the azimuth and pitching adjustment data so that the image acquisition equipment continuously captures the light spots.

2. A video tracking system according to claim 1, further comprising a PAL output branch;

the FPGA calls a cache image and samples the cache image;

the FPGA superposes the light spot detection result into the sampled cache image and sends the superposed image to the PAL output branch;

and the PAL output branch circuit encodes the superposed images and outputs the analog video.

3. A video tracking system according to claim 1, further comprising a plurality of crystal units;

clock signals output by the crystal oscillator units are respectively connected to different BANKs of the FPGA and used as system working clocks of the FPGA.

4. A video tracking system according to claim 1, wherein the FPGA comprises n x n registers and a plurality of FIFO buffers, and n is derived from the spot size;

each FIFO buffer reads a line of pixels in the buffer image and stores the pixels into the register in a first-in first-out mode;

and the FPGA obtains all n x n window gray values in each frame of cache image from the register, and the window with the maximum gray value is used as the position of the light spot.

5. The video tracking system of claim 4, wherein the FPGA performs angle conversion on the center coordinates of the window with the maximum gray scale value according to the following formula:

X＝X_Pmax*1°/G；

Y＝Y_Pmax*1°/H；

wherein X is the converted central angle of the light spot in the X direction; y is the converted central angle of the light spot in the Y direction; x_PmaxThe rectangular coordinate is the X-direction rectangular coordinate of the window center with the maximum gray value; y is_PmaxThe rectangular coordinate of the center Y direction of the window with the maximum gray value; g is the number of pixels in the X direction of the image; h is the number of pixels in the Y direction of the image.

6. A video tracking method, comprising:

receiving an image signal, processing the image signal and storing the image signal as a cache image;

calling a cache image, detecting light spots in the cache image and positioning the light spots;

obtaining azimuth and pitching adjustment data according to the light spot positioning result, and sending the azimuth and pitching adjustment data to the azimuth pitching adjustment mechanism through the serial port communication branch;

the orientation and pitch adjustment mechanism adjusts the orientation and pitch of the image acquisition device according to the orientation and pitch adjustment data so that the image acquisition device continuously captures the light spots.

7. The video tracking method according to claim 6, wherein the step of locating the light spot after detecting the light spot in the buffered image comprises the sub-steps of:

processing the cache image by adopting an n-x-n register and a plurality of FIFO (first in first out) buffers, wherein n is obtained according to the size of the light spot;

each FIFO buffer reads a line of pixels in the buffer image and stores the pixels into a line of the register in a first-in first-out mode;

when new data in the FIFO buffer reaches the register, discarding the data in the register unit at the tail end of the register, moving the data in the register to the register unit at the tail end of the register, and recalculating the sum of the gray values in the n × n register after the data are moved as the gray value of the n × n window;

and selecting the window with the maximum gray value as the position of the light spot.

8. A method for video tracking according to claim 7, wherein recalculating the sum of the gray values in the n x n register as the gray value of the n x n window comprises the steps of:

the grey value of the n x n window is obtained according to the following formula:

P_(m)sum＝P_(m-1)sum+REG_(m)-P_(m)；

in the formula P_(m)sumIs the sum of the gray values obtained by recalculation; p_(m-1)sumThe gray value of the n x n window when the new data does not reach the register; REG_(m)The gray value of the new data is the gray value of the new data; p_(m)The gray value of the data is discarded for this time.

9. The method of claim 7, wherein the frame image is considered to have no flare when the difference between the gray value of the window with the largest gray value and the gray value of the other windows in the same frame is smaller than a threshold.

10. The video tracking method of claim 6, wherein deriving the azimuth and elevation adjustment data from the spot location comprises:

and performing angle conversion on the central coordinate of the window with the maximum gray value according to the following formula:

X＝X_Pmax*1°/G；

Y＝Y_Pmax*1°/H；

wherein X is the converted central angle of the light spot in the X direction; y is the converted central angle of the light spot in the Y direction; x_PmaxThe rectangular coordinate is the X-direction rectangular coordinate of the window center with the maximum gray value; y is_PmaxThe rectangular coordinate of the center Y direction of the window with the maximum gray value; g is the number of pixels in the X direction of the image; h is the number of pixels in the Y direction of the image;

and obtaining azimuth and pitching adjustment data according to the converted central angle data.

Technical Field

The invention relates to the technical field of computers, in particular to a video tracking system and a video tracking method.

Background

Tracking can be defined simply as estimating the trajectory in the image plane of an object as it moves around a scene, i.e. a tracking system assigns consistent labels to tracked objects in different frames of the same video. The tracking technology is one of the hot spots in the field of computer vision research, and has wide application prospects in various aspects such as military reconnaissance, accurate guidance, fire fighting, battlefield evaluation, security monitoring and the like.

The target tracking technology is one of the hot spots in the field of computer vision research, and has wide application prospects in various aspects such as military reconnaissance, accurate guidance, fire fighting, battlefield evaluation, security monitoring and the like. The non-directional motion of the target changes the appearance pattern of the target and the scene, the non-rigid target structure, the shielding between the targets and between the target and the scene, the motion of the camera and the like, so that the target tracking task becomes more difficult. Tracking is often applied in those application environments where knowledge of the position and shape of each frame of the target is required, and assumptions are often used to constrain the tracking problem in a particular application environment.

Tracking can be defined simply as estimating the trajectory in the image plane of an object as it moves around a scene, i.e. a tracking system assigns consistent labels to tracked objects in different frames of the same video. Target tracking is a challenging problem. Target tracking is an important task in the field of computer vision. With the increase of high-performance computers and the popularization of high-quality and low-cost cameras, the increasing demand for automatic video analysis has led to a great interest in target tracking algorithms.

Disclosure of Invention

The invention aims to solve the technical problems that a video tracking means in the prior art generally has the problems of complex and unstable system architecture, large calculated amount and the like, is not suitable for quickly and accurately positioning and tracking a light spot target, and aims to provide a video tracking system and a video tracking method to solve the problems.

The invention is realized by the following technical scheme:

a video tracking system comprises an FPGA, a storage unit, an image input branch and a serial port communication branch; the image input branch receives an image signal acquired by the image acquisition equipment and sends the image signal to the FPGA, and the FPGA processes the image signal and then sends the image signal to the storage unit to store the image signal as a cache image; the FPGA calls the cache image, detects the light spots in the cache image and positions the light spots; the FPGA obtains azimuth and pitching adjustment data according to the light spot positioning result, and sends the azimuth and pitching adjustment data to the azimuth and pitching adjustment mechanism through the serial port communication branch, and the azimuth and pitching adjustment mechanism adjusts the azimuth and pitching of the image acquisition equipment according to the azimuth and pitching adjustment data so that the image acquisition equipment continuously captures the light spots.

When the method is applied, the FPGA is used as a control chip, the light spots in the cache image are detected and positioned, the positioning result is sent to the azimuth pitching adjusting mechanism, and the light spots are tracked through the azimuth pitching adjusting mechanism, so that the light spots are continuously captured by the image acquisition equipment. The video tracking system is arranged on the laser rotary table, the laser rotary table performs circumferential scanning at a certain angular speed, when a target is detected, a reflection light spot appears in image input, and the video tracking system needs to detect the reflection light spot in real time; once the reflection light spot is detected, the video tracking system sends an instruction to control the rotary table to stop rotating, then the position of the reflection light spot is calculated, azimuth and pitching adjustment parameters are output to the rotary table, and finally the reflection light spot is located in the center of the image.

Further, the system also comprises a PAL output branch circuit; the FPGA calls a cache image and samples the cache image; the FPGA superposes the light spot detection result into the sampled cache image and sends the superposed image to the PAL output branch; and the PAL output branch circuit encodes the superposed images and outputs the analog video.

When the invention is applied, because the invention is mainly used for tracking the tail flame of an air target, such as a missile and a jet aircraft, a tracking image needs to be sent to a command system, such as a command tower and the like, for a commander to refer, the invention superposes a light spot detection result into a sampled cache image and sends the superposed image to a PAL output branch, the PAL output branch carries out analog video output after coding the superposed image, and the superposed image can enable the personnel of the command system to quickly and accurately distinguish the tail flame and carry out command judgment.

Further, the device also comprises a plurality of crystal oscillator units; clock signals output by the crystal oscillator units are respectively connected to different BANKs of the FPGA and used as system working clocks of the FPGA.

When the invention is applied, clock signals output by a plurality of crystal oscillator units are provided for the FPGA as a system working clock, so that the input system clock is not in the same BANK, and a plurality of PLL resources in the FPGA can be used; therefore, the response speed of the FPGA is improved, the Cameralink input clock is an input clock of the image input branch, the Cameralink input clock needs to be connected to a global clock pin of the FPGA and is located in the same BANK of the FPGA with the Cameralink signal, and the problems of clock delay and phase jitter caused by crossing the BANK are reduced.

Furthermore, the FPGA comprises n × n registers and a plurality of FIFO buffers, and n is obtained according to the size of the light spot; each FIFO buffer reads a line of pixels in the buffer image and stores the pixels into the register in a first-in first-out mode; and the FPGA obtains all n x n window gray values in each frame of cache image from the register, and the window with the maximum gray value is used as the position of the light spot.

When the invention is applied, in order to improve the response speed of a system, the calculation amount of light spot detection needs to be greatly reduced, the invention realizes the reduction of the calculation amount through a register of n x n and a plurality of FIFO (first in first out) buffers, n is obtained according to the size of a light spot, namely the size of the light spot needs to be assumed, if the size of the light spot is n x n, the register selects n x n, namely, one register can completely store the light spot in one image; the buffered image is generally a gray scale image, and whether the window is a light spot can be judged by calculating the gray scale value in the n × n window, and because the FIFO buffer is adopted, for a complete image frame, a FIFO buffer stores a complete line of pixels, which are read into the register first in first out, the recalculation of the gray value each time the data stored in the register changes does not require re-reading of the entire register, the data output from the FIFO buffer, the data discarded by the register, and the data before the change are superimposed to obtain a new gray value, which greatly reduces the amount of computation, and when n is 5, the recalculation of the data in the register requires at least 24 summations, and by the method of the present invention, the calculation process can be shortened to less than 10 times, so the response time of the present invention can be shortened by at least 50%.

Further, the FPGA performs angle conversion on the center coordinate of the window with the maximum gray value according to the following formula: x ═ X_Pmax*1°/G；Y＝Y_Pmax1 °/H; wherein X is the converted central angle of the light spot in the X direction; y is the converted central angle of the light spot in the Y direction; x_PmaxThe rectangular coordinate is the X-direction rectangular coordinate of the window center with the maximum gray value; y is_PmaxThe rectangular coordinate of the center Y direction of the window with the maximum gray value; g is the number of pixels in the X direction of the image; h is the number of pixels in the Y direction of the image.

When the method is applied, the central coordinate is converted for conveniently adjusting the angle, and different from other video tracking technologies, the tracking object is the tail flame of the aircraft, the flying height of the aircraft is higher, so that the method can be adjusted through the angle relation, and the angle of the rotary table required to be adjusted in the next step can be known by converting the pixel coordinate into the angle coordinate, thereby further improving the tracking efficiency.

A video tracking method, comprising: receiving an image signal, processing the image signal and storing the image signal as a cache image; calling a cache image, detecting light spots in the cache image and positioning the light spots; obtaining azimuth and pitching adjustment data according to the light spot positioning result, and sending the azimuth and pitching adjustment data to the azimuth pitching adjustment mechanism through the serial port communication branch; the orientation and pitch adjustment mechanism adjusts the orientation and pitch of the image acquisition device according to the orientation and pitch adjustment data so that the image acquisition device continuously captures the light spots.

When the method is applied, the FPGA is used as a control chip, the light spots in the cache image are detected and positioned, the positioning result is sent to the azimuth pitching adjusting mechanism, and the light spots are tracked through the azimuth pitching adjusting mechanism, so that the light spots are continuously captured by the image acquisition equipment. The video tracking system is arranged on the laser rotary table, the laser rotary table performs circumferential scanning at a certain angular speed, when a target is detected, a reflection light spot appears in image input, and the video tracking system needs to detect the reflection light spot in real time; once the reflection light spot is detected, the video tracking system sends an instruction to control the rotary table to stop rotating, then the position of the reflection light spot is calculated, azimuth and pitching adjustment parameters are output to the rotary table, and finally the reflection light spot is located in the center of the image.

Further, the positioning the light spot after detecting the light spot in the cached image comprises the following sub-steps: processing the cache image by adopting an n-x-n register and a plurality of FIFO (first in first out) buffers, wherein n is obtained according to the size of the light spot; each FIFO buffer reads a line of pixels in the buffer image and stores the pixels into a line of the register in a first-in first-out mode; when new data in the FIFO buffer reaches the register, discarding the data in the register unit at the tail end of the register, moving the data in the register to the register unit at the tail end of the register, and recalculating the sum of the gray values in the n × n register after the data are moved as the gray value of the n × n window; and selecting the window with the maximum gray value as the position of the light spot.

When the invention is applied, in order to improve the response speed of a system, the calculation amount of light spot detection needs to be greatly reduced, the invention realizes the reduction of the calculation amount through a register of n x n and a plurality of FIFO (first in first out) buffers, n is obtained according to the size of a light spot, namely the size of the light spot needs to be assumed, if the size of the light spot is n x n, the register selects n x n, namely, one register can completely store the light spot in one image; the buffered image is generally a gray scale image, and whether the window is a light spot can be judged by calculating the gray scale value in the n × n window, and because the FIFO buffer is adopted, for a complete image frame, a FIFO buffer stores a complete line of pixels, which are read into the register first in first out, the recalculation of the gray value each time the data stored in the register changes does not require re-reading of the entire register, the data output from the FIFO buffer, the data discarded by the register, and the data before the change are superimposed to obtain a new gray value, which greatly reduces the amount of computation, and when n is 5, the recalculation of the data in the register requires at least 24 summations, and by the method of the present invention, the calculation process can be shortened to less than 12 times, so the response time of the present invention can be shortened by at least 50%.

Further, recalculating the sum of the gray values in the n × n register as the gray value of the n × n window comprises the steps of: the grey value of the n x n window is obtained according to the following formula:

P_(m)sum＝P_(m-1)sum+REG_(m)-P_(m)；

in the formula P_(m)sumIs the sum of the gray values obtained by recalculation; p_(m-1)sumThe gray value of the n x n window when the new data does not reach the register; REG_(m)The gray value of the new data is the gray value of the new data; p_(m)The gray value of the data is discarded for this time.

When the method is applied, the sum of the gray values in the n x n register can be well obtained through the formula, the calculation mode is simple, and the REG is_(m)Is a continuous addition array, the number of elements in the array is n, P_(m)The array is also a continuous addition array, and the number of elements in the array is n; i.e. the whole operand is n + n times, and if the whole register is recalculated, the operand is n x n times, it is obvious that in case n is larger than 2, the operand 2n of the present invention is much smaller than the recalculated operand n²And as n increases, the calculation amount gap is further widened.

Further, when the difference between the gray value of the window with the maximum gray value and the gray values of other windows of the same frame is smaller than a threshold value, the frame image is considered to have no light spot.

When the method is applied, when the equipment is actually operated, the situation that no light spot exists in a picture can be completely caused, so that a threshold value is designed for screening the light spot, and the threshold value can be a direct value of 30-80.

Further, the step of obtaining the azimuth and pitch adjustment data according to the spot positioning result comprises the following steps: and performing angle conversion on the central coordinate of the window with the maximum gray value according to the following formula: x ═ X_Pmax*1°/G；Y＝Y_Pmax1 °/H; wherein X is the converted central angle of the light spot in the X direction; y is the converted central angle of the light spot in the Y direction; x_PmaxThe rectangular coordinate is the X-direction rectangular coordinate of the window center with the maximum gray value; y is_PmaxThe rectangular coordinate of the center Y direction of the window with the maximum gray value; g is the number of pixels in the X direction of the image; h is the number of pixels in the Y direction of the image; and obtaining azimuth and pitching adjustment data according to the converted central angle data.

When the method is applied, the central coordinate is converted for conveniently adjusting the angle, and different from other video tracking technologies, the tracking object is the tail flame of the aircraft, the flying height of the aircraft is higher, so that the method can be adjusted through the angle relation, and the angle of the rotary table required to be adjusted in the next step can be known by converting the pixel coordinate into the angle coordinate, thereby further improving the tracking efficiency.

Compared with the prior art, the invention has the following advantages and beneficial effects:

the invention relates to a video tracking system and a method, wherein the video tracking system needs to detect a reflected light spot in real time; once the reflection light spot is detected, the video tracking system sends an instruction to control the rotary table to stop rotating, then the position of the reflection light spot is calculated, azimuth and pitching adjustment parameters are output to the rotary table, and finally the reflection light spot is located in the center of the image.

Drawings

The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:

FIG. 1 is a schematic diagram of a circuit architecture according to the present invention;

FIG. 2 is a schematic diagram of an embodiment of the present invention;

FIG. 3 is a schematic diagram of an embodiment of a crystal oscillator according to the present invention;

FIG. 4 is a schematic diagram of a register and FIFO according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to examples and accompanying drawings, and the exemplary embodiments and descriptions thereof are only used for explaining the present invention and are not meant to limit the present invention.