阅读说明：本技术 一种基于dsp+fpga的视觉目标跟踪系统 (Visual target tracking system based on DSP + FPGA ) 是由 孙鹏 唐俊 胡楷 朱鸿泰 张巍 周弦 于 2021-08-31 设计创作，主要内容包括：本发明涉及图像处理模式识别领域,具体涉及一种基于DSP+FPGA的视觉目标跟踪系统,主要处理步骤包括采集图像数据和接收串口指令、接收图像数据和上位机指令、视觉目标跟踪及数据后处理,以及系统的目标偏测量数据发送与图像显示,本发明利用DSP与FPGA之间的串行高速输入输出口、外部存储器接口实现数据的高速传输,并通过改变硬件连通的通用输入输出口(GPIO)引脚电平触发彼此的数据接收中断以通知各进程相关数据传输已完成,实现DSP与FPGA之间的高速数据交换。(The invention relates to the field of image processing mode identification, in particular to a visual target tracking system based on DSP + FPGA, which mainly comprises the steps of collecting image data, receiving a serial port instruction, receiving the image data and an upper computer instruction, tracking a visual target, post-processing the data, and sending target offset measurement data and displaying the image of the system.)

1. A visual target tracking system based on DSP + FPGA is characterized by comprising the following steps:

s1: controlling the time sequence of the serial port and the video decoding chip by adopting an FPGA processor to acquire image data and receive serial port instructions;

s2: receiving image data and an upper computer instruction by a DSP (digital signal processor) through SRIO (serial input/output), an EMIF (external memory interface) high-speed interface and GPIO (general purpose input/output) interruption;

s3: tracking a visual target and post-processing data;

s4: and the FPGA realizes the transmission of the deviation measurement data and the image display.

2. The system for tracking a visual target based on DSP + FPGA of claim 1, wherein the specific process in step S1 is:

s1.1: receiving tracking instruction data by controlling the time sequence of an external serial port chip, writing the instruction data into an EMIF space of a DSP in a transparent transmission mode, and triggering a corresponding GPIO pin of a DSP processor to be interrupted;

s1.2: controlling the time sequence of a video decoding chip connected with a camera according to the image data clock signal information to realize the acquisition of image data, and realizing the caching of the acquired image data by controlling an SDRAM cache unit;

s1.3: and writing the image data into a DDR3 memory connected with the DSP through the SRIO interface, and triggering the corresponding GPIO pin of the DSP processor to be interrupted.

3. The system for tracking a visual target based on DSP + FPGA of claim 1, wherein the specific process of step S2 is:

s2.1: after the DSP responds to the instruction interruption, reading instruction data from the EMIF space, analyzing according to a corresponding protocol and updating global instruction parameters;

s2.2: after the DSP responds to the image interrupt, the image data is read in the designated space of the DDR3 memory, and whether the read image is valid or not is judged by calculating the flag bit value of the image data.

4. The system for tracking a visual target based on DSP + FPGA of claim 1, wherein the specific process in step S3 is:

s3.1: after the DSP reads the effective image data, resetting the image data zone bit to enable the image data zone bit to be in an invalid state;

s3.2: judging the current processing state according to the instruction global variable information, and performing corresponding calculation on the image data;

s3.3: and performing offset measurement feedback coding and superimposed grid plate manufacturing according to the obtained target offset measurement data, writing the coded offset measurement data and character superimposed information into an EMIF space, and changing the level of a corresponding pin of the DSP to trigger offset measurement sending and character superimposed interruption of the FPGA.

5. The system for tracking the visual target based on the DSP + FPGA as claimed in claim 4, wherein the DSP opens up a char type data segment with the same size as the input image in the EMIF space, the DSP edits the data in the EMIF space, adds character information into the grid plate, completes the manufacture of the superimposed grid plate, and triggers the interruption of character superimposition of the FPGA through GPIO pins.

6. The system for tracking a visual target based on DSP + FPGA of claim 1, wherein the specific process in step S4 is:

s4.1: the FPGA responds to an interrupt function, reads the partial measurement feedback information from the EMIF space and controls the serial port chip to send the partial measurement feedback information out through the serial port;

s4.2: the FPGA reads the character superposition grid plate from the EMIF space, restricts the corresponding pixel point information of the cache image in the SDRAM by using the data of each point in the grid plate, realizes the character superposition of the image, encodes the image data of the superposed character information and sends the image data to the display.

Technical Field

The invention belongs to the field of image processing mode recognition, and particularly relates to a visual target tracking system based on a DSP + FPGA.

Background

In the field of computer vision, visual target tracking is a research hotspot and difficulty. The visual tracking estimates the shape, position and occupied area of a tracking target in a continuous video image sequence, and determines the information of the target such as the motion speed, direction and track, so as to realize the analysis and understanding of the target behavior. The method is widely applied to the fields of military navigation, intelligent monitoring, medical imaging and the like.

In the prior art at home and abroad, the realization of visual target tracking processing on a single FPGA or DSP processor is mainly researched; the FPGA platform is complex in programming and is not beneficial to rapid iterative upgrade of an algorithm program of a processing system; the DSP platform has strong data processing capacity, but the capacity of directly acquiring image data through a data interface is insufficient, and the engineering application is limited.

The embedded target tracking system based on the DSP and the FPGA has the advantages of small size, low power consumption, convenience in carrying and the like, and the advantages of strong complex algorithm processing of the DSP, large data volume and simple structure operation of the FPGA are utilized, so that the real-time performance and flexibility of the tracking system can be greatly improved, and the application prospect is very wide.

Disclosure of Invention

Aiming at the defects in the prior art, the invention provides a visual target tracking system based on a DSP and an FPGA, and aims to solve the technical problem of how to combine an original single DSP and an FPGA processor to improve the real-time performance and the flexibility of the tracking system.

In order to solve the technical problems, the technical scheme provided by the invention is as follows: a visual target tracking system based on DSP + FPGA comprises the following steps:

s1: controlling the time sequence of the serial port and the video decoding chip by adopting an FPGA processor to acquire image data and receive serial port instructions;

s2: receiving image data and an upper computer instruction by a DSP (digital signal processor) through an SRIO (serial peripheral input/output), an EMIF (external memory interface) high-speed interface and a GPIO (general purpose input/output) interrupt interface;

s3: tracking a visual target and post-processing data;

s4: and the FPGA realizes the transmission of the deviation measurement data and the image display.

Preferably, the specific process in step S1 is:

s1.1: the FPGA receives tracking instruction data by controlling the time sequence of an external serial port chip, writes the instruction data into an EMIF space of the DSP in a transparent transmission mode, and triggers the corresponding GPIO pin of the DSP to be interrupted;

s1.2: controlling the time sequence of a video decoding chip connected with a camera according to the image data clock signal information to realize the acquisition of image data, and realizing the caching of the acquired image data by controlling an SDRAM cache unit;

s1.3: and writing the image data into a DDR3 memory connected with the DSP through the SRIO interface, and triggering the corresponding GPIO pin of the DSP processor to be interrupted.

Preferably, the specific process of step S2 is:

s2.1: after the DSP responds to the instruction interruption, reading instruction data from the EMIF space, analyzing according to a corresponding protocol and updating global instruction parameters;

s2.2: after the DSP responds to the image interrupt, reading image data in a specified space of a DDR3 memory, and judging whether the read image is valid or not by calculating the sign bit value of the image data;

preferably, the specific process in step S3 is:

s3.1: after the DSP reads the effective image data, resetting the image data zone bit to enable the image data zone bit to be in an invalid state;

s3.2: judging the current processing state according to the instruction global variable information, and performing corresponding calculation on the image data;

s3.3: and performing offset measurement feedback coding and superimposed grid plate manufacturing according to the obtained target offset measurement data, writing the coded offset measurement data and character superimposed information into an EMIF space, and changing the level of a corresponding pin of the DSP to trigger offset measurement sending and character superimposed interruption of the FPGA.

Preferably, the DSP opens up a char type data section with the same size as the input image in the EMIF space, the DSP edits data in the EMIF space, character information is added into the grid plate to complete the manufacture of the superposed grid plate, and FPGA character superposition interruption is triggered through the GPIO pin.

Preferably, the specific process in step S4 is:

s4.1: the FPGA responds to an interrupt function, reads the partial measurement feedback information from the EMIF space and controls the serial port chip to send the partial measurement feedback information out through the serial port;

s4.2: the FPGA reads the character superposition grid plate from the EMIF space, restricts the corresponding pixel point information of the cache image in the SDRAM by using the data of each point in the grid plate, realizes the character superposition of the image, encodes the image data of the superposed character information and sends the image data to the display.

The beneficial effect that this technical scheme brought is: the invention relates to a visual target tracking system based on a DSP and an FPGA, wherein the DSP is responsible for instruction analysis, tracking processing, character superposition grid plate manufacturing and deflection measurement data coding; the FPGA is responsible for finishing image data acquisition and display, video character superposition and logic control of a serial port, high-speed data transmission is realized by utilizing a serial high-speed input/output port (SRIO) and an External Memory Interface (EMIF) between the DSP and the FPGA, and the levels of general purpose input/output ports (GPIO) pins communicated by hardware are changed to trigger mutual data receiving interruption so as to inform that the related data transmission of each process is finished, so that high-speed data exchange between the DSP and the FPGA is realized.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:

FIG. 1 is a block diagram of a software implementation of the present invention;

FIG. 2 is a block diagram illustrating an instruction update process according to the present invention;

FIG. 3 is a process flow diagram of the target tracking algorithm of the present invention;

FIG. 4 is a flow chart of the text overlay message transmission of the present invention;

FIG. 5 is a flow chart of the bias measurement process of the DSP module of the present invention.

Detailed Description

The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, and it should be understood that the preferred embodiments described herein are merely for purposes of illustration and explanation, and are not intended to limit the present invention.

The invention relates to a visual target tracking system based on DSP + FPGA, as shown in figure 1, mainly comprising the following steps:

s1: controlling the time sequence of the serial port and the video decoding chip by adopting an FPGA processor to acquire image data and receive serial port instructions;

s1.1: the FPGA receives tracking instruction data by controlling the time sequence of an external serial port chip, writes the instruction data into an EMIF space of the DSP in a transparent transmission mode, and triggers the corresponding GPIO pin of the DSP to be interrupted;

s1.2: the FPGA controls the time sequence of a video decoding chip connected with the camera according to the image data clock signal blanking area information to realize the acquisition of image data, and realizes the caching of the acquired image data through the control of the SDRAM cache unit.

S2: the DSP processor receives the upper computer instruction and the image data through the EMIF, the SRIO high-speed interface and the GPIO interrupt;

s2.1: after the DSP responds to the instruction interruption, reading instruction data from the EMIF space, analyzing according to a corresponding protocol and updating global instruction parameters;

the instruction updating of the tracking system adopts a data writing and interruption notification mode, the upper computer sends the instruction updating to the FPGA, the FPGA forwards data to an EMIF space of the DSP through an EMIF interface and triggers DSP hardware interruption by changing the level of a GPIO pin, and the DSP processes the instruction updating in a hardware interruption service function.

After the interrupt response, the detailed processing process is as shown in fig. 2, after the DSP instruction interrupt comes, the interrupt response function reads the instruction information sent by the serial port from the specified data address field of the EMIF, and moves the data to the static storage space, and then the DSP reads the length flag bit of the static space instruction data to obtain the length information of the frame data, and then performs instruction verification on the valid data bit of the data field according to the length information. The DSP reads the type zone bit of the static space instruction data, analyzes the instruction according to the agreed protocol and assigns values to the instruction global variable of the program, and then the DSP performs zero clearing processing on the instruction data storage space of the EMIF space and jumps out of the interrupt.

In the above DSP data moving content, the specific way is that when the DSP executes the platform initialization operation, a section of static storage space is opened up in the DDR3 memory for storing the instruction information sent by the serial port. After the instruction interrupt arrives, the interrupt response function reads instruction information sent by a serial port from a specified data address in the EMIF and moves the data to the section of static storage space, so that the instruction data in the EMIF space is prevented from being covered.

And simultaneously, the DSP reads the length zone bit of the instruction data segment to obtain the length information of the instruction data frame, and then performs instruction verification on effective instruction frame data in the data segment according to the length information to verify whether the frame instruction is wrong. Analyzing the data when the data is correct; if the data is wrong, adding 1 to the global variable value of the instruction error. After the data verification is completed, the DSP reads the type flag bit in the correct data frame and judges the instruction type; then, the instruction global variable of the program is assigned according to the agreed protocol. After the data analysis is completed, the DSP performs zero clearing processing on the instruction data section of the EMIF space and the instruction storage space of the DDR3 to avoid influencing the subsequent processing and jump out of the instruction interruption.

S2.2: after the DSP responds to the image interrupt, the image data is read in the designated space of the DDR3 memory, and whether the read image is valid or not is judged by calculating the flag bit value of the image data.

When the FPGA executes a blanking area of an image, the image data in the SDRAM is written into a DDR3 memory of the DSP through an SRIO interface and image receiving hardware interruption of the DSP is triggered through GPIO level, and in order to prevent read-write conflict between the image data written in by the FPGA and the image data read by the DSP, the research uses ping-pong operation to design the image storage of the DSP, so that the image data read by the DSP and the image data written in by the FPGA cannot use the same DDR3data segment at the same time.

In the above steps, the odd frame/field interrupt service sets the source code as:

interrupt void C66x_ISR_INTCA(void)

{

fpga_GetImageFromDDR3DATA_A();

}

even frame/field interrupt service sets source code:

interrupt void C66x_ISR_INTCB(void)

{

fpga_GetImageFromDDR3DATA_B();

}

in the code, when high-definition system video data is processed, the parity frames are processed separately; when standard definition type video data is processed, the odd-even field of the same frame of image is processed separately. When the FPGA transmits an odd frame/field image to the DSP, triggering an external interrupt A of the DSP; and when the FPGA transmits the even frame/field image to the DSP, triggering the external interrupt B of the DSP. When the FPGA writes odd frame/field DATA into a DDR3DATA _ A DATA segment of the DSP and triggers DSP image interruption through a binding pin of a DSP hardware interruption A, the DSP processes image DATA in the DDR3DATA _ B DATA segment or finishes processing the image DATA and is in an interruption waiting state; the DSP and the FPGA do not access the same DDR3data segment at the same time, so that access conflict is avoided.

S3: tracking a visual target and post-processing data;

s3.1: after the DSP reads the valid image data, whether the image read this time is valid is judged by calculating the sign bit value of the image data, if the image is valid, the sign bit is reset to enable the image to be in an invalid state, and the tracking instruction reading is executed; if the read image flag bit is invalid, writing an image invalid state into the tracking state and the offset measurement information in the tracking processing array;

s3.2: judging the current processing state according to the instruction global variable information in S2, and performing tracking calculation on the image data, as shown in fig. 3 below;

reading tracking instruction information, judging whether the DSP executes a tracking target instruction or not, and scheduling a tracking algorithm if the tracking processing is needed; and writing an idle thread state identifier into the tracking state and the offset measurement information in the tracking processing array without tracking calculation. When the tracking algorithm is scheduled for processing, processing and relevance judgment are carried out on a target in an image, when the relevance is less than or equal to a threshold value, the current tracking is judged to be invalid, and a tracking state of target loss is written into a tracking processing array; when the correlation is larger than the threshold value, the stability of the tracking target of the algorithm is judged;

s3.3: and performing offset measurement feedback coding and superimposed grid plate manufacturing according to the obtained target offset measurement data, writing the coded offset measurement data and character superimposed information into an EMIF space, and changing the level of a corresponding pin of the DSP to trigger offset measurement sending and character superimposed interruption of the FPGA.

In the above steps, the display control unit of the tracking system is controlled by the FPGA, the display information obtained by tracking processing needs to be transmitted to the FPGA module by the DSP module, the transmission flow is as shown in fig. 4, the DSP develops a char type data segment with the same size as the input image size in the EMIF space, the DSP edits the data in the EMIF space by the segment, adds the character information into the grid plate, completes the manufacture of the superimposed grid plate, and triggers the interruption of the character superimposition of the FPGA through the GPIO pin.

In addition, the offset measurement data is written into the EMIF space by the DSP, and the offset measurement data reading process of the FPGA is triggered by GPIO interrupt, and directly transmitted to the serial port for output, and the execution process is as shown in fig. 5.

In the above steps, in order to avoid the influence of the previous frame of offset measurement data on the current frame of data, the DSP platform EMIF spatial offset measurement data needs to be cleared.

After the tracking algorithm is processed, only the tracking state and the position of the target in the image are obtained. When tracking is performed, the tracking state and the target position need to be read, and when the tracking target is lost and in an idle state, only the tracking state needs to be read.

Defining the value of a horizontal deviation measurement X and a pitching deviation measurement Y as 0 and 0 when a target is at the central point of an image; when the target is positioned at the lower right of the image, the deflection measurement is a positive value; when the target is located at the upper left of the image, the deflection measure is negative. Because one char type character sent by the serial port cannot completely express the offset measurement information of the target offset measurement larger than 256, the target offset measurement needs to be converted, and the research uses two adjacent char type characters to express the offset measurement of the target, and the detailed calculation is as follows:

when the deviation measurement value is greater than 0:

ByteH = N/256

ByteL = N%256

when the deviation measurement value is less than 0:

ByteH = 255-N/256

ByteL = -N%256

in the above formula, ByteH represents that the target offset measurement is 8 bits higher, when ByteH is less than 4, the offset measurement is greater than 0, and when ByteH is greater than 251, the offset measurement is less than 0; ByteL indicates the lower 8 bits of the bias measure.

And writing the encoded obtained offset measurement frame data into an EMIF (external memory interface) space data section of the DSP, then changing the level of a corresponding GPIO (general purpose input/output) pin, and triggering the offset measurement data of the FPGA to be transmitted and interrupted.

S4: the FPGA realizes the transmission of the deviation measurement data and the image display;

s4.1: the FPGA responds to an interrupt function, reads the partial measurement feedback information from the EMIF space and controls the serial port chip to send the partial measurement feedback information out through the serial port;

s4.2: the FPGA reads the character superposition grid plate from the EMIF space, restricts the corresponding pixel point information of the cache image in the SDRAM by using the data of each point in the grid plate, realizes the character superposition of the image, encodes the image data of the superposed character information and sends the image data to the display.

Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.