阅读说明：本技术 一种csi相机高帧率图像采集方法 (High frame rate image acquisition method for CSI camera ) 是由 赵世杰 杨煦 赵复阳 宋伟铭 周中亚 刘敏 郭勇军 于 2021-08-05 设计创作，主要内容包括：本申请公开了一种CSI相机高帧率图像采集方法,包括：步骤1,将CSI控制器中的第一缓存帧地址设置为第二缓存帧地址；步骤2,当判定CSI控制器识别出的图像数据帧的帧头信号满足帧头中断阈值时,进入帧头中断服务,读取采集队列中的缓存帧地址配置为第三缓存帧地址,并根据CSI控制器中的第二缓存帧地址存储图像数据帧；步骤3,当判定CSI控制器识别出的图像数据帧的帧尾信号满足帧尾中断阈值时,进入帧尾中断服务,将第二缓存帧地址添加至交付队列,并将第三缓存帧地址配置为CSI控制器中存储下一帧图像数据帧的第二缓存帧地址。通过本申请中的技术方案,解决了英伟达平台图像数据采集过程中实时性较差以及线程图像采集过程中容易出现图像丢帧的问题。(The application discloses a CSI camera high frame rate image acquisition method, which comprises the following steps: step 1, setting a first cache frame address in a CSI controller as a second cache frame address; step 2, when judging that the frame header signal of the image data frame identified by the CSI controller meets a frame header interrupt threshold, entering frame header interrupt service, reading a cache frame address in an acquisition queue to configure the cache frame address as a third cache frame address, and storing the image data frame according to a second cache frame address in the CSI controller; and 3, when the frame tail signal of the image data frame identified by the CSI controller is judged to meet the frame tail interruption threshold, entering frame tail interruption service, adding the second buffer frame address to the delivery queue, and configuring the third buffer frame address as the second buffer frame address for storing the next frame of image data frame in the CSI controller. By the technical scheme, the problems that real-time performance is poor in an image data acquisition process of an English-Webda platform and image loss is easy to occur in a thread image acquisition process are solved.)

1. A CSI camera high frame rate image acquisition method is applicable to an image processing platform, a CSI controller is arranged in the image processing platform, and the method comprises the following steps:

step 1, setting a first cache frame address in the CSI controller as a second cache frame address;

step 2, when judging that the frame header signal of the image data frame identified by the CSI controller meets a frame header interrupt threshold, entering frame header interrupt service, reading a cache frame address in an acquisition queue to configure the cache frame address as a third cache frame address, and storing the image data frame according to a second cache frame address in the CSI controller;

and step 3, when the frame tail signal of the image data frame identified by the CSI controller is judged to meet a frame tail interruption threshold, entering frame tail interruption service, adding the second buffer frame address to a delivery queue, and configuring the third buffer frame address as the second buffer frame address for storing the next frame of image data frame in the CSI controller.

2. The CSI camera high frame rate image acquisition method of claim 1, wherein the image data frame is sent to the CSI controller by an image sensor, the image sensor configuring the header signal and the trailer signal of the image data frame according to a CSI protocol.

3. The CSI camera high frame rate image acquisition method of claim 1, wherein prior to step 1, the method further comprises:

the CSI controller reads the first buffer frame address in the acquisition queue, records the first buffer frame address as a reserved buffer frame address,

and reading a second buffer frame address in the acquisition queue and recording the second buffer frame address as the first buffer frame address.

4. The CSI camera high frame rate image acquisition method according to claim 3, wherein the step 2 specifically comprises:

step 21, when it is determined that the frame header signal of the image data frame identified by the CSI controller satisfies a frame header interrupt threshold, entering a frame header interrupt service, and determining whether the acquisition queue is empty;

step 22, if yes, configuring the reserved buffer frame address as the third buffer frame address,

if not, reading a buffer frame address in the acquisition queue to configure the buffer frame address as a third buffer frame address;

and step 23, storing the image data frame according to a second cache frame address in the CSI controller, starting transmission of the next frame of image data frame, resetting the frame header interrupt threshold value, and exiting the frame header interrupt service.

5. The CSI camera high frame rate image acquisition method according to claim 4, wherein the step 3 specifically comprises:

step 31, when it is determined that the end-of-frame signal of the image data frame identified by the CSI controller satisfies an end-of-frame interrupt threshold, entering an end-of-frame interrupt service, and determining whether the second buffer frame address is the reserved buffer frame address;

step 32, if yes, discarding the corresponding storage data in the second buffer frame address,

if not, adding the second cache frame address to a delivery queue;

and step 33, configuring the third buffer frame address as a second buffer frame address for storing the next frame of image data frame in the CSI controller, resetting the end-of-frame interrupt threshold, and exiting the end-of-frame interrupt service.

6. The CSI camera high frame rate image acquisition method of claim 4, wherein prior to step 23, the method further comprises:

judging whether the state of the transmission zone bit is in an enabling state, if so, performing frame tail exception processing on the previous frame of image data frame, and if not, modifying the transmission zone bit to be in the enabling state, wherein the frame tail exception processing process specifically comprises the following steps:

step A, judging whether a second cache frame address for storing a previous frame of image data frame is the reserved cache frame address;

step B, if yes, saving the third cache frame address as a second cache frame address for storing the current image data frame,

and if not, performing residual frame marking on the second cache frame address for storing the previous frame of image data frame, and adding the marked second cache frame address to the delivery queue.

7. The CSI camera high frame rate image acquisition method of claim 5, wherein prior to step 33, the method further comprises:

judging whether the state of a transmission zone bit is in an enabling state, if so, modifying the state of the transmission zone bit into a clearing state, and if not, performing frame header exception processing on the current image data frame, wherein the process of the frame header exception processing specifically comprises the following steps:

step C, judging whether the acquisition queue is empty, if so, configuring the reserved buffer frame address as the third buffer frame address,

if not, reading a buffer frame address in the acquisition queue to configure the buffer frame address as a third buffer frame address;

and D, configuring the second buffer frame address in the CSI controller as an address for storing the next frame of image data frame, and starting the transmission of the next frame of image data frame.

8. The CSI camera high frame rate image acquisition method of any one of claims 1 to 7, the method further comprising:

step 4, generating a task queue scheduling signal and quitting the frame tail interrupt service;

and step 5, reading the address in the delivery queue according to the task queue scheduling signal, setting the state of the address in the delivery queue to be a submittable state, and adding the submittable state to a completion queue, wherein the completion queue is used for providing the stored image data frame for a user.

9. The CSI camera high frame rate image acquisition method of claim 1, wherein the acquisition queue comprises a plurality of buffer frame addresses, and the first buffer frame address is a buffer frame address in the acquisition queue.

10. The CSI camera high frame rate image acquisition method of claim 1, wherein the image processing platform is the nvidia jetson tx2 platform.

Technical Field

The application relates to the technical field of image acquisition, in particular to a high-frame-rate image acquisition method for a CSI camera.

Background

A high-performance embedded vision and AI platform nvidia jetson tx2, which is introduced by Invida nvidia in 2019, integrates a Camera Serial Interface (CSI) image transmission Interface, and adopts a CSI transmission protocol to acquire image data. In the CSI transmission protocol, in order to separate image data on a transmission line from one frame to another, the CSI transmission protocol requires that a frame header signal (start of frame) and a frame trailer signal (end of frame) be added at the start and end positions of transmitting the image data, respectively, in order to resolve accurate single-frame image data.

The platform adopts a double-thread mode to collect images, and comprises a collection thread and a transmission completion waiting thread. In the field of industrial cameras, requirements for continuity of an image acquisition process and integrity of an image are high, and a trigger operation of a user needs to be supported, that is, the user sends a trigger signal, and an image sensor outputs a frame of acquired image. Through testing, the platform has at least the following problems:

(1) because the thread in the platform operation process is not always operated at all times, and a series of related operations need to be executed in sequence in the thread. As shown in fig. 1, since the frame header signal triggering image acquisition is completely controlled by the user, when a specific phase of the frame header signal in the acquisition thread arrives, such as a phase of configuring an acquisition buffer frame address, a phase of starting image transmission, and a phase of processing an error timeout, the acquisition thread cannot capture the frame header signal, so that a corresponding buffer frame address cannot be determined, and particularly when the acquisition frame rate of the image reaches above 330fps, the platform may lose frames, resulting in poor real-time performance of the platform image acquisition.

(2) Because image data can be interfered by external environment in the transmission process, a phenomenon that a frame header signal and/or a frame tail signal are lost exists, and timeout time is generally required to be set in order to ensure that a thread can normally run. However, in the thread waiting for completion of transmission, as shown in fig. 2, when the resolution of the acquired image is large and the data of one frame of image is large, the frame end signal cannot be received within the set timeout period (e.g., 200ms), and the frame image is determined to be an erroneous frame and discarded by the thread waiting for completion of transmission, which results in a frame loss phenomenon on the platform.

Disclosure of Invention

The purpose of this application lies in: the problems that the real-time performance is poor in the process of acquiring the image data of the nvidia jetson tx2 platform and image frame loss is easy to occur in the process of acquiring the thread image are solved.

The technical scheme of the application is as follows: the method is suitable for an image processing platform, a CSI controller is arranged in the image processing platform, and the method comprises the following steps: step 1, setting a first cache frame address in a CSI controller as a second cache frame address; step 2, when judging that the frame header signal of the image data frame identified by the CSI controller meets a frame header interrupt threshold, entering frame header interrupt service, reading a cache frame address in an acquisition queue to configure the cache frame address as a third cache frame address, and storing the image data frame according to a second cache frame address in the CSI controller; and 3, when the frame tail signal of the image data frame identified by the CSI controller is judged to meet the frame tail interruption threshold, entering frame tail interruption service, adding the second buffer frame address to the delivery queue, and configuring the third buffer frame address as the second buffer frame address for storing the next frame of image data frame in the CSI controller.

In any of the above technical solutions, further, the image data frame is sent to the CSI controller by the image sensor, and the image sensor configures a frame header signal and a frame trailer signal of the image data frame according to the CSI protocol.

In any one of the above technical solutions, further, before step 1, the method further includes: and the CSI controller reads a first buffer frame address in the acquisition queue and records the first buffer frame address as a reserved buffer frame address, and reads a second buffer frame address in the acquisition queue and records the second buffer frame address as a first buffer frame address.

In any one of the above technical solutions, further, the step 2 specifically includes: step 21, when judging that the frame header signal of the image data frame identified by the CSI controller meets a frame header interrupt threshold, entering frame header interrupt service, and judging whether an acquisition queue is empty; step 22, if yes, configuring the reserved buffer frame address as a third buffer frame address, and if not, reading the buffer frame address in the acquisition queue and configuring the buffer frame address as the third buffer frame address; and step 23, storing the image data frame according to the second cache frame address in the CSI controller, starting the transmission of the next frame of image data frame, resetting a frame header interrupt threshold value, and exiting frame header interrupt service.

In any one of the above technical solutions, further, step 3 specifically includes: step 31, when it is determined that the end-of-frame signal of the image data frame identified by the CSI controller satisfies the end-of-frame interrupt threshold, entering an end-of-frame interrupt service, and determining whether the second buffer frame address is a reserved buffer frame address; step 32, if yes, discarding the corresponding stored data in the second buffer frame address, and if no, adding the second buffer frame address to the delivery queue; and step 33, configuring the third buffer frame address as a second buffer frame address for storing the next frame of image data frame in the CSI controller, resetting the end-of-frame interrupt threshold, and exiting the end-of-frame interrupt service.

In any one of the above technical solutions, further, before step 23, the method further includes: judging whether the state of the transmission zone bit is in an enabling state, if so, performing frame tail exception processing on the previous frame of image data frame, and if not, modifying the transmission zone bit into the enabling state, wherein the frame tail exception processing process specifically comprises the following steps: step A, judging whether a second cache frame address for storing a previous frame of image data frame is a reserved cache frame address; and step B, if yes, saving the third cache frame address as a second cache frame address for storing the current image data frame, otherwise, carrying out residual frame marking on the second cache frame address for storing the previous image data frame, and adding the marked second cache frame address to a delivery queue.

In any one of the above technical solutions, further, before step 33, the method further includes: judging whether the state of the transmission zone bit is in an enabling state, if so, modifying the state of the transmission zone bit into a clearing state, and if not, performing frame header exception processing on the current image data frame, wherein the process of the frame header exception processing specifically comprises the following steps: step C, judging whether the acquisition queue is empty, if so, configuring the reserved cache frame address as a third cache frame address, and if not, reading the cache frame address in the acquisition queue and configuring the cache frame address as the third cache frame address; and D, configuring the second buffer frame address in the CSI controller as an address for storing the next frame of image data frame, and starting the transmission of the next frame of image data frame.

In any of the above technical solutions, further, the method further includes: step 4, generating a task queue scheduling signal and quitting the frame tail interrupt service; and step 5, reading the address in the delivery queue according to the task queue scheduling signal, setting the state of the address in the delivery queue to be a submittable state, and adding the submittable state to a completion queue, wherein the completion queue is used for providing the stored image data frame for a user.

In any of the above technical solutions, further, the acquisition queue includes a plurality of buffer frame addresses, and the first buffer frame address is a buffer frame address in the acquisition queue.

In any of the above technical solutions, further, the image processing platform is an nvidia jetson tx2 platform.

The beneficial effect of this application is:

according to the technical scheme, the frame head and the frame tail signals are used for triggering the interrupt service, so that the frame head interrupt service and the frame tail interrupt service are used for respectively configuring the cache frame addresses for storing the image data frames, the long preparation time before the cache address is configured in the thread image acquisition process is avoided, the real-time performance of the image is improved, and the problem of frame loss under the high acquisition frame frequency is solved.

In the preferred implementation manner of the present application, a reserved buffer frame address is further set, and frame head interrupt service and frame tail interrupt service are optimized correspondingly, so as to ensure that the interrupt service can obtain an effective buffer address, and further solve the problem of image frame loss caused by abnormal phenomena such as blocking, abnormal operation and the like of the interrupt service.

In a preferred implementation manner of the present application, the state of the transmission flag is determined by adding the transmission flag and using a frame header interrupt service and a frame tail interrupt service, so as to solve the abnormal problems of image data frame acquisition interruption, image acquisition failure and the like caused by loss of frame header and frame tail signals, and improve the reliability of image data acquisition by using a double interrupt service in an nvidia jetson tx2 platform.

Drawings

The advantages of the above and/or additional aspects of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a diagram illustrating a frame loss phenomenon in a collection thread in the prior art;

FIG. 2 is a diagram illustrating a frame loss phenomenon of a thread waiting for transmission completion in the prior art;

fig. 3 is a schematic flow diagram of a CSI camera high frame rate image acquisition method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a thread scheme versus interrupt scheme timing comparison according to one embodiment of the present application;

FIG. 5 is a schematic flow diagram of acquisition initiation process optimization according to one embodiment of the present application;

FIG. 6 is a schematic flow chart diagram of frame header interrupt service optimization according to one embodiment of the present application;

FIG. 7 is a schematic flow chart diagram of end-of-frame breakout service optimization according to one embodiment of the present application;

FIG. 8 is a schematic flow chart diagram of frame header interrupt service optimization according to another embodiment of the present application;

FIG. 9 is a schematic flow chart diagram of end-of-frame breakout service optimization according to another embodiment of the present application;

FIG. 10 is a schematic flow chart diagram of a task queue processing procedure according to one embodiment of the present application.

Detailed Description

In order that the above objects, features and advantages of the present application can be more clearly understood, the present application will be described in further detail with reference to the accompanying drawings and detailed description. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, however, the present application may be practiced in other ways than those described herein, and therefore the scope of the present application is not limited by the specific embodiments disclosed below.

The first embodiment is as follows:

as shown in fig. 3, the present embodiment provides a CSI camera high frame rate image acquisition method, which is applicable to an image processing platform, where the image processing platform is an embedded vision and AI platform nvidia jetson tx2, and a CSI controller is disposed in the image processing platform.

Through a high acquisition frame frequency test, when the acquisition frame rate reaches over 330fps, the platform can lose frames, and the real-time performance and the continuity of image acquisition are influenced. Through the analysis of the platform dual-thread image acquisition architecture, researchers find that, because the time occupied by each step in the execution process of the acquisition thread is long, if the acquisition thread is not scheduled in time or a frame header signal arrives at a specific stage in the acquisition thread, such as a stage of configuring an acquisition buffer frame address, a stage of starting image transmission and a stage of processing timeout errors, a corresponding buffer frame address cannot be configured before the frame header signal arrives, and at the moment, image transmission can be continued only after the arrival of the next frame header signal, so that an image corresponding to the current frame header signal is lost, and image frame loss is generated.

Therefore, in the embodiment, the original dual-thread service is replaced by the dual-interrupt service, because the priority of the interrupt is far higher than that of the thread, the interrupt service is generally executed immediately as long as the interrupt signal arrives, and the interrupt service is not interrupted by other low-priority interrupts and is not interrupted by system kernel scheduling, the real-time performance of the interrupt service is good.

In this embodiment, in order to acquire and store image data frames through a dual interrupt service, the address queues configured to store the image data frames in this embodiment include three address queues, which are an acquisition queue, a delivery queue, and a completion queue, respectively, where the acquisition queue includes a plurality of buffer frame addresses, and each buffer frame address corresponds to one data storage space.

When the image frame data is stored, a buffer frame address is extracted from the acquisition queue for data storage, and the buffer frame address is added to the delivery queue after the data storage is completed and then added to the completion queue so as to be submitted to a user.

In this embodiment, the process of acquiring an image data frame by using a dual interrupt service is at least divided into an acquisition start process, a process of switching a buffer frame address by using the dual interrupt service, and a task queue processing process, and the method specifically includes:

step 1, setting a first cache frame address in a CSI controller as a second cache frame address current buffer, wherein the second cache frame address current buffer is a storage address for storing an image data frame, the image data frame is sent to the CSI controller by an image sensor, and the image sensor configures a frame header signal and a frame tail signal of the image data frame according to a CSI protocol;

specifically, the task queue is initialized in the acquisition starting process, and a buffer frame address is read from the acquisition queue as a first buffer frame address for storing the image data frame in the CSI controller, so that when a frame header signal of a first frame image data frame is detected, the first buffer frame address can be directly used as a buffer frame address currently performing image transmission, that is, a second buffer frame address current buffer.

Step 2, when judging that the frame header signal of the image data frame identified by the CSI controller meets the frame header interrupt threshold, entering frame header interrupt service, reading a cache frame address in an acquisition queue to configure the cache frame address as a third cache frame address future buffer, and storing the image data frame according to a second cache frame address current buffer in the CSI controller;

specifically, in the process of switching the address of the buffer frame by the dual interrupt service, the following steps are first set: and when detecting that a frame head signal of the image data frame meets a frame head interrupt threshold, triggering frame head interrupt service, and taking out a cache frame address from the acquisition queue by the frame head interrupt service to be used as a storage position of the next frame of image data frame, namely a third cache frame address future buffer, and configuring the third cache frame address future buffer to the CSI controller to be used as a position for filling the subsequent image data frame.

And then, starting the transmission of the current image data frame by enabling the transmission register corresponding to the CSI controller, and storing the current buffer to the second buffer frame address.

It should be noted that the frame header interrupt threshold in this embodiment may be set to 1, that is, each time a frame header signal is received, the entering of the frame header interrupt service is triggered.

And 3, when the frame tail signal of the image data frame identified by the CSI controller is judged to meet the frame tail interrupt threshold, entering frame tail interrupt service, adding the second buffer frame address current buffer to a delivery queue, configuring the third buffer frame address future buffer as the second buffer frame address current buffer for storing the next frame of image data frame in the CSI controller, and using the delivery queue for outputting the image data frame.

Specifically, when it is detected that the end signal of the image data frame satisfies the end interrupt threshold, the end interrupt service is triggered, and the end interrupt service first adds the second buffer frame address current buffer, which is already filled with the current image frame data, to the delivery queue for subsequent data output. And then, configuring the third buffer frame address future buffer determined in the frame header interrupt service as the second buffer frame address current buffer for storing the next frame of image data frame.

In this embodiment, the process of switching the cache frame address by the dual interrupt service is realized by detecting the frame header signal and the frame tail signal, that is, in the frame header interrupt service, the cache frame address of the next frame is read in advance according to the frame header signal of the current image data frame; in the frame end interrupt service, the buffer frame address read in the frame head interrupt service is configured to be the buffer frame address of the next frame of image frame data stored by the CSI controller according to the frame end signal. The process of switching the buffer frame address by the double interrupt service optimizes the mode of configuring the buffer frame address of the image data frame, shortens the time of configuring the address and is beneficial to solving the problem of frame loss in the process of acquiring the high frame rate image.

As shown in fig. 4, when the frame rate of the acquisition reaches 330fps or more, the buffer frame address of the second frame image data cannot be configured in time in the acquisition thread, which results in the loss of the second frame image data. By adopting the technical scheme in the embodiment, since the third buffer frame address future buffer is configured in advance after the frame header signal of the first frame of image data is detected to enter the frame header interrupt service, when the frame header signal of the second frame of image data arrives, the real-time storage of the second frame of image data can be realized, and the real-time performance of the platform image acquisition is optimized.

Through actual measurement, the scheme in the embodiment can achieve no frame loss at 3000fps, and because no overtime process such as a thread scheme exists in the acquisition process, the problems of abnormal frames, residual frames and the like caused by triggering acquisition or overlarge resolution can be solved, and the image acquisition performance of the nvidia jetson tx2 platform is improved.

Example two:

in the process of carrying out double-interruption image data acquisition, statistics on a large number of image data acquisition processes shows that when the acquisition frame rate is high, the situation that the double-interruption scheme program is blocked or even cannot normally operate in the process of entering a field is generated, and a frame loss phenomenon is accompanied when the situation occurs.

Research shows that when the frequency of an acquisition frame in a certain time period is suddenly increased, or when a user takes out all buffer frame addresses in an acquisition queue and does not receive a newly released buffer frame address in a system, the situation that an effective buffer frame address cannot be read from the acquisition queue after the frame header interrupt service is started is caused, namely, the frame header interrupt service takes out an empty buffer frame address which does not have an address space for storing an image data frame, so that the subsequent frame tail interrupt service cannot be normally executed, that is, the next frame header interrupt service cannot be prepared and the transmission of the next frame image data frame is started, and thus an image processing platform is crashed or jammed.

In this embodiment, in order to prevent the image processing platform from being stuck, crashed or jammed, on the basis of the above embodiment, the embodiment optimizes the high frame rate image capturing method, as shown in fig. 5, before step 1, the method further includes:

and the CSI controller reads a first buffer frame address in the acquisition queue and records the first buffer frame address as a reserved buffer frame address, and reads a second buffer frame address in the acquisition queue and records the second buffer frame address as a first buffer frame address.

Specifically, in the acquisition starting process, the task queue is initialized, and a buffer frame address is extracted from the acquisition queue and taken out as a reserved buffer frame address. And if the condition that the effective cache frame address cannot be read from the acquisition queue after the frame header interrupt service is entered occurs, using the reserved cache frame address to receive the image data frame.

It should be noted that, because the above situation only occurs at the ultra-high acquisition frame rate, the processing bandwidth of the platform is already exceeded, and frame loss is unavoidable, the reserved buffer frame address may not be submitted to the delivery queue, and is only used to ensure that the frame header and the frame tail interrupt service can normally operate, and ensure that the acquisition start process normally operates.

And then, taking out the second buffer frame address from the acquisition queue, recording the second buffer frame address as the first buffer frame address, and taking the second buffer frame address current buffer of the first frame of image data frame after the acquisition starting process. And setting a frame head interruption threshold and a frame tail interruption threshold, and starting a CSI image transmission channel and image sensor image data transmission.

It should be noted that the image sensor is a source of image data, and when a frame header signal in the image data of the image sensor arrives, a frame header interrupt service is started; when the end of frame signal comes, the end of frame interrupt service is started. The acquisition starting process is executed only once and can be used as an entrance of a second stage 'double-interrupt service switching buffer frame address process', and the subsequent cyclic image acquisition process is realized by the second stage.

Correspondingly, step 2 and step 3 in the above embodiment are optimized, as shown in fig. 6, step 2 specifically includes:

step 21, when judging that the frame header signal of the image data frame identified by the CSI controller meets a frame header interrupt threshold, entering frame header interrupt service, and judging whether an acquisition queue is empty;

step 22, if yes, configuring the reserved buffer frame address as a third buffer frame address future buffer,

if not, reading the buffer frame address in the acquisition queue to configure the buffer frame address as a third buffer frame address future buffer;

and step 23, storing the image data frame according to the second buffer frame address current buffer in the CSI controller, starting the transmission of the next frame of image data frame, resetting a frame header interrupt threshold value, and exiting the frame header interrupt service.

Specifically, when the current image data frame has been transmitted to the head or the tail of the frame, two interrupt services in the second stage are triggered, and the two interrupt services configure the address of the buffer frame to be used when the next image data frame is transmitted.

Therefore, in the process of frame header interrupt service, it is necessary to judge whether the acquisition queue is empty, and if so, in order to ensure that the interrupt service can normally operate, the reserved buffer frame address is configured as a third buffer frame address future buffer; and if not, configuring the buffer frame address in the read acquisition queue as a third buffer frame address future buffer. Enabling a transmission register corresponding to the CSI controller, starting the transmission of the current next frame of image data frame, and storing the image data frame according to a second buffer frame address current buffer in the CSI controller.

And sets (resets) the frame header interrupt threshold value for triggering the frame header interrupt service next time, and prepares for the next frame image data frame transmission to enter the frame header interrupt. And then, enabling the frame head to be interrupted, waiting for the arrival of a frame head signal of the next frame of image data frame, and ending the frame head interruption service.

In this embodiment, the main function of the frame header interrupt service is to set an address for transmitting the next image data frame, start transmission of the image data frame corresponding to the current frame header signal, simultaneously save the address as the third buffer frame address future buffer, and provide the frame tail interrupt service with the second buffer frame address current buffer to replace.

As shown in fig. 7, step 3 specifically includes:

step 31, when it is determined that the frame tail signal of the image data frame identified by the CSI controller satisfies the frame tail interrupt threshold, entering a frame tail interrupt service, and determining whether the second buffer frame address current buffer is the reserved buffer frame address;

step 32, if yes, discarding the corresponding stored data in the second buffer frame address current buffer, if no, adding the second buffer frame address current buffer to the delivery queue;

and step 33, configuring the third buffer frame address future buffer as the second buffer frame address current buffer for storing the next frame of image data frame in the CSI controller, resetting the frame end interrupt threshold, and exiting the frame end interrupt service.

Specifically, after entering the end-of-frame interrupt service, it is determined whether an address for storing the current image data frame is a reserved buffer frame address. If so, the reserved buffer frame address is not submitted to the delivery queue, so that the stored data is discarded, and the third buffer frame address future buffer determined in the frame header interrupt service is directly configured as the second buffer frame address current buffer for storing the next frame of image data frame in the CSI controller.

If not, adding the second buffer frame address current buffer to the delivery queue, and then configuring the third buffer frame address future buffer determined in the frame header interrupt service as the second buffer frame address current buffer for storing the next frame of image data frame in the CSI controller.

And setting a frame tail interruption threshold value for triggering frame tail interruption service next time, and preparing for the next frame of image data frame transmission to enter frame tail interruption. And enabling the end-of-frame interrupt, waiting for the frame of the next frame of image data frame to arrive as a signal, and exiting the end-of-frame interrupt service.

In this embodiment, the frame end interrupt service mainly functions to hand over the second buffer frame address current buffer where the transmission of the current image data frame is completed to the upper layer, and set the third buffer frame address future buffer to the second buffer frame address current buffer.

Example three:

as can be understood by those skilled in the art, in the nvidia jetson tx2 platform, image data is transmitted on a wire through a level signal specified by the CSI protocol, and due to factors such as electromagnetic interference, the level signal transmitted on the wire may have a jitter phenomenon, and in a serious case, the CSI controller may incorrectly identify the level signal. When the CSI controller checks the signals of the frame head and the frame tail, if a certain level signal is identified incorrectly, the CSI controller can directly discard the wrong data packet, so that the loss of the frame head and the frame tail occurs, and the subsequent data transmission is influenced.

In order to avoid abnormal problems of image data frame acquisition interruption, image non-acquisition and the like caused by loss of signals of a frame header and a frame tail in the process of double-interruption image data acquisition, on the basis of the above embodiment, the embodiment optimizes the high frame rate image acquisition method, as shown in fig. 8, before step 23, the method further includes:

judging whether the state of the transmission zone bit is in an enabling state, if so, performing frame tail exception processing on the previous frame of image data frame, and if not, modifying the transmission zone bit into the enabling state, wherein the frame tail exception processing process specifically comprises the following steps:

step A, judging whether a second buffer frame address current buffer for storing the previous frame of image data frame is a reserved buffer frame address;

step B, if yes, saving the third buffer frame address future buffer as the second buffer frame address current buffer for storing the current image data frame,

and if not, carrying out residual frame marking on the second buffer frame address current buffer for storing the previous frame of image data frame, and adding the marked second buffer frame address current buffer to the delivery queue.

Specifically, a variable is added to the frame header interrupt service as a transmission flag bit to determine whether the loss of the frame header and the frame tail signals occurs, and the following definitions are performed:

a. when the transmission is started or one image is transmitted, the transmission flag bit is cleared;

b. when the frame header arrives, the transmission flag bit is enabled.

Therefore, the following conclusions can be drawn:

1. under the normal transmission condition, the transmission flag bit in the frame header interrupt service is in a clear state, and the frame header interrupt service modifies the state of the transmission flag bit into an enable state; the transmission flag bit in the tail-of-frame interrupt service should be in an enabled state, and the tail-of-frame interrupt service will modify the state flag bit into a cleared state.

2. If the state of the transmission flag bit is enabled in the frame header interrupt service, it indicates that the frame end signal corresponding to the previous frame of image data frame is lost.

3. If the state of the transmission flag bit is a clear state in the frame end interrupt service, it indicates that the frame head signal of the current image data frame is lost.

Therefore, after the frame header interrupt service is entered and the third buffer frame address future buffer is configured, the state of the transmission flag bit is judged, and when the transmission flag bit is in the enabled state, it indicates that the frame end signal of the previous frame of image data frame is lost due to interference, and frame end exception processing needs to be performed, which specifically includes:

judging whether the second buffer frame address current buffer is a reserved buffer frame address, if so, saving the third buffer frame address future buffer as the second buffer frame address current buffer; if the buffer address is not the reserved buffer frame address, the buffer is marked as a residual frame and then added to the delivery queue, and the third buffer frame address future buffer is saved as the second buffer frame address current buffer. And adding the frame of image data to a completion queue from the delivery queue, and acquiring a corresponding state after the user acquires the frame of image data in the completion queue, and judging whether the frame of image data is a residual frame so as to select a subsequent processing mode by self.

When the clear status indicates that the image data frame is transmitted normally, step 23 is executed.

Similarly, as shown in fig. 9, before step 33, the method further includes:

judging whether the state of the transmission zone bit is in an enabling state, if so, modifying the state of the transmission zone bit into a clearing state, and if not, performing frame header exception processing on the current image data frame, wherein the process of the frame header exception processing specifically comprises the following steps:

step C, judging whether the acquisition queue is empty, if so, configuring the reserved buffer frame address as a third buffer frame address future buffer,

if not, reading the buffer frame address in the acquisition queue to configure the buffer frame address as a third buffer frame address future buffer;

and D, configuring the second buffer frame address in the CSI controller as an address for storing the next frame of image data frame, and starting the transmission of the next frame of image data frame.

Specifically, in the frame end interrupt service, if the transmission flag bit is not in the enabled state, indicating that the frame header signal of the current image data frame is lost, at this time, a process similar to that in the frame header interrupt service is performed:

judging whether the acquisition queue is empty, if so, configuring the reserved cache frame address as a third cache frame address future buffer in order to ensure that the interrupt service can normally run; and if not, configuring the buffer frame address in the read acquisition queue as a third buffer frame address future buffer. Enabling a transmission register corresponding to the CSI controller, starting the transmission of the current next frame of image data frame, and storing the image data frame according to a second buffer frame address current buffer in the CSI controller.

Example four:

on the basis of the above embodiment, in order to shorten the interrupt processing time and avoid the problem that the interrupt service time is too long, which may cause other platform services to be unable to be scheduled in time, thereby causing deadlock and performance degradation, the embodiment puts some data processing flows that are not necessarily executed in the interrupt service into the task queue processing process, so that before the end of the frame end interrupt service, it is set that it sends a task queue scheduling signal, and the specific task is scheduled and executed by the platform, so as to greatly shorten the execution time of the interrupt service and ensure the scheduling performance and response performance of the system.

As shown in fig. 10, the method further includes:

step 4, generating a task queue scheduling signal and quitting the frame tail interrupt service;

and step 5, reading the address in the delivery queue according to the task queue scheduling signal, setting the state of the address in the delivery queue to be a submittable state, and adding the submittable state to a completion queue, wherein the completion queue is used for providing the stored image data frame for a user.

Specifically, after the address in the delivery queue is added to the completion queue, the camera driving frame may send an image completion signal to notify the user that the image data can be taken out from the completion queue.

The technical scheme of the present application is described in detail above with reference to the accompanying drawings, and the present application provides a CSI camera high frame rate image acquisition method, which includes: step 1, setting a first cache frame address in a CSI controller as a second cache frame address; step 2, when judging that the frame header signal of the image data frame identified by the CSI controller meets a frame header interrupt threshold, entering frame header interrupt service, reading a cache frame address in an acquisition queue to configure the cache frame address as a third cache frame address, and storing the image data frame according to a second cache frame address in the CSI controller; and 3, when the frame tail signal of the image data frame identified by the CSI controller is judged to meet the frame tail interruption threshold, entering frame tail interruption service, adding the second buffer frame address to the delivery queue, and configuring the third buffer frame address as the second buffer frame address for storing the next frame of image data frame in the CSI controller. By the technical scheme, the problems that real-time performance is poor in an image data acquisition process of an English-Webda platform and image loss is easy to occur in a thread image acquisition process are solved.

The steps in the present application may be sequentially adjusted, combined, and subtracted according to actual requirements.

The units in the device can be merged, divided and deleted according to actual requirements.

Although the present application has been disclosed in detail with reference to the accompanying drawings, it is to be understood that such description is merely illustrative and not restrictive of the application of the present application. The scope of the present application is defined by the appended claims and may include various modifications, adaptations, and equivalents of the invention without departing from the scope and spirit of the application.