Target algorithm testing method, device and system

文档序号：1415834 发布日期：2020-03-10 浏览：22次中文

阅读说明：本技术 一种目标算法的测试方法、装置和系统 (Target algorithm testing method, device and system ) 是由倪晶静谢春宁刘有文于 2018-08-29 设计创作，主要内容包括：本发明实施例提供了一种目标算法的测试方法、装置和系统。所述方法包括：摄像设备向解码设备发送摄像设备采集的原始数据,原始数据包括摄像设备录制的视频流或摄像设备采集的第一格式的图像数据；解码设备与摄像设备通过以太网接口实现通信连接；摄像设备接收解码设备发送的当前帧图像数据,当前帧图像数据是解码设备对原始数据进行解码后得到的第二格式的单帧图像数据；摄像设备对当前帧图像数据进行预处理；摄像设备将预处理后的数据作为目标算法的测试数据,并执行目标算法的测试流程,得到当前帧图像数据对应的测试结果。应用本发明实施例提供的方案可以测试目标算法在实际应用时的运行效果。(The embodiment of the invention provides a method, a device and a system for testing a target algorithm. The method comprises the following steps: the method comprises the steps that the camera shooting equipment sends original data collected by the camera shooting equipment to the decoding equipment, wherein the original data comprises video streams recorded by the camera shooting equipment or image data in a first format collected by the camera shooting equipment; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface; the method comprises the steps that the camera shooting equipment receives current frame image data sent by the decoding equipment, wherein the current frame image data is single frame image data in a second format obtained after the decoding equipment decodes original data; the method comprises the steps that image data of a current frame are preprocessed by the camera equipment; and the camera equipment takes the preprocessed data as the test data of the target algorithm and executes the test flow of the target algorithm to obtain the test result corresponding to the current frame image data. The scheme provided by the embodiment of the invention can be used for testing the running effect of the target algorithm in actual application.)

1. A method for testing a target algorithm, the method comprising:

the method comprises the steps that the camera shooting equipment sends original data collected by the camera shooting equipment to decoding equipment, wherein the original data comprises video streams recorded by the camera shooting equipment or image data in a first format collected by the camera shooting equipment; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface;

the image pickup equipment receives current frame image data sent by the decoding equipment, wherein the current frame image data is single frame image data in a second format obtained after the decoding equipment decodes the original data;

the camera equipment preprocesses the current frame image data;

and the camera equipment takes the preprocessed data as the test data of the target algorithm and executes the test flow of the target algorithm to obtain the test result corresponding to the current frame image data.

2. The method according to claim 1, wherein before the step of the image pickup apparatus receiving the current frame image data transmitted by the decoding apparatus, the method further comprises:

the camera equipment and the decoding equipment establish Socket programming interface communication connection;

the step in which the image pickup apparatus receives the current frame image data transmitted by the decoding apparatus includes:

the camera equipment receives Socket indication information sent by the decoding equipment, wherein the Socket indication information is used for indicating the camera equipment to read the current frame image data according to the mounted shared directory; and the Socket indication information is sent by the decoding equipment after the current frame image data is obtained and written into the storage area corresponding to the shared directory.

3. The method according to claim 1, characterized in that the camera device comprises a main processor and at least one co-processor, the target algorithm comprising at least a first sub-algorithm and a second sub-algorithm;

the main processor is configured to: receiving current frame image data sent by the decoding equipment, preprocessing the current frame image data and sending the preprocessed current frame image data to the coprocessor;

the coprocessor is configured to: taking the preprocessed current frame image data as test data of a first sub-algorithm, executing a test flow of the first sub-algorithm to obtain a test result corresponding to the current frame image data, and sending the test result to a main processor;

the main processor is further configured to: outputting a test result from the coprocessor; or taking the test result from the coprocessor as test data of a second sub-algorithm, and executing the test flow of the second sub-algorithm to obtain the test result of the second sub-algorithm.

4. The method according to claim 2, wherein prior to the step of the image pickup apparatus preprocessing the current frame image data, the method further comprises:

the image pickup equipment reads image data from the current frame image data and buffers the image data in a first annular buffer; or reading image data and image parameters from the current frame image data, caching the image data in the first ring buffer, and caching the image parameters in the second ring buffer; and

and sending response information of the Socket indication information to the decoding equipment, wherein the response information is used for indicating the decoding equipment to send the next frame of image data to the camera equipment.

5. The method according to claim 4, wherein the step of the image pickup apparatus preprocessing the current frame image data includes:

acquiring image data of the current frame image data from the first annular buffer, and performing at least one of compression, clipping and format conversion on the image data to obtain image data of an interested area;

acquiring image parameters of the current frame image data from the second ring buffer;

acquiring algorithm parameters of the target algorithm;

and generating preprocessed data according to the algorithm parameters, the image parameters of the current frame image data and the image data of the region of interest.

6. The method of any one of claims 1 to 5, wherein the target algorithm is at least one of an image processing algorithm, an image classification algorithm, and an image recognition algorithm, the target algorithm comprising one or more sub-algorithms.

7. A method for testing a target algorithm, the method comprising:

the method comprises the steps that a decoding device receives original data collected by a camera device, wherein the original data comprises a video stream recorded by the camera device or image data in a first format collected by the camera device; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface;

the decoding equipment decodes the original data to obtain current frame image data, wherein the current frame image data is single frame image data in a second format;

the decoding apparatus transmits current frame image data to the image pickup apparatus.

8. The method according to claim 7, wherein before the step of the decoding apparatus transmitting the current frame image data to the image pickup apparatus, the method further comprises:

the decoding equipment and the camera equipment establish Socket programming interface communication connection;

the step in which the decoding apparatus transmits current frame image data to the image pickup apparatus includes:

after the decoding device decodes the current frame image data, writing the current frame image data into a storage area corresponding to a shared directory; and are

And sending Socket indication information to the image pickup equipment, wherein the Socket indication information is used for indicating the image pickup equipment to read the current frame image data according to the mounted shared directory.

9. The method according to claim 7 or 8, wherein after the step of the decoding apparatus transmitting the current frame image data to the image pickup apparatus, the method further comprises:

the decoding equipment receives response information of Socket indication information sent by the camera equipment, wherein the response information is sent by the camera equipment after the camera equipment reads image data from the current frame image data and buffers the image data in a first annular buffer; or, the response information is sent after the image pickup apparatus reads image data and image parameters from the current frame image data, buffers the image data in the first ring buffer, and buffers the image parameters in the second ring buffer;

and the decoding equipment transmits the next frame of image data to the camera equipment according to the response information.

10. A test apparatus for a target algorithm, the test apparatus being located in an image pickup device, the apparatus comprising:

the original data sending module is used for sending original data collected by the camera equipment to decoding equipment, wherein the original data comprises video streams recorded by the camera equipment or image data in a first format collected by the camera equipment; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface;

an image data receiving module, configured to receive current frame image data sent by the decoding device, where the current frame image data is single frame image data in a second format obtained after the decoding device decodes the original data;

the image data processing module is used for preprocessing the current frame image data by the camera equipment;

and the algorithm testing module is used for taking the preprocessed data as the testing data of the target algorithm and executing the testing process of the target algorithm to obtain the testing result corresponding to the current frame image data.

11. A test apparatus for a target algorithm, the test apparatus being located in a decoding device, the apparatus comprising:

the system comprises an original data receiving module, a data processing module and a data processing module, wherein the original data receiving module is used for receiving original data collected by a camera device, and the original data comprises a video stream recorded by the camera device or image data in a first format collected by the camera device; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface;

the original data decoding module is used for decoding the original data to obtain current frame image data, and the current frame image data is single frame image data in a second format;

and the image data sending module is used for sending the current frame image data to the camera equipment.

12. A system for testing a target algorithm, comprising: an image pickup apparatus and a decoding apparatus;

the camera device is used for sending original data collected by the camera device to a decoding device, wherein the original data comprises a video stream recorded by the camera device or image data in a first format collected by the camera device; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface;

the decoding equipment is used for receiving the original data collected by the camera equipment; decoding the original data to obtain current frame image data; sending current frame image data to the camera equipment; the current frame image data is single frame image data in a second format;

the camera device is also used for receiving the current frame image data sent by the decoding device; preprocessing the current frame image data; and taking the preprocessed data as test data of a target algorithm, and executing a test flow of the target algorithm to obtain a test result corresponding to the current frame image data.

13. The camera equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-6 when executing a program stored in the memory.

14. The decoding device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 7 to 9 when executing a program stored in the memory.

15. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.

16. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any of the claims 7-9.

Technical Field

The present invention relates to the field of testing technologies, and in particular, to a method, an apparatus, and a system for testing a target algorithm.

Background

Algorithms involved in the field of video surveillance are used to identify and analyze targets and their behavior in videos.

Since the development process of the algorithm is implemented on other devices except the image pickup device, in order to verify the validity of the algorithm, after the algorithm is developed, the operation effect of the algorithm when the algorithm is applied to the image pickup device needs to be tested. Therefore, how to quickly and effectively test the operation effect of the algorithm in application is an urgent problem to be solved.

Disclosure of Invention

The embodiment of the invention aims to provide a method, a device and a system for testing a target algorithm, a camera device, a decoding device and a computer readable storage medium, so as to test the running effect of the algorithm in practical application. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a method for testing a target algorithm, where the method includes:

the camera equipment preprocesses the current frame image data;

In a second aspect, an embodiment of the present invention further provides a method for testing a target algorithm, where the method includes:

the decoding equipment decodes the original data to obtain current frame image data, wherein the current frame image data is single frame image data in a second format;

the decoding apparatus transmits current frame image data to the image pickup apparatus.

In a third aspect, an embodiment of the present invention further provides a device for testing a target algorithm, where the device is located in an image capturing apparatus, and the device includes:

the image data processing module is used for preprocessing the current frame image data by the camera equipment;

In a fourth aspect, an embodiment of the present invention further provides a device for testing a target algorithm, where the device is located in a decoding apparatus, and the device includes:

the original data decoding module is used for decoding the original data to obtain current frame image data, and the current frame image data is single frame image data in a second format;

and the image data sending module is used for sending the current frame image data to the camera equipment.

In a fifth aspect, an embodiment of the present invention further provides a test system for a target algorithm, including: an image pickup apparatus and a decoding apparatus;

In a sixth aspect, an embodiment of the present invention further provides an image capturing apparatus, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the testing method of the target algorithm in the first aspect when executing the program stored in the memory.

In a seventh aspect, an embodiment of the present invention further provides a decoding device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the testing method of the target algorithm in the second aspect when executing the program stored in the memory.

In an eighth aspect, the embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method for testing the target algorithm of the first aspect.

In a ninth aspect, the embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method for testing the target algorithm of the second aspect.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic connection diagram of a decoding apparatus and an image capturing apparatus according to an embodiment of the present invention;

fig. 2 is a schematic diagram of connection between another decoding apparatus and an image capturing apparatus according to an embodiment of the present invention;

fig. 3 is a schematic diagram of data interaction between a decoding apparatus and a main processor of an image capturing apparatus according to an embodiment of the present invention;

fig. 4 is a schematic flowchart of a method for testing a target algorithm applied to an image capturing apparatus according to an embodiment of the present invention;

fig. 5 is an interaction diagram of a decoding apparatus and an image capturing apparatus according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating an annular buffer according to an embodiment of the present invention;

FIG. 7 is a schematic diagram illustrating data flow of each sub-thread of a host processor and a sub-thread of a coprocessor according to an embodiment of the present invention;

FIG. 8 is a flowchart illustrating an operation of an image data management thread according to an embodiment of the present invention;

fig. 9 is a schematic flowchart of a working process of an image data sending thread according to an embodiment of the present invention;

FIG. 10 is a schematic flowchart illustrating a working process of a coprocessor algorithm application thread according to an embodiment of the present invention;

fig. 11 is a schematic flowchart illustrating a process of a coprocessor processing result receiving thread during operation according to an embodiment of the present invention;

fig. 12 is a schematic flowchart illustrating a process of processing a thread when a coprocessor processes a result according to an embodiment of the present invention;

fig. 13 is a flowchart illustrating a method for testing a target algorithm applied to a decoding device according to an embodiment of the present invention;

fig. 14 is a schematic flowchart of a decoding device according to an embodiment of the present invention during operation;

fig. 15 is a schematic structural diagram of a testing apparatus for a target algorithm applied to an image capturing device according to an embodiment of the present invention;

fig. 16 is a schematic structural diagram of a testing apparatus for a target algorithm applied to a decoding device according to an embodiment of the present invention;

fig. 17 is a schematic structural diagram of an image capturing apparatus according to an embodiment of the present invention;

fig. 18 is a schematic structural diagram of a decoding device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In general, the development environment of an algorithm in an image pickup apparatus is a windows environment; in real products, the algorithms in the camera device are typically run on the processors of the embedded device, which are typically linux environments. Therefore, it is necessary to test whether the operational effect of the algorithm in the image pickup apparatus is consistent with or close to the operational effect of the algorithm in the development environment. That is, how to quickly and effectively test the operation effect of the algorithm in practical application is an urgent problem to be solved.

In order to test the operation effect of a target algorithm in practical application, embodiments of the present invention provide a method, an apparatus, a system, a camera device, a decoding device, and a computer-readable storage medium for testing a target algorithm.

For clarity of description of the solution, in the first aspect, a detailed description is first made on a test system of a target algorithm provided in an embodiment of the present invention.

As shown in fig. 1, a test system of a target algorithm provided in an embodiment of the present invention may include a decoding device 110 and a camera device 120, where the camera device may include a main processor 121 and a coprocessor 122. In general, the image pickup apparatus is an image pickup type apparatus with an ISP image capturing module.

The decoding device 110 and the camera device 120 can be connected through an ethernet port; the main processor 121 and the coprocessor 122 are connected by internal hardware, so that the main processor 121 and the coprocessor 122 can perform data transmission. It is understood that the main processor 121 and the coprocessor 122 may be connected through a PCI-E bus, a USB bus, and the like, and of course, the connection manner of the main processor and the coprocessor is not particularly limited in the embodiment of the present invention.

The operating environment of the decoding equipment is a linux environment; the main processor and the coprocessor are both located in the camera equipment, and the running environment of the main processor and the coprocessor is also a linux environment. The embodiment of the invention can be suitable for the camera equipment which takes an ARM platform as a main processor framework and has a linux system environment on the ARM.

It should be noted that the target algorithm may include one or more algorithms, and the target algorithm may be split into two types of algorithms, which are: a first sub-algorithm and a second sub-algorithm; the first sub-algorithm can be packaged into an algorithm library corresponding to the coprocessor in advance, so that the first sub-algorithm can run on the coprocessor; the second sub-algorithm may be pre-packaged into a corresponding algorithm library of the main processor, such that the second sub-algorithm can run on the main processor.

The interaction process of the decoding apparatus and the image pickup apparatus will be described in detail below.

In practical applications, the flow of capturing images by the image capturing apparatus may be as follows: the image sensor generates an image data stream; the ISP module carries out image signal processing on the image data stream to obtain the image data stream after the image signal processing; and inputting the image data stream after the image signal processing into a target algorithm packaged in the camera equipment in advance, carrying out image processing on the image data after the image signal processing by the target algorithm to obtain a processing result, and uploading the single image frame of which the processing result meets the triggering snapshot rule to an upper layer application.

In the embodiment of the present invention, after the image capturing device acquires the raw data, the acquired raw data may be sent to the decoding device, where the raw data may be a video stream recorded by the image capturing device or image data acquired by the image capturing device. After receiving the original data sent by the image pickup device, the decoding device decodes the original data to obtain single-frame image data of one frame, namely, an image data stream is generated, so that the image data stream can be sent to a main processor of the image pickup device. That is, the decoding apparatus may functionally replace the image sensor and the ISP module. As shown in fig. 2, as can be seen from fig. 2, the decoding apparatus 210 may functionally replace the image sensor 220 and the ISP module 220 shown in the dashed line box of fig. 2, and the decoding apparatus 210 may transmit the image data stream to the main processor of the image capturing apparatus.

After the decoding apparatus obtains the single-frame image data, the single-frame image data may be transmitted to a main processor of the image pickup apparatus. Alternatively, in one embodiment, as shown in fig. 3, the decoding apparatus 310 and the image capturing apparatus 320 may establish a Socket programming interface communication connection. Since the operating environments of the decoding device and the image pickup device are both linux environments, after the decoding device decodes the single-frame image data, the single-frame image data can be written into the storage area corresponding to the shared directory and can send Socket indication information to the image pickup device, and after the image pickup device receives the Socket indication information, the main processor 321 of the image pickup device can acquire the single-frame image data from the mounted shared directory.

After the main processor of the decoding device acquires the single-frame image data, the main processor of the decoding device can preprocess the single-frame image data and send the preprocessed single-frame image data to the coprocessor, wherein the preprocessing can be image scaling, format conversion, matting operation of an interested region and the like. In an embodiment, after acquiring a frame of single-frame image data, a main processor of a decoding device may read image data from the frame of single-frame image data and buffer the image data in a first ring buffer, or read image data and image parameters from a current frame of image data, buffer the image data in the first ring buffer, buffer the image parameters in a second ring buffer, and send response information of Socket indication information to the decoding device, and after receiving the response information of the Socket indication information, the decoding device may write next frame of single-frame image data into a shared file. In this embodiment, when the main processor of the image capturing apparatus preprocesses single-frame image data, the main processor may obtain image data of the single-frame image data from the first ring buffer, may obtain image parameters of the single-frame image data from the second ring buffer, and obtain algorithm parameters of a target algorithm; and generating preprocessed data according to the algorithm parameters, the image parameters of the current frame image data and the image data of the interested area.

After receiving the preprocessed single-frame image data, the coprocessor can take the preprocessed current-frame image data as test data of a first sub-algorithm, execute a test flow of the first sub-algorithm to obtain a test result corresponding to the single-frame image data, and send the test result to the main processor;

after the main processor receives the test result sent by the coprocessor, the test result from the coprocessor can be directly output; or taking the test result from the coprocessor as the test data of the second sub-algorithm, executing the test flow of the second sub-algorithm to obtain the test result of the second sub-algorithm, and outputting the test result.

Therefore, the technical scheme provided by the embodiment of the scheme can achieve the purpose of testing the running effect of the target algorithm in actual application. In addition, the scheme provided by the embodiment of the invention uses the camera equipment as hardware for testing, does not need to design and set up a hardware environment independently, belongs to a testing means on a software level, and is convenient for realizing the testing of a target algorithm.

In a second aspect, an embodiment of the present invention provides a method for testing a target algorithm applied to an image capturing apparatus.

The target algorithm testing method provided by the embodiment of the invention can be applied to camera equipment. In a specific application, the image pickup apparatus may be: a camera device having an Image sensor and an ISP (Image Signal Processing) module is not limited to this. It is understood that the flow of the image capturing apparatus capturing the image may be: the image sensor generates image data; the ISP module carries out image signal processing on the image data to obtain image data after the image signal processing; and inputting the image data after the image signal processing into a target algorithm packaged in the camera equipment in advance, carrying out image processing on the image data after the image signal processing by using the target algorithm to obtain a processing result, and uploading the single frame of the image of which the processing result meets the triggering snapshot rule to an upper application.

In order to test the operation effect of the target algorithm, in the embodiment of the present invention, the image capturing apparatus may be communicatively connected to the decoding apparatus, for example, the image capturing apparatus and the decoding apparatus may be connected through an ethernet port by hardware, but is not limited thereto. The decoding device in the embodiment of the invention can take the image data collected by the camera device as input, decode the image data into single frame image data, send each frame image data obtained by decoding to the camera device for processing, preprocess each frame image data after the camera device receives each frame image data to obtain preprocessed data, take the preprocessed data as the test data of the target algorithm, and execute the test flow of the target algorithm, thereby realizing the test of the target algorithm.

Moreover, the image data acquired by the camera device may be a video stream, that is, a video file recorded by the camera device, or an image file in a format such as jpeg, raw, or the like; therefore, the decoding apparatus can decode not only the video file but also the image file in the format of jpeg, raw, or the like, and can decode each file in the directory containing the video file or the image file.

For example, a directory contains a plurality of video files, and the decoding device can traverse each video file to be decoded under the directory and decode each video file to be decoded; for another example, a directory contains several jpeg format image files, and the decoding device can traverse each jpeg image file to be decoded under the directory and decode each jpeg image file to be decoded.

Moreover, after the decoding apparatus decodes the image data acquired by the imaging apparatus, the format of the obtained single frame image data is generally the YUV format.

A method for testing a target algorithm applied to an image capturing apparatus according to an embodiment of the present invention will be described in detail below.

As shown in fig. 4, a method for testing a target algorithm applied to an image capturing apparatus according to an embodiment of the present invention includes the following steps:

s410, the camera shooting device sends original data collected by the camera shooting device to the decoding device, wherein the original data comprises video streams recorded by the camera shooting device or image data of a first format collected by the camera shooting device; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface.

The image pickup apparatus has a function of collecting original data. The raw data may be a video stream, i.e. a video file recorded by the camera device, or may be image data in the first format. The first format may be jpeg, raw, and the like, and the first format is not particularly limited in the embodiment of the present invention.

After the image pickup apparatus has acquired the raw data, the raw data may be transmitted to a decoding apparatus so that the decoding apparatus decodes the raw data into single-frame image data. It is understood that the image pickup apparatus is communicatively connected to the decoding apparatus. The decoding device and the camera device can be in communication connection through an Ethernet interface.

And S420, the image pickup device receives current frame image data sent by the decoding device, wherein the current frame image data is single frame image data in a second format obtained after the decoding device decodes the original data.

After the decoding apparatus decodes the original data into single-frame image data, the single-frame image data is transmitted to the image pickup apparatus, so that the image pickup apparatus receives the image data of one frame. It is to be understood that any one of the frame image data received by the image pickup apparatus may be the current frame image data. The format of the current frame image data may be YUV format. The embodiment of the invention does not specifically limit the format of the current frame image data.

And S430, preprocessing the current frame image data by the camera.

After receiving the current frame image data, the image pickup device may perform preprocessing on the current frame image data. For example, after receiving the current frame image data, the image capturing apparatus may perform preprocessing operations such as image scaling, format conversion, and cropping out a region of interest on the current frame image data. As can be seen, the preprocessing may be image scaling, format conversion, clipping, and the like, and the preprocessing is not specifically limited in the embodiment of the present invention.

And S440, the camera device takes the preprocessed data as the test data of the target algorithm, and executes the test flow of the target algorithm to obtain the test result corresponding to the current frame image data.

After the image pickup device preprocesses the current frame image data, preprocessed data can be obtained. At this time, the image pickup apparatus can execute a test flow of the target algorithm. Specifically, the camera device calls a target algorithm to process the preprocessed data, so as to obtain a processing result corresponding to the preprocessed data.

The target algorithm may be: image processing algorithms, image classification algorithms, image recognition algorithms, and the like. And different target algorithms are used for outputting different processing results after the preprocessed data are processed. For example, when the target algorithm is an image recognition algorithm, the output processing result is an image recognition result; and when the target algorithm is an image compression algorithm, the output processing result is an image compression result.

Of course, the processing result corresponding to the preprocessed data may be not only: after the image processing is carried out on the preprocessed data, an image processing result is obtained; the processing result corresponding to the preprocessed data may also be: in the process of processing the preprocessed data, the performance parameters of the target algorithm, for example, the processing result corresponding to the preprocessed data, may be: in the process of processing the preprocessed data, consumed time, occupied memory and the like are used for evaluating parameters of the performance of the target algorithm.

And after the processing result corresponding to the preprocessed data is obtained, the processing result corresponding to the preprocessed data can be output in a file form, and the processing result representing the operation effect of the target algorithm can be obtained through the file, so that the test result corresponding to the current frame image data is obtained.

For example, when the target algorithm is a face recognition algorithm, the processing result corresponding to the preprocessed data may include feature information of the face recognized in the image, such as age, gender, and the like, and the accuracy of the target algorithm may be determined by comparing the obtained feature information with actual feature information, thereby achieving the operation effect of testing the target algorithm.

For another example, when the target algorithm is a face recognition algorithm, the processing result corresponding to the preprocessed data may further include time taken to recognize feature information of a face in the image, and the time may be used to evaluate an operation rate of the target algorithm, so that an operation effect of testing the target algorithm is achieved.

It can be understood that the target algorithm may be pre-packaged in a dynamic algorithm library, and the dynamic algorithm library may be stored in the image capturing apparatus, or may be stored in other apparatuses, as long as the image capturing apparatus can call the dynamic algorithm library, which is not limited in the embodiment of the present invention.

For the image pickup apparatus having a dual processor architecture of a main processor and a coprocessor, in order to reduce the amount of computation when the main processor of the image pickup apparatus performs image processing, in one implementation, the image pickup apparatus may include the main processor and at least one coprocessor, and the target algorithm includes at least a first sub-algorithm and a second sub-algorithm;

the main processor is used for: receiving current frame image data sent by decoding equipment, preprocessing the current frame image data and sending the preprocessed current frame image data to a coprocessor;

the coprocessor is used for: taking the preprocessed current frame image data as test data of a first sub-algorithm, executing a test flow of the first sub-algorithm to obtain a test result corresponding to the current frame image data, and sending the test result to a main processor;

the main processor is further configured to: outputting the test result from the coprocessor; or taking the test result from the coprocessor as test data of the second sub-algorithm, and executing the test flow of the second sub-algorithm to obtain the test result of the second sub-algorithm.

It will be appreciated that a coprocessor is a processor that is developed and deployed to assist a host processor in performing processing tasks that it cannot perform or that perform efficiently and effectively.

In this embodiment, the target algorithm may include one or more algorithms, and the target algorithm may be split into two types of algorithms, respectively: a first sub-algorithm and a second sub-algorithm; the first sub-algorithm can be packaged into an algorithm library corresponding to the coprocessor in advance, so that the first sub-algorithm can run on the coprocessor; the second sub-algorithm may be pre-packaged into a corresponding algorithm library of the main processor, such that the second sub-algorithm can run on the main processor.

It should be noted that the coprocessor side usually processes an image with a small size of one frame or an image with a fixed size including a certain region of interest, and therefore preprocessing operations such as image scaling, format conversion, and region of interest matting operation are performed on the host processor side. That is, after receiving the current frame image data sent by the decoding device, the main processor may pre-process the current frame image data and then send the pre-processed current frame image data to the coprocessor.

After receiving the preprocessed current frame image data sent by the main processor, the coprocessor calls a first sub-algorithm, the preprocessed current frame image data is used as test data of the first sub-algorithm, the first sub-algorithm carries out image processing on the preprocessed current frame image data, namely, a test flow of the first sub-algorithm is executed, a test result from the coprocessor is obtained, and the test result is sent to the main processor; after receiving the test result from the coprocessor, the main processor can directly output the test result of the coprocessor; the test results from the co-processor may be further processed.

The specific process of the main processor performing post-processing on the test result from the coprocessor may be: and calling a second sub-algorithm, taking the test result from the coprocessor as the input of the second sub-algorithm, and further processing the test result from the coprocessor by using the second sub-algorithm, namely executing the test flow of the second sub-algorithm, thereby obtaining the test result of the current frame of image data. For example, the test results from the coprocessor are the targets present in the image; after the main processor receives the target sent by the coprocessor, the received target can be tracked.

As can be seen from the above description, an algorithm related to further processing the test result from the coprocessor in the target algorithm may be used as the second sub-algorithm; and taking the algorithm except the second sub-algorithm in the target algorithm as the first algorithm.

According to the scheme provided by the embodiment of the invention, after the camera shooting equipment collects the original data, the original data is sent to the decoding equipment; after receiving the original data, the decoding equipment decodes the original data to obtain single-frame image data and sends the single-frame image data to the camera equipment; after receiving the current frame image data sent by the decoding equipment, the camera equipment preprocesses the current frame image data, uses the preprocessed data as the test data of the target algorithm, executes the test flow of the target algorithm, and obtains the test result corresponding to the current frame image data. Therefore, the aim of testing the running effect of the target algorithm in actual application can be achieved through the scheme. In addition, the scheme provided by the embodiment of the invention uses the camera equipment as hardware for testing, does not need to design and set up a hardware environment independently, belongs to a testing means on a software level, and is convenient for realizing the testing of a target algorithm.

On the basis of the embodiment shown in fig. 4, in an implementation, before the step of receiving, by the image capturing apparatus, the current frame image data transmitted by the decoding apparatus, the method for testing the target algorithm may further include:

the camera equipment and the decoding equipment establish Socket programming interface communication connection;

the step of the image pickup apparatus receiving the current frame image data transmitted by the decoding apparatus may include:

the method comprises the steps that the camera equipment receives Socket indication information sent by the decoding equipment, wherein the Socket indication information is used for indicating the camera equipment to read current frame image data according to a mounted shared directory; the Socket indication information is sent after the decoding device obtains the current frame image data and writes the current frame image data into the storage area corresponding to the shared directory.

It is to be understood that, when the decoding apparatus and the image capturing apparatus are in the same operating environment, the directory on the decoding apparatus may be previously mounted on the image capturing apparatus, and thus, the directory previously mounted on the image capturing apparatus becomes a shared directory, so that the decoding apparatus and the image capturing apparatus can share image data. Specifically, after the shared directory is determined, the decoding apparatus may write each frame of image data obtained by decoding into the shared directory, and the imaging apparatus may read each frame of image data from the shared directory, thereby implementing sharing of the image data.

In this embodiment, the image pickup apparatus and the decoding apparatus may establish a Socket programming interface communication connection before the decoding apparatus transmits the current frame image data to the image pickup apparatus. Therefore, the decoding equipment can write the current frame image data into the storage area corresponding to the shared directory after decoding the original data to obtain the current frame video data; sending Socket indication information to the camera equipment; after receiving the Socket instruction information, the image capturing device analyzes the Socket instruction information to obtain an analysis result, and it can be understood that the analysis result may be: the current frame image data is read from the mounted shared directory, so that the image pickup apparatus can execute the step of reading the current frame image data from the mounted shared directory.

The above process will be described in detail with reference to the schematic drawings. As shown in fig. 5, the decoding apparatus and the image pickup apparatus are communicatively connected to a Socket programming interface 2 through a Socket programming interface 1. The above process can be mainly divided into 5 steps of S1-S5, specifically:

s1, after the decoding equipment obtains the current frame image data, writing the current frame image data into a shared directory;

s2, the decoding equipment sends Socket indication information to the camera equipment;

s3, the camera equipment analyzes the Socket indication information;

s4, the image pickup apparatus reads the current frame image data from the shared directory;

s5, the image pickup apparatus sends a response message to the decoding apparatus, and after the decoding apparatus receives the response message, it can know that the image pickup apparatus has successfully received the response message, and read the current frame image data from the shared file.

Therefore, the aim of testing the running effect of the target algorithm in actual application can be achieved through the scheme. In addition, the scheme provided by the embodiment of the invention uses the camera equipment as hardware for testing, does not need to design and set up a hardware environment independently, and belongs to a testing means on a software level; moreover, the camera device and the decoding device establish Socket programming interface communication connection, so that the camera device and the decoding device can realize image data sharing, and the target algorithm test is convenient to realize.

It can be understood that the image capturing apparatus may obtain multiple frames of image data from the decoding apparatus, and when the image capturing apparatus performs image processing on the obtained multiple frames of image data, the image capturing apparatus generally needs to perform processing according to the order of obtaining the frames of image data.

In order to ensure that the image capturing apparatus can process each frame of image data according to the sequence of acquiring each frame of image data, in an embodiment, before the step of preprocessing the current frame of image data by the image capturing apparatus, the method for testing the target algorithm may further include:

the image pickup equipment reads image data from the current frame image data and buffers the image data in a first annular buffer; or reading image data and image parameters from the current frame image data, caching the image data in a first ring buffer, and caching the image parameters in a second ring buffer; and

In this embodiment, the image pickup apparatus may read image data from the current frame image data and buffer the read image data in the ring buffer after receiving the current frame image data. It can be understood that the ring buffer is a first-in first-out circular buffer, which can ensure that the image pickup apparatus reads image data from the ring buffer according to the sequence of buffering.

In addition, in some application scenarios, while the decoding apparatus transmits the image data of the current frame image data to the decoding apparatus, the decoding apparatus may also transmit image parameters related to the image data of the current frame image data to the decoding apparatus, so that the image pickup apparatus performs image processing according to the image data and the image parameters of the current frame image data. For example, in an application scenario of detecting a license plate, image data of current frame image data may be the license plate, and image parameters of the current frame image data may be information of an area where the license plate is located. In these application scenarios, the image data and the image parameters may be respectively buffered by using different ring buffers, that is, the image data may be buffered in the first ring buffer, the image parameters may be buffered in the second ring buffer, and in order to ensure that the image data of the current frame image data is buffered in the first ring buffer, the image parameters of the current frame image data may be buffered in the second ring buffer; and reading the image data of the current frame image data from the first ring buffer, and simultaneously reading the image parameters of the current frame image data from the second ring buffer, and managing the first ring buffer and the second ring buffer by the same group of read-write indexes. Therefore, the image data and the image parameters are cached by using the independent annular buffers respectively, so that the data reading speed can be improved, and the expansibility is enhanced.

The image pickup device reads image data from the current frame image data and buffers the image data in a first annular buffer; alternatively, the image data and the image parameters are read from the current frame image data, the image data is buffered in the first ring buffer, and the image parameters are buffered in the second ring buffer, and then response information of Socket indication information may be sent to the decoding device, where the response information is used to instruct the decoding device to send the next frame image data to the image pickup device, that is, after receiving the response information, the decoding device may send the next frame image data to the decoding device.

Accordingly, the step of the image capturing apparatus performing the pre-processing on the current frame image data may include:

acquiring image data of current frame image data from the first annular buffer, and performing at least one of compression, cutting and format conversion on the image data to obtain image data of an interested area;

acquiring image parameters of the current frame image data from the second ring buffer;

acquiring algorithm parameters of a target algorithm;

and generating preprocessed data according to the algorithm parameters, the image parameters of the current frame image data and the image data of the interested area.

Specifically, since the image data of the current frame image data is buffered in the first ring buffer, the image parameters of the current frame image data are buffered in the second ring buffer. Accordingly, the image capturing apparatus can acquire image data of the current image data from the first ring buffer, and can acquire image parameters of the current image data from the second ring buffer. Also, after the image capturing apparatus acquires the image data of the current frame image data from the first ring buffer, the acquired image data may be subjected to preprocessing, such as compression, cropping, format conversion, or the like, to obtain image data of interest.

Fig. 6 is a schematic diagram of a circular buffer. As can be seen from the figure, the ring buffer has 8 buffer areas, which are numbered as: 1. 2, 3, 4, 5, 6, 7 and 8. Each buffer area may store image data or image parameters.

Moreover, different current frame image data may have different corresponding target algorithms and different corresponding algorithm parameters of the target algorithms. Therefore, when the camera device generates the preprocessed data, the algorithm parameters of the target algorithm can be obtained, and the algorithm parameters of the target algorithm, the image parameters of the current frame image data and the image data of the region of interest can be packaged to obtain the preprocessed data, so that the test of the target algorithm corresponding to the preprocessed data in the subsequent steps is facilitated.

Therefore, in the embodiment, the image data is cached in the annular buffer, so that the image data can be ensured to be read from the annular buffer by the camera equipment according to the caching sequence; in addition, the image data and the image parameters are respectively cached by using independent annular buffers, so that the data reading speed can be improved, and the expansibility is enhanced.

In addition, for an image capturing apparatus having a dual processor architecture of a main processor and a coprocessor, the function of the main processor 710 of the image capturing apparatus may be implemented by dividing into one main thread and four sub-threads, as shown in fig. 7, the four sub-threads are respectively: an image data management thread 711, an image data transmission thread 712, a coprocessor processing result receiving thread 713, and a coprocessor processing result post-processing thread 714. The image data management thread and the other three sub-threads can be in a parallel relation, and the other three sub-threads are in a serial relation. The functionality of coprocessor 720 may be divided into a main thread and a sub-thread, where the main thread may be used to: initializing a bottom layer of the coprocessor and starting the sub-thread, wherein the sub-thread comprises the following steps: the coprocessor algorithm uses threads 721, it should be noted that in order to better illustrate the connection between the four sub-threads of the main processor and one sub-thread of the coprocessor, the main thread of the main processor and the main thread of the coprocessor are not shown in fig. 7.

Wherein, the main thread of the main processor may be configured to: and finishing the initialization of the main processor, starting the coprocessor and assisting the coprocessor to finish the initialization.

The image data management thread may be used to perform the following steps, as shown in FIG. 8.

S810, judging whether a free image data buffer area exists in the first annular buffer; if the first circular buffer has a free image data buffer area, executing S820;

s820, buffering the image data of the current frame image data in a first annular buffer;

s830, judging whether image parameters of the current frame image data need to be read; if the image parameter of the current frame image data needs to be read, executing S840; otherwise, executing direct execution S850;

s840, buffering the image parameters of the current frame image data in a second ring buffer;

and S850, sending response information of the Socket indication information to the decoding device.

Wherein the write pointers of the first and second ring buffers are maintained by the image data management thread itself; while the read pointers for the first ring buffer and the second ring buffer are maintained by the image data send thread.

The image data sending thread may be used to perform the following steps, as shown in fig. 9.

S910, reading algorithm parameters of a target algorithm from the configuration file;

s920, judging whether the first annular buffer stores the image data of the current frame image data; whether the second ring buffer stores the image parameters of the current frame image data or not; if the determination result in S920 is yes, go to S930;

s930, reading the image data of the current frame image data from the first ring buffer, and reading the image parameters related to the image data of the current frame image data from the second ring buffer;

s940, image zooming, format conversion, clipping and the like are carried out on the image data read from the first annular buffer to obtain interested image data;

s950, packaging the algorithm parameters of the target algorithm, the image parameters of the current frame image data and the image data of the region of interest to obtain preprocessed data;

s960, sending the obtained preprocessed data to a coprocessor algorithm application thread.

The coprocessor algorithm application thread may be used to perform the following steps, as shown in FIG. 10.

S1010, receiving the preprocessed data sent by the image data sending thread;

s1020, the preprocessed data are used as test data of a first sub-algorithm, the first sub-algorithm carries out image processing on the preprocessed data, and a coprocessor processing result is obtained

And S1030, packaging the processing result of the coprocessor and sending the result to a coprocessor processing result receiving thread.

The coprocessor processing result receiving thread may be used to perform the following steps, as shown in FIG. 11.

S1110, receiving a coprocessor processing result sent by a coprocessor algorithm application thread;

s1120, analyzing the processing result of the coprocessor to obtain an analysis result;

s1130, judging whether the analysis structure is correct; if yes, go to step S1140;

and S1140, sending the coprocessor processing result to a coprocessor post-processing thread.

The coprocessor processes the results and the post-processing thread may be used to perform the following steps, as shown in FIG. 12.

S1210, receiving a coprocessor processing result sent by a coprocessor processing result receiving thread;

s1220, judging whether to output the processing result of the coprocessor; if so, go to S1230; if not, executing S1240;

s1230, outputting the processing result of the coprocessor;

and S1240, taking the processing result of the coprocessor as the test data of the second sub-algorithm, and carrying out post-processing on the processing result of the coprocessor by the second sub-algorithm to obtain the test result corresponding to the image data of the current frame.

In addition, in an implementation manner, the image data sending thread and the coprocessor processing result post-processing thread may be blocked threads, that is, after the coprocessor processing result post-processing thread completes one frame of image data, the image data sending thread sends the next frame of image data to the coprocessor algorithm application thread. The purpose of blocking is to enable the image data processed by the main processor to correspond one-to-one to the results of the coprocessor processing. And after the coprocessor processes the result and the processing thread processes the current frame of image data, the main processor returns to the image data sending thread to execute the subsequent operations of counting time consumption, outputting the result in a file form and the like.

Therefore, the aim of testing the running effect of the target algorithm in actual application can be achieved through the scheme. In addition, the scheme provided by the embodiment of the invention uses the camera equipment as hardware for testing, does not need to design and set up a hardware environment independently, belongs to a testing means on a software level, and is convenient for realizing the testing of a target algorithm. In addition, the camera equipment has a dual processor architecture of a main processor and a coprocessor, so that the calculation amount of the main processor is reduced, and the test efficiency of a target algorithm is improved.

In a third aspect, an embodiment of the present invention provides a method for testing a target algorithm applied to a decoding device.

The target algorithm testing method provided by the embodiment of the invention can be applied to decoding equipment. In order to test the operation effect of the target algorithm, in the embodiment of the present invention, the decoding apparatus may be communicatively connected to the image capturing apparatus, for example, the decoding apparatus and the image capturing apparatus may be connected through an ethernet port by hardware, but is not limited thereto. The decoding device in the embodiment of the invention can take the image data collected by the camera device as input, decode the image data into single frame image data, send each frame image data obtained by decoding to the camera device for processing, preprocess each frame image data after the camera device receives each frame image data to obtain preprocessed data, take the preprocessed data as the test data of the target algorithm, and execute the test flow of the target algorithm, thereby realizing the test of the target algorithm.

Moreover, after the decoding apparatus decodes the image data acquired by the imaging apparatus, the format of the obtained single frame image data is generally the YUV format.

The following describes a method for testing a target algorithm applied to a decoding device according to an embodiment of the present invention in detail.

As shown in fig. 13, a method for testing a target algorithm applied to a decoding device according to an embodiment of the present invention includes the following steps:

s1310, the decoding equipment receives original data collected by the camera equipment, wherein the original data comprises video streams recorded by the camera equipment or image data in a first format collected by the camera equipment; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface.

The image pickup apparatus has a function of collecting original data. Specifically, an image sensor of the image pickup apparatus generates image data; the ISP module carries out image signal processing on the image data, and original data can be obtained.

Also, the raw data may be a video stream, i.e., a video file recorded by the image pickup apparatus, or may be image data in the first format. The first format may be jpeg, raw, and the like, and the first format is not particularly limited in the embodiment of the present invention.

After the camera device collects the original data, the original data can be sent to the decoding device, and the decoding device can receive the original data collected by the camera device. It is understood that the image pickup apparatus is communicatively connected to the decoding apparatus. The decoding device and the camera device can be in communication connection through an Ethernet interface.

S1320, the decoding device decodes the original data to obtain the current frame image data, where the current frame image data is a single frame image data in the second format.

After receiving the original data, the decoding equipment decodes the original data to obtain single-frame image data; it is to be understood that any one frame image data may be the current frame image data. The format of the current frame image data may be YUV format. The embodiment of the invention does not specifically limit the format of the current frame image data.

S1330, the decoding apparatus transmits the current frame image data to the image capturing apparatus.

After the decoding apparatus obtains the single-frame image data, the single-frame image data may be transmitted to the image pickup apparatus, so that the image pickup apparatus receives the image data of one frame. Any one of the frames of image data received by the image pickup apparatus may be the current frame of image data. After the image pickup device receives the current frame image data, the method for testing the target algorithm in the first aspect can be executed.

On the basis of the embodiment shown in fig. 13, in an implementation, before the step of sending the current frame image data to the image capturing apparatus by the decoding apparatus, the method for testing the target algorithm may further include:

the decoding equipment and the camera equipment establish Socket programming interface communication connection;

the step of the decoding apparatus transmitting the current frame image data to the image pickup apparatus includes:

after decoding to obtain current frame image data, the decoding equipment writes the current frame image data into a storage area corresponding to the shared directory;

and sending Socket indication information to the camera equipment, wherein the Socket indication information is used for indicating the camera equipment to read the current frame image data according to the mounted shared directory.

In one embodiment, after the step of the decoding apparatus transmitting the current frame image data to the image pickup apparatus, the method of testing the target algorithm may further include:

the decoding equipment receives response information of Socket indication information sent by the camera equipment, wherein the response information is sent after the camera equipment reads image data from current frame image data and buffers the image data in a first annular buffer; or the response information is sent by the camera equipment after reading the image data and the image parameters from the current frame image data, buffering the image data in the first ring buffer and buffering the image parameters in the second ring buffer;

the decoding apparatus transmits the next frame of image data to the image pickup apparatus based on the response information.

In order to ensure that the camera equipment can process each frame of image data according to the sequence of acquiring each frame of image data, the camera equipment reads the image data from the current frame of image data and buffers the image data in a first annular buffer; or reading image data and image parameters from the current frame image data, caching the image data in a first ring buffer, and caching the image parameters in a second ring buffer; the image pickup apparatus may read image data from the current frame image data and buffer the read image data in the ring buffer after receiving the current frame image data. It can be understood that the ring buffer is a first-in first-out circular buffer, which can ensure that the image pickup apparatus reads image data from the ring buffer according to the sequence of buffering.

In order to describe the workflow of the decoding apparatus more clearly, the workflow of the decoding apparatus is explained in detail below. As shown in fig. 14.

S1410, the decoding device and the camera device establish Socket programming interface communication connection;

s1420, the decoding equipment receives original data collected by the camera equipment and sent by the camera equipment;

s1430, the decoding device determines whether the received original data is a directory containing video files or image files, if so, the step S1440 is executed; otherwise, directly executing step S1450;

s1440, traversing each video file or image file in the directory;

s1450, decoding the video file or the image file to obtain single-frame image data of one frame;

s1460, writing the current frame image data into a shared directory, and sending Socket indication information to the camera equipment, wherein the Socket indication information is used for indicating the camera equipment to read single frame image data from the shared directory;

and S1470, judging whether response information of the Socket indication information is received or not by the decoding equipment, if so, returning to execute S1460, and if not, continuously judging whether the response information of the Socket indication information is received or not.

In a fourth aspect, an embodiment of the present invention provides a device for testing a target algorithm, where the device is located in an image capturing apparatus, and as shown in fig. 15, the device includes:

a raw data sending module 1510, configured to send, to a decoding device, raw data collected by the image capturing device, where the raw data includes a video stream recorded by the image capturing device or image data in a first format collected by the image capturing device; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface;

an image data receiving module 1520, configured to receive current frame image data sent by the decoding apparatus, where the current frame image data is single frame image data in a second format obtained after the decoding apparatus decodes the original data;

an image data processing module 1530, configured to pre-process the current frame image data by the image capturing apparatus;

the algorithm testing module 1540 is configured to use the preprocessed data as testing data of a target algorithm, and execute a testing process of the target algorithm to obtain a testing result corresponding to the current frame image data.

Optionally, the apparatus may further include:

the first communication establishing module is used for establishing Socket programming interface communication connection with the decoding equipment before the image data receiving module of the camera equipment receives the current frame image data sent by the decoding equipment;

the image data receiving module is specifically configured to:

receiving Socket indication information sent by the decoding equipment, wherein the Socket indication information is used for indicating the camera equipment to read the current frame image data according to the mounted shared directory; and the Socket indication information is sent by the decoding equipment after the current frame image data is obtained and written into the storage area corresponding to the shared directory.

Optionally, the image capturing apparatus includes a main processor and at least one coprocessor, and the target algorithm at least includes a first sub-algorithm and a second sub-algorithm;

Optionally, the apparatus may further include:

an image data reading module, configured to read image data from the current frame image data before the image data processing module performs preprocessing on the current frame image data, and cache the image data in a first circular buffer; or reading image data and image parameters from the current frame image data, caching the image data in the first ring buffer, and caching the image parameters in the second ring buffer; and

and the response information sending module is used for sending response information of the Socket indication information to the decoding equipment, and the response information is used for indicating the decoding equipment to send the next frame of image data to the camera equipment.

Optionally, the image data processing module is specifically configured to:

acquiring image parameters of the current frame image data from the second ring buffer;

acquiring algorithm parameters of the target algorithm;

and generating preprocessed data according to the algorithm parameters, the image parameters of the current frame image data and the image data of the region of interest.

Optionally, the target algorithm is at least one of an image processing algorithm, an image classification algorithm and an image recognition algorithm, and the target algorithm includes one or more sub-algorithms.

In a fifth aspect, an embodiment of the present invention provides a device for testing a target algorithm, where the device is located in a decoding apparatus, as shown in fig. 16, and the device includes:

a raw data receiving module 1610, configured to receive raw data collected by a camera apparatus, where the raw data includes a video stream recorded by the camera apparatus or image data in a first format collected by the camera apparatus; the decoding equipment and the camera equipment are in communication connection through an Ethernet interface;

an original data decoding module 1620, configured to decode the original data to obtain current frame image data, where the current frame image data is single frame image data in a second format;

an image data sending module 1630, configured to send the current frame image data to the image capturing apparatus.

Optionally, the apparatus may further include:

the second communication establishing module is used for establishing Socket programming interface communication connection with the camera shooting equipment before the image data sending module sends the current frame image data to the camera shooting equipment;

the image data sending module is specifically configured to:

after the original data decoding module decodes the current frame image data, writing the current frame image data into a storage area corresponding to a shared directory; and are

Optionally, the apparatus further comprises:

the response information receiving module is used for receiving response information of Socket indication information sent by the camera equipment after the image data sending module sends the current frame of image data to the camera equipment, wherein the response information is sent after the camera equipment reads the image data from the current frame of image data and buffers the image data in a first annular buffer; or, the response information is sent after the image pickup apparatus reads image data and image parameters from the current frame image data, buffers the image data in the first ring buffer, and buffers the image parameters in the second ring buffer;

and sending the next frame of image data to the camera equipment according to the response information.

In a sixth aspect, an embodiment of the present invention further provides an image pickup apparatus, as shown in fig. 17, including a processor 1701, a communication interface 1702, a memory 1703, and a communication bus 1704, wherein the processor 1701, the communication interface 1702, and the memory 1703 communicate with each other via the communication bus 1704,

a memory 1703 for storing a computer program;

the processor 1701 is configured to implement the test method of the target algorithm described in the first aspect when executing the program stored in the memory 1703.

The communication bus mentioned in the above-mentioned image pickup apparatus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the above-described image pickup apparatus and other apparatuses.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

In a seventh aspect, an embodiment of the present invention further provides a decoding apparatus, as shown in fig. 18, including a processor 1801, a communication interface 1802, a memory 1803, and a communication bus 1804, where the processor 1801, the communication interface 1802, and the memory 1803 complete communication with each other through the communication bus 1804,

a memory 1803 for storing a computer program;

the processor 1801 is configured to implement the method for testing the target algorithm according to the second aspect when executing the program stored in the memory 1803.

The communication bus mentioned in the above decoding apparatus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the decoding device and other devices.

In an eighth aspect, in a further embodiment provided by the present invention, there is further provided a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to execute the method for testing the target algorithm described in the first aspect above.

In a ninth aspect, in a further embodiment provided by the present invention, there is also provided a computer-readable storage medium having stored therein instructions which, when run on a computer, cause the computer to execute the method for testing the target algorithm described in the second aspect above.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus, the system, the image pickup apparatus, the decoding apparatus, and the computer-readable storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

33页详细技术资料下载

上一篇：一种医用注射器针头装配设备

下一篇：外部触发的线性相机检测系统及其影像均匀度处理方法

Target algorithm testing method, device and system

相关技术

网友询问留言