Soft-hard combined and high-efficiency transmission video decoding method
阅读说明：本技术 一种软硬结合且高效传输的视频解码方法 (Soft-hard combined and high-efficiency transmission video decoding method ) 是由 高娟 于 2021-06-23 设计创作，主要内容包括：本发明属于linux系统下视频解码技术,涉及一种在linux系统下使用软硬结合的高效传输的视频解码设计方法。其特点在于：首先,移植ffmpeg到海思平台,对ffmpeg进行源码修改使其适配海思芯片。其次,启动PCIE通道与主机进行数据传输交互；然后,启动ffmpeg的动态库对网络传输的数据进行动态过滤,去除错误帧,同时获取图像与参数信息结合的数据包；最后,将完整数据包再传送给海思芯片的硬解码模块,从而完成解码任务。本发明使用芯片的硬解码模块完成解码过程,有效减少解码时间,采用PCIE通道传输解码前后的图像,极大地减少了数据传输时延。(The invention belongs to a video decoding technology under a linux system, and relates to a video decoding design method for efficient transmission under the linux system by using soft and hard combination. It is characterized in that: first, ffmpeg is transplanted to the Haisi platform, and source code modification is performed on ffmpeg to make it fit to the Haisi chip. Secondly, starting a PCIE channel to perform data transmission interaction with the host; then, starting a dynamic library of the ffmpeg to dynamically filter the data transmitted by the network, removing error frames, and acquiring a data packet combining the image and the parameter information; and finally, transmitting the complete data packet to a hard decoding module of the Haisi chip so as to complete a decoding task. The invention uses the hard decoding module of the chip to complete the decoding process, effectively reduces the decoding time, and adopts the PCIE channel to transmit the images before and after decoding, thereby greatly reducing the data transmission time delay.)
1. A method for decoding video with soft and hard combining and high transmission efficiency, comprising the steps of:
1) the compiling attribute and parameter of the ffmpeg are configured, and the ffmpeg dynamic library is transplanted to the Haisi platform;
2) configuring PCIE equipment;
the PCIE of the Haesi platform is configured into slave equipment, and the PCIE equipment of the host is configured into master equipment; meanwhile, making transmission rules of the master equipment and the slave equipment;
3) creating a PCIE data reading thread, and starting a PCIE reading task to acquire original image data transmitted by a host;
4) creating an ffmpeg acquisition data packet thread, and acquiring a complete data packet of a compressed image by using an ffmpeg dynamic library
5) Decoding the data packet of the compressed image by using a hard decoding module of the Haisi platform to obtain decoded image data;
6) and transmitting the decoded image data back to the host by using the PCIE channel.
2. The method of claim 1, wherein the master device and the slave device in step 2) have transmission rules as follows:
writing specified contents of handshake after the main equipment is electrified, and detecting corresponding contents and successfully handshaking after the slave equipment is electrified;
the method comprises the steps that a main device and a slave device distribute read-write PCIE addresses for each channel, and different read-write addresses are not conflicted with each other;
after the handshake of the main equipment is successful, the address content of the received data of each channel is written in a specified PCIE address field;
the main device judges whether the data exists or not by inquiring whether the content length of the fixed address field is greater than 0 or not, and clears the content length after the data reading is finished;
before data transmission, the slave device firstly judges whether the master device has read the last data through the length of the fixed address segment, if so, the slave device writes the decoded image data and writes the length, and if not, the slave device abandons the data writing task and waits for the next decoded image data to be transmitted.
3. The method according to claim 1, wherein the creating of the PCIE read data thread in step 3) is as follows:
31) applying for a read-write exclusive lock of the PCIE channels, protecting the read-write operation of each PCIE channel, and not allowing a plurality of PCIE channels to perform the read-write operation at the same time;
32) checking whether the main device has new image data available for reading, namely whether the read data length in the fixed address is not zero or not, and simultaneously, the read address of the main device is not zero;
33) reading data in a specified read address field according to a protocol rule, and specifically operating: setting a destination address of a PCIE reading task as a PCIE reading address of the slave equipment; setting a source address as a PCIE writing address of the master device;
34) sending a data reading task command to the slave equipment through an ioctl function; judging whether the return value of the data reading task is successful, and if the data reading task is successful, resetting the reading length to be zero so that the main equipment can transmit data next time;
35) and unlocking the read-write mutual exclusion lock of the PCIE channel, and releasing the resources.
4. The method according to claim 1 or 3, wherein the step 6) of returning the decoded image data to the host using the PCIE channel includes:
61) acquiring a read-write mutual exclusion lock of a PCIE channel;
62) copying the decoded image parameter data to a DMA virtual address space;
63) judging whether the write length in the Haisi platform chip is cleared by the host, if not, reporting an error and exiting, terminating the image sending task, and returning to the step 61 after receiving new decoded image parameter data); otherwise, go to step 64);
64) acquiring a target physical address of data sent to the main equipment, if the target physical address is zero, reporting an error and exiting, terminating an image sending task, and returning to the step 61 after receiving new decoded image parameter data); otherwise, go to step 65); the PCIE of Haisi is configured as a slave device, and the PCIE device of a host is a master device;
65) acquiring a physical initial address and an offset address specified by a protocol of DMA transmission, taking the sum of the physical initial address and the offset address as a source address of a DMA transmission task, and taking a physical address vxworks _ phy _ addr of a host PCIE as a destination address of the task;
66) setting the transmission length of the task as the length sei _ len of parameter frame data;
67) judging whether the operation handle of the current PCIE is larger than zero, if so, using a system function ioctl to issue a PCIE writing task and entering the next step, if not, reporting an error and exiting the process, terminating the image sending task, and returning to the step 61 after receiving new decoded image parameter data);
68) judging whether the task state of the current PCIE writing task is finished, if so, entering the next step, and if not, reporting an error and exiting the process; returning to step 61) after receiving new decoded image parameter data;
69) setting a source address of a second DMA transmission task as image frame group data, taking an address of a host physical address vxworks _ phy _ addr offset parameter frame byte number sei _ len as a target address of the task, and setting an image frame length frame _ len as the transmission length of the second task;
610) repeat 67) and 68), proceed to the next step;
611) writing the sent image frame length frame _ len into a memory address specified by a protocol to prompt a host computer that data can be read;
612) reading image frame data and parameter frame data by using a host to finish the return work;
613) and unlocking the mutual exclusion lock of the PCIE channel, releasing the PCIE channel resource, and returning to the step 61) after receiving the new decoded image parameter data.
5. The method according to claim 4, wherein the step 1) of migrating the ffmpeg dynamic library to the Haisi platform comprises:
11) firstly, configuring ffmpeg compiling attribute, and configuring parameters according to a platform type, a cpu type, a codec attribute, a format conversion attribute and a cross compiling attribute;
12) modifying acodec.h files, increasing parameter frame length SEI _ len and array SEI _ BUF, and determining parameter frame SIZE SEI _ BUF _ SIZE according to application layer requirements;
13) adding a parameter frame acquiring function in a decode _ nal _ sei _ prefix function in a hevc _ sei.c file;
14) executing a configure command, and generating decoding libraries libavcodec, libavformat, libavutil and libswscale under the subfolder lib of the configuration folder;
15) and finally, copying the dynamic library to a/usr/lib path of a decoding board card.
6. The method according to claim 5, wherein the step 4) creates ffmpeg acquisition packet threads, specifically:
421) acquiring an image data array first address pointer bufPtr and a length bufLen read from a PCIE channel;
422) judging whether the current data length bufLen is larger than 0, if so, continuing the next step, and if not, exiting the decoding process of the data, and waiting for image data transmitted by the PCIE channel next time;
423) transmitting a data array head address pointer bufPtr and a length bufLen to a soft decoding module, segmenting image data by frames by using a library function av _ parser _ parse2, if a complete image frame data packet can be successfully obtained from an array, recording and continuing the next step, and if not, exiting the decoding process; waiting for the image data array transmitted by the PCIE channel;
424) storing the length ret of the image frame divided in the data array at the time, removing the length ret of the image frame data packet divided at the time from the total length bufLen of the data array, and moving a first address pointer bufPtr forwards for ret;
425) and putting the image complete data packet segmented this time into a queue to be decoded.
7. The method according to claim 6, wherein the step 5) of decoding the compressed image data packets comprises:
51) initializing a hard decoding module according to the image parameters and the decoding type, configuring the size of a video data buffer area in the hard decoding module, and starting the decoding module;
52) dynamically applying for the size buf of a buffer area of a video image frame data packet;
53) circularly judging whether the decoding is finished or not, if so, exiting the processing flow and carrying out the step 57), and if not, carrying out the next step;
54) setting parameters of a current frame to be decoded: stream end identifier, frame head identifier, frame tail identifier;
55) putting the image frame data into buf;
56) calling a dynamic library function to send buf data to a hard decoding module;
57) monitoring the decoding state of the hard decoding module in real time, if the decoding is wrong, restarting the hard decoding module in a soft mode and resetting parameters, and if the decoding is normal, calling a library function to obtain a decoded image;
58) and calling a library function to stop sending the video stream to the hard decoding module, closing a decoding channel, unbinding the binding relationship among the modules and clearing resources.
8. A processing apparatus, comprising:
a memory for storing a computer program;
a processor for calling and running the computer program from the memory to perform the method of any of claims 2 to 7.
9. A computer-readable storage medium, having stored thereon a computer program or instructions, which, when executed, implement the method of any one of claims 2 to 7.
10. A computer program product, characterized in that it comprises instructions which, when run on a computer, cause the computer to carry out the method of any one of claims 2 to 7.
Technical Field
The invention belongs to a video decoding technology under a linux system, and particularly relates to a video decoding method with soft and hard combination and efficient transmission.
Background
Hi3559AV100 is a professional 8K Ultra HD Mobile Camera SOC, provides digital video recording of 8K30/4K120 broadcast-level image quality, supports multi-channel Sensor input, supports H.265 coding output or video-level RAW data output, integrates high-performance ISP processing, and simultaneously adopts an advanced low-power-consumption process and a low-power-consumption architecture design, and provides excellent image processing capability for users.
Hi3559AV100 supports the leading multi-channel 4K Sensor input in the industry, multi-channel ISP image processing, the high dynamic range technical standard of HDR10 and multi-channel panoramic hardware splicing. In support of 8K30/4K120 video recording, Hi3559AV100 provides hardened 6-Dof digital anti-shake, reducing reliance on mechanical holders.
PCI-express (peripheral component interconnect express) is a high-speed serial computer expansion bus standard, originally named "3GIO", which was proposed by Intel in 2001, to replace the old PCI, PCI-X and AGP bus standards. PCIe belongs to high-speed serial point-to-point double-channel high-bandwidth transmission, connected devices distribute independent channel bandwidth and do not share bus bandwidth, and the PCIe mainly supports functions of active power management, error reporting, end-to-end reliable transmission, hot plug, quality of service (QOS) and the like. PCIe is renamed to PCI-Express, which is called PCI-e for short, after being authenticated and issued by PCI-SIG (PCI Special interest group). Its main advantages are high data transmission rate and high development potential.
However, the Hi3559AV100 belongs to the hard decoding category, and when the protocol frame does not completely conform to the decoding protocol or the number of error frames is large, the decoding efficiency is low or decoding is impossible. In addition, the hard decoding module cannot return the decoded image frame with the parameter frame. Meanwhile, under the condition of large image data, the network transmission delay is large, and the performance requirement cannot be met.
Disclosure of Invention
The technical problem solved by the invention is as follows: the defects of the prior art are overcome, the video decoding method with soft and hard combination and high-efficiency transmission is provided, the decoding process is completed by using the hard decoding module of the chip, the decoding time is effectively reduced, and the data transmission time delay is greatly reduced by adopting a PCIE channel to transmit images before and after decoding.
The technical solution of the invention is as follows:
in a first aspect, a method for decoding a video with soft and hard combination and high efficiency transmission includes the following steps:
1) the compiling attribute and parameter of the ffmpeg are configured, and the ffmpeg dynamic library is transplanted to the Haisi platform;
2) configuring PCIE equipment;
the PCIE of the Haesi platform is configured into slave equipment, and the PCIE equipment of the host is configured into master equipment; meanwhile, making transmission rules of the master equipment and the slave equipment;
3) creating a PCIE data reading thread, and starting a PCIE reading task to acquire original image data transmitted by a host;
4) creating an ffmpeg acquisition data packet thread, and acquiring a complete data packet of a compressed image by using an ffmpeg dynamic library
5) Decoding the data packet of the compressed image by using a hard decoding module of the Haisi platform to obtain decoded image data;
6) and transmitting the decoded image data back to the host by using the PCIE channel.
Optionally, the master device and the slave device in step 2) have the following transmission rules:
writing specified contents of handshake after the main equipment is electrified, and detecting corresponding contents and successfully handshaking after the slave equipment is electrified;
the method comprises the steps that a main device and a slave device distribute read-write PCIE addresses for each channel, and different read-write addresses are not conflicted with each other;
after the handshake of the main equipment is successful, the address content of the received data of each channel is written in a specified PCIE address field;
the main device judges whether the data exists or not by inquiring whether the content length of the fixed address field is greater than 0 or not, and clears the content length after the data reading is finished;
before data transmission, the slave device firstly judges whether the master device has read the last data through the length of the fixed address segment, if so, the slave device writes the decoded image data and writes the length, and if not, the slave device abandons the data writing task and waits for the next decoded image data to be transmitted.
Optionally, the creating a PCIE read data thread in step 3) is specifically as follows:
31) applying for a read-write exclusive lock of the PCIE channels, protecting the read-write operation of each PCIE channel, and not allowing a plurality of PCIE channels to perform the read-write operation at the same time;
32) checking whether the main device has new image data available for reading, namely whether the read data length in the fixed address is not zero or not, and simultaneously, the read address of the main device is not zero;
33) reading data in a specified read address field according to a protocol rule, and specifically operating: setting a destination address of a PCIE reading task as a PCIE reading address of the slave equipment; setting a source address as a PCIE writing address of the master device;
34) sending a data reading task command to the slave equipment through an ioctl function; judging whether the return value of the data reading task is successful, and if the data reading task is successful, resetting the reading length to be zero so that the main equipment can transmit data next time;
35) and unlocking the read-write mutual exclusion lock of the PCIE channel, and releasing the resources.
Optionally, the method for returning the decoded image data to the host using the PCIE channel in step 6) specifically includes:
61) acquiring a read-write mutual exclusion lock of a PCIE channel;
62) copying the decoded image parameter data to a DMA virtual address space;
63) judging whether the write length in the Haisi platform chip is cleared by the host, if not, reporting an error and exiting, terminating the image sending task, and returning to the step 61 after receiving new decoded image parameter data); otherwise, go to step 64);
64) acquiring a target physical address of data sent to the main equipment, if the target physical address is zero, reporting an error and exiting, terminating an image sending task, and returning to the step 61 after receiving new decoded image parameter data); otherwise, go to step 65); the PCIE of Haisi is configured as a slave device, and the PCIE device of a host is a master device;
65) acquiring a physical initial address and an offset address specified by a protocol of DMA transmission, taking the sum of the physical initial address and the offset address as a source address of a DMA transmission task, and taking a physical address vxworks _ phy _ addr of a host PCIE as a destination address of the task;
66) setting the transmission length of the task as the length sei _ len of parameter frame data;
67) judging whether the operation handle of the current PCIE is larger than zero, if so, using a system function ioctl to issue a PCIE writing task and entering the next step, if not, reporting an error and exiting the process, terminating the image sending task, and returning to the step 61 after receiving new decoded image parameter data);
68) judging whether the task state of the current PCIE writing task is finished, if so, entering the next step, and if not, reporting an error and exiting the process; returning to step 61) after receiving new decoded image parameter data;
69) setting a source address of a second DMA transmission task as image frame group data, taking an address of a host physical address vxworks _ phy _ addr offset parameter frame byte number sei _ len as a target address of the task, and setting an image frame length frame _ len as the transmission length of the second task;
610) repeat 67) and 68), proceed to the next step;
611) writing the sent image frame length frame _ len into a memory address specified by a protocol to prompt a host computer that data can be read;
612) reading image frame data and parameter frame data by using a host to finish the return work;
613) and unlocking the mutual exclusion lock of the PCIE channel, releasing the PCIE channel resource, and returning to the step 61) after receiving the new decoded image parameter data.
Optionally, the step 1) of transplanting the ffmpeg dynamic library to the haisi platform specifically comprises:
11) firstly, configuring ffmpeg compiling attribute, and configuring parameters according to a platform type, a cpu type, a codec attribute, a format conversion attribute and a cross compiling attribute;
12) modifying acodec.h files, increasing parameter frame length SEI _ len and array SEI _ BUF, and determining parameter frame SIZE SEI _ BUF _ SIZE according to application layer requirements;
13) adding a parameter frame acquiring function in a decode _ nal _ sei _ prefix function in a hevc _ sei.c file;
14) executing a configure command, and generating decoding libraries libavcodec, libavformat, libavutil and libswscale under the subfolder lib of the configuration folder;
15) and finally, copying the dynamic library to a/usr/lib path of a decoding board card.
Optionally, step 4) creates an ffmpeg acquisition packet thread, specifically:
421) acquiring an image data array first address pointer bufPtr and a length bufLen read from a PCIE channel;
422) judging whether the current data length bufLen is larger than 0, if so, continuing the next step, and if not, exiting the decoding process of the data, and waiting for image data transmitted by the PCIE channel next time;
423) transmitting a data array head address pointer bufPtr and a length bufLen to a soft decoding module, segmenting image data by frames by using a library function av _ parser _ parse2, if a complete image frame data packet can be successfully obtained from an array, recording and continuing the next step, and if not, exiting the decoding process; waiting for the image data array transmitted by the PCIE channel;
424) storing the length ret of the image frame divided in the data array at the time, removing the length ret of the image frame data packet divided at the time from the total length bufLen of the data array, and moving a first address pointer bufPtr forwards for ret;
425) and putting the image complete data packet segmented this time into a queue to be decoded.
Optionally, the method for decoding the data packet of the compressed image in step 5) specifically includes:
51) initializing a hard decoding module according to the image parameters and the decoding type, configuring the size of a video data buffer area in the hard decoding module, and starting the decoding module;
52) dynamically applying for the size buf of a buffer area of a video image frame data packet;
53) circularly judging whether the decoding is finished or not, if so, exiting the processing flow and carrying out the step 57), and if not, carrying out the next step;
54) setting parameters of a current frame to be decoded: stream end identifier, frame head identifier, frame tail identifier;
55) putting the image frame data into buf;
56) calling a dynamic library function to send buf data to a hard decoding module;
57) monitoring the decoding state of the hard decoding module in real time, if the decoding is wrong, restarting the hard decoding module in a soft mode and resetting parameters, and if the decoding is normal, calling a library function to obtain a decoded image;
58) and calling a library function to stop sending the video stream to the hard decoding module, closing a decoding channel, unbinding the binding relationship among the modules and clearing resources.
In a second aspect, a processing apparatus comprises:
a memory for storing a computer program;
a processor for calling and running the computer program from the memory to perform the method of the first aspect.
A computer readable storage medium having stored thereon a computer program or instructions which, when executed, implement the method of the first aspect.
A computer program product comprising instructions for causing a computer to perform the method of the first aspect when the computer program product is run on a computer.
Compared with the prior art, the invention has the advantages that:
the parameter frame information and the data frame information in the original frame are acquired by adopting the ffmpeg decoding library, and a complete data packet is combined to the hard decoding module, so that the decoding time delay is reduced, and the decoding function is completed. Meanwhile, the PCIE channel is adopted to transmit image big data, so that transmission delay is greatly reduced.
The system is in linux, a Haisi Hi3559AV100 chip is used as a hard decoding module, an FFMPEG decoding library is used for acquiring a complete data packet, compressed frame parameter information and image frame information are effectively acquired, decoding time is reduced, a PCIE channel is started to efficiently transmit image data, and data transmission delay is greatly reduced.
Drawings
Fig. 1 is a flow chart of a video decoding scheme for efficient transmission with soft and hard combining according to the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the detailed description.
With reference to fig. 1, in order to solve the problem of video decoding in the linux system, ffmpeg-based soft decoding and haisi chip module hard decoding are combined, and meanwhile, PCIE channels are used to transmit image data. The present invention will be further described below.
PCIE (peripheral Components Interconnect express) is a high-speed serial computer expansion bus standard. PCIE belongs to high-speed serial point-to-point double-channel high-bandwidth transmission, and connected equipment distributes independent channel bandwidth without sharing bus bandwidth. PCIE devices communicate through logical connections called interconnects or links. Links are point-to-point communication channels between two PCI Express ports that allow them to send and receive ordinary PCI requests and interrupts. PCIE uses a shared parallel bus architecture, where the PCI host and all devices share a common set of address, data, and control lines.
1. Transplanting ffmpeg to Haisi platform
11) First, ffmpeg compiling attribute is configured, and parameters are configured according to the platform type, the cpu type, the codec attribute, the format conversion attribute and the cross compiling attribute.
12) Modifying acodec.h files, increasing parameter frame length SEI _ len and array SEI _ BUF, and determining parameter frame SIZE SEI _ BUF _ SIZE according to application layer requirements;
13) adding a parameter frame acquiring function in a decode _ nal _ sei _ prefix function in a hevc _ sei.c file: obtaining a parameter SIZE in a function, assigning the SIZE to SEI _ len, judging whether the SIZE is smaller than or equal to SEI _ BUF _ SIZE, copying data in a context parameter array gb to SEI _ BUF if a condition is met, wherein the copy length is the SIZE, and an array subscript i of the copied SEI _ BUF is an index of the gb array divided by 8, namely SEI _ BUF [ i ] ═ gb _ BUF [ index/8 ];
14) then, a configure command is executed, and the generated decoding libraries libavcodec, libavformat, libavutil and libswscale are located under the subfolder lib of the configuration folder.
15) Finally, copying the dynamic library to a/usr/lib path of a decoding board card;
2. configuring PCIE devices
Firstly, according to the transmission requirement, the PCIE device of haisi is configured as a slave device, and the PCIE device of the host is a master device.
Then, a drive file of the PCIE is configured. Automatically loading 3559 drive files in the system path/etc/profile of the Hai card: hi35xx _ dev _ slv.ko, irq _ map _ slv.ko, mcc _ drv _ slv.ko, mcc _ usrdev _ slv.ko, pcit _ dma _ slv.ko.
Finally, a transmission rule of the master equipment and the slave equipment is formulated, and the rule is as follows:
writing specified contents of handshake after the main equipment is electrified, and detecting corresponding contents and successfully handshaking after the slave equipment is electrified;
the method comprises the steps that a main device and a slave device distribute read-write PCIE addresses for each channel, and different read-write addresses are not conflicted with each other;
after the handshake of the main equipment is successful, the address content of the received data of each channel is written in a specified PCIE address field;
the main device judges whether the data exists or not by inquiring whether the content length of the fixed address field is greater than 0 or not, and clears the content length after the data reading is finished;
the slave equipment judges whether the master equipment reads the last data (namely the length of the fixed address segment is 0) through the length of the fixed address segment before transmitting the data, if so, the decoded image data is written in and the length is written in, and if not, the data writing task is abandoned, and the slave equipment waits for receiving the next decoded image data and then transmits the data;
3. obtaining data transmitted by a host through PCIE
Creating a PCIE read data thread, the thread being specifically operative to:
31) applying for a read-write exclusive lock of the PCIE channels, protecting the read-write operation of each PCIE channel, and not allowing a plurality of PCIE channels to perform the read-write operation at the same time;
32) checking whether the main device has new image data available for reading, namely whether the read data length in the fixed address is not zero or not, and simultaneously, the read address of the main device is not zero;
33) reading data in a specified read address field according to a protocol rule, and specifically operating: setting a destination address of a PCIE reading task as a PCIE reading address of the slave equipment; setting a source address as a PCIE writing address of the master device;
34) sending a data reading task command to the slave equipment through an ioctl function; and judging whether the return value of the data reading task is successful, and if the data reading task is successful, resetting the reading length to zero so as to enable the main equipment to transmit data next time.
35) And unlocking the read-write mutual exclusion lock of the PCIE channel, and releasing the resources.
4. Obtaining a complete data packet of a compressed image using an ffmpeg dynamic library
A usage context is created for ffmpeg and a soft decoder is specified that starts the requirements, a protocol analysis can be performed on the raw data. Meanwhile, original data are circularly obtained from a data area received by the network, a library function is called to obtain the length of a data packet which can be combined into a complete image frame, the length is removed from the data area received by the network, and the method is circularly repeated until the data area has no data. The method comprises the following specific steps:
41) initializing decoding library usage environment
First, a decoder type is set, and an HEVC (h265) type decoder is adopted. Then, initializing context environment of the decoder, and dynamically applying for image frame storage space according to the size of each frame of image data. Finally, the decoder is turned on according to the parameters.
42) Creating an ffmpeg acquisition data packet thread, wherein the thread comprises the following specific flows:
421) acquiring an image data array first address pointer bufPtr and a length bufLen read from a PCIE channel;
422) and judging whether the current data length bufLen is greater than 0, if so, continuing the next step, and if not, exiting the decoding process of the data, and waiting for the image data transmitted by the PCIE channel next time.
423) Transmitting a data array head address pointer bufPtr and a length bufLen to a soft decoding module, segmenting image data by frames by using a library function av _ parser _ parse2, if a complete image frame data packet can be successfully obtained from an array, recording and continuing the next step, and if not, exiting the decoding process; and waiting for the image data array transmitted by the PCIE channel.
424) And storing the length ret of the image frame divided in the data array at the time, removing the length ret of the image frame divided at the time from the total length bufLen of the data array, and moving the first address pointer bufPtr forwards for ret.
425) And putting the image complete data packet segmented this time into a queue to be decoded.
5. And sending the complete data packet to be decoded to a hard decoding module.
Starting a hard decoding module of the chip, dynamically applying for the size of an image buffer area according to the size of an image to be decoded, copying the original image data into the buffer memory, and carrying out hard decoding according to the configuration information. Meanwhile, a real-time monitoring thread is started aiming at the condition that the decoding module cannot work due to the error frame, the decoding state of the real-time monitoring thread is analyzed, and a soft reset measure is implemented, so that the decoding module can continuously perform decoding work.
51) Initializing a hard decoding module according to the image parameters and the decoding type, configuring the size of a video data buffer area of the hard decoding module, and starting the decoding module;
52) dynamically applying for the size buf of a buffer area of a video image frame data packet;
53) circularly judging whether the decoding is finished or not, if so, exiting the processing flow and carrying out the step 57), and if not, carrying out the next step;
54) setting parameters of a current frame to be decoded: stream end identifier, frame head identifier, frame tail identifier;
55) putting the image frame data into buf;
56) calling a dynamic library function to send buf data to a hard decoding module;
57) monitoring the decoding state of the hard decoding module in real time, if the decoding is wrong, restarting the hard decoding module in a soft mode and resetting parameters, and if the decoding is normal, calling a library function to obtain a decoded image;
58) and calling a library function to stop sending the video stream to the hard decoding module, closing a decoding channel, unbinding the binding relationship of each module and clearing resources.
6. Sending the decoded image to the host through PCIE
After a target image after hard decoding is obtained, image data is copied to a DMA (direct memory access) transmission memory space, a DMA transmission task is started to send the image data to a PCIE (peripheral component interface express) channel at the host end, and the host is informed of reading the data by writing the sending length into an address field specified by a protocol. The method comprises the following specific steps:
61) acquiring a read-write mutual exclusion lock of a PCIE channel;
62) copying the decoded image parameter data to a DMA virtual address space;
63) judging whether the writing length in the Haisi platform chip is cleared by the host (namely the last frame of image data is taken away by the host), if not, reporting an error and exiting, terminating the image sending task, and returning to the step 61 after receiving new decoded image parameter data); otherwise, go to step 64);
64) acquiring a target physical address of data sent to the main equipment, if the target physical address is zero, reporting an error and exiting, terminating an image sending task, and returning to the step 61 after receiving new decoded image parameter data); otherwise, go to step 65); the PCIE of the Haesi platform is configured into slave equipment, and the PCIE equipment of the host is master equipment;
65) acquiring a physical initial address and an offset address specified by a protocol of DMA transmission, taking the sum of the physical initial address and the offset address as a source address of a DMA transmission task, and taking a physical address vxworks _ phy _ addr of a host PCIE as a destination address of the task;
66) setting the transmission length of the task as the length sei _ len of parameter frame data;
67) judging whether the operation handle of the current PCIE is larger than zero, if so, using a system function ioctl to issue a PCIE writing task and entering the next step, if not, reporting an error and exiting the process, terminating the image sending task, and returning to the step 61 after receiving new decoded image parameter data);
68) judging whether the task state of the current PCIE writing task is finished, if so, entering the next step, and if not, reporting an error and exiting the process; returning to step 61) after receiving new decoded image parameter data;
69) setting a source address of a second DMA transmission task as image frame group data, taking an address of a host physical address vxworks _ phy _ addr offset parameter frame byte number sei _ len as a target address of the task, and setting an image frame length frame _ len as the transmission length of the second task;
610) repeat 67) and 68), proceed to the next step;
611) writing the sent image frame length frame _ len into a memory address specified by a protocol to prompt a host computer that data can be read;
612) reading image frame data and parameter frame data by using a host to finish the return work;
613) and unlocking the mutual exclusion lock of the PCIE channel, releasing the PCIE channel resource, and returning to the step 61) after receiving the new decoded image parameter data.
The steps 2), 3) and 6) of the invention start the PCIE channel to transmit the data, thereby effectively reducing the transmission delay.
The invention belongs to a video decoding technology under a linux system, and aims at the characteristics of Hi3559AV100 chips and the attribute of PCIE equipment in an application scene that the system is the linux, so that a video decoding scheme design method based on the combination of soft decoding and hard decoding under the linux system is designed and realized.
By utilizing the technical scheme and adopting the operation steps, the video decoding and transmission problems under the linux system can be realized, and the method is verified by an algorithm and is tested experimentally. The result shows that the scheme can adopt a soft decoding method to dynamically filter the error frame and obtain the complete data packet in order to solve the problem of image decoding, and uses a hard decoding module of a chip to finish the decoding process, thereby effectively reducing the decoding time.
Those skilled in the art will appreciate that those matters not described in detail in the present specification are well known in the art.