阅读说明：本技术 一种安防高清录像倒播方法及装置 (Security protection high-definition video reverse broadcasting method and device ) 是由 林德雄 江盛欣 黄仝宇 汪刚 宋一兵 侯玉清 刘双广 于 2019-03-22 设计创作，主要内容包括：本发明属于视频技术处理领域，具体涉及一种安防高清录像倒播方法及装置，所述的装置包括：分析模块、第一缓存模块、第二缓存模块、缓存管理模块和播放显示模块，分析模块，用于对目标码流进行分析，输出每帧的帧类型以及帧唯一标识；第一缓存模块，用于将分析模块输出的帧类型以及帧唯一标识按照播放顺序和GOP的方式进行存储；第二缓存模块，用于存储第一缓存模块的帧类型或帧唯一标识；缓存管理模块，设置第二缓存模块的缓存策略，并根据缓存策略将第一缓存模块中的内容交换到第二缓存模块中；播放显示模块，用于逆序播放第二缓存模块中的图像。本方案通过倍速因子分段多级缓存法，试图在CPU与内存消耗方面找到一个平衡，避免CPU与内存消耗过大，并满足多路及倍速流畅倒放的要求。(The invention belongs to the field of video technical processing, and particularly relates to a security high-definition video reverse broadcasting method and a security high-definition video reverse broadcasting device, wherein the security high-definition video reverse broadcasting device comprises the following components: the system comprises an analysis module, a first cache module, a second cache module, a cache management module and a play display module, wherein the analysis module is used for analyzing a target code stream and outputting a frame type and a frame unique identifier of each frame; the first cache module is used for storing the frame type and the unique frame identifier output by the analysis module according to the playing sequence and the GOP mode; the second cache module is used for storing the frame type or the frame unique identifier of the first cache module; the cache management module is used for setting a cache strategy of the second cache module and exchanging the content in the first cache module to the second cache module according to the cache strategy; and the playing and displaying module is used for playing the images in the second cache module in a reverse order. According to the scheme, a balance is tried to be found in the aspects of CPU and memory consumption by a speed-multiplying factor segmented multilevel cache method, the CPU and memory consumption is prevented from being overlarge, and the requirements of multipath and speed-multiplying smooth reverse playing are met.)

1. The utility model provides a security protection high definition video reverse playing device, the device include: the device comprises an analysis module, a first cache module, a second cache module, a cache management module and a play display module, and is characterized in that the analysis module is used for analyzing a target code stream and outputting a frame type and a frame unique identifier of each frame;

the first cache module is used for storing the frame type and the unique frame identifier output by the analysis module according to the playing sequence and the GOP mode;

the second cache module is used for storing the frame type or the frame unique identifier of the first cache module;

the cache management module is used for setting a cache strategy of the second cache module according to a cache time length m multiple speed factor s, a play frame rate fps, a GOP (group of pictures) buffer segmentation last frame pe and a segmentation first frame ps, and switching the content in the first cache module to the second cache module according to the cache strategy;

and the playing and displaying module is used for playing the images in the second cache module in a reverse order.

2. The security protection high-definition video reverse-playing device according to claim 1, wherein the first cache module sequentially stores the decoded image information according to a multiple speed factor, and the second cache module is used for storing the images exchanged by the first-level cache and playing and displaying the images in a reverse order.

3. The security protection high definition video reverse playing device according to claim 2, wherein the first cache module caches a part of segment images corresponding to the latest decoded GOP, the second cache module caches image segments to be displayed exchanged by the first cache module for reverse playing and displaying, and the first cache module can cache decoded images of a previous segment in parallel in the reverse playing and displaying process of images in the second cache module.

4. The security protection high definition video reverse playing device according to claim 2, wherein in the first cache module, according to the difference between the speed doubling content requirement and the frame rate of playing and displaying, frames in part of the speed doubling content are discarded.

5. The security protection high definition video reverse playing device according to claim 4, characterized in that according to a frame rate difference selective frame loss strategy, interval frame loss is selected, and one frame loss frame is stored periodically.

6. The security protection high definition video reverse playing device according to claim 3, wherein the length of the first buffer module and the second buffer module is the maximum time consumed by the computer client to decode a group of GOPs.

7. The security protection high definition video reverse playing device according to claim 1, wherein the GOP is stored in a structured manner, the framed target code stream is divided into I frames as a division boundary, the I frames and non-I frame images therebetween are sequentially arranged according to a playing and displaying sequence, and then the I frames of each GOP are sequentially arranged according to the playing and displaying sequence of the I frames of each GOP.

8. The security protection high definition video reverse playing device according to claim 7, wherein the storage information of each frame during GOP structured storage comprises a frame unique identifier, a frame type and position information of the frame in the cache.

9. The security protection high definition video playback device of claim 1, wherein the analysis module analyzes a complete frame, a type of the frame, a unique frame identifier, and timestamp information of the frame, wherein the type of the frame can be used for determining the decoding position.

10. A security high-definition video reverse broadcasting method is characterized by comprising the following steps:

1) sequentially framing an input code stream and sequentially outputting complete video frames and frame related information to a first buffer area A, wherein the frame related information comprises a frame type and a frame unique identifier;

2) the first buffer area A organizes and stores the input frame related information according to a GOP mode and stores the frame related information in a sequencing mode according to a playing sequence;

3) acquiring a first group of GOPs arranged in the first buffer area A according to a playing sequence, and inputting frame related information such as a frame type or a frame unique identifier into a second buffer area B;

4) setting a caching strategy of a second buffer area B and setting a decoding mark fdc to be in an un-decoded state according to a cached time length m multiple speed factor s, a playing frame rate fps, a GOP (group of picture) buffering segmented last frame pe and a segmented first frame ps;

5) judging whether the pe pointing position in the second buffer area B is smaller than the I frame position of the GOP, if so, deleting the first group of GOP target code stream in the first buffer area A, skipping to execute the step 3), and if not, executing the step 6);

6) judging fdc whether the decoding is finished, if yes, executing step 8), and if not, executing step 7);

7) sequentially inputting each frame of target code stream data of a first group of GOPs in the first buffer area A from the frame I to the frame pe to point to the decoding module for decoding, sequentially collecting decoded images, caching frames needing to be buffered according to a caching strategy in the second buffer area B, and setting fdc to be in a decoded state;

8) judging whether frames marked to be cached in a caching mark in frame intervals pointed by ps and pe in the second buffer area B are cached in corresponding decoded images, if so, executing a step 9), and if not, executing a step 5);

9) and sequentially switching the buffer images in the second buffer area B to the third buffer area C, sequentially displaying in a reverse order according to the playing and displaying sequence, and executing the step 4).

Technical Field

The invention belongs to the field of video technical processing, and particularly relates to a security high-definition video reverse playing method and device.

Background

The current encoding format in the market is mainly a video format file which is compressed and encoded between frames and in frames in a segmented, i.e. a Group of sequential frames, such as h264, h265, mpeg-4, etc., the frames of this type of video encoding are divided into multiple types, such as I, P, B frames, wherein the I frames appear periodically, and a Group of GOPs (Group of Pictures) is formed between two I frames with the I frame as a boundary, as shown in fig. 1. I decoding can be correctly decoded without referring to other frames, but P frame decoding can be correctly decoded only by referring to the previous I frame, otherwise, the video can be displayed in a screen-lost manner or cannot be played. When the video is played backwards, because the I frame is always behind the P frame depending on the I frame, the I frame needs to be found for the P frame, and the video can be decoded after the I frame depending on the P frame is found.

Disclosure of Invention

The invention aims to provide a security high-definition video reverse playing method and a security high-definition video reverse playing device.

The invention is realized by the following technical scheme:

the utility model provides a security protection high definition video reverse playing device, the device include: the device comprises an analysis module, a first cache module, a second cache module, a cache management module and a play display module, and is characterized in that the analysis module is used for analyzing a target code stream and outputting a frame type and a frame unique identifier of each frame;

the first cache module is used for storing the frame type and the unique frame identifier output by the analysis module according to the playing sequence and the GOP mode;

the second cache module is used for storing the frame type or the frame unique identifier of the first cache module;

the cache management module is used for setting a cache strategy of the second cache module according to a cache time length m multiple speed factor s, a play frame rate fps, a GOP (group of pictures) buffer segmentation last frame pe and a segmentation first frame ps, and switching the content in the first cache module to the second cache module according to the cache strategy;

and the playing and displaying module is used for playing the images in the second cache module in a reverse order.

Preferably, the first buffer module stores the decoded image information in sequence according to the multiple speed factor, and the second buffer module is used for storing the images exchanged by the first-level buffer and playing and displaying in reverse order.

Preferably, the first buffer module buffers a part of segment images corresponding to the latest decoded GOP, the second buffer module buffers image segments to be displayed, which are exchanged by the first buffer module, for reverse play and display, and the first buffer module can buffer the decoded images of the previous segment in parallel during the reverse play and display of the images in the second buffer module.

Preferably, in the first buffer module, according to the frame rate difference between the requirement of the double-speed content and the playing display, the frame in the partial double-speed content is discarded.

Preferably, according to the frame rate difference selective frame loss strategy, intermittent frame loss is selected, and one frame loss frame is periodically selected and stored.

Preferably, the first buffer module and the second buffer module have a length that is the maximum time consumed by the computing client to decode a group of GOPs.

Preferably, the GOP is stored in a structured manner, the framed target code stream is sequentially arranged according to the playing and displaying sequence of the I frames and the non-I frame images therebetween by taking the I frame as a division boundary, and then the I frame playing and displaying sequence of each GOP is arranged between the GOPs.

Preferably, the storage information of each frame when the GOP structure is stored includes a frame unique identifier, a frame type and position information of the frame in the buffer.

Preferably, the analysis module analyzes a complete frame, a type of the frame, a unique frame identifier, and timestamp information of the frame, wherein the type of the frame can be used for determining the decoding position.

The invention also provides a security high-definition video reverse broadcasting method, which comprises the following steps:

1) sequentially framing an input code stream and sequentially outputting complete video frames and frame related information to a first buffer area A, wherein the frame related information comprises a frame type and a frame unique identifier;

2) the first buffer area A organizes and stores the input frame related information according to a GOP mode and stores the frame related information in a sequencing mode according to a playing sequence;

3) acquiring a first group of GOPs arranged in the first buffer area A according to a playing sequence, and inputting frame related information such as a frame type or a frame unique identifier into a second buffer area B;

4) setting a caching strategy of a second buffer area B and setting a decoding mark fdc to be in an un-decoded state according to a cached time length m multiple speed factor s, a playing frame rate fps, a GOP (group of picture) buffering segmented last frame pe and a segmented first frame ps;

5) judging whether the pe pointing position in the second buffer area B is smaller than the I frame position of the GOP, if so, deleting the first group of GOP target code stream in the first buffer area A, skipping to execute the step 3), and if not, executing the step 6);

6) judging fdc whether the decoding is finished, if yes, executing step 8), and if not, executing step 7);

7) sequentially inputting each frame of target code stream data of a first group of GOPs in the first buffer area A from the frame I to the frame pe to point to the decoding module for decoding, sequentially collecting decoded images, caching frames needing to be buffered according to a caching strategy in the second buffer area B, and setting fdc to be in a decoded state;

8) judging whether frames marked to be cached in a caching mark in frame intervals pointed by ps and pe in the second buffer area B are cached in corresponding decoded images, if so, executing a step 9), and if not, executing a step 5);

9) and sequentially switching the buffer images in the second buffer area B to the third buffer area C, sequentially displaying in a reverse order according to the playing and displaying sequence, and executing the step 4).

Compared with the prior art, the invention has at least the following beneficial effects or advantages:

the security high-definition video reverse broadcasting method and the security high-definition video reverse broadcasting device adopt a segmented multi-stage storage mode, and save the memory. In addition, under the condition of not increasing the memory consumption, the playing and displaying of the second-level cache and the decoding cache of the first level are carried out in parallel, and smooth playing can be met. In addition, for a target code stream with 25fps, 50 frame size GOP and 1920 × 1080 resolution, only ceil (49/8) -ceil (n/8) +1 time per frame is required to be decoded in the scheme, and under a double-speed-factor frame loss strategy, the decoding times are fewer, and the memory is still kept at 48M.

Drawings

The present invention will be described in further detail with reference to the accompanying drawings;

FIG. 1 is a diagram of a group of GOP frame information in the prior art;

FIG. 2 is an overall frame diagram of the back sowing device of the present invention;

FIG. 3 is a diagram of frame information in buffer B of the present invention;

fig. 4 is a diagram of GOP-related information recorded in the buffer B of the present invention;

fig. 5 is a diagram of a process of setting a cache policy in the buffer B according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention provides a security high-definition video reverse-broadcasting device, as shown in fig. 2, comprising: the device comprises an analysis module, a decoding module and a playing display module, wherein a target code stream is analyzed by the analysis module to obtain a frame type and a frame unique identifier of each frame, structured storage is carried out in a GOP mode, a multi-level cache strategy is segmented according to a multiple speed factor, decoded information of each frame is sequentially and selectively stored in a first-level cache, namely a buffer area B, the content in the first-level cache is switched to a second-level cache, namely a buffer area C, according to the multi-level cache strategy, and images in the second-level cache are directly and sequentially pushed to the playing display module to be played and displayed in a reverse order.

The invention also provides a security high-definition video reverse broadcasting method which is applied to a security high-definition video reverse broadcasting device and comprises the following steps:

step 1) inputting the obtained target code stream into an analysis module, sequentially framing the input code stream by the analysis module, sequentially outputting complete video frames and frame related information into a buffer area A, and executing step 2), wherein the frame related information comprises related parameters such as frame types, frame unique identifiers and the like.

Step 2) the buffer area A organizes and stores the input frame related information according to a GOP mode, sorts and stores the frame related information according to a playing sequence, and executes the step 3);

the playing sequence here is defined as the sequence of playing and displaying, and has two levels:

frame ordering in the GOP of the same group, which may be defined as the chronological order of display of each frame, may typically be ordered by display Timestamp (Presentation Timestamp), but is not limited to being ordered by display Timestamp, e.g., frame1 is displayed before frame2, then in the current GOP frame1 should be arranged before frame2, i.e., frame1, frame2, …;

and ordering among GOPs, wherein the ordering among the GOPs is ordered according to the playing display sequence, typically according to the I-frame display time stamp, but not limited to the ordering according to the display time stamp.

Step 3) obtaining a first group of GOP arranged in the playing sequence in the buffer area A, inputting frame related information such as frame type or frame unique identification into the buffer area B, and executing step 4).

Step 4) according to the time length m and the speed factor s of the buffer, the playing frame rate fps (frames Per second), the last frame pe of the GOP buffer segment and the first frame ps of the segment are set, the buffer strategy of the buffer area B is set, the decoding mark fdc is set to be in an undecoded state, and the step 5 is executed;

4.1 the buffering time length m is a buffering size defined by a unit of time ms (millisecond), for example, a buffering size of 300ms, and for a video with a common 25 frame rate, a display time per frame is ceil (1000/25) ═ 40ms (ceil is defined as rounding up), and then a buffering time span of 300ms can buffer ceil (300/40) ═ 8 frames. The size of m can be determined by testing the maximum value of the time consumed by decoding a group of GOPs through a relevant test procedure or presetting a fixed value according to an empirical value, the value of m is generally less than half of the display time of GOP playing, for example, GOP of 2s and 50 frames is played in span, and the theoretical value of m is less than 2000/2-1000 ms, namely m <1000 ms.

The 4.2 speed factor s is determined by the external playback magnification, e.g. the speed factor is set to 4 when the external playback is performed at 4 times.

4.3 fps is an external pre-value, e.g. 25 fps.

4.4 ps is a frame position cursor indicating that the current decoding round needs to calculate whether the first frame is buffered.

4.5 pe is a frame position cursor indicating whether the current decoding round needs to calculate the last frame of the buffer.

4.6 buffer strategy of buffer B, as shown in FIGS. 3-5, the detailed setup steps are as follows:

4.6.1, judging whether the caching strategy of the GOP is set for the first time, if so, executing the step 4.6.2, and if not, executing the step 4.6.3;

4.6.2, first, recording related information of all frames contained in the current buffered GOP to be decoded, as shown in fig. 3, where the information includes a unique frame identifier, a frame type, a buffer flag (whether the frame is buffered, 1: identifier buffer, 0: identifier not buffered), a decoded data address, etc., and initializing the buffer flag to be 0 (not buffered state), if ps > is 0, setting ps pe to the last frame position of the group of GOPs, otherwise, setting ps pe to the last frame position + pe, and executing step 4.6.4;

4.6.3 updating pe value, where ps is the position of the frame before pe, i.e. ps equals pe-1, executing step 4.6.4;

4.6.4, modifying the tag value whether each frame is cached in the GOP related information according to the multiple speed factor s, the frame rate fps, the cache size m and the ps value, and updating the pe value;

for example: as shown in fig. 4 and fig. 5, assuming that the GOP size is 50 frames, and each frame is arranged in sequence from 0,1,2 …,48,49, fps is 25, s is 4, m is 300, it is known from m that ceil (300/(1000/25)) -8 frames can be currently buffered, it is required to play video content of which fps is 25, 4 is 100 frames in actual 1s, and fps is 25 only requires to play and display 25 frames, so frame loss buffering strategy can be selected, typically optimized uniform frame loss strategy can be selected, for example, 49 th frame is selected to be stored, 48 th, 47, 46 th frame is discarded, 45 th frame is stored, … th, and so on until a marker s ceil (m/(1000/s)) is set, i.e. 4 is 8 is 32 or I frame in encountered; at the end of marking, all the remaining unmarked frames are set to be not buffered. The setting result of this time is 49, 45, 41, 37, 33, 29, 25, 21 frame markers are buffered, the rest frame markers are not buffered, and the ps value is recorded as 18.

And 5) judging whether the position of the pe pointing in the buffer area B is smaller than the position of the I frame of the GOP, namely pe <0, if so, deleting the target code stream of the first group of GOP in the buffer area A, skipping to execute the step 3), and if not, executing the step 6).

Step 6) determining fdc whether the decoding is done, if yes, go to step 8), if not, go to step 7).

Step 7) sequentially inputting each frame of target code stream data of the first group of GOPs in the buffer area A from the frame I to the frame pe to point to the decoding module for decoding, sequentially collecting decoded images, caching frames needing to be buffered according to a caching strategy in the buffer area B, setting fdc to be in a decoded state, and executing the step 8);

if the GOP contains B frames, the decoding module must also be input with the P frame nearest to the pe-pointed frame as the boundary.

And 8) judging whether frames marked as frames needing to be cached in the frame intervals respectively pointed by ps and pe in the buffer area B have cached the corresponding decoded images, if so, executing the step 9), and if not, executing the step 5).

9) Sequentially exchanging the buffer images in the buffer area B into the buffer area C, sequentially pushing the buffer images to a playing display module for reverse-order display according to the playing display order, and executing the step 4);

9.1 buffer C is used for direct buffering of the playback display, which should be the same size as buffer B;

9.2 the time selection of the segmented image in the buffer B to the buffer C is exchanged, the playing and displaying in the buffer C can be typically selected to be completed, and after the space is released, the segmented image in the buffer B is triggered to be completely exchanged to the buffer C at one time; preferably, in C, the images in the buffer display one frame per play, i.e. triggering B to swap one frame into C.

In summary, compared with solution 1, in the embodiment of the present invention, a segmented multilevel storage method is adopted, for example, 25fps, 50 frame size GOP, and a target code stream with 1920 × 1080 resolution is adopted, after decoding, solution 1 adopts a full storage method, which needs to consume 3 × 50 — 150M memory, and in the embodiment, a segmented storage method is adopted, for example, the segmented cache length is set to 300ms, then a frame with a maximum parallel cache length of 300 × 2 — 600ms in the two-level segmented cache in the embodiment needs to consume ceil (300/(1000/25))/2 × 3 — 48M memory (ceil, function name, used for returning a minimum integer greater than or equal to a specified expression), and obviously, the embodiment saves more memory. In addition, under the condition that memory consumption is not increased, the scheme 1 cannot decode and cache in parallel in the playing and displaying process, so that the requirement of smooth playing cannot be met.

Compared with the technical scheme 2, also for a target code stream with a resolution of 1920 × 1080, like 25fps, a GOP with a size of 50 frames, with the technical scheme 2, similarly, assuming that a segment buffer memory occupies 300ms, that is, 8 frames, in this scheme, although the memory occupies 3 × 8 — 24M, the 41 th, 40 th and previous n frames in the GOP in the same group need to be decoded (49-8-n +1+1) times, in this embodiment, only ceil (49/8) -ceil (n/8) +1 times per frame needs to be decoded, and under a double-rate-factor frame dropping strategy, the number of decoding times will be less, and the memory still remains at 48M, obviously, on the basis of little memory, CPU consumption is greatly saved in this embodiment.

The above-mentioned embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, and it should be understood that the above-mentioned embodiments are only examples of the present invention and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement and the like made without departing from the spirit and scope of the invention are also within the protection scope of the invention.