阅读说明：本技术 基于小波变换的码流预测方法及系统 (Code stream prediction method and system based on wavelet transformation ) 是由 车俊岭 王慧 于 2021-10-27 设计创作，主要内容包括：本发明公开的一种基于小波变换的码流预测方法及系统,首先将当前帧画面拆分为多个图像块,然后对各个图像块进行小波变换,得到若干个子频带；再然后将各个子频带按照重要性分图层进行传输；其次根据各图层的DWT系数的绝对值之和预测各图层经过编码后的编码码流,进而得到当前帧的各图层的编码码流；最后根据预测出的当前帧的各图层的编码码流,确定当次待编码的图层数,使得当前帧每个图层在编码后所产生的码流与当前的网络带宽相匹配,保证传输质量。(The invention discloses a code stream prediction method and a code stream prediction system based on wavelet transformation.A current frame picture is firstly split into a plurality of image blocks, and then each image block is subjected to wavelet transformation to obtain a plurality of sub-frequency bands; then, transmitting each sub-band according to the importance map layer; secondly, predicting the coded code stream of each layer according to the sum of the absolute values of the DWT coefficients of each layer, and further obtaining the coded code stream of each layer of the current frame; and finally, determining the number of layers to be coded at the time according to the predicted code stream of each layer of the current frame, so that the code stream generated after each layer of the current frame is coded is matched with the current network bandwidth, and the transmission quality is ensured.)

1. A code stream prediction method based on wavelet transform is characterized by comprising the following steps:

step 1, splitting a current frame picture into a plurality of image blocks;

step 2, performing wavelet transformation on each image block to obtain a plurality of sub-bands;

step 3, transmitting each sub-frequency band according to the importance map layer;

step 4, predicting the coded code stream of each layer according to the sum of the absolute values of the DWT coefficients of each layer, and further obtaining the coded code stream of each layer of the current frame;

and step 5, determining the number of the layers to be coded at the current time according to the predicted code stream of each layer of the current frame, so that the code stream generated after each layer of the current frame is coded is matched with the current network bandwidth.

2. The wavelet transform-based code stream prediction method of claim 1, wherein 10 subbands are obtained after each image block is processed by using three-level wavelet transform in step 2.

3. The wavelet transform-based code stream prediction method of claim 1, wherein quantization processing is performed on each sub-band in step 2 to eliminate inter-pixel redundancy.

4. The wavelet transform-based code stream prediction method of claim 1, wherein in step 4, the predicted code streams of the same layer of each image block are added to obtain the predicted code stream size of each layer of the current frame.

5. The wavelet transform-based code stream prediction method according to claim 1, wherein in step 5, the number of layers to be coded is the number of layers of a current frame that can be carried by a target bandwidth.

6. The wavelet transform-based code stream prediction method according to claim 1, wherein the method for determining the number of picture layers to be coded in step 5 is as follows:

s1, if the code stream predicted by the first layer of the current frame is smaller than the target bandwidth, the number of the picture layers to be coded is one;

s2, calculating the difference between the target bandwidth and the predicted code stream of the first layer of the current frame, comparing the difference with the predicted code stream of the second layer of the current frame, and when the predicted code stream of the second layer is less than the difference, the number of the picture layers to be coded is two;

and S3, repeating the step S2 until the code stream predicted by the nth layer is larger than the difference value, wherein the number of the layers to be coded is n-1 layers.

7. A system of wavelet transform-based code stream prediction method according to any one of claims 1-6, comprising,

the splitting module is used for splitting a current frame picture into a plurality of image blocks;

the wavelet transformation module is used for performing wavelet transformation on each image block output by the splitting module to obtain a plurality of sub-bands;

the layer module is used for transmitting each sub-band output by the wavelet transform module according to importance map layers;

the prediction module is used for coding code streams of all layers of the current frame;

and the layer module to be coded is used for determining the number of the layers to be coded at the current time according to the coded code stream of each layer of the current frame predicted by the prediction module.

8. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 6 when executing the computer program.

Technical Field

The invention relates to the technical field of image coding, in particular to a code stream prediction method and a code stream prediction system based on wavelet transformation.

Background

The size of the code stream generated by the existing video coding method is only related to the number of the changed macro blocks in the current frame, and due to the limitation of network bandwidth, if the size of the code stream generated by each frame is not controlled, the size of the code stream generated by coding is not matched with the size of the network bandwidth, so that network congestion and packet loss are caused, and the quality of a video picture at a decoding end is influenced finally.

Therefore, a code stream prediction method needs to be designed to match the coded code stream of the current frame with the current network bandwidth so as to ensure the video picture quality.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a code stream prediction method based on wavelet transformation, so that the code stream generated after each layer of a current frame is coded is matched with the current network bandwidth, and the video picture quality is ensured.

The invention is realized by the following technical scheme:

a code stream prediction method based on wavelet transformation comprises the following steps:

step 1, splitting a current frame picture into a plurality of image blocks;

step 2, performing wavelet transformation on each image block to obtain a plurality of sub-bands;

step 3, transmitting each sub-frequency band according to the importance map layer;

step 4, predicting the coded code stream of each layer according to the sum of the absolute values of the DWT coefficients of each layer, and further obtaining the coded code stream of each layer of the current frame;

and step 5, determining the number of the layers to be coded at the current time according to the predicted code stream of each layer of the current frame, so that the code stream generated after each layer of the current frame is coded is matched with the current network bandwidth.

Preferably, after each image block is processed by using three-level wavelet transform in step 2, 10 subbands are obtained.

Preferably, in step 2, quantization processing is performed on each sub-band to eliminate redundancy between pixels.

Preferably, in step 4, the predicted code streams of the same layer of each image block are added to obtain the predicted code stream size of each layer of the current frame.

Preferably, the number of layers to be coded in step 5 is the number of layers of the current frame that can be carried by the target bandwidth.

Preferably, the method for determining the number of map layers to be encoded in step 5 is as follows:

s1, if the code stream predicted by the first layer of the current frame is smaller than the target bandwidth, the number of the picture layers to be coded is one;

s2, calculating the difference between the target bandwidth and the predicted code stream of the first layer of the current frame, comparing the difference with the predicted code stream of the second layer of the current frame, and when the predicted code stream of the second layer is less than the difference, the number of the picture layers to be coded is two;

and S3, repeating the step S2 until the code stream predicted by the nth layer is larger than the difference value, wherein the number of the layers to be coded is n-1 layers.

A system of code stream prediction method based on wavelet transform comprises,

the splitting module is used for splitting a current frame picture into a plurality of image blocks;

the wavelet transformation module is used for performing wavelet transformation on each image block output by the splitting module to obtain a plurality of sub-bands;

the layer module is used for transmitting each sub-band output by the wavelet transform module according to importance map layers;

the prediction module is used for coding code streams of all layers of the current frame;

and the layer module to be coded is used for determining the number of the layers to be coded at the current time according to the coded code stream of each layer of the current frame predicted by the prediction module.

A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of a wavelet transform-based code stream prediction method when executing the computer program.

Compared with the prior art, the invention has the following beneficial technical effects:

the code stream prediction method based on wavelet transform provided by the invention has the advantages that the size of the code stream generated by coding after wavelet transform is predicted, and then the data and the number of transmission image layers transmitted each time are determined according to the current network bandwidth, so that the code stream generated by each image layer of the current frame after coding is matched with the current network bandwidth, and the transmission quality is ensured.

Drawings

FIG. 1 is a DWT variation process for a tile of the present invention;

FIG. 2 shows the DWT variation results of the inventive tile.

FIG. 3 is a flow chart of wavelet transform-based code stream prediction according to the present invention;

Detailed Description

The present invention will now be described in further detail with reference to the attached drawings, which are illustrative, but not limiting, of the present invention.

Referring to fig. 1 to 3, a code stream prediction method based on wavelet transform includes the following steps:

step 1, splitting a current frame picture into a plurality of image blocks.

Specifically, in this embodiment, the current frame picture is divided into a plurality of image blocks of 16 × 16 pixels, where each image block is a macroblock.

The current frame picture is divided into a plurality of macro blocks, and one macro block is composed of a luminance pixel block and two additional chrominance pixel blocks. In general, a luminance block is a 16x16 sized block of pixels, while the size of two chrominance image blocks depends on the sampling format of their image, e.g., for a YUV420 sampled image, a chrominance block is an 8x8 sized block of pixels. In each image, a plurality of macro blocks are arranged in a form of a slice, and a video coding algorithm codes the macro blocks one by taking the macro blocks as units to organize a continuous video code stream.

And 2, performing wavelet transformation on each image block to obtain a plurality of sub-bands, and preferentially transmitting the sub-bands containing the basic information of the image in the transmission process.

As shown in fig. 1 and fig. 2, in the present embodiment, after each image block is processed by using three-level wavelet transform, 10 subbands are obtained, including LL3, HL3, LH3, HH3, HL2, LH2, HH2, HL1, LH1, and HH1, after the image data is converted into a frequency domain, the low-frequency component includes the basic features of the image, and the high-frequency component supplements the detail information of the image, such as the edge contour information of the image signal. Therefore, the above sub-bands are sorted by importance into LL3, HL3, LH3, HH3, HL2, LH2, HH2, HL1, LH1, HH1 according to the visual characteristics of human eyes. The LL3 sub-band contains basic information of an image, is also a DWT coefficient of a pixel point most concerned by human eyes, and is preferentially transmitted in the transmission process.

And 3, quantizing each sub-band to eliminate redundancy among pixels.

For compression, after the image block is subjected to DWT transform, the LL3 subbands are first subjected to difference operation to eliminate inter-pixel redundancy, specifically, the first value of the LL3 subband is kept unchanged, the previous value is subtracted from each subsequent value, and then the obtained data is quantized.

And 4, carrying out layered transmission on each sub-band obtained in the step 3 according to importance, carrying out zero filling on the position which is not received after a decoding end receives a part of DWT coefficients, and generating a complete current frame after the decoding end receives superposition of a plurality of layers.

The layering of the sub-bands needs to be determined according to actual conditions, and the layering form in the embodiment is as follows:

a first layer: LL3

A second layer: HL3, LH3, HH3

And a third layer: HL2

A fourth layer: LH2, HH2

And a fifth layer: HL1

A sixth layer: LH1, HH1

And 5, predicting the encoded code stream of each layer of each image block after huffman encoding according to the sum of the absolute values of the DWT coefficients of each layer, and further obtaining the encoded code stream of each layer of the current frame.

Before Huffman coding is carried out, the size of the code stream of each image layer of each image block after Huffman coding needs to be predicted, the Huffman coding target is the quantized DWT coefficient, the size of the DWT coefficient determines the size of the generated code stream, and the larger the absolute value is, the larger the code stream is. Therefore, the size of the coded code stream can be predicted according to the sum of the absolute values of the DWT coefficients of each layer.

According to multiple tests, the corresponding relation between the sum of the absolute values of the DWT coefficients of each layer and the coded code stream in the example can be obtained as follows:

a first layer: preBit [1] (total [1 ]/6);

a second layer: PreBit 2 total 2/4

And a third layer: PreBit [3] total [3]/3

A fourth layer: PreBit [4] (total [4 ]/1)

And a fifth layer: PreBit [5] (total [5 ]/1)

A sixth layer: PreBit [6] (total [6 ]/1)

Under different scenes, different types of images are transmitted, and corresponding coefficient values may also be different and need to be determined according to actual conditions.

And 6, determining the number of the layers to be coded at the time according to the predicted code stream of each layer of the current frame.

In order to accurately predict the code stream of each layer of the current frame, the predicted code streams of the same layer of each image block are added to obtain the predicted code stream size of each layer of the current frame.

It should be noted that the number of layers to be coded is the number of layers of the current frame that can be carried by the target bandwidth. When the target bandwidth is larger, the number of layers to be coded is larger, and when the target bandwidth is smaller, the number of layers to be coded is smaller.

The following describes a process of calculating the number of layers to be coded at this time:

s1, if the code stream predicted by the first layer of the current frame is smaller than the target bandwidth, the number of the picture layers to be coded is one;

s2, calculating the difference between the target bandwidth and the predicted code stream of the first layer of the current frame, comparing the difference with the predicted code stream of the second layer of the current frame, and when the predicted code stream of the second layer is less than the difference, the number of the picture layers to be coded is two;

and S3, repeating the step S2 until the code stream predicted by the nth layer is larger than the difference value, wherein the number of the layers to be coded is n-1 layers.

For example, the predicted code stream of each layer of the current frame is pre _ lay [ x ].

Wherein x is the number of drawing layers, and x is 0, 1, 2, 3, 4, 5; the target bandwidth is the coarse _ bit; the number of layers to be coded is layer.

Judging whether pre _ lay [ x ] is smaller than the good _ bit, if so, indicating that the target bandwidth can at least transmit the 1 st layer, and continuing to execute the subsequent steps; presetting x as 0 and layer as 0, and determining the number of layers to be coded as follows:

layer＝layer+1；

goal_bit＝goal_bit–pre_lay[x]；

and judging whether the good _ bit is larger than zero, if so, x is equal to x +1, and if not, determining that the number of the graph layers to be coded is layer.

The invention also provides a system of the code stream prediction method based on the wavelet transform, which comprises a splitting module, a wavelet transform module, a layer module, a code stream prediction module and a layer module to be coded.

The splitting module is used for splitting a current frame picture into a plurality of image blocks;

the wavelet transformation module is used for performing wavelet transformation on each image block output by the splitting module to obtain a plurality of sub-bands;

the layer module is used for transmitting each sub-band output by the wavelet transform module according to importance map layers;

the code stream prediction module is used for coding code streams of all layers of the current frame;

and the layer module to be coded is used for determining the number of the layers to be coded at the current time according to the coded code stream of each layer of the current frame predicted by the prediction module.

In an exemplary embodiment, a terminal device is further provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the wavelet transform-based code stream prediction method when executing the computer program. The processor may be a Central Processing Unit (CPU), other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.

In an exemplary embodiment, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the wavelet transform-based codestream prediction method. The computer storage medium may be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical memory (e.g., CD, DVD, BD, HVD, etc.), and semiconductor memory (e.g., ROM, EPROM, EEPROM, nonvolatile memory (NANDFLASH), Solid State Disk (SSD)), etc.

The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.