阅读说明：本技术 信号增强编码结构 (Signal enhancement coding structure ) 是由 圭多·梅尔迪 洛伦佐·西卡雷利 西蒙妮·费拉拉 于 2019-08-01 设计创作，主要内容包括：公开了一种输入信号的编码方法,所述方法包括：通过向编码器(120E)馈送输入信号(10)的下采样(105D)版本来生成基本编码信号(基本)。所述方法还包括通过以下方式生成第一量化残差信号(1级)：解码(120D)所述基本编码信号来生成基本解码信号；并利用所述基本解码信号与所述输入信号的所述下采样版本之间的差值(110-S)生成第一残差信号；量化(120-1)所述第一残差信号来生成所述第一量化残差信号。所述方法还包括通过以下方式生成第二残差信号：逆量化(120-1i)所述第一量化残差信号来生成所述第一残差信号的重构版本；使用所述第一残差信号的重构版本来校正(110-C)所述基本解码信号,以创建校正后的解码版本；对所述校正后的解码版本上采样(105U)；并利用所述校正后的解码信号与所述输入信号之间的差值(100-S)生成所述第二残差信号。(A method of encoding an input signal is disclosed, the method comprising: a base encoded signal (base) is generated by feeding a down-sampled (105D) version of the input signal (10) to an encoder (120E). The method further comprises generating a first quantized residual signal (level 1) by: decoding (120D) the base encoded signal to generate a base decoded signal; and generating a first residual signal using a difference (110-S) between the base decoded signal and the down-sampled version of the input signal; quantizing (120-1) the first residual signal to generate the first quantized residual signal. The method further comprises generating a second residual signal by: inverse quantizing (120-1i) the first quantized residual signal to generate a reconstructed version of the first residual signal; correcting (110-C) the base decoded signal using the reconstructed version of the first residual signal to create a corrected decoded version; upsampling (105U) the corrected decoded version; and generating the second residual signal using a difference (100-S) between the corrected decoded signal and the input signal.)

1. A method of encoding an input signal, the method comprising:

generating a base encoded signal by feeding a down-sampled version of an input signal to an encoder;

generating a first quantized residual signal by:

decoding the basic coded signal to generate a basic decoded signal, an

Generating the first residual signal using a difference between the base decoded signal and a down-sampled version of the input signal;

quantizing the first residual signal to generate the first quantized residual signal;

generating a second residual signal by:

inverse quantizing the first quantized residual signal to generate a reconstructed version of the first residual signal;

correcting the base decoded signal using the reconstructed version of the first residual signal to create a corrected decoded version;

upsampling the corrected decoded version; and are

Generating the second residual signal using a difference between the corrected decoded signal and the input signal.

2. The method of claim 1, wherein the step of generating a first quantized residual signal comprises transforming the first residual signal before quantization to generate a first transformed quantized residual signal, and the step of generating a second residual signal comprises inverse transforming the first transformed quantized residual signal after inverse quantizing the first transformed quantized residual signal.

3. The method of claim 2, wherein the transforming step comprises directionally decomposing the first residual signal to create a component corresponding to the first residual signal.

4. The method of claim 3, wherein the directional decomposition transform comprises a 2 x 2 kernel or a 4 x 4 kernel.

5. The method of claim 3 or claim 4, wherein the directional decomposition transform is based on a Hadamard transform.

6. The method of claim 3 or claim 5, wherein the transformation generates the following components: mean, vertical, horizontal, and diagonal values.

7. The method according to any of the preceding claims, wherein the step of generating a second residual signal comprises quantizing the second residual signal to generate a second quantized residual signal.

8. The method according to any of the preceding claims, wherein the step of generating the second residual signal comprises transforming the second residual signal to generate a second transformed quantized residual signal before quantizing the second residual signal.

9. The method of claim 8, wherein the step of transforming the second residual signal comprises directional decomposition transforming the second residual signal to create a component corresponding to the second residual signal.

10. The method as claimed in claim 9 when dependent thereon, wherein the directional decomposition transform comprises a 2 x 2 kernel or a 4 x 4 kernel.

11. The method of claim 9 or claim 10, wherein the directional decomposition transform is based on a hadamard transform.

12. The method of any of claims 9 to 11, wherein the transformation generates the components of: mean, vertical, horizontal, and diagonal values.

13. The method of any of the preceding claims, wherein one or more of the first and second residual signals are entropy encoded.

14. The method according to any of the preceding claims, wherein quantizing the first residual signal comprises dividing the first residual signal by a predetermined step size, and inverse quantizing the first quantized residual signal comprises multiplying by a predetermined step size.

15. The method of claim 13 or claim 14, wherein the entropy encoding comprises one or both of run-length encoding and huffman encoding.

16. A method according to any preceding claim, wherein the input signal is a video signal.

17. The method of claim 16, wherein the input signal is a frame of a video signal.

18. A computer program adapted to perform the method of any of claims 1 to 17.

19. A computer-readable storage medium or data carrier comprising a computer program adapted to perform the method of any of claims 1 to 17.

20. An encoder comprising a processor and a memory, the encoder being adapted to perform the method of any of claims 1 to 17.

Technical Field

The present disclosure relates to a method and apparatus for encoding and/or decoding a signal. In particular, but not exclusively, the present disclosure relates to a method and apparatus for encoding video and/or image signals, but which may be extended to any other type of data to be compressed and decompressed.

Background

At present, there is an urgent need to create flexible solutions for signal encoding and decoding schemes, especially in the field of video encoding and decoding. It is also important to provide the highest quality video output to the viewer possible and to implement it in a manner that is backwards compatible with prior art and decoder hardware.

It is an object of the present disclosure to provide a solution to one or more of the above-mentioned needs.

Disclosure of Invention

The present disclosure provides a method, a computer program, a computer readable medium and an encoder as claimed in the appended claims.

Drawings

Embodiments of the invention will now be described with reference to the accompanying drawings, in which:

FIG. 1 is a block diagram depicting an encoding process;

FIG. 2 is a block diagram depicting a decoding process corresponding to the encoding process shown in FIG. 1;

FIG. 3 is a block diagram depicting the encoding process shown in FIG. 1 in greater detail;

FIG. 4 is a block diagram depicting the decoding process shown in FIG. 2 in greater detail;

FIG. 5 is a flow chart describing an encoding method;

fig. 6 is a block diagram illustrating a modified version of the encoder of fig. 3.

Detailed Description

The present disclosure describes a hybrid backward compatible encoding technique. The technique is a flexible, adaptable, efficient and computationally inexpensive encoding format that combines different video encoding formats and base codecs (i.e., encoder-decoders) (e.g., AVC/h.264, HEVC/h.265, or any other existing or future codec, non-standard algorithms such as VP9, AV 1) with at least two enhancement stages of encoded data.

The general structure of the coding scheme uses a downsampled source signal encoded by a base codec, adding a first level of correction or enhancement data to the decoded output of the base codec to generate a correction picture, and then adding another level of correction or enhancement data to an upsampled version of the correction picture.

Thus, a stream is considered to be a base stream and one or more enhancement streams, of which there are typically two enhancement streams. It is noted that in general the base stream can be decoded by a hardware decoder, while the enhancement stream is adapted to be implemented by a software process with a suitable power consumption.

The structure creates a plurality of degrees of freedom, and has great flexibility and adaptability in many cases, so that the coding format is suitable for various use cases including OTT transmission, live broadcast, live ultra high definition UHD (ultra high definition) live broadcast and the like. This structure also provides low complexity video coding.

Although the decoded output of the base codec is not intended for viewing, it is a fully decoded lower resolution video, making the output compatible with existing decoders and, where deemed appropriate, usable as a lower resolution output.

The codec format uses a minimum number of relatively simple coding tools. When combined synergistically, they may provide an improvement in visual quality compared to full resolution pictures encoded by a base codec, while yielding flexibility in the manner of use.

The methods and apparatus are based on an overall algorithm built on top of existing encoding and/or decoding algorithms (e.g., MPEG standards such as AVC/h.264, HEVC/h.265, non-standard algorithms such as VP9, AV1, etc.) that serve as baselines for the enhancement layer. The enhancement layer operates according to different encoding and/or decoding algorithms. The idea behind the overall algorithm is to hierarchically encode/decode the video frames instead of using block-based methods like the MPEG series of algorithms. Hierarchically encoding a frame includes generating a residual for the entire frame, then generating a reduced or decimated frame, and so on.

The block diagram shown in fig. 1 describes the encoding process. The encoding process shown is divided into two parts by a dashed line. Below the dashed line is the basic level of the encoder 100, which can be implemented efficiently in hardware. Above the dotted line is an enhancement stage that can be effectively implemented in software. The encoder 100 may include only the enhancement level process or a combination of the base level process and the enhancement level process as needed. The encoder 100 can be effectively implemented in software, especially at the enhancement level. For example, such an arrangement allows a legacy hardware encoder that provides a basic level to be upgraded with firmware (e.g., software) updates, where the firmware is configured to provide an enhanced level. In the new device both the basic and the enhancement stage can be provided in hardware and/or a combination of hardware and software.

The general level of the encoder topology is as follows. The encoder 100 comprises an input I for receiving an input signal 10. Input I is connected to downsampler 105D and processing block 100-0. The down sampler 105D outputs to the base codec 120 at the base stage of the encoder 100. The down sampler 105D also outputs to processing block 100-1. Processing block 100-1 passes the output to upsampler 105U, which upsampler 105U in turn outputs to processing block 100-0. Each processing block 100-0 and 100-1 includes one or more of the following modules: a transform block 110, a quantization block 120, and an entropy coding block 130.

An input signal 10, such as full (or highest) resolution video in this example, is processed by an encoder 100 to generate various encoded streams. A first coded stream (coded base stream) is generated by feeding a downsampled version of the input video 10 to a base codec 120 of the base level (e.g., AVC, HEVC, or any other codec) using a downsampler 105D. A base reconstruction is created by reconstructing the encoded base stream and then taking the difference between the base reconstruction and the down-sampled version of the input video 10 to create a second encoded stream (encoded level 1 stream). The differential signal is then processed at block 100-1 to create an encoded level 1 stream. The block 100-1 includes a transform block 110-1, a quantization block 120-1, and an entropy coding block 130-1. The third encoded stream (encoded level 0 stream) is created by upsampling the base reconstructed corrected version using the upsampler 105U and taking the difference between the base reconstructed corrected version and the input signal 10. The differential signal is then processed at block 100-0 to create an encoded level 0 stream. The block 100-0 includes a transform block 110-0, a quantization block 120-0, and an entropy coding block 130-0.

The encoded elementary stream may be referred to as a base layer or base level.

The corresponding decoding process is depicted in the block diagram of fig. 2. The decoding process shown is divided into two parts by a dashed line. Below the dashed line is the basic level of the decoder 200, which can be implemented efficiently in hardware. Above the dotted line is an enhancement stage that can be effectively implemented in software. The decoder 200 may include only the enhancement level process or a combination of the base level process and the enhancement level process as needed. The decoder 200 may be efficiently implemented in software, especially at the enhancement level, and may suitably use conventional decoding techniques, especially conventional hardware techniques. By "legacy technology", it is meant older technology that has been previously developed and sold in the marketplace, and that is inconvenient and/or very expensive to replace, but which can still be used to decode the signal.

The general level decoder topology is as follows. Decoder 200 includes an input (not shown) for receiving one or more input signals including an encoded elementary stream, an encoded level 1 stream, an encoded level 0 stream, and an optional header containing further decoding information. The decoder 200 includes a base decoder 220 at a base level and processing blocks 200-1 and 200-0 at an enhancement level. An upsampler 205U is also disposed between processing blocks 200-1 and 200-0 to provide an upsampled version of the signal output by processing block 200-1 to processing block 200-0.

The decoder 200 receives one or more input signals and directs the three streams generated by the encoder 100. The encoded elementary stream is directed to and decoded by a base decoder 220, the base decoder 220 corresponding to the base codec 120 used in the encoder 100 and its role is to reverse the encoding process at the base level. The encoded level 1 stream is processed by block 200-1 of decoder 200 to reconstruct the first residual created by encoder 100. The block 200-1 corresponds to the processing block 100-1 in the encoder 100, and the processing of the block 100-1 is inverted at the base level or largely inverted. The output of the base decoder 220 is combined with the first residual obtained from the encoded level 1 stream. The combined signal is up-sampled by an up-sampler 205U. The encoded level 0 stream is processed through block 200-0 to recreate the other residuals created by encoder 100. The block 200-0 corresponds to the processing block 100-0 of the encoder 100 and is used at the base stage to invert or substantially invert the processing of the block 100-0. The upsampled signal from upsampler 205U is combined with other residuals obtained from encoding the level 0 stream to create a level 0 reconstruction of input signal 10.

As described above, the enhancement stream may include two streams, i.e., an encoded level 1 stream (first enhancement level) and an encoded level 0 stream (second enhancement level). Encoding the level 1 stream provides a set of correction data that can be combined with a decoded version of the base stream to generate a correction picture.

Fig. 3 shows the encoder 100 of fig. 1 in more detail. The encoded elementary stream is created directly by the base encoder 120E and may be quantized and entropy encoded as needed. In some cases, these subsequent processes may be performed by the base encoder 120E as part of the encoding. To generate the encoded level 1 stream, the encoded elementary stream is decoded at the encoder 100 (i.e., a decoding operation is applied to the encoded elementary stream at the base decoding block 120D). The base decoding block 120D is shown as part of the basic level of the encoder 100 and is separate from the corresponding base encoding block 120E. For example, the base decoder 120D may be a decoding component that supplements an encoding component in the form of the base encoder 120E with a base codec. In other examples, conversely, base decoding block 120D may be part of an enhancement stage, and in particular may be part of processing block 100-1.

Returning to fig. 3, the difference between the decoded base stream output by base decoding block 120D and the downsampled input video results (i.e., subtraction operation 110-S is applied to the downsampled input video and the decoded base stream to generate a first set of residuals). The term "residual" is used herein in the same manner as known in the art. That is, "residual" means an error or difference between a reference signal or frame and a desired signal or frame. Here, the reference signal or frame is the decoded base stream, and the desired signal or frame is the downsampled input video. Thus, the residuals used in the first enhancement stage can be considered correction signals, since they are able to "correct" the future decoded base stream into the closest or closer approximation of the down-sampled input video used in the base coding operation. This is very useful because it can correct anomalies or other characteristics of the base codec, especially the motion compensation algorithms, quantization and entropy coding, and block adjustments applied by the base codec.

The first set of residuals is processed at block 100-1 shown in fig. 1. The components of the block are shown in more detail in figure 3. In particular, the first set of residuals is transformed, quantized, and entropy encoded to generate an encoded level 1 stream. As shown in fig. 3, transform operation 110-1 is applied to the first set of residuals; quantization operation 120-1 is applied to the set of transformed residuals to generate a set of quantized residuals; and applies entropy encoding operation 130-1 to the quantized set of residuals to generate an encoded level 1 stream for the first enhancement level. However, it should be noted that in other examples, only the quantization step 120-1 may be performed, or only the transformation step 110-1 may be performed. Entropy coding may not be used or may be selectively used except for the transform step 110-1 and/or the quantization step 120-1. The entropy encoding operation may be any suitable type of entropy encoding, such as a huffman encoding operation or a Run Length Encoding (RLE) operation or a combination of both.

As described above, the enhancement stream may include an encoded level 1 stream (first enhancement level) and an encoded level 0 stream (second enhancement level). The first level enhancement can be considered as enabling the corrected video at the base level, i.e. e.g. correcting anomalies of the encoder. The second level of enhancement may be considered another enhancement level for converting the corrected video to the original input video or an analog close thereto. For example, the second level enhancement may add detail lost during downsampling and/or help correct errors introduced by one or both of the transform operation 110-1 and the quantization operation 120-1.

Referring to fig. 1 and 3, to generate a coded level 0 stream, another enhancement information level is created by generating and coding another set of residuals at block 100-0. Another set of residuals is the difference between the up-sampled version of the corrected version of the decoded base stream (reference signal or frame) (by up-sampler 105U) and the input signal 10 (desired signal or frame).

To reconstruct a corrected version of the decoded base stream generated at the decoder 200, at least part of the processing steps of the block 100-1 are reversed to simulate the processing of the decoder 200 and account for at least part of the losses and anomalies in the transformation and quantization processes. To this end, the block 100-1 includes an inverse quantization block 120-1i and an inverse transform block 110-1 i. The quantized first set of residuals are inverse quantized at inverse quantization block 120-li and inverse transformed at inverse transform block 110-1i of encoder 100 to reproduce a decoder-side version of the first set of residuals.

The decoded base stream from decoder 120D is combined with the decoder-side version of the first set of residuals (i.e., summing 110-C of the decoded base stream and the decoder-side version of the first set of residuals, summing 110-C generates a reconstruction of the down-sampled version of the input video, which is likely generated at the decoder-i.e., the reconstructed base codec video). As shown in fig. 1 and 3, the reconstructed base codec video is up-sampled by an up-sampler 105U.

The up-sampled signal (i.e., the reference signal or frame) is then compared to the input signal 10 (i.e., the desired signal or frame) to create another set of residuals (i.e., the difference operation 100-S is applied to the up-sampled reconstructed stream to generate another set of residuals). Then, another set of residuals is processed at block 100-0 as an encoded level 0 stream (i.e., an encoding operation is applied to the other set of residuals to generate another encoded enhancement stream).

In particular, another set of residuals is transformed (i.e., a transform operation 110-0 is performed on the other set of residuals to generate another set of transformed residuals). The transformed residuals are then quantized and entropy encoded in the manner described above with respect to the first set of residuals (i.e., quantization operation 120-0 is applied to the set of transformed residuals to generate another set of quantized residuals, and entropy encoding operation 130-0 is applied to the other set of quantized residuals to generate an encoded level 0 stream containing another level of enhancement information). However, only the quantization step 120-1, or only the transformation and quantization steps may be performed. In addition, entropy encoding may be selectively used. Preferably, the entropy coding operation may be a huffman coding operation or a run length coding RLE operation or a combination of both.

Thus, as shown in fig. 1 and 3 and described above, the output of the encoding process is one base stream at the base stage and one or more enhancement streams at the enhancement stage, which preferably includes a first enhancement stage and another enhancement stage.

An encoded base stream and one or more enhancement streams are received at a decoder 200. Fig. 4 shows the decoder of fig. 2 in more detail.

The encoded base stream is decoded at a base decoder 220 to generate a base reconstruction of the input signal 10 received at the encoder 100. In actual use, this basic reconstruction may be used to provide a visual reproduction of the signal 10 at a lower quality level. However, the main purpose of this basic reconstructed signal is to lay a good foundation for a high quality reproduction of the input signal 10. To this end, the decoded elementary stream is provided to the processing block 200-1. Processing block 200-1 also receives the encoded level 1 stream and inverts any encoding, quantization and transforms that have been applied by encoder 100. The block 200-1 includes an entropy decoding process 230-1, an inverse quantization process 220-1, and an inverse transformation process 210-1. Alternatively, only one or more steps may be performed, depending on the operations performed at the respective block 100-1 of the encoder. By performing these respective steps, a decoded level 1 stream comprising a first set of residuals may be used at the decoder 200. The first set of residuals is combined with the decoded base stream from the base decoder 220 (i.e., the sum of the decoded base stream and the decoded first set of residuals 210-C is performed to generate a reconstruction of a down-sampled version of the input video, i.e., the reconstructed base codec video). As shown in fig. 2 and 4, the reconstructed base codec video is upsampled by an upsampler 205U.

Further, optionally in parallel, the encoded level 0 stream is processed at block 200-0 of fig. 2 to produce another set of decoded residuals. Similar to processing block 100-0, processing block 200-0 includes entropy decoding process 230-0, inverse quantization process 220-0, and inverse transformation process 210-0. Of course, these operations will correspond to operations performed at block 100-0 of encoder 100, and one or more steps may be omitted as desired. Block 200-0 generates a decoded level 0 stream that includes another set of residuals and adds these residuals to the output from upsampler 205U in operation 200-C to create a level 0 reconstruction of input signal 10.

Thus, as shown in fig. 1-4 and described above, the output of the decoding process is the basic reconstruction, and is a higher level original signal reconstruction. This embodiment is particularly suitable for creating encoded and decoded video at different frame resolutions. For example, the input signal 10 may be an HD video signal including frames of 1920 × 1080 resolution. In some cases, the display device may use both the base reconstruction and the level 0 reconstruction. For example, in the case of using network traffic, a level 0 stream may be more disturbed than a level 1 stream and a base stream (since it may contain as much as 4 times the amount of data that the sampling reduces the dimension in each direction by 2). In this case, the display device may revert to displaying the base reconstruction when traffic occurs while interrupting the level 0 stream (e.g., when level 0 reconstruction is not available) and then return to displaying the level 0 reconstruction when network conditions improve. A similar approach may be applied when the decoding device is resource constrained, e.g., a set top box performing a system update may have the capability to operate the base decoder 220 to output a base reconstruction, but may not have the processing power to compute a level 0 reconstruction.

The encoding arrangements of fig. 1 and 3 also enable a video distributor to distribute video to a set of heterogeneous devices. Only users with the base decoder 220 can view the base reconstruction, while users with the enhancement level can view the higher quality level 0 reconstruction. In the comparative case, two complete video streams of different resolutions are required to serve both sets of devices. Since the level 0 enhancement stream and the level 1 enhancement stream encode residual data, the level 0 enhancement stream and the level 1 enhancement stream can be encoded more efficiently. For example, the distribution of residual data is usually a large part around 0 (i.e. where there is no difference) and is usually taken within a small range around 0, especially after quantization. In contrast, a complete video stream of different resolutions will have a different distribution, with a non-zero mean or median value, requiring a higher bit rate to be transmitted to the decoder.

Description of the tools

It was mentioned above how a set of tools can be applied to each enhancement stream (or input video) throughout the process. An overview of each tool and its function in the overall process as shown in figures 1 to 4 is given below.

Down sampling

A downsampling process is applied to the input video to generate a downsampled video to be encoded by the base codec. Typically, downsampling reduces the resolution of the picture. The down-sampling may be performed in both the vertical and horizontal directions, or may be performed only in the horizontal direction. And any suitable downsampling process may be used.

Level 1 (L-1) coding

The input to the tool comprises L-1 residuals obtained by taking the difference between the decoded output of the base codec and the down-sampled video. The L-1 residual is then transformed, quantized, and encoded.

Transformation of

The transform tool uses a directional decomposition transform, such as a hadamard transform.

In this process, there are two types of transformations that are particularly useful. Both have small kernels (i.e. 2 x 2 or 4 x 4) that can be directly applied to the residual. Further details regarding transformations may be found in, for example, patent applications PCT/EP2013/059847 or PCT/GB2017/052632, which are incorporated herein by reference. In another example, the encoder may select between different transforms to be applied, for example between a 2 x 2 kernel and a 4 x 4 kernel. This makes the coding of the residual more flexible. The selection may be based on an analysis of the data to be transformed.

The transform may convert the residual information into four planes, which may generate the following components, for example: mean, vertical, horizontal, and diagonal values.

Quantization

Any known quantization scheme can be used to create the residual signal as a quantum, so that certain variables can only take some discrete amount. In one case, the quantization includes dividing by a predetermined step size. This can be applied to two levels (0 and 1). For example, the quantization at block 120-1 may include dividing the transformed residual by the step size. The step size may be predetermined, for example, selected based on a desired quantization level. In one case, the divide-by-step size can be converted to a multiply-by-inverse-step size, which can be more efficiently implemented in hardware. In this case, the inverse quantization, as in block 120-li, may comprise multiplication by the step size.

Entropy coding

The quantized coefficients are encoded using an entropy encoder. In the entropy coding scheme, the quantized coefficients are first encoded using Run Length Encoding (RLE), and then the encoded output is processed using a huffman encoder. However, when entropy encoding is required, only one of these schemes may be used.

Level 1 (L-1) decoding

The input to the tool includes L-1 encoded residuals, which are passed through an entropy decoder, inverse quantizer and inverse transform module. The operations performed by these modules are the inverse operations performed by the above modules.

Upsampling

A combination of the decoded L-1 residual and the base decoded video is upsampled to generate an upsampled reconstructed video. The upsampling will be described in more detail below.

Level 0 (L-0) coding

The input to the tool includes L-0 residuals, which are obtained by taking the difference between the upsampled reconstructed video and the input video. The L-0 residue is then transformed, quantized and encoded as described below. The transformation, quantization and coding are performed in the same manner as described in connection with the L-1 coding.

Level 0 (L-0) decoding

The input to the tool includes the encoded L-0 residual. The decoding process of the L-0 residual is performed by an entropy decoder, an inverse quantizer and an inverse transform module. The operations performed by these modules are the inverse operations performed by the above modules.

Residual data structure

In the above encoding/decoding algorithm, there are typically 3 planes of data (e.g., YUV or RGB for image or video data), each plane having two quality levels (LoQ) which are described as level 0 (or LoQ-0 or highest level, full resolution) and level 1 (LoQ-1 or lower, small-scale resolution such as half-resolution), respectively.

Introduction to the basic coding procedure

Fig. 5 is a flowchart describing a basic encoding method. The specific method comprises the following steps:

step 510: a base encoded signal is generated from an up-sampled version of an input signal.

Step 520: the base encoded signal is decoded.

Step 530: the down-sampled version is compared to the decoded version to create a first residual signal.

Step 540: the first residual signal is quantized.

Step 550: inverse quantizing the quantized first residual signal.

Step 560: the decoded first decoded signal is combined with the inverse quantized first residual signal.

Step 570: the combined signal is upgraded.

Step 580: the input signal is compared to the upgrade signal to create a second residual signal.

Of course, the method may include features compatible with the description of fig. 1-4. In particular, the method may further comprise transforming and inverse transforming the first residual signal.

Fig. 6 is a block diagram illustrating a modified version of the encoder of fig. 3.

It can be seen that the transform block 110-1 and the entropy coding block 130-1 are removed, and thus the quantization step 120-1 and the inverse quantization step 120-1i are performed only at the encoder 100. Likewise, only the transform and inverse transform may be applied. The reason for performing one or both of the inverse quantization step and the inverse transform step at the encoder 100 is to take into account artifacts, noise or other imperfections introduced into the level 1 residual signal in the level 0 residual. Of course, it may also be desirable to configure entropy coding block 130-1 to reduce the bit rate of the coded level 1 stream. As can be seen from fig. 6, the coded level 0 stream containing the other set of residuals is only quantized similarly by the quantization block 120-0. It is also possible to send another set of residuals without any coding or to perform only the transformation process 110-0.

As shown in fig. 3 and 6, the base codec 120 may be divided into two components, i.e., a base encoder 120E and a base decoder 120D. It is noted that the encoding and decoding portions may be placed in a single codec module (hardware or software), or they may be separate modules or components. The base decoder 120D may reside at an enhancement stage of the encoder 100, if desired.

In the examples described herein, the residual may be considered an error or difference at a particular quality or resolution level. In the depicted example, there are two sets of residuals (level 1 and level 0) because there are two quality or resolution levels. Each set of residuals described herein simulates a different form of error or difference. For example, a level 1 residual typically corrects features of the base encoder, e.g., corrects artifacts introduced by the base encoder during the encoding process. In contrast, for example, a level 0 residual can often correct for the complex effects introduced by changes in quality levels, as well as the differences introduced by a level 1 correction (e.g., artifacts generated by a level 1 encoding pipeline over a wider spatial range, such as a 4 or 16 pixel region). This means that the operations performed on one set of residuals do not necessarily have the same effect on another set of residuals, e.g. each set of residuals may have a different statistical pattern and associated set.