阅读说明：本技术 色度残差缩放的延迟降低方法和装置 (Chroma residual scaling delay reduction method and apparatus ) 是由 林芷仪 庄子德 陈庆晔 徐志玮 于 2020-03-13 设计创作，主要内容包括：公开了一种视频解码的方法和装置。根据一种方法,色度残差缩放因子是基于并置亮度块的相邻预测或重建的亮度样本导出的,其中并置亮度块的相邻预测或重建的亮度样本对应于沿着并置亮度块的顶部边界的M个样本和沿并置亮度块的左边界的N个样本中的样本。根据导出的色度残差缩放因子,将色度缩放应用于色度残差块的色度残差样本。在另一种方法中,色度残差缩放因子是基于并置亮度处理数据单元外部的一个或多个重建的亮度样本得出的。在另一种方法中,色度残差缩放因子在数据流的APS(自适应参数集)中发信或从中解析出来。(A method and apparatus for video decoding are disclosed. According to one approach, the chroma residual scaling factor is derived based on neighboring predicted or reconstructed luma samples of the collocated luma block, wherein the neighboring predicted or reconstructed luma samples of the collocated luma block correspond to samples from the M samples along a top boundary of the collocated luma block and the N samples along a left boundary of the collocated luma block. Chroma scaling is applied to chroma residual samples of the chroma residual block in accordance with the derived chroma residual scaling factor. In another approach, the chroma residual scaling factor is derived based on concatenating one or more reconstructed luma samples outside the luma processing data unit. In another approach, the chroma residual scaling factor is signaled in or parsed from an APS (adaptive parameter set) of the data stream.)

1. A method of video decoding, the method comprising:

receiving chroma residual data associated with a current chroma processing data unit in a picture, wherein the picture is divided into a plurality of non-overlapping processing data units, each processing data unit comprising a luma processing data unit and one or more chroma processing data units;

deriving one or more chroma residual scaling factors based on one or more reconstructed luma samples outside of a collocated luma processed data unit associated with the current chroma processed data unit; and

chroma scaling is applied to chroma residual samples of the current chroma processing data unit according to the derived one or more chroma residual scaling factors.

2. The method of claim 1, wherein the one or more reconstructed luma samples outside the collocated luma processed data unit correspond to one or more reconstructed luma samples of one or more previously coded luma processed data units.

3. The method of claim 2, wherein the one or more reconstructed luma samples of the one or more previously coded luma processed data units correspond to one or more reconstructed luma samples along a top boundary of the collocated luma processed data unit, one or more reconstructed luma samples along a left boundary of the collocated luma processed data unit, or both.

4. The method of claim 1, wherein the luma size of the one or more non-overlapping processed data units is equal to Min (CtbSizeY, 64) multiplied by Min (CtbSizeY, 64), and wherein CtbSizeY specifies the luma width and the luma height of the luma codec tree block.

5. A video decoding apparatus, the apparatus comprising one or more electronic circuits or processors arranged to:

receiving chroma residual data associated with a current chroma processing data unit in a picture, wherein the picture is divided into a plurality of non-overlapping processing data units, each processing data unit comprising a luma processing data unit and one or more chroma processing data units;

deriving one or more chroma residual scaling factors based on one or more reconstructed luma samples outside of a collocated luma processed data unit associated with the current chroma processed data unit; and

chroma scaling is applied to chroma residual samples of the current chroma processing data unit according to the derived one or more chroma residual scaling factors.

6. A method of video decoding, the method comprising:

receiving chroma residual data associated with a current chroma processing data unit in a picture, wherein the picture is divided into a plurality of non-overlapping processing data units, each processing data unit comprising a luma processing data unit and one or more chroma processing data units;

deriving one or more chroma residual scaling factors from a first coding unit based on one or more reconstructed luma samples, wherein the first coding unit covers an upper-left position of a collocated luma processed data unit associated with the current chroma processed data unit; and

chroma scaling is applied to chroma residual samples of the chroma residual data associated with the current chroma processing data unit according to the derived one or more chroma residual scaling factors.

7. The method of claim 6, wherein the one or more non-overlapping processed data units have a luminance size equal to Min (CtbSizeY, 64) multiplied by Min (CtbSizeY, 64), and wherein CtbSizeY specifies a luminance width and a luminance height of the luminance codec tree block.

8. The method of claim 6, wherein the one or more reconstructed luma samples outside the first codec unit covering the collocated luma processed data unit correspond to one or more reconstructed luma samples of one or more previously codec luma processed data units.

9. The method of claim 6, wherein the one or more reconstructed luma samples of the one or more previously coded luma units correspond to one or more reconstructed luma samples along a top boundary of the first Codec Unit (CU), one or more reconstructed luma samples along a left boundary of the first Codec Unit (CU), or both, wherein the first codec unit covers the collocated luma units.

10. The method of claim 9, wherein the number of reference reconstructed luma samples along the top boundary of the first codec unit covering the collocated luma unit data is equal to the width or height of the luma unit data.

11. The method of claim 9, wherein the number of reference reconstructed luma samples along the left boundary of the first codec unit covering the collocated luma unit data is equal to the width or height of the luma unit data.

12. A video encoding and decoding method, the method comprising:

receiving a current chroma residual block;

signaling one or more chroma residual scaling factors in or parsing the video data stream at an adaptive parameter set level; and

chroma scaling is applied to chroma residual samples of the current chroma residual block.

13. A video decoding apparatus, the apparatus comprising one or more electronic circuits or processors arranged to:

receiving a current chroma residual block;

signaling one or more chroma residual scaling factors in or parsing the video data stream at an adaptive parameter set level; and

chroma scaling is applied to chroma residual samples of the current chroma residual block.

14. A method of video decoding, the method comprising:

receiving a current chroma residual block;

deriving one or more chroma residual scaling factors based on neighboring predicted or reconstructed luma samples of a collocated luma block associated with the current chroma residual block, wherein the neighboring predicted or reconstructed luma samples of the collocated luma block correspond to samples among M samples along a top boundary of the collocated luma block and N samples along a left boundary of the collocated luma block, wherein M and N are positive integers; and

chroma scaling is applied to chroma residual samples of the current chroma residual block in accordance with the one or more derived chroma residual scaling factors.

15. The method of claim 14, wherein the neighboring predicted or reconstructed luma samples of the collocated luma block correspond to the M samples along the top boundary of the collocated luma block.

16. The method of claim 14, wherein the neighboring predicted or reconstructed luma samples of the collocated luma block correspond to the N samples along the left boundary of the collocated luma block.

17. The method of claim 14, wherein the neighboring predicted or reconstructed luma samples of the collocated luma block correspond to both the M samples along the top boundary and the N samples along the left boundary of the collocated luma block.

18. The method of claim 14, wherein if boundary samples at an upper left position of the collocated luma block are available, deriving the one or more chroma residual scaling factors using the boundary samples at the upper left position of the collocated luma block.

19. The method of claim 18, wherein if boundary samples at an upper left position of the collocated luma block are not available, deriving the one or more chroma residual scaling factors using left boundary samples along a left boundary of the collocated luma block or top boundary samples along a top boundary of the collocated luma block.

20. A video decoding apparatus, the apparatus comprising one or more electronic circuits or processors arranged to:

receiving a current chroma residual block;

deriving one or more chroma residual scaling factors based on neighboring predicted or reconstructed luma samples of a collocated luma block associated with the current chroma residual block, wherein the neighboring predicted or reconstructed luma samples of the collocated luma block correspond to samples among M samples along a top boundary of the collocated luma block and N samples along a left boundary of the collocated luma block, wherein M and N are positive integers; and

chroma scaling is applied to chroma residual samples of the current chroma residual block in accordance with the one or more derived chroma residual scaling factors.

[ technical field ] A method for producing a semiconductor device

The invention relates to video codec for color video data, wherein a luminance map is applied to the luminance component. In particular, techniques for deriving and/or signaling one or more chroma scaling factors (scaling factors) for chroma residual scaling are disclosed.

[ background of the invention ]

Multifunctional video codec (VVC) is an emerging video codec standard developed by the joint video Experts Group (joint video Experts Group) consisting of the ITU-T16 th research Group video codec expert Group and ISO/IEC JTC1SC29/WG11 (Moving Picture Experts Group, abbreviated MPEG)). VVC is based on HEVC (high efficiency video codec) video standard with improved and new codec tools. For example, the remolding (reshap) process is a new codec tool employed in VTM-4.0(VVC test model version 4.0). The reshaping process is also called LMCS (Luma Mapping and Chroma Scaling). When applying reshaping, the video samples are codec and reconstructed in the reshaped domain before loop filtering (loop filter). The reshaped domain reconstructed sample is converted into the original domain by using inverse reshaping. The loop-filtered original domain reconstructed samples are stored in a decoded picture buffer. For Inter mode (Inter mode), the Motion Compensated (MC) predictor is converted to a reshaped domain by using forward reshaping. Fig. 1 shows an example of a reshaping process at the decoder side.

As shown in fig. 1, the data stream is decoded by a CABAC (context adaptive binary arithmetic coding) decoder (i.e., CABAC)^-1) Inverse quantization (i.e. Q)^-1) And inverse transformation (T)^-1) Deriving a reconstructed luminance residual Y_res. The reconstructed luma residual is provided to a luma reconstruction block 120 to generate a reconstructed luma signal. For intra mode, the predictor is from an intra prediction block 130. For inter mode, the predictor comes from the motion compensation block 140. Since the reshaping is applied to the luminance signal on the encoder side, forward reshaping 150 is used for the predictor from motion compensation block 140 before it is provided to reconstruction block 120. Inverse reshaping 160 is applied to the reconstructed luminance signal from reconstruction block 120 to recover an un-reshaped reconstructed luminance signal. Then, the signal is stored in a decoded picture bufferThe loop filter 170 is applied to the unshaped reconstructed luminance signal before in (DPB) 180.

When applying reshaping, chroma residual scaling is also applied. Chroma residual scaling may compensate for the interaction between the luma signal and the chroma signal, as shown in fig. 2. In fig. 2, the upper part corresponds to luminance decoding and the lower part corresponds to chrominance decoding.

Chroma residual scaling is applied at the TU level at the encoder side and the decoder side, respectively, according to the following equation:

encoder side C_ResScale＝C_Res*C_Scale＝C_Res/C_ScaleInv (1)

Decoder side C_Res＝C_ResScale/C_Scale＝C_ResScale*C_ScaleInv (2)

In the above equation, C_ResIs the original chrominance residual signal, C_ResScaleIs the scaled chrominance residual signal. C_ScaleIs a scaling factor calculated for the inter mode predictor using fwdllut (i.e., forward look-up table) and converted to its inverse C_ScaleInvTo perform multiplication on the decoder side instead of division, thereby reducing implementation complexity. The scaling operations at the encoder and decoder sides are both implemented by fixed-point integer arithmetic (fixed-point integer arithmetric) by the following formula:

c’＝sign(c)*((abs(c)*s+2^{CSCALE_FP_PREC-1})>>

CSCALE_FP_PREC) (3)

in the above equation, c is the chroma residual, s is the chroma residual scaling factor in the scscalelnv [ piexedx ], piexedx is determined by the corresponding average luminance value of TU, CSCALE _ FP _ PREC is a constant for specified precision. To derive the scaling factor, the predictor of the entire TU is used. The value of C _ ScaleInv was calculated as follows:

if the mode is the intra-frame mode, calculating the average value of the intra-frame prediction brightness values; if it is an inter mode, an average value of forward reshaped inter prediction luminance values is calculated. In other words, the average luminance value acgY 'is calculated in the remoulded domain'_TU。

Find index idx, where avgY'_TUBelonging to the inverse mapping PWL.

C_ScaleInv＝cScaleInv[idx]

Deriving the chroma scaling factor C as performed by block 210 in FIG. 2_ScaleInvThe step (2). Derived chroma scaling factor C_ScaleInvChroma residual for transform scaling, which is decoded by CABAC (context adaptive binary arithmetic coding) (i.e. CABAC)^-1) Inverse quantization (i.e. Q)^-1) And inverse transformation (T)^-1) To reconstruct. The reconstruction block 220 reconstructs a chrominance signal by adding a predictor to the reconstructed chrominance residual. For intra mode, the predictor is from an intra prediction block 230. For inter mode, the predictor comes from the motion compensation block 240. Then, a loop filter 270 is applied to the reconstructed chrominance signal before storing the signal in a chrominance Decoded Picture Buffer (DPB) 280.

Fig. 3 shows an example of a luminance map. In fig. 3A, there is shown 1: 1 mapping, where the output (i.e., reshaped luminance) is the same as the input. Since the histogram of luminance samples is typically not flat, using intensity reshaping may help improve performance in terms of RDO (rate-distortion optimization). Statistics of luminance samples are computed for an image region (e.g., picture). The mapping curve is then determined based on the statistics. Generally, a piece-wise linear (PWL) mapping curve is used. Fig. 3B shows an example of a piecewise linear (PWL) mapping with 3 segments, where two adjacent segments have different slopes. Dashed line 340 corresponds to 1: 1 mapping. If the samples ranging from 0 to 340 have a large spatial variance and a small number of occurrences (numbers of occurrences), then the input ranges 0-340 are mapped to a smaller output range (i.e., 0-170), as indicated by line segment 310 of FIG. 3B. If the samples ranging from 340 to 680 have small spatial variations and occur more frequently, the input range 340-680 is mapped to a larger output range (i.e., 170-850), as indicated by the segment 320 in FIG. 3B. If the samples ranging from 680 to 1023 have large spatial variations and occur less frequently, the input range 680-1023 is mapped to a smaller output range (i.e., 850-1023), as indicated by segment 330 in FIG. 3B. Fig. 3B is intended to illustrate a simple PWL mapping. In practice, the PWL map may have more or fewer segments.

Intra-frame sub-block partitioning (ISP for short) and sub-block transform (SBT for short)

To generate better intra mode predictors, intra sub-block partitioning (ISP) may be applied. When the ISP is applied, the luminance component is divided into a plurality of sub TBs. The sub-TBs are reconstructed one by one. For each sub-TU, the reconstructed samples of the neighboring sub-TBs may be used as neighboring reconstructed samples for intra prediction. The chroma component TB is not divided into sub-TBs as is the case with luminance.

Similar to ISPs, sub-block transformations (SBTs) may be applied to inter modes. When SBT is applied, only part of the CU data is converted. For example, the current partition may be divided into two partitions by horizontal or vertical partitioning. Only one partition can be used for transcoding. The residual of the other partition is set to zero. For example, a CU is divided into two TUs or four TUs. Only one of the TUs has a non-zero coefficient.

Signaling of LMCS parameters

A syntax table of the LMCS parameters under consideration for VVC is shown in table 1.

Table 1.

In the above syntax table, the semantics of the syntax are defined as follows:

lmcs _ min _ bin _ idx specifies the minimum data sub (bin) index of the PWL (piecewise Linear) model of luminance mapping

lmcs _ delta _ max _ bin _ idx specifies the delta value (delta value) between 15 and the maximum data sub-index LmcsMaxBinIdx used in lmcs. This value should be in the range of 1 to 15 inclusive.

lmcs _ delta _ cw _ prec _ minus1 plus 1 is the number of data used to represent the syntax lmcs _ delta _ abs _ cw [ i ].

lmcs _ delta _ abs _ cw [ i ] is the absolute delta codeword value for the ith data element.

lmcs _ delta _ sign _ cw _ flag [ i ] is the sign (sign) of the variable lmcsDeltaCW [ i ].

The variable lmcsDeltaCW [ i ] is derived as follows:

lmcsDeltaCW[i]＝

(1-2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i].

the variable lmcsCW [ i ] specifies the number of codewords per interval in the mapped domain, where i is 0 … 15. It can be derived as follows:

OrgCW＝(1<<BitDepthY)/16

for i equal to 0 … lmcs _ min _ bin _ idx-1, lmcsCW [ i ] is set equal to 0.

For i ═ lmcs _ min _ bin _ idx.. LmcsMaxBinIdx, the following applies:

lmcsCW[i]＝OrgCW+lmcsDeltaCW[i]

the value of lmcsCW [ i ] should be in the range of (OrgCW > >3) to (OrgCW < <3-1) (including (OrgCW > >3) and (OrgCW < < 3-1)).

For i-LmcsMaxBinIdx +1 … 15, lmcsCW [ i ] is set equal to 0.

To represent the PWL model of the remodeling curve, three variables lmcspinvot [ i ] (i ═ 0 … 16), ScaleCoeff [ i ] (i ═ 0 … 15), and InvScaleCoeff [ i ] (i ═ 0 … 15) were derived as follows:

in the above derivation, SCALE _ FP _ PREC is a constant value for specifying precision.

In the LMCS process, the delay of chroma residual scaling may negatively impact processing speed by relying on corresponding luma data. Therefore, it is desirable to develop methods and apparatus to reduce the delay of chroma residual scaling.

[ summary of the invention ]

Methods and apparatus for video decoding are disclosed. According to one method of the present invention, a current chroma residual block is received. Deriving one or more chroma residual scaling factors based on neighboring predicted or reconstructed luma samples of the collocated luma block, wherein neighboring predicted or reconstructed luma samples of the collocated luma block associated with the current chroma residual block correspond to samples from the M samples along a top boundary of the collocated luma block and the N samples along a left boundary of the collocated luma block, wherein M and N are positive integers. Applying chroma scaling to chroma residual samples of a current chroma residual block in accordance with the derived one or more chroma residual scaling factors.

In one embodiment, neighboring predicted or reconstructed luma samples of the collocated luma block correspond to M samples along the top boundary of the collocated luma block. In another embodiment, neighboring predicted or reconstructed luma samples of the collocated luma block correspond to N samples along the left boundary of the collocated luma block. In yet another embodiment, neighboring predicted or reconstructed luma samples of the collocated luma block correspond to both M samples along the top boundary of the collocated luma block and N samples along the left boundary of the collocated luma block.

In one embodiment, the boundary samples at the top left position of the collocated luma block are used to derive the one or more chroma residual scaling factors if the boundary samples at the top left position of the collocated luma block are available. If boundary samples at the top left position of the collocated luma block are not available, either left boundary samples along the left boundary of the collocated luma block or top boundary samples along the top boundary of the collocated luma block are used to derive a scaling factor representing one or more chroma residual errors.

According to another method, chroma residual data associated with a current chroma processing data unit in a picture is received, wherein the picture is divided into a plurality of non-overlapping processing data units, and each processing data unit includes a luma processing data unit and one or more chroma processing data units. Deriving one or more chroma residual scaling factors based on one or more reconstructed luma samples outside of a collocated luma processed data unit associated with a current chroma processed data unit. Chroma scaling is then applied to chroma residual samples of the current chroma processed data unit in accordance with the derived one or more chroma residual scaling factors. According to a variant of the method, the chroma residual scaling factor is derived based on one or more reconstructed luma samples from a first Codec Unit (CU) covering a top-left position of the collocated luma processing data unit.

In one embodiment, the one or more reconstructed luma samples outside of a first Codec Unit (CU) covering the collocated luma processing data unit correspond to one or more reconstructed luma samples of one or more previously codec luma processing data units. In another embodiment, the one or more reconstructed luma samples of the one or more previously coded luma processing data units correspond to one or more reconstructed luma samples along a top boundary of the first Codec Unit (CU) covering collocated luma, one or more reconstructed luma samples along a left boundary of the first Codec Unit (CU) covering collocated luma, or both.

In one embodiment, the reconstructed luma samples outside the collocated luma processed data unit correspond to one or more reconstructed luma samples of one or more previously decoded luma processed data units. For example, the reconstructed luma samples of the one or more previously decoded luma processed data units correspond to one or more reconstructed luma samples along a top boundary of the collocated luma processed data units, one or more reconstructed luma samples along a left boundary of the collocated luma processed data units, or both.

In yet another method, the one or more chroma residual scaling factors are signaled in an APS (Adaptation Parameter Set) level of the video data stream at the encoder side or parsed from the APS level of the video data stream at the decoder side.

[ description of the drawings ]

Fig. 1 shows an exemplary block diagram of a video decoder incorporating a luminance reshaping process.

Fig. 2 shows an exemplary block diagram of a video decoder incorporating a luma reshaping process and chroma scaling.

Fig. 3A shows 1: an example of a 1 luminance map, where the output (i.e. reshaped luminance) is the same as the input.

Fig. 3B shows an example of a piecewise linear (PWL) luminance map with 3 segments.

Fig. 4 illustrates an example of deriving a chroma scaling factor based on luma samples reconstructed from a reference along a VPDU top boundary, a left boundary, or both, according to an embodiment of the invention.

Fig. 5 shows an example of deriving a chroma scaling factor based on a reference reconstructed luma sample TL, a or L position according to an embodiment of the present invention.

Fig. 6 illustrates a flow diagram of an exemplary decoding system for deriving one or more chroma residual scaling factors based on neighboring predicted or reconstructed luma samples of collocated luma blocks according to an embodiment of the present invention.

Fig. 7 shows a flow diagram of another exemplary decoding system for deriving one or more chroma residual scaling factors based on one or more reconstructed luma samples outside a collocated luma processing data unit according to an embodiment of the present invention.

Fig. 8 illustrates a flow diagram of an exemplary codec system in which one or more chroma residual scaling factors are signaled in or parsed from an APS (adaptive parameter set) level of a video data stream on an encoder side, according to an embodiment of the invention.

[ detailed description ] embodiments

The following description is of the best modes contemplated for carrying out the present invention. This description is made for the purpose of illustrating the general principles of the present invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

In chroma residual scaling, for a chroma TU, all corresponding luma predictors are used to derive one single scaling factor. The chroma sample reconstruction cannot be processed until the scaling factor is derived. It introduces new data dependencies for the cross-component process, resulting in longer delays in chroma sample reconstruction. In VVC, some decoder aids are introduced to perfect luma predictors to improve codec efficiency. These types of codec tools will also increase the critical path of the reconstruction loop. In inter and intra mode prediction, prediction samples of a CU/PU/TU may be divided into multiple MxN blocks, and the blocks may be processed sequentially or in parallel.

In one embodiment, to reduce the delay of chroma sample reconstruction, for a CU/PU/TU, it uses only the KxL luma samples above and to the left (e.g., luma predictor or luma reconstructed samples or luma residuals) or the M luma samples above and to the left for deriving one or more chroma residual scaling factors. K and L may be equal to 1, 2, 4, 8, 16, 32, or 64. One or more scaling factors are used for the entire chroma TU. For example, the top left 16x15 luma samples are used. In another example, the top left 1x1 luma samples are used. In another example, the top left 256 luma samples are used. In another example, the top left 1 luma sample is used. In another example, if the width and height of a luma CU/TU is greater than or equal to 16, the top left 16x16 luma samples are used; otherwise, a maximum of 256 top left luminance samples are used. In one example, when an ISP is applied, only the top left KxL block or top left M samples of the first ISP sub-TB are used to derive the chroma residual scaling factor. In another example, when applying SBT, only the top left KxL block or top left M samples of TUs with non-zero coefficients are used to derive the scaling factor. In another embodiment, only a portion of the corresponding luma samples are used to derive chroma residual scaling factors. For example, a portion of the intra-collocated luma CT/TU/PU boundary samples, e.g., a portion of the top row (top-row) and a portion of the left column of the intra-collocated luma CT/TU/PU boundary samples, are used to derive chroma residual scaling factors.

In another embodiment, to reduce the delay for chroma sample reconstruction for a CU/PU/TU, only neighboring boundary samples along the current TB (i.e., corresponding luma samples or luma samples referred to as collocated) are used to derive one or more chroma residual scaling factors. The samples may be predicted samples or reconstructed samples of neighboring blocks. In an embodiment, M samples along the top boundary are used to derive one or more chroma residual scaling factors. In an embodiment, N samples along the left boundary are used to derive one or more chroma residual scaling factors. In an embodiment, M samples along the top boundary and N samples along the left boundary are used to derive one or more chroma residual scaling factors. Here, M and N may be 1, 2, 4, 8, 16, 32, or 64. In another embodiment, samples located at the upper left corner of the L-shaped boundary are used to derive one or more chroma residual scaling factors. In another embodiment, if the top left adjacent sample is available, then the sample is used. Otherwise, one of the top neighboring samples or one of the left neighboring samples is used. In one example, if none of the above samples are available, the top left sample in the collocated luma block is used. One or more scaling factors are used for the entire chroma TU.

In another embodiment, to reduce the delay of chroma sample reconstruction when applying chroma residual scaling, it is proposed to divide chroma TUs into sub-blocks, e.g. KxL sub-blocks or sub-blocks with a block size equal to M. K and L may be 2, 4, 8, 16 or 32; m may be 4, 8, 16, 32, 64, 128, 256, 512, or 1024. For each KxL chroma residual sub-block, one or more scaling factors are derived. Different KxL chroma residual sub-blocks may have different scaling factors. For example, for an M × N block, where M is greater than K (i.e., a width threshold) and N is less than L (i.e., a height threshold), the M × N block is divided into M/K blocks of size (K × N).

In another embodiment, when the size/area/width/height of the chroma residual TU is smaller than the first threshold or larger than the second threshold, chroma residual scaling is not applied. For example, when TU size is less than or equal to 8 or 16 or 64, chroma residual scaling will be disabled. In another example, chroma residual scaling will be disabled when TU width or height is less than or equal to 2 or 4 or 8 or 16. In another example, chroma residual scaling is disabled when TU size is greater than or equal to 16, 64, 256, or 1024. In another example, chroma residual scaling is disabled when TU width or height is greater than or equal to 8 or 16 or 32. In another example, chroma residual scaling is disabled for certain prediction modes. For example, chroma residual scaling is disabled for blocks that have DMVR mode enabled, BIO mode, LIC mode, diffuse mode, or a combination of these modes enabled.

DMVR (decoder-side motion vector refinement) is a new codec tool developed in recent years. The DMVR derives MV refinement information at the decoder side to improve the coding and decoding performance. BIO is another new codec tool developed in recent years. The BIO derives a sample-level motion refinement from the assumption of optical flow and stable motion, where the current pixel in the B slice (bi-predictive slice) is predicted by one pixel in reference picture 0 and one pixel in reference picture 1. LIC (Local Illumination Compensation) is a method of performing inter prediction using neighboring samples of a current block and a reference block. It is based on a linear model using scaling factors and offsets.

In one embodiment, when the ISP is applied, only a part of the luma sub-TB is used to derive the chroma residual scaling factor. For example, only the first luma TB is used to derive the chroma residual scaling factor. Generating the scaling factor using the first luma TB may reduce a delay of chroma sample reconstruction. In another example, only the last luma TB is used to derive the chroma residual scaling factor.

In another embodiment, each luminance sub-TB may be considered as a separate TB when the ISP is applied. For each sub-TB, it may calculate its own chroma residual scaling factor. The above proposed method can also be applied, for example, dividing each luminance sub-TB into several KxL sub-blocks and deriving a scaling factor for each sub-block. For the chroma TB, even though it is not divided into a plurality of sub-TBs as much as the luminance when the transform is performed, the chroma TB is divided into a plurality of sub-regions when the chroma residual scaling is performed. Each sub-region corresponds to a luminance TB; each sub-region corresponds to one or more luminance sub-TBs; or one or more chrominance sub-regions correspond to one luminance sub-TB. For each chroma sub-region, if the luma sub-TB is divided into a plurality of sub-blocks to derive the scaling factor, it may be further divided into a plurality of sub-blocks.

In another embodiment, only luma partitions with non-zero coefficients are used to derive chroma residual scaling factors when SBT is applied. The luminance partition used may be divided into sub-blocks to derive the scaling factors. In another embodiment, when applying SBT, the luma samples of the entire CU may be used to derive one or more scaling factors.

In another embodiment, luma samples of a CU (not a TU or TB) are used to derive chroma residual scaling factors. When ISP is applied, the entire luma CU sample will be used to derive the chroma residual scaling factor. For example, the luma CU samples may be divided into sub-blocks to derive different scaling factors for different sub-blocks. The sub-blocks may cross ISP sub-TB boundaries.

In another embodiment, the chroma residual scaling factor derivation may be different for applying a transform or not applying a transform (e.g., transform skip). For chroma residual scaling factor derivation, the values/factors/constants or equations may be different. In another embodiment, the chroma residual scaling factor derivation may be different for different prediction modes or different residual energy levels.

At the encoder side, the scaling factor derivation typically includes deriving λ for the quantization parameter. In one embodiment, the entire TU prediction data is used to derive the λ value. For chroma residual scaling, the TU is still divided into sub-blocks. Each sub-block may derive its own scaling factor.

In the BIO and DMVR processes, the same type of process problems will be encountered. For example, for the BIO process, SAD (sum of absolute differences) calculation at TU/PU/CU level is performed. If the calculated cost is small enough, the BIO process may be disabled. For the DMVR process, if the entire CU/PU/TU is used to derive one MV difference (MVD), this is not a friendly design. Therefore, it is proposed to align the BIO with the DMVR, or even align the BIO and/or DMVR with the chroma residual scaling process, which divides the current block into KxL blocks. For example, for BIO and DMVR processes, the current block is divided into KxL blocks. For each KxL block, it can calculate the cost of its BIO early termination decision, and can also derive its MVD using the DMVR process. In another example, for a BIO or DMVR procedure, the current block is divided into a KxL block for performing the BIO and DMVR procedure, where the size of KxL (in units of luma sample precision) is the same as the basic unit of the chroma residual scaling procedure.

In another embodiment, different modes may use the reference luma samples at different locations.

In one embodiment, for a block that may reference neighboring reconstructed samples for the prediction process, the reference luma samples used for the scaling value derivation are from neighboring reconstructed samples or reference boundary samples used to generate the predictor of the current CU or TU. For example, if the current block is an intra prediction mode, the reference luma sample is the top left, top, or left reference boundary sample of the current CU. Thus, for intra sub-partition prediction (ISP) mode, chroma residual scaling values are derived using the top left, top or left boundary reconstructed samples of the L-shaped boundary reconstructed samples of the current CU/TU (not the sub-partition TU). In another example, if the current block is an intra prediction mode, the reference luma sample is an upper-left reference boundary sample of the current TU. Thus, for intra sub-partition prediction (ISP) mode, chroma residual scaling values are derived using the top-left, top, or left boundary reconstructed samples of the L-shaped boundary reconstructed samples of the current TU (sub-partition). The upper left L-shaped boundary reconstructed sample may be one sample.

In another example, if the current CU is inter prediction mode, but predicted by combining inter/intra modes (CIIP) or other prediction methods that require neighboring reconstructed samples, the reference luma samples may be reference boundary reconstructed samples or reference boundary samples used to generate a predictor for the current CU or TU (e.g., using the top left neighboring reconstructed samples), as described above. As is known in the art, CIIP is yet another codec tool developed in recent years. CIIP uses a weighted average of the inter and intra prediction signals to obtain CIIP prediction.

In another embodiment, if the current block is inter prediction mode, the reference luma sample may be the top-left luma prediction sample of the current CU or TU.

In one embodiment, if CIIP mode, the reference luma sample is the top left luma prediction sample of the inter predictor.

In another embodiment, if the current block is an inter prediction mode other than the CIIP mode, the reference luma sample may be the top-left luma prediction sample of the current CU or TU. In this embodiment, a block coded in the CIIP mode is regarded as an intra prediction mode, and any of the above-described methods related to the intra prediction mode may be applied.

In another embodiment, if the current block is an IBC mode, the decision to refer to luma samples is the same as inter prediction mode. As known in the art, IBC (intra block copy) is a new codec tool developed in recent years. IBC is similar to inter prediction mode. However, IBC uses reference pixels in the current frame instead of using reference pixels in previously coded frames.

In another embodiment, if the current block is an IBC mode, the determination of the reference luma sample is the same as the intra prediction mode.

When the one or more reference luma samples are prediction samples of the current CU or TU, different numbers of samples may be used as described in the above embodiments.

In one embodiment, the above embodiments of intra and inter prediction modes may be combined.

In one embodiment, for intra-prediction mode and CIIP mode, the reference luma sample is an upper left boundary reference sample used to generate the intra predictor, and for inter-prediction modes other than CIIP mode, the reference luma sample is an upper left luma prediction sample.

In one embodiment, for intra prediction mode, the reference luma sample is an upper left boundary reference sample used to generate the intra predictor, and for inter prediction mode, the reference luma sample is an upper left luma prediction sample except for CIIP mode. For CIIP mode, the reference luma sample is the top left luma prediction sample of the inter predictor. In other words, the prediction samples are mixed with the intra-prediction samples prior to use.

In one embodiment, the reference luma sample is the top left, top, or left (first available) boundary reconstructed sample for intra prediction mode and CIIP mode, and the reference luma sample is the top left luma prediction sample for inter prediction mode (except CIIP mode). In other words, the prediction samples are mixed with the intra-prediction samples prior to use.

In another example, only the upper left reconstructed samples are used.

If the reference sample is not available, the scaling factor is set to a default value. In one embodiment, the default value is equal to (1< < PREC), where PREC is a prediction of chroma scaling.

In one embodiment, for intra prediction mode, the reference luma sample is the top left, top, or left (first available) boundary reconstructed sample, and for inter prediction mode (except CIIP mode), the reference luma sample is the top left luma prediction sample. For CIIP mode, the reference luma sample is the top left luma prediction sample of the inter predictor. In other words, the prediction samples are mixed with the intra-prediction samples prior to use.

Mode-constrained and conditionally-prohibited chroma splitting within root blocks

In another embodiment, a root block is determined, and the luminance component of the root block may be further divided into smaller blocks. According to this embodiment, it is determined whether the chroma components of the root block can be further divided according to the prediction mode of the luma block within the same root block.

In previous approaches, three cases of "same mode" definitions are listed below:

the same mode means that all blocks within the root block must be intra prediction mode, or all blocks within the root block must be inter prediction mode, or all blocks within the root block must be IBC mode.

The same mode means that all blocks within the root block must be intra prediction mode or all blocks within the root block must be one of inter prediction mode and IBC prediction mode (inter/IBC mode).

The same mode means that all blocks within the root block must be either intra prediction mode and one of the IBC prediction modes (intra/IBC mode), or all blocks within the root block must be inter prediction mode.

In one embodiment, if all blocks within the current root block are inter prediction mode, inter/IBC mode, and inter prediction mode for case 1, case 2, and case 3, respectively, the division of the chroma components follows the luma block. If all blocks within the current root block are intra prediction mode, and intra/IBC mode for case 1, case 2, and case 3, respectively, the chroma components of this root block cannot be further split, thus resulting in multiple luma blocks corresponding to one chroma block.

In another embodiment, a root block is determined, and the luminance component of the root block may be further divided into smaller blocks. According to this embodiment, it is not possible to further partition the chrominance components of the root block. In this region, the luminance blocks may be the same pattern or may be different patterns.

In one embodiment, chroma residual scaling cannot be applied when the chroma component is not allowed to be further partitioned. In another embodiment, chroma residual scaling may still be applied when the chroma components are not allowed to be further split. The position of the reference luma sample may be different. In an embodiment, the upper left NxM luma prediction samples of the collocated luma block are used. N and M may be 1, 2, 4, 8, 16, 32, 64, and 128. In another embodiment, the reconstructed top boundary K of the current root block is used to reference the samples. In another embodiment, the reconstructed left boundary K reference samples of the current root block are used. K may be 1, 2, 4, 8, 16, 32, 64, and 128. In another embodiment, the reconstructed top left reference sample of the current root block is used.

In another embodiment, chroma residual scaling cannot be applied when the chroma components are not allowed to be further partitioned and the chroma root block is coded in intra mode. In another example, when the chroma component is not allowed to be further split and the chroma root block is coded in IBC mode, chroma residual scaling cannot be applied. In another embodiment, chroma residual scaling may still be applied when the chroma components are not allowed to be further split and the chroma root block is coded in intra mode. In another example, chroma residual scaling may still be applied when the chroma components are not allowed to be further split and the chroma root block is coded in IBC mode. The position of the reference luma sample may be different. In an embodiment, the upper left NxM luma prediction samples of the collocated luma block are used. N and M may be 1, 2, 4, 8, 16, 32, 64, and 128. In another embodiment, the reconstructed top boundary K reference samples of the current root block are used. In another embodiment, the reconstructed left boundary K reference samples of the current root block are used. K may be 1, 2, 4, 8, 16, 32, 64, and 128. In another embodiment, the reconstructed top left reference sample of the current root block is used.

In another embodiment, when the chroma block is in the chroma root block, chroma residual scaling cannot be applied. In another embodiment, when the chroma block is in the chroma root block, chroma residual scaling may still be applied. The position of the reference luma sample may be different. In an embodiment, the upper left NxM luma prediction samples of the collocated luma block are used. N and M may be 1, 2, 4, 8, 16, 32, 64, and 128. In another embodiment, the reconstructed top boundary K reference samples of the current root block are used. In another embodiment, K reference samples of the reconstructed left boundary of the current root block are used. K may be 1, 2, 4, 8, 16, 32, 64, and 128. In another embodiment, the reconstructed top left reference sample of the current root block is used.

The LMCS maps samples in the original domain to the reshaped domain for better data estimation. The mapping curve is approximated by a piecewise linear (PWL) model (approximate). To convert sample values from the original domain to the reshaped domain, a look-up-table (abbreviated LUT) is used. The number of entries (entry number) of the LUT is the same as the input sample dynamic range. For example, if 10 data inputs are used, a 1024 entry LUT is used. If 14 data inputs are used, an 8192 entry LUT is used. In a hardware implementation, the cost of such a LUT is high. Therefore, a piecewise-linear model may be used. The input may be compared to each of the plurality of segments to find a segment to which the input belongs. In each segment, a corresponding output value may be calculated from the characteristics of the segment.

Various methods of an LMCS are disclosed according to embodiments of the invention.

Method 1-PCM mode with LMCS

The LMCS maps samples in the original domain to the reshaped domain for better data estimation. The mapping curve is approximated by a piecewise linear model. The sample values are converted from the original domain to the reshaped domain using a look-up table (LUT). The number of entries of the LUT is the same as the input sample dynamic range. For example, if 10 data inputs are used, a 1024 entry LUT is used. If 14 data inputs are used, an 8192 entry LUT is used.

In one embodiment, when Pulse Code Modulation (PCM) coding is used, LMCS is disabled, which may enable lossless coding. This is because the mapping process may introduce some rounding of the digits or may not accurately map them back to the original values after performing the forward and backward mapping, resulting in a lossy codec. According to one embodiment of the invention, one or more high level syntax of a PCM codec is signaled at the SPS/PPS/APS/slice/group of pictures/block/picture level and is signaled before the LMCS syntax. When it is determined that the tile/tile group/picture/slice/sequence uses PCM coding, syntax elements (reshaping tools or reshaping models) related to LMCS may be skipped, inferred as unused, or constrained as unused (e.g., encoder constraints to prohibit LMCS from being used for PCM coding).

In another embodiment, if PCM coding mode is applied in tile/tile group/slice/picture/sequence level regions, reshaping may still be applied. However, the mapping tables for forward and reverse reshaping should be identity mapping, e.g., input equal to output, or mapping function as a line with slope equal to 1.

In one example, a mapping table may be signaled, but should be an identity mapping table. In another example, no mapping table is signaled. A default identity mapping table is used. The default mapping is a simple identical mapping where the input is equal to the output.

In another embodiment, if CU/PU/TU level PCM coding and/or transform quantization bypass mode is applied, the residual or transformed residual should be coded in the original domain to achieve PCM coding. For example, predictors (e.g., inter mode predictor, intra block copy mode predictor, palette mode predictor) should also be located in the original domain. For intra-prediction or any other prediction mode (e.g., combined inter/intra prediction) that uses neighboring reconstructed samples to generate predictors, the neighboring reconstructed samples are converted to the original domain before generating the predictor. In another example, if a predictor (e.g., an intra mode predictor) is generated in the reshaped domain, the generated predictor is converted into the original domain. In this example, for an inter mode predictor, when using PCM mode, it will not pass through the forward reshaper to become a predictor of the reshaped domain. The residual data is coded in the original domain. The syntax is used to specify the field of reconstructed CU samples. Thus, when applying CU/PU/TU level PCM coding and/or transform quantization bypass mode, if intra prediction is used for prediction, only the neighboring reconstructed samples in the reshaped domain need to be inversely mapped to the original domain.

When a CU within a current frame is coded in lossy coding, forward mapping is needed if the neighboring reconstructed samples are in the original domain. After mapping the neighboring reconstructed samples to the reshaped domain, the reshaped neighboring reconstructed samples will be used to generate intra-predicted samples.

In another embodiment, if the current intra CU is coded in lossy coding, the neighboring reconstructed samples are considered as reshaped samples regardless of the domain to which they belong.

In another embodiment, if CU/PU/TU level PCM coding and/or transform quantization bypass mode is applied, the predictor can still be generated in the reshaped domain, but the reconstructed samples are not reconstructed by inverse mapping (to the original domain). However, the reconstructed sample in the reshaped domain should be the value obtained by PCM of the original sample. For example, for intra prediction or any other prediction mode that uses neighboring reconstructed samples to generate predictors, the neighboring samples need not be converted back to the original domain. The reshaped domain neighbor samples may be used to generate predictors. For inter prediction, the predictor may or may not be transformed by a forward mapping like lossy codec. In another embodiment, backward mapping may still be applied. However, the mapping table of the backward mapping is the same mapping, e.g. a one-to-one mapping with a slope equal to 1 or an output equal to the input.

In another embodiment, if CU/PU/TU level PCM coding and/or transform quantization bypass mode is applied, the forward and backward mapping is disabled or the same mapping (for all prediction modes) is used. In another embodiment, the residual/predictor/reconstructed samples can still be coded in the reshaped domain. However, there are encoder constraints or data stream consistency requirements that when applying PCM mode, the reconstructed samples can be converted into the original domain, and the original domain reconstructed samples should be identical to the input samples.

In one embodiment, if CU/PU/TU level PCM coding and/or transform quantization bypass mode is applied, chroma residual scaling is not applied, or the scaling factor is set to 1, or the scaling factor is limited to a range. For example, the scaling factor should be no greater than 1 or no less than 1. In another embodiment, chroma residual scaling is not applied when applying the transform skip mode. In another embodiment, when the transform skip mode is applied to the chroma component, chroma residual scaling is not applied.

In another embodiment, if CU/PU/TU level PCM coding and/or transform quantization bypass mode is applied, the residual or transformed residual should be coded in the re-reshaped domain, where the output of the mapping table is the same as the input. Thus, the mapping process does not introduce lossy codecs.

In another embodiment, if the CU/PU/TU level PCM mode and/or transform quantization bypass mode is used, the neighboring reconstructed samples are converted to the original domain. The prediction samples for the current block may still be transformed by reshaping. However, the mapping tables for forward and reverse reshaping should be one-to-one mappings, e.g. the output equals the input, or the mapping function corresponds to a line with a slope equal to 1.

Method 2-derivation of inverse scaling factor

In one embodiment, the inverse scaling factor may be derived as follows:

InvScaleCoeff[i]＝

OrgCW*((1<<SCALE_FP_PREC)/lmcsCW[i]).

thus, since the number of possible values for the denominator (e.g., lmcsCW [ i ]) is limited, a lookup table may be used to implement a division of values that are not powers of 2 (e.g., lmcsCW [ i ]). The lookup table contains the values of (1< < SCALE _ FP _ PREC)/lmcscW [ i ].

Method 3-LMCS with default number of codewords

In one embodiment, the number of codewords per data sub in the mapping domain (e.g., lmcsCW [ i ]) can be derived using a default number of codewords instead of using OrgCW, which depends only on the data depth of the input data.

In the proposed method, the variable lmcsCW [ i ] is derived according to the following formula, where i ═ lmcs _ min _ bin _ idx to LmcsMaxBinIdx:

lmcsCW[i]＝default_CW+lmcsDeltaCW[i],

where the default _ CW is derived at the decoder side or signaled by the re-encoder.

In one embodiment, if the default _ CW is derived at the decoder side, it can be derived from lmcs _ min _ bin _ idx and LmscMaxBinIdx. Default _ CW can be adjusted to a value greater than OrgCW if the sum of the number of data children less than lmcs _ min _ bin _ idx and the number of data children greater than LmscsMaxBinIdx is greater than lmcs _ min _ bin _ idx.

For example, if the sum of the number of data children less than lmcs _ min _ bin _ idx and the number of data children greater than LmcsMaxBinIdx is equal to 2, then default _ CW is derived as default _ CW ═ OrgCW + a, where a is a positive integer (e.g., 1, 2, 3 …).

Default _ CW is equal to OrgCW if the sum of the number of data children less than lmcs _ min _ bin _ idx and the number of data children greater than LmscMaxBinIdx is equal to 0.

In one embodiment, if default _ CW is signaled, two syntaxes, default _ delta _ abs _ CW and default _ delta _ sign _ CW _ flag, are signaled before lmcs _ delta _ CW _ minus 1.

The variable default _ delta _ abs _ CW represents the absolute difference between default _ CW and OrgCW, and the variable default _ delta _ sign _ CW _ flag represents that the increment value is positive or negative. The default _ delta _ sign _ CW _ flag is signaled only when the default _ delta _ abs _ CW is greater than 0.

In one embodiment, syntax default _ delta _ CW is signaled before lmcs _ delta _ CW _ prec _ minus1 if default _ CW is signaled.

The variable default _ delta _ CW represents the difference between default _ CW and OrgCW.

Method 4-remodeling Curve update

In one embodiment, the reshaping curve is updated in every frame or every other frame.

Chroma scaling with VPDU constraints

A picture may be divided into several non-overlapping MxN blocks. These MxN non-overlapping blocks, which are units of processing data, are referred to as VPDUs. M and N may be 64, or any predefined or signaled value, or a value related to the maximum transform block size.

In one embodiment, for chroma components, chroma residual scaling reconstructs samples using reference luma outside the current VPDU, e.g., a previously coded VPDU.

In one embodiment, the reference luma sample may be one or more regions. For example, the reference sample is a KxL block outside the current VPDU. K and L may be 2, 4, 8, 16 or 32. In detail, according to the present embodiment, the size of the current VPDU is equal to min (ctbsiyy, 64), and the reference luminance sample numbers of the top and left boundaries are equal to min (ctbsiyy, 64), respectively. The variable CtbSizeY specifies the luminance width and luminance height of the luminance codec tree block.

In another embodiment, the reference reconstructed luma samples are along the VPDU top boundary or the left boundary or both, as shown in fig. 4. The number of reference luminance samples is a power of 2.

In another embodiment, the reference reconstructed luma sample is only one sample value. In one embodiment, the location may be the upper left position of the L-shaped boundary of the current VPDU, such as the TL location in FIG. 5. In another embodiment, the location of the reference sample may be a location above the current VPDU, such as the a location in fig. 5. In another embodiment, the position of the reference sample may be a left position of the current VPDU, such as the L position in FIG. 5.

In another embodiment, the chroma scaling is derived only once in each VPDU, and the scaling factor is derived by the first CU in each VPDU. In detail, the size of the VPDU is equal to Min (CtbSizeY, 64), and the reference luma samples used to derive the chroma scaling factor are the same according to this embodiment for all blocks in the Min (CtbSizeY, 64) multiplied by Min (CtbSizeY, 64) region (i.e., in the same VPDU). The variable CtbSizeY specifies the luminance width and luminance height of the luminance codec tree block.

In another embodiment, a reference Codec Unit (CU) for chroma scaling is derived from the VPDU corresponding to the current block (e.g., chroma scaling is always derived by the first CU in the VPDU even if chroma scaling is not applicable to that CU). In detail, according to this embodiment, the size of the VPDU is equal to Min (CtbSizeY, 64), and the reference CU covers the upper left position of the current VPDU. The reference luma samples include a Min (CtbSizeY, 64) luma reconstructed sample along the top boundary of the reference CU and a Min (CtbSizeY, 64) luma reconstructed sample along the left boundary of the reference CU. In another embodiment, the scaling factor is set to a default value if no chroma scaling is applied to the first CU in the current VPDU. In one embodiment, the default value is equal to (1< < PREC), where PREC is a prediction of chroma scaling.

In another embodiment, the chroma scaling factors are shared in the picture/slice level. In another embodiment, the chroma scaling factor is shared in the APS level. In other words, for each signaled mapping curve, a chroma scaling factor is derived. In one example, the derivation of the chroma scale factor for each reshaping curve is done by averaging the scale factors in all intervals (segments). In another embodiment, the scaling factor is derived by selecting the majority of the scaling factors in all intervals (segments). In another embodiment, the scaling factor is derived by directly dividing the difference between the maximum and minimum luma samples by the difference between the maximum and minimum luma samples in the reshaped domain.

Luma residual with reduced delay

Instead of mapping luma prediction samples, the mapping may be applied to luma residuals only. In other words, the predicted samples of the luma component are in the original domain, and the residual of the luma component will be scaled by the scaling factor. The scaling factor is derived by referencing luma prediction samples at different locations or in different ways. The above method suggested for chroma scaling may also be applied for luma residual scaling.

In another embodiment, the scaling factor is an average of two scaling factors for two consecutive intervals.

In one embodiment, the scaling factor for luma residual scaling and chroma residual scaling is the same.

Signaling chroma scaling factor

Instead of implicitly deriving the chroma scaling factor at the decoder side, embodiments of the present invention signal the chroma scaling factor at the TB, TU, CU, CTU, VPDU, slice level, block level, or APS level.

In one embodiment, one or more chroma scaling factors are signaled in one APS.

In one embodiment, if the chroma scaling factor is signaled at the TU level, and if Cbfs (coded block flag) of Cb and Cr is equal to 0, then the chroma scaling factor is not signaled.

In another embodiment, if the chroma scaling factor is signaled at the TU level, and if the root Cbf is equal to 0, then the chroma scaling factor is not signaled.

In one embodiment, if a chroma scaling factor is signaled at the TB level for a chroma Cb component, and if Cbf of Cb is equal to 0, then no chroma scaling factor is signaled; for the chroma Cr component, if Cbf of Cr is equal to 0, no chroma scaling factor is signaled.

In some embodiments, the video encoder must follow the aforementioned syntax design in order to generate a legal data stream, and the video decoder can correctly decode the data stream only if the parsing process conforms to the aforementioned syntax design. When skipping the syntax in the data stream, the encoder and decoder should set the syntax value to an inferred value to ensure that the encoding and decoding results match.

Fig. 6 illustrates a flow diagram of an exemplary decoding system for deriving one or more chroma residual scaling factors based on neighboring predicted or reconstructed luma samples of collocated luma blocks according to an embodiment of the present invention. The steps shown in the flow diagrams, as well as other subsequent flow diagrams in the present disclosure, can be implemented as program code executable on one or more processors (e.g., one or more CPUs) at the encoder side and/or the decoder side. The steps shown in the flowcharts may also be implemented on hardware basis, e.g. one or more electronic devices or processors arranged to perform the steps in the flowcharts. According to the method, a current chroma residual block is received in step 610. In step 620, one or more chroma residual scaling factors are derived based on neighboring predicted or reconstructed luma samples of a collocated luma block associated with the current chroma residual block, wherein the neighboring predicted or reconstructed luma samples of the collocated luma block correspond to samples from M samples of a collocated luma block top boundary and N samples of a collocated luma block left boundary, where M and N are positive integers. Chroma scaling is then applied to chroma residual samples of the current chroma residual block according to the one or more chroma residual scaling factors in step 630.

Fig. 7 shows a flow diagram of another exemplary decoding system for deriving one or more chroma residual scaling factors based on one or more reconstructed luma samples outside a collocated luma processing data unit, according to an embodiment of the present invention. According to the method, chroma residual data associated with a current chroma processed data unit in a picture is received in step 710, wherein the picture is divided into a plurality of non-overlapping processed data units, and each processed data unit comprises a luma processed data unit and one or more chroma processed data units. In step 720, one or more chroma residual scaling factors are derived based on one or more reconstructed luma samples outside of the collocated luma processed data unit associated with the current chroma processed data unit. In step 730, chroma scaling is applied to chroma residual samples of the current chroma processing data unit according to the one or more chroma residual scaling factors.

Fig. 8 shows a flow diagram of an exemplary codec system in which one or more chroma residual scaling factors are signaled in an APS (adaptive parameter set) level of a video data stream on the encoder side or parsed from the APS level of a video data stream on the decoder side according to an embodiment of the invention. According to the method, a current chroma residual block is received in step 810. In step 820, one or more chroma residual scaling factors are signaled in or parsed out of the APS (adaptive parameter set) level of the video data stream. In step 830, chroma scaling is applied to chroma residual samples of the current chroma residual block.

The illustrated flow chart is intended to illustrate an example of video codec according to the present invention. One skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to practice the invention without departing from the spirit of the invention. In this disclosure, specific syntax and semantics have been used to illustrate examples for implementing embodiments of the present invention. Those skilled in the art can practice the invention by replacing syntax and semantics with equivalent syntax and semantics without departing from the spirit of the invention.

The previous description is presented to enable any person skilled in the art to practice the invention in the context of a particular application and its requirements. Various modifications to the described embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments. Thus, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In the previous detailed description, numerous specific details were set forth in order to provide a thorough understanding of the invention. However, those skilled in the art will appreciate that the present invention may be practiced.

The embodiments of the present invention as described above may be implemented in various hardware, software codecs, or a combination of both. For example, an embodiment of the invention may be program code integrated into one or more electronic circuits in a video compression chip or into video compression software to perform the processes described herein. Embodiments of the invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processes described herein. The invention may also relate to a number of functions performed by a computer processor, digital signal processor, microprocessor, or Field Programmable Gate Array (FPGA). The processors may be configured to perform certain tasks in accordance with the invention by executing machine-readable software code or firmware code that defines the specific methods embodied by the invention. Software code or firmware code may be developed in different programming languages and in different formats or styles. The software code may also be compiled for different target platforms. However, different codec formats, styles and languages of software code, and other means of configuring code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.