阅读说明：本技术 用于帧间预测插值的参考尺寸 (Reference size for inter-prediction interpolation ) 是由 张凯 张莉 刘鸿彬 许继征 王悦 于 2019-11-15 设计创作，主要内容包括：描述了用于实施视频处理技术的技术。在一个示例实施方式中,一种视频处理的方法包括对于尺寸为WxH的视频的当前块和该视频的比特流表示之间的转换,确定维度为(W+N-1)x(H+N-1)的第二块以用于该转换期间的运动补偿,其中第二块是基于维度为(W+N-1-PW)x(H+N-1-PH)的参考块而确定的。N表示滤波器尺寸,并且W、H、N、PW和PH是非负整数。PW和PH不都等于0。该方法还包括基于该确定执行该转换。(Techniques for implementing video processing techniques are described. In one example embodiment, a method of video processing includes, for a transition between a current block of video of size WxH and a bitstream representation of the video, determining a second block of dimension (W + N-1) x (H + N-1) for motion compensation during the transition, wherein the second block is determined based on a reference block of dimension (W + N-1-PW) x (H + N-1-PH). N denotes the filter size and W, H, N, PW and PH are non-negative integers. PW and PH are not both equal to 0. The method also includes performing the conversion based on the determination.)

1.A method of video processing, comprising:

for a conversion between a current block of video of size WxH and a bitstream representation of the video, determining a second block of dimension (W + N-1) x (H + N-1) for motion compensation during the conversion, wherein the second block is determined based on a reference block of dimension (W + N-1-PW) x (H + N-1-PH), wherein N represents a filter size, wherein W, H, N, PW and PH are non-negative integers, and wherein PW and PH are not both equal to 0, and

performing the conversion based on the determination.

2. The method of claim 1, wherein pixels in the second block that are outside of the reference block are determined by repeating one or more boundaries of the reference block.

3. The method of claim 1 or 2, wherein PH-0, and wherein at least a left or right boundary of the reference block is repeated to generate a second block.

4. The method of any one or more of claims 1-3, wherein PW is 0, and wherein at least a top boundary or a bottom boundary of the reference block is repeated to generate a second block.

5. The method according to claim 1 or 2, wherein PW >0 and PH >0, and the second block is generated by:

repeating at least a left boundary or a right boundary of the reference block; and

at least a top boundary or a bottom boundary of the reference block is then repeated.

6. The method according to claim 1 or 2, wherein PW >0 and PH >0, and the second block is generated by:

repeating at least a top boundary or a bottom boundary of the reference block; and

at least the left or right boundary of the reference block is then repeated.

7. The method of any one or more of claims 1 to 6, wherein the left boundary of the reference block is repeated M1 times, and wherein the right boundary of the reference block is repeated (PW-M1) times, M1 being a positive integer.

8. The method of any one or more of claims 1 to 7, wherein the top boundary of the reference block is repeated M2 times, and wherein the bottom boundary of the reference block is repeated (PH-M2) times, M2 being a positive integer.

9. The method of any one or more of claims 1-8, wherein at least one of PW or PH is different for different color components of the current block, a color component comprising at least a luma component or one or more chroma components.

10. The method of any one or more of claims 1-9, wherein at least one of PW or PH is a variable that depends on a size or shape of the current block.

11. The method of any one or more of claims 1 to 9, wherein at least one of PW or PH is a variable that depends on coding characteristics of the current block, the coding characteristics including unidirectional predictive coding or bidirectional predictive coding.

12. The method of claim 1, wherein pixels in the second block that are outside of the reference block are set to a single value.

13. The method of claim 12, wherein the single value is 1< (BD-1), BD being the bit depth of a pixel sample in the reference block.

14. The method according to claim 13, wherein BD is 8 or 10.

15. The method of claim 12, wherein the single value is derived based on pixel samples of a reference block.

16. The method of any one or more of claims 12-15, wherein the single value is signaled in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a slice, a coding tree unit row, a coding tree unit, a coding unit, or a prediction unit.

17. The method of claims 1 to 16, wherein if the current block is affine encoded, padding of pixels in the second block that are located outside of a reference block is disabled.

18. A method of video processing, comprising:

for a transition between a current block of video of size WxH and a bitstream representation of the video, determining a second block of dimension (W + N-1) x (H + N-1) for motion compensation during the transition, wherein W, H is a non-negative integer, and wherein N is a non-negative integer and is based on a filter size,

wherein, during the converting, a refined motion vector is determined based on a multi-point search according to a motion vector refinement operation on the original motion vector, and wherein a pixel long boundary of the reference block is determined by repeating one or more non-boundary pixels; and

performing the conversion based on the determination.

19. The method of claim 18, wherein whether the reference block is applicable for processing of the current block is determined based on a dimension of the current block.

20. The method of any one or more of claims 1-19, wherein interpolating the current block comprises:

interpolating a plurality of sub-blocks of the current block based on the second block, wherein each sub-block has a size W1xH1, and W1, H1 are non-negative integers.

21. The method of claim 20, wherein W1-H1-H8 and PW-PH 0.

22. The method of claim 20 or 21, wherein the second block is determined based entirely on an integer part of a motion vector of at least one of the plurality of sub-blocks.

23. The method of claim 22, wherein the reference block is determined based on the integer part of the motion vector of the upper-left sub-block of the current block if the maximum difference between the integer parts of the motion vectors of all of the plurality of sub-blocks is equal to or less than 1 pixel, and wherein each of a right boundary and a bottom boundary of the reference block is repeated once to obtain the second block.

24. The method of claim 22, wherein if a maximum difference between integer parts of motion vectors of all of the plurality of sub-blocks is equal to or less than 1 pixel, the reference block is determined based on an integer part of a motion vector of a lower-right sub-block of the current block, and each of a left boundary and a top boundary of the reference block is repeated once to obtain the second block.

25. The method of claim 20 or 21, wherein the second block is determined based entirely on the modified motion vector of one of the plurality of sub-blocks.

26. The method of claim 25, wherein if a maximum difference between integer parts of the motion vectors of all of the plurality of sub-blocks is equal to or less than two pixels, the motion vector of the top-left sub-block of the current block is modified by adding an integer-pixel distance to each component to obtain a modified motion vector, wherein the reference block is determined based on the modified motion vector, and wherein each of a left boundary, a right boundary, a top boundary, and a bottom boundary of the reference block is repeated once to obtain the second block.

27. The method of claim 25, wherein if a maximum difference between integer parts of the motion vectors of all of the plurality of sub-blocks is equal to or less than two pixels, the motion vector of a bottom-right sub-block of the current block is modified by subtracting an integer-pixel distance from each component to obtain a modified motion vector, wherein the reference block is determined based on the modified motion vector, and wherein each of a left boundary, a right boundary, a top boundary, and a bottom boundary of the reference block is repeated once to obtain the second block.

28. The method of any one or more of claims 1-27, wherein performing the transformation comprises generating the bitstream representation based on a current block of the video.

29. The method of any one or more of claims 1 to 27, wherein performing the transformation comprises generating a current block of the video from the bitstream representation.

30. A video processing apparatus comprising a processor, wherein the processor is configured to perform the method of one or more of claims 1 to 29.

31. A computer-readable medium having code stored thereon, which when executed by a processor, causes the processor to implement the method of any one or more of claims 1 to 29.

Technical Field

This patent document relates to image and video encoding and decoding.

Background

Digital video occupies the greatest bandwidth usage in the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video usage is expected to continue to grow.

Disclosure of Invention

This document discloses various video processing techniques that may be used by video encoders and decoders during encoding and decoding operations.

In one example aspect, a method of video processing is disclosed. The method comprises the following steps: determining, for a conversion between a current block of a video and a bitstream representation of the video using an affine coding tool, a first motion vector of a sub-block of the current block and a second motion vector that is a representative motion vector of the current block comply with a size constraint. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: an affine model including six parameters is determined for a conversion between a current block of video and a bitstream representation of the video. The affine model inherits from affine coding information of neighboring blocks of the current block. The method also includes performing a transformation based on the affine model.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a conversion between a block of video and a bitstream representation of the video, a determination is made as to whether a bi-predictive coding technique is applicable to the block based on a size of the block having a width W and a height H, where W and H are positive integers. The method also includes performing the conversion in accordance with the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for conversion between a block of video and a bitstream representation of the video, it is determined whether a coding tree partitioning process is applicable to the block based on a size of a sub-block that is a sub-coding unit of the block according to the coding tree partitioning process. The subblocks have a width W and a height H, where W and H are positive integers. The method also includes performing the conversion in accordance with the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video and a bitstream representation of the video, it is determined whether an index of a Bi-prediction with Coding unit level Weight (BCW) Coding mode is derived based on a rule regarding a position of the current block. In the BCW encoding mode, a weight set including a plurality of weights is used to generate a bidirectional predictor of a current block. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video encoded using a Combined Inter and Intra Prediction (CIIP) encoding technique and a bitstream representation of the video, an Intra Prediction mode for the current block is determined independently of Intra Prediction modes of neighboring blocks. CIIP encoding techniques use the inter and intra prediction values to derive a final prediction value for the current block. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video encoded using a Combined Inter and Intra Prediction (CIIP) encoding technique and a bitstream representation of the video, an intra prediction mode for the current block is determined according to a first intra prediction mode for a first neighboring block and a second intra prediction mode for a second neighboring block. The first neighboring block is encoded using an intra prediction encoding technique and the second neighboring block is encoded using a CIIP encoding technique. The first intra prediction mode is given a different priority than the second intra prediction mode. CIIP encoding techniques use the inter and intra prediction values to derive a final prediction value for the current block. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a conversion between a current block of video and a bitstream representation of the video, it is determined whether a Combined Inter and Intra Prediction (CIIP) process is applicable to a color component of the current block based on a size of the current block. CIIP encoding techniques use the inter and intra prediction values to derive a final prediction value for the current block. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video and a bitstream representation of the video, a determination is made whether to apply a Combined Inter and Intra Prediction (CIIP) encoding technique to the current block based on characteristics of the current block. CIIP encoding techniques use the inter and intra prediction values to derive a final prediction value for the current block. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video and a bitstream representation of the video, a determination is made whether to disable an encoding tool for the current block based on whether the current block is encoded using a Combined Inter and Intra Prediction (CIIP) encoding technique. The encoding tool comprises at least one of: Bi-Directional Optical Flow (BDOF), Overlapped Block Motion Compensation (OBMC), or decoder-side motion vector refinement (DMVR). The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a conversion between a block of video and a bitstream representation of the video, a first precision P1 of motion vectors for spatial motion prediction and a second precision P2 of motion vectors for temporal motion prediction are determined. P1 and/or P2 are scores, and neither P1 nor P2 are signaled in the bitstream representation. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a conversion between a block of video and a bitstream representation of the video, a motion vector (MVx, MVy) with a precision (Px, Py) is determined. Px is associated with MVx and Py is associated with MVy. MVx and MVy are stored as integers each having N bits, and MinX ≦ MVx ≦ MaxX and MinY ≦ MVy ≦ MaxY, where MinX, MaxX, MinY, and MaxY are real numbers. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video and a bitstream representation of the video, it is determined whether a shared Merge list is applicable to the current block according to an encoding mode of the current block. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video of size WxH and a bitstream representation of the video, a second block of dimension (W + N-1) x (H + N-1) is determined for motion compensation during the transition. The second block is determined based on the reference block having the dimension (W + N-1-PW) x (H + N-1-PH). N denotes the filter size, W, H, N, PW and PH are non-negative integers. PW and PH are not both equal to 0. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a transition between a current block of video of size WxH and a bitstream representation of the video, a second block of dimension (W + N-1) x (H + N-1) is determined for motion compensation during the transition. W, H is a non-negative integer, and N is a non-negative integer and is based on the filter size. During the conversion, a refined motion vector is determined based on a multi-point search according to a motion vector refinement operation on the original motion vector, and a pixel long boundary of the reference block is determined by repeating one or more non-boundary pixels. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: for a conversion of a block of video encoded using a combined inter-intra prediction (CIIP) encoding technique and a bitstream representation of the video, a prediction value at a location in the block is determined based on a weighted sum of the inter prediction value and the intra prediction value at the location. The weighted sum is based on adding an offset to an initial sum obtained based on the inter prediction value and the intra prediction value, and the offset is added before performing a right shift operation to determine the weighted sum. The method also includes performing the conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method comprises the following steps: the method comprises determining a size constraint between a representative motion vector of a current video block being affine-encoded and a motion vector of a sub-block of the current video block, and performing a conversion between a bit stream representation and pixel values of the current video block or sub-block by using the size constraint.

In another example aspect, another method of video processing is disclosed. The method comprises the following steps: determining one or more sub-blocks of the current video block for the affine-encoded current video block, wherein each sub-block has a size of MxN pixels, wherein M and N are multiples of 2 or 4, conforming motion vectors of the sub-blocks to size constraints, and performing a transformation between a bitstream representation and pixel values of the current video block conditionally on a trigger by using the size constraints.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: the method further includes determining that the current video block satisfies a size condition, and based on the determination, performing a conversion between a bitstream representation and pixel values of the current video block by excluding a bi-predictive coding mode of the current video block.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: the method further includes determining that the current video block satisfies a size condition, and performing a transition between a bitstream representation and pixel values of the current video block based on the determination, wherein the inter-prediction mode is signaled in the bitstream representation according to the size condition.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: determining that the current video block satisfies a size condition, and performing a transition between a bitstream representation and pixel values of the current video block based on the determination, wherein generation of the Merge candidate list during the transition depends on the size condition.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: determining that a sub-coding unit of the current video block satisfies a size condition, and performing a conversion between a bitstream representation and pixel values of the current video block based on the determination, wherein a coding tree partitioning process used to generate the sub-coding unit depends on the size condition.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: determining a weight index for a Generalized Bi-prediction (GBi) process for a current video block based on a location of the current video block, and performing a conversion between the current video block and a bitstream representation thereof using the weight index to implement the GBi process.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: determining that a current video block is encoded as an Intra-Inter Prediction (IIP) encoded block, and performing a conversion between the current video block and its bitstream representation using a simplified rule for determining an Intra Prediction Mode or Most Probable Mode (MPM) for the current video block.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: the method further includes determining that the current video block satisfies a simplification criterion, and performing the conversion by disabling the use of inter-intra prediction mode for the conversion between the current video block and the bitstream representation or by disabling additional encoding tools for the conversion.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: the conversion between the current video block and the bitstream representation of the current video block is performed using a motion vector based encoding process, where (a) during the conversion process, precision P1 is used to store spatial motion predictors and precision P2 is used to store temporal motion predictors, where P1 and P2 are fractions, or (b) precision Px is used to store x motion vectors and precision Py is used to store y motion vectors, where Px and Py are fractions.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: interpolating a small subblock of size W1xH1 in a large subblock of size W2xH2 of a current video block by extracting a (W2+ N-1-PW) (H2+ N-1-PH) block, pixel-filling the extracted block, performing boundary pixel repetition on the pixel-filled block, and obtaining pixel values of the small subblock, wherein W1, W2, H1, H2, and PW and PH are integers, and performing conversion between the current video block and a bitstream representation of the current video block using the interpolated pixel values of the small subblock.

In another example aspect, another method of video processing is disclosed. The method comprises the following steps: during a conversion of a current video block of dimension WxH and a bitstream representation of the current video block, performing a motion compensation operation by fetching (W + N-1-PW) × (W + N-1-PH) reference pixels during the motion compensation operation and filling the reference pixels outside the fetched reference pixels, and performing a conversion between the current video block and the bitstream representation of the current video block using a result of the motion compensation operation, wherein W, H, N, PW and PH are integers.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: determining that bi-prediction or uni-directional prediction of the current video block is not allowed based on the size of the current video block, and based on the determination, performing a conversion between a bitstream representation and pixel values of the current video block by disabling the bi-prediction or uni-directional prediction mode.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: determining that bi-prediction or uni-directional prediction of the current video block is not allowed based on the size of the current video block, and based on the determination, performing a conversion between a bitstream representation and pixel values of the current video block by disabling the bi-prediction or uni-directional prediction mode.

In yet another example aspect, a video encoder apparatus is disclosed. The video encoder comprises a processor configured to implement the above-described method.

In yet another example aspect, a video encoder apparatus is disclosed. The video encoder comprises a processor configured to implement the above-described method.

In yet another example aspect, a computer-readable medium having code stored thereon is disclosed. The code embodies one of the methods described herein in the form of processor executable code.

These and other features are described throughout this document.

Drawings

Fig. 1 shows an example of sub-block based prediction.

FIG. 2A shows a 4-parameter affine model.

FIG. 2B shows a 6-parameter affine model.

Fig. 3 shows an example of an affine motion vector field for each sub-block.

Fig. 4A shows an example of the AF _ MERGE candidate.

Fig. 4B shows another example of the AF _ MERGE candidate.

Fig. 5 shows candidate positions of the affine Merge mode.

Fig. 6 shows an example of constrained sub-block motion vectors of Coding Units (CUs) in affine mode.

Fig. 7A shows an example of dividing a CU into 135 degree portions of two triangle prediction units.

Fig. 7B shows an example of a 45-degree partition mode that partitions a CU into two triangle prediction units.

Fig. 8 shows an example of the positions of neighboring blocks.

Fig. 9 shows an example of repeated boundary pixels of a reference block before interpolation.

Fig. 10 shows an example of Coding Tree Unit (CTU) and CTU (region) lines. Shaded CTUs (regions) are on one CUT (region) line and unshaded CTUs (regions) are on the other CUT (region) line.

Fig. 11 is a block diagram of an example of a hardware platform for implementing a video decoder or video encoder apparatus described herein.

FIG. 12 is a flow diagram of an example method of video processing.

Fig. 13 shows an example of a motion vector difference MVD (0,1) mirrored between list 0 and list 1 in the DMVR.

Fig. 14 shows an example MV that can be examined in one iteration.

Fig. 15 shows the required reference samples and boundaries filled in for the calculation.

FIG. 16 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.

Fig. 17 is a flowchart representation of a method for video processing according to the present disclosure.

Fig. 18 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 19 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 20 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 21 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 22 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 23 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 24 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 25 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 26 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 27 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 28 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 29 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 30 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 31 is a flowchart representation of another method for video processing according to the present disclosure.

Fig. 32 is a flowchart representation of yet another method for video processing according to the present disclosure.

Detailed Description

For ease of understanding, section headings are used in this document and do not limit the applicability of the techniques and embodiments disclosed in each section to that section only.

1. Abstract

This patent document relates to video/image coding techniques. And in particular to reducing the bandwidth and line buffers of several coding tools in video/image coding. It can be applied to existing video coding standards (such as HEVC) or to standards to be finalized (multi-functional video coding). It can also be applied to future video/image coding standards or video/image codecs.

2. Background of the invention

Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. ITU-T makes H.261 and H.263, ISO/IEC makes MPEG-1 and MPEG-4 visualizations, and these two organizations jointly make the H.262/MPEG-2 Video and the H.264/MPEG-4 Advanced Video Coding (AVC) and the H.265/HEVC standards. Since h.262, video coding standards have been based on hybrid video coding structures, in which temporal prediction plus transform coding is utilized. In order to explore future Video coding techniques other than HEVC, VCEG and MPEG united in 2015 to form Joint Video Exploration Team (jmet). Since then, jfet has adopted many new methods and placed them into a reference software named Joint Exploration Model (JEM). In month 4 of 2018, the Joint Video Expert Team (jviet) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11(MPEG) holds in an effort to the VVC standard, with a 50% reduction in bitrate compared to HEVC.

2.1HEVC/VVC inter prediction

Interpolation filter

In HEVC, luma subsamples are generated by an 8-tap interpolation filter and chroma subsamples are generated by a 4-tap interpolation filter.

The filter is separable in two dimensions. The spots were filtered horizontally and then vertically.

2.2 sub-block based prediction techniques

Sub-block based prediction was first introduced into the video coding standard by HEVC annex I (3D-HEVC). With sub-block based Prediction, a block such as a Coding Unit (CU) or Prediction Unit (PU) is divided into several non-overlapping sub-blocks. Different sub-blocks may be assigned different Motion information, such as a reference index or a Motion Vector (MV), and Motion Compensation (MC) is performed separately for each sub-block. Fig. 1 illustrates the concept of sub-block based prediction.

To explore future video coding techniques other than HEVC, VCEG and MPEG united in 2015 into a joint video exploration team (jfet). Since then, jfet took many new approaches and placed them into a reference software named Joint Exploration Model (JEM).

In JEM, sub-block based Prediction is used for a variety of coding tools, such as affine Prediction, optional Temporal Motion Vector Prediction (ATMVP), Spatial-Temporal Motion Vector Prediction (STMVP), Bi-directional optical flow (BIO), and Frame-Rate Up Conversion (FRUC). Affine prediction is also employed into VVC.

2.3 affine prediction

In HEVC, only the translational Motion model is applied to Motion Compensated Prediction (MCP). In the real world, there are many kinds of movements such as zoom in/out, rotation, perspective movement, and other irregular movements. In VVC, a simplified affine transform motion compensated prediction is applied. As shown in fig. 2A and 2B, the affine motion field of a block is described by two (in a 4-parameter affine model) or three (in a 6-parameter affine model) control point motion vectors.

The Motion Vector Field (MVF) of the block is described by the following equations using a 4-parameter affine model in equation (1) (in which 4 parameters are defined as variables a, b, e, and f) and a 6-parameter affine model in equation (2) (in which 4 parameters are defined as variables a, b, c, d, e, and f), respectively:

wherein (mv)^h ₀,mv^h ₀) Is the motion vector of the Control Point (CP) at the top left corner, (mv)^h ₁,mv^h ₁) Is the motion vector of the upper right corner control point, and (mv)^h ₂,mv^h ₂) Is the lower left corner controlThe motion vector of the dotting, all three of which are called Control Point Motion Vectors (CPMV), (x, y) represents the coordinates of the representative point with respect to the top left sample point within the current block. The CP motion vectors can be signaled (like in affine AMVP mode) or derived on the fly (like in affine Merge mode). w and h are the width and height of the current block. In practice, division is performed by right-shifting and rounding operations. In the VTM, a representative point is defined as the center position of the subblock, for example, when the coordinates of the upper left corner of the subblock with respect to the upper left sampling point within the current block are (xs, ys), the coordinates of the representative point are defined as (xs +2, ys + 2).

In a division-free design, equations (1) and (2) are implemented as:

for the 4-parameter affine model shown in (1):

for the 6-parameter affine model shown in (2):

in the end of this process,

Off＝1＜＜(S-1)

where S represents the calculation accuracy. For example, in VVC, S ═ 7. In VVC, the MV in MC for the subblock with the top left sample point at (xs, ys) is calculated by equation (6), where x ═ xs +2, and y ═ ys + 2.

To derive the motion vector for each 4 × 4 sub-block, the motion vector for the center sample point of each sub-block is calculated according to equation (1) or (2) and rounded to a fractional accuracy of 1/16, as shown in fig. 3.

Affine models can inherit from spatially adjacent affine coding blocks (such as left, top right, bottom left, and top left adjacent blocks) as shown in fig. 4A. For example, if the neighborhood lower left block a in fig. 4A is encoded in affine mode, as represented by a0 in fig. 4B, the Control Point (CP) motion vector mv containing the upper left, upper right and lower left corners of the neighboring CU/PU of block a is extracted₀ ^N、mv₁ ^NAnd mv₂ ^N. And is based on mv₀ ^N、mv₁ ^NAnd mv₂ ^NCalculating the motion vector mv at the top left/top right/bottom left on the current CU/PU₀ ^C、mv₁ ^CAnd mv₂ ^C(for 6-parameter affine model only) it should be noted that in VTM-2.0, if the current block is affine encoded, the sub-block (e.g., 4 × 4 block in VTM) L T stores MV0 and RT stores MV1, if the current block is encoded with 6-parameter affine model, L B stores MV2, otherwise (with 4-parameter affine model), L B stores MV 2'. the other sub-blocks store MVs for MC.

It should be noted that when a CU is encoded with affine Merge mode (e.g., in AF _ Merge mode), it gets the first block encoded with affine mode from the valid neighborhood reconstructed block. And the selection order of the candidate blocks is from left, upper right, lower left, to upper left as shown in fig. 4A.

In affine Merge mode, the derived CP MV MV of the current block may be compared to the current block₀ ^C、mv₁ ^CAnd mv₂ ^CUsed as the CP MV. Or they may be used as MVPs for affine inter mode in VVC. It should be noted that for Merge mode, if the current block is encoded with affine mode, after deriving the CP MV of the current block, the current block may be further divided into a plurality of sub-blocks, and each block will be based onThe derived CP MV of the current block derives its motion information.

2.4 example embodiment in JFET

Unlike VTMs where only one affine spatial neighboring block can be used to derive affine motion of a block. In some embodiments, a separate affine candidate list is constructed for AF _ MERGE mode.

1) Inserting inherited affine candidates into a candidate list

Inherited affine candidates refer to candidates derived from valid neighborhood reconstruction blocks encoded with affine mode. As shown in FIG. 5, the scan order of the candidate blocks is A₁、B₁、B₀、A₀And B₂. When a block is selected (e.g. A)₁) When, two steps of process are applied:

a first, the three angular motion vectors of the CU covering the block are used to derive the two/three control points of the current block.

Deriving a sub-block motion for each sub-block within the current block based on the control point for the current block.

2) Insertion-built affine candidates

And if the candidate number in the affine Merge candidate list is less than MaxMumAffinic, inserting the constructed affine candidate into the candidate list.

The constructed affine candidates mean that the candidates are constructed by combining the neighborhood motion information of each control point.

The motion information of the control points is first derived from the specified spatial and temporal neighborhood shown in fig. 5. CPk (k ═ 1, 2,3, 4) denotes the kth control point. A. the₀、A₁、A₂、B₀、B₁、B₂And B₃Is the spatial position used to predict CPk (k 1, 2, 3); t is the temporal location used to predict CP 4.

The coordinates of the CP1, CP2, CP3, and CP4 are (0,0), (W,0), (H,0), and (W, H), respectively, where W and H are the width and height of the current block.

The motion information of each control point is obtained according to the following priority order:

a for CP1, check priorityStage B₂→B₃→A₂. If B is present₂Can be used, then B is used₂. Otherwise, if B₂Can be used, then B is used₃. If B is present₂And B₃Are all unusable, use A₂. If all three candidates are not available, no motion information for CP1 can be obtained.

B for CP2, check priority B1 → B0;

c for CP3, check priority A1 → A0;

2.d for CP4, T is used.

Second, a motion model is constructed using a combination of control points.

The motion vectors of the three control points are needed to calculate the transformation parameters in the 6-parameter affine model. The three control points may be selected from one of the following four combinations ({ CP1, CP2, CP4}, { CP1, CP2, CP3}, { CP2, CP3, CP4}, { CP1, CP3, CP4 }). For example, a 6-parameter Affine motion model, denoted Affine (CP1, CP2, CP3), is constructed using CP1, CP2, and CP3 control points.

The motion vectors of the two control points are needed to calculate the transformation parameters in the 4-parameter affine model. Two control points may be selected from one of the following six combinations ({ CP1, CP4}, { CP2, CP3}, { CP1, CP2}, { CP2, CP4}, { CP1, CP3}, { CP3, CP4 }). For example, CP1 and CP2 control points are used to construct a 4-parameter Affine motion model, denoted Affine (CP1, CP 2).

The combination of the constructed affine candidates is inserted into the candidate list in the following order: { CP1, CP2, CP3}, { CP1, CP2, CP4}, { CP1, CP3, CP4}, { CP2, CP3, CP4}, { CP1, CP2}, { CP1, CP3}, { CP2, CP3}, { CP1, CP4}, { CP2, CP4}, { CP3, CP4}

3) Inserting zero motion vectors

If the number of candidates in the affine Merge candidate list is less than MaxumAffinic, then zero motion vectors are inserted into the candidate list until the list is full.

2.5 affine Merge candidate list

2.5.1 affine Merge mode

In JFET-L0366, a candidate list of affine Merge patterns is constructed by searching for valid affine neighbors and combining the neighborhood motion information for each control point.

The affine Merge candidate list is constructed as follows:

1) inserting inherited affine candidates

Inherited affine candidates mean that the candidates are derived from affine motion models of their valid neighborhood affine coding blocks. In the common basis, as shown in fig. 5, the scan order of the candidate locations is: a1, B1, B0, a0 and B2.

After deriving the candidates, a complete pruning process is performed to check if the same candidates have been inserted into the list. If the same candidate exists, the derived candidate is discarded.

2) Insertion-built affine candidates

If the number of candidates in the affine Merge candidate list is less than MaxmumAffinic and (set to 5 in this contribution), the constructed affine candidate is inserted into the candidate list. The constructed affine candidates mean that the candidates are constructed by combining the neighborhood motion information of each control point.

The motion information of the control points is first derived from the specified spatial and temporal neighborhoods. CPk (k ═ 1, 2,3, 4) denotes the kth control point. A0, a1, a2, B0, B1, B2, and B3 are spatial positions for predicting CPk (k is 1, 2, 3); t is the temporal location used to predict CP 4.

The coordinates of the CP1, CP2, CP3, and CP4 are (0,0), (W,0), (H,0), and (W, H), respectively, where W and H are the width and height of the current block.

The motion information of each control point is obtained according to the following priority order:

for CP1, the checking priority is B₂→B₃→A₂. If B is present₂Can be used, then B is used₂. Otherwise, if B₂Can be used, then B is used₃. If B is present₂And B₃Are all unusable, use A₂. If all three candidates are not available, no motion information for CP1 can be obtained.

For CP2, the check priority is B1 → B0;

for CP3, the check priority is A1 → A0;

for CP4, T is used.

Next, affine Merge candidates are constructed using combinations of control points.

Motion information of three control points is required to construct a 6-parameter affine candidate. The three control points may be selected from one of the following four combinations ({ CP1, CP2, CP4}, { CP1, CP2, CP3}, { CP2, CP3, CP4}, { CP1, CP3, CP4 }). The combinations CP1, CP2, CP3, CP2, CP3, CP4, CP1, CP3, CP4 will be converted into a 6-parameter motion model represented by upper-left, upper-right and lower-left control points.

Motion information of two control points is required to construct a 4-parameter affine candidate. Two control points may be selected from one of the following six combinations ({ CP1, CP4}, { CP2, CP3}, { CP1, CP2}, { CP2, CP4}, { CP1, CP3}, { CP3, CP4 }). The combinations CP1, CP4, CP2, CP3, CP2, CP4, CP1, CP3, CP3, CP4 will be converted into a 4-parameter motion model represented by the upper left and upper right control points.

The combination of constructed affine candidates is inserted into the candidate list in the following order:

{CP1,CP2,CP3}、{CP1,CP2,CP4}、{CP1,CP3,CP4}、{CP2,CP3,CP4}、{CP1,CP2}、{CP1,CP3}、{CP2,CP3}、{CP1,CP4}、{CP2,CP4}、{CP3,CP4}

for a combined reference list X (X is 0 or 1), the reference index with the highest usage in the control points is selected as the reference index for list X, and the motion vectors pointing to different reference pictures will be scaled.

After deriving the candidates, a complete pruning procedure will be performed to check if the same candidates have been inserted into the list. If the same candidate exists, the derived candidate is discarded.

3) Filling with zero motion vectors

If the number of candidates in the affine Merge candidate list is less than 5, a zero motion vector with a zero reference index is inserted into the candidate list until the list is full.

2.5.2 example affine Merge modes

In some embodiments, the affine Merge mode may be simplified as follows:

1) by comparing the coding units covering neighboring positions, instead of comparing the derived affine candidates in VTM-2.0.1, the pruning process of inherited affine candidates is simplified. Up to 2 inherited affine candidates are inserted into the affine Merge list. The pruning process of the constructed affine candidates is completely removed.

2) The MV scaling operations are removed in the constructed affine candidates. If the reference indices of the control points are different, the constructed motion model is discarded.

3) The number of constructed affine candidates is reduced from 10 to 6.

4) In some embodiments, other Merge candidates predicted using sub-blocks such as ATMVP are also placed in the affine Merge candidate list. In this case, the affine Merge candidate list may be renamed with some other name such as the sub-block Merge candidate list.

2.6 control Point MV offset for example affine Merge mode

Generating a new affine Merge candidate based on the CPMV offset of the first affine Merge candidate. If the first affine Merge candidate enables the 4-parameter affine model, deriving 2 CPMVs for each new affine Merge candidate by offsetting the 2 CPMVs of the first affine Merge candidate; otherwise (6-parameter affine model enabled), 3 CPMVs are derived for each new affine Merge candidate by shifting the 3 CPMVs of the first affine Merge candidate. In uni-directional prediction, the CPMV offset is applied to the CPMV of the first candidate. In bi-directional prediction with list 0 and list 1 in the same direction, the CPMV offset is applied to the first candidate as follows:

MV_new(L0),i＝MV_old(L0)+MV_offset(i)equation (8)

MV_new(L1),i＝MV_old(L1)+MV_offset(i)Equation (9)

In bi-directional prediction with list 0 and list 1 in opposite directions, the CPMV offset is applied to the first candidate as follows:

MV_new(L0),i＝MV_old(L0)+MV_offset(i)equation (10)

MV_new(L1),i＝MV_old(L1)-MV_offset(i)Equation (11)

Various offset directions with different offset magnitudes may be used to generate new affine Merge candidates.

Two embodiments have been tested:

(1) generate 16 new affine Merge candidates with 8 different offset directions and 2 different offset magnitudes, as shown in the following offset set:

the offset set is { (4,0), (0,4), (-4,0), (0, -4), (-4, -4), (4, -4), (4,4), (-4,4), (8,0), (0,8), (-8,0), (0, -8), (-8, -8), (8,8), (-8,8) }.

For this design, the affine Merge list is increased to 20. The number of potential affine Merge candidates is 31 in total.

(2) Generate 4 new affine Merge candidates with 4 different offset directions and 1 offset magnitude, as shown in the following offset set:

the offset set is { (4,0), (0,4), (-4,0), (0, -4) }.

The affine Merge list remains 5 as VTM2.0.1 does. The four time-domain constructed affine Merge candidates are removed to keep the number of potential affine Merge candidates unchanged, e.g., 15 in total. Let the coordinates of CPMV1, CPMV2, CPMV3, and CPMV4 be (0,0), (W,0), (H,0), and (W, H). Note that CPMV4 is derived from the time domain MV as shown in fig. 6. The removed candidates are the following four time-domain correlated constructed affine Merge candidates: { CP2, CP3, CP4}, { CP1, CP4}, { CP2, CP4}, and { CP3, CP4 }.

2.7 Bandwidth problem for affine motion compensation

Since the current block is divided into a 4 × 4 sub-block for the luma component and a2 × 2 sub-block for the two chroma components for motion compensation, the total bandwidth requirement is much higher than the non-sub-block inter prediction.

2.7.1 example 1

The 4x4 block serves as the sub-block size of the uni-predictive affine coded CU, while the 8x4/4x8 block serves as the sub-block size of the bi-predictive affine coded CU.

2.7.2 example 2

For affine mode, the sub-block motion vectors of affine CUs are constrained within a predefined motion vector field. Suppose the motion vector of the first (upper left) sub-block is (v)_0x,v_0y) And the second sub-block is (v)_ix,v_iy)，v_ixAnd v_iyThe values of (a) show the following constraints:

v_ix∈[v_0x-H,v_0x+H]equation (12)

v_iy∈[v_0y-V,v_0y+V]Equation (13)

If the motion vector of any sub-block exceeds the predefined motion vector field, the motion vector is clipped. Fig. 6 gives an illustration of the idea of constrained sub-block motion vectors.

Assuming that the memory is retrieved per CU rather than per sub-block, the values H and V are chosen such that, in the worst case, the memory bandwidth of an affine CU does not exceed the memory bandwidth of the normal inter MC of the 8 × 8 bi-prediction block.

2.7.3 example 3

To reduce memory bandwidth requirements in affine prediction, each 8x8 block within a block is treated as a base unit. The MVs of all four 4x4 sub-blocks inside the 8x8 block are constrained such that the maximum difference between the integer part of the four 4x4 sub-blocks MVs does not exceed 1 pixel. The bandwidth is thus (8+7+1) × (8+7+1)/(8 × 8) ═ 4 spots/pixel.

In some cases, after calculating MVs of all sub-blocks inside the current block with the affine model, the MVs of the sub-blocks containing the control points are first replaced with the corresponding control points MVs. This means that MVs of the upper left, upper right and lower left sub-blocks are replaced by upper left, upper right and lower left control points MV, respectively. Then, for each 8x8 block in the current block, the MVs of all four 4x4 sub-blocks are clipped to ensure that the maximum difference between the integer parts of the four MVs does not exceed 1 pixel. It should be noted here that the sub-blocks containing control points (top left, top right and bottom left sub-blocks) use the corresponding control points MVs to participate in the MV cropping process. The MV at the upper right control point remains unchanged during the cropping process.

The clipping process applied to each 8x8 block is described as follows:

1. the maximum and minimum values MVminx, MVminy, MVmaxx, MVmaxy of the MV components are first determined for each 8 × 8 block, as follows:

a) obtaining a minimum MV component among four 4x4 subblock MVs

MVminx＝min(MVx0,MVx1,MVx2,MVx3)

MVminy＝min(MVy0,MVy1,MVy2,MVy3)

b) Using the integer part of MVminx and MVminy as the minimum MV component

MVminx＝MVminx>>MV_precision<<MV_precision

MVminy＝MVminy>>MV_precision<<MV_precision

c) The maximum MV component is calculated as follows:

MVmaxx＝MVminx+(2<<MV_precision)–1

MVmaxy＝MVminy+(2<<MV_precision)–1

d) if the upper right control point is in the current 8x8 block

If (MV1x > MVmaxx)

MVminx＝(MV1x>>MV_precision<<MV_precision)–(1<<MV_precision)

MVmaxx＝MVminx+(2<<MV_precision)–1

If (MV1y > MVmaxy)

MVminy＝(MV1y>>MV_precision<<MV_precision)–(1<<MV_precision)

MVmaxy＝MVminy+(2<<MV_precision)–1

2. The MV components of each 4x4 block within the 8x8 block are clipped as follows:

MVxi＝max(MVminx,min(MVmaxx,MVxi))

MVyi＝max(MVminy,min(MVmaxy,MVyi))

where (MVxi, MVyi) is the MV of the ith sub-block within an 8x8 block, where i is 0,1, 2, 3; (MV1x, MV1y) is the MV for the upper right control point; MV _ precision equals 4, corresponding to motion vector fractional precision of 1/16. Since the difference between the integer parts of MVminx and MVmaxx (MVminy and MVmaxy) is 1 pixel, the maximum difference between the integer parts of the four 4x4 subblocks MV does not exceed 1 pixel.

In some embodiments, a similar approach may also be used to handle planar patterns.

2.7.4 example 4

In some embodiments, a limitation of the affine mode for worst case bandwidth reduction. To ensure that the worst-case bandwidth of an affine block is not worse than the INTER-4 x 8/INTER-8 x4 block or even the INTER-9 x9 block, the motion vector difference between affine control points is used to decide whether the sub-block size of the affine block is 4x4 or 8x 8.

General affine limitation of worst case bandwidth reduction

Memory bandwidth reduction for affine mode is controlled by limiting the motion vector difference between affine control points (also called control point difference). Generally, if the control point difference satisfies the following constraint, the affine motion uses a 4x4 sub-block (i.e., a 4x4 affine pattern). Otherwise, it will use the 8x8 sub-block (8x8 affine pattern). The constraints of the 6-parameter and 4-parameter models are given as follows.

To derive the constraints of different block sizes (wxh), the motion vector differences of the control points are normalized to:

Norm(v_2x-v_0x)＝(v_2x-v_0x) 128/h equation (14)

In the 4-parameter affine model, (v)_2x-v_0x) And (v)_2y-v_0y) The settings were as follows:

(v_2x-v_0x)＝-(v_1y-v_0y)

(v_2y-v_0y)＝-(v_1x-v_0x) Equation (15)

Thus, (v)_2x-v_0x) And (v)_2y-v_0y) The specification of (a) is given as follows:

Norm(v_2x-v_0x)＝-Norm(v_1y-v_0y)

Norm(v_2y-v_0y)＝Norm(v_1x-v_0x) Equation (16)

The worst case bandwidth is guaranteed to reach the limit of INTER _4x8 or INTER _8x 4:

|Norm(v_1x-v_0x)+Norm(v_2x-v_0x)+128|+|Norm(v_1y-v_0y)+Norm(v_2y-v_0y)+128|+|Norm(v_1x-v_0x)-Norm(v_2x-v_0x)|+|Norm(v_1y-v_0y)-Norm(v_2y-v_0y) Equation (17) | < 128 × 3.25

Where the left hand side of equation (18) represents the shrinkage or span level of the sub-affine block and (3.25) represents a 3.25 pixel shift.

Guarantee worst case bandwidth up to INTER 9x9 limit

(4*Norm(v_1x-v_0x)＞-4*pel&&+4*Norm(v_1x-v_0x)pel)&&

(4*Norm(v_1y-v_0y)＞-pel&&4*Norm(v_1y-v_0y)＜pel)&&

(4*Norm(v_2x-v_0x)＞-pel&&4*Norm(v_2x-v_0x)pel)&&

(4*Norm(v_2y-v_0y)＞-4*pel&&4*Norm(v_2y-v_0y)pel)&&

((4*Norm(v_1x-v_0x)+4*Norm(v_2x-v_0x)＞-4*pel)&&

(4*Norm(v_1x-v_0x)+4*Norm(v_2x-v_0x)＜pel))&&

((4*Norm(v_1y-v_0y)+4*Norm(v_2y-v_0y)＞-4*pel)&&

(4*Norm(v_1y-v_0y)+4*Norm(v_2y-v_0y) < pel)) equation (18)

Where pel is 128 x16 (128 and 16 represent normalization factor and motion vector precision, respectively).

2.8 generalized Bi-prediction improvement

Some embodiments improve the gain complexity tradeoff for GBi and are employed in BMS2.1 GBi is also known as bi-prediction with CU-level weights (BCW) BMS2.1 GBi applies unequal weights to predicted values from L0 and L1 in bi-prediction mode in inter-prediction mode, multiple weight pairs including equal weight pairs (1/2 ) are evaluated based on rate-distortion optimization (RDO) and GBi indices of selected weight pairs are signaled to the decoder in Merge mode GBi indices are inherited from neighboring CUs in BMS2.1 i predicted values in bi-prediction mode are generated as shown in equation (19).

P_GBi＝(w₀*P_L0+w₁*P_L1+RoundingOffset_GBi)>>shiftNum_GBiEquation (19)

Wherein P is_GBiIs the final predictor of GBi. w is a₀And w₁Is the selected GBi weight pair and is applied to the predicted values for List 0 (L0) and List 1 (L1), respectively_GBiAnd shiftNum_GBiFor normalizing the final predicted values in GBi. Supported w₁The set of weights is { -1/4,3/8,1/2,5/8,5/4}, where five weights correspond to one equal weight pair and four unequal weight pairs. Hybrid gain, e.g. w₁And w₀The sum was fixed to 1.0. Thus, corresponding w₀The set of weights is {5/4,5/8,1/2,3/8, -1/4 }. The weight pair selection is at the CU level.

For non-low delay pictures, the weight set size is reduced from five to three, where w₁The set of weights is {3/8,1/2,5/8}, and w₀The set of weights is {5/8,1/2,3/8 }. The weight set size reduction for non-low delay pictures applies to BMS2.1 GBi and all GBi tests in this contribution.

In some embodiments, the following modifications are applied on the basis of the existing GBi design in BMS2.1 to further improve GBi performance.

2.8.1GBi encoder error repair

To reduce GBi encoding time, in current encoder designs, the encoder will store the uni-directional prediction motion vectors estimated from GBi weights equal to 4/8 and reuse them for uni-directional prediction search of other GBi weights. This fast encoding method is applied to both translational and affine motion models. In VTM2.0, a 6-parameter affine model and a 4-parameter affine model are employed. When the GBi weight is equal to 4/8, the BMS2.1 encoder does not distinguish between the 4-parameter affine model and the 6-parameter affine model when storing the uni-directional predictive affine MV. Thus, after encoding with GBi weights 4/8, a 4-parameter affine MV may be overwritten by a 6-parameter affine MV. For other GBi weights, the stored 6-parameter affine MV may be used for the 4-parameter affine ME, or the stored 4-parameter affine MV may be used for the 6-parameter affine ME. The proposed GBi encoder error repair is to separate the 4-parameter and 6-parameter affine MV storage. When the GBi weights are equal to 4/8, the encoder stores those affine MVs based on the affine model type and, for other GBi weights, reuses the corresponding affine MVs based on the affine model type.

CU size constraints of 2.8.2GBi

In this approach, GBi is disabled for small CUs. In inter prediction mode, if bi-prediction is used and the CU area is less than 128 luma samples, GBi is disabled without any signaling.

2.8.3 Merge mode with GBi

In Merge mode, the GBi index is not signaled. Instead, it is inherited from the neighboring block it is merged into. When a TMVP candidate is selected, then GBi is closed in the block.

2.8.4 affine prediction with GBi

GBi may be used when the current block is encoded with affine prediction. For affine inter mode, the GBi index is signaled. For affine Merge mode, the GBi index is inherited from the neighboring block it merges into. If the constructed affine model is selected, GBi is closed in this block.

2.9 example inter-Intra prediction modes (IIP)

With inter-intra prediction mode (also known as Combined Inter and Intra Prediction (CIIP)), multi-hypothesis prediction combines one intra prediction and one Merge-indexed prediction. Such a block is considered a special inter-coded block. In the Merge CU, when the flag is true, a flag is signaled to the Merge mode to select the intra mode from the intra candidate list. For the luminance component, the intra candidate list is derived from 4 intra prediction modes including DC mode, planar mode, horizontal mode, and vertical mode, and the size of the intra candidate list may be 3 or 4 depending on the shape of the block. When the CU width is greater than twice the CU height, the horizontal mode does not include the intra-mode list, and when the CU height is greater than twice the CU width, the vertical mode is removed from the intra-mode list. The prediction of one intra prediction mode selected by the intra mode index and one Mege index selected by the Mege index is combined using a weighted average. For the chroma component, DM is always applied without additional signaling.

The weights of the combined predictions are described below. Equal weights are applied when either DC mode or planar mode is selected or CB width or height is less than 4. For those CBs having a CB width and height greater than or equal to 4, when the horizontal/vertical mode is selected, one CB is first vertically/horizontally divided into four equal-area regions. Each weight set (denoted as (w _ intra)_i,w_inter_i) Wherein i is 1 to 4, and (w _ intra)₁,w_inter₁)＝(6,2)，(w_intra₂,w_inter₂)＝(5,3)，(w_intra₃,w_inter₃) (3,5), and (w _ intra)₄,w_inter₄) (2,6)) will be applied to the corresponding region. (w _ intra)₁,w_inter₁) For the region closest to the reference sample point, and (w _ intra)₄,w_inter₄) For the region furthest from the reference sample point. Then, the two weighted predictions can be addedAnd shifted right by 3 bits to compute a combined prediction. Further, intra-prediction modes for intra-hypotheses of predicted values may be saved for reference by subsequent neighboring CUs.

Assume intra and inter prediction values are PIntra and Pinter and weighting factors are w _ intra and w _ inter, respectively. The predicted value at location (x, y) is calculated as (PIntra (x, y) × w _ intra (x, y) + pin (x, y) × w _ inter (x, y)) > > N, where w _ intra (x, y) + w _ inter (x, y) ═ 2^ N.

Signaling of intra prediction mode in IIP coding blocks

When inter-intra mode is used, one of the four allowed intra prediction modes (DC mode, planar mode, horizontal mode and vertical mode) is selected and signaled. The three Most Probable Modes (MPMs) are made up of left and top neighboring blocks. Intra prediction modes of intra-coded neighboring blocks or IIP-coded neighboring blocks are regarded as one MPM. If the intra-prediction mode is not one of the four allowed intra-prediction modes, it will be rounded to the vertical mode or the horizontal mode according to the angle difference. The neighboring blocks must be on the same CTU line as the current block.

It is assumed that the width and height of the current block are width and height. If W >2 × H or H >2 × W, only one of the three MPMs can be used in the inter-intra mode. Otherwise, all four valid intra prediction modes can be used for inter-intra mode.

It should be noted that the intra prediction mode in inter-intra mode cannot be used to predict the intra prediction mode in normal intra coded blocks.

Inter-intra prediction can only be used when W × H > -64.

2.10 example triangle prediction modes

The concept of Triangular Prediction Mode (TPM) is to introduce a new triangle partition for motion compensated prediction. As shown in fig. 7A-7B, it divides the CU into two triangular prediction units in the diagonal or anti-diagonal direction. Each triangle prediction unit in the CU is inter predicted using its own uni-directional prediction motion vector and reference frame index derived from the uni-directional prediction candidate list. After the triangle prediction unit is predicted, adaptive weighting processing is performed on diagonal edges. The transform and quantization process is then applied to the entire CU. Note that this mode only applies to skip and Merge modes.

2.10.1 unidirectional prediction candidate list for TPM

As shown in FIG. 8, it is derived from seven neighboring blocks including five spatial neighboring blocks (1 to 5) and two temporal collocated blocks (6 to 7). in the order of the uni-directional predicted motion vector, the L0 motion vector of the bi-directional predicted motion vector, the L1 motion vector of the bi-directional predicted motion vector, and the average motion vector of the L0 and L1 motion vectors of the bi-directional predicted motion vector, the motion vectors of the seven neighboring blocks are collected and placed in the uni-directional predicted candidate list.

More specifically, it relates to the following steps:

1) from A₁、B₁、B₀、A₀、B₂Col and Col2 (corresponding to blocks 1-7 in fig. 8) obtain motion candidates without any pruning operation.

2) The variable numcurmergecand is set to 0.

3) For slave A₁、B₁、B₀、A₀、B₂Each motion candidate derived by Col and Col2 is added to the Merge list if the motion candidate is uni-directionally predicted (from list 0 or list 1), where numcurmergergecand is incremented by 1. This added candidate motion is named "original uni-directional predicted candidate". Full pruning is applied.

4) For slave A₁、B₁、B₀、A₀、B₂Each motion candidate derived by Col and Col2, and numcurrmemegared is less than 5, if the motion candidate is bi-predicted, the motion information from list 0 is added to the Merge list (i.e., modified to uni-prediction from list 0), and numcurrmemegared is incremented by 1. Such asThe added candidate motion is named "truncated list 0 predicted candidate". Full pruning is applied.

5) For slave A₁、B₁、B₀、A₀、B₂Each motion candidate derived by Col and Col2, and numcurrmemegared is less than 5, if the motion candidate is bi-predicted, the motion information from list 1 is added to the Merge list (i.e., modified to uni-directional prediction from list 1), and numcurrmemegared is increased by 1. This added candidate motion is named "truncated list 1 predicted candidate". Applications ofComplete pruning。

6) For slave A₁、B₁、B₀、A₀、B₂Each candidate motion derived by Col and Col2, and numcurmergecand is less than 5, if the candidate motion is bi-predicted,

if the Quantization Parameter (QP) of the slice 0 reference picture is smaller than the slice QP of the list 1 reference picture, the motion information of list 1 is first scaled to the list 0 reference picture and the average of two MVs (one from the original list 0 and the other one from the scaled MV of list 1) is added to the Merge list, i.e. the average uni-directional prediction from the list 0 motion candidate and numcurr mergecand is increased by 1.

Otherwise, first scale the motion information of list 0 to list 1 reference pictures and add the average of two MVs (one from the original list 1 and the other the scaled maximum from list 0) to the Merge list, i.e. the average uni-directional prediction from list 1 motion candidates and numcurmergergecand is increased by 1.

Full pruning is applied.

7) If numcurrMergeCand is less than 5, a zero motion vector candidate is added.

Decoder-side motion vector refinement (DMVR) in VVC

For DMVR in VVC, MVD mirroring between list 0 and list 1 is assumed as shown in fig. 13, and bilateral matching is performed to refine the MVs, e.g., find the best MVD among several MVD candidates the MVD represented by (MvdX, MvdY) of the MV L0 (L0X, L0Y) and MV L1 (L1X, L1Y) of the two reference picture lists MV. list 0 is defined as the best MVD for the SAD function defined as the derived between the reference block of list 0 using motion vectors (L0X + MvdX, &lttttranslation = L "&tttl &/ttt gt0Y + MvdY) in the list 0 reference picture and the SAD derived block of list 0 using motion vectors (L X + MvdX, &l &/ttttttttt 1Y + MvdY) in the list 1 reference picture L1.

The motion vector refinement process may iterate twice. In each iteration, up to 6 MVDs (in integer pixel accuracy) can be examined in two steps, as shown in fig. 14. In a first step, the MVD (0,0), (-1,0), (0, -1), (0,1) is examined. In a second step, one of the MVDs (-1, -1), (-1,1), (1, -1), or (1,1) may be selected and further examined. Assume that the function Sad (x, y) returns the Sad value of MVD (x, y). The MVDs examined in the second step (denoted by (MvdX, MvdY)) are determined as follows:

MvdX＝-1；

MvdY＝-1；

if (Sad (1,0) < Sad (-1,0))

MvdX＝1；

If (Sad (0,1) < Sad (0, -1))

MvdY＝1；

In the first iteration, the starting point is the signaled MV, and in the second iteration, the starting point is the signaled MV plus the best MVD selected in the first iteration. DMVR is applied only when one reference picture is a previous picture and the other reference picture is a subsequent picture, and both reference pictures have the same picture order count distance as the current picture.

To further simplify the process of DMVR, the following main features may be implemented in some embodiments:

1. early termination occurs when the (0,0) position SAD between list 0 and list 1 is less than the threshold.

2. Early termination occurs when the SAD between list 0 and list 1 at a certain position is zero.

Block size of DMVR: w H >64 & & H > 8, where W and H are the width and height of the block.

4. For DMVR with CU size >16 x16, the CU is divided into multiples of 16x16 subblocks. If only the width or height of a CU is greater than 16, it is only divided in the vertical or horizontal direction.

5. The reference block size (W +7) × (H +7) (for luminance).

6. Integer pixel search based on 25-point SAD (e.g., (-) 2 refine search range, single level)

7. DMVR based on bilinear interpolation.

8. And (3) sub-pixel refinement based on a parameter error surface equation. This process is only performed if the minimum SAD cost is not equal to zero and the optimal MVD is (0,0) in the last MV refinement iteration.

9. Luma/chroma MC w/reference block padding (if needed).

10. Refined MVs were used for MC and TMVP only.

2.11.1 use of DMVR

The DMVR may be enabled when the following conditions are true:

the DMVR enabled flag (e.g., SPS _ DMVR _ enabled _ flag) in the SPS is equal to 1.

The TPM flag, the interframe affine flag and the subblock Merge flag (ATMVP or affine Merge), the MMVD flag all equal to 0.

The Merge flag is equal to 1.

The current block is bi-directionally predicted, and the Picture Order Count (POC) distance between the current Picture and the reference Picture in list 1 is equal to the POC distance between the reference Picture in list 0 and the current Picture.

-the current CU height is greater than or equal to 8.

-the number of luminance samples (CU width x height) is greater than or equal to 64.

2.11.2 subpixel refinement based on "parametric error surface equation

The method is summarized as follows:

1. the parametric error surface fit is calculated only if the center position is the best cost position in a given iteration.

2. The center position cost and the cost at the (-1,0), (0, -1), (1,0) and (0,1) positions from the center are used to fit a 2-D parabolic error surface equation of the form

E(x,y)＝A(x-x₀)²+B(y-y₀)²+C

Wherein (x)₀,y₀) Corresponding to the position where the cost is the smallest and C corresponds to the smallest cost value. By solving 5 equations of 5 unknowns, (x)₀,y₀) The calculation is as follows:

x₀＝(E(-1,0)-E(1,0))/(2(E(-1,0)+E(1,0)-2E(0,0)))

y₀＝(E(0,-1)-E(0,1))/(2((E(0,-1)+E(0,1)-2E(0,0))

(x₀,y₀) Any desired sub-pixel precision can be calculated by adjusting the precision with which the division is performed (e.g., the number of bits over which the quotient is calculated). For 1/16^thPixel accuracy, only 4 bits in the absolute value of the quotient need to be calculated, which facilitates the fast shift subtraction-based implementation of the 2 divisions required for each CU.

3. Calculated (x)₀,y₀) Is added to the integer distance refinement MV to obtain the sub-pixel exact refinement increment MV.

2.11.3 reference spots required in DMVR

For a block of size W × H, (W +2 × offSet + filterSize-1) reference samples (H +2 × offSet + filterSize-1) are needed, assuming that the maximum allowed MVD value is +/-offSet (e.g. 2 in VVC) and the filter size is filterSize (e.g. 8in VVC for luminance, 4 for chrominance). To reduce memory bandwidth, only the center (W + filterSize-1) × (H + filterSize-1) reference samples are fetched, and other pixels are generated by repeating the fetched boundaries of the samples. An example of an 8 by 8 block is shown in fig. 15, extracting 15 by 15 reference spots, and repeating the boundaries of the extracted spots to generate a 17 by 17 region.

During motion vector refinement, bilinear motion compensation is performed using these reference samples. At the same time, these reference samples are also used to perform final motion compensation.

2.12 Bandwidth calculation for different Block sizes

Based on the current 8-tap luma interpolation filter and 4-tap chroma interpolation filter, the memory bandwidth per block unit (4: 2:0 color format, one MxN luma block with two M/2x N/2 chroma blocks) is shown in Table 1 below.

TABLE 1 example memory Bandwidth

Similarly, based on the current 8-tap luma interpolation filter and 4-tap chroma interpolation filter, the memory bandwidth of each MxN luma block unit is shown in table 2 below.

TABLE 2 example memory Bandwidth

Thus, regardless of the color format, the bandwidth requirements for each block size are arranged in descending order as follows:

4*4Bi>4*8Bi>4*16Bi>4*4Uni>8*8Bi>4*32Bi>4*64Bi>4*128Bi>8*16Bi>4*8Uni>8*32Bi>…。

motion vector accuracy problem in 2.12VTM-3.0

In VTM-3.0, the MV precision in storage is 1/16 luma pixels. When MV is signaled, the best precision is 1/4 luma pixels.

3. Examples of problems addressed by the disclosed embodiments

1. The bandwidth control method for affine prediction is not clear enough and should be more flexible.

2. Note that in HEVC design, the worst case memory bandwidth requirement is 8x8 bi-prediction, even Coding Units (CUs) can be split in asymmetric prediction modes, such as one 16x16 split into two PU. with sizes equal to 4x16 and 12x16 in VVC, one CU can be set to 4x16 due to the new QTBT split structure, and bi-prediction can be enabled.

3. New coding tools, such as GBi, introduce more line buffer problems

4. Inter-intra mode requires more memory and logic to signal the intra prediction mode used in inter-coded blocks.

The 5.1/16 luminance pixel MV precision requires higher memory storage.

6. To interpolate four 4x4 blocks in one 8x8 block, it needs to extract (8+7+1) × (8+7+1) reference pixels, and approximately 14% more pixels are needed when compared to the non-affine/non-planar mode 8x8 block.

7. The averaging operation in hybrid intra and inter prediction should be aligned with other coding tools such as weighted prediction, local illumination compensation, OBMC, and triangle prediction, where the offset is added before the shift.

4. Examples of the embodiments

The techniques disclosed herein may reduce the bandwidth and line buffers required in affine prediction and other new encoding tools.

The following description should be considered as an example to explain the general concept and not in a narrow sense. Furthermore, these embodiments may be combined in any manner.

In the following discussion, the width and height of the affine-coded current CU are w and h, respectively. Assume that the interpolation filter tap (in motion compensation) is N (e.g., 8, 6, 4, or 2) and the current block size is WxH.

Affine predicted bandwidth control

Example 1:suppose that the motion vector of the sub-block SB in the affine coding block is MV_SB(expressed as (MVx, MVy)), MV_SBMay be within a certain range with respect to the representative motion vector MV ' (MV ' x, MV ' y).

In some embodiments, MVx > ═ MV ' x-DH0 and MVx < ═ MV ' x + DH1 and MVy > ═ MV ' y-DV0 and MVy < ═ MV ' y + DV1, where MV ' ═ MV ' x, MV ' y. In some embodiments, DH0 may be equal to or not equal to DH 1; DV0 may or may not equal DV 1. In some embodiments, DH0 may or may not be equal to DV 0; DH1 may or may not be equal to DV 1. In some embodiments, DH0 may not be equal to DH 1; DV0 may not equal DV 1. In some embodiments, DH0, DH1, DV0, and DV1 may be signaled from the encoder to the decoder, such as in VPS/SPS/PPS/slice header/CTU/CU/PU. In some embodiments, DH0, DH1, DV0, and DV1 may be specified differently for different standard profiles (profiles)/levels. In some embodiments, DH0, DH1, DV0, and DV1 may depend on the width and height of the current block. In some embodiments, DH0, DH1, DV0, and DV1 may depend on whether the current block is uni-directional predicted or bi-directional predicted. In some embodiments, DH0, DH1, DV0, and DV1 may depend on the location of the sub-block SB. In some embodiments, DH0, DH1, DV0, and DV1 may depend on how the MV' is obtained.

In some embodiments, MV' may be a CPMV, such as MV0, MV1, or MV 2.

In some embodiments, MV 'may be MV in MC for one of the corner sub-blocks, such as MV 0', MV1 ', or MV 2' in fig. 3.

In some embodiments, MV' may be an MV derived for any location inside or outside the current block using an affine model of the current block. For example, it may be derived for the center position of the current block (e.g., x ═ w/2 and y ═ h/2).

In some embodiments, the MV' may be an MV in an MC for any sub-block of the current block, such as one of the central sub-blocks (C0, C1, C2, or C3 shown in fig. 3).

In some embodiments, if MV_SBIf the constraint is not satisfied, then MV_SBShould be clipped to the valid range. In some embodiments, the cropped MVs_SBIs stored into the MV buffer which will be used to predict MVs for subsequent coded blocks. In some embodiments, the MV before being clipped_SBIs stored into the MV buffer.

In some embodiments, if MV_SBIf the constraint is not satisfied, the bitstream is considered non-compliant (invalid).In one example, the MV may be specified in a standard_SBConstraints must or should be met. Any compliant encoder should comply with the constraint, otherwise the encoder is deemed to be non-compliant.

In some embodiments, the MV_SBAnd MV' may be represented with signaled MV precision (such as quarter-pixel precision). In some embodiments, the MV_SBAnd MV' may be represented with a stored MV precision, such as 1/16 precision. In some embodiments, the MV_SBAnd MV' may be rounded to a precision (such as integer precision) other than that signaled or stored.

Example 2:for affine coded blocks, each MxN (such as 8x4, 4x8, or 8x8) block within a block is considered a basic unit. The MVs of all 4x4 sub-blocks inside MxN are constrained such that the maximum difference between the integer part of the four 4x4 sub-blocks MVs does not exceed K pixels.

In some embodiments, whether and how this constraint is applied depends on whether the current block applies bi-directional prediction or uni-directional prediction. For example, the constraint applies only to bi-directional prediction and not to uni-directional prediction. As another example, M, N and K are different for bi-directional prediction and uni-directional prediction.

In some embodiments, M, N and K may depend on the width and height of the current block.

In some embodiments, whether or not to apply the constraint may be signaled from the encoder to the decoder, such as in VPS/SPS/PPS/slice header/CTU/CU/PU. For example, an on/off flag is signaled to indicate whether a constraint applies. As another example, M, N and K are signaled.

In some embodiments, M, N and K may be specified differently for different standard profiles/levels/hierarchies.

Example 3:the width and height of the sub-block may be calculated differently for different affine coding blocks.

In some embodiments, the calculation method is different for affine coding blocks with unidirectional prediction and bi-directional prediction. In one example, the sub-block size is fixed (such as 4x4, 4x8, or 8x4) for blocks with unidirectional prediction. In another example, for a block with bi-prediction, the sub-block size is calculated. In this case, the sub-block size may be different for two different bi-directionally predicted affine blocks.

In some embodiments, for bi-directionally predicted affine blocks, the width and/or height of the sub-block from reference list 0 and the width and/or height of the sub-block from reference list 1 may be different. In one example, assume that the width and height of the sub-block from reference list 0 are derived as Wsb0 and Hsb0, respectively; the width and height of the sub-blocks from reference list 1 are derived as Wsb1 and Hsb1, respectively. Then, the final width and height of the sub-blocks of both reference list 0 and reference list 1 are calculated as Max (Wsb0, Wsb1) and Max (Hsb0, Hsb1), respectively.

In some embodiments, the calculated width and height of the sub-block are applied only to the luminance component. For the chroma component, it is always fixed, such as a 4x4 chroma sub-block, which corresponds to an 8x8 luma block with a 4:2:0 color format.

In some embodiments, MVx-MV 'x and MVy-MV' y are calculated to determine the width and height of the sub-block. (MVx, MVy) and (MV 'x, MV' y) are defined in example 1.

In some embodiments, the MVs involved in the calculation may be represented in signaled MV precision (such as quarter-pixel precision). In one example, the MVs may be represented with a stored MV precision (such as 1/16 precision). As another example, the MVs may round to a precision (such as integer precision) different from the signaled or stored precision.

In some embodiments, the thresholds used in the calculations to decide the width and height of a sub-block may be signaled from the encoder to the decoder, such as in VPS/SPS/PPS/slice header/CTU/CU/PU.

In some embodiments, the thresholds used in the calculations to determine the width and height of the sub-blocks may be different for different standard profiles/levels/hierarchies.

Example 4:to interpolate the W1xH1 sub-block in one W2xH2 sub-block/block, first the (W2+ N-1-PW) ((H2 + N-1-PH)) block is extracted, thenThe pixel filling method described in example 6 (e.g., the boundary pixel repetition method) is applied later to generate a larger block, which is then used to interpolate the W1xH1 sub-block. For example, W2 ═ H2 ═ 8, W1 ═ H1 ═ 4, and PW ═ PH 0.

In some embodiments, the integer part of the MV of any W1xH1 sub-block may be used to extract the entire W2xH2 sub-block/block, thus possibly requiring a different boundary pixel repetition method. For example, if the maximum difference between the integer parts of all W1xH1 sub-blocks MV is no more than 1 pixel, the integer part of MV of the top left W1xH1 sub-block is used to extract the entire W2xH2 sub-block/block. The right and bottom boundaries of the reference block are repeated once. As another example, if the maximum difference between the integer parts of all W1xH1 sub-blocks MV is no more than 1 pixel, the integer part of MV of the bottom right W1xH1 sub-block is used to extract the entire W2xH2 sub-block/block. The left and top boundaries of the reference block are repeated once.

In some embodiments, the MV of any W1xH1 sub-block may be modified first and then used to extract the entire W2xH2 sub-block/block, thus possibly requiring a different boundary pixel repetition method. For example, if the maximum difference between the integer parts of all W1xH1 sub-blocks MV is no more than 2 pixels, then the integer parts of MV of the top left W1xH1 sub-block can be added by (1,1) (where 1 means 1 integer-pixel distance) and then used to extract the entire W2xH2 sub-block/block. In this case, the left, right, top and bottom boundaries of the reference block are repeated once. As another example, if the maximum difference between the integer parts of all W1xH1 sub-blocks MV is no more than 2 pixels, then the integer parts of the MVs of the bottom right W1xH1 sub-block may be added (-1, -1) (where 1 means 1 integer pixel distance) and then used to extract the entire W2xH2 sub-block/block. In this case, the left, right, top and bottom boundaries of the reference block are repeated once.

Bandwidth control for specific block size

Example 5:bidirectional prediction is not allowed if w and h of the current block satisfy one or more of the following conditions.

A.w is equal to T1 and h is equal to T2, or h is equal to T1 and w is equal to T2. In one example, T1-4 and T2-16.

B.w is equal to T1 and h is not greater than T2, or h is equal to T1 and w is not greater than T2. In one example, T1-4 and T2-16.

C.w is not greater than T1 and h is not greater than T2, or h is not greater than T1 and w is not greater than T2. In one example, T1-8 and T2-8. In another example, T1 ═ 8 and T2 ═ 4. In yet another example, T1-4 and T2-4.

In some embodiments, bi-prediction may be disabled for the 4x8 block. In some embodiments, bi-prediction may be disabled for 8x4 blocks. In some embodiments, bi-prediction may be disabled for the 4x16 block. In some embodiments, bi-prediction may be disabled for the 16x4 block. In some embodiments, bi-prediction may be disabled for 4x8, 8x4 blocks. In some embodiments, bi-prediction may be disabled for 4x16, 16x4 blocks. In some embodiments, bi-prediction may be disabled for 4x8, 16x4 blocks. In some embodiments, bi-prediction may be disabled for 4x16, 8x4 blocks. In some embodiments, bi-prediction may be disabled for a 4xN block, e.g., N < ═ 16. In some embodiments, bi-prediction may be disabled for Nx4 blocks, e.g., N < ═ 16. In some embodiments, bi-prediction may be disabled for an 8xN block, e.g., N < ═ 16. In some embodiments, bi-prediction may be disabled for Nx8 blocks, e.g., N < ═ 16. In some embodiments, bi-prediction may be disabled for 4x8, 8x4, 4x16 blocks. In some embodiments, bi-prediction may be disabled for 4x8, 8x4, 16x4 blocks. In some embodiments, bi-prediction may be disabled for 8x4, 4x16, 16x4 blocks. In some embodiments, bi-prediction may be disabled for 4x8, 8x4, 4x16, 16x4 blocks.

In some embodiments, the block size disclosed in this document may refer to one color component, such as a luminance component, and the decision as to whether to disable bi-prediction may be applied to all color components. For example, if bi-prediction is disabled according to the block size of the luminance component of a block, bi-prediction will also be disabled for corresponding blocks of other color components. In some embodiments, the block size disclosed in this document may refer to a color component, such as a luminance component, and the decision as to whether to disable bi-prediction may only be applied to that color component.

In some embodiments, if bi-prediction is disabled for a block and the selected Merge candidate is bi-predicted, only one MV from reference list 0 or reference list 1 of the Merge candidate is assigned to the block.

In some embodiments, if bi-prediction is disabled for a block, Triangle Prediction Mode (TPM) is not allowed for the block.

In some embodiments, how the prediction direction (unidirectional prediction from list 0/1, bi-directional prediction) is signaled may depend on the block dimensions. In one example, an indication of a unidirectional prediction from list 0/1 may be signaled when 1) block width block height <64 or 2) block width block height 64 but the width is not equal to the height. As another example, an indication of unidirectional prediction or bidirectional prediction from list 0/1 may be signaled when 1) block width by block height >64 or 2) n width by block height 64 and the width is equal to the height.

In some embodiments, both unidirectional prediction and bidirectional prediction may be disabled for a 4x4 block. In some embodiments, it may be disabled for affine coding blocks. Alternatively, it may be disabled for non-affine coding blocks. In some embodiments, indications of quadtree partitioning of 8x8 blocks, binary tree partitioning of 8x4 or 4x8 blocks, ternary tree partitioning of 4x16 or 16x4 blocks may be skipped. In some embodiments, the 4x4 block must be encoded as an intra block. In some embodiments, the MVs of the 4x4 block must be integer precision. For example, the IMV flag of a 4x4 block must be 1. As another example, the MVs of a 4x4 block must be rounded to integer precision.

In some embodiments, bi-directional prediction is allowed. However, assuming that the interpolation filter tap is N, only (W + N-1-PW) × (W + N-1-PH) reference pixels are fetched, instead of (W + N-1) × (H + N-1) reference pixels. At the same time, the pixels at the reference block boundaries (top, left, bottom and right boundaries) are repeated to generate the (W + N-1) × (H + N-1) block shown in FIG. 9, which is used for the final interpolation. In some embodiments, PH is zero, and only the left boundary or/and the right boundary is repeated. In some embodiments, PW is zero, and only the top boundary or/and the bottom boundary is repeated. In some embodiments, PW and PH are both greater than zero, and first the left boundary or/and the right boundary is repeated, then the top boundary or/and the bottom boundary is repeated. In some embodiments, both PW and PH are greater than zero, and first the top boundary or/and the bottom boundary is repeated, then the left boundary and the right boundary are repeated. In some embodiments, the left boundary is repeated M1 times and the right boundary is repeated PW-M1 times. In some embodiments, the top border is repeated M2 times and the bottom border is repeated PH-M2 times. In some embodiments, such a boundary pixel repetition method may be applied to some or all of the reference blocks. In some embodiments, PW and PH may be different for different color components, such as Y, Cb and Cr.

Fig. 9 shows an example of repeated boundary pixels of a reference block before interpolation.

Example 6:in some embodiments, (W + N-1-PW) ((W + N-1-PH) reference pixels (instead of (W + N-1) ((H + N-1) reference pixels) may be extracted for motion compensation of the WxH block. The spots outside the range (W + N-1-PW) (W + N-1-PH) but inside the range (W + N-1) (H + N-1) are filled in to perform the interpolation process. In one filling method, the pixels at the reference block boundaries (top, left, bottom and right boundaries) are repeated to generate a (W + N-1) × (H + N-1) block as shown in FIG. 11, which is used for final interpolation.

In some embodiments, PH is zero, and only the left boundary or/and the right boundary is repeated.

In some embodiments, PW is zero, and only the top boundary or/and the bottom boundary is repeated.

In some embodiments, PW and PH are both greater than zero, and first the left boundary or/and the right boundary is repeated, then the top boundary or/and the bottom boundary is repeated.

In some embodiments, both PW and PH are greater than zero, and first the top boundary or/and the bottom boundary is repeated, then the left boundary and the right boundary are repeated.

In some embodiments, the left boundary is repeated M1 times and the right boundary is repeated PW-M1 times.

In some embodiments, the top border is repeated M2 times and the bottom border is repeated PH-M2 times.

In some embodiments, such a boundary pixel repetition method may be applied to some or all of the reference blocks.

In some embodiments, PW and PH may be different for different color components, such as Y, Cb and Cr.

In some embodiments, the PW and PH may be different for different block sizes or shapes.

In some embodiments, PW and PH may be different for unidirectional prediction and bi-directional prediction.

In some embodiments, the padding may not be performed in affine mode.

In some embodiments, the spots outside the range (W + N-1-PW) (W + N-1-PH) but inside the range (W + N-1) (H + N-1) are set to a single value. In some embodiments, the single value is 1< < (BD-1), where BD is the bit depth of a sample point, such as 8 or 10. In some embodiments, a single value is signaled from the encoder to the decoder in VPS/SPS/PPS/slice header/CTU row/CTU/CU/PU. In some embodiments, the single value is derived from samples within a range (W + N-1-PW) (W + N-1-PH).

Example 7:instead of extracting (W + filterSize-1) (H + filterSize-1-PH) reference spots in the DMVR, (W + filterSize-1) ((H + filterSize-1)) reference spots may be extracted, and all other desired spots may be generated by repeating the boundaries of the extracted reference spots, where PW is 1-PW>0 and PH>＝0。

In some embodiments, the method set forth in example 6 may be used to fill in non-extracted samples.

In some embodiments, padding may not be performed again in the final motion compensation of the DMVR.

In some embodiments, whether the above method is applied may depend on the block dimension.

Example 8:the signaling method of inter _ pred _ idc may depend on whether w and h satisfy the condition in example 5. Table 3 below shows an example:

TABLE 3

Another example is shown in table 4 below:

TABLE 4

Table 5 below shows yet another example:

TABLE 5

Example 9:the Merge candidate list construction process may depend on whether w and h satisfy the condition in example 4. The following examples describe the cases when w and h satisfy the conditions.

In some embodiments, if one Merge candidate uses bi-directional prediction, only the prediction from reference list 0 is retained, and the Merge candidate is considered as a uni-directional prediction with reference to reference list 0.

In some embodiments, if one Merge candidate uses bi-directional prediction, only the prediction from reference List 1 is retained, and the Merge candidate is considered as a uni-directional prediction with reference to reference List 1.

In some embodiments, a Merge candidate is considered unavailable if it uses bi-prediction. That is, such a Merge candidate is removed from the Merge list.

In some embodiments, the Merge candidate list construction process for the triangle prediction mode is used instead.

Example 10:the coding tree partitioning process may depend on whether the width and height of the partitioned sub-CUs satisfy the conditions in example 5.

In some embodiments, if the width and height of the divided sub-CU satisfy the conditions in example 5, the division is not allowed. In some embodiments, signaling of the coding tree partitioning may depend on whether one partitioning is allowed. In one example, if one partition is not allowed, the codewords representing the partitions are omitted.

Example 11:the signaling of the skip flag or/and Intra Block Copy (IBC) flag may depend on whether the width and/or height of the Block meets certain conditions (e.g., the conditions mentioned in example 5).

In some embodiments, the condition is that the luminance block contains no more than X samples. For example, X ═ 16;

in some embodiments, the condition is that the luminance block contains X samples. For example, X ═ 16.

In some embodiments, the condition is that the width and height of the luminance block are both equal to X. For example, X ═ 4;

in some embodiments, when one or some of the above conditions are true, inter mode and/or IBC mode are not allowed for such blocks.

In some embodiments, if inter mode is not allowed for a block, a skip flag may not be signaled for it. Further, optionally, the skip flag may be inferred as false.

In some embodiments, if inter and IBC modes are not allowed for a block, a skip flag may not be signaled for it, and may be implicitly derived as false (e.g., a block is derived to be encoded in non-skip mode).

In some embodiments, if inter mode is not allowed for a block, but IBC mode is allowed for the block, a skip flag may still be signaled. In some embodiments, if the block is encoded in skip mode, the IBC flag may not be signaled and implicitly derived as true (e.g., the block is derived to be encoded in IBC mode).

Example 12:the signaling of the prediction mode may depend on whether the width and/or height of the block satisfy a certain condition (e.g., the condition mentioned in example 5).

In some embodiments, the condition is that the luminance block contains no more than 1X6 samples, e.g., X ═ 16.

In some embodiments, the condition is that the luminance block contains X samples, e.g., X ═ 16.

In some embodiments, the condition is that the width and height of the luminance block are both equal to X, e.g., X ═ 4.

In some embodiments, when one or some of the above conditions are true, inter mode and/or IBC mode are not allowed for such blocks.

In some embodiments, signaling of an indication of a particular mode may be skipped.

In some embodiments, if inter and IBC modes are not allowed for a block, signaling of indications of inter and IBC modes is skipped, and the remaining allowed modes, such as intra or palette (palette) mode, may still be signaled.

In some embodiments, if inter and IBC modes are not allowed for a block, the prediction mode may not be signaled. Further, alternatively, the prediction mode may be implicitly derived as an intra mode.

In some embodiments, if inter mode is not allowed for a block, signaling of an indication of inter mode is skipped, and the remaining allowed modes, such as intra mode or IBC mode, may still be signaled. Optionally, the remaining allowed modes may still be signaled, such as intra-frame mode or IBC mode or palette mode.

In some embodiments, if inter mode is not allowed for a block, but IBC mode and intra mode are allowed for it, an IBC flag may be signaled to indicate whether the block is encoded in IBC mode. Further, optionally, the prediction mode may not be signaled.

Example 13:signalling in delta modeThe notification may depend on whether the width and/or height of the block meets certain conditions (e.g., the conditions mentioned in example 5).

In some embodiments, the condition is that the luma block size is one of some particular sizes. For example, the particular dimensions may include 4x16 or/and 16x 4.

In some embodiments, when the above condition is true, the triangle mode may not be allowed, and the flag indicating whether the current block is encoded in the triangle mode may not be signaled and may be deduced as false.

Example 14:the signaling of the inter prediction direction may depend on whether the width and/or height of the block satisfy a certain condition (e.g., the condition mentioned in example 5).

In some embodiments, the condition is that the luma block size is one of some particular sizes. For example, the particular dimensions may include 8x4 or/and 4x8 or/and 4x16 or/and 16x 4.

In some embodiments, when the above condition is true, the block may be only uni-directionally predicted, and the flag indicating whether the current block is bi-directionally predicted may not be signaled and may be inferred to be false.

Example 15:the signaling of the SMVD (symmetric MVD) flag may depend on whether the width and/or height of the block satisfy a particular condition (e.g., the condition mentioned in example 5).

In some embodiments, the condition is that the luma block size is one of some particular sizes. In some embodiments, the condition is defined as whether a block size has no more than 32 samples. In some embodiments, the condition is defined as whether the block size is 4x8 or 8x 4. In some embodiments, the condition is defined as whether the block size is 4x4, 4x8, or 8x 4. In some embodiments, the particular dimensions may include 8x4 or/and 4x8 or/and 4x16 or/and 16x 4.

In some embodiments, when the particular condition is true, an indication of the use of SMVD (such as an SMVD flag) may not be signaled and may be inferred to be false. For example, the block may be set to be uni-directionally predicted.

In some embodiments, an indication of the use of SMVD (such as an SMVD flag) may still be signaled when a particular condition is true, however, only list 0 or list 1 motion information may be utilized in the motion compensation process.

Example 16:the motion vector (similar to the motion vector derived in the conventional Merge mode, ATMVP mode, MMVD Merge mode, MMVD skip mode, etc.) or the block vector for IBC may be modified according to whether the width and/or height of the block satisfy a certain condition.

In some embodiments, the condition is that the luma block size is one of some particular sizes. For example, the particular dimensions may include 8x4 or/and 4x8 or/and 4x16 or/and 16x 4.

In some embodiments, when the above condition is true, if the derived motion information is bi-directional predicted (e.g., inherited from a neighboring block with some offset), the motion vector or block vector of the block may be changed to a uni-directional motion vector. Such a process is called a conversion process, and the final unidirectional motion vector is called a "converted unidirectional" motion vector. In some embodiments, the motion information for reference picture list X (e.g., X is 0 or 1) may be maintained and discarded along with the motion information for list Y (Y is 1-X). In some embodiments, the motion information of reference picture list X (e.g., X is 0 or 1) and the motion information of list Y (Y is 1-X) may be jointly used to derive new motion candidate points for list X. In one example, the motion vector of the new motion candidate may be the average motion vector of the two reference picture lists. As another example, the motion information of list Y may be scaled to list X first. The motion vector of the new motion candidate may then be the average motion vector of the two reference picture lists. In some embodiments, the motion vector in the prediction direction X may not be used (e.g., the motion vector in the prediction direction X is changed to (0,0) and the reference index in the prediction direction X is changed to-1), and the prediction direction may be changed to 1-X, X ═ 0 or 1. In some embodiments, the converted unidirectional motion vector may be used to update the HMVP lookup table. In some embodiments, derived bi-directional motion information, e.g., bi-directional MVs prior to conversion to uni-directional MVs, may be used to update the HMVP lookup table. In some embodiments, the converted unidirectional motion vectors may be stored and may be used for motion prediction, TMVP, deblocking (deblocking), etc. of subsequent encoded blocks. In some embodiments, the derived bi-directional motion information, e.g., bi-directional MVs prior to conversion to uni-directional MVs, may be stored and may be used for motion prediction, TMVP, deblocking, etc. of subsequent encoded blocks. In some embodiments, the converted uni-directional motion vectors may be used for motion refinement. In some embodiments, the derived bi-directional motion information may be used for motion refinement and/or sample point refinement, such as with optical flow methods. In some embodiments, the prediction block generated from the derived bi-directional motion information may be first refined, after which the final prediction and/or reconstructed block of a block may be derived using only one prediction block.

In some embodiments, when the particular condition is true, the (bi-directionally predicted) motion vector may be converted to a uni-directional motion vector before being used as the base Merge candidate in MMVD.

In some embodiments, when the particular condition is true, the (bi-directionally predicted) motion vector may be converted to a uni-directional motion vector before being inserted into the Merge list.

In some embodiments, the converted uni-directional motion vector may only come from reference list 0. In some embodiments, when the current slice/slice group/picture is bi-predicted, the converted uni-directional motion vector may be from reference list 0 or list 1. In some embodiments, when the current slice/slice group/picture is bi-predicted, the converted uni-directional motion vectors from reference list 0 and list 1 may be interleaved (interleaved) in the Merge list or/and the MMVD base Merge candidate list.

In some embodiments, how the motion information is converted into a unidirectional motion vector may depend on the reference picture. In some embodiments, list 1 motion information may be utilized if all reference pictures of one video data unit (such as a slice/slice group) in display order are past pictures. In some embodiments, list 0 motion information may be utilized if at least one of the reference pictures of one video data unit (such as a slice/slice group) in display order is a past picture and at least one is a future picture. In some embodiments, how the motion information is converted to a unidirectional motion vector may depend on a low latency check flag.

In some embodiments, the conversion process may be invoked just prior to the motion compensation process. In some embodiments, the conversion process may be invoked just after the motion candidate list (e.g., Merge list) construction process. In some embodiments, the conversion process may be invoked before the add MVD process in the MMVD process is invoked. That is, the add MVD process follows the design of unidirectional prediction rather than bidirectional prediction. In some embodiments, the conversion process may be invoked before invoking the sample point refinement process in the PROF process. That is, the add MVD process follows the design of unidirectional prediction rather than bidirectional prediction. In some embodiments, the translation process may be invoked before invoking a BIO (aka BDOF) process. That is, for some cases, the BIO may be disabled because it has been converted to unidirectional prediction. In some embodiments, the conversion process may be invoked before the DMVR process is invoked. That is, for some cases, the DMVR may be disabled because it has been converted to unidirectional prediction.

Example 17:in some embodiments, how the motion candidate list is generated may depend on the block dimension.

In some embodiments, for certain block dimensions, all motion candidates derived from spatial and/or temporal blocks and/or HMVP and/or other types of motion candidates may be restricted to being uni-directionally predicted.

In some embodiments, for certain block dimensions, if one motion candidate derived from a spatial domain block and/or a temporal domain block and/or an HMVP and/or other type of motion candidate is bi-predictive, it may first be converted to uni-directional predictive before being added to the candidate list.

Example 18:whether sharing of the Merge list is allowed may depend on the encoding mode.

In some embodiments, the Merge list may not be allowed to be shared for blocks encoded in the regular Merge mode, and may be allowed to be shared for blocks encoded in the IBC mode.

In some embodiments, when a block partitioned from a parent shared node is encoded in the normal Merge mode, the update of the HMVP table may be disabled after encoding/decoding the block.

Example 19:in the examples disclosed above, the block size/width/height of the luminance block may also be changed to the block size/width/height of the chrominance blocks (such as Cb, Cr or G/B/R).

GBi mode line buffer reduction

Example 20:whether the GBi weighted index can be inherited or predicted from neighboring blocks (including CABAC context selection) depends on the location of the current block.

In some embodiments, the GBi weighted index cannot be inherited or predicted from a neighboring block that is not in the same coding tree unit (CTU, also known as a largest coding unit L CU) as the current block.

In some embodiments, the GBi weighted index cannot be inherited or predicted from neighboring blocks that are not on the same CTU line or CTU row as the current block.

In some embodiments, the GBi weighted index cannot be inherited or predicted from neighboring blocks that are not in the M × N region as the current block.

In some embodiments, the GBi weighted index cannot be inherited or predicted from neighboring blocks that are not in the same M × N region line or M × N region row as the current block.

In some embodiments, assuming that the top left corner (or any other location) of the current block is (x, y) and the top left corner (or any other location) of the neighboring block is (x ', y'), then it cannot be inherited or predicted from the neighboring block if one of the following conditions is met.

(1) x/M! x'/M. For example, M128 or 64.

(2) y/N! y'/N. For example, N-128 or 64.

(3) ((x/M | = x '/M) & (y/N | = y'/N)). For example, M-N-128 or M-N-64.

(4) ((x/M | = x '/M) | (y/N | = y'/N)). For example, M-N-128 or M-N-64.

(5) x > > M! X' > > M. For example, M ═ 7 or 6.

(6) y > > N! Y' > > N. For example, N ═ 7 or 6.

(7) (x > > M! & (y > > N! & (y' > > N) }. For example, M ═ N ═ 7 or M ═ N ═ 6.

(8) (x > > M! | (y > > N | = y' > > N)). For example, M ═ N ═ 7 or M ═ N ═ 6.

In some embodiments, a flag is signaled in the PPS or slice header or slice group header or slice to indicate whether GBi can be applied in the picture/slice group/slice. In some embodiments, whether and how GBi is used (such as how many candidate weights and values of weights) may be derived for a picture/slice. In some embodiments, the derivation may depend on information such as QP, temporal layer, POC distance, etc.

Fig. 10 shows an example of CTU (zone) lines. Shaded CTUs (regions) are on one CUT (region) line and unshaded CTUs (regions) are on the other CUT (region) line.

Simplification of inter-intra prediction (IIP)

Example 21:the encoding of intra prediction modes in an IIP encoded block is encoded independently of the intra prediction modes of neighboring blocks of the IIP encoding.

In some embodiments, only the intra-prediction mode of the intra-coded block may be used for encoding of the intra-prediction mode of the IIP coded block, such as during the MPM list construction process.

In some embodiments, intra-prediction modes in an IIP encoded block are encoded without mode prediction from any neighboring blocks.

Example 22:when the intra prediction mode of the IIP coding block and the intra prediction mode of the intra coding block are both used for encoding the intra prediction mode of the new IIP coding block, the IIP coding blockMay be lower than the intra-prediction mode of the intra-coded block.

In some embodiments, when deriving the MPM for an IIP encoded block, the intra prediction modes of both the IIP encoded block and the intra-encoded neighboring blocks are utilized. However, the intra prediction modes from the intra-coded neighboring blocks may be inserted into the MPM list before the intra prediction modes from the IIP-coded neighboring blocks.

In some embodiments, the intra-prediction modes from the intra-coded neighboring blocks may be inserted into the MPM list after the intra-prediction modes from the IIP-coded neighboring blocks.

Example 23:the intra prediction mode in the IIP encoded block may also be used to predict the intra prediction mode of the intra encoded block.

In some embodiments, the intra prediction mode in the IIP coding block may be used to derive the MPM of the normal intra coding block. In some embodiments, when intra-prediction modes in the IIP coding block and intra-prediction modes in the intra-coding block are used to derive MPMs for normal intra-coding blocks, the priority of the intra-prediction modes in the IIP coding block may be lower than the priority of the intra-prediction modes in the intra-coding block.

In some embodiments, the intra-prediction mode in the IIP encoded block may also be used to predict the intra-prediction mode of a normal intra-encoded block or an IIP encoded block only when one or more of the following conditions are met:

1. both blocks are in the same CTU line.

2. Both blocks are in the same CTU.

3. Both blocks are in the same M × N area, such as M-N-64.

4. The two blocks are in the same M × N area line, such as M-N-64.

Example 24:in some embodiments, the MPM construction process for the IIP encoded blocks should be the same as the MPM construction process for the normal intra-coded blocks.

In some embodiments, six MPMs are used for inter-coded blocks utilizing inter-intra prediction.

In some embodiments, only part of the MPM is used for the IIP encoded block. In some embodiments, the first is always used. Further, optionally, it is also not necessary to signal the MPM flag and the MPM index. In some embodiments, the first four MPMs may be utilized. Further, optionally, it is not necessary to signal the MPM flag, but the MPM index.

In some embodiments, each block may select one from the MPM list according to the intra prediction modes included in the MPM list, such as selecting the mode having the smallest index compared to a given mode (e.g., plane).

In some embodiments, each block may select a subset of modes from the MPM list and signal the mode index in that subset.

In some embodiments, the context used to encode the intra MPM mode is reused to encode the intra mode in the IIP encoded block. In some embodiments, different contexts used to encode intra MPM modes are used to encode intra modes in the IIP encoded block.

Example 25:in some embodiments, for angular intra prediction modes that do not include horizontal and vertical directions, equal weights are used for intra-prediction blocks and inter-prediction blocks generated for the IIP coding block.

Example 26:in some embodiments, for a particular location, a zero weight may be applied in the IIP encoding process.

In some embodiments, a zero weight may be applied to the intra prediction block used in the IIP encoding process.

In some embodiments, a zero weight may be applied to the inter-prediction block used in the IIP encoding process.

Example 27:in some embodiments, the intra prediction mode of the IIP encoded block can only be selected as one of the MPMs regardless of the size of the current block.

In some embodiments, no MPM flag is signaled and inferred to be 1 regardless of the size of the current block.

Example 28:for IIP encodingThe codeblock, luma-predicted chroma Mode (L uma-predicted-chroma Mode, L M) Mode instead of the Derived Mode (Derived Mode, DM), is used to perform intra prediction of the chroma components.

In some embodiments, both DM and L M may be allowed.

In some embodiments, the chroma component may allow for multiple intra prediction modes.

In some embodiments, whether multiple modes of the chroma components are allowed may depend on the color format. In one example, for the 4:4:4 color format, the allowed chroma intra prediction modes may be the same as for the luma component.

Example 29:inter-intra prediction may not be allowed in one or more of the following specific cases:

A.w-T1-h-T1, for example, T1-4

B.w > T1 h > T1, e.g., T1-64

C. (w ═ T1& & h ═ T2) | (w ═ T2& & h ═ T1), for example, T1 ═ 4, T2 ═ 16.

Example 30:for blocks that utilize bi-directional prediction, inter-intra prediction may not be allowed.

In some embodiments, if the selected Merge candidate for the IIP coding block uses bi-directional prediction, it will be converted to a uni-directional prediction Merge candidate. In some embodiments, only the prediction from reference list 0 is retained, and the Merge candidate is considered as a unidirectional prediction of reference list 0. In some embodiments, only the prediction from reference list 1 is retained, and the Merge candidate is considered as a unidirectional prediction with reference to reference list 1.

In some embodiments, a restriction is added that the selected Merge candidate should be a uni-directional prediction Merge candidate. Optionally, the range index of the signaled IIP coding block indicates the index of the uni-directional prediction Merge candidate (i.e. the bi-directional prediction Merge candidate is not counted).

In some embodiments, the Merge candidate list construction process used in delta prediction mode may be used to derive a motion candidate list for an IIP coding block.

Example 31:when inter-intra prediction is applied, some coding tools may not be allowed.

In some embodiments, bi-directional optical flow (BIO) is not applied to bi-directional prediction.

In some embodiments, Overlapped Block Motion Compensation (OBMC) is not applied.

In some embodiments, the decoder-side motion vector derivation/refinement process is not allowed.

Example 32: inter frame-The intra prediction process used in intra prediction may be different from the intra prediction process used in normal intra coded blocks.

In some embodiments, the neighboring samples may be filtered differently. In some embodiments, neighboring samples are not filtered prior to performing intra prediction for use in inter-intra prediction.

In some embodiments, for intra prediction used in inter-intra prediction, no position-dependent (position-dependent) intra prediction sample filtering process is performed. In some embodiments, multiline (multiline) intra prediction is not allowed in inter-intra prediction. In some embodiments, wide-angle intra prediction is not allowed in inter-intra prediction.

Example 33:it is assumed that intra and inter prediction values in mixed intra and inter prediction are PIntra and Pinter, and weighting factors are w _ intra and w _ inter, respectively. The predicted value at position (x, y) is calculated as (PINtra (x, y) × w _ intra (x, y) + PINter (x, y) × w _ inter (x, y) + offset (x, y))>>N, where w _ intra (x, y) + w _ iner (x, y) ═ 2^ N, and offset (x, y) ^ 2 (N-1). In one example, N ═ 3.

Example 34:in some embodiments, the MPM flags signaled in the normal intra and IIP coding blocks should share the same arithmetic coding context.

Example 35:in some embodiments, MPM is not required to encode the intra-prediction mode in the IIP encoded block. (assuming block widths and heights are w and h).

In some embodiments, the four modes { planar, DC, vertical, horizontal } are binarized to 00, 01, 10, and 11 (any mapping rule may be utilized, such as 00-planar, 01-DC, 10-vertical, 11-horizontal).

In some embodiments, the four modes { planar, DC, vertical, horizontal } are binarized to 0, 10, 110, and 111 (any mapping rule may be utilized, such as 0-planar, 10-DC, 110-vertical, 111-horizontal).

In some embodiments, the four modes { planar, DC, vertical, horizontal } are binarized to 1, 01, 001, and 000 (any mapping rule may be utilized, such as 1-planar, 01-DC, 001-vertical, 000-horizontal).

In some embodiments, only three modes { planar, DC, vertical } may be used when W > N × H (N is an integer such as 2) may be used. These three modes are binarized to 1, 01, 11 (any mapping rule may be utilized, such as 1-planar, 01-DC, 11-vertical).

In some embodiments, only three modes { planar, DC, vertical } may be used when W > N × H (N is an integer such as 2) may be used. These three modes are binarized to 0, 10, 00 (any mapping rule may be utilized, such as 0-plane, 10-DC, 00-vertical).

In some embodiments, only three modes { planar, DC, horizontal } may be used when H > N × W (N is an integer such as 2) may be used. These three patterns are binarized to 1, 01, 11 (any mapping rule may be utilized, such as 1-plane, 01-DC, 11-level).

In some embodiments, only three modes { planar, DC, horizontal } may be used when H > N × W (N is an integer such as 2) may be used. These three modes are binarized to 0, 10, 00 (any mapping rule may be utilized, such as 0-plane, 10-DC, 00-level).

Example 36:in some embodiments, only DC mode and planar mode are used in the IIP coding block. In some embodiments, a flag is signaled to indicate whether DC or planar is used.

Example 37:in some embodiments, the IIP is performed differently for different color components.

In some embodiments, the chroma components (such as Cb and Cr) are not inter-intra predicted.

In some embodiments, the intra-prediction mode for the chroma components in the IIP coding block is different from the intra-prediction mode for the luma components.

In some embodiments, how the different color components are IIP may depend on the color format (such as 4:2:0 or 4:4: 4).

In some embodiments, how the different color components are IIP may depend on the block size. For example, when the width or height of the current block is equal to or less than 4, inter-intra prediction is not performed on chrominance components such as Cb and Cr.

MV precision problem

In the following discussion, the precision of the MV for storage for spatial motion prediction is denoted as P1, and the precision of the MV for storage for temporal motion prediction is denoted as P2.

Example 38:p1 and P2 may be the same or different.

In some embodiments, P1 is a 1/16 luminance pixel and P2 is a 1/4 luminance pixel. In some embodiments, P1 is 1/16 luminance pixels and P2 is 1/8 luminance pixels. In some embodiments, P1 is 1/8 luminance pixels and P2 is 1/4 luminance pixels. In some embodiments, P1 is 1/8 luminance pixels and P2 is 1/8 luminance pixels. In some embodiments, P2 is a 1/16 luminance pixel and P1 is a 1/4 luminance pixel. In some embodiments, P2 is 1/16 luminance pixels and P1 is 1/8 luminance pixels. In some embodiments, P2 is 1/8 luminance pixels and P1 is 1/4 luminance pixels.

Example 39:p1 and P2 may not be fixed. In some embodiments, P1/P2 may be different for different standard profiles/levels/hierarchies. In some embodiments, P1/P2 may be different for pictures in different temporal layers. In some embodiments, P1/P2 may be different for pictures having different widths/heights. In some embodiments, the header may be VPS/SPS/PPS/stripe headerP1/P2 is signaled from the encoder to the decoder in/slice header/slice/CTU/CU.

Example 40:for MV (MVx, MVy), the precision of MVx and MVy may be different, denoted Px and Py.

In some embodiments, Px/Py may be different for different standard profiles/levels/hierarchies. In some embodiments, Px/Py may be different for pictures in different temporal layers. In some embodiments, Px may be different for pictures having different widths. In some embodiments, Py may be different for pictures with different heights. In some embodiments, Px/Py may be signaled from the encoder to the decoder in VPS/SPS/PPS/slice header/CTU/CU.

Example 41:before the MV (MVx, MVy) is put into storage for temporal motion prediction, it should be changed to the correct precision.

In some embodiments, if P1> -P2, MVx-Shift (MVx, P1-P2) and MVy-Shift (MVy, P1-P2). In some embodiments, if P1> -P2, MVx-SignShift (MVx, P1-P2) and MVy-SignShift (MVy, P1-P2). In some embodiments, if P1< P2, MVx ═ MVx < < < (P2-P1) and MVy < (P2-P1).

Example 42:it is assumed that MV (MVx, MVy) precisions Px and Py, and MVx or MVy is stored by an integer having N bits. The range of MVs (MVx, MVy) is MinX<＝MVx<MaxX, and MinY<＝MVy<＝MaxY。

In some embodiments, MinX may be equal to MinY, or it may not be equal to MinY. In some embodiments, MaxX may be equal to MaxY, or it may not be equal to MaxY. In some embodiments, { MinX, MaxX } may depend on Px. In some embodiments, { MinY, MaxY } may depend on Py. In some embodiments, { MinX, MaxX, MinY, MaxY } may depend on N. In some embodiments, { MinX, MaxX, MinY, MaxY } may be different for MVs stored for spatial and temporal motion prediction. In some embodiments, { MinX, MaxX, MinY, MaxY } may be different for different standard profiles/levels/hierarchies. In some embodiments, { MinX, MaxX, MinY, MaxY } may be different for pictures in different temporal layers. In some embodiments, { MinX, MaxX, MinY, MaxY } may be different for pictures with different widths/heights. In some embodiments, { MinX, MaxX, MinY, MaxY } may be signaled from the encoder to the decoder in VPS/SPS/PPS/slice header/CTU/CU. In some embodiments, { MinX, MaxX } may be different for pictures with different widths. In some embodiments, { MinY, MaxY } may be different for pictures with different heights. In some embodiments, MVx is clipped to [ MinX, MaxX ] before being placed in storage for spatial motion prediction. In some embodiments, MVx is clipped to [ MinX, MaxX ] before being placed in storage for temporal motion prediction. In some embodiments, MVy is clipped to [ MinY, MaxY ] before being placed in storage for spatial motion prediction. In some embodiments, MVy is clipped to [ MinY, MaxY ] before being placed in storage for temporal motion prediction.

Line buffer reduction for affine Merge mode

Example 43:the affine model (derived CPMV or affine parameters) inherited by the affine Merge candidate from the neighboring blocks is always a 6-parameter affine model.

In some embodiments, if the neighboring blocks are encoded with a 4-parameter affine model, the affine model is still inherited as a 6-parameter affine model.

In some embodiments, if the neighboring block is not in the same coding tree unit (CTU, also known as maximum coding unit L CU) as the current block, the 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model.

In some embodiments, assuming that the top left corner (or any other location) of the current block is (x, y) and the top left corner (or any other location) of the neighboring block is (x ', y'), then the 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model if the neighboring block satisfies one or more of the following conditions:

(a) x/M! x'/M. For example, M128 or 64.

(b) y/N! y'/N. For example, N-128 or 64.

(c) ((x/M | = x '/M) & (y/N | = y'/N)). For example, M-N-128 or M-N-64.

(d) ((x/M | = x '/M) | (y/N | = y'/N)). For example, M-N-128 or M-N-64.

(e) x > > M! X' > > M. For example, M ═ 7 or 6.

(f) y > > N! Y' > > N. For example, N ═ 7 or 6.

(g) (x > > M! & (y > > N! & (y' > > N) }. For example, M ═ N ═ 7 or M ═ N ═ 6.

(h) (x > > M! | (y > > N | = y' > > N)). For example, M ═ N ═ 7 or M ═ N ═ 6.