阅读说明：本技术 用于针对合并模式确定预测权重的方法、设备和系统 (Method, device and system for determining prediction weights for merge mode ) 是由 陈杰 廖如伶 叶琰 于 2020-02-28 设计创作，主要内容包括：本公开提供了用于针对合并模式确定预测权重的系统、设备和方法。一种示例性方法包括以下步骤：确定编码单元的继承仿射合并候选、构建仿射合并候选和零运动向量中的至少一者的权重；以及基于所确定的权重对所述编码单元进行双向预测。可以通过在比特流中显式地传输权重索引或者通过在解码器侧的隐式获得来确定当前编码单元的权重。(The present disclosure provides systems, devices and methods for determining prediction weights for merge modes. An exemplary method comprises the steps of: determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and bi-directionally predicting the coding unit based on the determined weights. The weight of the current coding unit can be determined by explicitly transmitting the weight index in the bitstream or by implicit acquisition at the decoder side.)

1. A method for processing video data, the method comprising the steps of:

determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and

bi-directionally predicting the coding unit based on the determined weights.

2. The method of claim 1, wherein the weight is determined based on a weight index transmitted in a bitstream.

3. The method of claim 1, wherein the determined weights comprise the weights of the build affine merge candidate, and further comprising the steps of:

in response to the plurality of control points associated with the build affine merge candidate having one or more weights, determining a weight of the build affine merge candidate based on a weight associated with an upper left control point or a weight associated with an upper right control point.

4. The method of claim 3, wherein the determined weights comprise the weights of the build affine merge candidate, and further comprising the steps of:

in response to a plurality of control points associated with the build affine merge candidate having the same weight, determining the weights of the plurality of control points as the weight of the build affine merge candidate; or

Determining a default value as a weight of the build affine merge candidate in response to the plurality of control points having different weights.

5. The method of claim 3, wherein the determined weights comprise the weights of the build affine merge candidate, and further comprising the steps of:

determining weights used by a majority of the plurality of control points from among weights of the plurality of control points corresponding to the build affine merge candidate; and

setting the determined weight as the weight of the constructed affine merging candidate.

6. The method of claim 3, wherein the determined weights comprise the weights of the build affine merge candidate, and further comprising the steps of:

determining differences between the equal weights and the weights of the plurality of control points, respectively;

determining a first control point having a weight that is least different from the equal weight from the plurality of control points; and

setting the weight of the first control point as the weight of the constructed affine merging candidate.

7. The method of claim 3, wherein the determined weights comprise the weights of the build affine merge candidate, and further comprising the steps of:

determining an average weight of a plurality of control points corresponding to the constructed affine merge candidate; and

setting the average weight as the weight of the constructed affine merging candidate.

8. The method of claim 1, wherein bi-directionally predicting the coding unit based on the determined weights comprises: processing the video data using an affine merge mode, and the method further comprises the steps of:

in constructing an affine merge candidate list, zero motion vectors with equal weight are inserted into the affine merge candidate list before zero motion vectors with unequal weight are inserted.

9. The method of claim 1, wherein bi-directionally predicting the coding unit based on the determined weights comprises: processing the video data using an affine merge mode, and the method further comprises the steps of:

determining an order of a plurality of construction affine merge candidates based on weights of control points corresponding to respective construction affine merge candidates among the plurality of construction affine merge candidates in constructing an affine merge candidate list,

the construction affine merge candidates having control points less weighted are assigned a higher priority than the construction affine merge candidates having control points more weighted.

10. The method of claim 9, wherein determining the order of the plurality of constructed affine merge candidates comprises:

in response to the weights of the control points corresponding to respective ones of the two build merge candidates having the same level of diversity, using a default order of the two build merge candidates.

11. The method of claim 9, further comprising the steps of:

determining availability of a first build affine merge candidate based on weights of control points corresponding to the first build affine merge candidate; and

in response to determining that the first build affine merge candidate is available, adding the first build affine merge candidate to the affine merge candidate list.

12. The method of claim 11, wherein determining the availability of the first build affine merge candidate comprises:

determining whether weights of two control points corresponding to the first build affine merge candidate have different signs; and

determining that the first build affine merge candidate is unavailable in response to determining that the weights of the two control points corresponding to the first build affine merge candidate have different signs.

13. A method for processing video data, the method comprising the steps of:

determining a weight of a first sub-block within a coding unit based on a weight of a control point of the coding unit;

bi-directionally predicting the first sub-block based on the determined weights.

14. The method of claim 13, wherein determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including one of the control points, using a weight of the one of the control points as a weight of the first sub-block, or

In response to the first sub-block not including the control point, determining a weight of the first sub-block according to a predefined rule.

15. The method of claim 13, wherein:

the coding unit is partitioned into four sub-blocks and has two control points; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including one of the two control points, using the weight of the one of the two control points as the weight of the first sub-block, or

In response to the first sub-block not including the two control points, setting a weight of the first sub-block to one of:

the weight of one of the two control points having a shorter distance from the first sub-block, or

A default value.

16. The method of claim 13, wherein:

the coding unit is partitioned into two sub-blocks and has three control points; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including only one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

In response to the first sub-block including at least two of the three control points, using a default value or a weight of one of the two control points as a weight of the first sub-block.

17. The method of claim 13, wherein:

the encoding unit is partitioned into four sub-blocks and has three control points having a first weight, a second weight, and a third weight, respectively; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including at least one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

In response to the sub-block not including the three control points, setting a weight of the first sub-block to one of:

an average of the first weight, the second weight, and the third weight,

a median of the first weight, the second weight, and the third weight,

the equal weight of the coding units,

one of the first weight, the second weight, and the third weight that is least different from the equal weight,

one of the first weight and the second weight that is less different from the equal weight, or

A default value.

18. A video processing device, the video processing device comprising:

a memory storing instructions; and

a processor configured to execute the instructions to cause the device to:

determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and

bi-directionally predicting the coding unit based on the determined weights.

19. A video processing device, the video processing device comprising:

a memory storing instructions; and

a processor configured to execute the instructions to cause the device to:

determining a weight of a first sub-block within a coding unit based on a weight of a control point of the coding unit;

bi-directionally predicting the first sub-block based on the determined weights.

20. A non-transitory computer readable medium storing a set of instructions executable by one or more processors of a video processing apparatus to cause the apparatus to perform a method comprising:

determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and

bi-directionally predicting the coding unit based on the determined weights.

Technical Field

The present disclosure relates generally to video data processing and, more particularly, to a method, apparatus and system for determining prediction weights for merge modes.

Background

Video coding is commonly used to convert and compress digital video signals, for example, to reduce consumed storage space or to reduce transmission bandwidth consumption associated with such signals.

Video coding systems may implement various tools or techniques to address different problems involved in the coding process. For example, in a video encoding process, input video data may be divided into block units of different sizes and processed block by block. In the prediction process, a block may be predicted using various prediction modes such as an intra prediction mode and an inter prediction mode. Based on the size of the block, different prediction modes may be applied. Various techniques have been proposed to improve the efficiency and accuracy of video coding and to reduce the computational complexity involved.

However, the application of different technologies may be subject to different conditions. For example, some techniques may require a coding unit to satisfy various conditions or to be encoded in a particular mode. As another example, some techniques may only be applicable to the processing of some motion vector candidates, but not to the processing of other motion vector candidates, in the weight prediction process. A combination of different techniques may not be feasible for processing all blocks or all sub-blocks. Therefore, it is desirable to appropriately combine or select the different techniques in consideration of respective applicable conditions of the different techniques.

Disclosure of Invention

Embodiments of the present disclosure provide methods, apparatuses, and systems for determining prediction weights in video data processing. According to some exemplary embodiments, the method performed by a decoder of video data comprises the steps of: determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and bi-directionally predicting the coding unit based on the determined weights. In some embodiments, the weights may be determined based on a weight index (weight index) of a signal transmitted in the bitstream.

In some embodiments, a method implemented by a decoder for obtaining weights for building affine merge candidates with multiple control points is provided. An exemplary method comprises the steps of: in response to the plurality of control points associated with the build affine merge candidate having one or more weights, determining a weight of the build affine merge candidate based on a weight associated with an upper left control point or a weight associated with an upper right control point.

In some embodiments, a method implemented by a decoder for obtaining weights for building affine merge candidates with multiple control points is provided. An exemplary method comprises the steps of: in response to a plurality of control points associated with a build affine merge candidate having the same weight, determining the weights of the plurality of control points as the weight of the build affine merge candidate; and determining a default (default) value as a weight of the build affine merge candidate in response to the plurality of control points having different weights.

In some embodiments, a method for determining prediction weights in video data processing is provided. One exemplary method comprises the steps of: determining a weight of a first sub-block within the coding unit based on the weight of the control point of the coding unit from among the weights of the plurality of control points; generating a motion predictor (predictor) for the first sub-block based on the determined weights; processing the coding unit based on the motion predictor.

In some embodiments, an apparatus for determining prediction weights in video data processing is provided. An exemplary video processing apparatus includes: a memory storing instructions; and a processor configured to execute the instructions to cause the device to: determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and bi-directionally predicting the coding unit based on the determined weights.

In some embodiments, an apparatus for determining prediction weights in video data processing is provided. An exemplary video processing apparatus includes: determining a weight of a first sub-block within a coding unit based on a weight of a control point of the coding unit; generating a motion predictor for the first sub-block based on the determined weights; processing the coding unit based on the motion predictor.

In some implementations, a non-transitory computer-readable medium is provided. An exemplary non-transitory computer readable medium stores a set of instructions executable by one or more processors of a video processing apparatus to cause the apparatus to perform a method comprising: determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and bi-directionally predicting the coding unit based on the determined weights.

In some implementations, a non-transitory computer-readable medium is provided. An exemplary non-transitory computer readable medium stores a set of instructions executable by one or more processors of a video processing apparatus to cause the apparatus to perform a method comprising: determining a weight of a first sub-block within a coding unit based on a weight of a control point of the coding unit; generating a motion predictor for the first sub-block based on the determined weights; and processing the coding unit based on the motion predictor.

Drawings

Embodiments and various aspects of the disclosure are illustrated in the following detailed description and drawings. The various features shown in the drawings are not drawn to scale.

Fig. 1 is a schematic diagram illustrating an exemplary video encoding and decoding system consistent with some embodiments of the present disclosure.

Fig. 2 is a schematic diagram illustrating an example video encoder that may be part of the example system of fig. 1 consistent with some embodiments of the present disclosure.

Fig. 3 is a schematic diagram illustrating an exemplary video decoder that may be part of the exemplary system of fig. 1 consistent with some embodiments of the present disclosure.

Fig. 4A is a schematic diagram of an example two-control-point affine motion model, according to some embodiments of the present disclosure.

Fig. 4B is a schematic diagram of an example three control point affine motion model, according to some embodiments of the present disclosure.

Fig. 5 is a schematic diagram illustrating an exemplary affine Motion Vector Field (MVF) for each sub-block in accordance with some embodiments of the present disclosure.

FIG. 6 is a schematic diagram illustrating exemplary locations of inherited affine motion predictors, according to some embodiments of the present disclosure.

Fig. 7 is a schematic diagram illustrating an example of control point motion vector inheritance, according to some embodiments of the present disclosure.

FIG. 8 is a schematic diagram illustrating example candidate locations for building an affine merge pattern in accordance with some embodiments of the present disclosure.

Fig. 9A is a schematic diagram illustrating horizontal sub-block partitioning for an encoding unit having two control points, according to some embodiments of the present disclosure.

Fig. 9B is a schematic diagram illustrating vertical sub-block partitioning for an encoding unit having two control points, according to some embodiments of the present disclosure.

Fig. 9C is a schematic diagram illustrating a sub-block partitioning into four sub-blocks for an encoding unit having two control points, according to some embodiments of the present disclosure.

Fig. 9D is a schematic diagram illustrating a sub-block partitioning into four sub-blocks for an encoding unit having two control points, according to some embodiments of the present disclosure.

Fig. 10A is a schematic diagram illustrating vertical sub-block partitioning for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 10B is a schematic diagram illustrating horizontal sub-block partitioning for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 10C is a schematic diagram illustrating horizontal sub-block partitioning for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 10D is a schematic diagram illustrating vertical sub-block partitioning for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 10E is a schematic diagram illustrating a sub-block partitioning into four sub-blocks for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 10F is a schematic diagram illustrating a sub-block partitioning into four sub-blocks for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 10G is a schematic diagram illustrating a sub-block partitioning into four sub-blocks for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 10H is a schematic diagram illustrating a sub-block partitioning into four sub-blocks for an encoding unit having three control points, according to some embodiments of the present disclosure.

Fig. 11 is a schematic diagram illustrating a sub-block partitioning into four sub-blocks for an encoding unit having four control points, according to some embodiments of the present disclosure.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, in which like reference numerals refer to the same or similar elements throughout the different views unless otherwise specified. The implementations set forth in the following description of exemplary embodiments do not represent all implementations consistent with the invention. Rather, the described implementations are merely examples of apparatus and methods consistent with aspects related to the invention as set forth in the claims below.

Fig. 1 is a block diagram illustrating an example video encoding and decoding system 100 that may utilize techniques compliant with various video encoding standards such as HEVC/h.265 and VVC/h.266. As shown in fig. 1, system 100 includes a source device 120, which source device 120 provides encoded video data to be later decoded by a destination device 140. Consistent with some implementations of the disclosure, each of source device 120 and destination device 140 may include any of a variety of devices, such as a desktop computer, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a mobile phone, a television, a camera, a wearable device (e.g., a smart watch or wearable camera), a display device, a digital media player, a video game console, a video streaming device, and so forth. Source device 120 and destination device 140 may be equipped for wireless communication or wired communication.

Referring to fig. 1, source device 120 may include a video source 122, a video encoder 124, and an output interface 126. Destination device 140 may include an input interface 142, a video decoder 144, and a display device 146. In some implementations, the source device and the destination device may further include other components or arrangements. For example, source device 120 may receive video data from an external video source (not shown), such as an external camera. Likewise, destination device 140 may interface with an external display device rather than include an integrated display device.

Although in the following description, some of the techniques are explained as being performed by a video encoding device, the techniques may also be performed by a video encoder/decoder (commonly referred to as a "CODEC"). Furthermore, the techniques of this disclosure may also be performed by a video preprocessor. Source device 120 and destination device 140 are merely examples of such encoding devices, where source device 120 generates encoded video data for transmission to destination device 140. In some implementations, source device 120 and destination device 140 may operate in a substantially symmetric manner such that each of source device 120 and destination device 140 includes video encoding and decoding components. Thus, system 100 may support one-way or two-way video transmission between source device 120 and destination device 140, e.g., for video streaming, video playback, video broadcasting, or video telephony.

The video source 122 of the source device 120 may include a video capture device (such as a video camera), a video archive containing previously captured video data, or a video feed interface for receiving video from a video content provider. As further alternatives, video source 122 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. Captured, pre-captured, or computer-generated video may be encoded by video encoder 124. The encoded video information may then be output by output interface 126 onto communication medium 160.

Output interface 126 may include any type of medium or device capable of transmitting encoded video data from source device 120 to destination device 140. For example, output interface 126 may include a transmitter or transceiver configured to transmit encoded video data from source device 120 directly to destination device 140 in real-time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 140.

The communication medium 160 may include a transitory medium such as a wireless broadcast or a wired network transmission. For example, communication medium 160 may include a Radio Frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). The communication medium 160 may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the internet. In some implementations, communication medium 160 may include a router, switch, base station, or any other device that may be used to facilitate communication from source device 120 to destination device 140. For example, a network server (not shown) may receive encoded video from source device 120 and provide the encoded video data to destination device 140, e.g., via a network transmission to destination device 140.

The communication medium 160 may also take the form of a storage medium (e.g., a non-transitory storage medium) such as a hard disk, a flash drive, a compact disk, a digital video disk, a blu-ray disk, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In some implementations, a computing device of a media production facility, such as a disc stamping facility, may receive encoded video data from source device 120 and produce a disc containing the encoded video data.

Input interface 142 of destination device 140 receives information from communication medium 160. The received information may include syntax information including syntax elements that describe the characteristics or processing of the blocks and other coded units. The syntax information is defined by the video encoder 124 and used by the video decoder 144. Display device 146 displays the decoded video data to a user and may include any of a variety of display devices, such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), a plasma display, an Organic Light Emitting Diode (OLED) display, or another type of display device.

As another example, the encoded video generated by source device 120 may be stored on a file server or storage device. The input interface 142 may access the stored video data from a file server or storage device via streaming or download. The file server or storage device may be any type of computing device capable of storing encoded video data and transmitting the encoded video data to the destination device 140. Examples of file servers include web servers supporting a website, File Transfer Protocol (FTP) servers, Network Attached Storage (NAS) devices, or local disk drives. The transmission of the encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

Video encoder 124 and video decoder 144 may each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), discrete logic, software, hardware, firmware, or any combinations thereof. When the techniques are implemented in part in the form of software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Each of video encoder 124 and video decoder 144 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in the respective device.

The video encoder 124 and the video decoder 144 may operate in accordance with any video coding standard, such as the general video coding (VVC/h.266) standard, the high efficiency video coding (HEVC/h.265) standard, the ITU-T h.264 (also known as the MPEG-4) standard, and so forth. Although not shown in fig. 1, in some implementations, the video encoder 124 and the video decoder 144 may each be integrated with an audio encoder and decoder, and may include appropriate MUX-DEMUX units or other hardware and software to handle encoding of both audio and video using a common data stream or separate data streams.

Fig. 2 is a schematic diagram illustrating an exemplary video encoder 200 consistent with the disclosed embodiments. For example, video encoder 200 may be used as video encoder 124 in system 100 (fig. 1). Video encoder 200 may perform intra-coding or inter-coding of blocks within a video frame, including video blocks or partitions or sub-partitions of video blocks. Intra-coding may rely on spatial prediction to reduce or remove spatial redundancy in video within a given video frame. Inter-frame coding may rely on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames of a video sequence. The intra mode may refer to a plurality of space-based compression modes. Inter modes (such as uni-directional prediction or bi-directional prediction) may refer to multiple time-based compressed modes.

Referring to fig. 2, an input video signal 202 may be processed block by block. For example, a video block unit may be a 16 × 16 pixel block (e.g., a Macroblock (MB)). The size of the video block unit may vary depending on the encoding technique used and the accuracy and efficiency desired. In HEVC, an extended block size (e.g., a Coding Tree Unit (CTU)) may be used to compress video signals with resolutions of, for example, 1080p and higher. In HEVC, a CTU may include up to 64 × 64 luma samples, corresponding chroma samples, and associated syntax elements. In VVC, the size of the CTU may be further increased to include 128 × 128 luma samples, corresponding chroma samples, and associated syntax elements. The CTUs may be further divided into Coding Units (CUs) using, for example, a quadtree, a binary tree, or a ternary tree. A CU may be further partitioned into Prediction Units (PUs) for which separate prediction methods may be applied. Each input video block may be processed using either spatial prediction unit 260 or temporal prediction unit 262.

The spatial prediction unit 260 performs spatial prediction (e.g., intra prediction) on the current block/CU using information on the same picture/slice including the current block. Spatial prediction may use pixels from already encoded neighboring blocks in the same video picture frame/slice to predict the current video block. Spatial prediction may reduce spatial redundancy inherent in video signals.

Temporal prediction unit 262 performs temporal prediction (e.g., inter prediction) on the current block using information from a different picture/slice than the picture/slice including the current block. The temporal prediction of a video block may be transmitted by one or more motion vectors. In unidirectional temporal prediction, only one motion vector indicating one reference picture is used to generate a prediction signal of a current block. On the other hand, in bi-directional temporal prediction, a prediction signal of a current block may be generated using two motion vectors (each motion vector indicating a corresponding reference picture). The motion vector may indicate an amount and direction of motion between the current block and one or more associated blocks in the reference frame. If multiple reference pictures are supported, one or more reference picture indices may be sent for the video block. One or more reference indices may be used to identify which reference picture(s) in the reference picture store or Decoded Picture Buffer (DPB)264 the temporal prediction signal is likely to come from.

The mode decision in the encoder and encoder control unit 280 may select the prediction mode, e.g. based on rate-distortion optimization. Based on the determined prediction mode, a prediction block may be obtained. The prediction block may be subtracted from the current video block at adder 216. The prediction residual may be transformed by the transform unit 204 and quantized by the quantization unit 206. The quantized residual coefficients may be inverse quantized at inverse quantization unit 210 and inverse transformed at inverse transform unit 212 to form a reconstructed residual. The reconstructed residual may be added to the prediction block at adder 226 to form a reconstructed video block. The reconstructed video block prior to loop filtering may be used to provide reference samples for intra prediction.

The reconstructed video block may undergo loop filtering at loop filter 266. For example, loop filtering such as a deblocking filter, Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF) may be applied. The reconstructed block after loop filtering may be stored in the reference picture storage 264 and may be used to provide inter prediction reference samples to encode other video blocks. To form the output video bitstream 220, the encoding mode (e.g., inter or intra), prediction mode information, motion information, and quantized residual coefficients may be sent to the entropy encoding unit 208 to further reduce the bit rate before the data is compressed and packed to form the bitstream 220.

Fig. 3 is a schematic diagram illustrating a video decoder 300 consistent with the disclosed embodiments. For example, the video decoder 300 may be used as the video decoder 144 in the system 100 (fig. 1). Referring to fig. 3, the video bitstream 302 may be unpacked or entropy decoded at an entropy decoding unit 308. The coding mode information may be used to determine whether to select the spatial prediction unit 360 or the temporal prediction unit 362. The prediction mode information may be transmitted to a corresponding prediction unit to generate a prediction block. For example, motion compensated prediction may be applied by the temporal prediction unit 362 to form a temporal prediction block.

The residual coefficients may be sent to an inverse quantization unit 310 and an inverse transform unit 312 to obtain a reconstructed residual. The prediction block and the reconstructed residual may be added together at 326 to form a reconstructed block prior to loop filtering. The reconstruction block may then undergo loop filtering at loop filter 366. For example, loop filtering such as deblocking filters, SAO, and ALF may be applied. The reconstructed block after the loop filtering may then be stored in the reference picture storage 364. The reconstructed data in the reference picture store 364 may be used to obtain decoded video 320 or to predict future video blocks. The decoded video 320 may be displayed on a display device, such as the display device 146 described in system 100 (fig. 1).

In some implementations, affine motion prediction may be applied. For example, in HEVC, a translational motion model is applied to Motion Compensated Prediction (MCP). While in the real world there are a wide variety of movements such as zoom in/out, rotation, perspective movement and other irregular movements. In VVC, block-based affine transform motion compensated prediction is applied. As shown in fig. 4A and 4B, the affine motion field of the block can be described by motion information of two control point motion vectors (4-parameter affine motion model, see fig. 4A) or three control point motion vectors (6-parameter affine motion model, see fig. 4B).

As shown in fig. 4A, v0 represents the control point motion vector for the top left corner Control Point (CP)402, and v1 represents the control point motion vector for the top right corner CP 404. For a 4-parameter affine motion model, the motion vector at sample position (x, y) in the block can be obtained as follows:

where (mv0x, mv0y) indicates the motion vector of the top left CP 402 and (mv1x, mv1y) indicates the motion vector of the top right CP 404.

As shown in FIG. 4B, in the 6-parameter affine motion model, v₀Control point motion vector, v, representing the top left corner Control Point (CP)406₁Control point motion vector representing the top right CP 408, and v₂The control point motion vector of the lower left corner CP 410 is represented. For a 6-parameter affine motion model, at sample position (x, y) in the blockThe motion vector of (a) can be obtained as follows:

where (mv0x, mv0y) indicates the motion vector of the upper left CP 406, (mv1x, mv1y) indicates the motion vector of the upper right CP 408, and (mv2x, mv2y) indicates the motion vector of the lower left CP 410.

In some embodiments, to simplify the motion compensated prediction process, block-based affine transform prediction may be applied. To obtain the motion vectors of the respective 4 × 4 luminance sub-blocks, the motion vectors of the center samples of the respective sub-blocks (as shown in fig. 5) may be calculated according to the above equations (1) and (2) and rounded to 1/16 fractional precision. Arrows in fig. 5 indicate motion vectors corresponding to the respective sub-blocks. A motion compensated interpolation filter may then be applied to generate a prediction for each sub-block using the obtained motion vectors. The subblock size of the chrominance component may also be set to 4 × 4. The motion vector of the 4 x 4 chroma sub-block may be calculated as the average of the motion vectors of the four corresponding 4 x 4 luma sub-blocks.

There are also two affine motion inter prediction modes when performing for translational motion inter prediction: affine merge mode and affine Advanced Motion Vector Prediction (AMVP) mode. For CUs having a width and a height greater than or equal to 8, an affine merging mode may be applied. In AMVP mode, a Control Point Motion Vector (CPMV) of a current CU may be generated based on motion information of spatially neighboring CUs. There may be up to five CPMV candidates. An index may be transmitted to indicate one candidate to use for the current CU. The following three types of CPMV candidates are used to form the affine merge candidate list: (1) inherited affine merge candidates extrapolated from CPMVs of neighboring CUs; (2) a constructed affine merging candidate CPMV obtained using translational Motion Vectors (MVs) of neighboring CUs; (3) and zero MV. These three types of candidates are described further below.

In VVC, there may be at most two inherited affine merge candidates. Two inherited affine merge candidates are derived from affine motion models of neighboring blocks, one from the left neighboring CU and one from the above neighboring CU. FIG. 6 shows candidate blocks A0, A1, B0, B1, and B2. For the left motion vector predictor, the neighboring blocks may be checked in the following order: a0- > A1. For the above motion vector predictor, neighboring blocks may be checked in the following order: b0- > B1- > B2. The first inheritance candidates from each side (left and top) can be selected. That is, on each side, the first neighboring block encoded in affine mode in check order is selected. Furthermore, pruning check cannot be performed between two inherited candidates.

When a neighboring affine CU is identified, its control point motion vector may be used to obtain the CPMV candidate in the affine merge list of the current CU. As shown in fig. 7, if the neighboring lower left block a is encoded in affine mode, a motion vector v of the upper left corner of the CU 704 containing block a can be implemented₂Motion vector v in the upper right corner₃And the motion vector v of the lower left corner₄. If block A is encoded with a 4-parameter affine model, it can be based on v₂And v₃Two CPMVs are calculated for the current CU 702. Alternatively, if block a is encoded with a 6-parameter affine model, it can be according to v₂、v₃And v₄Three CPMVs are calculated for the current CU 702.

The construction affine candidates refer to candidates constructed by combining adjacent motion information of the respective control points. As shown in fig. 8, the motion information for the four control points 802, 804, 806, and 808 may be derived from specified spatial and temporal neighbors. CPMV (compact peripheral component management video) system_k(k-1, 2,3,4) may be used to represent the kth control point CP_kThe motion vector of (2). For CPMV corresponding to CP 801₁The neighboring blocks B2, B3, and a2 may be checked according to the following order: b2->B3->A2. The MV of the first available block may be used. For CPMV corresponding to CP 804₂The neighboring blocks B1 and B0 may be checked in the following order: b1->B0. For CPMV corresponding to CP 806₃The neighboring blocks a1 and a0 may be checked in the following order: a1->A0. For CPMV corresponding to CP 808₄The Temporal Motion Vector Predictor (TMVP) can be used as the CPMV₄(if available).

After obtaining the MVs of the four control points 802-808, affine merging candidates may be constructed based on the motion information. The combination of control points MV can be used to construct candidates in the following order:

{CPMV₁，CPMV₂，CPMV₃}，{CPMV₁，CPMV₂，CPMV₄}，{CPMV₁，CPMV3，CPMV₄}，

{CPMV₂，CPMV₃，CPMV₄}，{CPMV₁，CPMV₂}，{CPMV₁，CPMV₃}

a combination of 3 CPMVs may construct a 6-parameter affine merge candidate, and a combination of 2 CPMVs may construct a 4-parameter affine merge candidate. In some embodiments, to avoid the motion scaling process, if the reference indices of the control points are different, the corresponding combination of control points MV may be discarded.

After examining the inherited affine merge candidate and constructing the affine merge candidate, if the affine merge candidate list is still not full, a zero MV may be inserted at the end of the list.

The affine AMVP mode may be applied to CUs having a width and height greater than or equal to 16. An affine flag at the CU level may be transmitted in the bitstream to indicate whether affine AMVP mode is used. If affine AMVP mode is used, another flag may be transmitted to indicate whether 4-parameter affine or 6-parameter affine is used. The difference of the CPMV of the current CU and its predictor (CPMVP) can be transmitted in the bitstream. The affine AMVP candidate list size is 2. The candidate list may be generated by using the following four types of CPMVP candidates in the following order: (1) inherited affine AMVP candidates extrapolated from the CPMVs of neighboring CUs; (2) constructing affine AMVP candidate CPMVP obtained by using the translation MVs of the adjacent CUs; (3) a translation MV from a neighboring CU; and (4) zero MV.

The checking order of the inherited affine AMVP candidate may be the same as the checking order of the inherited affine merge candidate, as described above. The only difference is that for AMVP candidates, only affine CUs with the same reference picture as the current block are considered. The pruning process is not applied when inserting the inherited affine motion predictor into the candidate list.

Constructing an AMVP candidate may result from a specified spatial neighbor, as shown in fig. 8. The same checking order used in the affine merge candidate construction may be used. In addition, reference picture indices of the neighboring blocks are also checked. A first block in check order is used that is inter-coded and has the same reference picture as in the current CU. Coding with 4-parameter affine mode on current CU and mv₀And mv₁They may be added as a candidate to the affine AMVP list when available. When the current CU is encoded with 6-parameter affine mode and all three CPMVs are available, they are added as a candidate to the affine AMVP list. Otherwise, the build AMVP candidate may be set as unavailable.

After examining the inherited AMVP candidates and building the AMVP candidates, the affine AMVP list candidates may still be less than 2. In this case, mv may be added in order when available₀、mv₁And mv₂As the panning MVs for predicting all control points MV of the current CU. Finally, if the affine AMVP candidate list is still not full, the list may be populated with zero MVs.

In some embodiments, bi-directional prediction (BWA) with weighted averaging may be applied. Conventionally, bi-directional prediction signals are generated by only averaging two prediction signals obtained from two different reference pictures and/or using two different motion vectors. In VVC, the bi-directional prediction mode extends beyond simple averaging to allow weighted averaging of the two prediction signals, as shown by:

P_bi-pred＝((8-w)*P₀+w*P₁+4)>>3 (3)

five weights are allowed in weighted average bi-directional prediction, w ∈ { -2, 3,4, 5, 10 }. The weights w may be determined in one of two ways for each bi-directionally predicted CU: 1) for non-merging CUs, the weight index may be transmitted after the motion vector difference; 2) for a merging CU, a weight index may be inferred from neighboring blocks based on the merging candidate index. Weighted average bi-prediction may be applied to CUs with 256 or more luma samples (which means that the CU width multiplied by the CU height is greater than or equal to 256). For low delay pictures, all 5 weights may be used. For non-low delay pictures, only 3 weights (w ∈ {3, 4, 5}) are used.

At the encoder side, a fast search algorithm can be applied to find the weight index without significantly increasing the complexity of the encoder. The application of these algorithms can be summarized based on the following. When combined with Adaptive Motion Vector Resolution (AMVR), the unequal weight (unity weight) is conditionally checked only for 1-pixel and 4-pixel motion vector precision if the current picture is a low latency picture. When combined with affine motion compensation, affine Motion Estimation (ME) is performed for unequal weights if and only if the affine mode is selected as the current best mode. The unequal weights are only conditionally checked when the two reference pictures in bi-prediction are identical.

In some embodiments, the unequal weights are not searched when certain conditions are met. Various factors may be considered, such as Picture Order Count (POC) distance between the current picture and its reference picture, coding Quantization Parameter (QP), and temporal level.

As described above, when BWA is combined with merge prediction, a weight index may be inferred from neighboring blocks based on the merge candidate index. However, in the case of affine merge prediction, this may not always be correct. As described above, for affine merging prediction, the following three types of CPMV candidates may exist in the affine merging candidate list: 1) inherited affine merge candidates extrapolated from CPMVs of neighboring CUs; 2) constructing affine merging candidate CPMVs obtained by using the translation MVs of the adjacent CUs; and 3) zero MV. When a candidate of the first type (inherited affine merge candidate) is selected, the BWA weight index of the neighboring CU is also inherited. Therefore, the weight index of the current CU used in the bi-directional prediction is the same as the weight index of the neighboring CU for inheriting the CPMV. However, when a second type of candidate (building affine merge candidate) or a third type of candidate (zero motion vector) is selected, the BWA weight index of the neighboring CU is not inherited. Conversely, in some techniques, equal weight may be used for the current CU by setting the current CU weight index to a default value.

The above design has at least the following disadvantages. Based on the above design, there may be an inconsistency of BWA weight index inheritance between the inherited affine merge candidate, which is two types of CPMV candidates in the same candidate list, and the build affine merge candidate. Thus, the encoder and decoder need to distinguish between these two candidate types and use different logic to generate the BWA weight for the current CU. The inherited affine merge candidate inherits the neighboring CUs, while the build affine merge candidate is fixed. This therefore increases the complexity of the processing.

Furthermore, if the build affine merge candidate is chosen, the weight of the current CU may be fixed, which loses the flexibility of weight selection. BWA is adopted by the VVC standard because it improves coding performance by providing more averaging weight options in inter prediction. However, using a fixed equal weight reduces the advantage of BWA, and thus may compromise coding performance.

Further, when the candidate list is not full after inserting the inherited affine merge candidate and constructing the affine merge candidate, zero MV is inserted into the affine merge candidate list. However, fixing the BWA weights for all zero MVs inserted into the candidate list means that the same MV candidate is repeated in the list, which introduces redundancy in the candidate signal transmission. In addition, as described above, the neighboring CU weights are not used for constructing affine merging candidates. This deviates from the design concept of inferring the weight index from neighboring blocks based on merging candidate indices when combined with BWA.

In order to solve the above problems, the present disclosure provides the following techniques.

One technique to address the above problem is through explicit weight signaling. With explicit signaling, the encoder may select the weight of the current CU and may explicitly transmit an index of the selected weight in the bitstream. This changes the idea of inferring the weight index from neighboring blocks based on the merge candidate index. For affine merge prediction, the weights of the current CU are no longer inferred from neighboring CUs. The encoder may select any value from the allowed candidate weights and transmit the selected value in the bitstream.

In some embodiments of the present disclosure, weights for all kinds of affine merge candidates (including inherited affine merge candidates, build affine merge candidates, and zero motion vectors) may be selected by the encoder and transmitted in the bitstream. In some embodiments, the weights for some, but not all, affine merge candidates are selected by the encoder and transmitted in the bitstream. For example, weights may be selected and transmitted only for building affine merge candidates; and weights are inferred from neighboring CUs or fixed to default values for other candidates (e.g., inherited affine merge candidates).

In some embodiments of the present disclosure, the weights may be transmitted by sending a weight index in the bitstream.

Another technique to solve the existing problem is through implicit weight acquisition. With this technique, the weight is not fixed for the CU using the constructed affine merging candidate. Instead, weights may be inferred from neighboring CUs. For each constructed affine merging candidate, a weight or weight index may be obtained from the CP of the constructed affine merging candidate. If the build affine merge candidate is used for a CU, the obtained weights or weight indices may also be used for the CU.

For example, CP_k(k ═ 1,2,3,4) can be used to denote the kth CP. For each CP, a corresponding CPMV may be obtained using the neighboring CUs. The neighboring CUs are also used to obtain the weight or weight index of the CP. In general, there are 4 control points. Thus, the building affine merge candidate may have 4 different weights or weight indices at most. Thus, a rule may be specified to obtain one weight or weight index from among at most 4 weights or weight indexes to construct an affine merge candidate. For example, multiple CPs (including the top-left CP and/or the top-right CP constructing affine merge candidates) have one or more weights. The weight to construct an affine merge candidate may be determined based on the weight associated with the top-left CP and/or the weight associated with the top-right CP. Similarly, in some embodiments, the lower left CP and/or the lower right CP constructing the affine merging candidate have one or more weights. The weight to construct an affine merge candidate may be determined based on the weight associated with the lower left CP and/or the weight associated with the lower right CP. Examples of the acquisition process are described further below. In some of the present disclosureIn an embodiment, if all of the CPs have the same weight or weight index, the weight or weight index may be used to construct an affine merge candidate. If not all CPs have the same weight or weight index, a default value corresponding to equal weight may be used.

For example, for a constructed affine merge candidate with 2 CPs (CPa, CPb),

if(CP_aweight＝＝CP_b weight)

the affine merge candidate weight＝CP_aweight

else

the affine merge candidate weight＝default value

further, for a constructed affine merging candidate having 3 CPs (CPa, CPb, CPc),

if(CP_aweight＝＝CP_bweight＆＆CP_b weight＝＝CP_c weight)

the affine merge candidate weight＝CP_aweight

else

the affine merge candidate weight＝default value

similarly, the above-described rule may also be applied to a build affine merge candidate having 4 or more CPs, and the above-described operation may also be applied to the weight index.

In some embodiments of the present disclosure, the weight or weight index that constructs the affine merge candidate may be the most likely weight or weight index. The most likely weight or weight index may correspond to the weight or weight index used by the majority of CPs that build affine merge candidates. For example, for a build affine merge candidate with 2 CPs (CPa, CPb), the weights may be determined based on:

if(CP_a weight＝＝CP_b weight)

the affine merge candidate weight＝CP_a weight

else

the affine merge candidate weight＝default value

for a constructed affine merge candidate with 3 CPs (CPa, CPb, CPc), weights may be determined based on:

if(CP_a weight＝＝CP_b weight)

the affine merge candidate weight＝CP_a weight

else if(CP_b weight＝＝CP_cweight)

the affine merge candidate weight＝CP_b weight

else if(CP_aweight＝＝CP_cweight)

the affine merge candidate weight＝CP_c weight

the above-described rule may also be applied to the construction of affine merging candidates having 4 or more CPs, and the determination of weight indexes.

In some embodiments of the present disclosure, from all weights of the CP constructing the affine merging candidate, the weight with the smallest difference from the equal weight may be used as the weight or weight index constructing the affine merging candidate. For example, for a build affine merge candidate with 2 CPs (CPa, CPb), the weights may be determined based on:

Diff_a＝|CP_a weight-equal weight|

Diff_b＝|CP_b weight-equal weight|

if(Diff_a＜＝Diff_b)

the affine merge candidate weight＝CP_aweight

else

the affine merge candidate weight＝CP_b weight

for a constructed affine merge candidate with 3 CPs (CPa, CPb, CPc), weights may be determined based on:

Diff_a＝|CP_a weight-equal weight|

Diff_b＝|CP_b weight-equal weight|

Diff_c＝|CP_c weight-equal weight|

if(Diff_a＜＝Diff_b&&Diff_a＜＝Diff_c)

the affine merge candidate weight＝CP_a weight

else if(Diff_b＜＝Diff_a&&Diff_b＜＝Diff_c)

the affine merge candidate weight＝CP_b weight

else

the affine merge candidate weight＝CP_cweight

alternatively, for a build affine merge candidate with 2 CPs (CPa, CPb), the weights may be determined based on:

if(CP_a weight＞＝equal weight＆＆CP_b weight＞＝equal weight)

the affine merge candidate weight＝min(CP_a weight，CP_b weight)

else if(CP_a weight＜＝equal weight＆＆CP_b weight＜＝equal weight)

the affine merge candidate weight＝max(CP_a weight，CP_bweight)

for a constructed affine merge candidate with 3 CPs (CPa, CPb, CPc), weights may be determined based on:

if(CP_a weight＞＝equal weight＆＆CP_b weight＞＝equal weight＆＆CP_cweight＞＝equal weight)

the affine merge candidate weight＝min(CP_a weight，CP_b weight，CP_cweight)

else if(CP_a weight＜＝equal weight＆＆CP_b weight＜＝equal weight＆＆CP_c weight＜＝equal weight)

the affine merge candidate weight＝max(CP_a weight，CP_b weight，CP_cweight)

else

the affine merge candidate weight＝equal weight，or the one closer to the equal weight of the two on the same side of equal weight

similarly, the above-described rule may also be applied to the construction affine merging candidate having 4 or more CPs, and the determination of the weight index.

In some embodiments, the average of the CP weights or weight indices may be used as the weight to construct an affine merge candidate. For example, for a build affine merge candidate with 2 CPs (CPa, CPb), the weights may be determined based on:

weight＝integer part of(CP_a weight+CP_b weight)/2

res＝(CP_a weight+CP_b weight)％2

if(res＝＝0)

the affine merge candidate weight＝weight

else

the affine merge candidate weight＝round(CP_a weight+CP_b weight)/2 towards to equal weight

further, for a constructed affine merging candidate having 3 CPs (CPa, CPb, CPc), the weight may be determined based on:

weight＝integer part of(CP_aweight+CP_b weight+CP_c weight)/3

res＝(CP_a weight+CP_b weight+CP_c weight)％3

if(res＝＝0)

the affine merge candidate weight＝weight

else

the affine merge candidate weight＝round(CP_aweight+CP_b weight+CP_cweight)/3 towards to equal weight

similarly, the above-described rule may also be applied to the construction affine merging candidate having 4 or more CPs, and the determination of the weight index.

According to some embodiments of the present disclosure, methods of using BWA weights for zero MV candidates are also provided. As described above, after inserting the inherited affine merge candidate and the build affine merge candidate into the affine merge candidate list (if the candidate list is not full), the candidate list will be filled with zero MVs until the candidate list is full. Consistent with some embodiments of the present disclosure, the candidate list may be populated with zero MVs having different BWA weights, such that the populated zero MV candidates may provide different predictors.

For example, the weight order of zero MVs to be padded in the candidate list may be {4,4}/8, {5,3}/8, {3,5}/8, { -2,10}/8, {10, -2 }/8. After obtaining the inherited affine merge candidate and constructing the affine merge candidate, if the candidate list is not full, zero MVs with equal weights {4,4}/8 may be inserted first. If the list is still not full, zero MVs with weights {5,3}/8 can be inserted. Thereafter, if the list is still not full, zero MVs with weights {3,5}/8, { -2,10}/8, and {10, -2}/8 can be inserted until the list is full. If the candidate list is still not full after inserting zero MVs with all weights, zero MVs with different reference indices may be inserted.

In VVC, there may be a plurality of building affine merge candidates. The order of the candidate control point sets/combinations is as follows: { CP₁，CP₂，CP₃}、{CP₁，CP₂，CP₄}、{CP₁，CP₃，CP₄}、{CP₂，CP₃，CP₄}、{CP₁，CP₂}、{CP₁，CP₃}. The encoder and decoder may check from the first combination to the last combination. If a combination is available, it may be placed in a candidate list. If a combination is not available, the next combination may be checked. The index of the candidate selected in the candidate list may be indicated in the bitstream with a variable length code. Smaller indices may be encoded with shorter length codes. The order of the candidates in the list may affect coding efficiency. Fixing of affine merge candidates may not be optimally constructed for individual video sequencesAnd (4) sequencing.

To solve the above problem, an adaptive order of merging candidates may be implemented. For example, the weight of the CP in the combination is checked. From the examination results, the order of the respective combinations can be determined.

In some embodiments of the present disclosure, all combinations of CPs having the same weight or weight index are assigned the highest priority and are ranked first in the list. The CP has the most diverse weight or combination of weight indices has a lower priority and is ranked later in the list. In some embodiments of the present disclosure, the CPs may be classified into different classes according to CP weights. Combinations where all CPs belong to the same class may be assigned the highest priority and may be ranked first in the list. The CP located in the most diverse combinations may be assigned a lower priority and ranked later in the list.

Based on the above, the more consistent the weight of the CP in a combination, the higher the priority of the combination. For example, for the combination { CPMV₁，CPMV₂，CPMV₃And { CPMV }and₁，CPMV₂，CPMV₄}. If CP₁、CP₂And CP₃Having the same weight or weight index or belonging to the same class, CP₄Have different weights or weight indices or belong to different classes, { CPMV₁，CPMV₂，CPMV₃May be assigned a higher priority. That is, { CPMV, in the list₁，CPMV₂，CPMV₃Are arranged in (CPMV)₁，CPMV₂，CPMV₄Before. Alternatively, if CP₁、CP₂And CP₃All of the weights or weight indexes or corresponding classes are different, but the CP₄Weight or weight index or corresponding class of (1) and CP₁Is the same or the weight index or corresponding class, { CPMV₁，CPMV₂，CPMV₄May be less like and assigned a higher priority. That is, { CPMV, in the list₁，CPMV₂，CPMV₄Are arranged in (CPMV)₁，CPMV₂，CPMV₃Before. If two combinations have the same weight or weight index or classA level of diversity, a default order may be assigned to the two combinations.

In the VVC, the construction affine candidate refers to a candidate constructed by combining adjacent motion information of the respective control points. The following combinations of control points are used to build affine merge candidates in order: { CP₁，CP₂，CP₃}、{CP₁，CP₂，CP₄}、{CP₁，CP₃，CP₄}、{CP₂，CP₃，CP₄}、{CP₁，CP₂}、{CP₁，CP₃}. The encoder and decoder may check the availability of the respective combinations. If available, the build affine merge candidate may be placed into a candidate list. If a combination is not available, the availability of the next combination may be checked. In the prior art, the usability condition is focused on whether the neighboring CU is inter-coded. The BWA weight is not considered.

Consistent with some embodiments of the present disclosure, new availability conditions may be applied. The weights or weight indices of the individual CPs in the combination can be checked. The availability of the combination may be determined from the examination result.

In some embodiments, a combination may be set as an unavailable combination if the weights of two CPs in the combination have different signs (signs). For example, in the combination { CP₁，CP₂，CP₃In, if CP₁Is (-2, 10) and CP₂Is (10, -2), the combination may be set as unavailable. As another example, in the combination { CP₁，CP₂，CP₃In, if CP₁Is (-2, 10) and CP₂Is (4, 4), the combination may be set as unavailable.

In VVC, BWA weights are applied to CU or PU levels. This means that each pixel within a CU or PU has the same weight in the prediction process. However, in affine motion prediction, sub-block based motion compensation may be applied. The motion vectors of the respective sub-blocks within the current CU may be obtained from the motion vectors of the control points of the current CU. A motion compensated interpolation filter is then applied to generate a predictor for each sub-block using the obtained motion vectors. To match the motion vector and BWA weights, sub-block level weights may be obtained according to some embodiments of the present disclosure.

In some embodiments, the weights of the sub-blocks within the current CU or PU may be obtained by the weights of the control points of the current CU or PU. Thus, each sub-block may have a different weight.

In some embodiments, the weight w of the CP may be based on_cp0、w_cp1And w_cp2Interpolating respective pixels w_x,yWherein (x, y) indicates the coordinates of the respective pixel. Various interpolation filters may be used without limitation. To simplify the interpolation process, the weights may be obtained at the sub-block level. For example, (x, y) may be the center coordinates of the individual sub-blocks. Further, the sub-block size may be the same as the sub-block size used in affine motion prediction.

In some embodiments of the present disclosure, a CU or PU may be divided into, for example, 2 or 4 sub-blocks. For each sub-block, the weight of the control point contained in the sub-block may be used for the sub-block. For sub-blocks without control points in the combination of control points currently constructing affine merging candidates, predefined rules may be used to obtain weight values. For example, as shown in FIG. 9A, for the 2-CP combination { CP₁，CP₃A CU or PU 910 may be horizontally partitioned into two sub-blocks 911 and 912. CP (CP)₁The weight W1 of (c) can be used for the upper sub-block 911. CP (CP)₃The weight of W3 may be used for the bottom sub-block 912. As another example, as shown in FIG. 9B, { CP ] is combined for 2-CP₁，CP₂A CU or PU 920 may be vertically partitioned into two sub-blocks 921 and 922. CP (CP)₁May be used for the left sub-block 921. CP (CP)₂The weight of (d) may be used for the right sub-block 922.

Alternatively, for 2-CP combination { CP₁，CP₃As shown in fig. 9C, a CU or PU 930 may be partitioned into four sub-blocks 931, 932, 933, and 934. CP (CP)₁May be used for the upper left sub-block 931. CP (CP)₃May be used for the lower left sub-block 932. The weight of the upper right sub-block 933 may be according to the CP₁Right of (1)Reacquire, or set to a default value. The weight of the lower right sub-block 934 may be according to CP₃Is obtained or set to a default value. Similarly, for a combination of 2 CPs CP₁，CP₂As shown in fig. 9D, the CU or PU 940 may be partitioned into four sub-blocks 941, 942, 943 and 944. CP (CP)₁The weight of (c) may be used for the upper left sub-block 941. CP (CP)₂The weight of (d) may be used for the top right sub-block 942. The weight of the lower left sub-block 943 may be according to the CP₁Is obtained or set to a default value. The weight of the lower right sub-block 944 may be based on CP₂Is obtained or set to a default value.

Exemplary partitions for three CPs are further provided below. As shown in FIG. 10A, CP is combined for 3-CP₁，CP₂，CP₄A CU or PU 1010 may be vertically partitioned into two sub-blocks 1011 and 1012. CP (CP)₁The weight of (d) may be used for the left sub-block 1011. CP (CP)₂The weight of (d) may be used for the right sub-block 1012. For 3-CP combinations { CP₁，CP₃，CP₄As shown in fig. 10B, a CU or PU 1020 may be horizontally partitioned into two sub-blocks 1021 and 1022. CP (CP)₁May be used for the upper sub-block 1021. CP (CP)₃May be used for the lower sub-block 1022. For 3-CP combinations { CP₂，CP₃，CP₄CU or PU 1030 may be horizontally split into sub-blocks 1031 and 1032, as shown in fig. 10C. CP (CP)₂May be used for the top sub-block 1031. CP (CP)₃The weight of (c) may be used for the bottom sub-block 1032. Alternatively, as shown in fig. 10D, CU or PU 1040 may be vertically partitioned into sub-blocks 1041 and 1042. CP (CP)₂The weights of (d) may be used for the right sub-block 1042. CP (CP)₃May be used for the left sub-block 1041.

A CU or PU may be further partitioned into 4 sub-blocks. As shown in FIG. 10E, CP is combined for 3 CPs₁，CP₂，CP₃The CU or PU 1050 may be partitioned into four sub-blocks 1051, 1052, 1053, and 1054. CP (CP)₁The weight of (d) may be used for the upper left sub-block 1051. CP (CP)₂The weight of (d) may be used for the upper right sub-block 1052. CP (CP)₃The weight of (d) may be used for the lower left sub-block 1053. The weight of the lower right sub-block 1054 may be setIs one of the following: CP (CP)₁、CP₂、CP₃Average of the three weights of (a); CP (CP)₂Weight sum CP of₃The average of the weights of (a); CP (CP)₁、CP₂And CP₃The median of the three weights of (a); CP (CP)₁、CP₂And CP₃The one of the three weights of (a) that is least different from the equal weight; CP (CP)₂Weight sum CP of₃The one of the weights of (a) that is least different from the equal weight; equalizing the weight; or another default value.

As shown in FIG. 10F, CP is combined for 3 CPs₁，CP₂，CP₄A CU or PU 1060 may be partitioned into four sub-blocks 1061, 1062, 1063, and 1064. CP (CP)₁The weight of (d) may be used for the upper left sub-block 1061. CP (CP)₂The weight of (d) may be used for the upper right sub-block 1062. CP (CP)₄The weight of (d) may be used for the lower right sub-block 1064. The weight of the lower left sub-block 1063 may be set to one of: the average of all three weights; CP (CP)₁Weight sum CP of₄The average of the weights of (a); the median of the three weights; CP (CP)₁、CP₂、CP₄The one of the three weights of (a) that is least different from the equal weight; CP (CP)₁Weight sum CP of₄The weight of (a) is less different from the equal weight; equalizing the weight; or another default value.

As shown in FIG. 10G, { CP ] is combined for 3 CPs₁，CP₃，CP₄CU or PU 1080 may be partitioned into four sub-blocks 1071, 1072, 1073, and 1074. CP (CP)₁The weight of (c) may be used for the upper left sub-block 1071. CP (CP)₃Can be used for the lower left sub-block 1073, and CP₄The weight of (c) may be used for the lower right sub-block 1074. The weight of the upper right sub-block 1072 may be set to one of: the average of all three weights; CP (CP)₁Weight sum CP of₄The average of the weights of (a); the median of the three weights; CP (CP)₁、CP₃、CP₄The one of the three weights of (a) that is least different from the equal weight; CP (CP)₁Weight sum CP of₄The weight of (a) is less different from the equal weight; equalizing the weight; or anotherA default value.

As shown in FIG. 10H, CP is combined for 3 CPs₂，CP₃，CP₄The CU or PU 1080 may be partitioned into four sub-blocks 1081, 1082, 1083 and 1084. CP (CP)₂Can be used for the upper right sub-block 1082. CP (CP)₃The weight of (d) can be used for the lower left sub-block 1083. CP (CP)₄Can be used for the lower right sub-block 1084. The weight of the top left sub-block 1081 may be set to one of: the average of all three weights; CP (CP)₂Weight sum CP of₃The average of the weights of (a); the median of the three weights for the control point; CP (CP)₂、CP₃、CP₄The one of the three weights of (a) that is least different from the equal weight; CP (CP)₂Weight sum CP of₃The weight of (a) is less different from the equal weight; equalizing the weight; or another default value.

Exemplary partitions for 4 CPs are further provided below. As shown in FIG. 11, for the combination { CP₁，CP₂，CP₃，CP₄A CU or PU 1100 may be partitioned into four sub-blocks 1101, 1102, 1103, and 1104. CP (CP)₁The weight of (c) may be used for the upper left sub-block 1101. The weight of CP2 may be used for the top right sub-block 1102. CP (CP)₃The weight of (c) may be used for the lower left sub-block 1103. CP (CP)₄The weight of (d) may be used for the lower right sub-block 1104.

In some embodiments, a non-transitory computer-readable storage medium comprising instructions is also provided and the instructions may be executed by a device (such as the disclosed encoder and decoder) for performing the above-described method. Common forms of non-transitory media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, or any other FLASH memory, NVRAM, cache, registers, any other memory chip or cartridge, and network versions thereof. The device may include one or more processors (CPUs), input/output interfaces, network interfaces, and/or memory.

It should be noted that relational terms such as "first" and "second" may be used herein only to distinguish one entity or operation from another entity or operation, and do not require or imply any actual relationship or order between such entities or operations. Furthermore, the words "comprising," "having," "containing," and "including," and other similar forms are intended to be equivalent in meaning and open ended in that one or more items following any one of these words are not intended to be an exhaustive list of such item or items, nor are they intended to be limited to only the listed item or items.

As used herein, unless otherwise specifically noted, the term "or" encompasses all possible combinations unless otherwise not feasible. For example, if a claims database may include a or B, then the database may include a or B, or a and B, unless explicitly stated otherwise or not feasible. As a second example, if it is stated that a database may include A, B or C, then the database may include a, or B, or C, or a and B, or a and C, or B and C, or a and B and C, unless explicitly stated otherwise or not feasible.

It is to be understood that the above-described embodiments may be realized by hardware, or software (program code), or a combination of hardware and software. If implemented by software, the software may be stored in the computer-readable medium described above. The software, when executed by a processor, may perform the disclosed methods. The computing unit and other functional units described in this disclosure may be implemented by hardware, or software, or a combination of hardware and software. One of ordinary skill in the art will also appreciate that a plurality of the above modules/units may be combined into one module/unit, and each of the above modules/units may be further divided into a plurality of sub-modules/sub-units.

Some embodiments of the disclosure may be further described using the following clauses:

1. a method for processing video data, the method comprising the steps of:

determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and

bi-directionally predicting the coding unit based on the determined weights.

2. The method of clause 1, wherein the weight is determined based on transmitting a weight index in the bitstream.

3. The method of clause 1, wherein the determined weights include the weight of the build affine merge candidate, and the method further comprises the steps of:

in response to the plurality of control points associated with the build affine merge candidate having one or more weights, determining a weight of the build affine merge candidate based on a weight associated with an upper left control point or a weight associated with an upper right control point.

4. The method of clause 1 or 3, wherein the determined weights include the weight of the build affine merge candidate, and the method further comprises the steps of:

in response to a plurality of control points associated with the build affine merge candidate having the same weight, determining the weights of the plurality of control points as the weight of the build affine merge candidate; or

Determining a default value as a weight of the build affine merge candidate in response to the plurality of control points having different weights.

5. The method of clause 1 or 3, wherein the determined weights include the weight of the build affine merge candidate, and the method further comprises the steps of:

determining weights used by a majority of the plurality of control points from among weights of the plurality of control points corresponding to the build affine merge candidate; and

setting the determined weight as the weight of the constructed affine merging candidate.

6. The method of clause 1 or 3, wherein the determined weights include the weight of the build affine merge candidate, and the method further comprises the steps of:

determining differences between the equal weights and the weights of the plurality of control points, respectively;

determining a first control point having a weight that is least different from the equal weight from the plurality of control points; and

setting the weight of the first control point as the weight of the constructed affine merging candidate.

7. The method of clause 1 or 3, wherein the determined weights include the weight of the build affine merge candidate, and the method further comprises the steps of:

determining an average weight of a plurality of control points corresponding to the constructed affine merge candidate; and

setting the average weight as the weight of the constructed affine merging candidate.

8. The method of any of clauses 1-7, wherein bi-directionally predicting the coding unit based on the determined weights comprises: processing the video data using an affine merge mode, and the method further comprises the steps of:

in constructing an affine merge candidate list, zero motion vectors with equal weight are inserted into the affine merge candidate list before zero motion vectors with unequal weight are inserted.

9. The method of any of clauses 1-7, wherein bi-directionally predicting the coding unit based on the determined weights comprises: processing the video data using an affine merge mode, and the method further comprises the steps of:

determining an order of a plurality of construction affine merge candidates based on weights of control points corresponding to respective construction affine merge candidates among the plurality of construction affine merge candidates in constructing an affine merge candidate list,

the construction affine merge candidates having control points less weighted are assigned a higher priority than the construction affine merge candidates having control points more weighted.

10. The method of clause 9, wherein determining the order of the plurality of constructed affine merge candidates comprises:

in response to the weights of the control points corresponding to each of the two build merge candidates having the same level of diversity, using a default order of the two build merge candidates.

11. The method of clause 9 or 10, further comprising the steps of:

determining availability of a first build affine merge candidate based on weights of control points corresponding to the first build affine merge candidate; and

in response to determining that the first build affine merge candidate is available, adding the first build affine merge candidate to the affine merge candidate list.

12. The method of clause 11, wherein determining the availability of the first build affine merge candidate comprises:

determining whether weights of two control points corresponding to the first build affine merge candidate have different signs; and

determining that the first build affine merge candidate is unavailable in response to determining that the weights of the two control points corresponding to the first build affine merge candidate have different signs.

13. A method for processing video data, the method comprising the steps of:

determining a weight of a first sub-block within a coding unit based on a weight of a control point of the coding unit;

bi-directionally predicting the first sub-block based on the determined weights.

14. The method of clause 13, wherein determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including one of the control points, using a weight of the one of the control points as a weight of the first sub-block, or

In response to the first sub-block not including the control point, determining a weight of the first sub-block according to a predefined rule.

15. The method of clause 13, wherein:

the coding unit is partitioned into four sub-blocks and has two control points; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including one of the two control points, using the weight of the one of the two control points as the weight of the first sub-block, or

In response to the first sub-block not including the two control points, setting a weight of the first sub-block to one of:

the weight of one of the two control points having a shorter distance from the first sub-block, or

A default value.

16. The method of clause 13, wherein:

the coding unit is partitioned into two sub-blocks and has three control points; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including only one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

In response to the first sub-block including at least two of the three control points, using a default value or a weight of one of the two control points as a weight of the first sub-block.

17. The method of clause 13, wherein:

the encoding unit is partitioned into four sub-blocks and has three control points having a first weight, a second weight, and a third weight, respectively; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

in response to the first sub-block including at least one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

In response to the sub-block not including the three control points, setting a weight of the first sub-block to one of:

an average of the first weight, the second weight, and the third weight,

a median of the first weight, the second weight, and the third weight,

the equal weight of the coding units,

one of the first weight, the second weight, and the third weight that is least different from the equal weight,

one of the first weight and the second weight that is less different from the equal weight, or

A default value.

18. A video processing device, the video processing device comprising:

a memory storing instructions; and

a processor configured to execute the instructions to cause the apparatus to:

determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and

bi-directionally predicting the coding unit based on the determined weights.

19. The apparatus of clause 18, wherein the weight is determined based on transmitting a weight index in the bitstream.

20. The apparatus of clause 18, wherein the determined weights comprise the weights of the build affine merge candidates, and the processor is further configured to execute the instructions to cause the apparatus to:

determining a weight of the build affine merge candidate based on a weight associated with an upper left control point or a weight associated with an upper right control point if a plurality of control points associated with the build affine merge candidate have one or more weights.

21. The apparatus of clause 18 or 20, wherein the determined weights comprise the weights of the build affine merge candidates, and the processor is further configured to execute the instructions to cause the apparatus to:

determining weights of a plurality of control points associated with the build affine merge candidate as the weight of the build affine merge candidate if the plurality of control points have the same weight; or

Determining a default value as the weight of the build affine merge candidate if the plurality of control points have different weights.

22. The apparatus of clause 18 or 20, wherein the determined weights comprise the weights of the build affine merge candidates, and the processor is further configured to execute the instructions to cause the apparatus to:

determining weights used by a majority of the plurality of control points from among weights of the plurality of control points corresponding to the build affine merge candidate; and

setting the determined weight as the weight of the constructed affine merging candidate.

23. The apparatus of clause 18 or 20, wherein the determined weights comprise the weights of the build affine merge candidates, and the processor is further configured to execute the instructions to cause the apparatus to:

determining differences between the equal weights and the weights of the plurality of control points, respectively;

determining a first control point having a weight that is least different from the equal weight from the plurality of control points; and

setting the weight of the first control point as the weight of the constructed affine merging candidate.

24. The apparatus of clause 18 or 20, wherein the determined weights comprise the weights of the build affine merge candidates, and the processor is further configured to execute the instructions to cause the apparatus to:

determining an average weight of a plurality of control points corresponding to the constructed affine merge candidate; and

setting the average weight as the weight of the constructed affine merging candidate.

25. The apparatus of any of clauses 18 to 24, wherein bi-directionally predicting the coding unit based on the determined weights comprises: processing the video data using an affine merge mode, and the processor is further configured to execute the instructions to cause the apparatus to:

in constructing an affine merge candidate list, zero motion vectors with equal weight are inserted into the affine merge candidate list before zero motion vectors with unequal weight are inserted.

26. The apparatus of any of clauses 18 to 24, wherein bi-directionally predicting the coding unit based on the determined weights comprises: processing the video data using an affine merge mode, and the processor is further configured to execute the instructions to cause the apparatus to:

determining an order of a plurality of construction affine merge candidates based on weights of control points corresponding to respective construction affine merge candidates among the plurality of construction affine merge candidates in constructing an affine merge candidate list,

the construction affine merge candidates having control points less weighted are assigned a higher priority than the construction affine merge candidates having control points more weighted.

27. The apparatus of clause 26, wherein, in determining the order of the plurality of constructed affine merge candidates, the processor is further configured to execute the instructions to cause the apparatus to:

in response to the weights of the control points corresponding to each of the two build merge candidates having the same level of diversity, using a default order of the two build merge candidates.

28. The apparatus of clause 26 or 27, wherein the processor is further configured to execute the instructions to cause the device to:

determining availability of a first build affine merge candidate based on weights of control points corresponding to the first build affine merge candidate; and

adding the first build affine merge candidate to the affine merge candidate list if the first build affine merge candidate is available.

29. The apparatus of clause 28, wherein, in determining the availability of the first build affine merge candidate, the processor is further configured to execute the instructions to cause the apparatus to:

determining whether weights of two control points corresponding to the first build affine merge candidate have different signs; and

determining that the first build affine merge candidate is unavailable if the weights of the two control points corresponding to the first build affine merge candidate have different signs.

30. A video processing device, the video processing device comprising:

a memory storing instructions; and

a processor configured to execute the instructions to cause the apparatus to:

determining a weight of a first sub-block within a coding unit based on a weight of a control point of the coding unit;

bi-directionally predicting the first sub-block based on the determined weights.

31. The apparatus of clause 30, wherein, in determining the weight of the first sub-block within the coding unit, the processor is further configured to execute the instructions to cause the apparatus to:

if the first sub-block includes one of the control points, using a weight of the one of the control points as a weight of the first sub-block, or

Determining a weight of the first sub-block according to a predefined rule if the first sub-block does not include the control point.

32. The apparatus of clause 30, wherein:

the coding unit is partitioned into four sub-blocks and has two control points; and is

In determining the weight of the first sub-block within the encoding unit, the processor is further configured to execute the instructions to cause the apparatus to:

in response to the first sub-block including one of the two control points, using the weight of the one of the two control points as the weight of the first sub-block, or

In response to the first sub-block not including the two control points, setting a weight of the first sub-block to one of:

the weight of one of the two control points having a shorter distance from the first sub-block, or

A default value.

33. The apparatus of clause 30, wherein:

the coding unit is partitioned into two sub-blocks and has three control points; and is

In determining the weight of the first sub-block within the encoding unit, the processor is further configured to execute the instructions to cause the apparatus to:

in response to the first sub-block including only one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

In response to the first sub-block including at least two of the three control points, using a default value or a weight of one of the two control points as a weight of the first sub-block.

34. The apparatus of clause 30, wherein:

the encoding unit is partitioned into four sub-blocks and has three control points having a first weight, a second weight, and a third weight, respectively; and is

In determining the weight of the first sub-block within the encoding unit, the processor is further configured to execute the instructions to cause the apparatus to:

in response to the first sub-block including at least one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

In response to the sub-block not including the three control points, setting a weight of the first sub-block to one of:

an average of the first weight, the second weight, and the third weight,

a median of the first weight, the second weight, and the third weight,

the equal weight of the coding units,

one of the first weight, the second weight, and the third weight that is least different from the equal weight,

one of the first weight and the second weight that is less different from the equal weight, an

A default value.

35. A non-transitory computer readable medium storing a set of instructions executable by one or more processors of a video processing apparatus to cause the apparatus to perform a method comprising:

determining a weight of at least one of an inherited affine merge candidate, a constructed affine merge candidate, and a zero motion vector of an encoding unit; and

bi-directionally predicting the coding unit based on the determined weights.

36. The non-transitory computer-readable medium of clause 35, wherein the weight is determined based on a weight index transmitted in the bitstream.

37. The non-transitory computer-readable medium of clause 35, wherein the determined weights comprise the weights of the build affine merge candidate, and the set of instructions are executable by the one or more processors of the apparatus to cause the apparatus to further perform the following:

determining a weight of the build affine merge candidate based on a weight associated with an upper left control point or a weight associated with an upper right control point if a plurality of control points associated with the build affine merge candidate have one or more weights.

38. The non-transitory computer-readable medium of clause 35 or 37, wherein the set of instructions is executable by the one or more processors of the device to cause the device to further perform the following:

determining weights of a plurality of control points associated with the build affine merge candidate as the weight of the build affine merge candidate if the plurality of control points have the same weight; and

determining a default value as the weight of the build affine merge candidate if the plurality of control points have different weights.

39. The non-transitory computer-readable medium of clause 35 or 37, wherein the set of instructions is executable by the one or more processors of the device to cause the device to further perform the following:

determining weights used by a majority of the plurality of control points from among weights of the plurality of control points corresponding to the build affine merge candidate; and

setting the determined weight as the weight of the constructed affine merging candidate.

40. The non-transitory computer-readable medium of clause 35 or 37, wherein the set of instructions is executable by one or more processors of the apparatus to cause the apparatus to further perform the following:

determining differences between the equal weights and the weights of the plurality of control points, respectively;

determining a first control point having a weight that is least different from the equal weight from the plurality of control points; and

setting the weight of the first control point as the weight of the constructed affine merging candidate.

41. The non-transitory computer-readable medium of clause 35 or 37, wherein the set of instructions is executable by the one or more processors of the device to cause the device to further perform the following:

determining an average weight of a plurality of control points corresponding to the constructed affine merge candidate; and

setting the average weight as the weight of the constructed affine merging candidate.

42. The non-transitory computer-readable medium of any one of clauses 35-41, wherein the set of instructions is executable by the one or more processors of the apparatus to cause the apparatus to further perform the following:

in constructing an affine merge candidate list, zero motion vectors with equal weight are inserted into the affine merge candidate list before zero motion vectors with unequal weight are inserted.

43. The non-transitory computer-readable medium of any one of clauses 35-41, wherein the set of instructions is executable by the one or more processors of the apparatus to cause the apparatus to further perform the following:

determining an order of a plurality of construction affine merge candidates based on weights of control points corresponding to respective construction affine merge candidates among the plurality of construction affine merge candidates in constructing an affine merge candidate list,

the construction affine merge candidates having control points less weighted are assigned a higher priority than the construction affine merge candidates having control points more weighted.

44. The non-transitory computer-readable medium of clause 43, wherein the set of instructions is executable by the one or more processors of the device to cause the device to further perform the operations of:

if the weights of the control points corresponding to each of the two build merge candidates have the same diversity level, the default order of the two build merge candidates is used.

45. The non-transitory computer-readable medium of clause 43 or 44, wherein the set of instructions is executable by the one or more processors of the device to cause the device to further perform the following:

determining availability of a first build affine merge candidate based on weights of control points corresponding to the first build affine merge candidate; and

adding the first build affine merge candidate to the affine merge candidate list if the first build affine merge candidate is available.

46. The non-transitory computer-readable medium of clause 45, wherein the set of instructions is executable by the one or more processors of the device to cause the device to further perform the operations of:

determining whether weights of two control points corresponding to the first build affine merge candidate have different signs; and

determining that the first build affine merge candidate is unavailable if the weights of the two control points corresponding to the first build affine merge candidate have different signs.

47. A non-transitory computer readable medium storing a set of instructions executable by one or more processors of a video processing apparatus to cause the apparatus to perform a method comprising:

determining a weight of a first sub-block within a coding unit based on a weight of a control point of the coding unit;

bi-directionally predicting the first sub-block based on the determined weights.

48. The non-transitory computer readable medium of clause 47, wherein determining the weight of the first sub-block within the coding unit comprises:

if the first sub-block includes one of the control points, using a weight of the one of the control points as a weight of the first sub-block, or

Determining a weight of the first sub-block according to a predefined rule if the first sub-block does not include the control point.

49. The non-transitory computer-readable medium of clause 47, wherein the set of instructions are executable by the one or more processors of the device to cause the device to further perform the operations of:

partitioning the coding unit into four sub-blocks, wherein the coding unit has two control points; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

if the sub-block includes one of the two control points, using the weight of the one of the two control points as the weight of the first sub-block, or

If the first sub-block does not include the two control points, setting a weight of the first sub-block to one of:

the weight of one of the two control points having a shorter distance from the first sub-block, or

A default value.

50. The non-transitory computer-readable medium of clause 47, wherein the set of instructions are executable by the one or more processors of the device to cause the device to further perform the operations of:

partitioning the coding unit into two sub-blocks, wherein the coding unit has three control points; and is

The step of determining weights of the sub-blocks within the coding unit comprises:

if the first sub-block includes only one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

If the first sub-block includes two of the three control points, using a default value or a weight of one of the two control points as a weight of the first sub-block.

51. The non-transitory computer-readable medium of clause 47, wherein the set of instructions are executable by the one or more processors of the device to cause the device to further perform the operations of:

partitioning the coding unit into four sub-blocks, wherein the coding unit has three control points with a first weight, a second weight, and a third weight; and is

The step of determining the weight of the first sub-block within the coding unit comprises:

if the first sub-block includes at least one of the three control points, using the weight of the one of the control points as the weight of the first sub-block, or

If the first sub-block does not include the three control points, setting a weight of the first sub-block to one of:

an average of the first weight, the second weight, and the third weight,

a median of the first weight, the second weight, and the third weight,

the equal weight of the coding units,

one of the first weight, the second weight, and the third weight that is least different from the equal weight,

one of the first weight and the second weight that is less different from the equal weight, or

A default value.

In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Certain adaptations and modifications of the described embodiments can be made. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims. The order of steps shown in the figures is also intended for illustrative purposes only and is not intended to be limited to any particular order of steps. Thus, those skilled in the art will appreciate that the steps may be performed in a different order while performing the same method.

In the drawings and specification, there have been disclosed exemplary embodiments. However, many variations and modifications may be made to these embodiments. Accordingly, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation. The disclosed embodiments are not limited to the above examples, but rather are defined by the following claims, along with their full scope of equivalents.