Binarization of segmentation in connection with extended quadtree
阅读说明：本技术 与扩展四叉树相关的分割的二值化 (Binarization of segmentation in connection with extended quadtree ) 是由 张凯 张莉 刘鸿彬 王悦 于 2019-08-28 设计创作，主要内容包括：一种视频处理的方法,该方法涉及与扩展四叉树相关的分割的二值化,包括：对于视频区域的视频块的编码表示和视频块之间的转换,确定对视频区域的至少一个视频块启用了扩展四叉树(EQT)分割；以及基于该确定来执行转换。EQT分割将视频块划分成多个子块,使得多个子块中的至少一个子块具有与视频块的一半宽度乘以视频块的一半高度不同的尺寸。视频块的编码表示是使用来自包括码字和分割类型之间的映射的表的码字而表示的。(A method of video processing involving binarization of a segmentation associated with an extended quadtree, comprising: determining that Extended Quadtree (EQT) partitioning is enabled for at least one video block of a video region for a transition between an encoded representation of the video block and the video block; and performing a conversion based on the determination. The EQT partitioning divides the video block into a plurality of sub-blocks such that at least one sub-block of the plurality of sub-blocks has a size that is different from half the width of the video block multiplied by half the height of the video block. An encoded representation of a video block is represented using codewords from a table that includes a mapping between codewords and partition types.)
1. A method of video processing, comprising:
determining that extended quadtree EQT partitioning is enabled for at least one video block of a video region for a transition between an encoded representation of the video block and the video block; and
a conversion is performed based on the determination,
wherein the EQT partitioning divides the video block into a plurality of sub-blocks such that at least one of the plurality of sub-blocks has a size that is different from half a width of the video block multiplied by half a height of the video block, and
wherein an encoded representation of the video block is represented using codewords from a table comprising a mapping between codewords and partition types.
2. The method of claim 1, wherein the table is as follows:
。
3. The method of claim 2, wherein the first and second light sources are selected from the group consisting of,
wherein the first flag and the second flag are included in an encoded representation of the video block,
wherein the first flag indicates that either EQT partitioning or BT partitioning is applied to the video block,
wherein the second flag indicates that horizontal partitioning or vertical partitioning is applied to the video block, and
wherein the first flag precedes the second flag in the encoded representation of the video block.
4. The method of claim 1, wherein the table is as follows:
5. the method of claim 4, wherein the first and second light sources are selected from the group consisting of,
wherein the first flag and the second flag are included in an encoded representation of the video block,
wherein the first flag indicates whether horizontal partitioning or vertical partitioning is applied to the video block,
wherein the second flag indicates that the EQT partition or the BT partition is applied to the video block, and
wherein the first flag precedes the second flag in the encoded representation of the video block.
6. The method of claim 1, wherein the table is as follows:
。
7. The method of claim 1, wherein the table is as follows:
。
8. The method of claim 1, wherein the table is as follows:
9. the method of claim 1, wherein the table is as follows:
。
10. The method of claim 1, wherein the table is as follows:
。
11. The method of claim 10, wherein the first and second light sources are selected from the group consisting of,
wherein the first flag, the second flag and the third flag are included in an encoded representation of the video block,
wherein the first flag indicates that either EQT partitioning or BT partitioning is applied to the video block,
wherein the second flag indicates whether an equal EQT partition or a non-equal partition is applied to the video block in response to the EQT partition being applied,
wherein the third flag indicates that horizontal partitioning or vertical partitioning is applied to the video block, and
wherein the first flag precedes the second flag and the second flag precedes the third flag in the encoded representation of the video block.
12. The method of claim 1, wherein the table is as follows:
。
13. The method of claim 12, wherein the first and second light sources are selected from the group consisting of,
wherein the first flag, the second flag and the third flag are included in an encoded representation of the video block,
wherein the first flag indicates whether horizontal partitioning or vertical partitioning is applied to the video block,
wherein the second flag indicates that either EQT partitioning or BT partitioning is applied to the video block,
wherein the third flag indicates that an equal EQT partition or a non-equal partition is applied to the video block in response to the EQT partition being applied, and
wherein the first flag precedes the second flag and the second flag precedes the third flag in the encoded representation of the video block.
14. The method of claim 1, wherein the table is as follows:
。
15. The method of claim 1, wherein the table is as follows:
16. the method of claim 1, wherein the table is as follows:
。
17. The method of claim 1, wherein the table is as follows:
。
18. The method of claim 1, wherein the video region comprises one or more video blocks.
19. The method of claim 1, wherein performing a conversion comprises generating an encoded representation of a video block.
20. The method of claim 1, wherein performing a conversion comprises generating a video block.
21. A decoder apparatus comprising a processor, wherein the processor is configured to implement the method of one or more of claims 1 to 20.
22. An encoder apparatus comprising a processor, wherein the processor is configured to implement the method of one or more of claims 1 to 20.
23. A computer-readable program medium having code stored thereon, the code comprising instructions, wherein the instructions, when executed by a processor, cause the processor to implement the method of one or more of claims 1 to 20.
Technical Field
This document relates to the field of video coding and compression.
Background
Despite advances in video compression technology, digital video still accounts for the largest bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video usage is expected to continue to grow.
Disclosure of Invention
Techniques for the coexistence of extended quadtree partitioning and binary tree partitioning of rule-based video blocks are disclosed. In an advantageous aspect, the rules enable efficient signaling of segmented video data in a compressed video bitstream.
A first exemplary method of video processing, comprising: determining that Extended Quad Tree (EQT) partitioning is enabled for at least one video block of a video region for a transition between an encoded representation of the video block and the video block; and performing a transform based on the determination, wherein the EQT partitioning divides the video block into a plurality of sub-blocks such that at least one of the plurality of sub-blocks has a size that is different from half the width of the video block multiplied by half the height of the video block.
A second exemplary method of video processing, comprising: determining that Extended Quadtree (EQT) partitioning is enabled for at least one video block of a video region for a transition between an encoded representation of the video block and the video block; and performing a transformation based on the determination, wherein the EQT partitioning divides the video block into a plurality of sub-blocks such that at least one of the plurality of sub-blocks has a size that is different from half the width of the video block multiplied by half the height of the video block, and wherein the encoded representation of the video block is represented using a codeword from a table that includes a mapping between codewords and partition types.
A third exemplary method of video processing, comprising: determining that Extended Quadtree (EQT) partitioning is enabled for at least one video block of a video region for a transition between an encoded representation of the video block and the video block; and performing a conversion based on the determination, wherein the EQT partitioning divides the video block into a plurality of sub-blocks such that at least one of the plurality of sub-blocks has a size that is different from half a width of the video block multiplied by half a height of the video block, wherein the encoded representation of the video block comprises one or more bits of a Binary bit string encoded using a Context Adaptive Binary Arithmetic Coding (CABAC) technique, and wherein the Binary bit string indicates a type of partitioning of the partitioned video block.
A fourth exemplary method of video processing, comprising: determining, based on a rule, whether Extended Quadtree (EQT) partitioning is enabled for at least one video block of the video region and/or whether the video block of the video region is partitioned using Binary Tree (BT) partitioning, for a transition between an encoded representation of the video block of the video region and the video block; and performing a conversion based on the determination.
In another example aspect, a visual media encoder apparatus is disclosed. The encoder implements the above method.
In yet another aspect, a visual media decoder device is disclosed. The decoder implements the above method.
In yet another aspect, the disclosed technology is embodied in the form of computer executable program code and stored on a computer readable medium.
These and other aspects are further described throughout this document.
Drawings
Fig. 1 is a block diagram illustrating an example of a video encoder implementation.
Fig. 2 illustrates macroblock partitioning in the h.264 video coding standard.
Fig. 3 shows an example of dividing a Coding Block (CB) into Prediction Blocks (PB).
Fig. 4 shows an example implementation for subdividing a CTB into CBs and Transform Blocks (TBs). The solid lines indicate CB boundaries and the dashed lines indicate TB boundaries, including example CTBs with partitions and corresponding quadtrees.
Fig. 5 shows an example of a Quad Tree Binary Tree (QTBT) structure for segmenting video data.
Fig. 6 shows an example of (a) quadtree splitting (b) vertical binary tree splitting (c) horizontal binary tree splitting (d) vertical center side ternary tree splitting (e) horizontal center side ternary tree splitting.
Fig. 7A to 7K show examples of dividing a pixel block.
Fig. 8A to 8D show examples of extended quadtree splitting.
Fig. 9A and 9B illustrate example partitions of a Largest Coding Unit (LCU) of video data.
Fig. 10 shows an example of generalized ternary tree partitioning of an M × N block.
Fig. 11A to 11F depict examples of excluding the BT mode from the EQT division.
Fig. 12 shows an example of dividing blocks.
FIG. 13 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.
Fig. 14A-14B are flow diagrams illustrating examples of methods for video processing.
Fig. 15A-15B depict examples of rules for EQT segmentation.
FIG. 16 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.
Detailed Description
1. Overview
This patent document relates to picture/video Coding, and more particularly to a partitioning structure, i.e. how to divide one Coding Tree Unit (CTU) into multiple Coding Units (CUs) at picture/slice edges. It can be applied to existing video coding standards, such as HEVC, or to-be-completed standards (multi-functional video coding). It may also be applied to future video coding standards or video codecs.
2. Introduction to
Video coding standards have evolved primarily by developing the well-known ITU-T and ISO/IEC standards. ITU-T produces H.261 and H.263, ISO/IEC produces MPEG-1 and MPEG-4 Video, and both organizations jointly produce H.262/MPEG-2 Video and H.264/MPEG-4 Advanced Video Coding (AVC) and H.265/HEVC standards. Starting from h.262, video coding standards are based on hybrid video coding structures, in which temporal prediction plus transform coding is utilized. An example of a typical HEVC encoder framework is depicted in fig. 1.
Partition tree structure in 2.1 H.264/AVC
The core of the coding layer in the previous standard is a macroblock, which contains a 16 × 16 block of luma samples and, in the typical case of 4:2:0 color sampling, two corresponding blocks of 8 × 8 chroma samples. FIG. 2 shows an example of MB partitioning in H.264/AVC.
Intra-coded blocks exploit spatial correlation between pixels using spatial prediction. Two segmentations are defined: 16 × 16 and 4 × 4.
Inter-coded blocks use temporal prediction instead of spatial prediction by estimating motion between pictures. Motion may be estimated independently for a 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, 4 × 4 (see fig. 5) macroblock or any sub-macroblock partition thereof. Only one Motion Vector (MV) per sub-macroblock partition is allowed.
Partition tree structure in 2.2 HEVC
In HEVC, CTUs are divided into CUs by using a quadtree structure, denoted as a coding tree, to accommodate various local characteristics. The decision whether to encode a picture region using inter-picture (temporal) or intra-picture (spatial) prediction is made at the CU level. Each CU may be further divided into one, two, or four PUs depending on the PU partition type. Within one PU, the same prediction process is applied and the relevant information is sent to the decoder on a PU basis. After obtaining the residual block by applying a prediction process based on the PU partition type, the CU may be partitioned into Transform Units (TUs) according to another quadtree structure similar to the coding tree of the CU. One of the key features of the HEVC structure is that it has multiple partitioning concepts, including CU, PU and TU.
In the following, various features involved in hybrid video coding using HEVC are highlighted as follows.
1) Coding Tree unit and Coding Tree Block (CTB) structure: the analog structure in HEVC is a Coding Tree Unit (CTU) that has a size selected by the encoder and may be larger than a conventional macroblock. The CTU consists of luma CTB and corresponding chroma CTB and syntax elements. The size L × L of the luminance CTB may be chosen to be L ═ 16, 32, or 64 samples, with larger sizes generally enabling better compression. HEVC then supports the partitioning of CTBs into smaller blocks using a tree structure and quadtree-like signaling.
2) Coding Unit (CU) and Coding Block (CB): the quad-tree syntax of the CTU specifies the size and location of its luma and chroma CBs. The root of the quadtree is associated with the CTU. Therefore, the size of the luminance CTB is the maximum supported size of the luminance CB. The division of the CTUs into luma and chroma CBs is signaled jointly. One luma CB and typically two chroma CBs and associated syntax form a Coding Unit (CU). A CTB may contain only one CU or may be partitioned to form multiple CUs, and each CU has an associated tree of partitioned Prediction Units (PUs) and Transform Units (TUs).
3) Prediction unit and Prediction Block (PB): the decision whether to encode a picture region using inter-frame or intra-frame prediction is made at the CU level. The PU partition structure has a root at the CU level. The luma and chroma CB may then be further partitioned and predicted from the luma and chroma Prediction Blocks (PB) depending on the base prediction type decision. HEVC supports variable PB sizes from 64 × 64 down to 4 × 4 samples.
Fig. 3 depicts an example of a mode that divides CB into PBs subject to certain size limitations. For intra-predicted CBs, only M and M/2 are supported.
4) TU and transform block: the prediction residual is encoded using a block transform. The TU tree structure has a root at the CU level. The luma CB residual may be the same as the luma Transform Block (TB) or may be further divided into smaller luma TBs. The same applies to chroma TB. Integer basis functions similar to those of the Discrete Cosine Transform (DCT) are defined for square TB sizes 4 × 4, 8 × 8, 16 × 16, and 32 × 32. For the 4 × 4 Transform of the luma intra prediction residual, an integer Transform derived from the form of a Discrete Sine Transform (DST) is alternately specified.
Fig. 4 shows an example of subdividing a CTB into CBs and Transform Blocks (TBs). The solid line indicates the CB boundary and the dashed line indicates the TB boundary. Left: CTB and its segmentation, right: a corresponding quadtree.
2.3 quad Tree plus binary Tree Block Structure with larger CTU in JEM
To explore future Video coding techniques for super HEVC, Joint Video exploration team (jfet) was created jointly by VCEG and MPEG in 2015. Since then, JFET has adopted many new approaches and incorporated them into a reference software named Joint Exploration Model (JEM).
2.3.1 QTBT (quad Tree plus binary Tree) Block segmentation Structure
Unlike HEVC, the QTBT structure removes the concept of multiple partition types, i.e., it removes the separation of CU, PU and TU concepts and supports greater flexibility of CU partition shapes. In the QTBT block structure, a CU may have a square or rectangular shape. As shown in fig. 5, a Coding Tree Unit (CTU) is first divided by a quadtree structure. The leaf nodes of the quadtree are further partitioned by a binary tree structure. There are two partition types in binary tree partitioning, symmetric horizontal partitioning and symmetric vertical partitioning. The binary tree leaf nodes are called Coding Units (CUs) and the segmentation is used for prediction and transform processing without any further segmentation. This means that CU, PU and TU have the same block size in the QTBT coding block structure. In JEM, a CU sometimes consists of Coded Blocks (CBs) of different color components, e.g., one CU contains one luma CB and two chroma CBs in the case of P-and B-slices of the 4:2:0 chroma format, and sometimes consists of CBs of a single component, e.g., one CU contains only one luma CB or only two chroma CBs in the case of I-slices.
The following parameters are defined for the QTBT segmentation scheme.
-CTU size: root node size of quadtree, same concept as in HEVC
-MinQTSize: minimum allowed quadtree leaf node size
-MaxBTSize: maximum allowed binary tree root node size
-MaxBTDepth: maximum allowed binary tree depth
-MinBTSize: minimum allowed binary tree leaf node size
In one example of the QTBT segmentation structure, the CTU size is set to 128 × 128 luma samples with two corresponding 64 × 64 chroma sample blocks, MinQTSize is set to 16 × 16, MaxBTSize is set to 64 × 64, MinBTSize (for both width and height) is set to 4 × 4, and MaxBTDepth is set to 4. Quadtree partitioning is first applied to CTUs to generate quadtree leaf nodes. The quad tree leaf nodes may have sizes from 16 × 16 (i.e., MinQTSize) to 128 × 128 (i.e., CTU size). If the leaf quadtree node is 128 x 128, it is not further partitioned by the binary tree because it exceeds the MaxBTSize (i.e., 64 x 64). Otherwise, the leaf quadtree nodes may be further partitioned by the binary tree. Thus, the leaf nodes of the quadtree are also the root nodes of the binary tree, and their binary tree depth is 0. When the binary tree depth reaches MaxBTDepth (i.e., 4), no further partitioning is considered. When the width of the binary tree node is equal to MinBTSize (i.e., 4), no further horizontal partitioning is considered. Similarly, when the height of the binary tree node is equal to MinBTSize, no further vertical partitioning is considered. The leaf nodes of the binary tree are further processed by prediction and transformation processes without any further partitioning. In JEM, the maximum CTU size is 256 × 256 luma samples.
Fig. 5 (left) shows an example of block segmentation by using QTBT and fig. 5 (right) shows the corresponding tree representation. The solid lines indicate quad-tree partitions and the dashed lines indicate binary tree partitions. In each partition (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which partition type (i.e., horizontal or vertical) to use, where 0 indicates horizontal partitioning and 1 indicates vertical partitioning. For a quadtree partition, there is no need to indicate the partition type, since a quadtree partition always partitions a block horizontally and vertically to generate 4 sub-blocks having equal sizes.
Furthermore, the QTBT scheme supports the ability for luminance and chrominance to have separate QTBT structures. Currently, luminance and chrominance CTBs in one CTU share the same QTBT structure for P-and B-stripes. However, for an I-stripe, luminance CTB is partitioned into CUs by a QTBT structure, and chrominance CTB is partitioned into chrominance CUs by another QTBT structure. This means that a CU in an I-slice consists of either a coded block of the luma component or a coded block of the two chroma components, and a CU in a P-slice or a B-slice consists of coded blocks of all three color components.
In HEVC, inter prediction of small blocks is restricted to reduce memory access for motion compensation so that bi-prediction is not supported for 4 × 8 and 8 × 4 blocks and inter prediction is not supported for 4 × 4 blocks. In the QTBT of JEM, these restrictions are removed.
2.4 VVC Tree
Tree types other than quadtrees and binary trees are also supported. In an embodiment, two other Triplet Tree (TT) segmentations, i.e. horizontal and vertical center-side triplets, are introduced, as shown in fig. 6(d) and 6 (e).
Fig. 6(a) -6(e) depict the following. Fig. 6(a) -quadtree splitting, fig. 6(b) vertical binary tree splitting, fig. 6(c) horizontal binary tree splitting, fig. 6(d) vertical center side ternary tree splitting, and fig. 6(e) horizontal center side ternary tree splitting.
There are two levels of trees, the area tree (quadtree) and the prediction tree (binary or ternary). The CTU is first partitioned by a Region Tree (RT). The RT leaves may be further partitioned using a Prediction Tree (PT). PT leaves may be further partitioned with PT until a maximum PT depth is reached. The PT leaf is a basic coding unit. For convenience, it is still referred to as a CU. A CU cannot be further partitioned. Both prediction and transform are applied to the CU in the same way as JEM. The whole partition structure is called a multi-type tree.
2.5 extended quadtree example
1. An extended quad-tree (EQT) partitioning structure corresponding to a block partitioning process including an extended quad-tree partitioning process for a block of video data, wherein the extended quad-tree partitioning structure represents partitioning the block of video data into final sub-blocks, and when the extended quad-tree partitioning process decides to apply extended quad-tree partitioning to a given block, the given block is always divided into four sub-blocks; decoding the final subblock based on the video bitstream; and decoding a block of video data based on the final sub-block decoded according to the derived EQT structure.
The EQT splitting process may be recursively applied to a given block to generate an EQT leaf node. Alternatively, when the EQT is applied to a block, it may be further divided into BT and/or QT and/or TT and/or EQT and/or other kinds of partition trees for each of the sub-blocks resulting from the EQT.
b. In one example, EQT and QT may share the same depth increment process and the same limit of leaf node size. In this case, when the size of a node reaches the minimum allowed quadtree leaf node size or the EQT depth of the node reaches the maximum allowed quadtree depth, the segmentation of one node may be implicitly terminated.
c. Alternatively, EQT and QT may share different depth increment processes and/or leaf node size constraints. Partitioning of a node by an EQT may be terminated implicitly when the size of the node reaches the minimum allowed EQT leaf node size or the EQT depth associated with the node reaches the maximum allowed EQT depth. Further, in one example, the EQT depth and/or the minimum allowed EQT leaf node size may be signaled in a Sequence Parameter Set (SPS), and/or a Picture Parameter Set (PPS), and/or a slice header, and/or a CTU, and/or a region, and/or a slice, and/or a CU.
d. Instead of using the current quadtree partitioning applied to a square, for a block with an mxn (M and N are non-zero positive integer values, equal or not) size, in EQT, a block may be equally partitioned into four partitions, such as M/4 xn or mxn/4 (examples are depicted in fig. 7A and 7B), or equally partitioned into four partitions with the partition size depending on the maximum and minimum values of M and N. In one example, a 4 × 32 block may be divided into four 4 × 8 sub-blocks, and a 32 × 4 block may be divided into four 8 × 4 sub-blocks.
e. Instead of using the current quadtree partitioning applied to the tiles, for blocks having sizes of M × N (M and N are non-zero positive integer values, equal or not), in EQT, a block may be partitioned unequally into four partitions, such as two partitions of size equal to (M × w0/w) × (N × h0/h) and two other partitions of size equal to (M × w0)/w) × (N (h-h 0)/h).
For example, w0 and w may be equal to 1 and 2, respectively, i.e., the width is halved, while the height may use other ratios than 2:1 to get the sub-blocks. Examples of this are depicted in fig. 7C and 7E. Alternatively, h0 and h may be equal to 1 and 2, respectively, i.e., height halved, while width may use other ratios than 2: 1. Examples of this are depicted in fig. 7D and 7F.
Fig. 7G and 7H show two alternative examples of quadtree splitting.
Fig. 7I shows a more general case of quadtree partitioning with partitions of different shapes.
Fig. 7J and 7K show a general example of fig. 7A and 7B.
In general, fig. 7A to 7K show the following dimensions. (a) M × N/4(b) M/4 × N, (c) subblock width fixed to M/2, height equal to N/4 or 3N/4, smaller for the top two partitions (d) subblock height fixed to N/2, width equal to M/4 or 3M/4, smaller for the left two partitions (e) subblock width fixed to M/2, height equal to 3N/4 or N/4, smaller for the bottom two partitions (f) subblock height fixed to N/2, width equal to 3M/4 or M/4, smaller for the right two partitions (g) M × N/4 and M/2 × N/2; (h) n × M/4 and N/2 × M/2(i) M1 × N1, (M-M1) × N1, M1 × (N-N1) and (M-M1) × (N-N1) (j) M × N1, M × N2, M × N3 and M × N4, where N1+ N2+ N3+ N4 ═ N (k) M1 × N, M2 × N, M3 × N and M4 × N, where M1+ M2+ M3+ M4 ═ M.
2. A Flexible Tree (FT) partition structure corresponding to a block partition process including an FT partition process for a block of video data, wherein the FT partition structure represents a partition of the block of video data into final sub-blocks, and when the FT partition process decides to apply FT partition to a given block, the given block is divided into K sub-blocks, where K may be greater than 4; decoding the final subblock based on the video bitstream; and decoding a block of video data based on the final sub-block decoded according to the derived FT structure.
a. The FT segmentation process may be recursively applied to a given block to generate FT-leaf nodes. The segmentation of a node is implicitly terminated when the node reaches a minimum allowed FT leaf node size or the FT depth associated with the node reaches a maximum allowed FT depth.
b. Alternatively, when FT is applied to a block, it may be further divided into BT, and/or QT, and/or EQT, and/or TT, and/or other kinds of partition trees for each of the sub-blocks resulting from FT.
c. Alternatively, additionally, FT depth or minimum allowed FT leaf node size or minimum allowed split size of FT may be signaled in a Sequence Parameter Set (SPS), and/or Picture Parameter Set (PPS), and/or slice header, and/or CTU, and/or region, and/or slice, and/or CU.
d. Similar to EQT, all sub-blocks resulting from FT partitioning may have the same size; alternatively, the size of the different sub-blocks may be different.
e. In one example, K is equal to 6 or 8. Some examples are depicted in fig. 8A-8D, which show examples of FT segmentation: (a0K ═ 8, M/4 × (N)/2 (b) K ═ 8, M/2 × (N)/4 (c) K ═ 6, M/2 × (N)/2, and M/4 × (N)/2, (d) K ═ 6, M/2 × (N)/2, and M/2 × (N)/4
3. For TT, the partition restriction in the horizontal or vertical direction may be removed.
a. In one example, a Generalized TT (GTT) partitioning pattern may be defined as a division of both horizontal and vertical. Examples are shown in fig. 9A and 9B.
4. The proposed method can be applied under certain conditions. In other words, when the condition(s) are not satisfied, there is no need to signal the segmentation type.
f. Alternatively, the proposed method can be used to replace existing split tree types. Alternatively, furthermore, the proposed method may only be used as a replacement under certain conditions.
g. In one example, the condition may include a picture and/or a slice type; and/or block size; and/or a coding mode; and/or whether a block is located at a picture/slice boundary.
h. In one example, the proposed EQT may be treated in the same manner as QT. In this case, when the indication that the partition tree type is QT, a more flag/indication of the detailed quadtree partition mode may be further signaled. Alternatively, the EQT may be considered as an additional segmentation mode.
i. In one example, signaling the partitioning methods of the EQT or FT or GTT may be conditional, i.e., one or some of the EQP/FT/GTT partitioning methods may not be used in some cases, and no bits corresponding to signaling these partitioning methods are signaled.
2.6 edge handling in JFET-K0287
In jfet-K0287, a boundary processing method is proposed for multi-function video coding (VVC). A similar approach was also used for AVS-3.0.
The solution for constrained quadtree boundary partitioning in VVC is not optimized. jfet-K0287 proposes a boundary segmentation method using a conventional block segmentation syntax to maintain the continuity of a Context Adaptive Binary Arithmetic Coding (CABAC) engine and to match picture boundaries.
The generic boundary partition obtains the following rules (both encoder and decoder):
the exact same partitioning syntax (e.g., VTM-1.0 as in fig. 10) for normal blocks (non-bounded) is used for the bounded block (bounded block), the syntax needs to remain unchanged.
If no partition pattern is resolved for the boundary CU, Forced Boundary Partitioning (FBP) is used to match the picture boundaries.
After forced boundary segmentation (non-single boundary segmentation), there is no further segmentation.
FIG. 10 shows an example of unchanged syntax and changed semantics for generic boundary segmentation. The forced boundary segmentation is described as follows:
if the size of the block is larger than the maximum allowed BT size, then FBP is performed using a forced QT in the current forced partition level;
otherwise, if the bottom-right sample of the current CU is below the bottom picture boundary without extending the right boundary, then FBP is performed using the forcing level BT in the current forcing partition level;
otherwise, if the bottom-right sample of the current CU is located to the right of the right picture boundary, but not below the bottom boundary, then FBP is performed using forced vertical BT in the current forced partition level;
otherwise, if the lower right sample of the current CU is located to the right of the right picture boundary and below the bottom boundary, then FBP is performed using forced QT in the current forced partition level.
3 limitations of the current technology
There may be some redundancy between the partitions of EQT and QT/BT/TT. For example, for an M × N block, it may be divided into vertical BT three times (first into two M/2 × N partitions, then for each M/2 × N partition, further vertical BT partitions are applied) to get four M/4 × N partitions. Furthermore, to obtain four M/4N partitions, the block may choose to use EQT directly as in FIG. 7B.
2. How to efficiently signal the EQT in the bitstream remains an unresolved problem.
4 example embodiment
To address this issue, as well as other possible improvements in video coding techniques, several embodiments are proposed to handle the case of EQT. The benefits of these embodiments will sometimes be explicitly described, but will also be understood by those skilled in the art.
The following list should be considered as an example to explain the general concept. These examples should not be construed in a narrow sense. Furthermore, these embodiments may be combined in any manner.
Example 1 in one embodiment, when EQT is enabled, certain BT partitioning is not allowed under certain conditions. In this case, an indication that BT splitting is not allowed may be signaled in the methods from example 2 and/or example 3.
a. When EQT as M × N/4 is enabled (as depicted in fig. 7A), in one example as shown in fig. 11A, if the lower part is divided from the parent block with horizontal BT and the upper part is also divided with horizontal BT, horizontal BT is not allowed in that block.
b. When EQT as M/4 × N is enabled (as depicted in fig. 7B), in one example as shown in fig. 11B, if the right portion is divided from the parent block with vertical BT and the left portion is also divided with vertical BT, then vertical BT is not allowed in that block.
c. When EQT as M × N/4 is enabled (as depicted in fig. 7A), in one example as shown in fig. 11C, if the center portion is divided from the parent block by level TT, then level BT is not allowed in that block.
d. When EQT as M/4 × N is enabled (as depicted in fig. 7B), in one example as shown in fig. 11D, vertical BT is not allowed in the parent block if the center portion is divided from the block by vertical TT.
e. When EQT is enabled as in the leftmost side of fig. 7G, in one example as shown in fig. 11E, vertical BT is not allowed in the block of the center portion divided from the parent block by the horizontal TT.
f. When EQT is enabled as in the leftmost side of fig. 7H, in one example as shown in fig. 11F, horizontal BT is not allowed in the block if the center portion is divided from the parent block by vertical TT.
Example 2. in some embodiments, syntax elements for a partition are binarized in the same manner if some partitions are not allowed and if they are allowed. The syntax signaling method is not specifically designed to avoid disallowed segmentation. A canonical encoder must avoid choosing an unallowable partition. A bitstream that includes an impermissible partition is considered an irregular bitstream.
a. In one example, disallowed partitions cannot be signaled in a canonical bitstream. The disallowed partitioning is signaled and the bitstream is determined by the decoder to be non-canonical.
Example 3. in some embodiments, no partitioning is allowed, i.e., no code words to indicate a partitioning that is not allowed, can be signaled from the encoder to the decoder.
a. In one example as shown in fig. 11A, if the lower portion is divided from the parent block with the level BT and the upper portion is also divided with the level BT, there is no codeword to represent the level BT of the block.
b. In one example as shown in fig. 11B, if the right portion is divided from the parent block with a vertical BT and the left portion is also divided with a vertical BT, there is no codeword to represent the vertical BT of the block.
c. In one example as shown in fig. 11C, if the center portion is divided from the parent block by the level TT, there is no codeword to represent the level BT of the block.
d. In one example as shown in fig. 11D, if the center portion is divided from the parent block by vertical TT, there is no codeword to represent the vertical BT of the block.
e. In one example as shown in fig. 11E, if the center portion is divided from the parent block by horizontal TT, there is no codeword to represent the vertical BT of the block.
f. In one example as shown in fig. 11F, if the center portion is divided from the parent block by the vertical TT, there is no codeword to represent the horizontal BT of the block.
Example 4. in one example, the binarization of the segmentation is shown in fig. 12. The code words are represented as follows:
TABLE 1
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal BT
3
0101
Vertical BT
4
0110
Horizontal EQT
5
0111
Vertical EQT
It should be noted that all table exchanges for all "0" and "1" are equivalent.
a. In one example, a flag is signaled to indicate whether BT or EQT applies, followed by a flag indicating whether horizontal or vertical splitting applies.
b. Alternatively, a flag is signaled to indicate whether horizontal or vertical partitioning applies followed by a flag indicating whether BT or EQT applies. An example of a representation of a codeword is as follows:
TABLE 2
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal BT
3
0101
Horizontal EQT
4
0110
Vertical BT
5
0111
Vertical EQT
Example 5. if there is no codeword to represent horizontal BT, as claimed in example 3. a. An example of a representation of a codeword is as follows:
TABLE 3
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
010
Horizontal EQT
3
0110
Vertical BT
4
0111
Vertical EQT
An alternative example of a representation of a codeword is as follows:
TABLE 4
Example 6. if there is no codeword to represent vertical BT, as claimed in example 3. b. An example of a representation of a codeword is as follows:
TABLE 6
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal BT
3
0101
Vertical EQT
4
0110
Horizontal EQT
An alternative example of a representation of a codeword is as follows:
TABLE 7
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal BT
3
0101
Horizontal EQT
4
011
Vertical EQT
Example 7 in one example, four types of EQTs (an equal horizontal EQT as shown in fig. 7A, an equal vertical EQT as shown in fig. 7B, a non-equal horizontal EQT as shown in fig. 7G, and a non-equal vertical EQT as shown in fig. 7H) may be selected by the encoder and signaled to the decoder.
a. In one example, a flag is signaled to indicate whether BT or EQT is applied. If an EQT is selected, a subsequent flag is signaled to indicate whether an equal EQT or a non-equal EQT is selected. After those one or two flags, a flag indicating whether horizontal or vertical partitioning is signaled. An exemplary codeword table is as follows.
TABLE 8
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal BT
3
0101
Vertical BT
4
01100
Equal level EQT
5
01101
Equal vertical EQT
6
01110
Non-equal level EQT
7
01111
Non-equal vertical EQT
a. Alternatively, a flag is signaled to indicate whether horizontal or vertical partitioning applies followed by a flag indicating whether BT or EQT applies. If an EQT is selected, a subsequent flag is signaled to indicate whether an equal EQT or a non-equal EQT is selected. An exemplary codeword table is as follows.
TABLE 9
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal BT
3
01010
Equal level EQT
4
01011
Non-equal level EQT
5
0110
Vertical BT
6
01110
Equal vertical EQT
7
01111
Non-equal vertical EQT
b. If there is no codeword to represent the level BT, as claimed in example 3. a. An example of a representation of a codeword is as follows:
watch 10
Another example of a representation of a codeword is as follows:
TABLE 11
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
010
Vertical BT
3
01100
Equal level EQT
4
01101
Equal vertical EQT
5
01110
Non-equal level EQT
6
01111
Non-equal vertical EQT
c. If there is no codeword to represent vertical BT, as claimed in example 3. b. An example of a representation of a codeword is as follows:
TABLE 12
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
010
Horizontal BT
3
01100
Equal level EQT
4
01101
Equal vertical EQT
5
01110
Non-equal level EQT
6
01111
Non-equal vertical EQT
An alternative example of a representation of a codeword is as follows:
watch 13
Code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal BT
3
01010
Equal level EQT
4
01011
Non-equal level EQT
6
0110
Equal vertical EQT
7
0111
Non-equal vertical EQT
Example 8 each bin (bit) of the bin string used for the indication of the partition type may be CABAC encoded with one or more contexts.
a. In one example, no further partitioning (no partitioning) mode is considered as one of the partition types.
b. Alternatively, only a portion of the binary digits of the binary digit string may be coded in the context, and the remaining binary digits may be coded in a bypass mode (i.e., without the context).
c. For binary bits encoded with context, one or more contexts may be used.
d. The context may depend on:
(a) the position or index of the binary bit.
(b) Segmentation of spatial/temporal neighboring blocks.
(c) The current partition depth (e.g., QT depth/BT depth) of the current block.
(d) Split depths (e.g., QT depth/BT depth) of spatial/temporal neighboring blocks and/or spatial/temporal non-neighboring blocks.
(e) Coding modes of spatial/temporal neighboring blocks.
(f) Slice type/picture type.
(g) Color components.
(h) Statistics of partition types based on previously coded blocks.
Example 9 whether and how redundant segmentation is not allowed may depend on the color component, such as Y/Cb/Cr.
Example 10 whether and how redundancy partitioning is not allowed may be signaled from the encoder to the decoder in VPT/SPS/PPS/slice header/CTU/CU/group of CTUs/group of CUs.
Example 11 in some embodiments, EQT is not allowed after BT partitioning. If one or more parent blocks of the current block are divided by BT, EQT is not allowed for the current block. Fig. 15A and 15B show two examples of block division from a parent block, where fig. 15A is divided with a BT of one depth, and fig. 15B is divided with a BT of two depths.
a. The signaling of the partitioning depends on whether the block is partitioned from the parent block with BT at one or more depths. In one example, if a block is partitioned from a parent block with BT at one or more depths, there is no codeword to represent the EQT.
b. Alternatively, EQT is allowed after BT partitioning.
Example 12 in some embodiments, EQT is not allowed after TT partitioning. EQT is not allowed for the current block if one or more parent blocks of the current block are divided by TT.
a. Alternatively, EQT is allowed after TT partitioning.
Example 13 in some embodiments, EQT is not allowed after QT partitioning. If one or more parent blocks of the current block are divided with QT, EQT is not allowed for the current block.
a. Alternatively, the EQT partition is followed by allowing the EQT.
Example 14. in some embodiments, QT is not allowed after EQT partitioning. QT is not allowed for the current block if one or more parent blocks of the current block are partitioned with EQT.
a. Alternatively, QT is allowed after EQT partitioning.
Example 15. in some embodiments, TT is not allowed after EQT partitioning. If one or more parent blocks of the current block are divided with EQT, TT is not allowed for the current block.
a. Alternatively, TT is allowed after EQT division.
Example 16. in some embodiments, BT is not allowed after EQT partitioning. BT is not allowed for the current block if one or more parent blocks of the current block are divided with EQT.
a. Alternatively, BT is allowed after EQT partitioning.
Example 17 in some embodiments, EQT is not allowed after EQT partitioning. If one or more parent blocks of the current block are divided with EQT, then EQT is not allowed for the current block.
a. Alternatively, the EQT is allowed after the EQT division.
Example 18 in some embodiments, EQT is not allowed if the shape or size of the current block meets some conditions. (assuming that the width and height of the current block are W and H, and T1, T2, and T are some integers)
a. If W > -T1 and H > -T2, then EQT is not allowed;
b. if W > -T1 or H > -T2, then EQT is not allowed;
c. if W < ═ T1 and H < ═ T2, then EQT is not allowed;
d. if W < ═ T1 or H < ═ T2, then EQT is not allowed;
e. if W × H < ═ T, then EQT is not allowed;
f. if W × H > -T, not allowing EQT;
g. if H < ═ T, the level EQT is not allowed;
h. if H > -T, the level EQT is not allowed;
i. if W < ═ T, then vertical EQT is not allowed;
j. if W > -T, vertical EQT is not allowed;
example 19 in some embodiments, EQT is allowed if the shape or size of the current block meets some condition. (assuming that the width and height of the current block are W and H, and T1, T2, and T are some integers)
a. If W > -T1 and H > -T2, then EQT is allowed;
b. if W > -T1 or H > -T2, then EQT is allowed;
c. if W < ═ T1 and H < ═ T2, then EQT is allowed;
d. if W < ═ T1 or H < ═ T2, then EQT is allowed;
e. if W × H < ═ T, then EQT is allowed;
f. if W × H > -T, then EQT is allowed;
g. if H < ═ T, then allow level EQT;
h. if H > -T, the level EQT is allowed;
i. if W < ═ T, then vertical EQT is allowed;
j. if W > -T, then vertical EQT is allowed;
example 20 in some embodiments, EQT is not allowed if the depth of the current block satisfies some conditions.
a. If the division depth is less than T, not allowing EQT;
b. if the partition depth > T, not allowing EQT;
c. if the QT partition depth is less than T, not allowing EQT;
d. if QT partition depth > T, EQT is not allowed;
e. if BT division depth > T, not allowing EQT;
f. if the BT division depth is less than T, not allowing EQT;
g. if the TT division depth is less than T, not allowing the EQT;
h. if TT division depth > T, not allowing EQT;
i. if the EQT partition depth is less than T, not allowing the EQT;
j. if EQT partition depth > T, not allowing EQT;
example 21 in some embodiments, EQT is allowed if the depth of the current block meets some condition.
a. If the division depth is less than T, allowing EQT;
b. if the partition depth > T, then allowing EQT;
c. if the QT partition depth is less than T, allowing EQT;
d. if QT partition depth > T, then EQT is allowed;
e. if BT division depth > T, allowing EQT;
f. if the BT division depth is less than T, allowing the EQT;
g. if the TT division depth is less than T, allowing the EQT;
h. if TT division depth > T, then allowing EQT;
i. if the EQT division depth is less than T, allowing the EQT;
if the EQT partition depth > T, allowing the EQT;
FIG. 13 illustrates a block diagram of an example embodiment of a hardware device 1300 that may be used to implement portions of the presently disclosed technology. Hardware device 1300 may be a laptop, smart phone, tablet, camcorder, or other type of device capable of processing video or other types of visual media (e.g., images). The device 1300 includes a processor or controller 1302 that processes data and a memory 1304 that communicates with the processor 1302 and stores and/or buffers data. For example, the processor 1302 may include a Central Processing Unit (CPU) or a Micro Controller Unit (MCU). In some embodiments, processor 1302 may include a Field-Programmable Gate-Array (FPGA). In some embodiments, device 1300 includes or communicates with a Graphics Processing Unit (GPU), a Video Processing Unit (VPU), and/or a wireless communication Unit for various visual and/or communication data Processing functions of the smartphone device. For example, the memory 1304 may include and store processor executable code that, when executed by the processor 1302, configures the device 1300 to perform various operations, such as, for example, receiving information, commands, and/or data, processing the information and data, and transmitting or providing the processed information/data to another device, such as an actuator or an external display. To support various functions of the device 1300, the memory 1304 may store information and data such as instructions, software, values, images, and other data processed or referenced by the processor 1302. For example, various types of Random Access Memory (RAM) devices, Read Only Memory (ROM) devices, flash Memory devices, and other suitable storage media may be used to implement the storage functions of Memory 1304. Device 1300 may further include dedicated video processing circuitry 1306 for performing recalculation functions such as transformation and decoding. In some embodiments, the video processing circuit 1306 may be implemented within the processor 1302. In some embodiments, video processing circuit 1306 may be a Graphics Processor Unit (GPU).
Fig. 14A is a flow diagram of an example method 1400 of processing video data. The video data may be, for example, digital video or digital images. The method 1400 comprises: for a transition between an encoded representation of a video block of a video region and the video block, it is determined (1402) that Extended Quadtree (EQT) partitioning is enabled for at least one video block of the video region. The method 1400 also includes performing (1404) the converting based on the determining, wherein the EQT partitioning divides the video block into a plurality of sub-blocks such that at least one sub-block of the plurality of sub-blocks has a size that is different from half the width of the video block multiplied by half the height of the video block.
In some embodiments, the determining of the transition between the encoded representation of the video block of the video region and the video block includes determining that Binary Tree (BT) partitioning of the video block is not allowed based on a rule. In some embodiments, the rule specifies: for an M × N video block, where M and N are integers, and for EQT partitioning in an M × N/4 structure, if the lower portion of the video block is partitioned from the parent block with a horizontal BT and the upper portion of the video block is also partitioned with a horizontal BT, the horizontal BT is not allowed in the lower portion of the video block.
In some embodiments, the rule specifies: for an M × N video block, where M and N are integers, and for EQT partitioning in an M/4 × N structure, vertical BT is not allowed in the right portion of the video block if the right portion of the video block is partitioned from the parent block with vertical BT and the left portion of the video block is also partitioned with vertical BT.
In some embodiments, the rule specifies: for an M × N video block, where M and N are integers, and for an EQT split of an M × N/4 structure, if the video block includes a central portion divided from a parent block with a horizontal Ternary Tree (TT) split, horizontal BT is not allowed in the central portion of the video block.
In some embodiments, the rule specifies: for an M × N video block, where M and N are integers, and for EQT partitioning in an M/4 × N structure, vertical BT is not allowed in the central portion of the video block if the video block includes a central portion that is partitioned from a parent block with vertical Ternary Tree (TT) partitioning.
In some embodiments, the rule specifies: for an mxn video block, where M and N are integers, vertical BT is not allowed in the central portion of the video block if the video block includes a central portion divided from a parent block by horizontal treeing (TT) partitioning.
In some embodiments, the rule specifies: for an mxn video block, where M and N are integers, horizontal BT is not allowed in the central portion of the video block if the video block includes a central portion divided from a parent block by vertical treeing (TT) partitioning.
In some embodiments, syntax elements of the encoded representation of the video block are generated using a binarization process that is the same as a binarization process for syntax elements of another encoded representation of another video block to which the rule is not applied.
In some embodiments, the bitstream includes an encoded representation of the video block and includes codewords of a BT segmentation process for the video block that is not allowed based on the rule, wherein an indication of BT segmentation for the video block that is not allowed based on the rule is not included in the bitstream.
In some embodiments, the bitstream includes an encoded representation of a video block and does not include codewords of the BT segmentation process for video blocks that are not allowed based on the rule.
In some embodiments, if the lower portion of the video block is divided from the parent block at level BT and the upper portion of the video block is also divided at level BT, the bitstream does not include codewords of level BT of the lower portion of the block.
In some embodiments, if the right portion of the video block is divided from the parent block with a vertical BT and the left portion of the video block is also divided with a vertical BT, the bitstream does not include codewords of the vertical BT of the right portion of the block.
In some embodiments, if a video block includes a central portion divided from a parent block by a horizontal Ternary Tree (TT), the bitstream does not include codewords of the horizontal BT of the central portion of the block.
In some embodiments, if a video block includes a central portion divided from a parent block by a vertical Ternary Tree (TT), the bitstream does not include codewords of the vertical BT of the central portion of the block.
In some embodiments, if a video block includes a central portion divided from a parent block by a horizontal Ternary Tree (TT), the bitstream does not include codewords of vertical BT for the central portion of the block.
In some embodiments, if a video block includes a central portion divided from a parent block by a vertical Ternary Tree (TT), the bitstream does not include codewords of the horizontal BT of the central portion of the block. In some embodiments, the video region comprises one or more video blocks.
In some embodiments, for the method shown in fig. 14A, the encoded representation of the video block is represented using a codeword from a table that includes a mapping between codewords and partition types.
In some embodiments, the table is as follows:
in some embodiments, a first flag and a second flag are included in the encoded representation of the video block, the first flag indicating that EQT partitioning or BT partitioning is applied to the video block, the second flag indicating that horizontal partitioning or vertical partitioning is applied to the video block, and the first flag precedes the second flag in the encoded representation of the video block.
In some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal Binary Tree (BT)
3
0101
Horizontal EQT
4
0110
Vertical BT
5
0111
Vertical EQT
In some embodiments, a first flag and a second flag are included in the encoded representation of the video block, the first flag indicating that horizontal partitioning or vertical partitioning is applied to the video block, the second flag indicating that EQT partitioning or BT partitioning is applied to the video block, and the first flag precedes the second flag in the encoded representation of the video block.
In some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
010
Horizontal EQT
3
0110
Vertical Binary Tree (BT)
4
0111
Vertical EQT
In some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
010
Vertical Binary Tree (BT)
3
0110
Horizontal EQT
4
0111
Vertical EQT
In some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal Binary Tree (BT)
3
0101
Vertical EQT
4
0110
Horizontal EQT
In some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal Binary Tree (BT)
3
0101
Horizontal EQT
4
011
Vertical EQT
In some embodiments, the table is as follows:
in some embodiments, a first flag, a second flag, and a third flag are included in an encoded representation of a video block, the first flag indicating that EQT partitioning or BT partitioning is applied to the video block, the second flag indicating that equal EQT partitioning or non-equal partitioning is applied to the video block in response to EQT partitioning being applied, the third flag indicating that horizontal partitioning or vertical partitioning is applied to the video block, and the first flag precedes the second flag and the second flag precedes the third flag in the encoded representation of the video block.
In some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
0100
Horizontal Binary Tree (BT)
3
01010
Equal level EQT
4
01011
Non-equal level EQT
5
0110
Vertical BT
6
01110
Equal vertical EQT
7
01111
Non-equal vertical EQT
In some embodiments, a first flag, a second flag, and a third flag are included in an encoded representation of a video block, the first flag indicating that horizontal partitioning or vertical partitioning is applied to the video block, the second flag indicating that EQT partitioning or BT partitioning is applied to the video block, the third flag indicating that equal EQT partitioning or non-equal partitioning is applied to the video block in response to EQT partitioning being applied, and the first flag precedes the second flag and the second flag precedes the third flag in the encoded representation of the video block.
In some embodiments, the table is as follows:
in some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
010
Vertical Binary Tree (BT)
3
01100
Equal level EQT
4
01101
Equal vertical EQT
5
01110
Non-equal level EQT
6
01111
Non-equal vertical EQT
In some embodiments, the table is as follows:
code word
Binary bit string
Segmentation type
0
1
Quadtree
1
00
Without division
2
010
Horizontal Binary Tree (BT)
3
01100
Equal level EQT
4
01101
Equal vertical EQT
5
01110
Non-equal level EQT
6
01111
Non-equal vertical EQT
In some embodiments, the table is as follows:
in some embodiments, the video region comprises one or more video blocks. In some embodiments, performing the conversion includes generating an encoded representation of the video block. In some embodiments, performing the conversion includes generating a video block.
In some embodiments, for the method shown in fig. 14A, the encoded representation of the video block includes one or more bits of a bin string encoded using a Context Adaptive Binary Arithmetic Coding (CABAC) technique, and the bin string indicates a partition type of the partitioned video block.
In some embodiments, CABAC techniques encode a binary string to indicate a no partition mode as one partition type of a partitioned video block.
In some embodiments, the one or more bits include a first set of bits and a second set of bits, wherein the first set of bits is encoded with CABAC techniques, the second set of bits is encoded with bypass mode without utilizing context, and the first set of bits is different from the second set of bits.
In some embodiments, the one or more bits are encoded using a CABAC technique that utilizes one or more contexts. In some embodiments, the one or more contexts used for encoding depend on at least one of: (a) a location or index of bits, (2) a partition of spatially or temporally adjacent video blocks, (3) a current partition of video blocks, (4) a partition depth of spatially or temporally adjacent video blocks or a partition depth of spatially or temporally non-adjacent video blocks, (5) a coding mode of spatially or temporally adjacent video blocks, (6) a slice type or picture type of video blocks, (7) a color component of video blocks, or (8) a statistic of partition types from previously coded video blocks. In some embodiments, the video region comprises one or more video blocks. In some embodiments, performing the conversion includes generating an encoded representation of the video block. In some embodiments, performing the conversion includes generating a video block.
Fig. 14B is a flow diagram of an example method 1420 of processing video data. The video data may be, for example, digital video or digital images. The method 1420 includes: for a transition between an encoded representation of a video block of a video region and the video block, it is determined (1422) based on a rule whether Extended Quadtree (EQT) partitioning is enabled for at least one video block of the video region and/or whether a Binary Tree (BT) partitioning is used to partition the video blocks of the video region. The method 1420 further includes performing (1424) a transformation based on the determination.
In some embodiments, the rule specifies that EQT partitioning is not allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by BT partitioning. In some embodiments, the encoded representation of the video block does not include the codeword of the EQT partition in response to the video block being partitioned from the one or more parent video blocks using BT partitioning at the one or more depths. In some embodiments, the rule specifies that EQT partitioning is allowed for a video block after one or more parent video blocks of the video block are partitioned by BT partitioning.
In some embodiments, the rule specifies that EQT partitioning is not allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by a Ternary Tree (TT) partitioning. In some embodiments, the rule specifies that EQT partitioning is allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by a Ternary Tree (TT) partitioning. In some embodiments, the rule specifies that EQT partitioning is not allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by Quadtree (QT) partitioning.
In some embodiments, the rule specifies that EQT partitioning is allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by Quadtree (QT) partitioning. In some embodiments, the rule specifies that after one or more parent video blocks of a video block are divided by EQT partitioning, no Quadtree (QT) partitioning is allowed for the video block. In some embodiments, the rule specifies that a Quadtree (QT) partition is allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by EQT partitioning. In some embodiments, the rule specifies that after one or more parent video blocks of a video block are divided by EQT partitioning, no Ternary Tree (TT) partitioning is allowed for the video block.
In some embodiments, the rule specifies that after one or more parent video blocks of a video block are divided by EQT partitioning, a Ternary Tree (TT) division is allowed for the video block. In some embodiments, the rule specifies that BT partitioning is not allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by EQT partitioning. In some embodiments, the rule specifies that BT partitioning is allowed for video blocks after one or more parent video blocks of the video blocks are partitioned by EQT partitioning. In some embodiments, the rule specifies that EQT partitioning is not allowed for a video block after one or more parent video blocks of the video block are partitioned by EQT partitioning. In some embodiments, the rule specifies that EQT partitioning is allowed for the video block after one or more parent video blocks of the video block are partitioned by EQT partitioning.
In some embodiments, the rule is based on whether the video block is a luma block or a chroma block. In some embodiments, the rule is signaled in a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a slice header, a Coding Tree Unit (CTU), a Coding Unit (CU), a group of CTUs, or a group of CUs.
In some embodiments, the rule specifies that EQT partitioning is not allowed if the dimensionality of the video block satisfies one of the following conditions: (a) if W > -T1 and H > -T2, then EQT is not allowed; (b) if W > -T1 or H > -T2, then EQT is not allowed; (c) if W < ═ T1 and H < ═ T2, then EQT is not allowed; (d) if W < ═ T1 or H < ═ T2, then EQT is not allowed; (e) if W × H < ═ T, then EQT is not allowed; (f) if W × H > -T, not allowing EQT; (g) if H < ═ T, the level EQT is not allowed; (h) if H > -T, the level EQT is not allowed; (i) if W < ═ T, then vertical EQT is not allowed; and (j) disallowing vertical EQT if W > ═ T, where the width and height of the video block in pixels are W and H, respectively, and where T1, T2, and T are integers.
In some embodiments, the rule specifies that EQT partitioning is allowed if the shape or size of the video block satisfies one of the following conditions: (a) if W > -T1 and H > -T2, then EQT is allowed; (b) if W > -T1 or H > -T2, then EQT is allowed; (c) if W < ═ T1 and H < ═ T2, then EQT is allowed; (d) if W < ═ T1 or H < ═ T2, then EQT is allowed; (e) if W × H < ═ T, then EQT is allowed; (f) if W × H > -T, then EQT is allowed; (g) if H < ═ T, then allow level EQT; (h) if H > -T, the level EQT is allowed; (i) if W < ═ T, then vertical EQT is allowed; and (j) if W > ═ T, then allowing vertical EQT, where the width and height of the video block in pixels are W and H, respectively, and where T1, T2, and T are integers.
In some embodiments, the rule specifies that EQT partitioning is not allowed if the depth of the video block satisfies one of the following conditions: (a) if the division depth is less than T, not allowing EQT; (b) if the partition depth > T, not allowing EQT; (c) if the QT partition depth is less than T, not allowing EQT; (d) if QT partition depth > T, EQT is not allowed; (e) if BT division depth > T, not allowing EQT; (f) if the BT division depth is less than T, not allowing EQT; (g) if the TT division depth is less than T, not allowing the EQT; (h) if TT division depth > T, not allowing EQT; (i) if the EQT partition depth is less than T, not allowing the EQT; and (j) disallowing EQT if the EQT partition depth > T, where T is an integer.
In some embodiments, the rule specifies that EQT partitioning is allowed if the depth of the video block satisfies one of the following conditions: (a) if the division depth is less than T, allowing EQT; (b) if the partition depth > T, then allowing EQT; (c) if the QT partition depth is less than T, allowing EQT; (d) if QT partition depth > T, then EQT is allowed; (e) if BT division depth > T, allowing EQT; (f) if the BT division depth is less than T, allowing the EQT; (g) if the TT division depth is less than T, allowing the EQT; (h) if TT division depth > T, then allowing EQT; (i) if the EQT division depth is less than T, allowing the EQT; and (j) allowing the EQT if the EQT partition depth > T, where T is an integer.
In some embodiments, the video region comprises one or more video blocks. In some embodiments, EQT partitioning is enabled for one video block in the video region or for multiple video blocks in the video region. In some embodiments, the EQT partitioning divides the video block into a plurality of sub-blocks such that at least one sub-block of the plurality of sub-blocks has a size that is different from half the width of the video block multiplied by half the height of the video block. In some embodiments, EQT partitioning divides a video block into four sub-blocks having a height that is the same as the height of the video block. In some embodiments, EQT partitioning divides a video block into four sub-blocks having the same width as the width of the video block. In some embodiments, performing the conversion includes generating an encoded representation of the video block. In some embodiments, performing the conversion includes generating a video block.
In some embodiments, the determining operations 1402, 1422 may include receiving a bitstream via a network connection or a locally stored copy of the bitstream, where the bitstream includes an encoded representation of the video block. In some embodiments, for example, during encoding, the bitstream may be received in the form of previously encoded bits and may be reconstructed from a local memory of the encoder.
In some embodiments, for example, in a visual media decoder device, the performing 1404, 1424 can include generating uncompressed visual data (e.g., video or images) and storing it in a display memory or displaying it to a user interface.
In some embodiments, for example, in a visual media encoder apparatus, performing the conversion 1404, 1424 can include generating a compressed bitstream representation of the visual media and storing the compressed bitstream for future use or to transfer the compressed bitstream to memory or another device over a communication connection.
In some embodiments, a visual media decoder device, comprising: a processor configured to implement the above-described method(s) 1400, 1420, and additional features thereof. In some embodiments, a visual media encoder apparatus, comprising: a processor configured to implement the above-described method(s) 1400, 1420, and additional features thereof.
In some embodiments, a computer readable program medium may have code stored thereon. The code includes instructions that, when executed by the processor, cause the processor to implement the method(s) 1400, 1420, and additional features thereof described throughout this document.
From the foregoing it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the invention. Accordingly, the presently disclosed technology is not limited, except as by the appended claims.
With respect to the methods described herein, a minimum allowed leaf node size or a maximum allowed quadtree depth may be indicated in a field representing a bitstream of encoded video. The segmentation process for a given block may be selectively performed based on the condition that the given block is encoded. The condition may be related to the coding mode used to code a given block or a size parameter associated with a given block.
Using the disclosed techniques, in some embodiments, a method of continuously partitioning a video block may include partitioning the block using one of a plurality of partitioning possibilities including BT, TT, QT, or EQT. Various rules described herein may be used to control which segmentation possibilities are used at which depth of segmentation of the block. Furthermore, repeated end results of the segmentation are avoided, allowing signaling of the final segmentation based on a smaller number of bits.
FIG. 16 is a block diagram of an example video processing system in which the disclosed techniques may be implemented. Fig. 16 is a block diagram illustrating an example video processing system 1600 in which various techniques disclosed herein may be implemented. Various embodiments may include some or all of the components of system 1600. The system 1600 can include an input 1602 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. The input 1602 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of Network interfaces include wired interfaces such as ethernet, Passive Optical Network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.
The system 1600 can include an encoding component 1604 that can implement various encoding methods described in this document. The encoding component 1604 may reduce the average bit rate of the video from the input 1602 to an output of the encoding component 1604 to produce an encoded representation of the video. Thus, the encoding technique is sometimes referred to as a video compression or video transcoding technique. The output of the encoding component 1604 may be stored or transmitted via communications over a connection as represented by component 1606. A stored or communicatively transmitted bitstream (or encoded) representation of video received at input 1602 may be used by component 1608 to generate pixel values or displayable video for transmission to display interface 1610. The process of generating user-viewable video from a bitstream representation is sometimes referred to as video decompression. Further, while certain video processing operations are referred to as "encoding" operations or tools, it should be understood that encoding tools or operations are used at the encoder and that corresponding decoding tools or operations that reverse the encoded results will be performed by the decoder.
Examples of a peripheral bus Interface or a display Interface may include a Universal Serial Bus (USB) or a High Definition Multimedia Interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, and the like. The techniques described in this document may be embodied in various electronic devices, such as mobile phones, laptops, smart phones, or other devices capable of performing digital data processing and/or video display.
The disclosed and other embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.
A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (Application Specific Integrated Circuit).
Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.
Only some embodiments and examples are described and other embodiments, enhancements and variations can be made based on what is described and illustrated in this patent document.
- 上一篇：一种医用注射器针头装配设备
- 下一篇：一种基于酒店的用户数据分布式存储方法