Intra-partition coding concept

文档序号：653520 发布日期：2021-04-23 浏览：11次中文

阅读说明：本技术 分区内编码概念 (Intra-partition coding concept ) 是由圣地亚哥·德·卢桑·埃尔南德斯菲利普·默克勒海科·施瓦尔茨德特勒夫·马尔佩托马斯·威于 2019-07-12 设计创作，主要内容包括：通过提供内预测编码技术来进行图像的基于块编码,根据所述内预测编码技术,使用特定内编码模式,通过将预定块分割为分区来对所述图像的特定块进行内预测编码,为了重构的目的,使用针对所述特定块用信号通知的内预测编码模式顺序地对所述分区进行空间预测,随后使用预测残差对由此获得的预测器进行校正,使得针对先前分区,在处理下一分区、随后处理当前分区时所述样本的重构对于解码器可用。描述了各种改进。(Block-based coding of an image is performed by providing an intra-prediction coding technique according to which a particular intra-coding mode is used, an intra-prediction coding is performed on a particular block of the image by dividing a predetermined block into partitions, the partitions are sequentially spatially predicted using the intra-prediction coding mode signaled for the particular block for reconstruction purposes, and the thus obtained predictor is subsequently corrected using a prediction residual, such that reconstruction of the samples is available to a decoder when processing the next partition followed by processing the current partition for the previous partition. Various improvements are described.)

1. A decoder for block-based decoding of images (12) from a data stream (14) is configured to

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

Sequentially reconstructing the partitions of the predetermined blocks according to a predetermined partition order (126) sequentially traversing the partitions by, for a current partition and before proceeding with a subsequent partition,

deriving (122) a predictor for the current partition by populating the current partition in dependence on one or more already reconstructed samples neighboring the current partition in a manner dependent on the intra coding mode;

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition,

wherein the decoder

Is configured to select the predetermined partitioning order (126) for the predetermined block from at least two different orders depending on the data stream.

2. The decoder of claim 1, configured to

Setting the predetermined partition order (126) in dependence on syntax elements signaled for the predetermined block, in addition to the intra-coding mode.

3. The decoder of any of claims 1 or 2, configured to

Selecting, for the deriving of the predictor, the one or more already reconstructed samples from at least two different orders in a manner dependent on a selection of the predetermined partition order (126) for the predetermined block.

4. The decoder of claim 3, configured to

The selection of the one or more already reconstructed samples is performed in dependence on the selection of the predetermined partition order (126) in such a way that in case of one of the at least two different orders the one or more already reconstructed samples are located on two opposite sides of the current partition and in case of another of the at least two different orders the one or more already reconstructed samples are located on only one of the two opposite sides.

5. Decoder according to any of claims 3 or 4,

wherein one of the at least two different orders traverses the partitions starting from the partition furthest from the top left corner of the predetermined block and another of the at least two different orders traverses the partitions starting from the partition closest to the top left corner of the predetermined block.

6. Decoder according to any of claims 1 to 5,

is configured to perform the segmentation along a predetermined dimension (104) such that the partitions are as wide as the predetermined block (80) in a direction perpendicular to the predetermined dimension (104), and to select the width of the partitions measured along the predetermined dimension (105) from at least two different width settings depending on the data stream.

7. A decoder for block-based decoding of images (12) from a data stream (14) is configured to

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition,

wherein the decoder

Is configured to perform the segmentation along a predetermined dimension (104) such that the partitions are as wide as the predetermined block (80) in a direction perpendicular to the predetermined dimension (104), and to select the width of the partitions measured along the predetermined dimension (104) from at least two different width settings depending on the data stream.

8. The decoder according to any of claims 6 or 7, configured to perform said selection depending on the intra coding mode for the predetermined block.

9. The decoder according to any of claims 6 to 8, configured to perform said selection depending on whether the intra coding mode for the predetermined block is an angular mode.

10. The decoder of any of claims 6 to 8, configured to perform the selection in dependence on an index in the data stream for the predetermined block that indexes on the at least two different width settings.

11. Decoder according to any of claims 1 to 10,

wherein the decoder is configured to decode the prediction residuals from the data stream for all partitions before sequentially reconstructing the partitions of the predetermined block.

12. A decoder for block-based decoding of an image (12) from a data stream (14), configured to:

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition,

wherein the decoder is configured to decode the prediction residuals from the data stream for all partitions before sequentially reconstructing the partitions of the predetermined block.

13. Decoder according to any of claims 1 to 12,

wherein the decoder is configured, when deriving the predictor for the current partition, to use, in a not yet clipped state, reconstructed samples of a partition preceding the current partition according to the predetermined partition order, the reconstructed samples being samples among one or more already reconstructed samples adjacent to the current partition, wherein the reconstructed samples are clipped from the not yet clipped state to a state clipped to an allowed sample value range to finally reconstruct the predetermined block.

14. A decoder for block-based decoding of images (12) from a data stream (14) is configured to

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition,

wherein the decoder is configured, when deriving the predictor for the current partition, to use reconstructed samples of a partition preceding the current partition according to the predetermined partition order in a not yet clipped state, the reconstructed samples being samples among one or more already reconstructed samples adjacent to the current partition, wherein the reconstructed samples are clipped from the not yet clipped state to a state clipped to an allowed range of sample values to finally reconstruct the predetermined block.

15. Decoder according to any of claims 1 to 14,

wherein the decoder is configured to decode the prediction residual from the data stream in a transform domain and to retransform it into a spatial domain for use in correcting the predictor.

16. Decoder according to any of the claims 1 to 15,

wherein the decoder is configured to decode the prediction residual from the data stream in a transform domain on a partition-by-partition basis.

17. Decoder according to any of claims 1 to 16,

wherein the decoder is configured to decode the prediction residual from the data stream in a transform domain by, for each partition,

decoding an encoded partition flag (188) from the data stream;

setting prediction residuals for respective partitions to zero if the coded partition flag is not set, an

Decoding a transform of the prediction residual of a corresponding partition from the data stream if the coded partition flag is set.

18. The decoder according to claim 17, wherein,

is configured to sequentially decode the encoded partition flags for the partitions from the data stream and infer that an encoded partition flag for a last partition in partition order is set.

19. Decoder according to any of claims 17 or 18,

configured to decode encoded partition flags for respective partitions from the data stream by using context-dependent entropy decoding that uses contexts that depend on encoded partition flags decoded for previous partitions that precede the respective partitions in the predetermined partition order.

20. The decoder according to any of claims 1 to 19, configured to decode the intra coding mode (116) for the predetermined block (80) by,

deriving a list of most probable intra-prediction modes, the list forming a suitable subset of a set of intra-prediction modes supported by the decoder,

decoding a most probable mode list pointer from the data stream, the pointer pointing into the list of most probable intra-prediction modes.

21. Decoder according to any of the claims 1 to 19, the predetermined block being an intra-prediction block of a first type, wherein the decoder is configured to process an intra-prediction block by,

if the intra-prediction block is of the second type,

deriving a most probable flag from the data stream, the decoder declaring the most probable flag to be in a first flag state if the intra-predicted block is of a first type,

if the most probable flag assumes the first flag state,

deriving a list of most probable intra-prediction modes, the list forming a suitable subset of a set of intra-prediction modes supported by the decoder,

decoding a most probable mode list pointer from the data stream, the pointer pointing into the list of most probable intra-prediction modes,

if the most probable flag assumes the second flag state,

decoding further pointers from the data stream, the further pointers pointing into a remaining set of intra-prediction modes, the remaining set comprising intra-prediction modes of the set of intra-prediction modes that are disjoint from the list of most probable intra-prediction modes,

the intra-prediction block is intra-predicted either entirely or using recursive quadtree partitioning.

22. The decoder according to any of claims 20 and 21, configured to derive a list of most probable intra-prediction modes, the list being different in case the predetermined block is an intra-prediction block of a first type than in case the predetermined block is an intra-prediction block of a second type.

23. Decoder according to any of claims 20 and 21, configured to derive a list of most probable intra-prediction modes, which is different in case the predetermined block is an intra-prediction block of a first type than in case the predetermined block is an intra-prediction block of a second type, wherein the decoder is configured to fill the list of most probable intra-prediction modes by avoiding filling in a DC and/or planar mode of the set of intra-prediction modes, or preferring an angular mode of the set of intra-prediction modes with respect to a DC and/or planar mode of the set of intra-prediction modes when filling.

24. Decoder according to any of claims 20 to 23, configured to derive a list of most probable intra-prediction modes, which is different in case the predetermined block is an intra-prediction block of a first type than in case the predetermined block is an intra-prediction block of a second type, wherein the decoder is configured to fill the list of most probable intra-prediction modes by preferring, when filling, angular modes of the set of intra-prediction modes that are closer to a predetermined dimension (104), the predetermined block being partitioned into partitions in the predetermined dimension such that the partitions are as wide as the predetermined block (80) compared to angular modes of the set of intra-prediction modes that are further away from the predetermined dimension (104).

25. Decoder according to any of the claims 1 to 24,

wherein the decoder is configured to decode the prediction residual from the data stream in a transform domain by, for each partition,

decoding a transform of the prediction residual of the corresponding partition from the data stream in units of sub-blocks by, for each sub-block, the transform being partitioned into the sub-blocks,

decoding an encoded sub-block flag (188) from the data stream;

inferring that a transform coefficient of the transform within the corresponding sub-block is zero if the encoded sub-block flag is not set, an

Decoding the transformed transform coefficients within the corresponding sub-block from the data stream if the encoded sub-block flag is set.

26. The decoder of claim 25, configured to partition the transform into the sub-blocks such that the number of coefficients of each sub-block is a predetermined number independent of partition size, and/or such that the transform is partitioned into the sub-blocks one-dimensionally along a dimension along which the transform is less wide.

27. Decoder according to any of the claims 1 to 26,

wherein the decoder is configured to

Decoding the prediction residual from the data stream in a transform domain on a partition-by-partition basis, and decoding a transform of the prediction residual of a predetermined partition from the data stream by,

decoding a last position indication (190) from the data stream, the last position indication indicating a last transform coefficient position of the transform along a predetermined scan order in which transform coefficients of the one-dimensional transform are scanned; and

transform coefficients of the transform along the predetermined scan order up to the last transform coefficient position are decoded from the data stream (198) and transform coefficients exceeding the last transform coefficient position along the predetermined scan order are inferred to be zero.

28. The decoder according to any of claims 1 to 27,

wherein the decoder is configured to

Decoding the prediction residual from the data stream in a transform domain on a partition-by-partition basis by using a transform for each partition, wherein the transform is of a type selected based on one or more of the intra prediction mode, a dedicated syntax element, and a size of the partition.

29. The decoder according to any of claims 1 to 28,

is configured to

Decoding a partition mode flag (160) for the predetermined block of the image from the data stream;

performing the segmentation and the sequential reconstruction if the segmentation mode flag indicates a first segmentation mode;

reconstructing the predetermined block through spatial intra prediction in a manner depending on the intra coding mode if the coding partition flag indicates a second partition mode.

30. The decoder according to claim 29, wherein,

is configured to perform the spatial intra prediction by deriving a predictor for the predetermined block by filling the predetermined block in its entirety in dependence on one or more already reconstructed samples neighboring the predetermined block in a manner dependent on the intra coding mode, and then reconstructing the predetermined block by correcting the predictor using a prediction residual signaled in the data stream, in case the coding partition flag indicates the second partition mode, or

In a case where the coding partition flag indicates the second division mode, performing the spatial intra prediction by dividing the predetermined block into leaf blocks of a hierarchical multi-tree subdivision of the predetermined block along horizontal and vertical subdivision leaf block boundaries, and sequentially reconstructing the leaf blocks of the predetermined block according to a predetermined leaf block order in which the leaf blocks are sequentially traversed.

31. Decoder according to any of claims 29 or 30,

is configured to, if the intra-prediction mode is an angular mode,

if the partition mode flag (160) indicates a first partition mode,

-upon filling the current partition in dependence of one or more already reconstructed samples adjacent to the current partition in a manner dependent on the intra coding mode, subjecting the already reconstructed samples to a first interpolation filtering dependent on the angular mode;

if the coded partition flag indicates the second partition mode,

filling the predetermined block depending on one or more already reconstructed samples adjacent to the predetermined block in a manner depending on the intra coding mode by subjecting the already reconstructed samples to a second interpolation filtering depending on the angular mode when reconstructing the predetermined block by spatial intra prediction,

wherein the first interpolation filtering differs from the second interpolation filtering in an average number of already reconstructed samples (134) on which each sample (134) of the predictor within the predetermined block depends.

32. The decoder according to any of claims 7 to 31,

is configured to

A partition dimension flag (114) for the predetermined block of the image is decoded from the data stream, and the partition dimension (104) is set to horizontal or vertical depending on the partition dimension flag.

33. The decoder according to claim 32, wherein,

is configured to

Decoding the partition dimension flag (114) by using context-dependent entropy decoding using a context that depends on the intra coding mode.

34. Decoder according to any of the claims 32 to 33,

is configured to

Decoding the partition dimension flag (114) by using context-dependent entropy decoding using one of three contexts, including

Signaling an intra-coding mode other than the angle mode,

the intra coding mode of the horizontal mode is signaled,

the intra coding mode of the vertical mode is signaled.

35. Decoder according to any of the claims 7 to 34,

is configured to

Setting the predetermined partition order (126) depending on the intra-coding mode.

36. The decoder according to any of claims 1 to 35,

is configured to, if the intra-prediction mode is an angular mode,

-performing, when filling the current partition in a manner dependent on the intra-coding mode, one or more already reconstructed samples adjacent to the current partition, the already reconstructed samples being filtered dependent on the angular mode without being pre-filtered.

37. The decoder according to any of claims 1 to 36,

is configured to be, in use,

decoding, from the data stream, an assignment of blocks into which the picture is subdivided, the assignment of blocks being assigned to an inter-prediction coding mode and an intra-prediction coding mode to distinguish an inter-prediction block from an intra-prediction block, wherein the predetermined block is an intra-prediction block.

38. An encoder for block-based encoding of an image into a data stream, configured to

Determining an intra coding mode for a predetermined block of the picture and signaling the intra coding mode in the data stream;

partitioning the predetermined block into partitions;

sequentially predicting the partitions of the predetermined block according to a predetermined partition order of sequentially traversing the partitions by, for a current partition and before proceeding with a subsequent partition,

deriving a predictor for the current partition by populating the current partition in dependence on one or more already coded samples adjacent to the current partition in a manner dependent on the intra coding mode;

determining a prediction residual for use in correcting the predictor to reconstruct the current partition,

signaling the prediction residuals for the partitions in the data stream,

wherein the encoder

Is configured to select the predetermined partition order (126) for the predetermined block from at least two different orders.

39. The encoder of claim 38, configured to

Signaling the predetermined partition order (126) by providing syntax elements in the data stream for the predetermined blocks in addition to the intra-coding mode.

40. The encoder according to any of claims 38 or 39, configured to

Selecting, for the deriving of the predictor, the one or more already reconstructed samples in a manner dependent on a selection of the predetermined partition order (126) for the predetermined block from at least two different orders.

41. The encoder of claim 40, configured to

42. The encoder according to any one of claims 40 or 41,

43. Encoder according to any of the claims 38 to 42,

is configured to perform the segmentation along a predetermined dimension (104) such that the partitions are as wide as the predetermined block (80) in a direction perpendicular to the predetermined dimension (104), and to select a width of the partitions measured along the predetermined dimension (104) from at least two different width settings.

44. An encoder for block-based encoding of an image into a data stream, configured to

Determining an intra coding mode for a predetermined block of the picture and signaling the intra coding mode in the data stream;

partitioning the predetermined block into partitions;

determining a prediction residual for use in correcting the predictor to reconstruct the current partition,

signaling the prediction residuals for the partitions in the data stream,

wherein the encoder

Is configured to perform the segmentation along a predetermined dimension (104) such that the partitions are as wide as the predetermined block (80) in a direction perpendicular to the predetermined dimension (104), and to select a width of the partitions measured along the predetermined dimension (104) from at least two different width settings.

45. The encoder according to any of claims 43 or 44, configured to perform said selection in dependence on the intra coding mode for the predetermined block.

46. The encoder according to any of claims 43 to 45, configured to perform said selection in dependence on whether the intra coding mode for the predetermined block is an angular mode.

47. The encoder according to any of claims 43 to 46, configured to perform the selection in dependence on an index in the data stream for the predetermined block that indexes on the at least two different width settings.

48. Encoder according to any of the claims 38 to 47,

wherein the encoder is configured, when deriving the predictor for the current partition, to use, in a not yet clipped state, reconstructed samples of a partition preceding the current partition according to the predetermined partition order, the reconstructed samples being samples among one or more already reconstructed samples adjacent to the current partition, wherein the reconstructed samples are clipped from the not yet clipped state to a state clipped to an allowed sample value range for finally reconstructing the predetermined block for use as a prediction reference for a sequentially encoded block.

49. An encoder for block-based encoding of an image into a data stream, configured to

Determining an intra coding mode for a predetermined block of the picture and signaling the intra coding mode in the data stream;

partitioning the predetermined block into partitions;

determining a prediction residual for use in correcting the predictor to reconstruct the current partition,

signaling the prediction residuals for the partitions in the data stream,

wherein the encoder is

Wherein the encoder is configured, when deriving the predictor for the current partition, to use, in a not yet clipped state, reconstructed samples of a partition preceding the current partition according to the predetermined partition order, the reconstructed samples being samples among one or more already reconstructed samples adjacent to the current partition, wherein the reconstructed samples are clipped from the not yet clipped state to a state clipped to an allowed sample value range for finally reconstructing the predetermined block for use as a prediction reference for a sequentially encoded block.

50. The encoder according to claim 49, wherein the encoder is a digital encoder,

wherein the encoder is configured to encode the prediction residual into the data stream in a transform domain from which the prediction residual is to be retransformed into a spatial domain for use in correcting the predictor.

51. Encoder according to any of claims 38 or 50,

wherein the encoder is configured to encode the prediction residual into the data stream in a transform domain on a partition-by-partition basis.

52. Encoder according to any of the claims 38 to 51,

wherein the encoder is configured to encode the prediction residual into the data stream in a transform domain by, for each partition,

encoding an encoded partition flag into the data stream;

wherein in case the coded partition flag is set, signaling that the prediction residual for the respective partition is to be set to zero, an

Wherein in case the coding partition flag is not set, signaling that the prediction residue of the respective partition is to be transformed in order to obtain a transformation of the prediction residue of the respective partition and to encode it into the data stream.

53. The encoder according to claim 52, wherein the encoder is a digital encoder,

configured to sequentially encode into the data stream the encoded partition flags for the partitions, except for the encoded partition flag for the last partition in partition order, if all previous encoded partition flags are not set, which will then be inferred as set.

54. The encoder according to any one of claims 52 or 53,

is configured to encode the encoded partition flags for the respective partitions into the data stream by using context-dependent entropy encoding that uses contexts that depend on encoded partition flags for previous partitions that precede the respective partitions in the predetermined partition order.

55. The encoder of any of claims 38 to 54, configured to

Encoding the intra coding mode (116) for the predetermined block (80) by:

deriving a list of most probable intra-prediction modes, the list forming a suitable subset of a set of intra-prediction modes supported by the encoder,

encoding a most probable mode list pointer into the data stream, the most probable mode list pointer pointing into the list of most probable intra-prediction modes.

56. The encoder according to any of claims 38 to 54, the predetermined block being an intra-prediction block of a first type, wherein the encoder is configured to process an intra-prediction block by

If the intra-prediction block is of the second type,

encoding a most probable flag into the data stream that would be inferred to be in a first flag state if the intra-predicted block was of the first type,

if the most probable flag assumes the first flag state,

deriving a list of most probable intra-prediction modes, the list forming a suitable subset of a set of intra-prediction modes supported by the encoder,

encoding a most probable mode list pointer into the data stream, the most probable mode list pointer pointing into the list of most probable intra-prediction modes,

if the most probable flag assumes the second flag state,

encoding further pointers into the data stream, the further pointers pointing into a remaining set of intra-prediction modes, the remaining set comprising intra-prediction modes of the set of intra-prediction modes that are disjoint from the list of most probable intra-prediction modes,

the intra-prediction block is intra-predicted either entirely or using recursive quadtree partitioning.

57. The encoder according to any of claims 55 and 56, configured to derive a list of most probable intra-prediction modes, the list being different in case the predetermined block is an intra-prediction block of a first type than in case the predetermined block is an intra-prediction block of a second type.

58. The encoder according to any of claims 55 and 56, configured to derive a list of most probable intra-prediction modes, the list being different in case the predetermined block is an intra-prediction block of a first type than in case the predetermined block is an intra-prediction block of a second type, wherein the encoder is configured to fill the list of most probable intra-prediction modes by avoiding filling in a DC and/or planar mode of the set of intra-prediction modes, or preferring an angular mode of the set of intra-prediction modes with respect to a DC and/or planar mode of the set of intra-prediction modes when filling.

59. Encoder according to any of claims 55 to 58, configured to derive a list of most probable intra-prediction modes, which is different in case the predetermined block is an intra-prediction block of a first type than in case the predetermined block is an intra-prediction block of a second type, wherein the encoder is configured to fill the list of most probable intra-prediction modes by preferring, when filling, angular modes of the set of intra-prediction modes that are closer to a predetermined dimension (104), the predetermined block being partitioned into partitions in the predetermined dimension such that the partitions are as wide as the predetermined block (80) compared to angular modes of the set of intra-prediction modes that are further away from the predetermined dimension (104).

60. Encoder according to any of the claims 33 to 59,

wherein the encoder is configured to encode the prediction residual into the data stream in a transform domain by, for each partition,

encoding a transform of the prediction residuals of the respective partitions into the data stream in sub-block units by, for each sub-block, partitioning the transform into the sub-block

Encoding an encoded sub-block flag (188) into the data stream, wherein

If the encoded sub-block flag is not set, the transform coefficients of the transform within the corresponding sub-block will be inferred to be zero, an

Encoding the transformed transform coefficients within the respective sub-block from the data stream if the encoded sub-block flag is set.

61. The encoder according to claim 60, configured to partition the transform into the sub-blocks such that the number of coefficients of each sub-block is a predetermined number independent of partition size, and/or such that the transform is partitioned into the sub-blocks one-dimensionally along a dimension along which the transform is less wide.

62. Encoder according to any of the claims 38 to 61,

wherein the encoder is configured to

Encoding the prediction residues into the data stream in a transform domain on a partition-by-partition basis, and encoding a transform of a prediction residue of a predetermined partition into the data stream by,

encoding a last position indication into the data stream, the last position indication indicating a last transform coefficient position of a transform along a predetermined scan order in which transform coefficients of the one-dimensional transform are scanned; and

encoding into the data stream transform coefficients of the transform along the predetermined scan order up to the last transform coefficient position, wherein transform coefficients of the transform exceeding the last transform coefficient position along the predetermined scan order should be set to zero.

63. Encoder according to any of the claims 38 to 62,

wherein the encoder is configured to

Encoding the prediction residual into the data stream in a transform domain on a partition-by-partition basis by using a transform for each partition, wherein the transform is of a type selected based on one or more of the intra prediction mode, a dedicated syntax element, and a size of the partition.

64. Encoder according to any of the claims 38 to 63,

configured to encode a partition mode flag for the predetermined block of the image into the data stream;

performing the partitioning, the sequentially proceeding, and the signaling of the prediction residual of the partition if the partition mode flag indicates a first partition mode;

spatially intra-predicting the predetermined block in a manner dependent on the intra-coding mode and signaling a prediction residual of the predetermined block if the coded partition flag indicates a second partition mode.

65. The encoder according to claim 64, wherein the encoder is a digital video encoder,

is configured to, in a case where the coding partition flag indicates the second partition mode, perform the spatial intra prediction by deriving a predictor for the predetermined block depending on one or more already-coded samples adjacent to the predetermined block as a whole in a manner depending on the intra coding mode, and determining the prediction residual of the predetermined block to reconstruct the predetermined block by correcting the predictor using the prediction residual, or

In a case where the coding partition flag indicates the second division mode, performing the spatial intra prediction by partitioning the predetermined block into hierarchical multi-tree-subdivided leaf blocks of the predetermined block along horizontal and vertical subdivided leaf block boundaries, and determining the prediction residual of the predetermined block to sequentially reconstruct the leaf blocks of the predetermined block according to a predetermined leaf block order sequentially traversing the leaf blocks.

66. The encoder according to any one of claims 64 or 65,

is configured to, if the intra-prediction mode is an angular mode,

if the partition mode flag indicates the first partition mode,

performing a first interpolation filtering dependent on the angular mode on the already encoded samples when filling the current partition in a manner dependent on the intra coding mode depending on one or more already reconstructed samples adjacent to the current partition;

if the coded partition flag indicates the second partition mode,

filling the predetermined block depending on the one or more already reconstructed samples adjacent to the predetermined block in a manner depending on the intra coding mode by subjecting the already coded samples to a second interpolation filtering depending on the angular mode when reconstructing the predetermined block by spatial intra prediction,

wherein the first interpolation filtering differs from the second interpolation filtering in an average number of already reconstructed samples on which each sample of the predictor within the predetermined block depends.

67. The encoder according to any one of claims 44 to 66,

configured to determine whether the partition dimension is horizontal or vertical,

encoding a partition dimension flag for the predetermined block of the image into the data stream, the partition dimension flag indicating whether the partition dimension is horizontal or vertical.

68. The encoder according to claim 67, wherein the encoder is a digital video encoder,

is configured to encode the partition dimension indicator by using context-dependent entropy coding using a context that depends on the intra-coding mode.

69. Encoder according to any of the claims 67 to 68,

configured to encode the partition dimension flag by using context-dependent entropy coding using one of three contexts including

Signaling an intra-coding mode other than the angle mode,

the intra coding mode of the horizontal mode is signaled,

the intra coding mode of the vertical mode is signaled.

70. Encoder according to any of the claims 44 to 49,

is configured to set the predetermined partition order in dependence on the intra-coding mode.

71. The encoder according to any one of claims 38 to 70,

is configured to, if the intra-prediction mode is an angular mode,

-performing, when filling the current partition in dependence on one or more already coded samples adjacent to the current partition in a manner dependent on the intra coding mode, the already coded samples with filtering dependent on the angular mode without pre-filtering.

72. Encoder according to any of the claims 38 to 71,

is configured to be, in use,

encoding into the data stream an assignment of blocks into which the image is subdivided, the assignment of blocks being assigned to an inter-prediction coding mode and an intra-prediction coding mode to distinguish between inter-prediction blocks and intra-prediction blocks, wherein the predetermined block is an intra-prediction block.

73. The encoder of any of claims 38 to 72, configured to

Encoding into the data stream a partition mode flag for the predetermined block of the picture, the partition mode flag indicating that the predetermined block is encoded using the partitioning, the sequentially proceeding, and the signaling of the prediction residuals for the partitions if set, or indicating that the predetermined block is encoded using an alternative to spatial intra prediction using an intra prediction mode determined for the predetermined block if not set,

by passing

Performing an encoding cost test on the predetermined block to identify, from the set of supported intra-prediction modes, a supported intra-prediction mode that is preferred in terms of encoding cost when the predetermined block is encoded using the supported intra-prediction modes in the alternative to obtain the intra-coding mode for the predetermined block and a reference encoding cost associated therewith when the predetermined block is encoded using the intra-coding mode in the alternative, and

in sequentially proceeding, determining a coding sub-cost associated with the prediction residual for use when correcting the predictor to reconstruct the current partition, adding the coding sub-cost to a sum of previous coding costs associated with prediction residuals of previous partitions of the predetermined block to obtain an updated coding cost sum, checking whether the updated coding cost sum exceeds the reference coding cost and aborting the sequentially proceeding in case the coding cost exceeds the reference coding cost, and coding the partition mode flag as not set, and coding the partition mode flag as set if the updated coding cost sum obtained for a last partition in the predetermined partition order still does not exceed the reference coding cost,

using a coding cost minimization scheme to decide

An intra-coding mode of the set of supported intra-prediction modes, an

The partition mode flag is set or not set.

74. A method for block-based decoding of an image (12) from a data stream (14), comprising:

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition; and

a predetermined partition order (126) for the predetermined block is selected from at least two different orders depending on the data stream.

75. A method for block-based decoding of an image (12) from a data stream (14), comprising:

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition,

wherein the partitioning along a predetermined dimension (104) is performed such that the partitions are as wide as the predetermined block (80) in a direction perpendicular to the predetermined dimension (104), and the width of the partitions measured along the predetermined dimension (104) is selected from at least two different width settings depending on the data stream.

76. A method for block-based decoding of an image (12) from a data stream (14), comprising:

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition,

wherein the prediction residuals are decoded from the data stream for all partitions before sequentially reconstructing the partitions of the predetermined block.

77. A method for block-based decoding of an image (12) from a data stream (14), comprising

-decoding an intra coding mode (116) for a predetermined block (80) of the image from the data stream (14);

partitioning the predetermined block (80) into partitions (102)_i；112_i)；

by using the prediction residual (120) signaled in the data stream_i) Correcting (122) the predictor, reconstructing the current partition,

wherein, for the derivation of the predictor for the current partition, reconstructed samples of a partition preceding the current partition according to the predetermined partition order are used in a not yet clipped state, the reconstructed samples being samples among one or more already reconstructed samples adjacent to the current partition, wherein the reconstructed samples are clipped from the not yet clipped state to a state clipped to an allowed range of sample values to finally reconstruct the predetermined block.

78. A method for block-based encoding of an image into a data stream, comprising

Determining an intra coding mode for a predetermined block of the picture and signaling the intra coding mode in the data stream;

partitioning the predetermined block into partitions;

determining a prediction residual for use in correcting the predictor to reconstruct the current partition,

signaling the prediction residuals for the partitions in the data stream,

the predetermined partition order (126) for the predetermined block is selected from at least two different orders.

79. A method for block-based encoding of an image into a data stream, comprising

Determining an intra coding mode for a predetermined block of the picture and signaling the intra coding mode in the data stream;

partitioning the predetermined block into partitions;

determining a prediction residual for use in correcting the predictor to reconstruct the current partition,

signaling the prediction residuals for the partitions in the data stream,

80. A method for block-based encoding of an image into a data stream, comprising

Determining an intra coding mode for a predetermined block of the picture and signaling the intra coding mode in the data stream;

partitioning the predetermined block into partitions;

determining a prediction residual for use in correcting the predictor to reconstruct the current partition,

signaling the prediction residuals for the partitions in the data stream,

81. Computer program having a program code for performing a method according to any of claims 74 to 80 when run on a computer.

82. A data stream produced by the method of any one of claims 78 to 80.

Technical Field

The present application relates to the concept of intra coding for use in block-based codecs, such as, for example, hybrid video codecs.

Background

Given a particular block, intra prediction in HEVC is performed by extrapolating decoded boundary samples of neighboring blocks according to certain modes, namely 33 angular modes and DC and planar modes [1 ]. This intra prediction mode, which minimizes the data rate distortion cost, is then signaled to the decoder. Although known codecs support many Intra Prediction Modes (IPMs), intra prediction implemented by them is still the subject of development to find better intra predictors leading to higher coding efficiency. This applies not only to HEVC but also to other block-based codecs that use intra prediction. Due to the fact that more accurate predictors reduce the prediction residual, finding a set of intra prediction modes suitable for efficiently encoding the interior of the block requires considering from a signaling overhead point of view the signaling overhead of signaling the intra prediction modes and the resulting quality of the predictors derived from these intra prediction modes, thereby reducing the signaling overhead associated with encoding the prediction residual. In order for the signaling overhead associated with the intra prediction mode to be low, the intra prediction block should be large, i.e. the granularity of the signaled intra prediction mode should remain coarse, but on the other hand spatial prediction of larger blocks tends to be less accurate due to the fact that the average sample distance of the samples inside the intra prediction block, i.e. the samples to be predicted, is higher than the average sample distance of already decoded/coded samples, i.e. reference samples, adjacent to the block. HEVC slightly alleviates this left-right dilemma by allowing a transform residual block, which forms a leaf block that subdivides a coding unit by multi-tree subdivision, to inherit the intra-prediction mode of its corresponding coding unit. However, this still requires signaling overhead for signaling from the encoder to the decoder the sub-partitioning of the respective intra-coded coding unit into transform blocks.

Therefore, it would be advantageous to have techniques that can further improve the coding efficiency of intra-coding.

Disclosure of Invention

It is therefore an object of the present invention to provide a more efficient inner coding technique.

This object is achieved by the subject matter of the independent claims of the present application.

The present invention relates to block-based coding of an image using an intra-prediction coding concept, and according to the intra-prediction coding concept, a specific block of the image is intra-predicted by dividing a predetermined block into partitions whose number of partitions is two or more using a specific intra-coding mode. For reconstruction purposes, the partitions are spatially predicted sequentially using the intra-prediction coding mode signaled for a particular block, and the prediction residuals are then used to correct the predictor thus obtained, so that for the previous partition, the reconstructed samples can be made available to the decoder when processing the next partition, then the current partition. According to a first aspect, the partition order of processing partitions of a predetermined block is selected from at least two different orders. Explicit or implicit signaling techniques may be used to keep the selection synchronized between the encoder and decoder. Thus, the selection is such that when one partitioning order is applied to a predetermined block, another order may be applied to another block having the same size and shape and partitioned into partitions in the same manner, which is also encoded using the intra-partition prediction technique. The free selection of the partitioning processing order for the encoder, or the presentation of the dependency of the partitioning processing order on other parameters than size, shape and partitioning, is effective in improving the effectiveness of the intra-partition prediction mode.

According to a further aspect of the invention, which may be combined with or used without the previous aspect, the partitioning is performed along a dimension such that the partitions are as wide as the predetermined block in a direction perpendicular to the predetermined dimension, while the width of the partitions measured along the predetermined dimension is selected from at least two different width settings or options. Explicit or implicit signaling techniques may be used to keep the selection synchronized between the encoder and decoder. Thus, the selection may be such that partitions may be changed between blocks of the same size and shape, but the overhead associated with the change may be kept reasonably low. For example, the selection may be performed according to an intra coding mode for the predetermined block, such as depending on whether the intra coding mode for the predetermined block is an angle mode. The selection may also be made based on an index in the data stream for the predetermined block that indexes at least two different width settings. The partition may be one or more samples wide along the partitioning dimension. Within a block, the partition width along the dividing/predetermined direction may vary. One may be one sample wide, i.e. a one-dimensional stripe, and the other more than one sample wide, i.e. a two-dimensional field of samples. The signaling overhead with respect to the segmentation may be removed or may be kept low. Thus, the partitioning enables the overall signaling overhead of the predetermined block for signaling the intra-prediction mode in the data stream to be low, while still providing the encoder and decoder with the opportunity to reduce the average distance of the samples of the predetermined block from the neighboring reference samples they have reconstructed/encoded. The latter are located at least partly within the predetermined block itself, i.e. within previously processed partitions for which prediction residuals have been determined so as to be available for correcting the predictor for the partition in which they are located.

According to a further aspect of the invention, which may be combined with any of the preceding aspects or used without the preceding aspects, the decoder operation may become more efficient in implementation by decoding prediction residuals from the data stream for all partitions before sequentially reconstructing the partitions of the predetermined block.

And according to a still further aspect of the invention, which may be combined with any of the preceding aspects or used without the preceding aspects, the decoder operation becomes more efficient in implementation by using reconstructed samples of a partition preceding the current partition according to a predetermined partition order, in a not yet sliced state, which reconstructed samples are samples among one or more already reconstructed samples adjacent to the current partition, when deriving the predictor for the current partition, wherein the reconstructed samples are sliced from the not yet sliced state to a state sliced to an allowed range of sample values for finally reconstructing the predetermined block, which means that, on the encoder side, the slicing is performed only to obtain a reconstructed version of such samples, for use as a prediction reference for a subsequent coding block, in order to maintain reference synchronization with the decoder.

Advantageous aspects of the invention are the subject matter of the dependent claims.

Drawings

Preferred embodiments of the present application are described below with reference to the accompanying drawings, in which:

fig. 1 shows a block diagram of an apparatus for predictive coding an image as an example of an encoder for an intra prediction technique in which embodiments according to the present application may be implemented;

FIG. 2 shows a block diagram of an apparatus for predictive decoding of pictures, suitable for the apparatus of FIG. 1, as an example of a decoder for intra prediction techniques in which embodiments according to the application may be implemented;

fig. 3 shows a diagram illustrating an example of the relationship between a prediction residual signal, a prediction signal and a reconstructed signal to show the possibilities of setting the subdivision for coding mode selection, transform selection and transform performance, respectively;

fig. 4 shows a schematic diagram illustrating the partitioning process of an intra coded block according to an embodiment that allows selection among different partitioning dimensions, i.e. among horizontal and vertical partitions;

FIG. 5 shows a schematic diagram illustrating the sequential processing of partitions of intra-coded blocks processed according to a partitioning option;

FIG. 6 shows a schematic diagram illustrating the prediction derivation of the padding process for a partition;

fig. 7 shows an example of an intra-partition prediction block divided according to horizontal and vertical partition modes, respectively, and having two different intra-prediction modes associated therewith, respectively, in order to show the likelihood of relevant decisions exhibiting a partition order on the intra-prediction modes associated with the intra-prediction block;

FIG. 8 shows a diagram illustrating the possible signalization spent by intra-predicted blocks 80 processed using the partitioning option;

FIG. 9 shows a diagram illustrating one possible way of transmitting the prediction residual for a partition, according to an embodiment;

FIG. 10 shows a schematic diagram illustrating the partial sum of the coding costs involved in determining partitions of an intra-prediction mode, so as to be able to abort the test when it is clearly not getting better than any normal intra-prediction mode; and

FIG. 11 shows a flow diagram of the mode or operation of an encoder to perform partition mode testing according to an embodiment; and

fig. 12a and 12b show an example of an alternative segmentation of intra-coded blocks as a further embodiment.

Detailed Description

The following description of the figures begins with the presentation of descriptions of an encoder and decoder for a block-based predictive codec for encoding pictures of video to form an example for an encoding architecture in which embodiments for an intra-predictive codec may be built. The former encoder and decoder are described with respect to fig. 1 to 3. A description of embodiments of the intra-prediction techniques of the present application, and a description of how these techniques may be built into the encoders and decoders of fig. 1 and 2, respectively, are then presented together, but the embodiments described with respect to fig. 4 and subsequent figures may also be used to form encoders and decoders that do not operate according to the encoding architecture on which the encoders and decoders of fig. 1 and 2 are based.

Fig. 1 illustrates an apparatus for predictively encoding an image 12 into a data stream 14, using transform-based residual coding. The apparatus or encoder is indicated using reference numeral 10. Fig. 2 shows a corresponding decoder 20, i.e. an apparatus 20 configured to predictively decode a picture 12' from the data stream 14 also using transform-based residual decoding, wherein an apostrophe has been used to indicate: the image 12' reconstructed by the decoder 20 deviates from the image 12 originally encoded by the apparatus 10 in terms of coding losses introduced by the quantization of the prediction residual signal. Although fig. 1 and 2 exemplarily use transform-based prediction residual coding, embodiments of the present application are not limited to such prediction residual coding. The same is true for the other details described with respect to fig. 1 and 2, as will be outlined below.

The encoder 10 is configured to perform a spatial to spectral transformation of the prediction residual signal and to encode the prediction residual signal thus obtained as a data stream 14. Likewise, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and to perform a spectral-to-spatial transformation on the prediction residual signal thus obtained.

Internally, the encoder 10 may include a prediction residual signal former 22 that generates a prediction residual 24 in order to measure the deviation between the prediction signal 26 and the original signal, i.e. the image 12. For example, the prediction residual signal former 22 may be a subtractor that subtracts the prediction signal from the original signal, i.e., the image 12. The encoder 10 then further comprises a transformer 28 which performs a spatial to spectral transformation of the prediction residual signal 24 to obtain a prediction residual signal 24' in the frequency domain, which is then quantized by a quantizer 32, which is also comprised by the encoder 10. The prediction residual signal 24 "thus quantized is encoded into the bitstream 14. To this end, the encoder 10 may optionally comprise an entropy encoder 34 which entropy encodes the transformed and quantized prediction residual signal into the data stream 14. The prediction residual 26 is generated by a prediction stage 36 of the encoder 10 on the basis of a prediction residual signal 24 "which is encoded into the data stream 14 and which can be decoded from the data stream 14. To this end, the prediction stage 36 may internally comprise a dequantizer 38, as shown in fig. 1, which dequantizes the prediction residual signal 24 "so as to obtain a frequency-domain prediction residual signal 24" ', which corresponds to the signal 24' apart from quantization losses, followed by an inverse transformer 40, which inversely transforms, i.e. a spectral-to-spatial transformation, the prediction residual signal 24 "', so as to obtain a prediction residual signal 24" "which corresponds to the original prediction residual signal 24 apart from quantization losses. The combiner 42 of the prediction stage 36 then performs a recombination, such as by adding the prediction signal 26 to the prediction residual signal 24 "", in order to obtain a reconstructed signal 46, i.e. a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to the signal 12'. The prediction module 44 of the prediction stage 36 then generates the prediction signal 26 on the basis of the signal 46 by using, for example, spatial prediction, i.e. intra-prediction, and/or temporal prediction, i.e. inter-prediction.

Likewise, decoder 20 may be internally made up of components corresponding to prediction stage 36 and interconnected in a manner corresponding to prediction stage 36. In particular, the entropy decoder 50 of the decoder 20 may entropy decode the quantized frequency-domain prediction residual signal 24 "from the data stream, wherein the inverse quantizer 52, the inverse transformer 54, the combiner 56 and the prediction module 58 are interconnected and cooperate in the manner described above for the modules of the prediction stage 36 to restore the reconstructed signal on the basis of the prediction residual signal 24", as shown in fig. 2, such that the output of the combiner 56 produces the reconstructed signal, i.e. the image 12'.

Although not specifically described above, it is clear that the encoder 10 may set some encoding parameters, including for example prediction modes, motion parameters, etc., according to some optimization scheme, such as for example in a way that optimizes some data rate and distortion related criterion, i.e. encoding cost. For example, the encoder 10 and decoder 20 and respective modules 44, 58 may support different prediction modes, such as intra-coding and inter-coding modes, respectively. The granularity at which the encoder and decoder switch between these prediction mode types may correspond to the subdivision of the images 12 and 12' into coded segments or blocks, respectively. In units of these coding sections, for example, an image may be subdivided into intra-coded blocks and inter-coded blocks. Intra-coded blocks are predicted based on the spatial, already encoded/decoded neighborhood of the respective block, as will be outlined in more detail below. Several intra-coding modes may exist and may be selected for respective intra-coded sections, including directional or angular intra-coding modes, according to which the respective section is padded into the respective intra-coded section by extrapolating sample values of a neighborhood along a particular direction, wherein the particular direction is specific to the respective directional intra-coding mode. The intra-coding modes may, for example, further comprise one or more further modes, such as a DC coding mode, assigning a DC value to all samples within the respective generated intra-coded section according to the prediction used by the mode for the respective intra-coded block; and/or an in-plane coding mode according to which the prediction of the respective block is approximated or determined as a spatial distribution of sample values, the spatial distribution being described by a two-dimensional linear function over the sample positions of the respective intra-coded block, with a drive tilt and offset of a plane defined by the two-dimensional linear function on the basis of neighboring samples. In contrast to this, the inter-coded blocks may be predicted, for example, temporally. For inter-coded blocks, a motion vector may be signaled within the data stream, the motion vector indicating the spatial displacement of the portion of the previously coded picture of the video to which picture 12 belongs, at which position the previously coded/decoded picture is sampled to obtain a prediction signal for the respective inter-coded block. This means that in addition to the encoding of the residual signal comprised by the data stream 14, such as the entropy encoded transform coefficient levels representing the quantized frequency domain prediction residual signal 24 ", the following parameters may be encoded in the data stream 14: coding mode parameters for assigning coding modes to various blocks, prediction parameters for some blocks, such as motion parameters for inter-coded sections, and further optional parameters, such as parameters that control and signal the subdivisions of fig. 12 and 12', respectively. The decoder 20 uses these parameters to subdivide the image in the same manner as the encoder, to assign the same prediction modes to the sections, and to perform the same prediction to generate the same prediction signal.

Fig. 3 shows the relation between the reconstructed signal, i.e. the reconstructed image 12', on the one hand, and the combination of the prediction residual signal 24 "" and the prediction signal 26, on the other hand, as signaled in the data stream. As already stated above, the combination may be an addition. The prediction signal 26 is shown in fig. 3 as subdividing the image area into intra-coded blocks, which are indicated diagrammatically with hatching, and inter-coded blocks are indicated diagrammatically with no hatching. The subdivision may be any subdivision, such as a conventional subdivision of the image area into rows and columns of blocks or blocks, or a multi-tree subdivision of the image 12 into leaf blocks of different sizes, such as a quad-tree subdivision into blocks or the like, a mixture of which is illustrated in fig. 3, where the image area is first subdivided into rows and columns of root blocks and then further subdivided according to a recursive multi-tree subdivision. Again, for an intra-coded block 80, an intra-coded mode may be encoded into the data stream 14, which assigns one of several supported intra-coded modes to the respective intra-coded block 80. Further details are described below. For inter-coded block 82, one or more motion parameters may be encoded into data stream 14. In general, the inter-coded block 82 is not limited to being time-coded. Alternatively, the inter-coded block 82 may be any block predicted from: previously encoded portions outside the current picture 12 itself, such as previously encoded pictures of the video to which the picture 12 belongs, or pictures of another view or pictures of a lower layer of the layered architecture in the case of scalable encoders and decoders, respectively. The prediction residual signal 24 "" in fig. 3 is also illustrated as a subdivision of the image area into blocks 84. These blocks may be referred to as transform blocks in order to distinguish them from the encoded blocks 80 and 82. In practice, fig. 3 shows that the encoder 10 and decoder 20 can use two different subdivisions of the image 12 and image 12', respectively, into blocks, one subdivision into encoded blocks 80 and 82, respectively, and another subdivision into block 84, respectively. These two subdivisions may be the same, i.e. each encoded block 80 and 82 may form a transform block 84 at the same time, but in the case shown in fig. 3, the subdivision into transform blocks 84 forms an extension into a subdivision into encoded blocks 80/82, for example, such that any boundary between two of the blocks 80 and 82 covers the boundary between two of the blocks 84, or alternatively, each block 80/82 may coincide with either one of the transform blocks 84, or with one cluster of transform blocks 84. However, the subdivisions may also be determined or selected independently of one another, such that transform block 84 may alternatively span block boundaries between blocks 80/82. Thus, there is also a similar statement for the subdivision into transform blocks 84 as those set forth with respect to the subdivision into blocks 80/82, i.e., blocks 84 may be the result of a conventional subdivision of an image area into blocks/blocks arranged in rows and columns, the result of a recursive multi-tree subdivision of an image area, or a combination thereof or any other kind of blocking. Additionally, it should be noted that the blocks 80, 82, and 84 are not limited to being square, rectangular, or any other shape.

Fig. 3 shows that the combination of the prediction signal 26 and the prediction residual signal 24 "" directly results in the reconstructed signal 12'. It should be noted, however, that according to alternative embodiments, more than one prediction signal 26 may be combined with the prediction residual signal 24 "" to produce the image 12'.

In fig. 3, the transform section 84 should have the following meaning. Transformer 28 and inverse transformer 54 perform their transforms in units of these transform sections 84. For example, many codecs use some DST or DCT on all transform blocks 84. Some codecs allow the transform to be skipped so that for some of the transform sections 84, the prediction residual signal is coded directly in the spatial domain. However, according to the embodiments described below, the encoder 10 and the decoder 20 are configured in such a way that they can support a plurality of transforms. For example, the transforms supported by the encoder 10 and decoder 20 may include:

o DCT-II (or DCT-III), where DCT stands for discrete cosine transform

Omicron DST-IV, where DST stands for discrete sine transform

οDCT-IV

οDST-VII

Omicron equal transformation (IT)

Of course, while transformer 28 will support all forward transformed versions of these transforms, decoder 20 or inverse transformer 54 will support its corresponding reverse or inverse transformed version:

omicron inverse DCT-II (or inverse DCT-III)

Omicron inverse DST-IV

Omicron inverse DCT-IV

Omicron inverse DST-VII

Omicron equal transformation (IT)

The following description provides more detail about which transforms the encoder 10 and decoder 20 may support. In any case, it should be noted that the set of supported transforms may include only one transform, such as one spectral to spatial or spatial to spectral transform.

As already outlined above, fig. 1-3 have been presented as examples in which the intra-prediction concepts described further below can be implemented to form specific examples for encoders and decoders according to the present application. In this regard, the encoder and decoder of fig. 1 and 2 represent possible implementations of the encoder and decoder, respectively, described below. As will be outlined in more detail below, the encoder of fig. 1 and the decoder of fig. 2, when constructed in the encoder and decoder of fig. 1 and 2 with the subsequently explained embodiments for intra prediction according to the present application, at least as an option, support processing of the intra prediction block 80 in a manner described in more detail below. Thus, the embodiments described hereinafter relate to an encoder equal to the encoder 10 of fig. 1, which processes the intra-predicted block 80 in a manner described in more detail below, and which also applies to the decoder of fig. 2, and therefore fig. 2 represents an example of a decoder according to an embodiment, in which intra-coded blocks are processed in a manner outlined in more detail below. However, fig. 1 and 2 are only specific examples. However, an encoder according to embodiments of the present application may perform block-based encoding of the image 12 using the techniques outlined in more detail below, and differs from the encoder of fig. 1, such as for example in that it is not a video encoder, in that it does not support intra prediction, or in that the subdivision into blocks 80 is performed in a different way than in the example of fig. 3, or even in that this encoder does not use transform prediction residual coding, but directly encodes the prediction residual, for example in the spatial domain. Likewise, a decoder according to an embodiment of the present application may perform block-based decoding of the image 12 'from the data stream 14 using intra-prediction techniques outlined further below, but may also differ from the decoder 20 of fig. 2, for example in that it is not a video decoder, but a still image decoder, in that it does not support intra-prediction, or in that it subdivides the image 12' into blocks in a different way than described with reference to fig. 3, and/or in that it derives prediction residuals from the data stream 14 not in the transform domain but, for example, in the spatial domain.

It has been said that the following description focuses on the description of intra prediction according to embodiments of the present application. According to the intra-prediction presented herein, the intra-prediction block, such as block 80 in fig. 4, is allowed to be divided into partitions. According to some embodiments, the partitions are fixed one-dimensional horizontal partitions or one-dimensional vertical partitions, but as it is mentioned many times herein, this situation will be understood as an example that may be varied in order to obtain alternative embodiments in which partitions are either wider, partitions or even have additional properties, instead of partitioning blocks along a certain partitioning direction/dimension, partitioning into one-and/or two-dimensional partitions is performed in any other way, including for example an operation using multi-tree partitioning, first defining a partition column, then partitioning the above-mentioned partition column into partitions, respectively, or otherwise. The availability to process blocks in the manner described below may be provided for intra-predicted blocks 80 of any size or limited to blocks 80 within a predetermined range of block sizes, such as blocks larger than a particular size. "one-dimensional" in relation to a partition refers to the fact that a partition is only one sample wide along the partition dimension, whereas one-dimensional or 1D in relation to a partition refers to the partitioning of a block into partitions along one direction, resulting in partitions/stripes that extend across the block along a direction perpendicular to the partitioning direction.

See, for example, fig. 4. Fig. 4 shows on the left side an intra-predicted block 80, i.e. a block to be decoded or a block to be encoded. It has a dimension W × H. That is, it is a block of size W × H, where H is the height of the block 80 and W is the width of the block 80, each measured in units of samples. According to fig. 4, two partitioning or splitting options are available, namely horizontal splitting 100, according to which horizontal splitting 100 a block 80 is partitioned or split into a plurality of partitions 102 along a vertical axis, i.e. a splitting dimension 104₁、102₂、102₃And 102₄. According to the example of FIG. 4, which is an example applied in the following description, each partition 102₁To 102₄Is one sample wide, as indicated by double-headed arrow 106, such that the partition 102 created by block 80₁To 102₄Is equal to H, i.e. the height of the block 80 in units of samples 108 of the block 80, but it should be clear that the partitioning may also be performed by the encoder and decoder in a different way depending on the agreement between encoder and decoder, such as, for example, the partitioning of the block 80 along the dimension 104 may result in the partition 102_iFor example, the predetermined number is greater than two, or a mixture of: the size of the block 80 is evenly distributed over a predetermined number of partitions along the partition dimension. And according to still further embodiments, the one-dimensional nature of the segmentation itself may even be altered to yield another segmentation concept, which is broadly referred to belowFor FIPS, the one-dimensional nature of the segmentation itself is that it occurs along a particular direction, the resulting partitions being like stripes that extend completely across the block in a direction transverse to the direction of segmentation.

Another encoding option, depicted in FIG. 4 and indicated by reference numeral 110, corresponds to partitioning the block 80 into vertical partitions 112₁，112₂，...112₈. That is, according to option 110, block 80 is partitioned into partitions 112 along a horizontal axis, i.e., horizontal partition dimension 104_i. In the case of option 100, each partition 102_iAs wide as the block 80, i.e. having the width W of the block, and the partition 112_iThe height H of the block 80 is taken, i.e. has a height H. In summary, in a manner similar to that described for option 100, vertical partition 110 may divide block 80 into W partitions 112_iWhere W represents the lateral width of the block 80 measured in samples 108, such that each partition 112_iHorizontally one sample wide, however, the partitioning according to option 110 may also be done in another way agreed between encoder and decoder.

Thus, according to fig. 4, the encoder is free to divide the block 80 into H W × 1 partitions 102 according to the horizontal partitioning option 100_iOr divided into W1 XH partitions 112 according to the vertical split option 110_iAnd the selection of a partition option for a block 80 by the encoder may be signaled in the data stream 14 for the block 80, such as, for example, by means of a corresponding partition dimension flag 114 in the data stream 14. However, it should be clear that embodiments of the present application also cover encoders and decoders that, in the preset case, use only one of the options 100 and 110, without the need for the flag 114 in the data stream. Even further, in other examples, the flag 114 may be transmitted in the data stream 80 from the encoder to the decoder depending on the intra-coding mode 116 signaled in the data stream 14 for the block 80. As described above, the intra-coding mode may indicate one of a set of available/supported intra-coding modes, including, for example, an angular mode and, optionally, one or more non-angular modesEquation, such as DC mode or planar mode. That is, according to an alternative embodiment, which will not be discussed further below, the flag 114 may be transmitted in the data stream 14 in a manner that is conditionally dependent on the intra-coding mode 116. According to the embodiment described below, a flag 114 is present in the data stream 14 for the block 80, which is independent of the intra-coding mode 116 signaled in the data stream 14 for the block 80. However, a dependency may exist relating to flag switching between the process of segmentation of intra coded block 80 just discussed and the different ways of processing intra coded block 80 that will be outlined below.

According to an embodiment of the present application, each of the partitions 102/112 is predicted, transformed, quantized, and encoded separately, and the partitions are processed sequentially in this manner. Thus, reconstructed samples of a certain partition will be able to be used to predict any subsequent partitions 102/112 in partition order in the partition into which block 80 has been partitioned, and in this way the process of intra-prediction loops through partition 102/112 into which block 80 has been partitioned. Fig. 5 exemplarily shows an intra-prediction block 80 divided according to option 100. For each partition 102 of block 80₁To 102₄Subject to prediction, i.e. the respective partition 102_iAnd a prediction residual-related task, i.e. performing a correction of the predictor using the prediction residual. The latter task may be performed by combining the prediction residual and the predictor. This is done in the decoder for reconstruction. As a task related to the prediction residual, the encoder performs a determination of the prediction residual, e.g. including transformation and quantization, and uses the prediction residual to correct the predictor, i.e. by filling the decoded picture buffer in the decoder with the reconstruction of the picture in order to keep the prediction loop synchronized with the decoder. The just mentioned tasks, i.e. the processing of prediction and residual, are for the partition 102₁To 102₄Individually and in partition 102₁To 102₄In which it is performed sequentially. After these two steps for the currently processing partition, the next partition 102 according to the partition order is processed in the same manner_i. The partitioning order is exemplarily illustrated in fig. 5 using three arrows 126.

FIG. 5 shows that the partition comprising the upper left-most pixel of block 80 will be processed first, followed by the processing of the immediately adjacent lower adjacent partition 102₂Etc., corresponding to partition 102 in fig. 5₁To 102₄But this order is merely an example and as the subsequent description will clearly appear, the partition order may be selected in a manner dependent on other settings such as the intra-coding mode and/or the size of the block 80, the former dependencies of which will be discussed below.

In the example discussed further below, the partition order 126 only varies between the ways in which partitions 102/112 are traversed in such a way that immediately following partitions are immediately adjacent to each other, such that in the case of partition type 100 the partition order is from top to bottom or from bottom to top, and in the case of partition type 110 is left to right or right to left, respectively. However, it should be mentioned that other examples are also conceivable. For example, the partition order may be selected as follows: such that the partitions are scanned in two scans in the contiguous order just outlined, with every other partition being processed from top to bottom, bottom to top, left to right, or right to left in the first scan, optionally applicable, and then the remaining partitions in between being processed in the same sequential direction or in the opposite direction.

In any case, FIG. 5 shows the first partition 102₁The partition that is to be processed first and is currently processed. For the first partition, here illustratively 102₁For forming the partition 102₁Set 118 of adjacent samples of the predictor of (2)₁The selection may be based only on samples that lie outside the boundary of the block 80, since no samples of the block 80 have been processed, i.e. reconstructed or encoded, while the first partition of the block 80 is being processed. That is, the set 118 has been reconstructed in the encoder using any prediction and correction of the corresponding predictor using the prediction residual transmitted in the data stream₁The sample of (1). They belong to previously encoded/decoded image blocks and may be inter-codedBlocks that are either intra-coded or coded in any other way are encoded. As for forming the first partition 102₁Of the predictor 118₁Depending on the intra coding mode assigned to the block 80, and the exact position. This intra-coding mode is used jointly, or equivalently, for each partition of processing block 80, as will be discussed below. To complete the first partition 102₁In the case of an encoder, the determination of pass depends on the set 118₁One or more already reconstructed/encoded samples in to fill the partition 102₁Derived in the decoder and encoder for this partition 102₁I.e. determined by transformation and quantization as outlined above, and then this prediction residual-in a version transmitted in the data stream, i.e. comprising quantization losses-is used to reconstruct this partition 102 by correcting the predictor using the prediction residual in the data stream 14₁. For example, FIG. 5 at 120₁Is illustratively shown for partition 102₁The prediction residual of (2). That is, 120₁Including corresponding to partition 102₁Of the prediction residual, wherein the data 120₁The description of which will be discussed in more detail below.

Turning now to the next partition in the partition order, partition 102 in the example of FIG. 5₂. Currently, the situation has changed because of the use of the export for the partition 102₂The set of neighbouring reconstructed/coded samples of the predictor of (2) may now be composed of samples located outside the block 80 and/or samples within the block 80, i.e. samples located in any already processed partition, here the partition 102 in the example currently described in fig. 5₁Since for these samples the prediction residual has already been determined and is already available in the data stream 14. That is, the encoder and decoder derive for this partition 102₂Followed by prediction residual determination in the encoder and the use of the prediction residual for correcting the predictor in the encoder and decoder, respectively. This processing then continues with the next partition in the queue, i.e., the next partition in the partition order, sequentially processingAll partitions of block 80 are processed.

As already mentioned above, it is possible to select the partition order 126 in a manner other than traversing the partitions such that the immediately consecutive partitions are immediately adjacent partitions. That is, the partition order may be skipped from one partition to the next. This means by filling the corresponding partition 102_iTo derive a set 118 of neighboring samples used by the respective predictor_iNot limited to directly adjacent samples of the respective partitions, as shown in fig. 5. This is also relevant to the selection of the start of the partition order 126. Imagine, for example, partition 102₄Was the first partition in the partition order. Its predictor may then pass through a set 118 that depends on neighboring samples not shown in FIG. 5₄It is filled and derived, which collects samples located around the block 80 up to the left and top of the block 80. In the collection 118₄Will not interact with the partition 102₄Directly adjacent. Incidentally, this will correspond to the case where the last sample column is filled in the conventional intra-prediction padding of the entire block of the block 80. The same may be true for any subsequently processed partition, i.e. the second and other partitions in the partition order. That is, their neighboring sample sets 118_iMay also include a corresponding partition 102_iNot directly adjacent samples. Even further, any second or subsequent processing partition 102 traverses partitions in a manner that does not restrict partition order in which consecutive partitions are directly adjacent to each other_iReference sample set 118_iNot only can collections be located in corresponding partitions 102_iAnd may also be samples located at the left and top of the corresponding partition 102₁Lower sample, depending on whether any block 80 is partitioned earlier than partition 102 according to partition order₁Is processed. That is, set 180_iMay include being located in partition 102_iMore than two sided samples.

Briefly summarized, FIG. 5 illustrates the sequential processing of partitions 102/112 of block 80, herein illustratively directed only to horizontal partitions, but the same description applies also to vertical partition 112_iThe vertical mode 110. For each partition 102_iCorresponding prediction residual 102_iIs included in the data stream 14. Data 120₁To 120₄Together forming the prediction residual, i.e. 120, for block 80. It should be remembered that according to an alternative embodiment of the present application, the transform residual coding may not be used, i.e. the prediction residual 120 of the block 80 may be signaled directly in the spatial domain in the data stream 14, for example. In this case, for the corresponding partition 102₁To 102₄Data 120 of₁To 120₄The partition separation field, as shown in fig. 5, may not be included in the data stream 14, where each data portion 120_iThe representation is for the corresponding partition 102_iSignaling of certain transformations of (a). Conversely, in this case, the prediction residual 120 for block 80 may form a field of data 14. In this alternative embodiment, the decoder will be processing a partition 102_iThe time from field 120 is collected about the application to this partition 102_iInformation of the prediction residual of (1). This procedure can also be applied when a fully reversible version of the transform is used, so that the quantization can be done in the spatial domain.

Thus, fig. 5 shows for each partition 102 in the encoder and decoder_iTwo tasks are performed, namely: 1) prediction derivation tasks 122, resulting in the generation of data for the corresponding partition 102₁I.e. for the respective partition 102_iA predicted sample value of each sample of (a); and 2) the prediction residual related task performed thereafter, namely prediction residual derivation at the encoder, including quantizing the prediction residual to bring it into the data stream 14, and reconstructing the respective partition 102 by combining or correcting the prediction residual and the predictor_iIn order to obtain samples for this partition 102_iThe reconstructed sample of (1). To predict the derivation task, the reconstructed samples may be used as partitions 102 for subsequent sequential processing in the partition order 126_jOf the neighboring sample set 118_jThe reservoir of (2).

Before continuing further on to describe details of embodiments of the present application, FIG. 6 illustrates partitioning 1 by populating the current process02_iThe process of prediction derivation 122 is undertaken, wherein it should be recalled that the illustration with respect to the horizontal partition 102 is merely selected for illustration, and the same description applies to the vertical partition 112. FIG. 6 illustrates a current process partition 102_iAnd its corresponding reconstructed/encoded set of neighboring samples 118_i. As already described above with respect to FIG. 5, set 118_iMay not be limited to being directly adjacent or contiguous to partition 102_iOf the sample 128. However, due to the segmentation, when averaged over all samples of block 80, in partition 102_iSample and set 118 of_iIs lower than performing intra prediction of block 80 in a manner known from e.g. h.264 or HEVC. As described with reference to FIG. 5, predictor derivation or padding 122 is for each partition 102 using the intra-prediction mode associated with block 80_iPerformed, wherein the mode indicates one of a set of available intra-prediction modes. This set may include angle or direction patterns that are in the neighboring sample set 118_iIs copied to partition 102_iAre different from each other in the angle or direction 132 along which they are in the sample 134. To perform such replication, partition 102_iMay be based on the set 118 being positioned relative to the sample 134 in a direction facing opposite the direction 132_iIs derived from the number of adjacent samples 134. For example, the number is used to derive the number in the sample set 118_iIs defined by the kernel of the interpolation filter at the inter-pixel locations between the samples 128. For example, FIG. 6 shows collection 118_iIs used to calculate the currently processed partition 102_iOf one sample 134. Due to the relatively small average distance 130, the partition 102_iThe number of reference samples 134 per sample 134 may be kept low. More details will be given below. However, for completeness, it should be noted that the set of available intra-prediction modes may also include a DC mode according to which DC values are assigned to partitions 102_iBy all samples 134 in the set of adjacent samples 118_iPerforming averaging to derive the DAnd C value. Furthermore, a planar mode may exist according to which the prediction values for the samples 134 are by way of the partition 102_iIs defined by a linear function at the sample position within which the adjacent sample 118 is located_iThe slope and offset of this linear function are derived on the basis of (a). Further, it should be noted that neighbor set 118_iMay differ according to the intra-prediction mode selected for block 80 and may be, for example, particularly different between angular and non-angular modes DC/plane.

For example, in a prior art JEM decoder, 67 intra prediction modes are available: of which 65 are angular modes and of which two are DC and planar model non-directional structures. The 1D partition mode, i.e. the predictor derivation 122 performed for partition 102/112 outlined above and below, according to which the block 80 is partitioned/divided along dimension 104, with the resulting partitions extending across the entire width of the block transverse to dimension 104, one sample wide or more than one sample wide along direction 104, the embodiments directly resulting therefrom by summarizing the partitioning process described further below using the name "flexible intra-prediction sub-partition mode" can be combined with any of them or, stated differently, can be implemented using any of them. As already described for fig. 5, all partitions 102/112, such as one block 80 of a coding unit CU, use the same associated intra prediction mode as the block 80, thereby avoiding excessive overhead in the signaling, since the prediction mode 116 only needs to be transmitted once in the data stream 14 for the block 80.

That is, if the sub-partition is two-dimensional, prediction 122 can be performed in the same manner as the normal case outlined in the JEM decoder. However, if it is a 1-D sub-partition, i.e., a line, whether horizontal or vertical, is calculated for the currently processed partition 102/112, so that the prediction process 122 will be adjusted accordingly as compared to JEM. In the case of a partition order chosen to traverse partitions in such a way that consecutive partitions are directly adjacent to each other, the prediction process 122 may correspond to the normal case of JEM, but only to the first line (if a partition is single sample wide, otherwise it is as in the conventional case), i.e. the one closest to the reconstructed/encoded neighborhood. In some cases, HEVC and JEM both allow the use of certain filters applied on the reference samples 128 or on the resulting predictor. This is useful in general situations to better predict samples within the prediction block 80 that are far away from the reference sample 128 to reduce boundary discontinuities. However, by using the partition that becomes partition 102/112, it is possible, and should be the goal, to take advantage of the high degree of correlation between nearby pixels.

That is, a reduced average distance 130 should be utilized. Excessive smoothing can degrade this quality. Thus, the encoder or decoder should be able to perform two types of intra prediction, i.e. using segmented intra prediction as outlined for fig. 4 to 6, and then the intra filter, i.e. the filter involved in the predictor derivation 122, is either disabled or the number of contributing samples 134 for at least each of the partitioned samples 134 is reduced relative to the number of samples that contribute to one sample in the general case, where the intra prediction for the block 80 is performed on the block or according to HEVC, i.e. decomposed into leaf blocks subdivided into a hierarchical architecture quadtree subdividing the block 80 into rectangular blocks.

As is clear from the discussion presented above, to perform the prediction residual correlation task 124, the decoder decodes, for example, a transform of the corresponding prediction residual of the currently processed partition from the data stream 14 and performs an inverse transform, such as a spectral to spatial transform, on this transform to produce a prediction residual, which is then used by combining/adding to correct the predictor obtained at 122. The same operations are done in the encoder to keep the prediction loop synchronized with the decoder. In addition, the encoder performs a transform of the prediction error using the predictor determined 122 for the current processing partition, such as a spatial to spectral transform followed by quantization of these transform coefficients, which is then encoded into the data stream 14 to produce the current processing partition 102_iCorresponding data 120 of_i. As for the conversion, canTo process all partitions 102/112 within block 80 using the same transform. It may be DCT-II but other than in the case of planar mode, for example, wherein DST-VII may be used. Thus, if the block 80 is encoded using an intra-prediction mode in a partitioning manner as outlined so far with reference to fig. 4 to 7, and as will be further outlined below, all tools related to transforms and inverse transforms that the encoder and decoder may use for other blocks, such as transform hopping, i.e. encoding in the spatial domain, EMT (EMT ═ explicit multi-core transform), NSST (NSST ═ mode-dependent inseparable quadratic transform), or others, may be disabled to avoid unnecessary overhead bits. Even alternatively, the transform may be a linear transform, the type of which is selected based on one or more of an intra-prediction mode, a dedicated syntax element, and a predetermined partition order.

Some description has been made regarding the partition order 126 for sequentially processing the partitions 102/112 of the current processing block 80. It should be emphasized that this embodiment is merely an example, and that the partitioning order may be static, or may be changed in different ways, according to alternative embodiments, according to other embodiments for which the examples listed below are intended. Fig. 7 indicates possible partitioning/processing orders by inscribed numbers, shown using arrow 126 in fig. 5. Here, this order follows the inscribed numbers in ascending order. Fig. 5 represents an example where the order 126 starts at the partition comprising the top left pixel/sample 140 of the block 80, leading down to the lowest partition. Similarly, if the partition type is vertical, the processing order will advance right again, starting with the leftmost partition that includes the top-left pixel/sample. However, this is not the best case for all existing intra prediction modes. This is illustrated in fig. 7, which also shows the vertical and horizontal division of the block 80 for the diagonal pattern, i.e., the replication angle/direction 132 points 45 ° from bottom left to top right, and 34, i.e., the replication angle/direction 132 points-45 ° from top left to bottom right. In the former case, if the partitioning is horizontal, starting from the upper left corner of block 80, a partition will be generated for which the reconstructed samples will not have any impact on the prediction of the subsequent partitions. Therefore, it is more reasonable to start from the lower left corner of the block, so that the reconstructed samples of each partition can be used to predict the next partition in partition order. However, in vertical segmentation, this is not necessary as it can be observed in the previous figures. On the other hand, the mode 34 does not have any of these problems considering that the sample enters the horizontal and vertical divisions from both sides. Thus, a normal processing order can be adopted in the two divisions.

Table 1 shows a complete list of processing orders according to the intra prediction mode and the partition type.

Table I: processing order according to intra-mode and partition type. HOR _ DIR and VER _ DIR are horizontal and vertical modes, respectively, and VDIA _ DIR is vertical diagonal mode

To summarize the embodiments described so far with respect to signaling overhead, please refer to fig. 8. FIG. 8 illustrates a transmission for a block 80 according to an embodiment of the present application. In particular, there is intra-prediction mode signaling 116 that signals as to which intra-prediction mode is to be applied to the block 80. Thus, the signalization 116 indicates, for example, one of the angular modes, or one of the available modes, which includes angular modes as well as non-angular modes, such as DC and planar. In addition to the signalization 116, there is a partitioning flag 160 encoded into the data stream 14 by the encoder and decoded from the block 80 by the decoder, which indicates whether the partitioning process according to fig. 4 to 7 is applied to the block 80, or whether it is processed "normally", such as entirely or entirely, i.e. only samples outside the block 80 are used to form the reference sample store 118 to predict each sample within the block 80. Again, as described further below, this embodiment directly means an alternative embodiment obtained by modifying the above-described embodiment. Alternatively, the marker 160 may switch between the segmentation process discussed with respect to fig. 4-7 in one aspect, and the decomposition of the block 80 using a quadtree subdivision that subdivides the block 80 into a transformBlocks, which are then processed sequentially, but suffer from the following disadvantages: the decomposition must be signaled in the data stream 14. If the segmentation marker 160 indicates a segmentation according to fig. 4, the data stream 14 contains a segmentation dimension flag 114 for the block 80, which segmentation dimension flag 114 switches between the segmentation types 100 and 110 discussed with respect to fig. 4. And then, for each partition of the block 80 into which the block 80 is subdivided/partitioned, if the partition flag 160 indicates this partition option, the data stream 14 includes signaling/data 120₁With the prediction residuals of the respective partitions encoded therein in the transform domain, such as indicated above.

With respect to fig. 8, it should be noted that the prediction residual data 120₁、120₂.., may be encoded into the data stream 14 in an order corresponding to the partitioning/encoding order 126. As discussed above, the latter can be uniquely determined by the intra-prediction mode indicated by the signalization 116. However, an alternative embodiment would be to determine the segmentation order 126 at least in part based on optional additional signalization in the data stream 14.

Another alternative proposed to be described herein is the fact that: the signalization 116 may alternatively be used to indicate whether a segmentation option is used. In other words, one syntax element can generally assume responsibility for the signaling of 116 and 160. Such syntax elements will assume one of a range of values, each of which corresponds to a combination of intra prediction modes' and an indication of whether block partitioning is used. In this case, it would also be possible to provide segmentation options for only a subset of the intra-prediction modes. And finally, it should be noted that the partition flag 160 may also be conditionally transmitted in the data stream 14 only in case the intra-prediction mode indicated by the signalization 116 assumes a certain subset of the available intra-prediction modes.

FIG. 9 exemplarily shows a partition 102/112 having a certain partition_iData 120 of prediction residual_iPossible morphology of (2). According to the embodiment of fig. 9, the prediction residual is encoded into the data stream 14 in the transform domain. That is, the encoder generates prediction residues by transform 180A transform 182 of the difference, wherein the decoder derives the prediction residual and spatial domain by an inverse transform 184. Fig. 9 shows transform coefficients 186 of the transform 182, which correspond to, for example, different spectral frequencies f. Data 120_iA coded block flag CBF 188 may be included that indicates whether the transform 182 includes any significant transform coefficients 186, i.e., whether the transform 182 is all zero. If CBF 188 is set, transform 182 is not zero and data 120 is_iA Last Position (LP) syntax element 190 may be included that indicates 192 the last position along the increased spectral frequency (see axis 194) of the significant transform coefficient, i.e., non-zero transform coefficient 186, starting with the lowest or DC coefficient 196. Then, the data 120_iSignaling 198 is included which signals the transform coefficients from 196 to 192.

That is, FIG. 9 shows each partition 102_i/112_iIts prediction residual may be encoded into the data stream 14 by CBF 188, LP 190 and transform coefficient data 198. That is, for a block 80 having n partitions 102/112, for each partition having non-n-zero CBFs 188, there will be n CBFs 188 and one LP 190, and the transform coefficient data 198 is only for the partition having the non-zero CBF 188 associated therewith. Coefficient data 198 may be encoded in the same way as a normally processed intra-predicted block, i.e. the partition flag 160 indicates the block 80 of non-partition options, with the following exceptions: each LP 190 requires only one coordinate if the partition is a single sample wide (otherwise it requires two coordinates as is often the case), i.e., x represents the horizontal split 100 and y represents the vertical split 110. However, in the case of a two-dimensional partition, LP 190 indicates the last position along a scan direction or path, either using the sorted indication or using x and y coordinates. The context of each CBF 188 may be selected as the value of the previously encoded CBF, i.e., the CBF of the previous partition in partition order 126. Further, due to the segmentation, the transform coefficient data 198 relates to different shapes. That is, the transforms 182 also have different shapes. In the case where the partition is a one-dimensional partition, the transform 182 is a one-dimensional transform, as discussed with respect to FIG. 4. That is, transform 182 may be a W/H long vector of transform coefficients 186, which depends on the partitionCut type 100 or 110.

The following should be noted with respect to the flags 160 and 114 of FIG. 8, and their encoding. Flag 160, which indicates whether block 80 is to be partitioned into partition 102/112, defines the conditions under which flag 114 is to be checked for transmission in data stream 14 for block 80. In particular, if flag 160 indicates a partition into partitions 102/112, flag 114 exists in data stream 14 and is transmitted to the decoder to signal which type of partition 100/110, i.e., horizontal or vertical, is to be performed. Like the flag CBF, the flag 114 may also be encoded using context-dependent entropy encoding/decoding. Depending on the intra-prediction mode of block 80, the context of flag 114 may have three possibilities: 0 represents the non-angle mode, 1 represents the horizontal mode, and 2 represents the vertical mode.

Although FIG. 9 shows that the CBF 188 may appear once at each partition i of the current block 80, additionally or alternatively, the partition 120 of the current block_iMay each be divided into one or more sub-blocks, with respect to the data 120ⁱEach sub-block within a sub-block signals an encoded sub-block flag indicating whether the transform coefficients 186 within the sub-block are all zero or at least one of the coefficients is non-zero. Thus, only the encoded sub-block flag signals that coefficients 186 within the sub-block where non-zero coefficients exist are to be encoded. The encoded sub-block flag signals that other coefficients within the sub-block, without any non-zero coefficients, will be inferred to be zero at the decoder side. Note that each partition 120 is a partition_iBeing transformed separately, the sub-blocks belonging to a partition differ in the spectral composition of the partition transform 182 and differ in the transform coefficients 186 they include outside the transform. For example, as long as the corresponding partition 102_i/112_iHave a size in both x (partition width) and y (partition height) greater than or equal to 4 samples 140, and therefore, as long as the corresponding partition 102_i/112_iHaving a size greater than or equal to 4 coefficients 186 in both x and y, the sub-blocks may be arranged in 4 x 4 coefficient blocks. In the case of 4 × N partitioning, the sub-blocks form one row of m 4 × 4 sub-blocks, where m × 4 ═ N and m is an integer. In the case of an N × 4 partition, the sub-blocks form one column of m 4 × 4 sub-blocks, where m × 4 ═ N and m is an integer. For wider partitions, an array of 4 × 04 sub-blocks arranged in rows and columns may be produced. However, depending on the embodiment, such partitioning wider than 4 samples and/or 4 samples wide may not occur. Whether or not the occurrence occurs, for a narrower partition, i.e. less than 4 samples in one of its dimensions, i.e. less than 4 samples wide in at least one dimension x or y, the partitioning of its sub-blocks of the transform 180 into sub-blocks may be done, each of which collects a different group of coefficients of the transform 180, so that the sub-blocks have the minimum number M of coefficients in all possible cases of the current block size. That is, the partitions may be set as large as the block width N along one dimension, with the partitioning occurring along the other dimension 104. Thus, the transform 180 of each partition may have a size of 1 × 1N, 2 × 2N, N × 31, or N × 42. In practice, the transform 180 for a certain partition may have a number of coefficients equal to the number of samples in that partition. In the case of 1 × N partitioning/transformation, these sub-blocks may form one column of M1 × M sub-blocks, where M × M ═ N and M is an integer. In the case of an N × 1 partition, the sub-blocks form one row of M × 1 sub-blocks, where M × M ═ N and M is an integer. In the case of a 2 xn partition/transform, the sub-blocks may form one column of M2 × sub-blocks (M/2), where M × N (M/2) and M is an integer. In the case of an N × 2 partition, the sub-blocks may form one row of M (M/2) × 2 sub-blocks, where M × (M/2) ═ N and M is an integer. This is exemplarily shown in table 1 for an example case of M-16 for the least number of coefficients.

Table 1: entropy coding coefficient group size

Although fig. 9 shows that the CBF 188 may occur once for each partition i of the current block 80, it may be agreed between the decoder and the encoder that at least one of the n partitions in the partition of the current block 80 has a non-zero CBF 188. For this reason, if n is the number of sub-partitions,and the first n-1 sub-partitions in the coding order have produced zero CBF, then the CBF of the nth partition will be inferred to be 1. Therefore, it is not necessary to decode it and it is not encoded. Thus, data 120_nThe CBF that will be lost is in the data 120₁To 120_n-1Will signal a CBF of zero and the decoder will infer this CBF to signal the presence of at least one non-zero coefficient in the transform of the partition.

The following may hold for the intra coding mode signaling 116. The coded mode signalization 116 may be transmitted as a pointer or index to one of a list of Most Probable Modes (MPMs). Next, the MPM list may be determined by the encoder and the encoder in the same manner based on the intra prediction mode used for the previously encoded/decoded intra prediction block, such as a spatially and/or temporally adjacent intra prediction mode. Thus, the MPM list may represent a suitable subset of the available/supported intra-prediction modes, i.e. one or more of the aforementioned angular modes and/or DC and planar modes. As mentioned above, there may be intra-predicted blocks using LIP or ISP schemes, such as block 80 in the figure, but also those that are intra-predicted in a classical manner, i.e. either in blocks or in units of transform blocks into which such intra-predicted blocks are partitioned using recursive quadtree partitioning. Both types of intra-prediction blocks may support the same set of available/supported intra-prediction modes. Although for the latter normal/legacy intra-prediction block, an MPM flag may be signaled in the data stream-the decoder decodes it and the encoder encodes it-indicating whether the mode of the block is selected from the MPM list, in this case a pointer/index to the MPM list is passed-the decoder decodes it and the encoder encodes it-in the case of an intra-prediction block using LIP or ISP schemes, such as block 80, the MPM flag will be inferred to signal the MPM list restriction. If, for a certain normal/legacy intra-prediction block, the MPM flag signals that no MPM mode is used, no index/pointer exists for the block in the data stream, but instead a replacement pointer/index pointing to the remaining list of intra-prediction modes is transmitted in the data stream for the block. The remaining list may also be a suitable subset of the available/supported intra-prediction mode set and may in particular be a complementary set of MPM lists compared to the available/supported intra-prediction mode set, i.e. each member of the available/supported intra-prediction mode set will be either a member of the MPM list or a member of the remaining set. The pointers/indices pointing to the MPM list may be VLC coded, while the pointers/indices pointing to the remaining set may be coded using a fixed length code. Of course, the MPM flag may be transmitted even for intra-predicted blocks of the LIP or ISP schemes, and the encoder may freely select any mode from the available/supported intra-prediction mode set, with the setting of the MPM flag depending on whether the selected mode is in the MPM list or in the remaining set.

The MPM list may be the same for normal/legacy intra prediction blocks and for ISP/LIP intra prediction blocks, i.e. may be decided by the encoder and the encoder in the same way. However, regardless of the use of an MPM list with no restrictions on the MPM list and without inference with an MPM flag to signal intra-ISP/LIP prediction blocks, the MPM list may instead be decided differently for intra-ISP/LIP prediction blocks to accommodate statistics of the ISP/LIP mode. For example, it may be changed to exclude the DC intra mode from the MP list and prioritize the horizontal intra mode for ISP horizontal split, i.e., horizontal direction 104, and the vertical intra mode for ISP vertical split, i.e., vertical direction 104. That is, for a normal/legacy intra-prediction block, the MPM list may form a suitable subset of the set of available/supported intra-prediction modes, which are selected and ordered according to some particular concept. For the ISP/LIP intra-predicted block 80, the MPM index may point to an MPM list depending on the partitioning direction 104 signaled by the flag 114 and/or the appropriate subset forming the set of available/supported intra-prediction modes minus the DC mode or minus the DC and planar modes, i.e. the appropriate subset of angular modes in the set of available/supported intra-prediction modes. The MPM list construction based on previously used intra-prediction modes that are previously encoded/decoded may be such that the angular mode of the preferred intra-prediction direction is closer to the horizontal dimension in case the flag 114 indicates that the partitioning direction 104 is to be horizontal, and closer to the vertical dimension in case the flag 114 indicates that the partitioning direction 104 is to be vertical.

With respect to the description just presented, please note again that the juxtaposition between the intra-prediction mode of the normal process and the intra-prediction mode using the partition process as outlined herein is not required. That is, the encoder and decoder may inevitably process the intra-predicted block 80 using the partitioning presented herein, and thus, for example, the partition flag 160 may become obsolete. However, if the partitioning option signaled by flag 160 is available as a decision for the encoder, the following description reveals the possibility of how the encoder performs the decision, or finds whether the partition mode should be used for a particular block 80, and which partition type, i.e. horizontal or vertical, is best. To perform this operation, the encoder should test two options for different intra prediction modes for each block. The encoder will therefore be slower than if it had only one option, such as the normal option, because more options have to be tested. To reduce this effect, the partition mode signaled by flag 160 may be tested in a manner coded according to the following strategy, wherein reference is made to fig. 10 and 11.

1) The 1D partition mode (or a flexible intra-prediction sub-partition according to a modified embodiment described later) is the last intra mode to be tested.

2) Let C_minBecomes the minimum cost so far when the 1D partition mode (or flexible intra-prediction sub-partition according to a modified embodiment described later) is to be tested.

3) A combination of intra-mode and partition type to be tested is selected.

4) A block is divided into N partitions, and let i denote the index of each of the partitions, where i ═ 1, N.

5) After each partition is coded, its sub-cost J_iIs calculated. Thus, we can know the sum of all available sub-costs after partition i has been encoded, which isThe process is illustrated in FIG. 10, and thus, it illustrates the accumulation of partition sub-costs to obtain the final cost for the entire block.

6) After each partition is processed, expression S_i<C_minIs evaluated. If true, we continue to encode the partition until the end. Otherwise, it is guaranteed that this test pattern will not yield a ratio C_minLower RD cost, so the process is interrupted and we continue to the next intra mode and partition type combination.

7) If all partitions are coded, the test pattern is the best pattern and C_minAre updated accordingly.

The advantage of this procedure is that it avoids processing unnecessary partitions, since it is known that the 1D partitioning (or flexible intra-prediction sub-partitioning according to the modified embodiment explained later) mode will not generate a better cost than the lowest cost already existing. Furthermore, it does not have any disadvantages in terms of RD loss. The whole process is as shown in the flow chart of fig. 11.

It is again noted that all the above embodiments show a segmentation in one sample wide band transverse to the direction 104, which may alternatively be done in a way that results in a wider partition, resulting in a two-dimensional partition, unless some details in the description of the embodiments make use of one-dimensional properties. Other alternatives regarding segmentation are described below.

In other words, embodiments have been described in the above description, which may be briefly described as including a line-based intra prediction (LIP) coding mode facility according to which an intra-predicted wxh block 80 is partitioned into 1-D partitions 102/112 or lines. Each of the resulting lines 102/112 is then predicted and a residual signal is generated, transformed, quantized, and entropy encoded, and finally the coefficients are described as being transmitted to a decoder. The reconstructed samples of such a line may be used to predict the next line, and so on. The process is repeated until all 1-D partitions within the original block have been encoded.

However, modifications are possible with respect to this LIP concept. Which was indicated previously in the discussion of fig. 4, embodiments of the present application are not limited to partitions 102/112 having a one sample width along the splitting/partitioning direction 104. In the embodiment described below, the definition of the width along the dividing direction 104 is based on 1) whether the intra prediction mode is an angular mode or a non-angular mode, and 2) the width of the intra prediction block along the direction 104.

1) The wxh (where W and H are assumed to be powers of 2) block 80 may be split horizontally or vertically (e.g., indicated by syntax element 114 communicated to the decoder) into K equal partitions 102/112 of size W × H, the values of which are listed in table II. According to table II, a block with W-16, H-8, predicted using the angle-free intra mode and divided vertically (i.e., dimension 104 is vertical), for example, would be divided into 4 partitions 102, all of which have the size W-16 and H-2. This example is shown in figure 12 a. If the same block 80 is predicted using the angular intra mode, it is divided into 8 partitions 102, each partition 102 having a size w-16 and h-1.

Table II: values of w, h and K for additional layout example 1

2) The wxh (where W and H are assumed to be powers of 2) block 80 may instead be split horizontally or vertically (e.g., indicated with a syntax element 114 communicated to the decoder) into K equal partitions of size W × H, where the value of K is not fixed (and thus it is communicated to the decoder with the syntax element), and may range to any power of 2 between 2 and S, where S is the value of the dimension being split (width for vertical splitting and height for horizontal splitting). The values of w and h were obtained as described in table III.

Variables of	Vertical division	Horizontal division
			w	W/K	W
h	H	H/K

Table III: values of w and h for additional layout example 2

Instead, the width of the partition for block 80 along dimension 104 may be signaled directly.

3) The wxh (where W and H are assumed to be powers of 2) block 80 may alternatively be split horizontally or vertically (e.g., indicated with a syntax element 114 communicated to the decoder) into K partitions (where K depends on W and H) with W_i×h_iA dimension, where i ═ 1, 2. If the segmentation is horizontal, let S be H, S_i＝h_iAnd if the division is vertical, let S equal W, S_i＝w_i. S is described in Table IV for various values of S_iS measures the width of the block 80 along dimension 104, and S_iThe width of partition i is measured along dimension 104.

Table IV: s for different partition layouts_iValue of (A)

The options used by the decoder are fixed or can be implicitly determined from the existing parameter values at the decoder side.

4) The wxh (where W and H are assumed to be powers of 2) block 80 may alternatively be split horizontally or vertically (e.g., indicated with a syntax element 114 communicated to the decoder) into K partitions (where K depends on W and H) with W_i×h_iA dimension, where i ═ 1, 2. If the division is horizontal, let S be H, S_i＝h_iAnd if the division is vertical, let S equal W, S_i＝w_i。s_iWill be determined by a syntax element indicating which of the three options presented in example 3) is to be used for dividing the block into sub-partitions.

Thus, as shown in examples 1 to 4 above, the segmentation may be performed along one dimension 104 such that the partitions are as wide as the predetermined block perpendicular to the predetermined dimension, while the width of the partitions measured along the predetermined dimension 104 is selected from at least two different width settings or options. The concept of explicit or implicit signaling may be used to keep the selection synchronized between the encoder and decoder. Thus, the selection is such that although the split can be changed between blocks of the same size and shape, the overhead associated with the change is kept fairly low. For example, the selection may be performed according to an intra coding mode for the predetermined block, such as depending on whether the intra coding mode for the predetermined block is an angular mode. The selection may also depend on the index in the data stream, the predetermined block for indexing at least two different width settings as shown in example 4. The partition may be one or more samples wide along the partitioning dimension. Within a block, the partition width along the dividing/predetermined direction may vary. One may be one sample wide, i.e. a one-dimensional stripe, and the other more than one sample wide, a two-dimensional field of samples.

Furthermore, in the case of some of the LIP concept embodiments presented above, it is proposed that two types of processing orders-referred to as partition orders 126 in the description above-may be used for each block 80, with the final choice of which one to use depending on the intra-mode of the block 80. However, as already mentioned above, there are alternatives for deciding the partition order 126. Processing order 126 indicates in which order child partitions 102/112 are to be processed. Possible processing orders 126 include starting with the top left sub-partition and then continuing in a predetermined order, such as down in the case of horizontal partitioning or right in the case of vertical partitioning, or in another case partitioning into partitions in any other order, such as in a raster scan fashion. In general, the processing order 126 may be determined by existing parameters available at the decoder side at the time of the corresponding sub-partition to be decoded/processed. That is, for the decoder, the processing order 126 may be determined in real time, i.e., alternating among predicting the various sub-partitions 102/112. Such existing parameters may include intra mode, sub-partition index, size of original block, size of sub-partition, CBF flag of current partition, and so on. The order 126 may also be signaled directly to the decoder.

Even for intra-partition predicted blocks 80 that use the same partitioning into partitions 102/112, the motivation for changing the partition order may stem from the concept already discussed above with respect to fig. 5: the order 126 of processing at partition 102/112 is changed to enable the allocation of prediction accuracy among the various partitions 102/112. For example, as exemplified above, when the partition 102/112 that is to be processed first away from the adjacent sample set 118 is selected according to the partition order 126, this would mean that intra-prediction would be worse than if the partition were last processed in the partition order 126 otherwise, because in the latter case the references used to populate the partition would be located closer, i.e., at the partition adjacent to the partition at the side thereof facing the reference sample set 118. In the case of FIG. 5, for example, it is apparent that for partition 102₄The obtained intra prediction result will result in a larger prediction residual (such that the prediction will be obtained based on neighboring samples outside the block 80) in case the partition order 126 is first, and the last processed partition 102 according to the partition order 126₄Compared to the situation (in this case, partition 102)₄Will be relative to partition 102 based on coverage₄A contiguous set of samples immediately adjacent to the neighboring samples and comprisingSuch as partition 102₃To generate a prediction). However, the opposite is true for the remaining partitions. In the first processing of partition 102₄In the case of (2), partition 102 is used₄May be included into the neighboring sample set 118 of any subsequently processed partitions according to the partition order 126_iThereby enabling, for example, prediction of each partition 102 of fig. 5₁To 102₃The intra prediction mode for the block 80 is used from both sides of the partition. In processing partition 102₄In this case, the neighboring sample set 118_iLocated only in partition 102_iOne side of the respective partition for each of i 1.. 4. I.e. for partition 102₁To 102₃The situation is just the opposite. For a partition, partition 102 is processed first₄In the case of the last processing partition 102₄In contrast, the prediction residual will be lower. Still further, it should be considered that what has been explained above, i.e. the order may not only be switched between two options of successively traversing partitions side by side, but it is also possible to allow further partitioning of the order 126, such as processing every other partition first in a predetermined order and then processing the remaining partitions in the same order or in a reverse predetermined order. Which option is best can be determined by testing various ordering options at the encoder side while the respective additional signalizations for the block 80 are transmitted in the data stream in the order 126, or the order 126 can be selected at the encoder side and the decoder side in the same way depending on the syntax elements transmitted in the data stream for the block 80, or in the same way for adjacent blocks such that even for blocks 80 of the same size and shape and partitioned into partitions 102/112 in the same way, the order 126 is not the same and varies between these blocks 80.

Thus, the partition order of the partitions of processing block 80 may be selected from at least two different orders. The concept of explicit or implicit signaling may be used to keep the selection synchronized between the encoder and decoder. Thus, the selection is such that when the partition order is applied to a predetermined block, another order can be applied to another block, also usingIntra-partition prediction concepts are coded, have the same size and shape and are partitioned into partitions in the same way. There is a degree of freedom in selecting a partitioning processing order for an encoder, or a partitioning processing order depending on other parameters than size, shape, and division, which is effective in improving the effectiveness of the intra-partition prediction mode. When collecting for exporting a partition 102_iReference samples 118 of the predictor of_iThe effectiveness may be even greater in a manner that depends on the predetermined partition order 126 selected for the predetermined block from at least two different orders. In particular, as shown, a reference sample 118 is collected_iMay be positioned on two opposite sides of the current partition in the case of one available or selectable partition order, and may be positioned only on partition 120 in the case of another available partition order_iOne of the two opposite sides of (a). In the case of partitioning along dimension 104, a "side" may be partition 120_iFacing sides along a direction, such as upper and lower sides in the case of the vertical direction 104, and left and right sides in the case of the horizontal direction, but another partitioned case is briefly discussed below, a "side" may simply be defined as facing a reference sample set 118 located outside the block 80_iOn the side of the partition of those samples, and facing the reference sample set 118 located inside the block 80_iI.e. inside one of the previously processed partitions. For example, while one of the at least two different selectable orders may traverse partitions starting from the partition furthest from the top left corner of the predetermined block, another of the at least two different selectable orders may traverse partitions from the partition closest to the top left corner of the predetermined block.

Another issue that should be addressed relates to residual coding. As described above, this can be done using transform coding. Each sub-partition 102/112 may have its own Coded Block Flag (CBF)188, Last Position (LP) syntax element 190, and transform coefficients 198 in the data stream that will be transmitted to the decoder. Thus, for a block 80 having K sub-partitions 102/112, such as a CU, there will be K CBFs 188 and one LP 190 for each partition 102/112 having a non-zero CBF. The context used to encode each CBF 188 may depend on the CBF values of previously encoded sub-partitions within the same block-e.g., along the order 126. Additionally, another syntax element not yet mentioned may be transmitted in the data stream to the decoder to indicate whether the described concept is used on each block, or whether the intra-partition prediction concepts described herein are used on each intra-prediction block 80 within a range corresponding to the entire data stream or a certain picture or a slice, e.g. for a certain picture, or whether some of them will be signaled as a whole, i.e. as if only partitioned into one partition.

Likewise, it has been described above that each sub-partition may be transformed separately using a transform, thereby producing a transform for each partition 102/112 that is not quantized to all zeros. As a transform for a particular partition 102/112, a 2-D transform may be used, except in the case where one of the dimensions of partition 102/112 is one, in which case a 1-D transform would be applied. The transform kernel may be DCT-II or any other transform determined by existing parameters at the decoder side when the sub-partition is to be decoded. For example, the transformation may be selected according to an intra mode, a sub-partition index, and a combination of sub-partition dimensions, or some subset of the latter parameters. The decoder may also be signaled directly or differently, e.g. in the form of additional syntax elements transmitted separately for all partitions within a block 80 or for each partition 102/112 of a block 80.

One aspect relates to the fact that the aspect already discussed above is: after quantization in the spatial or some intermediate transform domain by transforming the prediction residual of each partition in a single partition (whose transform coefficients are then quantized), the residual of partition 102/112 of block 80 may be further transformed, albeit lossless or reversible. Thus, the decoder can obtain the transformed transform coefficient levels for the entire block 80, inverse lossless transform it to obtain the prediction residual for each partition 102/112 in the spatial or intermediate transform domain from the prediction residual in the spatial domain obtained by re-transforming each partition 102/112.

It should be noted that, according to variations of the above-described embodiments, the segmentation may be performed in a form other than the segmentation along the dimension 104. Intra-predictive coding using such variations may be given a name such as flexible intra-prediction sub-partitioning (FIPS). Instead of splitting into stripes 102/112, the partitioning/splitting of block 80 may produce any rectangular sub-partition of block 80. Flexible Intra Prediction Subdivision (FIPS) partitions W H Intra prediction blocks 80 into K rectangles of size W_i×h_iWherein i is 1, 2. The resulting layout of the sub-partitions is completely compact, which means that the sum of the areas of the sub-partitions is equal to the area of the original block, i.e.Fig. 12b shows an example of such a generalized partitioning of the block 80, where the reference numeral 102 has been reused for partitioning. In this case, the 16 × 8 block 80 is divided into K7 sub-partitions 102, the dimensions of which are defined in table V.

Sub-partition index	Width of	Height
			1	6	6
2	6	2
			3	2	8
4	5	3
			5	3	3
6	8	4
			7	8	1

Table V: the dimensions of the sub-partitions depicted in FIG. 12b

The above embodiments have so far produced an example segmentation of the just mentioned segmentation. For example, fig. 4 shows two examples of such FIPS segmentation of block 80. However, in this case, all of the child partitions 102/112 have the same size. In particular, perpendicular to dimension 104 they have the width of the block, while along dimension 104 they have an equal width, corresponding to a certain fraction of the block width along dimension 104. Similar to any of the partitions 102/112 described above, each of the generalized sub-partitions 102 generated in FIPS, for example, is predicted and the resulting residual signals are transformed, quantized, and encoded separately. Thus, its reconstructed value may be used to predict other sub-partitions 102 that have not yet been encoded/decoded. For example, in the case of block 80 depicted in fig. 12b, sub-partition No. 1 may be predicted first, and then sub-partitions No. 2 and No. 3 may be predicted using their reconstructed samples. The process is repeated until all sub-partitions have been encoded/decoded.

Summarizing FIPS generalization, the partitioning of intra-predicted blocks 80 into partitions 102/112 may be defined and signaled via the data stream in a manner different from that discussed above, such as, for example, using recursive multi-tree subdivision of blocks 80 into rectangular partitions of different sizes or into any other possibility that a partition may be defined. For example, the partitioning may be defined by the derivation of dimension 104 in the manner discussed above to determine whether the partition of block 80 is a horizontal slice 102 or a vertical slice 112 based thereon, and as far as the lateral width of the slices is concerned, i.e. the height of horizontal slice 102 and the width of vertical slice 112, respectively, are dependent on the intra-prediction mode. All other possibilities discussed above are still such as, for example, the possibility to change the processing/partitioning order 126, which, as mentioned above, may depend on the intra-prediction mode, some syntax elements and intra-prediction mode parameters transmitted in the data stream for block 80 or from some other information transmitted in the data stream such as information that may be derived from the prediction residuals transmitted in the data stream for the respective partition.

Furthermore, as already discussed above, the prediction residuals for the respective partitions of block 80 are quantized and encoded into the data stream sequentially, partition by partition, in an alternating manner with respect to the generation of the intra prediction for the partitions. This has been described with reference to fig. 5 and also with reference to fig. 10 and 11. However, the decoder does not need to perform decoding by alternating between decoding of the residual on the one hand and reconstruction of the respective partition by combining intra prediction and prediction residual on the other hand. That is, upon decoding a certain block 80, the decoder may decouple the decoding of the prediction residual 120 for each partition from the actual reconstruction process comprising separate intra prediction for each partition. Please refer to fig. 5: the decoder may decode the prediction residuals 120 of all partitions, i.e. the prediction residuals 120 of the block 80, from the data stream 14 according to one processing task₁To 120₄And the decoder may use the partition 102 according to another processing task_iIs predicted to be the residual 120_iSo as to partition-by-partition reconstruct the interior of block 80 in partition order 126. To this end, the decoder performs each partition 120 using the intra prediction mode of the block 80 in the second task_iThen adding the prediction residual 120 resulting from the first task_iAnd then proceeds to the next partition 120 in partition order 126_i+1The reconstruction of block 80 is continued to perform spatial prediction, after which it is reconstructed using the prediction residuals of the partitions to correct the intra prediction result. The decoder may complete a first task of fully deriving the prediction residual 120 from the data stream 14 before starting a second task of performing prediction and prediction correction using the prediction residual, or the decoder may perform both tasks in parallel in the measures provided to ensure that a certain partition 102 is guaranteed_iIs predicted to be the residual 120_iIs ready when needed, i.e. when the intra-prediction mode of the block has been used to obtain the partition 102_iAnd need to be corrected. In particular, during a first task or phase, the decoder may perform all inverse transforms on all non-zero partitions in parallel, i.e. on the signaled prediction residual 120_iAll inverse transforms are performed for all the partitions 102 that are non-zero.

As indicated in one aspect, according to the above embodiment, where the residual 120 of partition i_iQuantized in the transform domain, it may happen that: the reconstructed samples of a partition may leave, i.e. exceed or overrun, a certain allowed range of sample values. As discussed above, they may function as members of the reference samples 118j for partition j in the order 126. According to an embodiment, these samples are left as they are used for the purpose of predicting partition j in order 126, with the clipping of these samples of block 80 being performed as a final clipping step for the whole block 80, thereby increasing the implementation friendliness, e.g. at the decoder side. Thus, partition 102 is now being considered_iIn the derived predictor, this partition 102 is in accordance with a partition order 126_iReconstructed samples of the previous partition, which are among one or more already reconstructed samples 118_iCan be used as a reference for the current partition in the system, and can be limitedThe state of amplitude is used, in which after sequential reconstruction has been performed, in the case of reconstructing a predetermined block at the end, the reconstructed samples are clipped from the not-yet-clipped state to the state clipped to the allowable sample value range. At the encoder side, clipping is performed only to obtain a reconstructed version of such samples, to be used as a prediction reference for subsequently encoded blocks, in order to maintain synchronization with the reference of the decoder. However, this final clean-up type of slicing is merely exemplary, and slicing may instead be performed immediately, i.e., sliced before the reconstructed sample of partition i is used as a reference sample 118j for subsequent processing of partition j.

In the embodiments discussed so far, one of them will be described below using a specific example. Specifically, according to an embodiment, data stream 14 signals whether intra-coded block 80 encodes it using an LIP or ISP scheme via partition mode flag 160. The corresponding syntax element in the data stream 14 may be named intra _ sub _ modes _ flag. For example, if the flag is 1, the intra-coded block 80 may be encoded using an LIP or ISP scheme, otherwise the block 80 is encoded using normal intra prediction. For example, an LIP or ISP scheme may be used for the current intra coded block 80 as long as one or more conditions are met. For example, the one or more conditions may include: for example, intra-coded blocks 80 need to be larger than some minimum size in terms of the number of samples of blocks 80, and/or intra-coded blocks 80 may not be allowed to exceed a certain size, at least in the horizontal and vertical directions, for example, in order not to result in too large a transform size. More specifically, the LSP or ISP mode may only be available if block 80 is less than or equal to the just-mentioned maximum transform correlation size in at least one direction, i.e., horizontal or vertical. Thus, intra _ sub _ modes _ flag will only be present in the data stream if block 80 satisfies the just mentioned condition. Otherwise, the decoder may conclude that intra-coded block 80 is normally intra-coded. The partition dimension flag 114 may be further signaled for an intra coded block 80 in the case that the partition mode flag, intra _ sub _ partitions _ mode _ flag, indicates that the intra coded block 80 is an LSP or ISP coded block. However, it is possible that the intra _ sub partitions _ mode _ flag is not explicitly signaled to avoid, but is inferred to indicate a certain partition dimension 104 in some cases. For example, in the case where the width of an intra-coded block 80 exceeds the maximum transform size described above (but its height does not exceed), then the partition dimension 104 may be forced to be horizontal, and in the case where the height of a block 80 exceeds the just-mentioned maximum transform size (but its width does not exceed), then the dimension 104 may be forced to be vertical. In both cases, the intra _ sub _ partitions _ split _ flag will not be explicitly signaled in the data stream, but inferred accordingly by the decoder. The intra-coding mode 116 may be signaled in the data stream as described above, i.e. by using a list of most probable intra-prediction modes built at the side of the encoder and decoder. Although for the LIP or ISP intra-coded blocks 80, the data stream 14 may signal the intra-coding mode by means of an MPM list pointer code that inevitably points into a list of possible intra-prediction modes, said MPM list pointer being referred to, for example, as intra _ luma _ MPM _ IDX, which may precede the MPM flag in the data stream 14 in case the intra-coded blocks are not coded in the LIP or ISP scheme. If the MPM flag, e.g. called intra _ luma _ MPM _ flag, has a specific flag state instead of a pointer to the list of most probable intra prediction modes, the pointer code to the remaining list of intra prediction modes is signaled in the data stream. However, as mentioned above, this is only an example, and the set of intra prediction modes that may be signaled may be the same, i.e. cover all supported intra prediction modes, for both the normally coded intra prediction blocks and the LIP or ISP intra prediction blocks. For example, intra _ luma _ mpm _ flag may be transmitted for two types of intra-coded blocks. Alternatively, for both types of intra-coded blocks, the pointer code transmitted for both types of intra-predicted blocks may point directly to the complete list of supported intra-prediction modes without any MPM flags. If intra-coded block 80 is coded using an LIP or ISP scheme, the number of partitions 102/112 may be defined as follows. In particular, the encoder and decoder may determine the number of partitions according to the size of the block 80. No signal is spent in the data stream. For small block sizes, the number may be two, otherwise the number of partitions 102/112 may be four. The partitioning order in which intra-partition prediction and prediction residual coding in the data stream is performed may be sequentially directed from the leftmost partition in case of the horizontal direction n14 and from the uppermost partition in case of the vertical partitioning direction along the partitioning direction 104 to the farthest partition. This does not take any signal. As described above, the residual transform may be done for each partition 102/112. That is, each partition may be transformed individually. In contrast, in the case of a normal intra block 80, the number of transforms may depend on the size of intra block 80, as described below: if the intra-coded block is smaller than the above-mentioned maximum transform size in the horizontal and vertical directions, the residual of the intra-coded block 80 is coded using one transform, i.e., the residual of the block 80 is completely subjected to one transform. In case the horizontal direction exceeds the maximum transform size, the intra coded block 80 is divided horizontally into two halves or a corresponding number of transform blocks such that the halves or transform blocks satisfy the maximum transform size and the residual of the block 80 undergoes one transform for each half/transform block. This is also the case where block 80 exceeds the maximum transform size in the vertical direction. If the maximum transform size is exceeded in both the vertical and horizontal directions, four or a corresponding number of transforms are used to transform the residual of block 80 in the four quadrants of this block 80, or the residual of a conventional two-dimensional subdivision of transform block 80 into a corresponding number of transform blocks. Furthermore, the processing of the normal intra coded blocks 80 may deviate from the processing of the intra coded blocks 80 of the LIP or ISP coding because the normal intra coded blocks are intra predicted in whole blocks. That is, it is not subdivided. Further differences may involve coding of a transform of the prediction residual for coding block 80. For each transform, a coded block flag 188 may be conveyed, such as tu _ CBF _ luma, but for normal intra-coded blocks 80, this flag may inevitably be coded for each transform within block 80, which may be inferred to be one of the last transforms for block 80 in the case where block 80 is LIP or ISB coded and all previous CBFs of previous transforms are zero. Furthermore, the selection of the size of the sub-blocks within each transform may differ between the normal intra coding blocks 80 on the one hand and the LIP or ISP coding blocks 80 on the other hand. The details are as described above. However, the transform 182 may be equally sub-partitioned into sub-blocks for normal intra coded blocks and LIP or ISP coded blocks instead. For example, let log2SbW and log2SbH be the logarithmic pairs of the widths and heights of these sub-blocks, respectively, and log2TbWidth and log2TbHeight be the widths and heights of the transforms, respectively. The sub-block size may then be determined as follows:

the above pseudo code produces sub-blocks of the sizes listed in table 1. It is possible that a normal intra block 80 results in only 4 x 4 coefficient sub-blocks due to the inherent minimum size of the intra block 80 and the non-sub-division of the normal intra block. It should finally be noted that the example just outlined may result in different sizes of LIP or intra-ISP prediction blocks, including blocks partitioned into only two partitions 102/112, but that LIP or intra-ISP prediction blocks partitioned into more than two partitions exist, whether or not such LIP or intra-ISP prediction blocks exist.

Thus, the above description discloses, among other things, a block-based coding concept for an image 12 that involves signaling an intra-coding mode 116 in the data stream 14 for a predetermined block 80 of the image and partitioning the predetermined block 80 into partitions 102_i/112_i(ii) a Wherein the partitions of the predetermined block are to be reconstructed according to a predetermined partition order 126 that sequentially traverses the partitions, for a current partition and before proceeding to a subsequent partition, filling in by depending on one or more already reconstructed/encoded samples adjacent to the current partition in a manner dependent on the intra-coding modePre-partition, deriving predictor 122 for the current partition, and by using prediction residual 120 signaled in the data stream_iThe predictor 122 is corrected to reconstruct the current partition. As mentioned above, the segmentation may be performed along the predetermined dimension 104 such that the partitions are as wide as the predetermined blocks 80 in a direction perpendicular to the predetermined dimension 104, and the width of the partitions measured along the predetermined dimension 104 may be selected by the encoder and decoder from at least two different width settings depending on the data stream, such as depending on the size of the predetermined blocks, such as by selecting the number of partitions into which the predetermined blocks are segmented, the segmentation being according to the size of the dimensions of the blocks or the size of the blocks along the predetermined dimension. The prediction residual coding of all partitions can be done using any of the above concepts, in particular in context selection and/or coding syntax. Furthermore, any concept of mode delivery for including or excluding MPM list construction may be used.

Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the respective method, wherein a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such a device.

The data stream of the present invention may be stored on a digital storage medium or may be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the internet.

Embodiments of the present invention may be implemented in hardware or software, as desired for certain implementations. The implementation can be performed using a digital storage medium, such as a floppy disk, DVD, blu-ray, CD, ROM, PROM, EPROM, EEPROM or flash memory having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Accordingly, the digital storage medium may be computer-readable.

Some embodiments according to the invention comprise a data carrier with electronically readable control signals capable of cooperating with a programmable computer system so as to carry out one of the methods described herein.

Generally, embodiments of the invention can be implemented as a computer program product having a program code which is operable to perform one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

Other embodiments include a computer program, stored on a machine-readable carrier, for performing one of the methods described herein.

In other words, an embodiment of the inventive method is thus a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) comprising a computer program recorded thereon for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium is typically tangible and/or non-transitory.

A further embodiment of the inventive method is thus a data stream or a signal sequence representing a computer program for performing one of the methods described herein. The data stream or signal sequence may for example be arranged to be transmitted via a data communication connection, for example via the internet.

Further embodiments include a processing device, such as a computer, or a programmable logic device, configured or adapted to perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon a computer program for performing one of the methods described herein.

Further embodiments according to the invention include an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, or the like. The apparatus or system may for example comprise a file server for transmitting the computer program to the receiver.

In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform the functions of some or all of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the method is preferably performed by any hardware device.

The apparatus described herein may be implemented using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

The apparatus described herein, or any component of the apparatus described herein, may be implemented at least in part in hardware and/or software.

The methods described herein may be implemented using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

The methods described herein, or any components of the apparatus described herein, may be implemented at least in part in hardware and/or software.

The above-described embodiments are merely illustrative of the principles of the present invention. It is to be understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. Therefore, it is intended that the present disclosure be limited only by the scope of the appended patent claims and not by the specific details presented by the examples and illustrations herein.

52页详细技术资料下载

上一篇：一种医用注射器针头装配设备

下一篇：视频译码中的分块指示

Intra-partition coding concept

相关技术

网友询问留言