阅读说明：本技术 简化的最可能模式列表生成方案 (Simplified most probable mode list generation scheme ) 是由 赵亮 赵欣 李翔 刘杉 于 2019-12-04 设计创作，主要内容包括：一种帧内预测模式发送方法,所述帧内预测模式用于对已编码视频比特流中的当前块进行编码,所述方法包括：生成与所述当前块的零参考行相对应的第一最可能模式MPM列表,所述第一MPM列表包括多个角度帧内预测模式；生成与所述当前块的至少一个非零参考行相对应的第二MPM列表,所述第二MPM列表包括所述多个角度帧内预测模式；发送参考行索引,所述参考行索引指示在所述零参考行和所述至少一个非零参考行中用于对所述当前块进行编码的参考行；及,发送帧内模式索引,所述帧内模式索引指示所述第一MPM列表内的帧内预测模式或者所述第二MPM列表内的帧内预测模式。(An intra prediction mode transmission method for encoding a current block in an encoded video bitstream, the method comprising: generating a first MPM list corresponding to a zero reference line of the current block, the first MPM list including a plurality of angular intra prediction modes; generating a second MPM list corresponding to at least one non-zero reference line of the current block, the second MPM list comprising the plurality of angular intra prediction modes; transmitting a reference line index indicating a reference line used for encoding the current block in the zero reference line and the at least one non-zero reference line; and transmitting an intra-mode index indicating an intra-prediction mode within the first MPM list or an intra-prediction mode within the second MPM list.)

1. An intra prediction mode transmission method for encoding a current block in an encoded video bitstream, the method using at least one processor, comprising:

generating a first MPM list corresponding to a zero reference line of the current block, the first MPM list including a plurality of angular intra prediction modes;

generating a second MPM list corresponding to at least one non-zero reference line of the current block, the second MPM list comprising the plurality of angular intra prediction modes;

transmitting a reference line index indicating one of the at least one non-zero reference line and the zero reference line for encoding the current block; and a process for the preparation of a coating,

transmitting an intra-mode index indicating an intra-prediction mode within the first MPM list or an intra-prediction mode within the second MPM list.

2. The method of claim 1, wherein the first MPM list further includes at least one non-angular intra prediction mode, and the second MPM list does not include the at least one non-angular intra prediction mode.

3. The method of claim 2, wherein the at least one non-angular intra prediction mode comprises at least one of a planar mode and a DC mode.

4. The method of claim 1, wherein the first MPM list comprises a first plurality of indices corresponding to the plurality of angular intra-prediction modes, the second MPM list comprises a second plurality of indices corresponding to the plurality of angular intra-prediction modes, the first plurality of indices being the same as the second plurality of indices.

5. The method of claim 4, wherein the plurality of angular intra prediction modes comprises a first angular intra prediction mode and a second angular intra prediction mode;

the first angular intra prediction mode corresponds to a first index of the first plurality of indices;

the second angular intra-prediction mode corresponds to a second index of the first plurality of indices;

the first angular intra-prediction mode corresponds to a first index of the second plurality of indices;

the second angular intra-prediction mode corresponds to a second index of the second plurality of indices;

a first index of the first plurality of indices is the same as a first index of the second plurality of indices;

a second index of the first plurality of indices is the same as a second index of the second plurality of indices.

6. The method of claim 1, wherein if a first neighboring mode of a first neighboring block of the current block is a non-angular intra prediction mode and a second neighboring mode of a second neighboring block of the current block is an angular intra prediction mode, the first intra prediction mode of the first MPM list is the non-angular intra prediction mode and the second intra prediction mode of the first MPM list is the angular intra prediction mode.

7. The method of claim 6, wherein the first intra prediction mode of the first MPM list is the DC mode if the first neighboring mode is the DC mode and the reference row index indicates that the reference row is the zero reference row.

8. The method of claim 6, wherein the first intra prediction mode of the first MPM list is the planar mode if the first neighboring mode is the planar mode and the reference line index indicates that the reference line is the zero reference line.

9. The method of claim 6, wherein, if the reference row index indicates that the reference row is the zero reference row, the first intra prediction mode of the first MPM list is a planar mode, the second intra prediction mode of the first MPM list is the angular intra prediction mode, and a third intra prediction mode of the first MPM list is a DC mode.

10. The method of claim 1, wherein if the reference row index indicates that the reference row is the zero reference row, a first neighbor mode of a first neighbor block of the current block is a non-angular intra prediction mode, and a second neighbor mode of a second neighbor block of the current block is an angular intra prediction mode, then the first intra prediction mode of the first MPM list is the angular intra prediction mode and the second intra prediction mode of the first MPM list is the non-angular intra prediction mode.

11. The method of claim 1, wherein if the reference row index indicates that the reference row is the zero reference row, a first neighboring mode of a first neighboring block of the current block is a non-angular intra prediction mode, and a second neighboring mode of a second neighboring block of the current block is an angular intra prediction mode, determining a location of at least one of a planar mode and a DC mode within the first MPM list relative to the angular intra prediction mode according to a mode number of the angular intra prediction mode.

12. The method of claim 11, wherein, if the angular intra prediction mode is one of a horizontal mode and a vertical mode, a first intra prediction mode of the first MPM list is the angular intra prediction mode, and a second intra prediction mode of the first MPM list is at least one of the planar mode and the DC mode;

the first intra prediction mode of the first MPM list is at least one of the planar mode and the DC mode if the angular intra prediction mode is one mode other than the horizontal mode and the vertical mode, and the second intra prediction mode of the first MPM list is the angular intra prediction mode.

13. The method of claim 11, wherein, if the angular intra prediction mode is one of a horizontal mode and a vertical mode, a first intra prediction mode of the first MPM list is the angular intra prediction mode, and a second intra prediction mode of the first MPM list is at least one of the planar mode and the DC mode;

if the angular intra prediction mode is one other than the horizontal mode and the vertical mode, the first intra prediction mode of the first MPM list is a left neighbor mode of a left neighbor block of the current block, and the second intra prediction mode of the first MPM list is an upper neighbor mode of an upper neighbor block of the current block.

14. The method of claim 11, wherein, if the angular intra prediction mode is one in which no interpolation is used to generate prediction samples at fractional positions, a first intra prediction mode of the first MPM list is the angular intra prediction mode, and a second intra prediction mode of the first MPM list is at least one of the planar mode and the DC mode;

the first intra prediction mode of the first MPM list is at least one of the planar mode and the DC mode, and the second intra prediction mode of the first MPM list is the angular intra prediction mode, if the angular intra prediction mode is one in which prediction samples are generated at fractional positions using interpolation.

15. An intra prediction mode transmission apparatus for encoding a current block in an encoded video bitstream, comprising:

at least one memory for storing program code;

at least one processor configured to read the program code and operate according to instructions of the program code, the program code comprising:

generating, by the at least one processor, a first MPM list corresponding to a zero reference line of the current block, the first MPM list comprising a plurality of angular intra prediction modes;

second generating code for causing the at least one processor to generate a second MPM list corresponding to at least one non-zero reference line of the current block, the second MPM list comprising the plurality of angular intra prediction modes;

means for transmitting a reference line index indicating a reference line used for encoding the current block in the zero reference line and the at least one non-zero reference line; and a process for the preparation of a coating,

means for transmitting an intra-mode index indicating an intra-prediction mode within the first MPM list or an intra-prediction mode within the second MPM list.

16. The apparatus of claim 15, wherein the first MPM list further comprises at least one non-angular intra prediction mode, and the second MPM list does not comprise the at least one non-angular intra prediction mode.

17. The apparatus of claim 15, wherein the first MPM list comprises a first plurality of indices corresponding to the plurality of angular intra-prediction modes, the second MPM list comprises a second plurality of indices corresponding to the plurality of angular intra-prediction modes, the first plurality of indices being the same as the second plurality of indices.

18. The apparatus of claim 17, wherein the plurality of angular intra prediction modes comprises a first angular intra prediction mode and a second angular intra prediction mode;

the first angular intra prediction mode corresponds to a first index of the first plurality of indices;

the second angular intra-prediction mode corresponds to a second index of the first plurality of indices;

the first angular intra-prediction mode corresponds to a first index of the second plurality of indices;

the second angular intra-prediction mode corresponds to a second index of the second plurality of indices;

a first index of the first plurality of indices is the same as a first index of the second plurality of indices;

a second index of the first plurality of indices is the same as a second index of the second plurality of indices.

19. The apparatus of claim 15, wherein if the reference row index indicates that the reference row is the zero reference row, a first neighbor mode of a first neighbor block of the current block is a non-angular intra prediction mode, and a second neighbor mode of a second neighbor block of the current block is an angular intra prediction mode, then the first intra prediction mode of the first MPM list is the angular intra prediction mode and the second intra prediction mode of the first MPM list is the non-angular intra prediction mode.

20. A non-transitory computer-readable medium having stored thereon instructions, the instructions comprising at least one instruction, which when executed by an intra prediction mode transmitting device, is to encode a current block in an encoded video bitstream, cause the at least one processor to:

generating a first MPM list corresponding to a zero reference line of the current block, the first MPM list including a plurality of angular intra prediction modes;

generating a second MPM list corresponding to at least one non-zero reference line of the current block, the second MPM list comprising the plurality of angular intra prediction modes;

transmitting a reference line index indicating one of the at least one non-zero reference line and the zero reference line for encoding the current block; and a process for the preparation of a coating,

transmitting an intra-mode index indicating an intra-prediction mode within the first MPM list or an intra-prediction mode within the second MPM list.

Technical Field

The present application relates to advanced video coding and decoding techniques, and more particularly, to a simplified most probable mode list generation scheme for zero reference lines and non-zero reference lines.

Background

The H.265/HEVC (high efficiency video coding) standards were published in 2013 (version 1), 2014 (version 2), 2015 (version 3) and 2016 (version 4) by ITU-T VCEG (Q6/16) and ISO/IEC MPEG (JTC 1/SC 29/WG 11). In 2015, these two standardization bodies jointly formed a joint video exploration team (jfet), and focused on the potential needs of developing future video coding technology standards beyond the HEVC standard. In 10 months 2017, they issued a joint call, calling for proposals for video compression with capabilities beyond hevc (cfp). By 2018, 2, 15 days, a total of 22 CfP responses for Standard Dynamic Range (SDR), 12 CfP responses for High Dynamic Range (HDR), and 12 CfP responses for 360 video categories were submitted, respectively. In month 4 2018, all received CfP responses were evaluated at the 122 Moving Picture Experts Group (MPEG)/tenth jvt conference. As a result of this meeting, jfet formally initiated standardization of next generation video coding beyond HEVC. The new standard is named universal video coding (VVC) and jfet is renamed joint video experts group.

The intra prediction modes used in HEVC are shown in fig. 1. In HEVC, there are a total of 35 intra prediction modes, with mode 10 being the horizontal mode, mode 26 being the vertical mode, and mode 2, mode 18, and mode 34 being the diagonal modes. The intra prediction mode is identified by three Most Probable Modes (MPMs) and 32 remaining modes.

To codec an intra mode, a Most Probable Mode (MPM) list of size 3 is established based on intra modes of neighboring blocks. This MPM list will be referred to as an MPM list or master MPM list. If the intra mode is not from the MPM list, a flag is sent to indicate whether the intra mode belongs to the selected mode.

One example of an MPM list generation process for HEVC is shown below:

·If(leftIntraDir＝＝aboveIntraDir&&leftIntraDir>DC_IDX)

οMPM[0]＝leftIntraDir；

οMPM[1]＝((leftIntraDir+offset)％mod)+2；

οMPM[2]＝((leftIntraDir-1)％mod)+2；

·Else if(leftIntraDir＝＝aboveIntraDir)

οMPM[0]＝PLANAR_IDX；

οMPM[1]＝DC_IDX；

οMPM[2]＝VER_IDX；

·Else if(leftIntraDir！＝aboveIntraDir)

οMPM[0]＝leftIntraDir；

οMPM[1]＝aboveIntraDir；

οIf(leftIntraDir>0&&aboveIntraDir>0)

■MPM[2]＝PLANAR_IDX；

οElse

■MPM[2]＝(leftIntraDir+aboveIntraDir)<2VER_IDX:

DC_IDX；

here, leftIntraDir is used to indicate a mode in the left block, and above intradir is used to indicate a mode in the upper block. If the left block is not currently available, then leftIntraDir or above entry Dir will be DC _ IDX. Further, the variables "offset" and "mod" are constant values, which are set to 29 and 32, respectively.

Disclosure of Invention

An embodiment of the present application provides an intra prediction mode transmission method, where the intra prediction mode is used to encode a current block in an encoded video bitstream, and the method uses at least one processor, and includes:

generating a first MPM list corresponding to a zero reference line of the current block, the first MPM list including a plurality of angular intra prediction modes;

generating a second MPM list corresponding to at least one non-zero reference line of the current block, the second MPM list comprising the plurality of angular intra prediction modes;

transmitting a reference line index indicating a reference line used for encoding the current block in the zero reference line and the at least one non-zero reference line; and a process for the preparation of a coating,

transmitting an intra-mode index indicating an intra-prediction mode within the first MPM list or an intra-prediction mode within the second MPM list.

An embodiment of the present application further provides an intra prediction mode transmission apparatus, where the intra prediction mode is used to encode a current block in an encoded video bitstream, and the intra prediction mode includes:

at least one memory for storing program code;

at least one processor configured to read the program code and operate according to instructions of the program code, the program code comprising:

generating, by the at least one processor, a first MPM list corresponding to a zero reference line of the current block, the first MPM list comprising a plurality of angular intra prediction modes;

second generating code for causing the at least one processor to generate a second MPM list corresponding to at least one non-zero reference line of the current block, the second MPM list comprising the plurality of angular intra prediction modes;

means for transmitting a reference line index indicating a reference line used for encoding the current block in the zero reference line and the at least one non-zero reference line; and a process for the preparation of a coating,

means for transmitting an intra-mode index indicating an intra-prediction mode within the first MPM list or an intra-prediction mode within the second MPM list.

The present embodiments also provide a non-transitory computer-readable medium having instructions stored thereon, the instructions including at least one instruction, which when executed by an intra prediction mode transmitting device, causes the at least one processor to:

generating a first MPM list corresponding to a zero reference line of the current block, the first MPM list including a plurality of angular intra prediction modes;

generating a second MPM list corresponding to at least one non-zero reference line of the current block, the second MPM list comprising the plurality of angular intra prediction modes;

transmitting a reference line index indicating one of the at least one non-zero reference line and the zero reference line for encoding the current block; and a process for the preparation of a coating,

transmitting an intra-mode index indicating an intra-prediction mode within the first MPM list or an intra-prediction mode within the second MPM list.

Drawings

Other features, properties, and various advantages of the disclosed subject matter will be further apparent from the following detailed description and the accompanying drawings, in which:

fig. 1 shows an example of intra prediction modes in HEVC;

fig. 2 shows an example of a reference row adjacent to a coding block unit;

fig. 3 shows an example of an intra prediction mode in VVC;

FIG. 4 shows an example of the location of neighboring CUs;

FIG. 5 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment;

FIG. 6 shows a block diagram of the location of a video encoder and a video decoder in a streaming environment, in accordance with an embodiment;

FIG. 7 shows a block diagram of a video decoder according to an embodiment;

FIG. 8 shows a block diagram of a video encoder according to an embodiment;

FIG. 9 depicts a flowchart of an exemplary process for sending an intra prediction mode for encoding a current block in an encoded video stream, according to an embodiment; and

FIG. 10 shows a schematic diagram of a computer system, according to an embodiment.

Detailed Description

Multiline intra prediction is proposed to use more reference lines for intra prediction, and the encoder decides and signals which reference line is used to generate the intra predictor. The reference row index is sent before the intra prediction mode and in case of sending a non-zero reference row index only the most probable mode is allowed. In fig. 2, an example of 4 reference lines is depicted, where each reference line comprises six segments, i.e. segments a to F, and the upper left reference sample. In addition, the nearest samples from segments B and E are used to pad segments a and F, respectively.

In VVC, as shown in fig. 3, there are 95 intra prediction modes in total, in which the mode 18 is the horizontal mode, the mode 50 is the vertical mode, and the mode 2, the mode 34, and the mode 66 are the diagonal modes. Modes-1 to-14 and 67 to 80 are referred to as wide-angle intra prediction (WAIP) modes.

In VTM3.0, the size of the MPM list is set to 6 for both adjacent reference lines (also referred to as zero reference lines) and non-adjacent reference lines (also referred to as non-zero reference lines). The positions of the neighboring patterns used to derive the 6 MPM candidates are also the same for neighboring and non-neighboring reference rows, as shown in fig. 4. In fig. 4, blocks a and B represent upper and left neighboring coding units of a current coding unit, and variables candirapredmodea and candirapredmodeb indicate intra prediction modes associated with the blocks a and B, respectively. candirapredmodea and candirapredmodeb are initially set to INTRA _ plan. If block a (or B) is marked as available, candirapredmodea (or candirapredmodeb) is set to the actual intra prediction mode of block a (or B).

The MPM candidate derivation process is different for neighboring and non-neighboring reference rows. For the zero reference row, if both neighboring modes are planar mode or DC mode, a default mode is used to construct the MPM list, of which 2 are planar mode and DC mode, and the remaining 4 modes are angular intra prediction modes, which may also be referred to as angular default modes. For non-zero reference rows, if both neighboring modes are planar or DC, the MPM list is constructed using the 6-angle default mode.

An example of the MPM list derivation process is shown below, where candModeList [ x ], x ═ 0.. 5, represents 6 MPM candidates, intralumarefoliedx [ xCb ] [ yCb ] represents the reference row index of the block to be predicted, and intralumarefoliedx [ xCb ] [ yCb ] may be 0, 1, or 3.

-if candxtrapredmodeb is equal to candxtrapredmodea and candxtrapredmodea is greater than INTRA _ DC, candModeList [ x ], x ═ 0.. 5, derived as follows:

-if IntraLumaRefLineIdx [ xCb ] [ yCb ] is equal to 0:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝INTRA_PLANAR

candModeList[2]＝INTRA_DC

candModeList[3]＝2+((candIntraPredModeA+61)％64)

candModeList[4]＝2+((candIntraPredModeA-1)％64)

candModeList[5]＝2+((candIntraPredModeA+60)％64)

else (intralumeforenlieidx [ xCb ] [ yCb ] not equal to 0), then:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝2+((candIntraPredModeA+61)％64)

candModeList[2]＝2+((candIntraPredModeA-1)％64)

candModeList[3]＝2+((candIntraPredModeA+60)％64)

candModeList[4]＝2+(candIntraPredModeA％64)

candModeList[5]＝2+((candIntraPredModeA59)％64)

else, if candirapredmodeb does not equal candirapredmodea, and candirapredmodea or candirapredmodeb is greater than INTRA _ DC:

the variables minAB and maxAB are derived as follows:

minAB＝candModeList[(candModeList[0]>candModeList[1])？1：0]

maxAB＝candModeList[(candModeList[0]>candModeList[1])？0：1]

andmodelist [ x ], x ═ 0.. 5, if candxtrapremoda and candxtrapremobb are both greater than INTRA _ DC, derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

-if IntraLumaRefLineIdx [ xCb ] [ yCb ] is equal to 0:

candModeList[2]＝INTRA_PLANAR

candModeList[3]＝INTRA_DC

-if maxAB-minAB is in the range of [2, 62 ]:

candModeList[4]＝2+((maxAB+61)％64)

candModeList[5]＝2+((maxAB-1)％64)

-otherwise:

candModeList[4]＝2+((maxAB+60)％64)

candModeList[5]＝2+((maxAB)％64)

else (intralumeforenlieidx [ xCb ] [ yCb ] not equal to 0), then:

-if maxAB-minAB is equal to 1:

candModeList[2]＝2+((minAB+61)％64)

candModeList[3]＝2+((maxAB-1)％64)

candModeList[4]＝2+((minAB+60)％64)

candModeList[5]＝2+(maxAB％64)

else, if maxAB-minAB equals 2, then:

candModeList[2]＝2+((minAB-1)％64)

candModeList[3]＝2+((minAB+61)％64)

candModeList[4]＝2+((maxAB-1)％64)

candModeList[5]＝2+((minAB+60)％64)

else, if maxAB-minAB is greater than 61:

candModeList[2]＝2+((minAB-1)％64)

candModeList[3]＝2+((maxAB+61)％64)

candModeList[4]＝2+(minAB％64)

candModeList[5]＝2+((maxAB+60)％64)

-otherwise:

candModeList[2]＝2+((minAB+61)％64)

candModeList[3]＝2+((minAB-1)％64)

candModeList[4]＝2+((maxAB+61)％64)

candModeList[5]＝2+((maxAB-1)％64)

else (candirapredmodea or candirapredmodeb is greater than INTRA _ DC), candModeList [ x ], x ═ 0.. 5, derived as follows:

-if IntraLumaRefLineIdx [ xCb ] [ yCb ] is equal to 0:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

candModeList[2]＝1-minAB

candModeList[3]＝2+((maxAB+61)％64)

candModeList[4]＝2+((maxAB-1)％64)

candModeList[5]＝2+((maxAB+60)％64)

else (intralumeforenlieidx [ xCb ] [ yCb ] not equal to 0), then:

candModeList[0]＝maxAB

candModeList[1]＝2+((maxAB+61)％64)

candModeList[2]＝2+((maxAB-1)％64)

candModeList[3]＝2+((maxAB+60)％64)

candModeList[4]＝2+(maxAB％64)

candModeList[5]＝2+((maxAB+59)％64)

-otherwise:

-if IntraLumaRefLineIdx [ xCb ] [ yCb ] is equal to 0:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝(candModeList[0]＝＝INTRA_PLANAR)？INTRA_DC：

INTRA_PLANAR

candModeList[2]＝INTRA_ANGULAR50

candModeList[3]＝INTRA_ANGULAR18

candModeList[4]＝INTRA_ANGULAR46

candModeList[5]＝INTRA_ANGULAR54

else (intralumeforenlieidx [ xCb ] [ yCb ] not equal to 0), then:

candModeList[0]＝INTRA_ANGULAR50

candModeList[1]＝INTRA_ANGULAR18

candModeList[2]＝INTRA_ANGULAR2

candModeList[3]＝INTRA_ANGULAR34

candModeList[4]＝INTRA_ANGULAR66

candModeList[5]＝INTRA_ANGULAR26

in VTM4.0, the size of the MPM list is extended to 6. When intra _ luma _ MPM _ flag is true, it indicates that the current mode belongs to a candidate in the MPM list. Consider the following table 1:

table 1:

fig. 5 is a simplified block diagram of a communication system (300) according to an embodiment disclosed herein. The communication system (300) comprises at least two terminal devices (510) and (520) interconnected via a network (550). For unidirectional data transmission, the first terminal (510) encodes video data at a local location and then transmits to the second terminal (520) over the network (550). The second terminal (520) receives video data encoded by another terminal from the network (550), decodes the encoded video data, and displays the restored video data. Unidirectional data transmission is common in applications such as media services.

Fig. 5 shows a second pair of terminals (530, 540) capable of supporting bidirectional transmission of encoded video, such as occurs during a video conference. For bi-directional data transmission, each terminal (530, 540) encodes the captured video data at a local location and then transmits the encoded video data to another terminal over a network (550). Each terminal (530, 540) may also receive encoded video data sent by another terminal, decode the encoded video data, and display the recovered video data on a local display device.

In fig. 5, the terminal (510-540) may be a server, a personal computer, and a smart phone, but the principles disclosed herein may not be limited thereto. Embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and/or dedicated video conferencing equipment. The network (550) represents any number of networks that transport encoded video data between terminals (510-540), including, for example, wired and/or wireless communication networks. The communication network (550) may exchange data in circuit-switched and/or packet-switched channels. Representative networks may include telecommunications networks, local area networks, wide area networks, and/or the internet. For purposes of this application, the architecture and topology of the network (550) may be immaterial to the operation disclosed herein, unless explained below.

As an example of the subject matter disclosed in this application, fig. 6 shows the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed herein is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, and the like.

The streaming system may include an acquisition subsystem (613), which may include a video source (601), such as a digital camera, that creates an uncompressed stream of video samples (602). The sample stream (602) is depicted as a thick line to emphasize the high data volume video sample stream as compared to the encoded video code stream, which can be processed by an encoder (603) coupled to the camera (601). The video encoder (603) may comprise hardware, software, or a combination of hardware and software to implement or embody aspects of the disclosed subject matter as described in more detail below. The encoded video codestream (604) is depicted as a thin line to emphasize lower data amounts than the sample stream, which may be stored on a streaming server (605) for future use. At least one streaming client subsystem (606, 608) may access the streaming server (605) to retrieve a copy (607,609) of the encoded video data (604). The client (606) includes a video decoder (610), the video decoder (610) decoding an incoming copy of the encoded video data (607) and generating an output stream of video samples (611) that can be presented on a display (612) or another presentation device (not depicted). In some streaming systems, the video streams (604, 607,609) may be encoded according to some video encoding/compression standard. Examples of such standards include ITU-T H.265. The Video Coding standard under development is informally referred to as next generation Video Coding (VVC), and the present application may be used in the context of the VVC standard.

Fig. 7 is a block diagram of a video decoder (610) according to an embodiment of the disclosure.

The receiver (710) may receive at least one encoded video sequence to be decoded by the video decoder (610); in the same or another embodiment, the encoded video sequences are received one at a time, wherein each encoded video sequence is decoded independently of the other encoded video sequences. The encoded video sequence may be received from a channel (712), which may be a hardware/software link to a storage device that stores encoded video data. The receiver (710) may receive encoded video data as well as other data, e.g. encoded audio data and/or auxiliary data streams, which may be forwarded to their respective usage entities (not indicated). The receiver (710) may separate the encoded video sequence from other data. To prevent network jitter, a buffer memory (715) may be coupled between the receiver (710) and the entropy decoder/parser (720) (hereinafter "parser (720)"). The buffer memory (715) may not need to be configured or may be made smaller when the receiver (710) receives data from a store/forward device with sufficient bandwidth and controllability or from an isochronous network. For use over a traffic packet network, such as the internet, a buffer memory (715) may also be required, which may be relatively large and may be of an adaptive size.

The video decoder (610) may include a parser (720) to reconstruct symbols (721) from the encoded video sequence. The categories of these symbols include information for managing the operation of the video decoder (610), as well as potential information to control a display device (612), which is not an integral part of the decoder, but may be coupled to the decoder, as shown in fig. 7. The control Information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (720) may parse/entropy decode the received encoded video sequence. Encoding of the encoded video sequence may be performed in accordance with video coding techniques or standards and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without contextual sensitivity, and so forth. A parser (720) may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. A subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and so on. The entropy decoder/parser may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

The parser (720) may perform entropy decoding/parsing operations on the video sequence received from the buffer memory (715), thereby creating symbols (721). A parser (720) may receive the encoded data, selecting a particular symbol (721) to decode. Further, the parser (720) may determine whether a particular symbol (721) may be provided to the motion compensated prediction unit (753), the scaler/inverse transform unit (751), the intra picture prediction unit (752), or the loop filter (756).

The reconstruction of the symbol (721) may involve at least two different units depending on the type of the encoded video picture or a portion of the encoded video picture (e.g., inter and intra pictures, inter and intra blocks), among other factors. Which units are involved and the way in which they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (720). For the sake of brevity, such a subgroup control information flow between the parser (720) and at least two units below is not described.

In addition to the functional blocks already mentioned, the video decoder (610) may be conceptually subdivided into several functional units as described below. In a practical embodiment operating under business constraints, many of these units interact closely with each other and may be integrated with each other. However, for the purposes of describing the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate.

The first unit is a sealer/inverse transform unit (751). A sealer/inverse transform unit (751) receives the quantized transform coefficients as symbols (721) from the parser (720) along with control information including which transform mode to use, block size, quantization factor, quantization scaling matrix, etc. The sealer/inverse transform unit (751) may output a block comprising sample values, which may be input into an aggregator (755).

In some cases, the output samples of sealer/inverse transform unit (751) may belong to an intra-coded block; namely: predictive information from previously reconstructed pictures is not used, but blocks of predictive information from previously reconstructed portions of the current picture may be used. Such predictive information may be provided by intra picture prediction unit (752). In some cases, the intra picture prediction unit (752) employs reconstructed information extracted from the current (partially reconstructed) picture (756) to generate surrounding blocks of the same size and shape as the block being reconstructed. In some cases, the aggregator (755) adds, on a per sample basis, the prediction information generated by the intra prediction unit (752) to the output sample information provided by the scaler/inverse transform unit (751).

In other cases, the output samples of sealer/inverse transform unit (751) may belong to inter-coded and potential motion compensated blocks. In this case, the motion compensated prediction unit (753) may access the reference picture memory (757) to extract samples for prediction. After motion compensating the extracted samples according to the sign (721), the samples may be added by an aggregator (755) to the output of the sealer/inverse transform unit (751), in this case referred to as residual samples or residual signals, generating output sample information. The fetching of prediction samples by the motion compensated prediction unit (753) from addresses within the reference picture memory (757) may be controlled by a motion vector, and the motion vector is used by the motion compensated prediction unit (753) in the form of said symbol (721), said symbol (721) for example comprising X, Y and a reference picture component. Motion compensation may also include interpolation of sample values fetched from reference picture memory, motion vector prediction mechanisms, etc., when using sub-sample exact motion vectors.

The output samples of the aggregator (755) may be employed in the loop filter unit (756) by various loop filtering techniques. The video compression techniques may include in-loop filter techniques that are controlled by parameters included in the encoded video sequence (also referred to as an encoded video bitstream), and which are available to the loop filter unit (756) as symbols (721) from the parser (720). However, in other embodiments, the video compression techniques may also be responsive to meta-information obtained during decoding of previous (in decoding order) portions of the encoded picture or encoded video sequence, as well as to sample values previously reconstructed and loop filtered.

The output of the loop filter unit (756) may be a sample stream that may be output to a display device (612) and stored in a reference picture memory (757) for subsequent inter picture prediction.

Once fully reconstructed, some of the coded pictures may be used as reference pictures for future prediction. For example, once the encoded picture corresponding to the current picture is fully reconstructed and the encoded picture is identified (by, e.g., parser (720)) as a reference picture, the current reference picture (656) may become part of the reference picture memory (757) and a new current picture buffer may be reallocated before starting reconstruction of a subsequent encoded picture.

The video decoder (610) may perform decoding operations according to predetermined video compression techniques, such as in the ITU-T h.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence conforms to the syntax of the video compression technique or standard and the configuration files recorded in the video compression technique or standard. In particular, the configuration file may select certain tools from all tools available in the video compression technology or standard as the only tools available under the configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the limits defined by the level of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in units of, e.g., mega samples per second), the maximum reference picture size, and so on. In some cases, the limits set by the hierarchy may be further defined by a Hypothetical Reference Decoder (HRD) specification and metadata signaled HRD buffer management in the encoded video sequence.

In an embodiment, the receiver (710) may receive additional (redundant) data along with the encoded video. The additional data may be part of an encoded video sequence. The additional data may be used by the video decoder (610) to properly decode the data and/or more accurately reconstruct the original video data. The additional data may be in the form of, for example, a temporal, spatial, or signal-to-noise ratio (SNR) enhancement layer, redundant slices, redundant pictures, forward error correction codes, and so forth.

Fig. 8 is a block diagram of a video encoder (603) according to an embodiment of the disclosure.

The video encoder (603) may receive video samples from a video source (601) (not part of the encoder) that may capture video images to be encoded by the video encoder (603).

The video source (601) may provide a source video sequence in the form of a stream of digital video samples to be encoded by the video encoder (603), which may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit … …), any color space (e.g., bt.601y CrCB, RGB … …), and any suitable sampling structure (e.g., Y CrCB 4:2:0, Y CrCB 4:4: 4). In the media service system, the video source (601) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as at least two separate pictures that are given motion when viewed in sequence. The picture itself may be constructed as an array of spatial pixels, where each pixel may comprise at least one sample, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples can be readily understood by those skilled in the art. The following text focuses on describing the samples.

According to an embodiment, the video encoder (603) may encode and compress pictures of a source video sequence into an encoded video sequence (843) in real-time or under any other temporal constraint required by the application. It is a function of the controller (850) to implement the appropriate encoding speed. For simplicity, the couplings are not labeled in the figures. The parameters set by the controller may include rate control related parameters (picture skip, quantizer, lambda value of rate distortion optimization technique, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (850) may be used to have other suitable functions relating to the video encoder (603) optimized for a certain system design.

Some video encoders operate in what is known to those skilled in the art as a "coding loop". As a simple description, in an embodiment, the encoding loop may comprise an encoding part (hereinafter referred to as "source encoder") in the encoder (830), responsible for creating symbols based on the input picture to be encoded and the reference picture, and a (local) decoder (833) embedded in the encoder (603). The decoder (833) reconstructs the symbols to create sample data in a manner similar to how a (remote) decoder creates sample data (since any compression between symbols and encoded video streams is lossless in the video compression techniques considered in this application). The reconstructed sample stream (sample data) is input to a reference picture memory (834). Since the decoding of the symbol stream produces bit accurate results independent of decoder location (local or remote), the reference picture memory content also corresponds bit-accurately between the local encoder and the remote encoder. In other words, the reference picture samples that the prediction portion of the encoder "sees" are identical to the sample values that the decoder would "see" when using prediction during decoding. This reference picture synchronization philosophy (and the drift that occurs if synchronization cannot be maintained due to, for example, channel errors) is also used in some related techniques.

The operation of "local" decoder (833) may be the same as a "remote" decoder, such as video decoder (610) that has been described in detail above in connection with fig. 7. However, referring briefly also to fig. 6, when symbols are available and the entropy encoder (845) and parser (720) are able to losslessly encode/decode the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (610), including the buffer memory (715) and parser (720), may not be fully implemented in the local decoder (833).

At this point it can be observed that any decoder technique other than the parsing/entropy decoding present in the decoder must also be present in the corresponding encoder in substantially the same functional form. For this reason, the present application focuses on decoder operation. The description of the encoder techniques may be simplified because the encoder techniques are reciprocal to the fully described decoder techniques. A more detailed description is only needed in certain areas and is provided below.

During operation, in some embodiments, the source encoder (830) may perform motion compensated predictive coding. The motion compensated predictive coding predictively codes an input picture with reference to at least one previously coded picture from the video sequence that is designated as a "reference picture". In this way, the encoding engine (832) encodes differences between pixel blocks of an input picture and pixel blocks of a reference picture, which may be selected as a prediction reference for the input picture.

The local video decoder (833) may decode encoded video data for a picture that may be designated as a reference picture based on symbols created by the source encoder (830). The operation of the encoding engine (832) may be a lossy process. When the encoded video data can be decoded at a video decoder (not shown in fig. 6), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (833) replicates a decoding process, which may be performed on reference pictures by the video decoder, and may cause reconstructed reference pictures to be stored in a reference picture cache (834). In this way, the video encoder (603) may locally store a copy of the reconstructed reference picture that has common content (no transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.

Predictor (835) may perform a prediction search for coding engine (832). That is, for a new picture to be encoded, predictor (835) may search reference picture memory (834) for sample data (as candidate reference pixel blocks) or some metadata, such as reference picture motion vectors, block shapes, etc., that may be referenced as appropriate predictions for the new picture. The predictor (835) may operate on a block-by-block basis of samples to find a suitable prediction reference. In some cases, from search results obtained by predictor (835), it may be determined that the input picture may have prediction references taken from at least two reference pictures stored in reference picture memory (834).

The controller (850) may manage the encoding operations of the source encoder (830), including, for example, setting parameters and subgroup parameters for encoding the video data.

The outputs of all of the above functional units may be entropy encoded in an entropy encoder (845). The entropy encoder losslessly compresses the symbols generated by the various functional units according to techniques such as huffman coding, variable length coding, arithmetic coding, etc., to convert the symbols into an encoded video sequence.

The transmitter (840) may buffer the encoded video sequence created by the entropy encoder (845) in preparation for transmission over the communication channel (860), which may be a hardware/software link to a storage device that will store the encoded video data. The transmitter (840) may combine the encoded video data from the video encoder (830) with other data to be transmitted, such as encoded audio data and/or an auxiliary data stream (sources not shown).

The controller (850) may manage the operation of the video encoder (603). During encoding, the controller (850) may assign a certain encoded picture type to each encoded picture, but this may affect the encoding techniques applicable to the respective picture. For example, pictures may be generally assigned to any of the following picture types:

intra pictures (I pictures), which may be pictures that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs tolerate different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of variants of picture I and their corresponding applications and features.

Predictive pictures (P pictures), which may be pictures that may be encoded and decoded using intra prediction or inter prediction that uses at most one motion vector and reference index to predict sample values of each block.

Bi-predictive pictures (B-pictures), which may be pictures that can be encoded and decoded using intra-prediction or inter-prediction that uses at most two motion vectors and reference indices to predict sample values of each block. Similarly, at least two predictive pictures may use more than two reference pictures and associated metadata for reconstructing a single block.

A source picture may typically be spatially subdivided into at least two blocks of samples (e.g., blocks of 4 x 4, 8 x 8, 4 x 8, or 16 x 16 samples) and encoded block-wise. These blocks may be predictively encoded with reference to other (encoded) blocks that are determined according to the encoding allocation applied to their respective pictures. For example, a block of an I picture may be non-predictive encoded, or the block may be predictive encoded (spatial prediction or intra prediction) with reference to an already encoded block of the same picture. The pixel block of the P picture can be prediction-coded by spatial prediction or by temporal prediction with reference to one previously coded reference picture. A block of a B picture may be prediction coded by spatial prediction or by temporal prediction with reference to one or two previously coded reference pictures.

The video encoder (603) may perform encoding operations according to a predetermined video encoding technique or standard, such as the ITU-T h.265 recommendation. In operation, the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the encoded video data may conform to syntax specified by the video coding technique or standard used.

In an embodiment, the transmitter (840) may transmit the additional data while transmitting the encoded video. The source encoder (830) may treat such data as part of an encoded video sequence. The additional data may include temporal/spatial/SNR enhancement layers, redundant pictures and slices, among other forms of redundant data, SEI messages, VUI parameter set slices, etc.

As described above, at VTM3.0, the MPM list candidate derivation process may be different for adjacent reference rows and non-adjacent reference rows. As a result, the MPM list candidate derivation process may be complex in each case without significant benefit in terms of coding efficiency.

Further, in VTM3.0, when the transmitted reference row index is 0 and one of the left neighbor mode and the upper neighbor mode is equal to or smaller than the DC mode and the other is larger than the DC mode, the left neighbor mode is always inserted into the MPM list, followed by the upper neighbor mode even if the upper neighbor mode is the plane or the DC mode. This may not be the best solution because, according to statistics, the plane and DC modes are the most frequently used intra prediction modes.

The proposed methods can be used alone or in any order in combination. In an embodiment, the thread index of the nearest reference row may be 0, and the nearest reference row may be referred to as a zero reference row. The other lines may be referred to as non-zero reference rows. In the following description, candModeList may represent an MPM list, reflineindex may represent a reference line index of the current block, and candirapredmodea and candirapredmodeb may represent a left neighbor mode and an upper neighbor mode. If a neighbor mode is not a planar or DC mode, or a neighbor mode is generating prediction samples according to a given prediction direction, such as intra prediction modes 2 through 66 defined in VVC draft 2, that mode may be referred to as an angular intra prediction mode. If a mode is a planar mode or a DC mode, the mode is referred to as a non-angular intra prediction mode. Each intra-prediction mode may be associated with a mode number, which may be referred to as an intra-prediction mode index. For example, the plane, DC, horizontal, and vertical intra prediction modes may be associated with mode numbers 0, 1, 18, and 50, respectively.

In one embodiment, the variables minAB and maxAB may be derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

minAB＝candModeList[(candModeList[0]>candModeList[1])？1：0]

maxAB＝candModeList[(candModeList[0]>candModeList[1])？0：1]

in one embodiment, the variables offset and mod may be set according to either of two situations: offset 61, mod 64; offset is 62 and mod is 65.

In one embodiment, when sending non-zero reference row indices, the MPM list candidates may be derived using the same rule if the absolute mode number difference between the left neighboring mode and the upper neighboring mode is greater than or equal to a given threshold.

In one embodiment, the given threshold may be 0, which means that the MPM list candidates are derived using the same rule, regardless of the mode number difference between the left neighboring mode and the upper neighboring mode.

In another embodiment, both the left neighboring mode and the upper neighboring mode are angular intra prediction modes, and the given threshold may be 1, 2, or 3. In one example, MPM list candidates may be derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

candModeList[2]＝2+((minAB+offset)％mod)

candModeList[3]＝2+((minAB-1)％mod)

candModeList[4]＝2+((maxAB+offset)％mod)

candModeList[5]＝2+((maxAB-1)％mod)

in another embodiment, where the left neighboring mode and the upper neighboring mode are both angular intra prediction modes, and the mode number difference of the two modes is equal to 1 or 2, the MPM list candidates may be derived by using the same rule. In one example, MPM list candidates may be derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

candModeList[2]＝2+((minAB+offset)％mod)

candModeList[3]＝2+((maxAB-1)％mod)

candModeList[4]＝2+((minAB+offset-1)％mod)

candModeList[5]＝2+(maxAB％mod)

in another embodiment, if the left neighboring mode and the upper neighboring mode are not equal, the MPM list candidates may be derived by using the same rule regardless of a mode number difference between the left neighboring mode and the upper neighboring mode.

In one embodiment, if both the left neighboring mode and the upper neighboring mode are angular intra prediction modes but they are not equal, the MPM list candidates are derived by using the same rule regardless of a mode number difference between the left neighboring mode and the upper neighboring mode.

In one example, 6 MPM candidates may be derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

candModeList[2]＝2+((minAB+offset)％mod)

candModeList[3]＝2+((minAB-1)％mod)

candModeList[4]＝2+((maxAB+offset)％mod)

candModeList[5]＝2+((maxAB-1)％mod)

in another example, 6 MPM candidates may be derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

candModeList[2]＝2+((minAB+offset)％mod)

candModeList[3]＝2+((maxAB-1)％mod)

candModeList[4]＝2+((minAB-1)％mod)

candModeList[5]＝2+((maxAB+offset)％mod)

in another example, 6 MPM candidates may be derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

candModeList[2]＝2+((maxAB+offset)％mod)

candModeList[3]＝2+((maxAB-1)％mod)

candModeList[4]＝2+((minAB+offset)％mod)

candModeList[5]＝2+((minAB-1)％mod)

in another example, 6 MPM candidates may be derived as follows:

candModeList[0]＝candIntraPredModeA

candModeList[1]＝candIntraPredModeB

candModeList[2]＝2+((candIntraPredModeA+offset)％mod)

candModeList[3]＝2+((candIntraPredModeA-1)％mod)

candModeList[4]＝2+((candIntraPredModeB+offset)％mod)

candModeList[5]＝2+((candIntraPredModeB-1)％mod)

in another embodiment, if at least one of the left neighboring mode and the upper neighboring mode is an angular intra prediction mode, the MPM list candidate is derived by using the same rule regardless of a mode number difference between the left neighboring mode and the upper neighboring mode.

In an embodiment, when both neighboring modes are angular intra prediction modes, there may be copy mode candidates for non-zero reference rows in the MPM list.

In one embodiment, when both neighboring modes are angular intra prediction modes and the absolute value of the mode number difference of the two neighboring modes is equal to 1 or 2, or greater than a threshold, which may be a positive integer, e.g., one of 61, 62, 63, or 64, there are redundant mode candidates for non-zero reference rows in the MPM list.

In another embodiment, no copy mode candidates for the zero reference row in the MPM list are defined.

In another embodiment, when both neighboring modes are angular intra prediction modes and are not equal, there may be copy mode candidates for non-zero reference rows in the MPM list.

In one embodiment, the angular intra prediction modes used to construct the zero reference line MPM list may be used again for the non-zero line MPM list, and the indices of these angular MPM modes may be the same as the indices for the zero reference line and non-zero reference line MPM lists.

In one example, the MPM list for the zero line may be 26, 18, 0, 1, 25, 27, where 26, 18, 25, and 27 are angular intra prediction modes. Thus, the 4 angular intra prediction modes may also use the same index for non-zero lines, and one example of an MPM list for non-zero lines may be 26, 18, 17, 19, 25, 27.

In one embodiment, the default angular intra prediction modes used to construct the zero reference row MPM list are then used for the non-zero line MPM list, and the MPM indices for these angular MPM modes are the same as the MPM indices for the zero reference row and non-zero reference row MPM lists. The variable K is a positive integer. In one example, K is equal to 4.

In one example, the default modes for the zero and non-zero reference rows are { planar, DC, vertical, horizontal, vertical-K, vertical + K } and {2, 34, vertical, horizontal, vertical-K, vertical + K }, respectively. This can be described as follows:

candmodellist [0] ═ RefLineIdx? Plane: 2

candModeList[1]＝RefLineIdx＝＝0DC：34

candModelist [2] (vertical)

candModelist [3] ═ horizontal

candModelist [4] ═ vertical-K

candModelist [5] ═ vertical + K

In another example, the default mode for zero and non-zero reference lines may be formulated as follows, where candirapredmodea is equal to planar or DC mode:

candModeList[0]＝RefLineIdx＝＝0CandintrapredModeA：2

candModeList[1]＝RefLineIdx＝＝0？(1-candIntraPredModeA)：34

candModelist [2] (vertical)

candModelist [3] ═ horizontal

candModelist [4] ═ vertical-K

candModelist [5] ═ vertical + K

In one embodiment, if one neighbor mode is a plane or DC mode, or any non-angular intra prediction mode, and the other neighbor mode is an angular intra prediction mode, the plane or DC mode may always be inserted first into the MPM list with index 0 and the angular neighbor mode may always be inserted second into the MPM list with index 1.

In one embodiment, if a zero reference row index is transmitted, it is possible to always insert the planar mode first into the MPM list with index 0, the angle neighbor mode second into the MPM list with index 1, and the DC mode third into the MPM list with index 2. One example is as follows:

candModelist [0] ═ plane

candModeList[1]＝max(candIntraPredModeA，candIntraPredModeB)

candModeList[2]＝DC

candModeList[3]＝2+((mpm[1]+offset)％mod)

candModeList[4]＝2+((mpm[1]-1)％mod)

candModeList[5]＝2+((mpm[1]+offset-1)％mod)

In another embodiment, if a zero reference index is transmitted, a non-angular neighbor mode may always be inserted first into an MPM list with an index of 0, and an angular neighbor mode may always be inserted second into an MPM list with an index of 1, and an MPM candidate with an index of 2 may always be set to (1-candModeList [0 ]). One example is as follows:

candModeList[0]＝min(candIntraPredModeA，candIntraPredModeB)

candModeList[1]＝max(candIntraPredModeA，candIntraPredModeB)

candModeList[2]＝1-candModeList[0]

candModeList[3]＝2+((mpm[1]+offset)％mod)

candModeList[4]＝2+((mpm[1]-1)％mod)

candModeList[5]＝2+((mpm[1]+offset-1)％mod)

in another embodiment, if a zero reference row index is transmitted, the angular neighbor mode may always be inserted first into the MPM list with index 0 and the non-angular neighbor mode may always be inserted second into the MPM list with index 1, and the MPM candidate with index 2 may always be set to (1-candModeList [1 ]). One example is as follows:

candModeList[0]＝max(candIntraPredModeA，candIntraPredModeB)

candModeList[1]＝min(candIntraPredModeA，candIntraPredModeB)

candModeList[2]＝1-candModeList[1]

candModeList[3]＝2+((mpm[0]+offset)％mod)

candModeList[4]＝2+((mpm[0]-1)％mod)

candModeList[5]＝2+((mpm[0]+offset-1)％mod)

another example is as follows:

candModeList[0]＝max(candIntraPredModeA，candIntraPredModeB)

candModelist [1] ═ plane

candModeList[2]＝DC

candModeList[3]＝2+((mpm[0]+offset)％mod)

candModeList[4]＝2+((mpm[0]-1)％mod)

candModeList[5]＝2+((mpm[0]+offset-1)％mod)

In one embodiment, for the zero reference line, if one neighboring block is associated with a non-angular intra prediction mode, such as planar mode, DC mode, inter mode, intra-inter mode, or CPR mode, and the other neighboring mode is an angular intra prediction mode, whether the planar or DC mode is placed before the angular neighboring mode in the MPM list depends on the mode number of the angular neighboring mode.

In one embodiment, if the angle-neighboring mode is a vertical mode or a horizontal mode, the angle-neighboring mode is firstly inserted into the MPM list having an index of 0, and the plane or DC mode is secondly inserted into the MPM list having an index of 1. Otherwise, the plane or DC mode is inserted first into the MPM list with index 0 and the angle neighbor mode is inserted second into the MPM list with index 1. One example is as follows:

candModeList[0]＝(maxAB＝＝INTRA_ANGULAR50||maxAB＝＝INTRA_ANGULAR18)？maxAB：minAB

candModeList[1]＝(maxAB＝＝INTRA_ANGULAR50||maxAB＝＝INTRA_ANGULAR18)？minAB：maxAB

candModeList[2]＝1-minAB

candModeList[3]＝2+((maxAB+offset)％mod)

candModeList[4]＝2+((maxAB-1)％mod)

candModeList[5]＝2+((maxAB+offset-1)％mod)

in one embodiment, if the angle-neighboring mode is a vertical mode, or a horizontal mode, or a diagonal mode, e.g., mode 2, 34, 66, the angle-neighboring mode is inserted first into the MPM list with index 0, and the plane or DC mode is inserted second into the MPM list with index 1. Otherwise, the plane or DC mode is inserted first into the MPM list with index 0 and the angle neighbor mode is inserted second into the MPM list with index 1.

In one embodiment, if the angle-neighboring mode is a vertical mode or a horizontal mode, the angle-neighboring mode is firstly inserted into the MPM list having an index of 0, and the plane or DC mode is secondly inserted into the MPM list having an index of 1. Otherwise, the left neighbor mode of the block left-neighboring the current block is firstly inserted into the MPM list having an index of 0, and the upper neighbor mode of the block above the current block is secondly inserted into the MPM list having an index of 1. One example is as follows:

candModeList[0]＝(maxAB＝＝INTRA_ANGULAR50||maxAB＝＝INTRA_ANGULAR18)？maxAB：candIntraPredModeA

candModeList[1]＝(maxAB＝＝INTRA_ANGULAR50||maxAB＝＝INTRA_ANGULAR18)？minAB：candIntraPredModeB

candModeList[2]＝1-minAB

candModeList[3]＝2+((maxAB+offset)％mod)

candModeList[4]＝2+((maxAB-1)％mod)

candModeList[5]＝2+((maxAB+offset-1)％mod)

in one embodiment, if the angle-neighboring mode is a vertical mode, or a horizontal mode, or a diagonal mode, e.g., mode 2, 34, 66, the angle-neighboring mode is inserted first into the MPM list with index 0, and the plane or DC mode is inserted second into the MPM list with index 1. Otherwise, the left neighbor mode is inserted first into the MPM list with index 0, and the upper neighbor mode is inserted second into the MPM list with index 1.

In one embodiment, if the angular neighbor mode is a mode that generates prediction samples without interpolation at fractional positions, e.g., modes 2, 34, 66, horizontal, vertical, 72, 76, 78, 80, -6, 10, -12, -14, the angular neighbor mode is inserted first into the MPM list with index 0 and the planar or DC mode is inserted second into the MPM list with index 1. Otherwise, the plane or DC mode is inserted first into the MPM list with index 0, and the angle neighbor mode is inserted second into the MPM list with index 1.

In one embodiment, the determination of whether to place the planar mode or the DC mode before the angular neighbor mode in the MPM list also depends on the location of the neighboring angular intra prediction mode, e.g., whether it is from a left neighboring block or an upper neighboring block.

In one embodiment, if the left angle neighboring mode is a horizontal mode or the upper angle neighboring mode is a vertical mode, the angle neighboring mode is inserted first into the MPM list having an index of 0 and the plane or DC mode is inserted second into the MPM list having an index of 1. Otherwise, the plane or DC mode is inserted first into the MPM list with index 0, and the angle neighbor mode is inserted second into the MPM list with index 1.

In one embodiment, if the left angle neighbor mode is a horizontal-like mode and it does not generate prediction samples using interpolation at fractional positions, e.g., mode 2, horizontal, -6, -10, -12, -14, then the angle neighbor mode is inserted first into the MPM list with index 0 and the plane or DC mode is inserted second into the MPM list with index 1. Otherwise, the plane or DC mode is inserted first into the MPM list with index 0, and the angle neighbor mode is inserted second into the MPM list with index 1.

In one embodiment, if the upper angle neighbor mode is a vertical-like mode and it does not generate prediction samples using interpolation at fractional positions, e.g., mode 66, vertical, 72, 76, 78, 80, then the angle neighbor mode is inserted first into the MPM list with index 0 and the plane or DC mode is inserted second into the MPM list with index 1. Otherwise, the plane or DC mode is inserted first into the MPM list with index 0, and the angle neighbor mode is inserted second into the MPM list with index 1.

FIG. 9 is a flow diagram of an exemplary process 900 for transmitting an intra prediction mode for encoding a current block in an encoded video bitstream. In some embodiments, at least one of the process blocks of fig. 9 may be performed by the decoder 610. In some embodiments, at least one of the process blocks of fig. 9 may be performed by another device or group of devices separate from the decoder 610 or including the decoder 610, such as the encoder 603.

As shown in fig. 9, the process 900 may include generating a first MPM list corresponding to a zero reference line of the current block (block 910). The first MPM list may include a plurality of angular intra prediction modes.

As further shown in fig. 9, process 900 may include generating a second MPM list corresponding to at least one non-zero reference line of the current block (block 920). The second MPM list may include the same plurality of angular intra prediction modes.

As further shown in fig. 9, the process 900 may include transmitting a reference line index indicating one of at least one non-zero reference line and a zero reference line for encoding the current block (block 930).

As further shown in fig. 9, the process 900 may include determining whether the reference row index indicates that the reference row is a zero reference row (block 940). Based on the reference row index indicating that the reference row is a zero reference row, the process 900 then sends an intra-mode index indicating an intra-prediction mode within the first MPM list (block 950). When the reference row index indicates that the reference row is one of the at least one non-zero reference row, the process 900 then sends an intra-mode index indicating an intra-prediction mode within the second MPM list (block 950).

In one embodiment, the first MPM list further includes at least one non-angular intra prediction mode, and the second MPM list does not include at least one non-angular intra prediction mode.

In one embodiment, the at least one non-angular intra prediction mode includes at least one of a planar mode and a DC mode.

In one embodiment, the first MPM list includes a first plurality of indices corresponding to a plurality of angular intra-prediction modes, the second MPM list includes a second plurality of indices corresponding to a plurality of angular intra-prediction modes, and the first plurality of indices are the same as the second plurality of indices.

In one embodiment, the plurality of angular intra-prediction modes includes a first angular intra-prediction mode corresponding to a first index of the first plurality of indices and a second angular intra-prediction mode corresponding to a second index of the first plurality of indices, the first angular intra-prediction mode corresponding to a first index of the second plurality of indices, the second angular intra-prediction mode corresponding to a second index of the second plurality of indices, the first index of the first plurality of indices being the same as the first index of the second plurality of indices and the second index of the first plurality of indices being the same as the second index of the second plurality of indices.

In one embodiment, if a first neighboring mode of a first neighboring block of the current block is a non-angular intra prediction mode and a second neighboring mode of a second neighboring block of the current block is an angular intra prediction mode, the first intra prediction mode of the first MPM list may be the non-angular intra prediction mode and the second intra prediction mode of the first MPM list may be the angular intra prediction mode.

In one embodiment, if the first neighbor mode is a DC mode and the reference row index indicates that the reference row is a zero reference row, the first intra prediction mode of the first MPM list may be the DC mode.

In one embodiment, if the first neighboring mode is a planar mode and the reference row index indicates that the reference row is a zero reference row, the first intra prediction mode of the first MPM list may be the planar mode.

In one embodiment, if the reference row index indicates that the reference row is a zero reference row, the first intra prediction mode of the first MPM list may be a planar mode, the second intra prediction mode of the first MPM list may be an angular intra prediction mode, and the third intra prediction mode of the first MPM list may be a DC mode.

In one embodiment, if the reference line index indicates that the reference line is a zero reference line, the first neighboring mode of the first neighboring block of the current block is a non-angular intra prediction mode, and the second neighboring mode of the second neighboring block of the current block is an angular intra prediction mode, the first intra prediction mode of the first MPM list may be an angular intra prediction mode, and the second intra prediction mode of the first MPM list may be a non-angular intra prediction mode. Although fig. 9 shows exemplary blocks of the process 900, in some embodiments, the process 900 may include more blocks, fewer blocks, different blocks, or a different arrangement of blocks than shown in fig. 9. Additionally or alternatively, two or more blocks of process 900 may be performed in parallel.

Further, the above proposed method may be implemented by a processing circuit (e.g., at least one processor, or at least one integrated circuit). In one embodiment, at least one processor executes a program stored in a non-transitory computer readable medium to perform at least one of the methods set forth above.

The techniques described above may be implemented as computer software via computer readable instructions and physically stored in at least one computer readable medium. For example, fig. 10 illustrates a computer system (1200) suitable for implementing certain embodiments of the disclosed subject matter.

The computer software may be encoded in any suitable machine code or computer language, and may be encoded by any suitable mechanism for assembling, compiling, linking, etc., code comprising instructions that are directly executable by at least one computer Central Processing Unit (CPU), Graphics Processing Unit (GPU), etc., or executable by code translation, microcode, etc.

The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, internet of things devices, and so forth.

The components illustrated in FIG. 10 for the computer system (1200) are exemplary in nature and are not intended to limit the scope of use or functionality of the computer software implementing embodiments of the present application in any way. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiments of the computer system (1200).

The computer system (1200) may include some human interface input devices. Such human interface input devices may respond to input from at least one human user by tactile input (e.g., keyboard input, swipe, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gesture), olfactory input (not shown). The human-machine interface device may also be used to capture media that does not necessarily directly relate to human conscious input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still-image cameras), video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

The human interface input device may include at least one of the following (only one of which is depicted): keyboard 1001, mouse 1002, touch pad 1003, touch screen 1004, data glove 1204, joystick 1005, microphone (1006), scanner 1007, camera 1008.

The computer system (1200) may also include certain human interface output devices. Such a human interface output device may stimulate the perception of at least one human user by, for example, tactile output, sound, light, and smell/taste. Such human interface output devices may include tactile output devices (e.g., tactile feedback through touch screen 1010, data glove 1204, or joystick 1005, but there may also be tactile feedback devices not used as input devices), audio output devices (e.g., speaker 1009, headphones (not shown)), visual output devices (e.g., screen 1010 including Cathode Ray Tube (CRT) screen, Liquid Crystal Display (LCD) screen, plasma screen, Organic Light Emitting Diode (OLED) screen), each with or without touch screen input functionality, each with or without tactile feedback functionality — some of which may output two-dimensional visual output or output more than three-dimensional through means such as stereoscopic visual output; virtual reality glasses (not shown), a holographic display and a smoke box (not shown)), and a printer (not shown).

The computer system (1200) may also include human-accessible storage devices and their associated media such as optical media including compact disc read-only/rewritable (CD/DVD ROM/RW) or similar media (1021) with CD/DVD, thumb drives (1022), removable hard drives or solid state drives (1023), conventional magnetic media such as magnetic tape and floppy disks (not shown), ROM/ASIC/PLD based application specific devices such as a security dongle (not shown), and the like.

Those skilled in the art will also appreciate that the term "computer-readable medium" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

The computer system (1200) may also include an interface to at least one communication network. For example, the network may be wireless, wired, optical. The network may also be a local area network, a wide area network, a metropolitan area network, a vehicular network, an industrial network, a real-time network, a delay tolerant network, and so forth. The network also includes ethernet, wireless local area networks, local area networks such as cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), television wired or wireless wide area digital networks (including cable, satellite, and terrestrial broadcast television), automotive and industrial networks (including CANBus), and so forth. Some networks typically require external network interface adapters for connecting to some general purpose data port or peripheral bus 1049 (e.g., a USB port of computer system (1200)); other systems are typically integrated into the core of the computer system (1200) by connecting to a system bus as described below (e.g., an ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1200) may communicate with other entities. The communication may be unidirectional, for reception only (e.g., wireless television), unidirectional for transmission only (e.g., CAN bus to certain CAN bus devices), or bidirectional, for example, to other computer systems over a local or wide area digital network. Each of the networks and network interfaces described above may use certain protocols and protocol stacks.

The human interface device, human accessible storage device, and network interface described above may be connected to the core (1040) of the computer system (1200).

The core (1040) may include at least one Central Processing Unit (CPU)1041, a Graphics Processing Unit (GPU)1042, a special programmable processing unit in the form of a Field Programmable Gate Array (FPGA)1043, a hardware accelerator 1044 for specific tasks, and the like. These devices, as well as Read Only Memory (ROM)1045, random access memory 1046, internal mass storage 1047 (e.g., internal non-user accessible hard drives, solid state drives, etc.), and the like, may be connected by a system bus 1248. In some computer systems, system bus 1248 may be accessed as at least one physical plug so as to be extendable by additional central processing units, graphics processing units, and the like. Peripheral devices may be attached directly to the system bus 1248 of the core or connected through a peripheral bus 1049. The architecture of the peripheral bus includes peripheral controller interface PCI, universal serial bus USB, etc.

The CPU 1041, GPU 1042, FPGA 1043 and accelerator 1044 can execute certain instructions that, in combination, can constitute the computer code described above. The computer code may be stored in the ROM 1045 or the RAM 1046. Transitional data may also be stored in RAM1046, while persistent data may be stored in, for example, internal mass storage 1047. Fast storage and retrieval for any memory device can be achieved through the use of cache memories, which can be closely associated with at least one CPU 1041, GPU 1042, mass storage 1047, ROM 1045, RAM1046, and the like.

The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present application, or they may be of the kind well known and available to those having skill in the computer software arts.

By way of example, and not limitation, a computer system having architecture (1200), and in particular core 1040, may provide functionality as a processor (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in at least one tangible computer-readable medium. Such computer-readable media may be media associated with the user-accessible mass storage described above, as well as certain storage of the non-volatile core 1040, such as core internal mass storage 1047 or ROM 1045. Software implementing various embodiments of the present application may be stored in such devices and executed by core 1040. The computer-readable medium may include one or more memory devices or chips, according to particular needs. The software may cause the core 1040, and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.), to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in the RAM1046 and modifying such data structures in accordance with the software defined processes. Additionally or alternatively, the computer system may provide functionality that is logically hardwired or otherwise embodied in circuitry (e.g., accelerator 1044) that may operate in place of or in conjunction with software to perform certain processes or certain portions of certain processes described herein. Where appropriate, reference to software may include logic and vice versa. Where appropriate, reference to a computer-readable medium may include circuitry (e.g., an Integrated Circuit (IC)) storing executable software, circuitry comprising executable logic, or both. The present application includes any suitable combination of hardware and software.

While at least two exemplary embodiments have been described in this application, various modifications, permutations and various substitute equivalents thereof are within the scope of this application. It will thus be appreciated that those skilled in the art will be able to devise various systems and methods which, although not explicitly shown or described herein, embody the principles of the application and are thus within its spirit and scope.