Method and apparatus for CABAC-based entropy coding
阅读说明：本技术 用于基于cabac的熵编码的方法和设备 (Method and apparatus for CABAC-based entropy coding ) 是由 南廷学 于 2019-06-14 设计创作，主要内容包括：根据本公开的实施方式的一种由解码设备执行的图片解码方法包括以下步骤：基于当前块的分割可用性信息来推导分割标志语法元素的上下文索引；基于推导出的所述上下文索引来确定上下文模型；使用所确定的所述上下文模型基于CABAC对所述分割标志语法元素的值进行解码；基于所述分割标志语法元素的值来从所述当前块中推导当前编码单元；基于针对所述当前编码单元的帧间预测或帧内预测来推导预测块；以及基于所述预测块来生成重构块。(A picture decoding method performed by a decoding apparatus according to an embodiment of the present disclosure includes the steps of: deriving a context index of a partition flag syntax element based on partition availability information of the current block; determining a context model based on the derived context index; decoding a value of the partition flag syntax element based on CABAC using the determined context model; deriving a current coding unit from the current block based on a value of the partition flag syntax element; deriving a prediction block based on inter prediction or intra prediction for the current coding unit; and generating a reconstructed block based on the prediction block.)
1. A picture decoding method performed by a decoding apparatus, the method comprising the steps of:
deriving a context index of a partition flag syntax element based on partition availability information of the current block;
determining a context model based on the derived context index;
decoding a value of the partition flag syntax element on a context-based adaptive binary arithmetic coding (CABAC) basis by using the determined context model;
deriving a current coding unit from the current block based on a value of the partition flag syntax element;
deriving a prediction block based on inter prediction or intra prediction for the current coding unit; and
generating a reconstructed block based on the prediction block,
wherein the partition flag syntax element comprises a directional partition flag indicating whether the current block is to be vertically partitioned,
wherein the division availability information includes BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a binary tree BT structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a ternary tree TT structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure, and
wherein a context index of the directional division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
2. The picture decoding method according to claim 1, wherein whether to derive a context index of the directional division flag is determined based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
3. The picture decoding method according to claim 2, wherein if the value of the BT horizontal division available flag information and the value of the TT horizontal division available flag information are 0 or the value of the BT vertical division available flag information and the value of the TT vertical division available flag information are 0, a context index of the directional division flag is not derived.
4. The picture decoding method according to claim 3, wherein a context index of the directional segmentation flag is not derived when a value of (availableBTHOR | | availableTTHOR) & & (availableBlister | | availableTTVER) is 0, wherein availableBTHOR represents BT horizontal segmentation available flag information, availableTTHOR represents the TT horizontal segmentation available flag information, availableBTVER represents the BT vertical segmentation available flag information, and availableTTVER represents the TT vertical segmentation available flag information.
5. The picture decoding method of claim 1, wherein the partition syntax element further comprises a BT partition flag indicating whether the current block is partitioned based on the BT structure, and
wherein a context index of the BT division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
6. The picture decoding method according to claim 1, wherein a context index of the directional partition flag is derived based on,
CtxIdx＝(availableBTHOR&&availableTTHOR)*2+(availableBTVER&&availableTTVER),
in this formula, availableBTHOR represents the BT horizontal split available flag information, availableTTHOR represents the TT horizontal split available flag information, availableBTVER represents the BT vertical split available flag information, and availableTTVER represents the TT vertical split available flag information.
7. The picture decoding method according to claim 5, wherein a context index of the BT segmentation flag is derived based on the following formula,
CtxIdx＝(availableBTHOR&&availableBTVER)*2+(availableTTHOR&&availableTTVER),
in the formula, availableBTHOR represents the BT horizontal split available flag information, availableTTHOR represents the TT horizontal split available flag information, availableBTVER represents the BT vertical split available flag information, and availableTTVER represents the TT vertical split available flag information.
8. A picture encoding method performed by an encoding apparatus, the method comprising the steps of:
deriving a partition flag syntax element representing a partition structure of the current block;
deriving a context index of the partition flag syntax element based on partition availability information for the current block;
determining a context model based on the derived context index; and
encoding a value of the segmentation marker syntax element based on CABAC using the determined context model,
wherein the partition flag syntax element comprises a directional partition flag indicating whether the current block is vertically partitioned,
wherein the segmentation availability information comprises: BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a BT structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a TT structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure, and
wherein a context index of the directional division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
9. The picture encoding method according to claim 8, wherein whether to derive a context index of the directional division flag is determined based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
10. The picture encoding method of claim 9, wherein if a value of the BT horizontal division available flag information and a value of the TT horizontal division available flag information are 0 or a value of the BT vertical division available flag information and a value of the TT vertical division available flag information are 0, a context index of the directional division flag is not derived.
11. The picture coding method according to claim 10, wherein when a value of (availableBTHOR | | availableTTHOR) & & (availableBTVER | | availableTTVER) is 0, a context index of the directional segmentation flag is not derived, wherein availableBTHOR denotes the BT horizontal segmentation availability flag information, availableTTHOR denotes the TT horizontal segmentation availability flag information, availableBTVER denotes the BT vertical segmentation availability flag information, and availableTTVER denotes the TT vertical segmentation availability flag information.
12. The picture encoding method of claim 8, wherein the partition syntax element further comprises a BT partition flag indicating whether the current block is partitioned based on the BT structure, and
wherein a context index of the BT division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
13. The picture encoding method according to claim 8, wherein a context index of the directional partition flag is derived based on the following formula,
CtxIdx＝(availableBTHOR&&availableTTHOR)*2+(availableBTVER&&availableTTVER),
in the formula, availableBTHOR represents the BT horizontal split available flag information, availableTTHOR represents the TT horizontal split available flag information, availableBTVER represents the BT vertical split available flag information, and availableTTVER represents the TT vertical split available flag information.
14. The picture encoding method of claim 12, wherein a context index of the BT segmentation flag is derived based on the following formula,
CtxIdx＝(availableBTHOR&&availableBTVER)*2+(availableTTHOR&&availableTTVER),
in this formula, availableBTHOR represents the BT horizontal split available flag information, availableTTHOR represents the TT horizontal split available flag information, availableBTVER represents the BT vertical split available flag information, and availableTTVER represents the TT vertical split available flag information.
15. A decoding apparatus that performs picture decoding includes:
an entropy decoder configured to derive a context index of a partition flag syntax element based on partition availability information of a current block, determine a context model based on the derived context index, decode a value of the partition flag syntax element based on CABAC using the determined context model, and derive a current coding unit from the current block based on the value of the partition flag syntax element;
a predictor configured to derive a prediction block based on inter prediction or intra prediction for the current coding unit; and
an adder configured to generate a reconstructed block based on the prediction block,
wherein the split flag syntax element comprises: a directional partition flag indicating whether the current block is vertically partitioned,
wherein the segmentation availability information comprises: BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a BT structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a TT structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure, and
wherein a context index of the directional division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
Technical Field
The present disclosure relates to a still image or video image encoding/decoding method, and more particularly, to a method and apparatus for partitioning a region into a Quadtree (QT) structure and using a context model when CABAC entropy encoding is performed based on a Binary Tree (BT) and a Ternary Tree (TT) structure.
Background
Demands for high-resolution and high-quality images such as High Definition (HD) images and Ultra High Definition (UHD) images are increasing in various fields. Since the image data has high resolution and high quality, the amount of information or bits to be transmitted increases relative to conventional image data. Therefore, when image data is transmitted using a medium such as a conventional wired/wireless broadband line or stored using an existing storage medium, transmission costs and storage costs thereof increase.
Accordingly, there is a need for an efficient image compression technique for efficiently transmitting, storing, and reproducing information of high-resolution and high-quality images.
Disclosure of Invention
Technical problem
An object of the present disclosure is to provide a method and apparatus for enhancing image coding efficiency.
Another object of the present disclosure is to provide a method and apparatus for enhancing image efficiency based on CABAC.
It is still another object of the present disclosure to provide a method and apparatus for determining a block structure and block segmentation based on a Quadtree (QT) structure, a Binary Tree (BT) structure, and a Ternary Tree (TT) structure in a process of compressing an image.
It is still another object of the present disclosure to provide a method and apparatus for encoding and decoding an image more efficiently using a context model when entropy encoding is performed according to CABAC based on a QT structure, a BT structure, and a TT structure.
Technical scheme
Exemplary embodiments of the present disclosure provide a picture decoding method performed by a decoding apparatus. The method comprises the following steps: deriving a context index of a partition flag syntax element based on partition availability information of the current block; determining a context model based on the derived context index; decoding a value of the partition flag syntax element on a context-based adaptive binary arithmetic coding (CABAC) basis by using the determined context model; deriving a current coding unit from the current block based on a value of the partition flag syntax element; deriving a prediction block based on inter prediction or intra prediction for the current coding unit; and generating a reconstructed block based on the prediction block, wherein the partition flag syntax element comprises: a directional partition flag indicating whether the current block is to be vertically partitioned, the partition availability information including: BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a binary tree BT structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a ternary tree TT structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure, and a context index of the directional division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
Another exemplary embodiment of the present disclosure provides a decoding apparatus that performs picture decoding. The decoding apparatus includes: an entropy decoder configured to derive a context index of a partition flag syntax element based on partition availability information of a current block, determine a context model based on the derived context index, decode a value of the partition flag syntax element based on CABAC using the determined context model, and derive a current coding unit from the current block based on the value of the partition flag syntax element; a predictor configured to derive a prediction block based on inter prediction or intra prediction for the current coding unit; and an adder configured to generate a reconstructed block based on the prediction block, wherein the partition flag syntax element comprises: a directional partition flag indicating whether the current block is vertically partitioned, the partition availability information including: BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a BT structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a TT structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure, and a context index of the directional division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
Still another exemplary embodiment of the present disclosure provides a picture encoding method performed by an encoding apparatus. The method comprises the following steps: deriving a partition flag syntax element representing a partition structure of the current block; deriving a context index of the partition flag syntax element based on partition availability information for the current block; determining a context model based on the derived context index; and encoding a value of the partition flag syntax element based on CABAC by using the determined context model, wherein the partition flag syntax element includes: a directional partition flag indicating whether the current block is to be vertically partitioned, the partition availability information including: BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a BT structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a TT structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure, and a context index of the directional division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
Still another exemplary embodiment of the present disclosure provides an encoding apparatus that performs picture encoding. The encoding device includes: an entropy encoder configured to derive a partition flag syntax element representing a partition structure of a current block, derive a context index of the partition flag syntax element based on partition availability information of the current block, determine a context model based on the derived context index, and encode a value of the partition flag syntax element based on CABAC by using the determined context model, wherein the partition flag syntax element includes: a directional partition flag indicating whether the current block is to be vertically partitioned, the partition availability information including: BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a BT structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a TT structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure, and a context index of the directional division flag is derived based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
Technical effects
The present disclosure can enhance overall image/video compression efficiency.
The present disclosure can enhance the efficiency of image coding based on intra prediction.
The present disclosure can improve efficiency of CABAC-based image coding.
The present disclosure can improve pipeline latency when CCLM based intra prediction is implemented in hardware.
The present disclosure may determine a block structure and block segmentation based on a Quadtree (QT) structure, a Binary Tree (BT) structure, and a Ternary Tree (TT) structure in a process of compressing an image.
The present disclosure may more efficiently encode and decode an image using a context model when performing entropy encoding according to CABAC based on a QT structure, a BT structure, and a TT structure.
Drawings
Fig. 1 is a diagram for schematically illustrating a configuration of an encoding apparatus according to an exemplary embodiment.
Fig. 2 is a diagram for schematically illustrating a configuration of a decoding apparatus according to an exemplary embodiment.
Fig. 3a to 3c are diagrams illustrating examples of QT segmentation, BT segmentation, and TT segmentation.
Fig. 4 is a block diagram illustrating a configuration of an entropy encoder according to an exemplary embodiment.
Fig. 5 is a block diagram illustrating a configuration of an entropy decoder according to an exemplary embodiment.
FIG. 6 is a flowchart illustrating operation of an entropy encoder according to an exemplary embodiment.
Fig. 7 is a flowchart illustrating an operation of an entropy decoder according to an exemplary embodiment.
Fig. 8 is a flowchart illustrating an operation of an encoding apparatus according to an exemplary embodiment.
Fig. 9 is a block diagram illustrating a configuration of an encoding apparatus according to an exemplary embodiment.
Fig. 10 is a flowchart illustrating an operation of a decoding apparatus according to an exemplary embodiment.
Fig. 11 is a block diagram illustrating a configuration of a decoding apparatus according to an exemplary embodiment.
Fig. 12 is a diagram of a content streaming system structure according to an exemplary embodiment.
Detailed Description
Exemplary embodiments of the present disclosure provide a picture decoding method performed by a decoding apparatus. The method comprises the following steps: deriving a context index of a partition flag syntax element based on partition availability information of the current block; determining a context model based on the derived context index; decoding a value of a partition flag syntax element based on context-based adaptive binary arithmetic coding (CABAC) using the determined context model; deriving a current coding unit from the current block based on a value of a partition flag syntax element; deriving a prediction block based on inter prediction or intra prediction for a current coding unit; and generating a reconstructed block based on the prediction block, wherein the partition flag syntax element includes a directional partition flag indicating whether the current block is vertically partitioned; the split availability information includes: BT horizontal division available flag information indicating whether a current block can be horizontally divided based on a Binary Tree (BT) structure, TT horizontal division available flag information indicating whether a current block can be horizontally divided based on a Ternary Tree (TT) structure, BT vertical division available flag information indicating whether a current block can be vertically divided based on a BT structure, and TT vertical division available flag information indicating whether a current block can be vertically divided based on a TT structure; and deriving a context index of the directional division flag based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information.
Since the present disclosure may be modified and practiced in various ways, specific exemplary embodiments will be illustrated in the drawings and described in detail. However, these embodiments are not intended to limit the present disclosure. The terminology used in the following description is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. Singular references include plural references as long as it is clearly read differently. Terms such as "including" and "having" are intended to indicate the presence of features, numbers, steps, operations, elements, components or combinations thereof used in the following description, and therefore it should be understood that the possibility of one or more different features, numbers, steps, operations, elements, components or combinations thereof being present or added is not excluded.
Furthermore, the elements in the figures described in this disclosure are drawn separately for the purpose of convenience to illustrate different specific functions, which does not mean that these elements are implemented by separate hardware or separate software. For example, two or more of these elements may be combined to form a single element, or one element may be divided into a plurality of elements. Embodiments in which elements are combined and/or divided are within the present disclosure without departing from the concepts thereof.
The following description relates to video/image coding. For example, the method/exemplary embodiment disclosed in this document may be applied to a method disclosed in a general Video coding (VVC) standard, a basic Video coding (EVC) standard, an AOMedia Video1(AV1) standard, a second generation audio Video coding (AVs2) standard, or a next generation Video/image coding standard (e.g., h.267, h.268, etc.).
Hereinafter, examples of the present embodiment will be described in detail with reference to the accompanying drawings. In addition, like reference numerals are used to designate like elements throughout the drawings, and the same description of the like elements will be omitted.
In the present disclosure, a video may mean a set of a series of images according to the passage of time. Generally, a picture means a unit of an image representing a specific time, and a slice is a unit constituting a part of the picture. One picture may be composed of a plurality of slices, and the terms picture and slice may be mixed with each other as occasion demands.
A pixel or pixel (pel) may mean the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. The samples may generally represent pixels or values of pixels, may represent only pixels of a luminance component (pixel values), and may represent only pixels of a chrominance component (pixel values).
The unit indicates a basic unit of image processing. The unit may include at least one of a specific region and information related to the region. Alternatively, the units may be mixed with terms such as blocks, regions, and the like. Typically, an mxn block may represent a set of samples or transform coefficients arranged in M columns and N rows.
Fig. 1 is a diagram briefly illustrating the structure of an encoding apparatus to which the present disclosure is applied. Hereinafter, the encoding/decoding apparatus may include a video encoding/decoding apparatus and/or an image encoding/decoding apparatus, and the video encoding/decoding apparatus may be used as a concept including the image encoding/decoding apparatus, or the image encoding/decoding apparatus may be used as a concept including the video encoding/decoding apparatus.
Referring to fig. 1, the video encoding apparatus 100 may include a picture divider 105, a predictor 110, a residual processor 120, an entropy encoder 130, an adder 140, a filter 150, and a memory 160. The residual processor 120 may include a subtractor 121, a transformer 122, a quantizer 123, a re-arranger 124, an inverse quantizer 125, and an inverse transformer 126.
The picture divider 105 may divide an input picture into at least one processing unit.
In one example, a processing unit may be referred to as a Coding Unit (CU). In this case, the coding units may be recursively separated from the Largest Coding Unit (LCU) according to a binary quadtree tree (QTBT) structure. For example, one coding unit may be separated into coding units of deeper depths based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and a ternary tree structure may be applied later. Alternatively, a binary/ternary tree structure may be applied first. The encoding process according to the present embodiment may be performed based on the final encoding unit which is not further separated. In this case, the maximum coding unit may be used as the final coding unit based on coding efficiency or the like according to image characteristics, or the coding unit may be recursively separated into coding units of lower depths as necessary and a coding unit having an optimal size may be used as the final coding unit. Here, the encoding process may include processes such as prediction, transformation, and reconstruction, which will be described later.
In another example, a processing unit may include a Coding Unit (CU), a Prediction Unit (PU), or a Transformer (TU). The coding units may be separated from a maximum coding unit (LCU) into coding units of deeper depths according to a quadtree structure. In this case, the maximum coding unit may be directly used as the final coding unit based on coding efficiency or the like according to image characteristics, or the coding unit may be recursively separated into coding units of deeper depths as needed, and a coding unit having an optimal size may be used as the final coding unit. When a minimum coding unit (SCU) is set, the coding unit may not be separated into coding units smaller than the minimum coding unit. Here, the final coding unit refers to a coding unit divided or separated into prediction units or transformers. The prediction unit is a unit divided from the coding unit, and may be a unit of sample prediction. Here, the prediction unit may be divided into subblocks. The transformer may be divided from the encoding unit according to a quadtree structure, and may be a unit that derives transform coefficients and/or a unit that derives residual signals from the transform coefficients. Hereinafter, the coding unit may be referred to as a Coding Block (CB), the prediction unit may be referred to as a Prediction Block (PB), and the transformer may be referred to as a Transform Block (TB). A prediction block or prediction unit may refer to a specific region in the form of a block in a picture and includes an array of prediction samples. In addition, a transform block or transformer may refer to a specific region in the form of a block in a picture and include an array of transform coefficients or residual samples.
The predictor 110 may perform prediction on a processing target block (hereinafter, it may mean a current block or a residual block), and may generate a prediction block including prediction samples for the current block. The unit of prediction performed in the predictor 110 may be an encoding block, or may be a transform block, or may be a prediction block.
The predictor 110 may determine whether intra prediction or inter prediction is applied to the current block. For example, the predictor 110 may determine whether to apply intra prediction or inter prediction in units of CUs.
In the case of intra prediction, the predictor 110 may derive prediction samples of the current block based on reference samples other than the current block in a picture (hereinafter, current picture) to which the current block belongs. In this case, the predictor 110 may derive the prediction sample based on an average or interpolation of neighboring reference samples of the current block (case (i)), or may derive the prediction sample based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block (case (ii)). Case (i) may be referred to as a non-directional mode or a non-angular mode, and case (ii) may be referred to as a directional mode or an angular mode. In intra prediction, the prediction modes may include 33 directional modes and at least two non-directional modes as an example. The non-directional mode may include a DC mode and a planar mode. The predictor 110 may determine a prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.
In the case of inter prediction, the predictor 110 may derive a prediction sample for the current block based on a sample specified by a motion vector on a reference picture. The predictor 110 may derive a prediction sample for the current block by applying any one of a skip mode, a merge mode, and a Motion Vector Prediction (MVP) mode. In case of the skip mode and the merge mode, the predictor 110 may use motion information of neighboring blocks as motion information of the current block. In case of the skip mode, unlike the merge mode, a difference (residual) between the prediction sample and the original sample is not transmitted. In case of the MVP mode, motion vectors of neighboring blocks are used as a motion vector predictor (predictor) to derive a motion vector of the current block.
In the case of inter prediction, the neighboring blocks may include spatially neighboring blocks existing in the current picture and temporally neighboring blocks existing in the reference picture. A reference picture including temporally adjacent blocks may also be referred to as a collocated picture (colPic). The motion information may include a motion vector and a reference picture index. Information such as prediction mode information and motion information may be (entropy) encoded and then output as a bitstream.
When motion information of temporally adjacent blocks is used in the skip mode and the merge mode, the highest picture in the reference picture list may be used as a reference picture. Reference pictures included in the reference picture list may be aligned based on a Picture Order Count (POC) difference between the current picture and the corresponding reference picture. POC corresponds to display order and can be distinguished from coding order.
The subtractor 121 generates a residual sample, which is the difference between the original sample and the predicted sample. If skip mode is applied, no residual samples may be generated as described above.
The transformer 122 transforms the residual samples in units of transform blocks to generate transform coefficients. The transformer 122 may perform transformation based on the size of a corresponding transform block and a prediction mode applied to a prediction block or an encoding block spatially overlapping with the transform block. For example, if intra prediction is applied to a prediction block or a coding block overlapping with a transform block and the transform block is a 4 × 4 residual array, residual samples may be transformed using a Discrete Sine Transform (DST) transform kernel and, in other cases, the residual samples are transformed using a Discrete Cosine Transform (DCT) transform kernel.
The quantizer 123 may quantize the transform coefficients to generate quantized transform coefficients.
The reorderer 124 reorders the quantized transform coefficients. The reorderer 124 may rearrange the quantized transform coefficients in the form of blocks into one-dimensional vectors by a coefficient scanning method. Although reorderer 124 is depicted as a separate component, reorderer 124 may be part of quantizer 123.
The entropy encoder 130 may perform entropy encoding on the quantized transform coefficients. Entropy encoding may include encoding methods such as exponential golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), and the like. In addition to the quantized transform coefficients, the entropy encoder 130 may perform encoding on information (e.g., syntax element values, etc.) required for video reconstruction together or separately according to entropy encoding or according to a pre-configured method. Entropy coding information may be transmitted or stored in units of Network Abstraction Layers (NALs) in the form of a bitstream. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, blu-ray, HDD, SDD, and the like.
The inverse quantizer 125 inverse-quantizes the values (transform coefficients) quantized by the quantizer 123, and the inverse transformer 126 inverse-transforms the values inverse-quantized by the inverse quantizer 125 to generate residual samples.
Adder 140 adds the residual samples to the prediction samples to reconstruct the picture. The residual samples may be added to the prediction samples in units of blocks to generate reconstructed blocks. Although the adder 140 is described as a separate component, the adder 140 may be part of the predictor 110. In addition, the adder 140 may be referred to as a reconstructor or a reconstruction block generator.
Filter 150 may apply deblocking filtering and/or sample adaptive offset to the reconstructed picture. Artifacts at block boundaries in the reconstructed picture or distortion upon quantization can be corrected by deblocking filtering and/or sample adaptive shifting. After deblocking filtering is completed, sample adaptive offset may be applied in sample units. Filter 150 may apply an Adaptive Loop Filter (ALF) to the reconstructed picture. ALF may be applied to reconstructed pictures to which deblocking filtering and/or sample adaptive offset has been applied.
The memory 160 may store information required for reconstructing a picture (decoded picture) or encoding/decoding. Here, the reconstructed picture may be a reconstructed picture filtered by the filter 150. The stored reconstructed pictures may be used as reference pictures for (inter) prediction of other pictures. For example, the memory 160 may store (reference) pictures used for inter prediction. Here, a picture for inter prediction may be specified according to a reference picture set or a reference picture list.
Fig. 2 is a diagram briefly illustrating the structure of a video/image decoding apparatus to which the present disclosure is applied. Hereinafter, the video decoding apparatus may include an image decoding apparatus.
Referring to fig. 2, the video decoding apparatus 200 may include an entropy decoder 210, a residual processor 220, a predictor 230, an adder 240, a filter 250, and a memory 260. The residual processor 220 may include a reorderer 221, an inverse quantizer 222, and an inverse transformer 223. In addition, although not depicted, the video decoding apparatus 200 may include a receiver for receiving a bitstream including video information. The receiver may be configured as a separate module or may be included in the entropy decoder 210.
When a bitstream including video/image information is input, the video decoding apparatus 200 may reconstruct video/images/pictures in association with a process of processing the video information in the video encoding apparatus.
For example, the video decoding apparatus 200 may perform video decoding using a processing unit applied in the video encoding apparatus. Thus, the processing unit block of video decoding may be, for example, a coding unit, and in another example, a coding unit, a prediction unit, or a transformer. The coding units may be separated from the maximum coding unit according to a quadtree structure and/or a binary tree structure and/or a ternary tree structure.
A prediction unit and a transformer may be further used in some cases, and in this case, the prediction block is a block derived or divided from a coding unit, and may be a unit of sample prediction. Here, the prediction unit may be divided into subblocks. The transformer may be separated from the encoding unit according to a quadtree structure, and may be a unit that derives a transform coefficient or a unit that derives a residual signal from the transform coefficient.
The entropy decoder 210 may parse the bitstream to output information required for video reconstruction or picture reconstruction. For example, the entropy decoder 210 may decode information in a bitstream based on an encoding method such as exponential golomb encoding, CAVLC, CABAC, or the like, and may output a value of a syntax element required for video reconstruction and a quantized value of a transform coefficient with respect to a residual.
More specifically, the CABAC entropy decoding method may receive a bin corresponding to each syntax element in a bitstream, determine a context model using decoding target syntax element information and decoding information of a neighboring block and the decoding target block or information of a symbol/bin decoded in a previous step, predict bin generation probabilities according to the determined context model, and perform arithmetic decoding of the bins to generate symbols corresponding to each syntax element value. Here, the CABAC entropy decoding method may update the context model using information of the symbol/bin decoded for the context model of the next symbol/bin after determining the context model.
Information regarding prediction among information decoded in the entropy decoder 210 may be provided to the predictor 250, and residual values (i.e., quantized transform coefficients) on which entropy decoding has been performed by the entropy decoder 210 may be input to the re-arranger 221.
The reorderer 221 may rearrange the quantized transform coefficients into a two-dimensional block form. The reorderer 221 may perform rearrangement corresponding to coefficient scanning performed by the encoding apparatus. Although the reorderer 221 is described as a separate component, the reorderer 221 may be part of the inverse quantizer 222.
The inverse quantizer 222 may inverse quantize the quantized transform coefficients based on the (inverse) quantization parameter to output the transform coefficients. In this case, information for deriving the quantization parameter may be signaled from the encoding apparatus.
Inverse transformer 223 may inverse transform the transform coefficients to derive residual samples.
The predictor 230 may perform prediction on the current block and may generate a prediction block including prediction samples for the current block. The unit of prediction performed in the predictor 230 may be an encoding block, or may be a transform block or may be a prediction block.
The predictor 230 may determine whether to apply intra prediction or inter prediction based on information about prediction. In this case, a unit for determining which one is to be used between intra prediction and inter prediction may be different from a unit for generating prediction samples. In addition, the unit for generating the prediction sample may also be different in inter prediction and intra prediction. For example, which is to be applied between inter prediction and intra prediction may be determined in units of CUs. Also, for example, in inter prediction, prediction samples may be generated by determining a prediction mode in units of PUs, and in intra prediction, prediction samples may be generated in units of TUs by determining a prediction mode in units of PUs.
In the case of intra prediction, the predictor 230 may derive prediction samples for the current block based on neighboring reference samples in the current picture. The predictor 230 may derive prediction samples for the current block by applying a directional mode or a non-directional mode based on neighboring reference samples of the current block. In this case, the prediction mode to be applied to the current block may be determined by using intra prediction modes of neighboring blocks.
In the case of inter prediction, the predictor 230 may derive a prediction sample for the current block based on a sample specified in a reference picture according to a motion vector. The predictor 230 may derive a prediction sample for the current block using one of a skip mode, a merge mode, and an MVP mode. Here, motion information (e.g., a motion vector and information on a reference picture index) required for inter prediction of a current block provided by the video encoding apparatus may be acquired or derived based on the information on prediction.
In the skip mode and the merge mode, motion information of neighboring blocks may be used as motion information of the current block. Here, the neighboring blocks may include spatially neighboring blocks and temporally neighboring blocks.
The predictor 230 may construct a merge candidate list using motion information of available neighboring blocks and use information indicated by a merge index on the merge candidate list as a motion vector of the current block. The merge index may be signaled by the encoding device. The motion information may include a motion vector and a reference picture. In the skip mode and the merge mode, when motion information of temporally adjacent blocks is used, a first-ordered picture in the reference picture list may be used as a reference picture.
In case of the skip mode, unlike the merge mode, a difference (residual) between the prediction sample and the original sample is not transmitted.
In case of the MVP mode, the motion vector of the current block may be derived using the motion vectors of neighboring blocks as a motion vector predictor. Here, the neighboring blocks may include spatially neighboring blocks and temporally neighboring blocks.
When the merge mode is applied, for example, the merge candidate list may be generated using motion vectors of the reconstructed spatially neighboring blocks and/or motion vectors corresponding to Col blocks, which are temporally neighboring blocks. The motion vector of the candidate block selected from the merge candidate list is used as the motion vector of the current block in the merge mode. The above information on prediction may include a merge index indicating a candidate block having a best motion vector selected from candidate blocks included in the merge candidate list. Here, the predictor 230 may derive a motion vector of the current block using the merge index.
When an MVP (motion vector prediction) mode is applied as another example, a motion vector predictor candidate list may be generated using motion vectors of reconstructed spatial neighboring blocks and/or motion vectors corresponding to Col blocks, which are temporal neighboring blocks. That is, motion vectors of the reconstructed spatial neighboring blocks and/or motion vectors corresponding to Col blocks, which are temporal neighboring blocks, may be used as motion vector candidates. The above-mentioned information on prediction may include a prediction motion vector index indicating a best motion vector selected from motion vector candidates included in the list. Here, the predictor 230 may select a prediction motion vector of the current block from among motion vector candidates included in the motion vector candidate list using the motion vector index. A predictor of an encoding apparatus may obtain a Motion Vector Difference (MVD) between a motion vector of a current block and a motion vector predictor, encode the MVD, and output the encoded MVD in the form of a bitstream. That is, the MVD may be obtained by subtracting the motion vector predictor from the motion vector of the current block. Here, the predictor 230 may acquire a motion vector included in the information on prediction and derive a motion vector of the current block by adding a motion vector difference to the motion vector predictor. In addition, the predictor may obtain or derive a reference picture index indicating a reference picture from the above-described information on prediction.
The adder 240 may add the residual samples to the prediction samples to reconstruct the current block or the current picture. The adder 240 may reconstruct the current picture by adding residual samples to prediction samples in units of blocks. When the skip mode is applied, no residual is sent, and thus the prediction samples may become reconstructed samples. Although the adder 240 is described as a separate component, the adder 240 may be part of the predictor 230. In addition, the adder 240 may be referred to as a reconstructor or a reconstruction block generator.
Filter 250 may apply deblock filtering, sample adaptive offset, and/or ALF to the reconstructed picture. Here, the sample adaptive offset may be applied in sample units after the deblocking filtering. The ALF may be applied after deblock filtering and/or applying a sample adaptive offset.
The memory 260 may store reconstructed pictures (decoded pictures) or information required for decoding. Here, the reconstructed picture may be a reconstructed picture filtered by the filter 250. For example, the memory 260 may store pictures for inter prediction. Here, a picture for inter prediction may be specified according to a reference picture set or a reference picture list. The reconstructed picture may be used as a reference picture for other pictures. The memory 260 may output the reconstructed pictures in output order.
Further, as described above, in performing video encoding, prediction is performed to improve compression efficiency. Accordingly, a prediction block including prediction samples for a current block that is a block to be encoded (i.e., an encoding target block) may be generated. Here, the prediction block includes prediction samples in a spatial domain (or a pixel domain). The prediction block is derived in the same manner in the encoding apparatus and the decoding apparatus, and the encoding apparatus may signal information on a residual between the original block and the prediction block (residual information) to the decoding apparatus instead of the original sample values of the original block, thereby improving image encoding efficiency. The decoding apparatus may derive a residual block including residual samples based on the residual information, add the residual block to the prediction block to generate a reconstructed block including reconstructed samples, and generate a reconstructed picture including the reconstructed block.
The residual information may be generated by a transform and quantization process. For example, the encoding apparatus may derive a residual block between an original block and a prediction block, perform a transform process on residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients, and may signal related residual information to the decoding apparatus (through a bitstream). Here, the residual information may include value information of a quantized transform coefficient, position information, a transform technique, a transform kernel, a quantization parameter, and the like. The decoding apparatus may perform an inverse quantization/inverse transformation process based on the residual information and may derive residual samples (or residual blocks). The decoding apparatus may generate a reconstructed picture based on the prediction block and the residual block. In addition, for inter prediction of a later reference picture, the encoding apparatus may also inverse quantize/inverse transform the quantized transform coefficients to derive a residual block, and generate a reconstructed picture based on the residual block.
Fig. 3a to 3c are diagrams illustrating examples of QT segmentation, BT segmentation, and TT segmentation.
In video coding, a block may be partitioned based on a Quadtree (QT). In addition, one sub-block partitioned by QT may be further recursively partitioned using QT. Leaf blocks that are no longer partitioned by QT may be partitioned by Binary Trees (BT), Ternary Trees (TT), Striped Trees (ST), and so on. BT can have two forms of segmentation such as horizontal BT (2N × N ) and vertical BT (N × 2N ). TT may have two forms of segmentation such as horizontal TT (2N × 1/2N ) and vertical TT (1/2N × 2N ). ST may have two forms of segmentation such as horizontal ST (2N × 1/2N,2N × 1/2N,2N × 1/2N,2N × 1/2N) and vertical ST (1/2N × 2N,1/2N × 2N,1/2N × 2N,1/2N × 2N). Each of BT, TT and ST may be further recursively segmented using BT, TT and ST.
FIG. 3a illustrates an example of QT partitioning. Block a may be split into four sub-blocks (a0, Al, a2, A3) by QT. The sub-block a1 may be further divided into four sub-blocks (B0, B1, B2, B3) by QT.
Fig. 3b illustrates an example of BT segmentation. The block B3, which is no longer split by QT, may be split into vertical BT (C0, C1) or horizontal BT (D0, D1). As with block C0, each sub-block may be further recursively partitioned in the form of a horizontal BT (E0, E1) or a vertical BT (F0, F1).
Fig. 3c illustrates an example of TT segmentation. The block B3, which is no longer split by QT, can be split into a vertical TT (C0, C1, C2) or a horizontal TT (D0, D1, D2). As with block C1, each sub-block may be further recursively split in the form of as horizontal TT (E0, E1, E2) or vertical TT (F0, F1, F2).
Table 1 below represents an example of encoding a quadtree syntax in the case of using BT and TT.
[ Table 1]
The syntax element split _ QT _ flag indicates whether the current (coded) block is split into four by QT. When the value of split _ QT _ flag is 0, it means that the current block is not divided by QT, and when the value of split _ QT _ flag is 1, it means that the current block is divided by QT. x0 and y0 represent the top left position of the luminance image.
The present specification defines specific information or concepts using specific terms or sentences. For example, the present specification uses specific terms or sentences to define specific syntax elements. As an example, this specification defines a syntax element indicating whether a current (coded) block is divided into 4 by QT as "split _ QT _ flag", and a syntax element indicating whether the current (coded) block is further divided as "split _ fur _ flag". However, the "split _ QT _ flag" may be replaced with various terms such as QT _ split _ flag and QT split flag, and the "split _ fur _ flag" may be replaced with various terms such as furter _ split _ flag and fur _ split _ flag, so that a specific term or sentence for determining specific information or concept in the present specification should not be construed as being limited to those in the entire specification, but needs to be construed by focusing on various operations, functions, and effects according to what the term denotes.
Table 2 below shows an example of syntax of the coding tree.
[ Table 2]
The syntax element split _ fur _ flag indicates whether the current (coded) block is further partitioned. When the value of the split _ fur _ flag is 0, it means that the current block is not divided any more, and when the value of the split _ fur _ flag is 1, it means that the current block is divided. The syntax element split _ BT _ flag indicates whether the current block is split based on BT or TT. When the value of split _ BT _ flag is 1, it means that the current block is divided based on BT, and when the value of split _ BT _ flag is 0, it means that the current block is divided based on TT.
The partition type (split type) of the current block may be determined as represented in table 3 below based on the syntax element split _ fur _ flag and the syntax element split _ bt _ flag.
[ Table 3]
spllit_fur_flag
split_bt_flag
SpplitType
0
NO_SPLIT
1
1
BT
1
0
TT
Referring to table 3, when the value of SPLIT _ fur _ flag is 0 and the current block is NO longer divided, it may be confirmed that the SPLIT type is determined to be NO _ SPLIT. In addition, when the value of split _ fur _ flag is 1 and the current block is divided, it may be confirmed that the split type is determined as BT when the value of split _ BT _ flag is 1, and the split type is determined as TT when the value of split _ BT _ flag is 0.
The syntax element split dir represents the partition direction of the current block. When the value of split _ dir is 0, it means that the current block is horizontally divided, and when the value of split _ dir is 1, it means that the current block is vertically divided. The partition mode (split mode) of the block can be derived based on split type and split _ dir as represented in table 4 below.
[ Table 4]
Referring to table 4, when the SplitType is BT, it may be confirmed that when the value of the SplitType is 0, the SplitMode is determined as PART _ BT _ HOR, i.e., the BT horizontal split mode, and when the value of the SplitMode is 1, the SplitMode is determined as PART _ BT _ VER, i.e., the BT vertical split mode. In addition, when the SplitType is TT, it can be confirmed that when the value of the SplitType is 0, the SplitMode is determined as PART _ TT _ HOR, i.e., TT horizontal split mode, and when the value of the SplitMode is 1, the SplitMode is determined as PART _ TT _ VER, i.e., TT vertical split mode.
When a region of each picture is partitioned into a QT structure, the encoder and the decoder may transmit a syntax element that adjusts the degree of partitioning of the QT in consideration of a relationship between performance and complexity. For example, syntax elements such as maximum QT block size (MaxQTSize), minimum QT block size (MinQTSize), and maximum QT block partitioning depth (MaxQTDepth) may be transmitted. Here, the maximum QT block size may mean the size of the maximum block that is first split by QT and is expressed in the form of a log2 scale. The minimum QT block size may mean the size of the smallest block divided by QT and is expressed in the form of a log2 scale. Additionally, the maximum QT split depth may mean how many QT splits are allowed from the maximum QT block size.
When an area of each image is divided into a QT structure and then a block is divided into a BT structure, the encoder and the decoder may transmit a syntax element that adjusts the degree of division of BT in consideration of the relationship between performance and complexity. For example, syntax elements such as maximum BT block size (MaxBTSize), minimum BT block size (MinBTSize), and maximum BT block partition depth (MaxBTDepth) may be transmitted. More specifically, the maximum BT block size may mean the size of the largest block that allows BT partitioning first, and is expressed in the form of a log2 scale. The minimum BT block size may mean the size of the minimum block divided by BT and expressed in the form of a log2 scale. In addition, the maximum BT partition depth may mean how many BT partitions are allowed from the maximum BT block size.
When a region of each picture is divided into a QT structure and then a block is divided into a TT structure, the encoder and the decoder may transmit a syntax element that adjusts the division of TT in consideration of the relationship between performance and complexity. For example, syntax elements such as maximum TT block size (MaxTTSize), minimum TT block size (MinTTSize), and maximum TT block partition depth (MaxTDepth) may be transmitted. More specifically, the maximum TT block size may mean the size of the largest block that allows TT segmentation first, and is expressed in the form of a log2 scale. The minimum TT block size may mean the size of the minimum block divided by TT and expressed in a log2 scale. In addition, the maximum TT split depth may mean how many times TT splits are allowed from the maximum TT block size.
Fig. 4 is a block diagram illustrating a configuration of an entropy encoder according to an exemplary embodiment.
Fig. 4 illustrates a schematic block diagram of an entropy encoder applying context-based adaptive binary arithmetic coding (CABAC) as an exemplary embodiment to which the present disclosure is applied.
The entropy encoder 400 to which the present disclosure is applied includes a binarization unit 410, a context modeling unit 420, a binary arithmetic coding unit 430, and a memory 460, wherein the binary arithmetic coding unit 430 may include a regular binary coding unit 440 and a bypass binary coding unit 450. Here, the regular binary encoding unit 440 and the bypass binary encoding unit 450 may also be referred to as a regular encoding engine and a bypass encoding engine, respectively.
The binarization unit 410 may receive a sequence of data symbols and perform binarization, thereby outputting a binary symbol (bin, bit) string consisting of binarized values of 0 or 1. The binarization unit 410 may map the syntax elements with the binary symbols. For binarization, various binarization processes may be used, such as unary code (U), truncated unary code (TU), K-th order exponential Golomb code (EGk), fixed length processes, and so forth. The binarization process may be selected based on the form of the syntax element.
The output bit string may be sent to the context modeling unit 420.
The context modeling unit 420 may select probability information necessary for encoding the current block from the memory to transmit the probability information to the binary arithmetic encoding unit 430. For example, the context modeling unit 420 may select a context memory based on a syntax element to be encoded and select probability information necessary for encoding a current syntax element through a bit index (binIdx). Here, the context means information on the occurrence probability of a symbol, and the context modeling may mean a process of estimating the probability of bits necessary for binary arithmetic coding by receiving bits as a binarization result.
The context modeling unit 420 may provide the (accurate) probability estimates necessary to achieve enhanced coding efficiency. Thus, different context models may be used for different binary symbols, and the probabilities of the context models may be updated based on the values of previously encoded binary symbols. At this time, the values of the previously encoded binary symbols are stored in the memory 460, and the context modeling unit 420 may use the values of the previously encoded binary symbols from the memory 460.
Binary symbols with similar distributions may share the same context model. The context model for each binary symbol may estimate the probability using at least one of syntax information of bits, a bin index (bin idx) representing a position of the bin in a bin string, and a probability of a bin included in a neighboring block of a block in which the bin is included.
The binary arithmetic encoding unit 430 includes a normal binary encoding unit 440 and a bypass binary encoding unit 450, and may perform entropy encoding on the output string and output compressed data bits.
The conventional binary encoding unit 440 may perform arithmetic encoding based on recursive interval division.
First, an interval (or range) having an initial value of 0 or 1 may be divided into two lower intervals based on the probability of the binary symbol. When converted to a binary fraction, the coded bits may provide an offset for selecting one of two lower intervals of values representing the decoded binary symbol.
After the binary symbol of the decoded mode, the interval may be updated so that the selected lower interval is the same, and the interval division process itself may be repeated. The bin and offset have limited bit precision, and therefore, whenever the bin is reduced to a particular value or less, renormalization may be required to prevent overflow. After each binary symbol is decoded, renormalization may occur.
The bypass binary encoding unit 450 performs encoding without a context model, and may perform encoding by fixing the probability of the currently encoded bin to 0.5. This method can be used when it is difficult to determine the probability of syntax or high-speed coding is required.
Fig. 5 is a block diagram illustrating a configuration of an entropy decoder according to an exemplary embodiment.
Fig. 5 illustrates a schematic block diagram of an entropy decoder to which context-based adaptive binary arithmetic coding (CABAC) is applied as an exemplary embodiment to which the present disclosure is applied.
The entropy decoder 500 according to an exemplary embodiment may include a context modeling unit 510, a binary arithmetic decoding unit 520, a memory 550, and an inverse binarization unit 560, and the binary arithmetic decoding unit 520 may include a normal binary decoding unit 530 and a bypass binary decoding unit 540.
The entropy decoder 500 may receive the bitstream and acknowledge the bypass flag therefrom. Here, the bypass flag indicates whether the bitstream is in a bypass mode, and the bypass mode may mean that encoding is performed by fixing the probability of dropping a currently encoded bin to 0.5 without using a context model.
If the bitstream is not in the bypass mode according to the bypass flag, the normal binary decoding unit 530 may perform binary arithmetic decoding according to the normal mode.
At this time, the context modeling unit 510 may select probability information necessary for decoding the current bitstream from the memory 550 to transmit the probability information to the conventional binary decoding unit 530.
Also, if the bitstream is in the bypass mode according to the bypass flag, the bypass binary decoding unit 540 may perform binary arithmetic decoding according to the bypass mode.
The inverse binarization unit 560 may receive the bin in binary form decoded by the binary arithmetic decoding unit 520 and convert the bin into a syntax element value in integer form to output the converted syntax element value.
FIG. 6 is a flowchart illustrating operation of an entropy encoder according to an exemplary embodiment.
The entropy encoder of the encoding apparatus according to an exemplary embodiment may perform binarization on syntax elements (S610).
The entropy encoder according to an exemplary embodiment may determine whether to perform binary arithmetic coding according to a normal mode or a bypass mode (S620). For example, the entropy encoder may confirm whether the bitstream is in the normal mode or the bypass mode based on the bypass flag, and when the bypass flag is 1, this may represent the bypass mode, and when the bypass flag is 0, this may represent the normal mode.
If it is determined that the binary arithmetic coding is to be performed according to the regular mode, the entropy encoder may select a probability model (S630) and perform the binary arithmetic coding based on the probability model (S640). In addition, the entropy encoder may update the probability model (S650), and select again an appropriate probability model based on the probability model updated in step S630.
If it is determined that the binary arithmetic coding is to be performed according to the bypass mode, the entropy encoder may perform the binary arithmetic coding based on a probability of 0.5 (S660).
Fig. 7 is a flowchart illustrating an operation of an entropy decoder according to an exemplary embodiment.
The entropy decoder of the decoding apparatus according to an exemplary embodiment may receive a bitstream (S710).
The entropy decoder according to an exemplary embodiment can confirm whether the bitstream is in the normal mode or the bypass mode by extracting a bypass flag from the bitstream (S720). Here, the bypass flag may be predetermined according to the kind of syntax.
If the bypass flag represents the normal mode, the entropy decoder may select a probability model (S730) and perform binary arithmetic decoding based on the selected probability model (S740). In addition, the entropy decoder may update the probability model (S750) and select an appropriate probability model again based on the probability model updated in step S730.
Also, if the bypass flag indicates the bypass mode, the entropy decoder may perform binary arithmetic decoding based on a probability of 0.5 (S760).
The entropy decoder may perform inverse binarization on the decoded bin string (S770). For example, the entropy decoder may receive a decoded bin in binary form and convert the bin into a syntax element value in integer form to output the syntax element value.
Exemplary embodiments of the present disclosure propose methods for determining a context model for CABAC entropy coding. As described above, split _ BT _ flag may indicate whether the current block is split into BT or TT. The syntax element split _ bt _ flag may be referred to as mtt _ split _ cu _ binary _ flag, etc. The context model for the syntax element split _ BT _ flag may be determined according to the number of BT and TT that can be split from the current block. Equation 1 below represents the value of the context index (CtxIdx) according to the block division availability information.
[ formula 1]
CtxIdx＝(availableBTHOR&&availableBTVER)*2+(availableTTHOR&&availableTTVER)
In this specification, "block division availability information" may indicate a block division condition. The block division availability information may include BT horizontal division available flag information indicating whether the current block can be horizontally divided based on a Binary Tree (BT) structure, TT horizontal division available flag information indicating whether the current block can be horizontally divided based on a Ternary Tree (TT) structure, BT vertical division available flag information indicating whether the current block can be vertically divided based on the BT structure, and TT vertical division available flag information indicating whether the current block can be vertically divided based on the TT structure. According to an exemplary embodiment, the block division availability information may be derived based on at least one of block size information, multi-type tree depth information, and maximum multi-tree size information.
In equation 1, availableBTHOR may represent BT horizontal split available flag information, availableBTVER may represent BT vertical split available flag information, availableTTHOR may represent TT horizontal split available flag information, and availableTTVER may represent TT vertical split available flag information. For example, the range of the value CtxIdx may be 0 or more and 3 or less.
The BT horizontal split available flag information may be indicated by various names such as available _ BT _ HOR, allowslitbthor, and availableBTHOR. Likewise, the BT vertical split available flag information may be indicated by available _ BT _ VER, allowslitbtver, or the like, the TT horizontal split available flag information may be indicated by available _ TT _ HOR, allowslittthor, or the like, and the TT vertical split available flag information may be indicated by available _ TT _ VER, allowslitttver, or the like.
Table 5 below shows an example of values of CtxIdx according to the values of availableBTHOR, availableBTVER, availableTTHOR, and availableTTVER.
[ Table 5]
availableBTHOR
availableBTVER
availableTTHOR
availableTTVER
Ctxldx
0
0
0
0
N/A
0
0
0
1
N/A
0
0
1
0
N/A
0
0
1
1
N/A
0
1
0
0
N/A
0
1
0
1
0
0
1
1
0
0
0
1
1
1
1
1
0
0
0
N/A
1
0
0
1
0
1
0
1
0
0
1
0
1
1
1
1
1
0
0
N/A
1
1
0
1
2
1
1
1
0
2
1
1
1
1
3
When each of the values availableBTHOR, availableBTVER, availableTTHOR, and availableTTVER is 0, it indicates that the segmentation according to each segmentation structure is unavailable, and when each value is 1, it indicates that the segmentation according to the segmentation structure is available. If the value of CtxIdx is N/A, it may mean that the value of CtxIdx does not have to be determined because signaling of split _ bt _ flag is unnecessary.
According to an exemplary embodiment, the split _ bt _ flag may be encoded using four context models. Table 6 below shows an example of an initial value (InitValue) of CABAC according to the value of CtxIdx.
[ Table 6]
Ctxldx
InitValue
0
152
1
153
2
154
3
152
As another example, when the value of CtxIdx is 0 or 3, the same initial value of CABAC may be used, and when having similar statistical properties, split _ bt _ flag may be encoded using three context models. That is, when the value of CtxIdx is 3, the value of CtxIdx may be replaced with 0. Table 7 below represents an example of the initial values of CABAC according to the value of CtxIdx when three context models are used.
[ Table 7]
Ctxldx
lnitValue
0
152
1
153
2
154
Another exemplary embodiment of the present disclosure proposes another method for determining a context model for CABAC entropy coding. As an example, the syntax element split _ BT _ flag may be signaled first to determine whether it is BT split or TT split, and then the syntax element split _ dir may be signaled to determine whether it is horizontal split or vertical split. In some cases, the syntax element split dir may be referred to as mtt _ split _ cu _ vertical _ flag. As another example, the syntax element split dir may be signaled first to determine whether it is horizontally split or vertically split, and then the syntax element split BT flag may be signaled to determine whether it is BT split or TT split.
At this time, a context model for the syntax element split _ dir may be determined according to the number of horizontal partitions and vertical partitions that can be split from the current block. Equation 2 below represents the value of the context index (CtxIdx) according to the block division availability information.
[ formula 2]
CtxIdx＝(availableBTHOR&&availableTTHOR)*2+(availableBTVER&&availableTTVER)
Table 8 below shows an example of the values of CtxIdx according to the values of availableBTHOR, availableBTVER, availableTTHOR, and availableTTVER.
[ Table 8]
availableBTHOR
availableTTHOR
availableBTVER
availableTTVER
Ctxldx
0
0
0
0
N/A
0
0
0
1
N/A
0
0
1
0
N/A
0
0
1
1
N/A
0
1
0
0
N/A
0
1
0
1
0
0
1
1
0
0
0
1
1
1
1
1
0
0
0
N/A
1
0
0
1
0
1
0
1
0
0
1
0
1
1
1
1
1
0
0
N/A
1
1
0
1
2
1
1
1
0
2
1
1
1
1
3
In table 8, the case where the value of CtxIdx is N/a may represent the case where the value of CtxIdx is not necessarily determined because signaling of split _ bt _ flag is unnecessary.
According to an exemplary embodiment, referring to table 8, if the values of availableBTHOR and availableTTHOR are 0 or the values of availableBTVER and availableTTVER are 0, it may be confirmed that CtxIdx of split dir is not derived.
According to an exemplary embodiment, referring to table 8, when the value of (availableBTHOR | | availableTTHOR) & & (availableBTVER | | availableTTVER) is 0, CtxIdx for which split _ dir is not derived may be confirmed.
According to an exemplary embodiment, the split _ dir _ flag may be encoded using four context models. Table 9 below shows an example of the initial value (InitValue) of CABAC according to the value of CtxIdx.
[ Table 9]
Ctxldx
InitValue
0
154
1
155
2
153
3
154
As another example, when the value of CtxIdx is 0 or 3, the same initial value of CABAC is used, and when having similar statistical properties, split _ bt _ flag may be encoded using three context models. That is, when the value of CtxIdx is 3, the value of CtxIdx may be replaced with 0. Table 10 below represents an example of initial values of CABAC according to the value of CtxIdx in the case of using three context models.
[ Table 10]
Ctxldx
InitValue
0
154
1
155
2
153
The exemplary embodiment may determine the block partition according to a transform size. The available transform size of a currently decoded image or picture may be transmitted in units of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Slice Header (SH), a header of a Network Abstraction Layer (NAL) unit, and the like. For available transform sizes, the corresponding transform sizes may also be transmitted one after the other, and the corresponding indices may also be transmitted based on a predefined set of transforms. The decoder terminal may receive and decode the index to derive the available transform size.
When the available transform size is determined, the size of the available block partitions may be determined. That is, the size of the available block partitions is defined for the currently decoded image and picture. The size of the available block partitions may be determined by transforming the available cases. Thus, the partitioning of blocks in which no transform exists may not be allowed.
The size of the available block partitions is determined for images and pictures, and then it may be determined whether a block may be partitioned in units of blocks (i.e., in units of CUs or PUs). The kind of available block partitioning is determined based on the size of the current block. That is, BT or TT available candidates may be configured horizontally and vertically. Alternatively, in addition to the 1/2 ratio split, it may be determined whether 1/4, 3/4 ratio split, 3/8, 5/8 ratio split, 1/3, 2/3 ratio split, 1/5, 4/5 ratio split, or the like is available.
The partition available block candidates are configured and then the final block partition can be determined by the block partition syntax sent to the decoder. If there is one available block candidate or there is no available block candidate, the corresponding syntax may not be transmitted.
Fig. 8 is a flowchart illustrating an operation of an encoding apparatus according to an exemplary embodiment, and fig. 9 is a block diagram illustrating a configuration of the encoding apparatus according to the exemplary embodiment.
The encoding apparatus illustrated in fig. 8 and 9 may perform operations corresponding to those of a decoding apparatus that will be illustrated in fig. 10 and 11 later. Therefore, contents to be described later with reference to fig. 10 and 11 can be equally applied to the encoding apparatuses illustrated in fig. 8 and 9.
Each step illustrated in fig. 8 may be performed by the encoding apparatus 100 illustrated in fig. 1. More specifically, S800 to S830 may be performed by the entropy encoder 130 illustrated in fig. 1. In addition, the operations according to S800 to S830 are performed based on some of the contents described above with reference to fig. 3 to 7. Therefore, the description of specific contents overlapping with those described above with reference to fig. 3 to 7 will be omitted or simplified.
As illustrated in fig. 9, an encoding apparatus according to an exemplary embodiment may include a picture segmentation module 105 and an entropy encoder 130. However, in some cases, all the components illustrated in fig. 9 may not be essential components of the encoding apparatus, and the encoding apparatus may be implemented by a larger number or a smaller number of components than those illustrated in fig. 9.
In the encoding apparatus according to the exemplary embodiment, each of the picture segmentation module 105 and the entropy encoder 130 may be implemented by a separate chip, or two or more components may also be implemented by a single chip.
The encoding apparatus according to an exemplary embodiment may derive a plurality of different coding units by dividing a current block within a current picture. More specifically, the picture segmentation module 105 of the encoding device may derive a plurality of different coding units by segmenting the current block within the current picture.
The encoding apparatus according to an exemplary embodiment may derive a partition flag syntax element representing a partition structure of a current block (S800). More specifically, the entropy encoder 130 of the encoding apparatus may derive a partition flag syntax element representing a partition structure of the current block.
According to an exemplary embodiment, the partition flag syntax element may include a directional partition flag indicating whether the current block is vertically partitioned, and the partition syntax element may include a BT partition flag indicating whether the current block is partitioned based on a BT structure. The directional split flag may be indicated by split _ dir, for example, and the BT split flag may be indicated by split _ BT _ flag, for example.
The encoding apparatus according to an exemplary embodiment may derive a context index of a partition flag syntax element based on partition availability information of a current block (S810). More specifically, the entropy encoder 130 of the encoding apparatus may derive a context index of a partition flag syntax element based on the partition availability information of the current block.
According to an exemplary embodiment, the partition availability information may include BT horizontal partition available flag information indicating whether the current block can be horizontally partitioned based on the BT structure, TT horizontal partition available flag information indicating whether the current block can be horizontally partitioned based on the TT structure, BT vertical partition available flag information indicating whether the current block can be vertically partitioned based on the BT structure, and TT vertical partition available flag information indicating whether the current block can be vertically partitioned based on the TT structure.
According to an example embodiment, the BT horizontal split available flag information may be indicated by availableBTHOR, the TT horizontal split available flag information may be indicated by availableTTHOR, the BT vertical split available flag information may be indicated by availableBTVER, and the TT vertical split available flag information may be indicated by availableTTVER.
According to an exemplary embodiment, a context index of a directional section flag may be derived based on BT horizontal section available flag information, TT horizontal section available flag information, BT vertical section available flag information, and TT vertical section available flag information.
According to an exemplary embodiment, whether to derive a context index of a directional section flag may be determined based on BT horizontal section available flag information, TT horizontal section available flag information, BT vertical section available flag information, and TT vertical section available flag information.
According to an exemplary embodiment, if the value of the BT horizontal division available flag information and the value of the TT horizontal division available flag information are 0, or the value of the BT vertical division available flag information and the value of the TT vertical division available flag information are 0, it is characterized in that the context index of the vertical division flag is not derived.
According to an exemplary embodiment, when the value of (availableBTHOR | | availableTTHOR) & & (availableBTVER | | availableTTVER) is 0, the context index of the vertical division flag may not be derived.
According to an exemplary embodiment, a context index of a BT split flag may be derived based on BT horizontal split available flag information, TT horizontal split available flag information, BT vertical split available flag information, and TT vertical split available flag information.
According to an exemplary embodiment, the context index of the directional segmentation flag may be derived based on equation 3 below.
[ formula 3]
CtxIdx＝(availableBTHOR&&availableTTHOR)*2+(availableBTVER&&availableTTVER)
According to an exemplary embodiment, the context index of the BT segmentation flag may be derived based on equation 4 below.
[ formula 4]
CtxIdx＝(availableBTHOR&&availableBTVER)*2+(availableTTHOR&&availableTTVER)
The encoding apparatus according to an exemplary embodiment may determine a context model based on the derived context index (S820). More specifically, the entropy encoder 130 of the encoding apparatus may determine the context model based on the derived context index.
The encoding apparatus according to an exemplary embodiment may encode the value of the partition flag syntax element based on CABAC using the determined context model (S830). More specifically, the entropy encoder 130 of the encoding apparatus may encode the value of the partition flag syntax element based on CABAC using the determined context model.
According to the encoding apparatus and the method for operating the encoding apparatus illustrated in fig. 8 and 9, the encoding apparatus may derive a partition flag syntax element representing a partition structure of a current block (S800), derive a context index of the partition flag syntax element based on partition availability information of the current block (S810), determine a context model based on the derived context index (S820), and encode a value of the partition flag syntax element based on CABAC using the determined context model (S830), and at this time, may be characterized in that the partition flag syntax element may include a directional partition flag representing whether the current block is vertically partitioned, the partition availability information may include BT horizontal partition available flag information representing whether the current block is horizontally partitioned based on the BT structure, TT horizontal partition available flag information representing whether the current block is horizontally partitioned based on the TT structure, and, The context index of the directional partition flag is derived based on BT vertical partition available flag information whether the BT structure can vertically partition the current block and TT vertical partition available flag information whether the TT structure can vertically partition the current block, and based on the BT horizontal partition available flag information, the TT horizontal partition available flag information, the BT vertical partition available flag information, and the TT vertical partition available flag information. That is, as illustrated in fig. 8 and 9, when entropy encoding according to CABAC is performed based on the QT structure, the BT structure, and the TT structure, encoding and decoding can be more efficiently performed using the context model.
Fig. 10 is a flowchart illustrating an operation of a decoding apparatus according to an exemplary embodiment, and fig. 11 is a block diagram illustrating a configuration of the decoding apparatus according to the exemplary embodiment.
Each step illustrated in fig. 10 may be performed by the decoding apparatus 200 illustrated in fig. 2. More specifically, S1000 to S1030 may be performed by the entropy decoder 210 illustrated in fig. 2, S1040 may be performed by the predictor 230 illustrated in fig. 2, and S1050 may be performed by the adder 240 illustrated in fig. 2. In addition, the operations according to S1000 to S1050 are performed based on some of the contents described above with reference to fig. 3 to 7. Therefore, the description of specific contents overlapping with those described above with reference to fig. 3 to 7 will be omitted or simplified.
As illustrated in fig. 11, the decoding apparatus according to an exemplary embodiment may include an entropy decoder 210, a predictor 230, and an adder 240. However, in some cases, all of the components illustrated in fig. 11 may not be essential components of the decoding apparatus, and the decoding apparatus may be implemented by a larger number or a smaller number of components than those illustrated in fig. 11.
In the decoding apparatus according to the exemplary embodiment, each of the entropy decoder 210, the predictor 230, and the adder 240 may be implemented by a separate chip, or two or more components may be implemented by a single chip.
The decoding apparatus according to an exemplary embodiment may derive a context index of a partition flag syntax element based on partition availability information of a current block (S1000). More specifically, the entropy decoder 210 of the decoding apparatus may derive a context index of a partition flag syntax element based on the partition availability information of the current block.
According to an exemplary embodiment, the partition flag syntax element may include a directional partition flag indicating whether the current block is vertically partitioned, and the partition syntax element may include a BT partition flag indicating whether the current block is partitioned based on a BT structure. The directional split flag may be indicated by split _ dir, for example, and the BT split flag may be indicated by split _ BT _ flag, for example.
According to an exemplary embodiment, the partition availability information may include BT horizontal partition available flag information indicating whether the current block can be horizontally partitioned based on the BT structure, TT horizontal partition available flag information indicating whether the current block can be horizontally partitioned based on the TT structure, BT vertical partition available flag information indicating whether the current block can be vertically partitioned based on the BT structure, and TT vertical partition available flag information indicating whether the current block can be vertically partitioned based on the TT structure.
As an example, the BT horizontal split available flag information may be indicated by availableBTHOR, the TT horizontal split available flag information may be indicated by availableTTHOR, the BT vertical split available flag information may be indicated by availableBTVER, and the TT vertical split available flag information may be indicated by availableTTVER.
According to an exemplary embodiment, a context index of a directional section flag may be derived based on BT horizontal section available flag information, TT horizontal section available flag information, BT vertical section available flag information, and TT vertical section available flag information.
According to an exemplary embodiment, whether to derive a context index of a directional section flag may be determined based on BT horizontal section available flag information, TT horizontal section available flag information, BT vertical section available flag information, and TT vertical section available flag information.
According to an exemplary embodiment, if the value of the BT horizontal division available flag information and the value of the TT horizontal division available flag information are 0, or the value of the BT vertical division available flag information and the value of the TT vertical division available flag information are 0, the context index of the directional division flag may not be derived.
According to an exemplary embodiment, when the value of (availableBTHOR | | availableTTHOR) & & (availableBTVER | | availableTTVER) is 0, the context index of the directional segmentation flag may not be derived.
According to an exemplary embodiment, a context index of a BT split flag may be derived based on BT horizontal split available flag information, TT horizontal split available flag information, BT vertical split available flag information, and TT vertical split available flag information.
According to an exemplary embodiment, the context index of the directional division flag may be derived based on the following equation 5.
[ formula 5]
CtxIdx＝(availableBTHOR&&availableTTHOR)*2+(availableBTVER&&availableTTVER)
According to an exemplary embodiment, the context index of the BT segmentation flag may be derived based on equation 6 below.
[ formula 6]
CtxIdx＝(availableBTHOR&&availableBTVER)*2+(availableTTHOR&&availableTTVER)
The decoding apparatus according to an exemplary embodiment may determine a context model based on the derived context index (S1010). More specifically, the entropy decoder 210 of the decoding apparatus may determine the context model based on the derived context index.
The decoding apparatus according to an exemplary embodiment may decode the value of the partition flag syntax element on a context-based adaptive binary arithmetic coding (CABAC) basis using the determined context model (S1020). More specifically, the entropy decoder 210 of the decoding apparatus may decode the value of the partition flag syntax element based on CABAC using the determined context model.
The decoding apparatus according to an exemplary embodiment may derive a current coding unit from the current block based on the value of the partition flag syntax element (S1030). More specifically, the entropy decoder 210 of the decoding apparatus may derive the current coding unit from the current block based on the value of the partition flag syntax element.
The decoding apparatus according to an exemplary embodiment may derive the prediction block based on inter prediction or intra prediction with respect to the current coding unit (S1040). More specifically, the predictor 230 of the decoding apparatus may derive the prediction block based on inter prediction or intra prediction with respect to the current coding unit.
The decoding apparatus according to an exemplary embodiment may generate a reconstructed block based on the prediction block (S1050). More specifically, the adder 240 of the decoding apparatus may generate a reconstructed block based on the prediction block.
According to the decoding apparatus and the method of operating the decoding apparatus illustrated in fig. 10 and 11, the decoding apparatus may derive a context index of a partition flag syntax element based on partition availability information of a current block (S1000), determine a context model based on the derived context index (S1010), decode a value of the partition flag syntax element based on context-based adaptive binary arithmetic coding (CABAC) using the determined context model (S1020), derive a current coding unit from the current block based on the value of the partition flag syntax element (S1030), derive a prediction block based on inter prediction or intra prediction with respect to the current coding unit (S1040), and generate a reconstructed block based on the prediction block (S1050), and at this time, may be characterized in that the partition flag syntax element may include a directional partition flag indicating whether the current block is vertically partitioned, the partition availability information may include Binary Tree (BT) horizontal partition available flag information indicating whether the current block can be horizontally partitioned based on a BT structure, TT horizontal partition available flag information indicating whether the current block can be horizontally partitioned based on a TT structure, BT vertical partition available flag information indicating whether the current block can be vertically partitioned based on the BT structure, and TT vertical partition available flag information indicating whether the current block can be vertically partitioned based on the TT structure; and deriving a context index of the directional division flag based on the BT horizontal division available flag information, the TT horizontal division available flag information, the BT vertical division available flag information, and the TT vertical division available flag information. That is, as illustrated in fig. 10 and 11, when entropy encoding according to CABAC is performed based on the QT structure, the BT structure, and the TT structure, encoding and decoding can be more efficiently performed using the context model.
The exemplary embodiments described in this disclosure may be implemented and executed by a processor, microprocessor, controller or chip. For example, the functional units illustrated in each of the figures may be implemented and executed by a computer, processor, microprocessor, controller, or chip.
In addition, the decoder and the encoder to which exemplary embodiments of the present disclosure are applied may be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater video device, a digital cinema video device, a surveillance camera, a video chat device, a real-time communication device such as a video communication device, a mobile streaming device, a storage medium, a camcorder, a video on demand (VoD) service providing device, an over-the-top video (OTT) video device, an internet streaming service providing device, a three-dimensional (3D) video device, a picture phone video device, a medical video device, and the like, and be used to process a video signal or a data signal. For example, an over-the-top (OTT) video device may include a game console, a blu-ray player, an internet access TV, a home theater system, a smart phone, a tablet PC, a Digital Video Recorder (DVR), and so forth.
In addition, the processing method to which the exemplary embodiment of the present disclosure is applied may be generated in the form of a program executed by a computer and stored in a computer-readable recording medium. The multimedia data having the data structure according to the present disclosure may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a blu-ray disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, the computer-readable recording medium includes media implemented in the form of carrier waves (e.g., transmission over the internet). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or may be transmitted through a wired or wireless communication network.
In addition, the exemplary embodiments of the present disclosure may be implemented as a computer program product by program code, and the program code may be executed on a computer by the exemplary embodiments of the present disclosure. The program code may be stored on a computer readable carrier.
Fig. 12 is a diagram of a content streaming system structure according to an exemplary embodiment.
A content streaming system to which the present disclosure is applied may mainly include an encoding server, a streaming server, a network server, a media storage, a user equipment, and a multimedia input device.
The encoding server serves to compress contents input from multimedia input devices such as a smart phone, a camera, and a camcorder into digital data to generate a bitstream, and transmit the bitstream to the streaming server. As another example, if a multimedia input device such as a smart phone, a camera, or a camcorder directly generates a bitstream, an encoding server may be omitted.
The bitstream may be generated by applying the encoding method or the bitstream generation method of the present disclosure, and the streaming server may temporarily store the bitstream in a process of transmitting or receiving the bitstream.
The streaming server transmits multimedia data to the user equipment upon a request of the user through the web server, and the web server serves as an intermediary for informing the user of what services exist. When a user requests a desired service from the web server, the web server transmits the service to the streaming server, and the streaming server transmits multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server serves to control instructions/responses between the respective devices in the content streaming system.
The streaming server may receive content from the media store and/or the encoding server. For example, if the content is received from an encoding server, the content may be received in real time. In this case, the streaming server may store the bit stream for a predetermined time to smoothly provide the streaming service.
Examples of user equipment may include mobile phones, smart phones, laptop computers, digital broadcast terminals, Personal Digital Assistants (PDAs), Portable Multimedia Players (PMPs), navigation devices, touch screen PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smart watches), glasses-type terminals (smart glasses), head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, and the like.
The respective servers in the content streaming system may operate as distributed servers, and in this case, data received by each server may be processed in a distributed manner.
The above-mentioned method according to the present disclosure may be implemented in the form of software, and the encoding apparatus and/or the decoding apparatus according to the present disclosure may be included in an apparatus that performs image processing, such as a TV, a computer, a smart phone, a set-top box, or a display device.
Each of the above-described parts, modules or units may be a processor or a hardware part that executes a continuously implemented process stored in a memory (or storage unit). Each step described in the above-mentioned exemplary embodiments may be performed by a processor or a hardware portion. The various modules/blocks/units described in the above-mentioned exemplary embodiments may operate as hardware/processors. Additionally, the methods provided by the present disclosure may be executed as code. The code may be written to a storage medium readable by a processor and, thus, readable by a processor provided by the device.
In the above-mentioned exemplary embodiments, the method is explained based on the flowchart by means of a series of steps or blocks, but the present disclosure is not limited to the order of the steps, and a certain step may occur in a different order or step from the above-mentioned order or step, or simultaneously with the above-mentioned order or step. In addition, one of ordinary skill in the art may appreciate that the steps shown in the flowcharts are not exclusive and that another step or one or more steps of the flowcharts may be eliminated without affecting the scope of the present disclosure.
When implemented by software according to exemplary embodiments of the present disclosure, the above-mentioned method may be implemented by modules (processes, functions, etc.) that perform the above-mentioned functions. The modules may be stored in a memory and executed by a processor. The memory may be internal or external to the processor and connected to the processor via various well-known means. The processor may include an Application Specific Integrated Circuit (ASIC), other chipset, logic circuit, and/or data processing device. The memory may include Read Only Memory (ROM), Random Access Memory (RAM), flash memory, memory cards, storage media, and/or other storage devices.
- 上一篇：一种医用注射器针头装配设备
- 下一篇：视频编码的方法和装置