阅读说明：本技术 发送方法 (Transmission method ) 是由 井口贺敬 远间正真 西孝启 加藤久也 于 2014-06-16 设计创作，主要内容包括：本申请的一方式所涉及的发送方法包括：生成编码流,所述编码流包括第1内容和至少一个第2内容；生成控制信息,所述控制信息对应于MPEG-DASH标准中定义的媒体演示描述,并且包括：第1识别符,用于识别各个第1内容；至少一个第2识别符,用于识别所述至少一个第2内容；等级识别符,用于识别第1参照内容,所述第1参照内容用于解码所述至少一个第2内容并且被包括在所述第1内容中；时刻信息,用于再现所述第1内容和所述至少一个第2内容；以及重构数据,用于重构所述编码流；从通信服务器向客户端装置发送所述编码流和所述控制信息。(A transmission method according to an aspect of the present application includes: generating an encoded stream, the encoded stream including a 1 st content and at least one 2 nd content; generating control information corresponding to a media presentation description defined in the MPEG-DASH standard and comprising: 1 st identifiers for identifying respective 1 st contents; at least one 2 nd identifier for identifying the at least one 2 nd content; a level identifier for identifying a 1 st reference content, the 1 st reference content being used for decoding the at least one 2 nd content and being included in the 1 st content; time information for reproducing the 1 st content and the at least one 2 nd content; and reconstruction data for reconstructing the encoded stream; the encoded stream and the control information are transmitted from a communication server to a client device.)

1. A transmission method, comprising:

generating an encoded stream, the encoded stream including a 1 st content and at least one 2 nd content;

generating control information corresponding to a media presentation description defined in the MPEG-DASH standard and comprising:

1 st identifiers for identifying respective 1 st contents;

at least one 2 nd identifier for identifying the at least one 2 nd content;

a level identifier for identifying a 1 st reference content, the 1 st reference content being used for decoding the at least one 2 nd content and being included in the 1 st content;

time information for reproducing the 1 st content and the at least one 2 nd content; and

reconstruction data for reconstructing the encoded stream; the encoded stream and the control information are transmitted from a communication server to a client device.

2. The transmission method of claim 1, wherein,

each of the 1 st content and the at least one 2 nd content is a video content or a sound content.

3. The transmission method of claim 1, wherein,

the 1 st content can be independently decoded.

4. The transmission method of claim 1, wherein,

the reconstruction data includes time offset information.

5. The transmission method of claim 1, wherein,

the reconstruction data represents a change in the 1 st content and the at least one 2 nd content.

6. The transmission method of claim 5, wherein,

the reconstruction data represents a removal of at least one content.

7. The transmission method of claim 1, wherein,

the reconstruction data is different from information used to decode the 1 st content and the at least one 2 nd content.

Technical Field

The present application relates to a method for transmitting hierarchically encoded data.

Background

Conventionally, a technique for transmitting encoded data in a predetermined multiplexing system is known. The encoded data is generated by encoding content including Video data and audio data based on a moving image Coding standard such as HEVC (High Efficiency Video Coding).

Examples of the predetermined multiplexing scheme include MPEG-2TS (moving picture Experts Group-2 Transport Stream), MMT (moving picture Experts Group-2 Transport Stream), and the like (see non-patent document 1).

In HEVC, hierarchical coding is possible. On the receiving side, the encoded data obtained by hierarchically encoding can be sorted according to the hierarchical level, thereby enabling selection of a frame rate of a video or the like.

Disclosure of Invention

A transmission method according to an aspect of the present application includes: generating an encoded stream, the encoded stream including a 1 st content and at least one 2 nd content; generating control information corresponding to a media presentation description defined in the MPEG-DASH standard and comprising: 1 st identifiers for identifying respective 1 st contents; at least one 2 nd identifier for identifying the at least one 2 nd content; a level identifier for identifying a 1 st reference content, the 1 st reference content being used for decoding the at least one 2 nd content and being included in the 1 st content; time information for reproducing the 1 st content and the at least one 2 nd content; and reconstruction data for reconstructing the encoded stream; the encoded stream and the control information are transmitted from a communication server to a client device.

A transmission method according to an aspect of the present application is a transmission method of encoded data in which a video is hierarchically encoded, including: a generation step of generating an encoded stream including a packet obtained by packing the encoded data, the packet being given a different packet ID according to at least a hierarchical level of the encoded data, and information indicating a correspondence relationship between the packet ID and the hierarchical level; and a transmission step of transmitting information indicating the generated encoded stream and the generated correspondence relationship.

The whole or specific aspects can be realized by a system, an apparatus, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. Further, these overall or specific aspects may be realized by any combination of a system, an apparatus, an integrated circuit, a computer program, and a recording medium.

Drawings

Fig. 1 is a diagram for explaining encoded data that is time-adaptively encoded.

Fig. 2 is a diagram 1 for explaining a data structure of a coded stream in MMT.

Fig. 3 is a diagram 2 for explaining a data structure of a coded stream in MMT.

Fig. 4 is a diagram showing a correspondence relationship between packet IDs and data (resources) in a coded stream according to embodiment 1.

Fig. 5 is a block diagram showing a configuration of a transmission device according to embodiment 1.

Fig. 6 is a flowchart of a transmission method according to embodiment 1.

Fig. 7 is a block diagram showing a configuration of a receiving apparatus according to embodiment 1.

Fig. 8 is a flowchart of a receiving method according to embodiment 1.

Fig. 9 is a diagram conceptually showing a reception method according to embodiment 1.

Fig. 10 is a block diagram showing a configuration of a receiving apparatus according to embodiment 2.

Fig. 11 is a diagram 1 for explaining an outline of a transmission/reception method according to embodiment 2.

Fig. 12 is a diagram 2 for explaining an outline of a transmission/reception method according to embodiment 2.

Fig. 13 is a diagram 1 for explaining an example of packetizing encoded data in MFU units obtained by division.

Fig. 14 is a diagram 2 for explaining an example of packetizing encoded data in MFU units obtained by division.

Fig. 15 is a diagram 3 for explaining an example of packetizing encoded data in MFU units obtained by division.

Fig. 16 is a diagram showing an example in which encoded data is arranged in the original order in MP4 data.

Fig. 17 shows an example 1 in which encoded data is arranged in MP4 data for each hierarchical level.

Fig. 18 is a diagram for explaining example 2 in which encoded data is arranged in MP4 data for each hierarchical level.

Detailed Description

(knowledge forming the basis of the present application)

The Video coding scheme hevc (high Efficiency Video coding) is time-adaptive, and can reproduce a 120fps Video as a 60fps Video, for example. Fig. 1 is a diagram for explaining encoded data resulting from time-adaptive (scalable) encoding.

In time-adaptive coded data, a Temporal ID (time ID) is assigned to each hierarchy. In fig. 1, for example, when a picture with a Temporal ID of 0 (I0, P4) and a picture with a Temporal ID of 1 (B2) are displayed, a video is displayed at 60fps, and when a picture with a Temporal ID of 2 (B1, B3) is additionally displayed, a video is displayed at 120 fps.

In the example of fig. 1, encoded data having a Temporal ID of 0 or 1 is set as a base layer (base hierarchical level), and data having a Temporal ID of 2 is set as an extension layer (extension hierarchical level).

A picture of the base layer can be independently encoded or can be decoded using other pictures of the base layer. In contrast, pictures of the extension layer cannot be decoded alone, and cannot be decoded if the reference picture located at the start point of the arrow in fig. 1 is not decoded. Therefore, a base layer picture, which is a reference picture of a picture of an extension layer, must be decoded first with respect to a picture of the extension layer.

In addition, the decoding order is different from the image display order. In the example of fig. 1, the image display order is (I0, B1, B2, B3, P4), and the decoding order is (I0, P4, B2, B1, B3). The image display order is determined based on a PTS (Presentation Time Stamp) assigned to each picture, and the decoding order is determined based on a DTS (Decode Time Stamp) assigned to each picture.

In the case of dividing a picture into a base layer and an extension layer, not only in the time-adaptive coding but also in the spatial-adaptive coding and SNR-adaptive coding, pictures belonging to the extension layer cannot be decoded alone. Pictures belonging to the extension layer must be decoded in coordination with pictures belonging to the base layer.

Preferably, the encoded data (hierarchically encoded) that can be appropriately encoded can be easily sorted on the receiving side (decoding side).

Therefore, a transmission method according to an aspect of the present application is a transmission method of encoded data in which a video is hierarchically encoded, including: a generation step of generating an encoded stream including a packet obtained by packing the encoded data, the packet being given a different packet ID according to at least a hierarchical level of the encoded data, and information indicating a correspondence relationship between the packet ID and the hierarchical level; and a transmission step of transmitting information indicating the generated encoded stream and the generated correspondence relationship.

In this way, the encoded data can be sorted for each hierarchical level by filtering the packet ID. That is, the encoded data can be easily sorted on the receiving side.

The hierarchical level may include a basic hierarchical level and an extended hierarchical level, the encoded data of the basic hierarchical level may be independently decodable or decodable by referring to decoded data of the other encoded data of the basic hierarchical level, and the encoded data of the extended hierarchical level may be decodable by referring to decoded data of the encoded data of the basic hierarchical level.

In the generating, a 1 st encoded stream and a 2 nd encoded stream may be generated, the 1 st encoded stream being the encoded stream including the packet in which the encoded data of the basic hierarchical level is packetized and not including the packet in which the encoded data of the extended hierarchical level is packetized, the 2 nd encoded stream being the encoded stream including the packet in which the encoded data of the extended hierarchical level is packetized and not including the packet in which the encoded data of the basic hierarchical level is packetized, and the transmitting may be configured to transmit the 1 st encoded stream using a 1 st transmission path and to transmit the 2 nd encoded stream using a 2 nd transmission path different from the 1 st transmission path.

In addition, in the generating step, the 1 st coded stream and the 2 nd coded stream may be generated in different multiplexing schemes.

In the generating step, one of the 1 st coded Stream and the 2 nd coded Stream may be generated according to MPEG2-TS (Moving Picture Experts Group-2 Transport Stream), and the other of the 1 st coded Stream and the 2 nd coded Stream may be generated according to MMT (Moving Picture Experts Group Transport).

One of the 1 st transmission line and the 2 nd transmission line may be a broadcast transmission line, and the other of the 1 st transmission line and the 2 nd transmission line may be a communication transmission line.

In addition, the generating step may generate the coded stream including the information indicating the correspondence relationship, and the transmitting step may transmit the coded stream including the information indicating the correspondence relationship.

The information indicating the correspondence may include one of information indicating that the coded stream can be decoded alone and information indicating another coded stream required for decoding the coded stream.

The whole or specific aspects can be realized by a system, an apparatus, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. Further, these overall or specific aspects may be realized by any combination of a system, an apparatus, an integrated circuit, a computer program, and a recording medium.

The embodiments are described below in detail with reference to the drawings.

The embodiments described below are all embodiments showing general or specific examples. The numerical values, shapes, materials, constituent elements, arrangement positions and connection modes of the constituent elements, steps, order of the steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present application. Among the components of the following embodiments, components not recited in the independent claims representing the uppermost concept will be described as arbitrary components.

(embodiment mode 1)

[ Transmission method ]

A transmission method (transmission apparatus) according to embodiment 1 will be described below with reference to the drawings. In embodiment 1, a transmission method for transmitting encoded data by MMT will be described as an example.

First, a data structure of an encoded stream of MMT is explained. Fig. 2 and 3 are diagrams for explaining the data structure of the coded stream of the MMT.

As shown in fig. 2, the encoded data is composed of a plurality of AUs (Access units). The encoded data is AV data encoded based on a moving picture encoding standard such as HEVC, for example. Specifically, the encoded data includes video data, audio data, metadata attached to these data, still images, files, and the like. When the encoded data is video data, 1 AU is a unit corresponding to 1 picture (1 frame).

In MMT, encoded data is converted into MP4 data (provided with an MP4 header) in a GOP (Group Of Picture) unit in accordance with the file format Of MP 4. The MP4 header included in the MP4 data contains the relative values of presentation time (PTS described above) and decoding time (DTS described above) of the AU. In addition, the MP4 header contains the serial number of the MP4 data. MP4 data (MP4 file) is an example of an MPU (media processing Unit) which is a data Unit defined in the MMT standard.

In the following, the case of transmitting MP4 data (file) will be described as an example, but the transmitted data may not be MP4 data. For example, data in a file format different from the MP4 file may be used, and the encoded data can be decoded on the receiving side by transmitting the encoded data and information necessary for decoding the encoded data (for example, information included in the MP4 header).

Also, as shown in fig. 3, the MMT encoded stream 10 includes program information 11, time offset information 12, and a plurality of MMT packets 13. In other words, the encoded stream 10 is a packet sequence of MMT packets 13.

The coded stream 10(MMT stream) is one of 1 or more streams constituting one MMT packet (package). The MMT package is comparable to, for example, one broadcast program content.

The program information 11 includes information indicating that the encoded stream 10 is a stream that can be encoded (including both the base layer and the extension layer), and information indicating the type of encoding that can be performed and the number of hierarchical levels (number of hierarchical levels). Here, the types of adaptive coding refer to temporal adaptability (scalability), spatial adaptability, SNR adaptability, and the like, and the number of layers refers to the number of layers such as a base layer and an extension layer. The program information 11 need not include all of the above information, and may include at least any one of the above information.

The program information 11 includes, for example, information indicating the correspondence between a plurality of resources and packet IDs. The resource (asset) is a data entity including data having the same transmission characteristics, and is, for example, any of video data, audio data, and the like. The program information 11 may include a descriptor indicating a hierarchical relationship between packet IDs (or resources).

Specifically, the program Information 11 is CI (Composition Information) and MPT (MMT Package Table) in MMT. The Program information 11 includes a PMT (Program Map Table) in MPEG2-TS and an MPD (Media presentation description) in MPEG DASH.

The time offset information 12 is time information for determining the PTS or DTS of the AU. The time offset information 12 is, for example, an absolute PTS or DTS of an AU belonging to the beginning of the base layer.

The MMT packet 13 is data obtained by packaging the MP 4. In embodiment 1, one MMT packet 13 contains one MP4 data (MPU). As shown in fig. 3, the MMT packet 13 includes a header 13a (MTT packet header, MPEG2-TS, TS packet header), and a payload 13 b.

In the payload 13b, MP4 data is stored. In addition, there are cases where data obtained by dividing MP4 is stored in payload 13 b.

The header 13a is accompanying information about the payload 13 b. For example, the header 13a includes a packet ID.

The packet ID is an identification number indicating a resource of data included in the MMT packet 13 (payload 13 b). The packet ID is an identification number unique to each resource constituting the MMT encapsulation.

The feature is that, in the encoded stream 10, the video data of the base layer and the video data of the extension layer are handled as different resources. That is, different packet IDs are assigned to the MMT packets 13 of the coded stream 10 according to the hierarchy level of the stored coded data. Fig. 4 is a diagram showing a correspondence relationship between packet IDs and data (resources) in the encoded stream 10. Fig. 4 shows an example of the correspondence relationship.

As shown in fig. 4, in embodiment 1, a packet ID "1" is given to an MMT packet 13 in which video data of a base layer (encoded data of a base layer level) is packetized. That is, the header 13a describes the packet ID "1". Then, a packet ID "2" is given to the MMT packet 13 in which video data of the extension layer (encoded data of the extension layer level) is packetized. That is, the header 13a describes the packet ID "2".

Similarly, a packet ID "3" is given to the MMT packet 13 in which the audio data is packetized, and a packet ID "4" is given to the MMT packet 13 in which the time offset information 12 is packetized. The MMT packet 13 into which the program information 11 is packetized is given a packet ID "5".

The correspondence as shown in fig. 4 is described in the program information 11 of the encoded stream 10. In the above correspondence, the MMT packet 13 to which the packet ID "1" is given and the MMT packet 13 to which the packet ID "2" is given are paired and include information indicating suitability.

The transmission method (transmission apparatus) of the coded stream 10 according to embodiment 1 as described above will be described. Fig. 5 is a block diagram showing a configuration of a transmission device according to embodiment 1. Fig. 6 is a flowchart of a transmission method according to embodiment 1.

As shown in fig. 5, the transmission device 15 includes an encoding unit 16, a multiplexing unit 17, and a transmission unit 18. The components of the transmission device 15 are realized by a microcomputer, a processor, a dedicated circuit, or the like.

In the transmission method of the bit stream 10 according to embodiment 1, first, the bit stream 10 is generated, and the bit stream 10 includes the MMT packet 13 to which the packet ID is given and information indicating the correspondence relationship between the packet ID and the hierarchy level (S11).

Specifically, when the encoded data output from the encoding unit 16 is packetized, the multiplexing unit 17 determines (selects) the packet ID based on the hierarchy level of the encoded data. Next, the multiplexing unit 17 generates the MMT packet 13 including the determined packet ID. On the other hand, the multiplexing unit 17 generates information indicating the correspondence relationship. The multiplexing unit 17 generates the encoded stream 10, and the encoded stream 10 includes the generated MMT packet 13 and information indicating the generated correspondence relationship.

The generated coded stream 10 is transmitted by the transmitter 18 via the transmission path (S12).

As described above, when the coded stream 10 including the MMT packet 13 to which different packet IDs are given according to the hierarchy level of the coded data is transmitted, the coded data can be easily sorted on the receiving side by using the conventional packet filter mechanism.

The information indicating the correspondence between the packet ID and the hierarchy level may be transmitted separately from the coded stream 10, without being included in the coded stream 10. When the receiving side has already grasped the correspondence between the packet ID and the hierarchy level, the receiving side may not transmit information indicating the correspondence between the packet ID and the hierarchy level.

For example, the information indicating the correspondence relationship may be included in program information repeatedly inserted into a continuous signal such as a broadcast signal, or may be acquired from a communication server before decoding is started.

[ receiving method ]

Hereinafter, a reception method (reception apparatus) according to embodiment 1 will be described with reference to the drawings. Fig. 7 is a block diagram showing a configuration of a receiving apparatus according to embodiment 1. Fig. 8 is a flowchart of a receiving method according to embodiment 1.

In the following description, the base layer is sometimes referred to as a hierarchical level a, and the extension layer is sometimes referred to as a hierarchical level B.

As shown in fig. 7, the reception device 20 includes a packet filter 21, a program information analysis unit 22, a control unit 23, a packet buffer 24, a decoding unit 25, and a presentation unit 26. Among the components of the receiver 20, the components other than the packet buffer 24 and the presentation unit 26 are specifically realized by a microcomputer, a processor, a dedicated circuit, or the like. The packet buffer 24 is a storage device such as a semiconductor memory. The presentation unit 26 is a display device such as a liquid crystal panel.

As shown in fig. 8, first, the packet filter 21 separates the MMT packets 13 included in the encoded stream 10 (S21), and outputs the program information 11 to the program information analysis unit 22. Here, the packet filter 21 recognizes in advance the packet ID of the MMT packet 13 including the program information 11 (or can acquire the packet ID of the MMT packet 13 including the program information 11 from other control information), and therefore can separate the MMT packet 13 including the program information 11 from the coded stream 10.

Next, the program information analysis unit 22 analyzes the program information 11 (S22). As described above, the program information 11 includes the correspondence between the packet ID and the resource.

On the other hand, the control unit 23 determines which level of encoded data (MMT packet 13) is extracted (S23). This determination may be made based on an input of the user accepted by an input accepting unit (not shown in fig. 7), or may be made in accordance with the specification of the presentation unit 26 (for example, the frame rate corresponding to the presentation unit 26).

The packet filter 21 extracts (filters) the encoded data (MMT packet 13) of the determined hierarchical level based on the control of the control unit 23 (S24). The control unit 23 can cause the packet filter 21 to extract the encoded data of the determined hierarchical level because the packet ID of each hierarchical level is recognized by the analysis of the program information analyzing unit 22.

Next, the packet buffer 24 buffers the encoded data extracted by the packet filter 21, and outputs the data to the decoding unit 25 at the time of DTS (S25). The time of the DTS is calculated based on the program information 11, the time offset information 12, and the time information transmitted in the header of the MP4, for example. In addition, when the same DTS is given to the encoded data of the base layer and the encoded data of the extension layer in spatial adaptability or the like, the decoding order may be rearranged so that the encoded data of the base layer is decoded before the encoded data of the extension layer.

The encoded data buffered in the packet buffer 24 is decoded by the decoding unit 25 and presented (displayed) by the presentation unit 26 at the time of the PTS (S26). The time of the PTS is calculated based on the program information 11, the time offset information 12, and time information in the header of the MP4, for example.

Such a receiving method is further explained using fig. 9. Fig. 9 is a diagram conceptually showing a reception method according to embodiment 1.

As shown in fig. 9, for example, when it is determined that the hierarchical level is hierarchical level a (the extraction target is only encoded data of the base layer), the packet filter 21 extracts all MMT packets 13 to which packet ID "1" is assigned, but does not extract MMT packets 13 to which packet ID "2" is assigned. As a result, a video with a low frame rate (for example, 60fps) is displayed by the presentation unit 26.

For example, when it is determined that the hierarchical level is the hierarchical level a + B (both the encoded data of the base layer and the encoded data of the extension layer are extracted as objects of extraction), the packet filter 21 extracts all the MMT packets 13 to which the packet IDs "1" or "2" are given. As a result, the presentation unit 26 displays a video image with a high frame rate (for example, 120 fps).

In this way, the receiving apparatus 20 can easily sort the encoded data of the basic hierarchy level and the encoded data of the extended hierarchy level by the packet filter 21.

(embodiment mode 2)

[ transmitting/receiving method ]

Hereinafter, a transmission method and a reception method (reception apparatus) according to embodiment 2 will be described with reference to the drawings. Fig. 10 is a block diagram showing a configuration of a receiving apparatus according to embodiment 2. Fig. 11 and 12 are diagrams for explaining an outline of a transmission/reception method according to embodiment 2. The block diagram of the transmission device, the reception method, and the flowchart of the transmission method are substantially the same as those described in embodiment 1 except that the hierarchical level ID is used, and therefore, the description thereof is omitted.

As shown in fig. 10, the receiving apparatus 20a according to embodiment 2 is different from the receiving apparatus 20 in that it includes a hierarchical filter 27.

As shown in fig. 11 (1), in the coded stream transmitted by the transmission method according to embodiment 2, the same packet ID is assigned to each of the MMT packet 13 of the base layer and the MMT packet 13 of the extension layer (fig. 10, packet ID: 1).

Further, the MMT packet 13 to which the same packet ID is given a hierarchical level ID, which is an identifier relating to a hierarchical level, independently of the packet ID. In the example of fig. 10, for example, a is assigned to the base layer and B is assigned to the extension layer as the hierarchical level ID. The packet ID and the hierarchy level ID are described in, for example, a header 13a (MMT header) corresponding to the MTT packet.

The hierarchy level ID may be specified as a new identifier, or may be realized by using personal user data (privateuser data) or another identifier.

When the TS header is used, the hierarchical ID may be defined as a new identifier or may be realized using an existing identifier. For example, it is possible to realize a function equivalent to the hierarchy level ID using one or both of the transport priority identifier and the elementary stream priority identifier.

As shown in fig. 11 (2) and fig. 12 (2), the transmitted encoded stream is packet-filtered by the packet filter 21 of the receiving apparatus 20 a. That is, the transmitted encoded stream is subjected to packet filtering based on the packet ID given to the packet header.

As shown in fig. 11 (3) and 12 (3), the MMT packet 13 after packet filtering is further hierarchically filtered by the hierarchical filter 27 based on the hierarchical level ID. The filtered encoded data is once buffered in the packet buffer 24, and then decoded by the decoding unit 25 at the time of DTS. As shown in fig. 11 (4) and 12 (4), the decoded data is presented at the time of the PTS by the presentation unit 26.

Here, in order to obtain a video in which only the base layer is decoded (for example, a video of 60fps), only the MMT packet 13 (encoded data) of the lower layer hierarchy level ID "a" may be decoded. Therefore, in the hierarchical filtering, only MMT packets 13 of the hierarchical level "a" are extracted.

On the other hand, in order to obtain a video (for example, a video of 120fps) in which the base layer and the extension layer are decoded, it is necessary to decode both the MMT packet 13 of the lower layer hierarchical level ID "a" and the MMT packet 13 of the higher layer hierarchical level ID "B". Therefore, in the hierarchical level filtering, both the MMT packet 13 of the hierarchical level ID "a" and the MMT packet 13 of the hierarchical level ID "B" are extracted.

As described above, the receiving method (receiving apparatus 20a) according to embodiment 2 has two systems, one of which is to filter only the MMT packet 13 having the hierarchical level ID "a" and decode and present only the video of the base layer, and the other of which is to filter the MMT packet 13 having the hierarchical level ID "a" or "B" and decode and present the video of the base layer + the extension layer.

In the packet filtering and the hierarchy filtering, what packet ID or hierarchy level ID is to be filtered is determined by considering information on the type and the number of hierarchies that can be encoded described in the program information 11, and what hierarchy encoded data is to be decoded and displayed by the receiving apparatus 20 a.

For example, the reception device 20a performs the above determination according to the processing capability of the reception device 20 a. The transmitting apparatus may also transmit information relating to the capability of the receiving apparatus 20a necessary for decoding and displaying the content as the signaling information, and in such a case, the receiving apparatus 20a may perform the determination by comparing the signaling information with the capability of the receiving apparatus 20 a.

Further, a filter unit may be provided in which the packet filter 21 and the layer filter 27 are integrated into one unit, and the filter unit may perform filtering based on the packet ID and the layer level ID together.

As described above, according to the transmission/reception method according to embodiment 2, encoded data can be sorted for each hierarchical level by filtering hierarchical level IDs. That is, the encoded data can be easily sorted on the receiving side. Further, since the packet ID and the hierarchical level ID are given separately, the encoded data of the base layer and the encoded data of the extension layer can be handled as the same stream in the packet filtering.

Further, by assigning a hierarchical level ID to a packet, desired hierarchical encoded data can be extracted only by a filtering operation without requiring reassembly (reassembling).

Further, by extracting desired hierarchically encoded data by the hierarchical filter, the receiving apparatus corresponding to decoding of only the base layer can reduce the memory for buffering packets of the enhancement layer.

[ example 1]

In the MMT, the MPU constituted by the MP4 data can be divided (Fragment) into MFUs (Media Fragment units), and the MMT packet 13 can be generated by adding a header 13a in units of MFUs. Here, the MFU can be divided into NAL unit units at minimum.

Hereinafter, an example of packetizing encoded data in MFU units divided will be described as specific example 1 of embodiment 2. Fig. 13, 14, and 15 are diagrams for explaining an example of packetizing encoded data in MFU units obtained by division. In fig. 13, 14, and 15, white AUs denote AUs of the base layer, and hatched AUs denote AUs of the extension layer (the same applies to fig. 16 to 18 below).

When the divided MFU is packetized, the same packet ID is assigned to the packet ID of the MMT packet header, and a hierarchical level ID is further assigned to the MMT packet header. In addition, in ' ftyp ', ' moov ', moof ', and the like, an MMT header indicating common data (common information) that is not related to a hierarchical level is given a common ID. In fig. 13, the hierarchy level a: base layer, hierarchy level B: extension level, hierarchy level Z: and sharing information. The base layer and the common information may have the same hierarchical level.

In the case of this structure, the encoded data of hierarchical level B is handled as one resource. The receiving device 20a performs filtering based on the packet ID, and then can perform filtering at a hierarchical level.

When it is desired to decode both the encoded data of the base layer and the encoded data of the extended layer, the receiving device 20a performs filtering based on the packet ID, and then extracts all the hierarchical level IDs by filtering based on the hierarchical level IDs. That is, in the hierarchical level filtering, the hierarchical level a: base layer, hierarchy level B: extension level and hierarchy level Z: all of the information is shared. The extracted data is the data shown in fig. 14.

When it is desired to decode only the encoded data of the base layer, the receiving apparatus 20a performs filtering based on the packet ID, and then performs filtering of the level a: base layer, and hierarchy level Z: and filtering and extracting the common information. The extracted data is the data shown in fig. 15 (a).

At this time, since the AU of the extension layer is removed, the decoding unit 25 acquires the AU of the base layer in a state where the AU is immediately adjacent as shown in fig. 15 (b). However, the time offset information or data size of the sample (AU) described in 'moof' is information generated in a state where an extension layer is included. Therefore, the information described in the header does not match the actual data.

Therefore, it is necessary to store information necessary for reconstructing MP4 data, for example, the size of an AU that has been excluded, offset information, and the like.

Therefore, when acquiring the AU of the base layer or the DTS and PTS, the decoding unit 25 may perform decoding processing assuming that the AU of the extension layer does not exist (is removed by filtering) in the header information such as 'moof' in the 'mdat'.

For example, offset information of an access unit (sample) in 'moof' is set as information that an AU of an extension layer exists. Therefore, when only the base layer is acquired, the decoding unit 25 subtracts the size of the removed AU from the offset information. The data obtained by the subtraction operation is schematically shown in fig. 15 (c).

Similarly, the DTS or PTS is also calculated based on the sample _ duration (difference in DTS between consecutive access units) and the sample _ composition _ time _ offset (difference in DTS and PTS of an access unit) corresponding to the AU of the removed extension layer.

Instead of the subtraction, header information for decoding data obtained by extracting only the base layer (header information for AU of only the base layer) may be written in the MP4 header in advance. The MP4 header may contain information for identifying header information when only the base layer is decoded and header information when both the base layer and the extension layer are decoded.

[ example 2]

Hereinafter, as specific example 2 of embodiment 2, an example in which the MPU is packaged in units of MPU without dividing the MPU will be described.

First, an example in which encoded data is arranged in the original order in MP4 data will be described. Fig. 16 is a diagram showing an example in which encoded data is arranged in the original order in MP4 data (in the case where AUs of different hierarchical levels are multiplexed at the same time).

In the case of configuring encoded data as it is into MP4 data, an AU of a base layer and an AU of an extension layer coexist on one track (track) in an 'mdat' frame (box). In this case, a hierarchy level ID is assigned to each AU. The hierarchy level ID of each AU (sample) is described in moov 'and moof'. Preferably, the AU of the base layer and the AU of the extension layer having a correspondence relationship are arranged in the same 'mdat' frame. When the MP4 data is packetized by attaching a transport header, the same packet ID is given.

In the above configuration, since the packet header cannot be assigned with the hierarchical level ID, filtering cannot be performed in units of packets. Filtering can be performed by parsing the MP4 data.

As another method, there is a method of dividing a base layer and an extension layer of encoded data into tracks and describing a correspondence relationship in a header.

The data thus packetized is packet-filtered at the receiving device 20a, and then is subjected to the determination of the level of the AU hierarchy at the time of MP4 data analysis, thereby extracting and decoding the AU of a desired layer.

Next, description will be given of example 1 in which encoded data is arranged in MP4 data for each hierarchical level. Fig. 17 shows an example 1 in which encoded data is arranged in MP4 data for each hierarchical level.

When the encoded data is arranged in the MP4 data for each hierarchical level, the encoded data is separated for each hierarchical level and arranged in the 'mdat' frame of the fragment for each hierarchical level. In this case, the hierarchy level ID is described in 'moof'. A hierarchy level ID indicating information common to the common header regardless of the hierarchy level is assigned to the common header.

The same packet ID is assigned to the packet header. In this case, the filtering cannot be performed on a packet-by-packet basis.

The data thus packed is subjected to packet filtering in the receiving apparatus 20a, and then the hierarchical level of the fragments is determined at the time of analysis of the MP4 data, and the fragments of a desired layer are extracted and decoded.

Finally, description will be given of example 2 in which encoded data is arranged in MP4 data for each hierarchical level. Fig. 18 is a diagram for explaining example 2 in which encoded data is arranged in MP4 data for each hierarchical level.

In this example, the encoded data is separated for each hierarchical level and arranged in an 'mdat' frame for each hierarchical level.

MP4 data of an AU holding a base layer and MP4 data of an AU holding an extension layer are generated.

The hierarchical level ID is described in either or both of the header and the transport header of the MP4 data. Here, the hierarchical level ID indicates a hierarchical relationship between MP4 data or transfer packets. In addition, the same packet ID is assigned to the packet header.

The data thus packed is packet-filtered in the receiving apparatus 20a, and then the packet of a desired layer is extracted and decoded based on the hierarchical level ID of the packet header.

[ modified examples ]

In embodiment 2, although the packet ID and the hierarchy level ID are given separately, a part of bits of the packet ID may be used to give the hierarchy level ID, or bits may be newly allocated as the extension packet ID. It is equivalent to assigning a hierarchical level ID using a part of bits of the packet ID, and assigning different packet IDs for each hierarchical level based on a rule that the same ID is assigned except for bits indicating the hierarchical level ID.

In embodiment 2, it has been described that, when both of the encoded data of the base layer and the encoded data of the extension layer are decoded, the data of the base layer + the extension layer are extracted by filtering by packet filtering or hierarchical filtering. However, the encoded data may be temporarily divided into a base layer and an extension layer in the hierarchical level filtering and then reconstructed.

(other embodiments)

The present application is not limited to the above-described embodiments.

In embodiments 1 and 2, the above-described encoded streams multiplexed in accordance with the MMT are described. However, the coded stream may be a stream multiplexed in accordance with another multiplexing method such as MPEG2-TS or RTP (real Transport protocol). Further, the MMT packet may be transmitted in accordance with MPEG-TS 2. In either case, the encoded data can be easily sorted at the receiving side.

In embodiments 1 and 2 described above, encoded data of the base layer and encoded data of the extension layer coexist in one encoded stream. However, a 1 st coded stream composed of coded data of the base layer and a 2 nd coded stream composed of coded data of the extension layer may be generated separately. Here, the 1 st encoded stream is, more specifically, an encoded stream including a packet in which encoded data of a base layer is packetized and not including a packet in which encoded data of an extension layer is packetized. The 2 nd encoded stream is an encoded stream including a packet in which the encoded data of the extension layer is packetized, and not including a packet in which the encoded data of the base layer is packetized.

In this case, the 1 st coded stream and the 2 nd coded stream may be generated in different multiplexing schemes. For example, one of the 1 st encoded stream and the 2 nd encoded stream may be generated according to MPEG2-TS, and the other of the 1 st encoded stream and the 2 nd encoded stream may be generated according to MMT.

When two coded streams are generated in different multiplexing schemes, a packet ID or a hierarchy level ID is assigned to each multiplexing scheme. In this case, the correspondence relationship between the packet ID and the hierarchy level or the correspondence relationship between the hierarchy level ID and the hierarchy level in each coded stream is described in the common program information. Specifically, the common program information is included in only one of the two encoded streams, or the common program information is included in both of the two encoded streams.

The receiving device performs packet filtering and hierarchy level filtering based on the correspondence described in the program information, and extracts and decodes encoded data of a desired hierarchy level. That is, one picture is displayed by two encoded streams.

Further, the 1 st coded stream and the 2 nd coded stream may be transmitted using (physically) different transmission paths. Specifically, for example, one of the 1 st coded stream and the 2 nd coded stream may be transmitted using a broadcast transmission line, and the other of the 1 st coded stream and the 2 nd coded stream may be transmitted using a communication transmission line. Such transmission is conceivable, for example, in the case of hierarchical transmission, bulk transmission in which transmission is performed across channels, or the like. In this case, the correspondence between the packet ID and the hierarchy level ID is described in the common program information. The program information may not be common. The receiving device may be able to recognize the correspondence between the packet ID and the hierarchy level or the correspondence between the hierarchy level ID and the hierarchy level.

In addition, although the above embodiments 1 and 2 describe the bit stream composed of two layers, i.e., one base layer and one extension layer, the extension layer may be configured in multiple layers, and the bit stream may be composed of 3 or more hierarchical levels. In this case, different packet IDs (or different hierarchy level IDs) are assigned to the 3 hierarchy levels.

In embodiments 1 and 2, the transmitting device 15 includes the encoding unit 16, but the transmitting device may not have an encoding function. In this case, an encoding device having an encoding function is provided independently of the transmission device 15.

Similarly, in embodiments 1 and 2, the receiving devices 20 and 20a include the decoding unit 25, but the receiving devices 20 and 20a may not have the decoding function. In this case, a decoding device having a decoding function is provided independently of the receiving devices 20 and 20 a.

In embodiments 1 and 2, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading out and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

In embodiment 1, the process executed by a specific processing unit may be executed by another processing unit. Further, the order of the plurality of processes may be changed, and a plurality of processes may be executed in parallel.

The whole or specific aspects of the present application can be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. Further, the whole or specific aspects of the present application may be implemented by any combination of a system, a method, an integrated circuit, a computer program, or a recording medium.

The present application is not limited to these embodiments or modifications thereof. As long as the present invention is not limited to the above-described embodiments, it is possible to provide a liquid crystal display device that can be easily manufactured and manufactured by a simple method.

A 1 st transmitting apparatus as an aspect of the present application transmits data that is time-adaptively encoded. Here, the data that is time-adaptively encoded includes data of a base layer that can be decoded using data contained in the layer, and data of an extension layer that cannot be decoded alone but must be decoded in cooperation with data of the base layer. Here, the data of the base layer is, for example, data for decoding a 60p video, and the data of the extension layer is, for example, data for decoding a 120p video by being combined with the data of the base layer.

The data of the base layer is transmitted as a 1 st resource to which a 1 st packet ID is given, and the data of the extension layer is transmitted as a 2 nd resource to which a 2 nd packet ID is given. The packet ID is described in the header of the packet in which data is stored. The transmission device multiplexes and transmits the data of the base layer, the data of the extension layer, and the program information. Here, the program information 11 may include, for example, identifiers indicating the hierarchical relationship of the respective package IDs (or resources corresponding to the respective package IDs). Here, the information indicating the hierarchical relationship includes, for example: information indicating that the 1 st packet ID (1 st resource) can be decoded by itself, and information indicating that the 2 nd packet ID (2 nd resource) cannot be decoded by itself and needs to be decoded using the 1 st packet ID (1 st resource) data.

Further, the program information may include: the stream constituting the program is at least one of information indicating that the stream is an adaptively-coded stream (a stream including both a base layer and an extension layer), information indicating a type of an adaptively-coded stream, information indicating the number of hierarchical layers, and information on hierarchical levels.

Further, the 1 st receiving apparatus of an aspect of the present application receives data that is time-adaptively encoded. Here, the data that is time-adaptively encoded includes data of a base layer that can be decoded using data contained in the layer, and data of an extension layer that cannot be decoded alone but must be decoded in cooperation with data of the base layer. Here, the data of the base layer is, for example, data for decoding a 60p video, and the data of the extension layer is, for example, data for decoding a 120p video by being combined with the data of the base layer.

The data of the base layer is transmitted as a 1 st resource to which a 1 st packet ID is given, and the data of the extension layer is transmitted as a 2 nd resource to which a 2 nd packet ID is given. The packet ID is described in the header of the packet in which data is stored. The transmission device multiplexes and transmits the data of the base layer, the data of the extension layer, and the program information. Here, the program information 11 may include, for example, identifiers indicating the hierarchical relationship of the respective package IDs (or resources corresponding to the respective package IDs). Here, the information indicating the hierarchical relationship includes, for example: information indicating that the 1 st packet ID (1 st resource) can be decoded by itself, and information indicating that the 2 nd packet ID (2 nd resource) cannot be decoded by itself and needs to be decoded using the 1 st packet ID (1 st resource) data.

Further, the program information may include: the stream constituting the program is at least one of information indicating a stream that can be coded appropriately (a stream including both a base layer and an extension layer), information indicating a type that can be coded appropriately, information indicating the number of hierarchical layers, and information related to hierarchical levels.

According to the 1 st transmitting device and the 2 nd receiving device described above, it is possible to acquire the packet ID (resource) necessary for decoding the data of the selected packet ID (resource) on the receiving side from the program information, and perform filtering based on the packet ID (resource). For example, when the 1 st resource to which the 1 st packet ID is given is reproduced, the data of the 1 st packet ID (1 st resource) can be decoded individually, and therefore the data of the 1 st packet ID (1 st resource) is acquired by filtering. On the other hand, when the 2 nd resource to which the 2 nd packet ID is given is reproduced, since the data of the 2 nd packet ID (2 nd resource) cannot be decoded alone and needs to be decoded using the data of the 1 st packet ID (1 st resource), the data of the 1 st packet ID (1 st resource) and the data of the 2 nd packet ID (2 nd resource) are acquired by filtering.

In the above configuration, the program information includes information indicating data of packet IDs (resources) other than the packet ID (resource) necessary for decoding the data of each packet ID (resource). According to this configuration, even when the data that can be encoded appropriately includes 3 layers, i.e., layer a, layer B decoded in cooperation with layer a, and layer C decoded in cooperation with layer a, the receiving apparatus can specify the packet ID (resource) to which the data necessary for decoding is transmitted without making a complicated determination when the layer to be reproduced is selected.

In particular, the above configuration is useful in which the packet ID (resource) for transmitting data necessary for decoding can be determined without making a complicated determination, considering that the depth of the hierarchy may be 3 or more, or data encoded by a plurality of kinds of suitable codes may be multiplexed and transmitted in the future.

Industrial applicability

The present invention is applicable to television broadcasting, moving image distribution, and the like as a transmission method of encoded data that enables a receiving side to easily sort hierarchically encoded data.

Description of the reference symbols

10 coded stream

11 program information

12 time offset information

13 MMT bag

13a head

13b payload

15 transmitting device

16 coding unit

17 multiplexing part

18 sending part

20, 20a receiver

21 pack filter

22 program information analysis unit

23 control part

24-pack buffer

25 decoding unit

26 presentation part