阅读说明：本技术 视频数据的处理，传输方法及装置，视频数据的处理系统 (Video data processing method, video data transmission method, video data processing system, video data transmission device and video data transmission device ) 是由 黄成� 于 2018-08-29 设计创作，主要内容包括：本发明提供了一种视频数据的处理,传输方法及装置,视频数据的处理系统。具体而言,视频数据的处理方法,包括：播放从第一视点切换至第二视点的全方向视频轨道；根据所述第二视点的视点信息和/或自所述第一视点切换前用户的观看方向,确定播放所述全方向视频轨道时用户的初始观看方向。通过本发明,解决了用户会由于视点切换前后视窗内容的突然变化而感到不适的问题,达到了提高用户观看体验效果。(The invention provides a video data processing method, a video data transmission method, a video data processing device, a video data transmission device and a video data processing system. Specifically, the video data processing method comprises the following steps: playing an omnidirectional video track switched from a first viewpoint to a second viewpoint; and determining the initial viewing direction of the user when the omnidirectional video track is played according to the viewpoint information of the second viewpoint and/or the viewing direction of the user before switching from the first viewpoint. By the method and the device, the problem that a user feels uncomfortable due to sudden change of the window content before and after viewpoint switching is solved, and the effect of improving the watching experience of the user is achieved.)

1. A method for processing video data, comprising:

playing an omnidirectional video track switched from a first viewpoint to a second viewpoint;

and determining the initial viewing direction of the user when the omnidirectional video track is played according to the viewpoint information of the second viewpoint and/or the viewing direction of the user before switching from the first viewpoint.

2. The method of claim 1, wherein the viewpoint information of the second viewpoint is determined according to a viewpoint group data box in the omnidirectional video track.

3. The method of claim 2, further comprising:

identifying the viewpoint group data box in the omnidirectional video track according to a specified track group type; wherein the view group data boxes of the omnidirectional video tracks corresponding to the same view point have the same track group identification.

4. A method according to any of claims 1 to 3, wherein the view information for the second view is determined from a dynamic view timing metadata track.

5. The method of claim 4, further comprising:

identifying the dynamic view timing metadata track according to a first sample entry type, the dynamic view timing metadata track to indicate dynamically changing view information of the omnidirectional video track.

6. The method of any one of claims 1 to 5, wherein the view information of the second view comprises at least one of: the 3D spatial position of the viewpoint, the GPS position of the viewpoint, the global coordinate system rotation direction of the viewpoint, the group of viewpoints, the identification of the viewpoint, the label of the viewpoint, and the type of the viewpoint.

7. The method of claim 1, further comprising:

when an initial viewing direction exists, playing the omnidirectional video track according to the initial viewing direction; and otherwise, playing the omnidirectional video track according to the watching direction of the user before switching from the first viewpoint.

8. The method of claim 7, further comprising:

identifying an initial viewing direction timing metadata track according to a second sample entry type, the initial viewing direction timing metadata track being used to indicate an initial viewing direction when playing the omnidirectional video track.

9. The method according to claim 6 or 7, characterized in that: and determining the rotation direction of the global coordinate system of the second viewpoint according to the rotation angle of the coordinate axis of the global coordinate system of the second viewpoint relative to the coordinate axis of the common reference coordinate system.

10. The method of claim 9, wherein determining an initial viewing direction of a user when the omnidirectional video track is played according to the viewpoint information of the second viewpoint and/or the viewing direction of the user before switching from the first viewpoint comprises:

determining an initial viewing direction of a user when the omnidirectional video track is played in the global coordinate system of the second viewpoint according to the global coordinate system rotation direction of the second viewpoint;

in a common reference coordinate system, the initial viewing direction of the user when the omnidirectional video track is played is consistent with the viewing direction of the user before switching from the first viewpoint.

11. The method of claim 1, wherein playing an omnidirectional video track that switches from the first viewpoint to the second viewpoint further comprises:

determining the second viewpoint according to the initial viewpoint identification in the initial viewpoint timing metadata track; or, the view with the minimum view identification value in all the views after view switching is selected as the second view.

12. The method of claim 11, wherein: identifying the initial view timing metadata track according to a third sample entry type, the initial view timing metadata track to indicate an initial view identification.

13. A method for transmitting video data, comprising:

receiving a media presentation description file, wherein the media presentation description file comprises a viewpoint descriptor for indicating viewpoint information of an omnidirectional video;

selecting a viewpoint for playing the omnidirectional video, and acquiring an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request;

and the viewpoint information of the viewpoint and/or the viewing direction of the user before switching from the viewpoint are used for determining the initial viewing direction of the user when the omnidirectional video track is played.

14. The method of claim 13, wherein obtaining an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request comprises:

and when the selected viewpoint is a dynamic viewpoint, requesting to acquire a dynamic viewpoint timing metadata file associated with the omnidirectional video file corresponding to the viewpoint.

15. The method of claim 13, wherein obtaining an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request further comprises:

determining a viewpoint group for playing the omnidirectional video;

when an initial viewpoint exists in the viewpoint group, requesting to acquire an omnidirectional video file corresponding to the initial viewpoint; and when the initial viewpoint does not exist in the viewpoint group, requesting to acquire the omnidirectional video file with the minimum viewpoint identification value in the viewpoint group.

16. The method of claim 14, further comprising:

judging whether an initial viewpoint exists in the viewpoint group or not according to the initial viewpoint flag bit and the viewpoint group identifier carried by the viewpoint descriptor;

or requesting to acquire initial viewpoint timing metadata files associated with all viewpoints;

and judging whether the initial viewpoint exists in the viewpoint group or not according to the initial viewpoint identifier indicated in the initial viewpoint timing metadata file, the initial viewpoint flag bit and the viewpoint group identifier.

17. The method according to any one of claims 13 to 16,

identifying the viewpoint descriptor according to the URI attribute in the media presentation description file, wherein the viewpoint information of the omnidirectional video indicated by the viewpoint descriptor at least comprises one of the following information: the 3D spatial position of the viewpoint, the GPS position of the viewpoint, the global coordinate system rotation direction of the viewpoint, the group of viewpoints, the identification of the viewpoint, the label of the viewpoint, and the type of the viewpoint.

18. An apparatus for processing video data, comprising:

the playing module is used for playing the omnidirectional video track switched from the first viewpoint to the second viewpoint;

and the determining module is used for determining the initial viewing direction of the user when the omnidirectional video track is played according to the viewpoint information of the second viewpoint and/or the viewing direction of the user before switching from the first viewpoint.

19. The apparatus of claim 18, wherein the determining module is further configured to play the omnidirectional video track in an initial viewing direction when the initial viewing direction exists; and otherwise, playing the omnidirectional video track according to the watching direction of the user before switching from the first viewpoint.

20. The apparatus of claim 19, wherein: the determining module is further configured to determine a global coordinate system rotation direction of the second viewpoint according to a rotation angle of a coordinate axis of the global coordinate system of the second viewpoint relative to a coordinate axis of a common reference coordinate system.

21. The apparatus of claim 20, wherein the determining module is further configured to: determining an initial viewing direction of a user when the omnidirectional video track is played in the global coordinate system of the second viewpoint according to the global coordinate system rotation direction of the second viewpoint; in a common reference coordinate system, the initial viewing direction of the user when the omnidirectional video track is played is consistent with the viewing direction of the user before switching from the first viewpoint.

22. A transmission apparatus of video data, comprising:

a receiving module, configured to receive a media presentation description file, where the media presentation description file includes a viewpoint descriptor for indicating viewpoint information of an omnidirectional video;

the request module is used for selecting a viewpoint for playing the omnidirectional video and acquiring an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request;

and the viewpoint information of the viewpoint and/or the viewing direction of the user before switching from the viewpoint are used for determining the initial viewing direction of the user when the omnidirectional video track is played.

23. The method of claim 22, wherein the requesting module is further configured to request to obtain a dynamic viewpoint timing metadata file associated with an omnidirectional video file corresponding to the viewpoint when the selected viewpoint is a dynamic viewpoint.

24. The method of claim 23, wherein the requesting module is further configured to determine a group of viewpoints for playing the omnidirectional video; when an initial viewpoint exists in the viewpoint group, requesting to acquire an omnidirectional video file corresponding to the initial viewpoint; and when the initial viewpoint does not exist in the viewpoint group, requesting to acquire the omnidirectional video file with the minimum viewpoint identification value in the viewpoint group.

25. A system for processing video data, comprising: the system comprises an omnidirectional video data processing server and an omnidirectional video data processing terminal;

the omnidirectional video data processing server is used for making the omnidirectional video collected by a plurality of viewpoints, generating viewpoint information of the omnidirectional video, and encapsulating the viewpoint information of the omnidirectional video track in one or a plurality of media files or one or a plurality of index information;

the omnidirectional video data processing terminal is used for transmitting the omnidirectional video, decapsulating the omnidirectional video, extracting viewpoint information of a plurality of viewpoints and playing the processed omnidirectional video.

26. The processing system of claim 25,

the media files include at least one of: the omnidirectional video track, the timing metadata track; the index information at least includes: a media presentation description file.

27. A storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the method of any of claims 1 to 12, 13 to 17 when executed.

28. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and the processor is configured to execute the computer program to perform the method of any one of claims 1 to 12 and 13 to 17.

Technical Field

The invention relates to the field of communication, in particular to a video data processing and transmitting method and device and a video data processing system.

Background

VR (Virtual Reality) refers to the simulation of a three-dimensional Virtual world by using computer technology, so that a user experiences a highly realistic Virtual space environment in the aspects of vision, hearing, touch, and the like. With the rapid development of VR technology, VR videos (also called omnidirectional videos) are applied more and more widely on demand and live broadcast, and immersive video service experience is brought to users.

For the same scene, multiple omnidirectional videos can be usually captured at different physical locations (viewpoints), which is called a multi-viewpoint omnidirectional video. The multi-view omni-directional video allows a user to switch between a plurality of omni-directional videos based on location information of different viewpoints. However, since there is overlapping content between all-directional videos photographed from consecutive viewpoints, if there is no correlation between the content of a user window before viewpoint switching and the content of an initial window after viewpoint switching, the user feels discomfort due to a sudden change in the content of the window before and after viewpoint switching.

In view of the above problems in the related art, no effective solution has been found at present.

Disclosure of Invention

Embodiments of the present invention provide a method and an apparatus for processing and transmitting video data, and a system for processing video data, so as to at least solve a problem that a user feels uncomfortable due to a sudden change of window content before and after viewpoint switching in the related art.

According to an embodiment of the present invention, there is provided a video data processing method including: playing an omnidirectional video track switched from a first viewpoint to a second viewpoint; and determining the initial viewing direction of the user when the omnidirectional video track is played according to the viewpoint information of the second viewpoint and/or the viewing direction of the user before switching from the first viewpoint.

Optionally, the viewpoint information of the second viewpoint is determined according to a viewpoint group data box in the omnidirectional video track.

Optionally, identifying the view group data box in the omnidirectional video track according to a specified track group type; wherein the view group data boxes of the omnidirectional video tracks corresponding to the same view point have the same track group identification.

Optionally, the view information of the second view is determined according to a dynamic view timing metadata track.

Optionally, the dynamic view timing metadata track is identified according to a first sample entry type, the dynamic view timing metadata track being used to indicate dynamically changing view information of the omnidirectional video track.

Optionally, the view information of the second view comprises at least one of: the 3D spatial position of the viewpoint, the GPS position of the viewpoint, the global coordinate system rotation direction of the viewpoint, the group of viewpoints, the identification of the viewpoint, the label of the viewpoint, and the type of the viewpoint.

Optionally, when there is an initial viewing direction, playing the all-direction video track according to the initial viewing direction; otherwise, the omnidirectional video track is played according to the watching direction of the user before switching from the first viewpoint.

Optionally, the global coordinate system rotation direction of the second viewpoint is determined according to a rotation angle of a coordinate axis of the global coordinate system of the second viewpoint relative to a coordinate axis of a common reference coordinate system.

Optionally, an initial viewing direction timing metadata track is identified according to a second sample entry type, the initial viewing direction timing metadata track being used to indicate an initial viewing direction when the omnidirectional video track is played.

Optionally, determining an initial viewing direction of the user when the omnidirectional video track is played according to the viewpoint information of the second viewpoint and/or the viewing direction of the user before switching from the first viewpoint includes: determining an initial viewing direction of a user when the omnidirectional video track is played in the global coordinate system of the second viewpoint according to the global coordinate system rotation direction of the second viewpoint; in the common reference coordinate system, the initial viewing direction of the user when the omnidirectional video track is played is consistent with the viewing direction of the user before the switching from the first viewpoint.

Optionally, playing an omnidirectional video track switched from the first viewpoint to the second viewpoint further comprises: determining the second viewpoint according to the initial viewpoint identifier in the initial viewpoint timing metadata track; or, the view with the minimum view identification value in all the views after view switching is selected as the second view.

Optionally, the initial viewpoint timing metadata track is identified according to a third sample entry type, the initial viewpoint timing metadata track indicating an initial viewpoint identification.

According to another embodiment of the present invention, there is provided a video data transmission method including: receiving a media presentation description file, wherein the media presentation description file comprises a viewpoint descriptor for indicating viewpoint information of a full-direction video; selecting a viewpoint for playing the omnidirectional video, and acquiring an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request; wherein the viewpoints comprise: and the viewpoint information of the viewpoint and/or the viewing direction of the user before switching the viewpoint are used for determining the initial viewing direction of the user when the omnidirectional video track is played.

Optionally, acquiring an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request includes: and when the selected viewpoint is a dynamic viewpoint, requesting to acquire a dynamic viewpoint timing metadata file associated with the all-direction video file corresponding to the viewpoint.

Optionally, acquiring an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request, further includes: determining a viewpoint group for playing the omnidirectional video; when an initial viewpoint exists in the viewpoint group, requesting to acquire an omnidirectional video file corresponding to the initial viewpoint; and when the initial viewpoint does not exist in the viewpoint group, requesting to acquire the omnidirectional video file with the minimum viewpoint identification value in the viewpoint group.

Optionally, according to the initial viewpoint flag bit and the viewpoint group identifier carried by the viewpoint descriptor, determining whether an initial viewpoint exists in the viewpoint group or requesting to acquire an initial viewpoint timing metadata file associated with all viewpoints; and judging whether an initial viewpoint exists in the viewpoint group or not according to the initial viewpoint identifier indicated in the initial viewpoint timing metadata file, the initial viewpoint flag bit and the viewpoint group identifier.

Optionally, the viewpoint descriptor is identified according to a URI attribute in the media presentation description file, and the viewpoint information of the omnidirectional video indicated by the viewpoint descriptor includes at least one of the following: the 3D spatial position of the viewpoint, the GPS position of the viewpoint, the global coordinate system rotation direction of the viewpoint, the group of viewpoints, the identification of the viewpoint, the label of the viewpoint, and the type of the viewpoint.

According to another embodiment of the present invention, there is provided a video data processing apparatus including: the playing module is used for playing the omnidirectional video track switched from the first viewpoint to the second viewpoint; and the determining module is used for determining the initial viewing direction of the user when the omnidirectional video track is played according to the viewpoint information of the second viewpoint and/or the viewing direction of the user before switching from the first viewpoint.

Optionally, the determining module is further configured to, when there is an initial viewing direction, play the omnidirectional video track according to the initial viewing direction; and otherwise, playing the omnidirectional video track according to the watching direction of the user before switching from the first viewpoint.

Optionally, the determining module is further configured to determine a global coordinate system rotation direction of the second viewpoint according to a rotation angle of a coordinate axis of the global coordinate system of the second viewpoint relative to a common reference coordinate system coordinate axis.

Optionally, the determining module is further configured to: determining an initial viewing direction of a user when the omnidirectional video track is played in the global coordinate system of the second viewpoint according to the global coordinate system rotation direction of the second viewpoint; in a common reference coordinate system, the initial viewing direction of the user when the omnidirectional video track is played is consistent with the viewing direction of the user before switching from the first viewpoint.

According to another embodiment of the present invention, there is provided a transmission apparatus of video data, including: a receiving module, configured to receive a media presentation description file, where the media presentation description file includes a viewpoint descriptor for indicating viewpoint information of an omnidirectional video; the request module is used for selecting a viewpoint for playing the omnidirectional video and acquiring an omnidirectional video file corresponding to the viewpoint according to the viewpoint descriptor request; wherein the viewpoint includes: and the viewpoint information of the viewpoint and/or the viewing direction of the user before switching the viewpoint are used for determining the initial viewing direction of the user when the omnidirectional video track is played.

Optionally, the requesting module is further configured to request to obtain a dynamic viewpoint timing metadata file associated with an omnidirectional video file corresponding to the viewpoint when the selected viewpoint is a dynamic viewpoint.

Optionally, the request module is further configured to determine a viewpoint group for playing the omnidirectional video; when an initial viewpoint exists in the viewpoint group, requesting to acquire an omnidirectional video file corresponding to the initial viewpoint; and when the initial viewpoint does not exist in the viewpoint group, requesting to acquire the omnidirectional video file with the minimum viewpoint identification value in the viewpoint group.

According to another embodiment of the present invention, there is provided a video data processing system including: a directional video data processing server and an omnidirectional video data processing terminal applied to the method; the omnidirectional video data processing server is used for making the omnidirectional video collected by a plurality of viewpoints, generating viewpoint information of the omnidirectional video, and encapsulating the viewpoint information of the omnidirectional video track in one or a plurality of media files or one or a plurality of index information; the omnidirectional video data processing terminal is used for transmitting the omnidirectional video, decapsulating the omnidirectional video, extracting viewpoint information of a plurality of viewpoints and playing the processed omnidirectional video.

Optionally, the media file includes at least one of: the omnidirectional video track, the timing metadata track; the index information at least includes: a media presentation description file.

According to a further embodiment of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when executed.

According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.

According to the invention, the initial viewing direction of the user is determined according to the switched viewpoint information and/or the switched viewpoint direction, so that the problem that the user feels uncomfortable due to the sudden change of the window content before and after viewpoint switching in the related technology can be solved, and the effect of improving the viewing experience of the user is achieved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention to a proper form. In the drawings:

fig. 1 is a block diagram of a hardware structure of a mobile terminal according to a video data processing method of an embodiment of the present invention;

fig. 2is a flowchart of a method of processing video data according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of view group data boxes in a projected omnidirectional video track according to an embodiment of the present invention;

fig. 4 is a diagram illustrating a dynamic viewpoint timing metadata track referencing a group of viewpoint tracks of an omnidirectional video, according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of another video data processing method according to an embodiment of the invention

FIG. 6 is a schematic illustration of a common reference frame according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating a further method for processing video data according to an embodiment of the present invention

Fig. 8 is a flow chart of a video data transmission method according to an embodiment of the present invention;

fig. 9 is a flowchart of an omni-directional video data transmission method according to an embodiment of the present invention;

fig. 10 is a flowchart of a method of processing video data according to an embodiment of the present invention;

fig. 11 is a block diagram of a video data processing apparatus according to an embodiment of the present invention;

fig. 12is a block diagram of a video data transmission apparatus according to an embodiment of the present invention;

fig. 13 is a block diagram of a video data processing system according to an embodiment of the present invention;

fig. 14 is a block diagram of another video data processing system according to an embodiment of the present invention.

Detailed Description

The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.