阅读说明：本技术 媒体处理和流传输的方法、系统以及存储介质 (Method, system and storage medium for media processing and streaming ) 是由 索达加伊拉吉 于 2020-10-14 设计创作，主要内容包括：本公开实施例提供了一种媒体处理和流传输的方法、系统以及存储介质。该媒体处理和流传输的方法,由基于网络的媒体处理NBMP源执行,包括发送第一请求,以在上行链路实时流框架FLUS源与FLUS接收端之间建立FLUS会话；在FLUS会话建立后,从FLUS源接收NBMP工作流管理器的地址；以及通过NBMP工作流管理器的地址对应的路径,向所述NBMP工作流管理器发送第二请求,以创建或更新用于处理媒体内容的工作流,所述路径不包括所述FLUS源和所述FLUS接收端。(The disclosed embodiments provide a method, system and storage medium for media processing and streaming. The method of media processing and streaming, performed by a network-based media processing NBMP source, comprises sending a first request to establish a FLUS session between an uplink real-time streaming framework FLUS source and a FLUS sink; receiving an address of an NBMP workflow manager from the FLUS source after the FLUS session is established; and sending a second request to an NBMP workflow manager to create or update a workflow for processing media content through a path corresponding to an address of the NBMP workflow manager, the path excluding the plus source and the plus receiver.)

1. A method of media processing and streaming, performed by a network-based media processing NBMP source, comprising:

sending a first request to a control source to establish a FLUS session between an uplink real-time streaming Framework (FLUS) source and a FLUS receiving end;

receiving an address of an NBMP workflow manager from the uplink real-time streaming Framework (FLUS) source after the FLUS session establishment; and

sending a second request to the NBMP workflow manager to create or update a workflow for processing media content through a path corresponding to an address of the NBMP workflow manager, the path excluding the uplink real-time streaming framework FLUS source and the FLUS receiver.

2. The method of claim 1, wherein the second request comprises a request to create the workflow.

3. The method of claim 1, wherein the second request comprises a request to update the workflow.

4. The method of claim 3, further comprising:

transmitting a second request to the uplink real-time flow framework, FLUS, source instructing the NBMP workflow manager to create the workflow, wherein the second request comprises a workflow description document, WDD.

5. The method of claim 4, further comprising:

receiving, by the FLUS receiver and the uplink real-time streaming framework FLUS source, an updated version of the workflow description document WDD from the NBMP workflow manager prior to sending the second request to update the workflow.

6. The method according to claim 4, characterized in that after said receiving the address of the NBMP workflow manager from the uplink real time streaming framework FLUS source, sending the second request to create the workflow.

7. The method of claim 1, further comprising:

receiving a report or notification from the NBMP workflow manager through the uplink real-time flow framework FLUS source and the FLUS receiver.

8. The method of claim 7, wherein the receiving the report or notification from the NBMP workflow manager comprises: receiving, by the uplink real-time flow framework, FLUS source and the FLUS receiver, a descriptor from the NBMP workflow manager, the descriptor including a parameter or event indicative of the report or notification.

9. The method of claim 1, wherein the address is a Uniform Resource Locator (URL).

10. The method of claim 9, wherein the path is a subzone path.

11. A system for media processing and streaming, comprising:

the system comprises a session request module, a control source and a server, wherein the session request module is used for sending a first request to the control source so as to establish a FLUS session between an uplink real-time streaming framework FLUS source and a FLUS receiving end; and

a workflow request module, configured to send a second request to an NBMP workflow manager through a path corresponding to an address of the NBMP workflow manager after receiving the address from the uplink real-time streaming framework plus source after the plus session is established, so as to create or update a workflow for processing media content, where the path does not include the uplink real-time streaming framework plus source and the plus receiver.

12. The system of claim 11, wherein the workflow request module comprises a workflow creation request submodule and the second request comprises a request to create the workflow.

13. The system of claim 11, wherein the workflow request module comprises a workflow update request submodule and the second request comprises a request to update the workflow.

14. An electronic device comprising a processor and a memory; the memory stores a computer program which, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 10.

15. A non-transitory computer readable medium, wherein the storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the method of any of claims 1 to 10.

Technical Field

Embodiments of the present disclosure relate to methods and systems for Media Processing and Streaming, and more particularly, to methods and systems for Network-Based Media Processing (NBMP) and Framework for Uplink Live Streaming (plus) for Moving Picture Experts Group (MPEG).

Background

The Moving Picture Experts Group (MPEG) network-based media processing (NBMP) project developed the concept of processing media on the cloud. However, current NBMP designs do not provide an explicit monitoring method. The entire contents of ISO/IEC DIS 23090-8 text for network-based media processing, ISO/IEC JTC 1/SC 29/WG 11 (N18657), dated 12.2019, are incorporated herein by reference.

The 3rd Generation Partnership Project (3 GPP) Framework for Uplink Live Streaming (plus) protocol provides a mechanism for Uplink Streaming of multimedia content from a source device to a network and sending/distributing the content to one or more target sites. "3 rd generation partnership project; technical specification group services and system aspects; the entire contents of the uplink flow (Release 16) of 3GPP TS 26.238V16.2.0 dated september 2019 are incorporated by reference in the present application.

Disclosure of Invention

In the NBMP standard, an NBMP source is an entity that provides workflow descriptions to a workflow manager to create, run, manage, and monitor media workflows. The interaction between the NBMP source and the workflow manager is through a set of NBMP operational APIs.

In the case of the 3GPP plus protocol, a source device of a media stream establishes an uplink session with a receiving end (sink) over a network. The FLUS API allows the source device to control the session and also allows the sink to provide feedback or remote control of the source device.

Current 3GPP FLUS protocol support includes NBMP Workflow Description Document (WDD) as part of the session control update of the source device. However, after workflow establishment, the protocol does not include interaction between the source device and the sink for management of NBMP workflows. Furthermore, the protocol does not provide an architecture for the NBMP workflow manager and/or tasks for providing reports and notifications to the source device so that it can receive feedback from the running NBMP workflow to be able to dynamically manage and modify the workflow.

Embodiments of the present disclosure address the above matters, and others.

Embodiments of the present disclosure provide for cloud-based workflow processing of uplink flows, wherein the cloud-based workflow processing has a control located at a client device of an uplink source. Embodiments of the present disclosure provide an architecture in which the source client is an NBMP source that directly manages the workflow after the FLUS session is established.

In accordance with one or more embodiments, a method of media processing and streaming is provided. The method comprises the following steps: the method comprises the steps that a network-based media processing NBMP source sends a first request to establish a FLUS session between an uplink real-time streaming Framework (FLUS) source and a FLUS receiving end; receiving, by the network-based media processing NBMP source, an address of an NBMP workflow manager from the uplink real-time streaming framework FLUS source after a FLUS session establishment; and sending, by the network-based media processing NBMP source, a second request to an NBMP workflow manager through a path of addresses of the NBMP workflow manager to create or update a workflow to process media content, the NBMP workflow manager not including the uplink real-time streaming framework plus source and the plus receiver.

According to an embodiment, the second request comprises a request to create a workflow.

According to an embodiment, the second request comprises a request to update the workflow.

According to an embodiment, the method further comprises: sending, by the network-based media processing NBMP source to the uplink real-time streaming framework FLUS source, a second request for instructing an NBMP workflow manager to create a workflow, wherein the second request comprises a workflow description document, WDD.

According to an embodiment, the method further comprises: receiving, by the network-based media processing NBMP source, an updated version of the workflow description document WDD from a NBMP workflow manager via a plus sink and the uplink live streaming framework plus source prior to sending the second request to update a workflow.

According to an embodiment, after said receiving said address of said NBMP workflow manager, sending said second request to create said workflow.

According to an embodiment, the method further comprises: receiving a report or notification from an NBMP workflow manager through the uplink real-time streaming framework FLUS source and FLUS receiver.

According to an embodiment, receiving the report or notification comprises receiving a descriptor from the NBMP workflow manager, the descriptor comprising a parameter or event indicative of the report or notification, through said uplink real-time streaming framework FLUS source and FLUS receiver.

According to an embodiment, the address is a Uniform Resource Locator (URL).

According to an embodiment, the path is a subdomain path.

In accordance with one or more embodiments, a system for media processing and streaming is provided. The system comprises: the session request module is used for enabling the NBMP source based on the network to send a first request so as to establish a FLUS session between the FLUS source and the FLUS receiving end of the uplink real-time streaming framework; and a workflow request module for sending a second request to the NBMP workflow manager through a path of addresses of the NBMP workflow manager to create or update a workflow for processing media content after the network-based media processing NBMP source receives an address from the uplink real-time streaming framework plus source after the plus session is established, the NBMP workflow manager not including the uplink real-time streaming framework plus source and the plus receiver.

According to an embodiment, the workflow request module includes a workflow creation request submodule and the second request includes a request to create a workflow.

According to an embodiment, the workflow request module includes a workflow update request submodule and the second request includes a request to update the workflow.

According to an embodiment, the system further comprises: a workflow creation request submodule for causing the network-based media processing NBMP source to send a second request to the uplink live streaming framework FLUS source instructing the NBMP workflow manager to create a workflow, wherein the second request comprises a workflow description document, WDD.

According to an embodiment, the workflow update request submodule is adapted to cause the network-based media processing NBMP source to send the second request for instructing the NBMP workflow manager to update the workflow after the NBMP source receives the updated version of the WDD.

According to an embodiment, said workflow creation request sub-module is configured to cause said network-based media processing NBMP source to send said second request for instructing said NBMP workflow manager to create said workflow, after said network-based media processing NBMP source receives said address of said NBMP workflow manager.

According to an embodiment, the network-based media processing NBMP source is adapted to receive reports or notifications from the NBMP workflow manager through the uplink real time streaming framework FLUS source and the FLUS receiver.

According to an embodiment, the network-based media processing NBMP source is configured to receive a descriptor from the NBMP workflow manager, the descriptor comprising a parameter or event indicative of the report or notification, over the uplink real-time streaming framework FLUS source and the FLUS receiver.

According to an embodiment, the address is a uniform resource locator, URL.

In accordance with one or more embodiments, there is provided an electronic device comprising a processor and a memory; the memory stores a computer program that, when executed by the processor, causes the processor to perform the method of embodiments of the disclosure.

According to one or more embodiments, a non-transitory computer-readable medium is provided, which stores computer instructions that, when executed on a computer, cause the computer to perform the method of an embodiment of the present disclosure.

By the method for media processing and streaming provided by the embodiment of the disclosure, a network-based media processing NBMP source sends a first request to establish a FLUS session between an uplink real-time streaming Framework (FLUS) source and a FLUS receiving end; receiving, by the network-based media processing NBMP source, an address of an NBMP workflow manager from the uplink real-time streaming framework FLUS source after a FLUS session establishment; and sending, by the network-based media processing NBMP source, a second request to an NBMP workflow manager through a path of addresses of the NBMP workflow manager to create or update a workflow to process media content, the NBMP workflow manager not including the uplink real-time streaming framework plus source and the plus receiver. In this way, the NBMP source may send a request to the NBMP workflow manager through a path of addresses of the NBMP workflow manager to create or update a workflow that handles media content, thereby enabling dynamic management and modification of the workflow.