阅读说明：本技术 一种用于媒体播放的缓冲队列管理方法、装置及存储介质 (Buffer queue management method and device for media playing and storage medium ) 是由 银国徽 于 2018-05-29 设计创作，主要内容包括：本公开提供了一种用于媒体播放的缓冲队列管理方法,包括：通过播放器接收来自网页的播放请求,其中,所述播放器内嵌于所述网页中播放媒体文件；根据所述播放请求的接收顺序,将所接收的播放请求存放于缓冲队列中进行排队；监听所述网页中与所述播放器对应的事件；响应于所监听到的事件,对所述缓冲队列中的播放请求执行与相应事件绑定的操作。本公开还提供了一种用于媒体播放的缓冲队列管理装置、以及存储介质。(The present disclosure provides a buffer queue management method for media playing, including: receiving a playing request from a webpage through a player, wherein the player is embedded in the webpage to play a media file; storing the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests; monitoring an event corresponding to the player in the webpage; and responding to the monitored events, and executing operations bound with the corresponding events on the playing requests in the buffer queue. The present disclosure also provides a buffer queue management apparatus for media playing, and a storage medium.)

1. A method for buffer queue management for media playback, the method comprising:

Receiving a playing request from a webpage through a player, wherein the player is embedded in the webpage to play a media file;

Storing the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests;

Monitoring an event corresponding to the player in the webpage;

And responding to the monitored events, and executing operations bound with the corresponding events on the playing requests in the buffer queue.

2. The method of claim 1, wherein the performing the operation of binding the play request in the buffer queue with the corresponding event comprises:

When the monitored event is a play pause event,

Emptying the play requests which are already allocated with connections and the play requests which are queued for waiting in the buffer queue,

Discontinuing allocation of the respective connection for the queued play request, and,

The connection that has been allocated for the dequeued play request is cancelled.

3. The method of claim 1, wherein the performing the operation of binding the play request in the buffer queue with the corresponding event comprises:

When the monitored event is a player shutdown event,

Emptying all the play requests in the buffer queue, and canceling the connection already allocated to the play requests out of the queue.

4. The method of claim 1, wherein the performing the operation of binding the play request in the buffer queue with the corresponding event comprises:

When the monitored event is a resolution switch event,

Emptying the play request for the original resolution in the buffer queue, and,

The connection already allocated for the play request of the original resolution is cancelled.

5. The method of claim 1, further comprising:

When the media data in the media file is obtained through the play request,

Packaging the media data and metadata corresponding to the media data into corresponding segmented media files;

and transmitting the segmented media file to the media element of the webpage through a media resource expansion interface for playing.

6. The method of claim 1,

The buffer queue has an upper limit of the number of concurrent connections that can be used, and the number of play request allocation connections in the buffer queue does not exceed the upper limit of the number of concurrent connections.

7. The method of claim 6,

The upper limit of the number of concurrent connections has a statically configured attribute, and the number of concurrent connections is less than the upper limit of the number of concurrent connections of the web page.

8. the method of claim 6, further comprising:

The upper limit of the number of concurrent connections has the attribute of dynamic self-adaptive configuration according to the characteristic parameters of the host equipment of the player, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the webpage.

9. a buffer queue management apparatus for media playback, the apparatus comprising: the device comprises a receiving module, a storage module, a monitoring module and a management module; wherein the content of the first and second substances,

the receiving module is used for receiving a playing request from a webpage through a player, wherein the player is embedded in the webpage to play a media file;

the storage module is used for storing the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests;

the monitoring module is used for monitoring an event corresponding to the player in the webpage;

And the management module is used for responding to the monitored events and executing the operation bound with the corresponding events on the playing requests in the buffer queue.

10. The apparatus according to claim 9, wherein the management module is specifically configured to:

When the monitored event is a play pause event,

Emptying the play requests which are already allocated with connections and the play requests which are queued for waiting in the buffer queue,

Discontinuing allocation of the respective connection for the queued play request, and,

The connection that has been allocated for the dequeued play request is cancelled.

11. The apparatus of claim 9, wherein the management module is further specifically configured to:

When the monitored event is a player shutdown event,

Emptying all the play requests in the buffer queue, and canceling the connection already allocated to the play requests out of the queue.

12. the apparatus of claim 9, wherein the management module is further specifically configured to:

When the monitored event is a resolution switch event,

Emptying the play request for the original resolution in the buffer queue, and,

The connection already allocated for the play request of the original resolution is cancelled.

13. The apparatus of claim 9, further comprising:

The packaging module is used for packaging the media data and the metadata corresponding to the media data into a corresponding segmented media file when the media data in the media file is obtained through the playing request after the received playing request is stored in a buffer queue for queuing by the storing module;

and the transmission module is used for transmitting the segmented media file to the media element of the webpage through a media resource expansion interface for playing.

14. a buffer queue management apparatus for media playback, comprising:

A memory for storing executable instructions;

A processor, configured to execute the executable instructions to implement the method for buffer queue management for media playback as claimed in any one of claims 1 to 8.

15. A storage medium storing executable instructions for implementing a buffer queue management method for media playback as claimed in any one of claims 1 to 8 when executed.

Technical Field

The present disclosure relates to network multimedia technologies, and in particular, to a method and an apparatus for managing a buffer queue for media playing, and a storage medium.

Background

At present, the media playing by using the webpage is a commonly used video playing scheme, and the complicated operation of installing a special client can be reduced.

However, when there are multiple media playing windows on the same page in the web page, because the web page cannot effectively manage some sudden events, the multiple media playing windows cannot be played normally at the same time, so that the phenomenon of network congestion inevitably occurs, and the user experience is seriously affected. In view of the above technical problems, no effective solution has been proposed in the related art.

Disclosure of Invention

In view of the above, it is desirable to provide a method, an apparatus and a storage medium for managing a buffer queue for media playing, which are at least used to improve the performance of a web page in managing events corresponding to a player.

in order to achieve the above purpose, the technical solution of the embodiment of the present disclosure is implemented as follows:

in a first aspect, an embodiment of the present disclosure provides a buffer queue management method for media playing, where the method includes:

Receiving a playing request from a webpage through a player, wherein the player is embedded in the webpage to play a media file;

Storing the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests;

monitoring an event corresponding to the player in the webpage;

And responding to the monitored events, and executing operations bound with the corresponding events on the playing requests in the buffer queue.

In a second aspect, an embodiment of the present disclosure further provides a buffer queue management apparatus for media playing, where the apparatus includes: the device comprises a receiving module, a storage module, a monitoring module and a management module; wherein the content of the first and second substances,

the receiving module is used for receiving a playing request from a webpage through a player, wherein the player is embedded in the webpage to play a media file;

The storage module is used for storing the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests;

The monitoring module is used for monitoring an event corresponding to the player in the webpage;

And the management module is used for responding to the monitored events and executing the operation bound with the corresponding events on the playing requests in the buffer queue.

In a third aspect, an embodiment of the present disclosure further provides a buffer queue management device for media playing, including:

A memory for storing executable instructions;

And the processor is used for realizing the buffer queue management method for media playing provided by the embodiment of the disclosure when the executable instruction is executed.

In a fourth aspect, the present disclosure also provides a storage medium storing executable instructions, where the executable instructions are executed to implement the buffer queue management method for media playing provided in the present disclosure.

The method, the device and the storage medium for managing the buffer queue for media playing provided by the embodiments of the present disclosure monitor an event corresponding to a player occurring in a web page in real time during the media playing process in the web page, and perform an operation of binding a playing request in the buffer queue with the corresponding event when the corresponding event is monitored. Therefore, when a plurality of media playing windows exist on the same page in the webpage, the monitored specific events can be effectively managed, the situation that the plurality of media playing windows cannot be played normally at the same time is avoided, the phenomenon of network blocking is avoided, the performance of the webpage for managing the specific events corresponding to the player can be effectively improved, and the watching experience of a user is greatly improved.

Drawings

FIG. 1 is a schematic view of an alternative construction of a container provided in accordance with an embodiment of the present disclosure;

Fig. 2 is a schematic diagram of an alternative package structure of an MP4 file according to an embodiment of the disclosure;

Fig. 3 is a schematic structural diagram of a media data container storing media data in a media file according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram of an alternative package structure of an FMP4 file provided by an embodiment of the present disclosure;

fig. 5 is a schematic flowchart of an alternative implementation of a buffer queue management method for media playing according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram illustrating an alternative implementation flow of packaging segmented media files according to an embodiment of the present disclosure;

Fig. 7 is a schematic flowchart illustrating a process of a player sending a segmented media file to a media element of a web page for decoding and playing through a media resource extension interface of the web page according to an embodiment of the present disclosure;

FIG. 8 is an alternative diagram of a player playing a segmented media file through a media resource extension interface of a web page according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of an embodiment of the present disclosure in which an MP4 file is converted into an FMP4 file and played through a media resource expansion interface;

Fig. 10 is a schematic flowchart of another alternative implementation of a buffer queue management method for media playing according to an embodiment of the present disclosure;

Fig. 11 is a schematic flowchart of another alternative implementation of a buffer queue management method for media playing according to an embodiment of the present disclosure;

Fig. 12 is a schematic flowchart of another alternative implementation of a buffer queue management method for media playing according to an embodiment of the present disclosure;

Fig. 13 is a schematic diagram illustrating an alternative functional structure of a buffer queue management apparatus for media playing according to an embodiment of the present disclosure;

Fig. 14 is a schematic diagram illustrating another alternative functional structure of a buffer queue management apparatus for media playing according to an embodiment of the present disclosure;

Fig. 15 is a schematic diagram of an alternative hardware structure of a buffer queue management apparatus for media playing according to an embodiment of the present disclosure.

Detailed Description

For the purpose of making the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be described in further detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present disclosure, and all other embodiments obtained by a person of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present disclosure.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure.

Before the present disclosure is explained in further detail, terms and expressions referred to in the embodiments of the present disclosure are explained, and the terms and expressions referred to in the embodiments of the present disclosure are applied to the following explanations.

1) a media file storing a file of encoded media data (e.g., at least one of audio data and video data) in a container (Box), which further includes metadata to express media information to ensure that the media data is correctly decoded.

For example, a formed media file in which media data is packaged in a Moving Picture Experts Group (MPEG) -4 packaging format is called an MP4 file. Typically, the MP4 file stores therein Video data encoded by the Advanced Video Coding (AVC, h.264) or MPEG-4(Part 2) specification and Audio data encoded by the Advanced Audio Coding (AAC) specification, although other encoding schemes for Video and Audio are not excluded.

2) a container (Box), also called a Box, an object-oriented member defined by a unique type identifier and a length, see fig. 1, fig. 1 is an alternative structural diagram of the container provided by the embodiment of the present disclosure, including a container Header (Box Header) and container Data (Box Data), which are filled with binary Data for expressing various information.

The container header includes a size (size) and a type (type), the size indicates a length occupied by the container in the media file, the type indicates a type of the container, referring to fig. 2, fig. 2 is a schematic diagram of an alternative package structure of an MP4 file provided by an embodiment of the present disclosure, and basic container types referred to in an MP4 file include a file type container (ftyp box), a metadata container (moov box), and a media data container (mdat box).

the container data portion may store specific data, where the container is referred to as a "data container," and may further encapsulate other types of containers, where the container is referred to as a "container of a container.

3) A Track (Track), also called a Stream (Stream), a time-ordered sequence of related samples (Sample) in a container of media data, a Track representing a sequence of video frames or a sequence of audio frames for media data, and possibly a subtitle Track synchronized with the sequence of video frames, a set of consecutive samples in the same Track being called a Chunk (Chunk).

4) A file type container, a container for storing the capacity (i.e. the length of occupied bytes) and the type of a file in a media file, as shown in fig. 2, the file type container is marked as "ftyp box", wherein the stored binary data describes the type and compatibility of the file according to the specified byte length.

5) A metadata container, a container in a media file for storing metadata (i.e., data describing multimedia data stored in the media data container), and information expressed by binary data stored in the metadata container in the MP4 file are referred to as media information.

As shown in fig. 2, the header of the metadata container represents the type of the container as "moov box" using binary data, and the container data part encapsulates an mvhd container for storing general information of an MP4 file, is independent of an MP4 file, and is related to the playing of an MP4 file, including a time length, a creation time, a modification time, and the like.

The media data container of the media file may include sub-containers corresponding to a plurality of tracks, such as an audio track container (audio track box) and a video track container (video track box), in which references and descriptions of media data of the corresponding tracks are included, and the necessary sub-containers include: a container (denoted tkhd box) for describing the characteristics and overall information of the track (e.g. duration, width, height), and a container (denoted mdia box) for recording media information of the track (e.g. information of media type and sample).

As for the sub-containers packaged in the mdia box, it may include: recording the relevant attributes and content of the track (denoted mdhd box), recording the playing procedure information of the media (denoted hdlr box), describing the media information of the media data in the track (denoted minf box); the minf box in turn has a sub-container (denoted as dinf box) for explaining how to locate the media information, and a sub-container (denoted as stbl box) for recording all the time information (decoding time/display time), position information, codec etc. of the samples in the track.

Referring to fig. 3, fig. 3 is a schematic structural diagram of a media data container for storing media data in a media file according to an embodiment of the present disclosure, and the time, type, capacity and location of a sample in the media data container can be interpreted by using media information identified from binary data in a stbl box, and each sub-container in the stbl box is described below.

the stsd box contains a sample description (sample description) table, and there may be one or more description tables in each media file according to different coding schemes and the number of files storing data, and the description information of each sample can be found through the description tables, and the description information can ensure correct decoding of the sample, and different media types store different description information, for example, the description information is the structure of the image in the case of video media.

the stts box stores the duration information of the samples and provides a table to map time (decoding time) and the serial numbers of the samples, and the samples at any time in the media file can be located through the sttx box; the stts box also uses other tables to map the sample size and pointers, where each entry in the table provides the serial number of consecutive samples within the same time offset and the offset of the sample, and increments these offsets to build a complete time-sample mapping table, and the calculation formula is as follows:

DT(n+1)＝DT(n)+STTS(n) (1)

where, stts (n) is the duration of the nth sample, DT (n) is the display time of the nth sample, and the arrangement of the samples is sorted according to the time sequence, so that the offset is always non-negative, and DT generally starts with 0, and takes the display time DT (i) of the ith sample as an example, the calculation formula is as follows:

DT(i)＝SUM(for j＝0to i-1of delta(j)) (2)

the sum of all offsets is the duration of the media data in the track.

the stss box records the sequence number of the key frame in the media file.

The sts box records the mapping relation between the samples and the blocks for storing the samples, the relation between the serial numbers of the samples and the serial numbers of the blocks is mapped through a table, and the blocks containing the specified samples can be found through table lookup.

The stco box defines the position of each block in the track, expressed in terms of the offset of the starting byte in the media data container, and the length (i.e., the size) relative to the starting byte.

The stsz box records the size (i.e., size) of each sample in the media file.

6) Media data container, a container for storing multimedia data in a media file, for example, a media data container in an MP4 file, as shown in fig. 3, a sample is a unit stored in the media data container, and is stored in a block of the media file, and the lengths of the block and the sample may be different.

7) And segmenting the media files, wherein the media files are divided into subfiles, and each segmented media file can be independently decoded.

taking an MP4 file as an example, media data in an MP4 file is divided according to key frames, the divided media data and corresponding metadata are packaged to form a segmented MP4(FMP4, Fragmented MP4) file, and metadata in each FMP4 file can ensure that the media data is correctly decoded.

For example, when converting an MP4 file as shown in fig. 2 into multiple FMP4 files, referring to fig. 4, fig. 4 is a schematic diagram of an optional packaging structure of FMP4 files provided in this disclosure, one MP4 file may be converted into multiple FMP4 files, and each FMP4 file includes three basic containers: moov containers, moof containers, and mdat containers.

The moov container includes MP4 file level metadata describing all media data in the MP4 file from which the FMP4 file is derived, such as the duration, creation time, and modification time of the MP4 file.

The moof container stores segment-level metadata describing media data packaged in the FMP4 file where it is located, ensuring that the media data in the FMP4 can be decoded.

The 1 moof container and the 1 mdat container constitute 1 segment of the segment MP4 file, and 1 or more such segments may be included in the 1 segment MP4 file, and the metadata encapsulated in each segment ensures that the media data encapsulated in the segment can be independently decoded.

8) media resource Extensions (MSE) interface, player-oriented interface implemented in web pages, interpreted by the browser's interpreter during loading in the web page, implemented by executing a front-end programming language (e.g., JavaScript), provides the player with the functionality to call the play Media stream of hypertext markup language (HTML) Media elements (Media elements), for example, to implement the play functionality of video/audio using video Element < video > and audio Element < audio >.

9) the streaming media format encapsulates media data into a media file of the streaming media, and the media file can be decoded and played without complete downloading and extra transcoding, namely, native support is provided for an encapsulation technology of downloading and playing simultaneously. A typical file in streaming media format includes: TS media file fragments based on HTTP Live Streaming (HLS, HTTP Live Streaming) technology, FLV (flash video) files, and the like.

10) non-streaming media format, which is a packaging technique that packages media data into media files and the media files can be decoded and played after being completely downloaded, a typical file in non-streaming media format includes: MP4 files, Windows Media Video (WMV) files, Advanced Streaming Format (ASF) files, and the like.

It should be noted that the MP4 file does not natively support streaming media format playback, but the technical effect of playing while downloading can be achieved by filling invalid binary data into the transcoded media stream of the player after online transcoding or the missing part of the partially downloaded MP4 file (for example, in the case of full downloading of ftyp container and moov container, the missing part of the mdat container is replaced by invalid binary data).

The following describes in detail an implementation process of a buffer queue management method for media playing in the embodiments of the present disclosure with reference to the accompanying drawings.

Fig. 5 is a schematic diagram of an optional implementation flow of the method for managing a buffer queue for media playing according to the embodiment of the present disclosure, where a terminal device implementing the embodiment of the present disclosure implements the method for managing a buffer queue for media playing by being applied to various types of players, and as for the terminal device, the terminal device may be various terminal devices such as a desktop computer and a notebook computer.

as an example, the player is an HTML5 (H5) player embedded in a web page loaded by a browser (or any application embedded in the browser), and is implemented by using a front-end oriented language (e.g., JavaScript).

As another example, a player is an application program for playing a media file, installed into a terminal device by running a pre-compiled and encapsulated installation package program in the terminal device.

as shown in fig. 5, for the implementation flow of the buffer queue management method for media playing in the embodiment of the present disclosure, the description will be made with reference to the steps shown in fig. 5.

Step 501: the method comprises the steps that a player receives a playing request from a webpage, wherein the player is embedded in the webpage to play a media file.

in the embodiment of the present disclosure, the player may be displayed in the playing window of the web page in an embedded manner, that is, the playing request is initiated by the player embedded in the web page. In practical applications, when the player plays a plurality of media files in a web page, the media files can be played in parallel through a plurality of concurrent playing windows in the web page.

The play request may be a request based on a hypertext Transfer Protocol (HTTP) initiated by the player through a web page, and of course, the play request may also be a request for the player to request to play a media file, for example, a request based on a Secure Socket Layer hypertext Transfer Protocol (HTTPs) initiated by the player through a web page, and the latter may prevent data from being intercepted and cracked by a third party in the transmission process through an encrypted transmission manner.

Here, the media file itself played in the web page may be a file supporting Streaming media playing, and is a set of a series of segmented media files, for example, a Streaming media file such as hypertext transfer protocol Live Streaming (HLS, HTTP Live Streaming) is a set of a series of continuous TS files (i.e., segmented media files), and each TS file can be independently decoded and played without complete downloading or additional transcoding, that is, each TS file natively supports downloading and playing while being played.

The media files played in the web page may also be files that do not support streaming media, such as MP4 files, and the segmented media files are segmented MP4 files (i.e., FMP4 files) that are independently decodable and playable by extracting media data and metadata from MP4 files, and filling in the encapsulation structure of FMP4 after proper processing (including calculating new metadata according to the encapsulation structure of FMP 4).

Step 502: and the player stores the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests.

The buffer queue exists independently of the webpage loaded by the embedded browser, and the buffer queue has an upper limit of the number of usable concurrent connections. When at least two playing requests initiated by the embedded player exist in the webpage, the received playing requests are stored in the buffer queue for queuing waiting according to the receiving sequence of the at least two playing requests, and corresponding connection is allocated to the playing requests in a mode of firstly entering the queue for firstly allocating.

In this embodiment of the present disclosure, after storing the received play request in a buffer queue for queuing, the method may further include: and distributing corresponding connection for the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence of the playing requests.

It should be noted that the number of the play request distribution connections in the buffer queue does not exceed the upper limit of the number of concurrent connections. Here, the connection allocated for the play request in the buffer queue is used for the player to request the media file to be played. The number of concurrent connections that can be used by the buffer queue in the embodiment of the present disclosure may be used to indicate the number of connections that can be used by the buffer queue at the same time, and generally, when the upper limit of the concurrent connection data of the buffer queue is not reached, each play request can be allocated with one connection to request media data from a server. For example, when the delay tolerance of the user to other services (such as non-video playing or web browsing) in the web page is higher, the value of the configurable concurrent connection number is larger than that of the configurable concurrent connection number when the delay tolerance of some services is lower.

Here, for allocating corresponding connections to the play requests in the buffer queue according to the upper limit of the number of concurrent connections that can be used by the buffer queue and the receiving order of the play requests, at least one of the following two ways may be adopted to implement:

Mode 1) when the connection allocated to the received play request does not reach the upper limit of the number of concurrent connections, according to the receiving sequence, allocating corresponding connections to the play requests in the buffer queue in sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Mode 2) when the connection allocated to the received play request reaches the upper limit of the number of concurrent connections, clearing the play request for releasing the connection in the buffer queue, and allocating corresponding connection to the play request in the buffer queue according to the receiving sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

For the above mode 1), a strategy of first-in first-distribution connection is mainly adopted, that is, corresponding connections are respectively distributed to all the play requests queued in the buffer queue according to the receiving sequence of the received play requests; as for the above mode 2), when the number of connections allocated to the received play request reaches the upper limit of the number of concurrent connections, the play requests without connection allocation may still be stored in the buffer queue, and for these requests, idle connections need to be waited for, so that it is to be monitored whether there is a situation that the play request releases the connection in the buffer queue in real time, and when it is monitored that there is a play request to release the connection in the buffer queue, the play request releasing the connection in the buffer queue may be cleared to obtain an idle connection.

here, the upper limit of the number of concurrent connections has a statically configured attribute, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

For example, the upper limit of the number of concurrent connections configured statically can be configured in the player through the attribute open interface. Specifically, the upper limit of the number of concurrent connections may support static configuration by an developer of a player or a service party operating a video through an attribute open interface. And the upper limit of the number of the concurrent connections which can be used by the buffer queue is greater than or equal to 2 and is less than or equal to the number of the concurrent playing windows in the webpage.

it should be noted that the upper limit of the number of concurrent connections is adaptively configured according to the requirement of the concurrent playing window in the web page. In the embodiment of the present disclosure, the minimum value of the upper limit of the number of concurrent connections is set to 2 instead of 1, mainly because there is a situation that two playing windows play simultaneously in a web page, and compared to the situation that the minimum value of the upper limit of the number of concurrent connections is 1, the minimum value of the upper limit of the number of concurrent connections is set to 2, which can avoid the situation that a playing window played later needs to wait because there is no response (because there are only 1 connection), and thus the two playing windows have a real-time response performance.

In this disclosure, an upper limit of the number of concurrent connections that can be used by the buffer queue may be preset, where the upper limit of the number of concurrent connections may be set by a player according to a connection number configuration specification of an operating system, or may be set by the player receiving the upper limit of the number of concurrent connections of a browser; of course, the setting may be performed autonomously by the player, for example, according to the characteristic parameters of the host device.

the following illustrates a scenario of player-autonomous setting: firstly, detecting characteristic parameters of host equipment of the player; then, according to the characteristic parameters, dynamically determining the upper limit of the number of concurrent connections for adapting the performance of the host device. That is, the upper limit of the number of concurrent connections has an attribute that is dynamically adaptively configured according to a characteristic parameter of a host device of the player, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

It should be noted that the upper limit of the number of concurrent connections in the embodiment of the present disclosure may be set individually for each web page, or may be an upper limit of the number of global connections set for all web pages in the browser.

it is understood that the upper limit of the number of concurrent connections here is an upper limit having a dynamic adaptation characteristic. Wherein, for dynamically determining the upper limit of the number of concurrent connections adapting to the performance of the host device according to the characteristic parameter, the following method may be adopted: and when the change of the characteristic parameter of the host equipment meets the change condition, determining the upper limit of the number of concurrent connections for adapting the change amplitude.

For example, when it is detected that a change in a characteristic parameter of a host device of the player has significant jitter, the upper limit of the number of concurrent connections that can be used by the buffer queue can be adjusted down according to the jitter amplitude; when a significant improvement in the change in the characteristic parameters of the host device of the player is detected, the upper limit of the number of concurrent connections that can be used by the buffer queue can be adaptively adjusted according to the improved quantization width.

Here, for the implementation manner of lowering or raising the upper limit of the number of concurrent connections, the upper limit of the number of concurrent connections may be lowered or raised in an equal proportion, or may be lowered or raised in a non-equal proportion, and the embodiment of the present disclosure is not specifically limited herein.

The following description will take an example in which the upper limit of the number of concurrent connections is adjusted in a stepwise manner by proportionally lowering or raising the upper limit of the number of concurrent connections. Assuming that the initial value of the upper limit of the number of concurrent connections that can be used by the buffer queue is 8, the jitter amplitude of the characteristic parameter is set to change once, and the initial value of the upper limit of the number of concurrent connections is reduced by a value in equal proportion, when it is detected that the change of the characteristic parameter of the host device of the player has obvious jitter, and the jitter amplitude is changed from 5 to 3, it can be seen that the jitter amplitude of the characteristic parameter is changed twice, and accordingly, the initial value of the upper limit of the number of concurrent connections can be reduced from 8 to 6. The improved quantization amplitude of the characteristic parameter is preset to be increased by a numerical value in an equal proportion every time the improved quantization amplitude of the characteristic parameter changes once, when the change of the characteristic parameter of the host equipment of the player is detected to be obviously improved, the improved quantization amplitude is changed from 2 to 5, and therefore the improved quantization amplitude of the characteristic parameter is changed three times, the initial value of the upper limit of the concurrent connection number can be increased from 8 to 11.

in another example, the upper limit of the number of concurrent connections may also be set by the player according to the network parameters of the web page, specifically: and detecting the network parameters of the webpage, and dynamically determining the upper limit of the number of concurrent connections according to the network parameters. For example, the network parameters may include the network bandwidth that the browser is able to use, and the network latency of the play request.

here, taking the network parameter as the network bandwidth that can be used by the browser as an example, when the network bandwidth is higher, the delay for retrieving the media data is smaller, so that a smaller upper limit of the number of concurrent connections can be configured at this time when the bandwidth is lower, that is, the network bandwidth and the upper limit of the number of concurrent connections are in a negative correlation relationship, that is, the higher the network bandwidth is, the smaller the upper limit of the number of concurrent connections is; the lower the network bandwidth, the larger the upper limit on the number of concurrent connections.

It should be noted that the media file described in the embodiments of the present disclosure may be in a streaming media format or a non-streaming media format.

In one embodiment, for the case that the media file is a streaming media file, the play request is used to request the server for a segmented media file within a given time period (a real-time play point for continuing the player), and the media element sent to the web page through the media resource extension interface of the web page is decoded, so as to realize continuous play of the media file.

As an example, the given period is a preload duration after the play point for preloading a portion of the media file after the play point for a smooth viewing experience. The length of the given period may be adapted by the player to network parameters or characteristic parameters of the hosting device to achieve an optimized utilization of terminal resources and/or network resources.

as an example, the given period may also be the length of at least one content unit after the playing point, wherein the content unit is divided and formed according to characters, scenes, plots and the like in the media file to represent the change of the content in the media file, so as to avoid the given period from being jumped by the user to consume unnecessary traffic to the greatest extent.

in one embodiment, for the case that the media file is a non-streaming media file, the player determines two key frames corresponding to a given time period in the media file by determining the given time period following a real-time playing point in the media file (the time period defined between the decoding of the key frames forms the given time period or includes the given time period), and then requests media data between the two key frames from the server, so as to construct a segmented media file capable of independently decoding the playing, and the media file is sent to a media element of a web page through a media resource extension interface of the web page for decoding, thereby realizing continuous playing of the media file.

The time corresponding to the playing point is a time measurement relative to a media time coordinate system (with the playing start time of the media file as a time origin), and the length of the given time interval is smaller than the length of the media file, for example, 5% of the predetermined proportion of the length of the media file, or a set length such as 10 minutes.

The following continues with a description of the manner in which the corresponding two key frames in a media file are determined from a given time period.

the time interval with the decoding time of the two key frames as the endpoint comprises a given time interval, the media data between the two key frames is used as the media data corresponding to the given time interval to construct a segmented media file for playing, and the given time interval is used for continuing the real-time playing point of the player, thereby realizing the continuous playing of the media file.

As for the play point, it may be a play time that is reached by continuously playing the media file (i.e., naturally playing without user intervention), for example, from a play point of 30 th minute to a play point of 40 th minute; or, the media file may reach the playing time of the media file by means of jumping (i.e., the user clicks the progress bar through the cursor to realize page jumping), where for example, the original playing point is 20% of the playing progress, and the jumping playing point is 30% of the playing progress.

in one embodiment, an implementation manner of determining two key frames (a first key frame and a second key frame whose decoding time is after the first key frame) is described according to a case where a playback point is a playback time that arrives by continuously playing a media file and a case where a video frame corresponding to the playback point and a video frame corresponding to an end time of a given period are normal frames or key frames.

Case 1) the video frame corresponding to the playing point is a normal frame, and since the player uses the media data between two key frames as a basic playing loading unit, the media data before the first key frame (the key frame whose decoding time is later than the key frame closest to the playing point in the key frames of the playing point) after the playing point is the loaded media data, and in order to avoid repeatedly acquiring the loaded media data, the first key frame in the two key frames of a given time period is: the first key frame in the media file whose decoding time is after the play point.

Case 2) the video frame corresponding to the playing point is a key frame, and the first key frame of the two key frames in the given period is: the corresponding key frame of the playing point, i.e. the key frame aligned with the start time of the given period.

case 3) if the video frame corresponding to the end time of the given time period is a normal frame, since the player uses the media data between two key frames as a basic playing loading unit, if the key frame before the end time is used as the second key frame of the given time period, the media data between the key frame and the video frame corresponding to the end time is missed to be acquired, and when the media file is played, the media data between the key frame before the end time and the video frame corresponding to the end time cannot be played and skipped, therefore, in order to ensure that the video frame corresponding to the end time of the given time period can be played normally without frame skipping, the second key frame of the two key frames of the given time period is: key frames closest to the end time among the key frames whose decoding time is later than the end time of the given period.

Case 4) the video frame corresponding to the end time of the given period is a key frame, and the second key frame of the two key frames of the given period is: the second key frame, which is time-aligned with the end time of the given period, is decoded, i.e., the key frame aligned with the end time of the given period.

In the above cases 1) and 3), the key frame crossing the playing point is used as the end point of the media data in a given period, so that it can be ensured that the video frame corresponding to the playing point has enough information for correct decoding, and frame skipping due to lack of decoded data (i.e. key frame) does not occur.

In the above cases 2) and 4), for the case that the playing point aligns the key frames, the aligned key frames are directly used as the end points of the media data in the given time period, so as to reduce the situation of requesting redundant data to the maximum extent and avoid the situation that the occupation of connection and traffic causes the delay of non-media playing service in the web page.

In another embodiment, an implementation manner of determining two key frames (a first key frame and a second key frame after the first key frame at the decoding time) is described according to a case where a video frame corresponding to a play point and a video frame corresponding to the end time of a given period are normal frames or key frames, where the play point is a play time that arrives by means of jumping.

Case 1) the video frame corresponding to the play point is a normal frame, and since the play point is reached by jumping, the first key frame before the play point and the media data between the play point are not loaded, and the first key frame is: in the media file, the first key frame before the playing point of the decoding time, that is, the time of the media data (that is, the corresponding relationship between the sequence number represented by the media information and the decoding time of the frame) is searched for the key frame whose decoding time is earlier than the starting time of the given period and is closest to the starting time.

By additionally requesting the media data between the playing point and the key frame before the playing point, the normal decoding of jumping to any playing point can be ensured, and the condition that the frame jumps due to the fact that the frame can not be decoded when the playing point corresponds to the common frame is avoided.

Case 2) the video frame corresponding to the play point is a key frame, and the first key frame is: the key frame corresponding to the playing point, that is, the key frame whose decoding time searched from the time of the media data (that is, the corresponding relationship between the sequence number represented by the media information and the decoding time of the frame) is aligned with the start time of the given period.

Case 3) the video frame corresponding to the end time of the given period is a normal frame, and the second key frame is: the key frame whose decoding time is later than the end time of the given period and is closest to the end time is decoded.

In the above cases 1) and 3), the key frame crossing the playing point is used as the end point of the media data in a given period, so that it can be ensured that the video frame corresponding to the playing point has enough information for correct decoding, and frame skipping due to lack of decoded data (i.e. key frame) does not occur.

Case 4) the video frame corresponding to the end time of the given period is a key frame, and the second key frame is: the key frames that are time aligned with the end time of a given period are decoded.

In the cases 2) and 4), the media data to be acquired is defined by aligning the key frames of the playing points, so that on the premise that the playing points can be correctly decoded, the situation of acquiring unnecessary media data is reduced to the greatest extent, occupation of connection and flow is reduced, and real-time performance of non-media playing services in the webpage is further ensured.

Firstly, packaging media data and metadata corresponding to the media data into corresponding segmented media files; and then, transmitting the segmented media file to a media element of the webpage through a media resource expansion interface for playing.

In one embodiment, the player is packaged into a corresponding segmented media file by: media data corresponding to a given time period in the media file is acquired from the server, wherein the given time period is used for continuing a playing point, the media data and metadata describing the media data are packaged according to a packaging structure of the segmented media file to form the segmented media file which can be used for being independently decoded by media elements of a webpage, and the description is continued with reference to fig. 6.

Referring to fig. 6, fig. 6 is a schematic diagram of an alternative implementation flow of encapsulating a segmented media file according to an example of the present disclosure, which will be described with reference to the steps shown in fig. 6.

Step 601: data representing the type and compatibility of the segmented media file is populated into a file type container of the segmented media file.

For example, taking as an example an FMP4 file packaged to form a package structure as shown in fig. 4, the type and length of a container (representing the entire length of the ftyp box) are filled in the header of the file type container of the FMP4 file, that is, the ftyp box, and data (binary data) representing that the file type is FMP4 and a compatible protocol is generated by filling in the data portion of the ftyp box.

Step 602: and filling metadata which represents the file level of the segmented media file into a metadata container of the segmented media file.

in one embodiment, the metadata describing the media data required to fill the nested structure is calculated from the nested structure of the metadata container in the segmented media file, based on the media data to be filled into the encapsulation structure of the segmented media file.

Still taking fig. 4 as an example, metadata representing the file level of the FMP4 file is calculated and filled into a metadata container (i.e., moov box) of the FMP4, in which three containers of mvhd, track, and video extension (mvex) are nested.

Step 603: correspondingly filling the extracted media data and the metadata describing the media data into a media data container in a segment container of the segmented media file and a metadata container at a segment level.

In one embodiment, one or more segments (fragments) may be encapsulated in a segmented media file, and for media data to be filled, one or more segmented media data containers (i.e., mdat boxes) of the segmented media file may be filled, and a segment-level metadata container (denoted as moof box) is encapsulated in each segment, wherein the filled metadata is used to describe the media data filled in the segment, so that the segments can be independently decoded.

In conjunction with fig. 4, for example, the media data to be filled is filled into 2 segments of the packaging structure of the FMP4 file, and each segment is filled with the media data; the metadata that needs to be filled into the metadata container (i.e., moof box) of the segmentation level of the corresponding segment is calculated and correspondingly filled into the child containers nested in the moof box, wherein the head of the moof box is called moof box, and the filled binary data is used to indicate the type of the container is "moof box" and the length of the moof box.

In one embodiment of filling data into the corresponding container in steps 601 to 603, when the filling operation is performed, a write operation function calling a class completes writing and merging of binary data in the memory buffer of the child container, and returns an instance of the class, where the returned instance is used for merging the child container and the child container having a nested relationship.

As an example of the stuffing data, a class MP4 for implementing a package function is established, and each sub-container in the segmented media file is packaged as a static method of class Stream; establishing classes (marked as Stream) for realizing binary data operation functions, wherein each class Stream is provided with a memory buffer area for storing binary data to be filled; converting multi-byte decimal data to be padded into binary data by a static method provided by Stream; merging and filling binary data to be filled into the sub-containers in the memory buffer area through a write operation function provided by the Stream-like instance; the static method provided by Stream returns a new Stream instance, and the merging of the current child container and other child containers with nested relation can be realized.

In one embodiment, the process of the player sending the segmented media file to the media resource extension interface of the web page for decoding and playing the media element of the web page is described.

referring to fig. 7, it is a schematic flowchart of a process that a player sends a segmented media file to a media element of a web page through a media resource extension interface of the web page for decoding and playing, and the steps shown in fig. 7 will be described.

step 701: the player adds the segmented media file to the media source object in the media resource extension interface.

Referring to fig. 8, fig. 8 is an optional schematic diagram of a player playing a segmented Media file through a Media resource extension interface of a web page according to an embodiment of the present disclosure, where when the player receives a play event of the Media file at a play window (corresponding to the play window) in the web page, the player creates a Media Source object by executing a MediaSource method encapsulated in the Media resource extension interface; executing an addSource buffer method packaged in a media resource expansion interface to create a buffer of a MediaSource object, namely a Source buffer (Source buffer) object, wherein one MediaSource object has one or more Source buffer objects, and each Source buffer object can be used for corresponding to a playing window in a webpage and is used for receiving a segmented media file to be played in the window.

In the process of playing the media file, a Parser (Parser) in the player continuously constructs a new segmented media file by parsing newly acquired media data, and adds the segmented media file to a SourceBuffer object of the same MediaSource object by executing an appdbuffer method of the SourceBuffer object.

Step 702: and the player calls the media resource expansion interface to create a virtual address corresponding to the media source object.

For example, the player executes the createObjectURL method encapsulated in the media resource extension interface, and creates a virtual address, i.e. a virtual URL, of the corresponding media source object, in which the Blob-type segmented media file is encapsulated.

In addition, the player sets the MediaSource object as the source (src) attribute of the virtual URL, i.e., binds the virtual URL with a media element in the web page, such as a video/audio element, which is also referred to as associating the media source object to the media element in the web page.

step 703: the player transmits a virtual address to the media element of the webpage, and the virtual address is used for playing the media element by taking the media source object as a data source.

For example, the player includes a statement that calls the media element to play the virtual URL, such as: < audio > virtual URL. When the browser interprets the corresponding statement in the player embedded in the webpage, the media element of the browser reads the segmented media file from the SourceBuffer object bound by the virtual URL, and decodes and plays the segmented media file.

Next, a process of converting the MP4 file into the FMP4 file and playing the converted file on a web page through the media resource extension interface by the player will be described.

referring to fig. 9, fig. 9 is a schematic diagram of converting an MP4 file into an FMP4 file and playing the file through a media resource extension interface according to an embodiment of the present disclosure, where the player requests to obtain partial media data in the MP4 file from a server based on a real address (http:// www.toutiao.com/a/b.mp4) of the media file, for example, data whose decoding time is in a given time period for continuing a playing point.

the player constructs an FMP4 file based on the acquired media data, and then adds the FMP4 file to a Source buffer object corresponding to the MediaSource object, because the virtual URL is bound to the MediaSource object, when the code for calling the audio/video element by the player is executed, the audio/video element reads a new FMP4 file which is continuously added from the Source buffer object of the MediaSource object and decodes the new FMP4 file, thereby realizing the continuous playing of the media file.

Step 503: and the player monitors the event corresponding to the player in the webpage.

Here, the event corresponding to the player includes at least one of the following specific events: a play pause event, a player shutdown event, and a resolution switch event.

It should be noted that, after storing the received play request in the buffer queue for queuing, monitoring the event corresponding to the player in the page, or while storing the received play request in the buffer queue for queuing, monitoring the event corresponding to the player in the page in real time, and as to which execution sequence is adopted, the embodiment of the present disclosure is not limited herein.

Step 504: and the player responds to the monitored events and performs operations bound with the corresponding events on the playing requests in the buffer queue.

Here, a binding relationship between a specific event and an operation for managing the buffer queue is preset, and when the specific event is monitored, an operation having a corresponding binding relationship with the monitored specific event can be queried according to the binding relationship, and the operation bound with the corresponding specific event is executed for the play request in the buffer queue.

In the embodiment of the present disclosure, when it is determined that the monitored event is a play pause event, for the operation of performing binding with the corresponding event on the play request in the buffer queue in this step 504, the following manner may be adopted: emptying the playing requests which are already distributed with the connection and the playing requests which are queued for waiting in the buffer queue, stopping distributing the corresponding connection for the playing requests which are queued for waiting, and canceling the connection which is already distributed for the playing requests which are out of the queue.

Specifically, when media playing is performed in a web page, when an event that a play pause is executed on a media file, such as a video file, in the web page is monitored, a play request corresponding to the video file in a buffer queue may be emptied, where the play request corresponding to the video file mainly includes: playing requests queued and waiting in the buffer queue, and playing requests already sent by using the connection in the buffer queue, namely playing requests already allocated with the connection; meanwhile, the following management operations can be carried out on the buffer queue: the allocation of connections to the queued play requests in the buffer queue is suspended, and the connections already allocated for the dequeued play requests are cancelled.

In this embodiment of the present disclosure, when it is determined that the monitored event is a player closing event, for the operation of performing binding with the corresponding event on the play request in the buffer queue in this step 504, the following manner may be adopted to implement: emptying all the play requests in the buffer queue, and canceling the connection already allocated to the play requests out of the queue.

for example, the player shutdown event may be a player destruction event, i.e., the player cannot be used at this time. When media playing is performed in a webpage, when it is monitored that a player closing event is performed in the webpage, an emptying operation can be performed on all playing requests stored in a buffer queue, wherein all playing requests stored in the buffer queue include: the playing requests which are stored in the buffer queue and are queued for waiting and the playing requests which are stored in the buffer queue and are already distributed and connected; meanwhile, the following management operations can be carried out on the buffer queue: the connection that has been allocated for the dequeued play request is cancelled.

In the embodiment of the present disclosure, when it is determined that the monitored event is a resolution switching event, for the operation of performing binding with the corresponding event on the play request in the buffer queue in this step 504, the following manner may be adopted: emptying the playing request aiming at the original resolution in the buffer queue, and canceling the connection which is already allocated to the playing request aiming at the original resolution.

For example, when media playing is performed in a web page, when an event that the resolution switching playing pause is performed on a media file, such as a video file, in the web page is monitored, for example, the resolution of the video file is switched from high definition to ultra definition, the following management operations may be performed on the buffer queue: emptying the playing request for the high definition resolution stored in the buffer queue, and canceling the connection already allocated to the playing request for the high definition resolution.

it should be noted that, when the connection allocated to the playback request with the original resolution, i.e., the playback request with the high resolution cannot be cancelled due to an abnormality, it is necessary to wait for the connection release to obtain an idle connection, and then respond to the new playback request using the idle connection, and discard the media data returned by the connection.

fig. 10 is a schematic flowchart of another alternative implementation of a buffer queue management method for media playing according to an embodiment of the present disclosure, where the buffer queue management method for media playing may be applied to various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file; as shown in fig. 10, an implementation process of a buffer queue management method for media playing in the embodiment of the present disclosure includes the following steps:

Step 1001: the player receives a play request from a web page.

Step 1002: the player stores the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests.

Step 1003: and the player allocates corresponding connection for the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence of the playing requests.

In this embodiment of the present disclosure, the number of play request allocation connections in the buffer queue does not exceed the upper limit of the number of concurrent connections. The connection allocated for the play request in the buffer queue is used for the player to request the media file to be played. The number of concurrent connections that can be used by the buffer queue may be used to indicate the number of connections that can be simultaneously used by the buffer queue, and generally, when the upper limit of the concurrent connection data of the buffer queue is not reached, each play request can be allocated with one connection to request media data from the server. For example, when the delay tolerance of the user to other services (such as non-video playing or web browsing) in the web page is higher, the value of the configurable concurrent connection number is larger than that of the configurable concurrent connection number when the delay tolerance of some services is lower.

For allocating corresponding connections to the play requests in the buffer queue according to the upper limit of the number of concurrent connections that can be used by the buffer queue and the receiving order of the play requests in this step 1003, at least one of the following two ways may be used to implement:

Mode 1) when the connection allocated to the received play request does not reach the upper limit of the number of concurrent connections, according to the receiving sequence, allocating corresponding connections to the play requests in the buffer queue in sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Mode 2) when the connection allocated to the received play request reaches the upper limit of the number of concurrent connections, clearing the play request for releasing the connection in the buffer queue, and allocating corresponding connection to the play request in the buffer queue according to the receiving sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Here, the upper limit of the number of concurrent connections has a statically configured attribute, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

The upper limit of the number of concurrent connections can be preset, wherein the upper limit of the number of concurrent connections can be set by a player according to a configuration specification of the number of connections of an operating system, and the upper limit of the number of concurrent connections of a browser can be received by the player for setting; of course, the setting may be performed autonomously by the player, for example, according to the characteristic parameters of the host device. The following illustrates the scenario of player settings: firstly, detecting characteristic parameters of host equipment of the player; then, according to the characteristic parameters, dynamically determining the upper limit of the number of concurrent connections for adapting the performance of the host device. That is, the upper limit of the number of concurrent connections also has the attribute of dynamic adaptive configuration according to the characteristic parameters of the host device, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

Specifically, for dynamically determining the upper limit of the number of concurrent connections that adapt to the performance of the host device according to the characteristic parameter, the following method may be adopted: and when the change of the characteristic parameter of the host equipment meets the change condition, determining the upper limit of the number of concurrent connections for adapting the change amplitude. For example, when it is detected that a change in a characteristic parameter of the host device has significant jitter, the upper limit of the number of concurrent connections that can be used by the buffer queue may be adaptively adjusted down according to the jitter amplitude; when a significant improvement in the change in the characteristic parameters of the host device is detected, the upper limit of the number of concurrent connections that can be used by the buffer queue can be adaptively adjusted according to the improved quantization width.

Step 1004: when the player obtains the media data in the media file through the connection allocated for the play request, the media data and the metadata corresponding to the media data are packaged into the corresponding segmented media file.

Here, the process of encapsulating the segmented media file into the corresponding segmented media file may be implemented by using an optional implementation flow diagram of encapsulating the segmented media file shown in fig. 6, which is not described in detail here.

Step 1005: the player transmits the segmented media file to the media element of the webpage through the media resource expansion interface for playing.

here, the process of playing the media element transferred to the web page through the media resource extension interface may be implemented by using the above-mentioned manner of the flow diagram shown in fig. 7, where the player sends the segmented media file to the media element of the web page through the media resource extension interface of the web page for decoding and playing, and details are not repeated here.

Step 1006: and the player monitors events corresponding to the player in the webpage.

Step 1007: when the event monitored by the player is a play pause event, emptying the play request which is already allocated with connection and the play request which is queued for waiting in the buffer queue, stopping allocating corresponding connection for the play request which is queued for waiting, and canceling the connection which is already allocated for the play request which is out of the queue.

Fig. 11 is a schematic flowchart of another alternative implementation of a buffer queue management method for media playing according to an embodiment of the present disclosure; the buffer queue management method for media playing can be applied to various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file; as shown in fig. 11, an implementation process of a buffer queue management method for media playing in the embodiment of the present disclosure includes the following steps:

Step 1101: the player receives a play request from a web page.

step 1102: the player stores the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests.

step 1103: and the player allocates corresponding connection for the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence of the playing requests.

In this step 1103, according to the upper limit of the number of concurrent connections that can be used by the buffer queue and the receiving order of the play requests, allocating corresponding connections to the play requests in the buffer queue may be implemented by at least one of the following two ways:

Mode 1) when the connection allocated to the received play request does not reach the upper limit of the number of concurrent connections, according to the receiving sequence, allocating corresponding connections to the play requests in the buffer queue in sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Mode 2) when the connection allocated to the received play request reaches the upper limit of the number of concurrent connections, clearing the play request for releasing the connection in the buffer queue, and allocating corresponding connection to the play request in the buffer queue according to the receiving sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Here, the upper limit of the number of concurrent connections has a statically configured attribute, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

the upper limit of the number of concurrent connections can be preset, wherein the upper limit of the number of concurrent connections can be set by a player according to a configuration specification of the number of connections of an operating system, and the upper limit of the number of concurrent connections of a browser can be received by the player for setting; of course, the setting may be performed autonomously by the player, for example, according to the characteristic parameters of the host device. The following illustrates the scenario of player settings: firstly, detecting characteristic parameters of host equipment of the player; then, according to the characteristic parameters, dynamically determining the upper limit of the number of concurrent connections for adapting the performance of the host device. That is, the upper limit of the number of concurrent connections also has the attribute of dynamic self-adaptive configuration according to the characteristic parameters of the host device, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

Specifically, for dynamically determining the upper limit of the number of concurrent connections that adapt to the performance of the host device according to the characteristic parameter, the following method may be adopted: and when the change of the characteristic parameter of the host equipment meets the change condition, determining the upper limit of the number of concurrent connections for adapting the change amplitude. For example, when it is detected that a change in a characteristic parameter of the host device has significant jitter, the upper limit of the number of concurrent connections that can be used by the buffer queue may be adaptively adjusted down according to the jitter amplitude; when a significant improvement in the change in the characteristic parameters of the host device is detected, the upper limit of the number of concurrent connections that can be used by the buffer queue can be adaptively adjusted according to the improved quantization width.

Step 1104: when the player obtains the media data in the media file through the connection allocated for the play request, the media data and the metadata corresponding to the media data are packaged into the corresponding segmented media file.

Here, the process of encapsulating the segmented media file into the corresponding segmented media file may be implemented by using an optional implementation flow diagram of encapsulating the segmented media file shown in fig. 6, which is not described in detail here.

Step 1105: the player transmits the segmented media file to the media element of the webpage through the media resource expansion interface for playing.

Here, the process of playing the media element transferred to the web page through the media resource extension interface may be implemented by using the above-mentioned manner of the flow diagram shown in fig. 7, where the player sends the segmented media file to the media element of the web page through the media resource extension interface of the web page for decoding and playing, and details are not repeated here.

Step 1106: and the player monitors events corresponding to the player in the webpage.

Step 1107: when the player monitors that the event in the webpage is a player closing event, all the playing requests in the buffer queue are emptied, and the connection which is already distributed to the playing requests which are out of the queue is cancelled.

fig. 12 is a schematic flowchart of another alternative implementation of a buffer queue management method for media playing according to an embodiment of the present disclosure; the buffer queue management method for media playing can be applied to various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file; as shown in fig. 12, an implementation process of a buffer queue management method for media playing in the embodiment of the present disclosure includes the following steps:

step 1201: the player receives a play request from a web page.

step 1202: the player stores the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests.

step 1203: and the player allocates corresponding connection for the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence of the playing requests.

For allocating corresponding connections to the play requests in the buffer queue according to the upper limit of the number of concurrent connections that can be used by the buffer queue and the receiving order of the play requests in this step 1203, at least one of the following two ways may be used to implement:

mode 1) when the connection allocated to the received play request does not reach the upper limit of the number of concurrent connections, according to the receiving sequence, allocating corresponding connections to the play requests in the buffer queue in sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Mode 2) when the connection allocated to the received play request reaches the upper limit of the number of concurrent connections, clearing the play request for releasing the connection in the buffer queue, and allocating corresponding connections to the play requests in the buffer queue according to the receiving sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Here, the upper limit of the number of concurrent connections has a statically configured attribute, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

The upper limit of the number of concurrent connections also has the attribute of dynamic self-adaptive configuration according to the characteristic parameters of the host equipment, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the webpage.

step 1204: when the player obtains the media data in the media file through the connection allocated for the play request, the media data and the metadata corresponding to the media data are packaged into the corresponding segmented media file.

Here, the process of encapsulating the segmented media file into the corresponding segmented media file may be implemented by using an optional implementation flow diagram of encapsulating the segmented media file shown in fig. 6, which is not described in detail here.

Step 1205: the player transmits the segmented media file to the media element of the webpage through the media resource expansion interface for playing.

Here, the process of playing the media element transferred to the web page through the media resource extension interface may be implemented by using the above-mentioned manner of the flow diagram shown in fig. 7, where the player sends the segmented media file to the media element of the web page through the media resource extension interface of the web page for decoding and playing, and details are not repeated here.

Step 1206: and the player monitors events corresponding to the player in the webpage.

Step 1207: when the player monitors that the event in the webpage is a resolution switching event, emptying the playing request aiming at the original resolution in the buffer queue and canceling the connection already allocated to the playing request aiming at the original resolution.

For example, during the process of playing media in a web page, when the player monitors an event in the web page that the playing of media files such as video files is paused by performing resolution switching, for example, the resolution of the video files is switched from high definition to super definition, the following management operations may be performed on the buffer queue: emptying the playing request for the high definition resolution stored in the buffer queue, and canceling the connection already allocated to the playing request for the high definition resolution.

It should be noted that, when the connection allocated to the playback request with the original resolution, i.e., the playback request with the high resolution cannot be cancelled due to an abnormality, it is necessary to wait for the connection release to obtain an idle connection, and then respond to the new playback request using the idle connection, and discard the media data returned by the connection.

In order to implement the above buffer queue management method for media playing, an embodiment of the present disclosure further provides a buffer queue management device for media playing, where the buffer queue management device for media playing may be an application program for running various types of players, such as an H5 player embedded in a web page loaded by a browser or playing a media file, and fig. 13 is an optional functional structure schematic diagram of the buffer queue management device for media playing provided by the embodiment of the present disclosure; as shown in fig. 13, the buffer queue management apparatus for media playing includes a receiving module 1301, a storing module 1302, a listening module 1303, and a management module 1304. Each module is explained in detail below.

The receiving module 1301 is configured to receive a playing request from a webpage through a player, where the player is embedded in the webpage to play a media file.

The storing module 1302 is configured to store the received play requests in a buffer queue for queuing according to the receiving sequence of the play requests.

The monitoring module 1303 is configured to monitor an event corresponding to the player in the web page.

The management module 1304 is configured to, in response to the monitored event, perform an operation bound to the corresponding event on the play request in the buffer queue.

In this embodiment of the present disclosure, when the monitored event is a play pause event, for the reason that the management module 1304 performs an operation bound to the corresponding event on the play request in the buffer queue, the following method may be adopted: emptying the playing requests which are already distributed with the connection and the playing requests which are queued for waiting in the buffer queue, stopping distributing the corresponding connection for the playing requests which are queued for waiting, and canceling the connection which is already distributed for the playing requests which are out of the queue.

In this embodiment of the present disclosure, when the monitored event is a player closing event, for the reason that the management module 1304 performs an operation bound to the corresponding event on the play request in the buffer queue, the following method may be adopted: emptying all the play requests in the buffer queue, and canceling the connection already allocated to the play requests out of the queue.

In this embodiment of the present disclosure, when the monitored event is a resolution switching event, for the reason that the management module 1304 performs an operation bound to the corresponding event on the play request in the buffer queue, the following method may be adopted: emptying the playing request aiming at the original resolution in the buffer queue, and canceling the connection which is already allocated to the playing request aiming at the original resolution.

here, the buffer queue has an upper limit of the number of concurrent connections that can be used, and the number of play request allocation connections in the buffer queue does not exceed the upper limit of the number of concurrent connections.

It should be particularly noted that the upper limit of the number of concurrent connections in the embodiment of the present disclosure has a statically configured attribute, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page. The upper limit of the number of concurrent connections in the embodiment of the present disclosure further has an attribute that is configured dynamically and adaptively according to a characteristic parameter of a host device of the player, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page. And when the characteristic parameter change of the host equipment meets the change condition, determining the upper limit of the number of concurrent connections adapting to the change amplitude.

It will be appreciated that the upper limit on the number of concurrent connections also has the property of being dynamically adaptively configured according to the network parameters of the web page. The network parameters may include network bandwidth that can be used by the browser and network delay of the play request.

fig. 14 is a schematic diagram of another optional functional structure of the buffer queue management device for media playing according to the embodiment of the present disclosure, and as shown in fig. 14, the buffer queue management device for media playing may further include: the encapsulating module 1305 is configured to encapsulate, after the storing module 1302 stores the received play request in a buffer queue for queuing, the media data and the metadata corresponding to the media data into a corresponding segmented media file when the media data in the media file is obtained through the play request.

A transmission module 1306, configured to transmit the segmented media file to a media element of the web page through a media resource extension interface for playing.

It should be noted that: the buffer queue management device for media playing provided in the above embodiments is only exemplified by the division of the above program modules when managing the buffer queue, and in practical applications, the above processing may be distributed to different program modules according to needs, that is, the internal structure of the buffer queue management device for media playing is divided into different program modules to complete all or part of the above described processing.

in practical applications, the storage module 1302, the monitoring module 1303, the management module 1304 and the encapsulation module 1305 in the above modules may be implemented by a Central Processing Unit (CPU), a microprocessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like on a player; the receiving module 1301 and the transmitting module 1306 in the above modules may be implemented in practical applications by a communication module (including a basic communication suite, an operating system, a communication module, a standardized interface and protocol, etc.) and a transceiver antenna.

In order to implement the above buffer queue management method for media playing, the embodiment of the present disclosure further provides a hardware structure of a buffer queue management device for media playing. A hardware structure of a buffer queue management apparatus for media playing, which can be an application program or the like running various types of players such as an H5 player embedded in a web page loaded by a browser, or playing a media file, implementing an embodiment of the present disclosure will now be described with reference to the accompanying drawings.

further description is made below on the hardware structure of the buffer queue management device for media playing according to the embodiment of the present disclosure, it is understood that fig. 15 only shows an exemplary structure of the buffer queue management device for media playing, and not a whole structure, and a part of or a whole structure shown in fig. 15 may be implemented as needed.

referring to fig. 15, fig. 15 is a schematic diagram of an alternative hardware structure of a buffer queue management apparatus for media playing according to an embodiment of the present disclosure, which can be used in various foregoing players in practical applications. The buffer queue management apparatus 1500 for media playing shown in fig. 15 may include: at least one processor 1501, memory 1502, a user interface 1503, and at least one network interface 1504. The various components of the buffer queue management device 1500 for media playback are coupled together by a bus system 1505. It will be appreciated that bus system 1505 is used to enable communications among the components by way of connections. Bus system 1505 may include a power bus, a control bus, and a status signal bus, in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 1505 in fig. 15.

The user interface 1503 may include a display, a keyboard, a mouse, a trackball, a click wheel, a key, a button, a touch pad, a touch screen, or the like, among others.

it will be appreciated that the memory 1502 can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.

the memory 1502 in the disclosed embodiment is used to store various types of data to support the operation of the buffer queue management device 1500 for media playback. Examples of such data include: any executable instructions for operating on the buffer queue management device for media playing 1500, such as the executable program 15021 and the operating system 15022, a program that implements the buffer queue management method for media playing of the embodiments of the present disclosure may be contained in the executable program 15021.

the buffer queue management method for media playing disclosed in the embodiments of the present disclosure may be applied to the processor 1501 or implemented by the processor 1501. Processor 1501 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the buffer queue management method for media playing described above may be implemented by integrated logic circuits of hardware in the processor 1501 or instructions in the form of software. The processor 1501 described above may be a general purpose processor, DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like. Processor 1501 may implement or perform the buffer queue management methods, steps, and logic blocks for media playback provided in the embodiments of the present disclosure. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the buffer queue management method for media playing provided by the embodiment of the present disclosure may be directly implemented as the execution of a hardware decoding processor, or implemented by the combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in the memory 1502, and the processor 1501 reads the information in the memory 1502, and completes the steps of the buffer queue management method for media playing provided by the disclosed embodiments in combination with the hardware thereof.

In this embodiment, the buffer queue management apparatus 1500 for media playing comprises a memory 1502 for storing executable instructions; the processor 1501 is configured to implement the buffer queue management method for media playing provided in the embodiment of the present disclosure when executing the executable instruction.

in an exemplary embodiment, the present disclosure further provides a storage medium, which may be a storage medium such as an optical disc, a flash memory, or a magnetic disc, and may be a non-transitory storage medium. The storage medium stores executable instructions thereon, and the executable instructions when executed implement the buffer queue management method for media playing provided by the embodiment of the disclosure.

In summary, the embodiments of the present disclosure have the following beneficial effects: when a plurality of media playing windows exist on the same page in the webpage, the monitored specific events can be effectively managed, the situation that the plurality of media playing windows cannot be played normally at the same time is avoided, the phenomenon of network blocking is avoided, the performance of the webpage for managing the specific events corresponding to the player can be effectively improved, and the watching experience of a user is greatly improved.

The technical solutions described in the embodiments of the present disclosure can be arbitrarily combined without conflict.

The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present disclosure, and all the changes or substitutions should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.