阅读说明：本技术 流媒体播放方法和装置 (Streaming media playing method and device ) 是由 黄思远 孙晔 高锐 李磊 于 2021-08-25 设计创作，主要内容包括：本申请公开了一种流媒体播放方法和装置,其中方法包括：在流媒体播放过程中,对于当前待下载的片段,确定下载粒度,根据所述下载粒度,请求下载所述片段；基于接收到的流媒体数据,进行所述流媒体的播放。本申请,按照一定的下载粒度,请求下载流媒体片段,细化了每次向服务器请求下载数据的粒度,如此,可以减小单次请求下载数据的传输时长,增强音视频数据下载的同步性,进而可以提高流媒体播放的流畅性和实时性。因此,采用本申请可以提高流媒体播放质量。(The application discloses a streaming media playing method and a streaming media playing device, wherein the method comprises the following steps: in the process of streaming media playing, determining a downloading granularity for a segment to be downloaded currently, and requesting to download the segment according to the downloading granularity; and playing the streaming media based on the received streaming media data. According to the method and the device, the streaming media fragments are requested to be downloaded according to a certain downloading granularity, and the granularity of requesting the server to download data at each time is refined, so that the transmission time of requesting the data to be downloaded at a single time can be shortened, the synchronism of audio and video data downloading is enhanced, and the fluency and the real-time performance of streaming media playing can be further improved. Therefore, the streaming media playing quality can be improved by adopting the method and the device.)

1. A method for playing streaming media, comprising:

in the process of streaming media playing, determining a downloading granularity for a segment to be downloaded currently, and requesting to download the segment according to the downloading granularity;

and playing the streaming media based on the received streaming media data.

2. The method of claim 1, wherein the download granularity comprises: a first download granularity for downloading segment description data for a segment and a second download granularity for downloading content data frames for a segment.

3. The method of claim 2, wherein the determining the download granularity comprises:

setting the first downloading granularity as a preset segment description data downloading granularity value; or, the first download granularity is calculated according to the current bandwidth and a preset first granularity duration;

setting the second download granularity as a preset data frame download granularity value; or, the second download granularity is calculated according to the current bandwidth and a preset second granularity duration.

4. The method of claim 1, further comprising: monitoring a real-time bandwidth based on the downloading condition of the streaming media data, and determining a matched target code rate based on the real-time bandwidth; and when the target code rate is different from the code rate of the current downloading segment, triggering code rate switching to request the data to be downloaded according to the segment corresponding to the target code rate.

5. The method of claim 4, wherein playing the streaming media further comprises:

when code rate switching occurs, loading the downloaded data corresponding to the switched code rate to a first decoder which is currently in a decoding stop state, and switching a second decoder which is currently in a decoding start state to the decoding stop state when the second decoder finishes decoding a currently played decoding unit so as to stop the second decoder from decoding, and switching the first decoder to the decoding start state so as to start the first decoder to decode.

6. The method of claim 4, wherein playing the streaming media further comprises:

and when code rate switching occurs, configuring the cache parameters according to the new target code rate.

7. The method of claim 4, wherein the monitoring the real-time bandwidth based on the downloading of the streaming media data comprises:

when a data packet of the streaming media is received, judging whether the bandwidth of the data packet needs to be filtered according to a preset bandwidth filtering condition; if not, updating the current bandwidth measurement parameter according to the downloading condition of the data packet;

and judging whether the current bandwidth meets the preset bandwidth measurement condition in real time according to the bandwidth measurement parameter and a preset sampling period, if so, triggering bandwidth sampling according to the bandwidth measurement parameter, acquiring the number of current bandwidth sampling points, and calculating the current bandwidth measurement value according to the sampled bandwidth value and the number of the bandwidth sampling points.

8. The method of claim 7, wherein the bandwidth filtering condition comprises:

the time interval between the current time and the latest request for downloading is less than a preset filtering time threshold;

and/or the data packet belongs to preset small-data-volume file data, wherein the small-data-volume file data comprises index file data and initialization fragment data.

9. The method of claim 7, wherein the bandwidth measurement parameters include a transmission time to be used for bandwidth sampling and a total amount of data that has been received without requiring bandwidth filtering.

10. The method of claim 9, wherein updating the current bandwidth measurement parameter comprises:

setting the transmission time to be used for bandwidth sampling as a time interval between the current time and the latest request for downloading;

and increasing the size of the data packet to the current total data amount.

11. The method of claim 9, wherein the monitoring real-time bandwidth based on the downloading of the streaming media data further comprises:

when the bandwidth of the data packet needs to be filtered and the downloading unit to which the data packet belongs has the downloading pause operation, the following conditions are met: when the time interval between the current time and the latest request for downloading is less than the preset filtering time threshold, setting the total data amount which is currently involved in bandwidth measurement as the total data amount which is currently received and does not need to filter the bandwidth, and timing the current sampling period to zero.

12. The method of claim 7, wherein the bandwidth measurement condition comprises:

the sampling period is currently reached;

or the quantity of data which does not participate in bandwidth measurement in the data which is currently received and does not need to filter the bandwidth is larger than a preset threshold value of the quantity of bandwidth measurement.

13. The method of claim 9, wherein triggering bandwidth sampling according to the bandwidth measurement parameter comprises:

and if the transmission time to be used for bandwidth sampling is within a preset time range, calculating the bandwidth value of the current sampling according to the transmission time to be used for bandwidth sampling and the quantity of data which does not participate in bandwidth measurement in the data which is currently received and does not need bandwidth filtering.

14. The method of claim 7, wherein obtaining the current number of bandwidth samples comprises:

dividing the playing time length of the currently received and undisplayed data by the sampling period to obtain the number of initial sampling points;

if the playing time of the maximum downloading unit of the currently played file is less than a preset time threshold, determining the number of the bandwidth sampling points matched with the number of the initial sampling points according to the mapping relation between the number of the preset initial sampling points and the number of the bandwidth sampling points, and setting the number of the bandwidth sampling points as the number of the current bandwidth sampling points, otherwise, setting the number of the current bandwidth sampling points as the preset maximum sampling point number threshold.

15. The method of claim 4, wherein the monitoring the real-time bandwidth based on the downloading of the streaming media data comprises:

when receiving the data packet of the streaming media, filtering the data packet which is not sent in saturation;

in the filtered data packet, a stable region of the data size and the transmission rate is found, and the current bandwidth is calculated based on the data size and the stable region of the transmission rate.

16. The method of claim 15, wherein filtering the understeered transmitted packets comprises:

calculating the preset time T_oAnd if the data size of the current data packet is smaller than or equal to the data size average value, filtering the current data packet.

17. The method according to claim 16, wherein the calculating a data size average of the data packets received within a preset time, and if the data size of the current data packet is smaller than or equal to the data size average, the filtering the current data packet specifically includes:

calculating the transmission time of the current data packet;

inserting the data size and transmission time of the current data packet into a bidirectional queue Q_oThe tail of (a);

if inserted into queue Q_oThe time of the queue element at the head is greater than the preset time T from the current time_oThen the queue Q is deleted_oA queue element of the head;

calculate the queue Q_oAverage of median data size;

if the data size of the current data packet is larger than the average value of the data sizes, inserting the data size and the transmission time of the current data packet into a bidirectional queue Q_fThe tail of (2).

18. The method of claim 17, wherein the finding of the stable region of the data size and the transmission rate in the filtered data packet, and the calculating of the current bandwidth based on the stable region of the data size and the transmission rate specifically comprises:

for the queue Q_fSorting according to the data size;

traversing the sorted queue Q_fAcquiring a group of queue elements with the same data size and the maximum number, and recording the queue elements in a queue Q_fPosition (2): starting position P_bEnd position P_eAnd the number of elements C_max；

If said C is_maxIf the value is greater than or equal to the preset value C, the queue Q is aligned_fMiddle P_bTo P_eThe queue elements in the region are sorted according to the transmission time, a stable region of the transmission rate is obtained based on the sorting result, and the data size is accumulated based on the stable region of the transmission rateObtaining S, accumulating the transmission time to obtain T, and calculating B as S/T as the current bandwidth;

if said C is_maxIf the value is less than the preset value C, accumulating the queue Q_oThe total data size S is obtained from the data size in the queue, and a queue Q is accumulated_oThe total transmission time T is obtained from the transmission time in (1), and B ═ S/T is calculated as the current bandwidth.

19. The method of claim 18, wherein the obtaining the stable region of the transmission rate based on the sorting result comprises:

taking preset N1 queue elements at a first middle position and preset N2 queue elements at a second middle position of the sequencing result, and forming a stable region of the transmission rate by the N1 queue elements and the N2 queue elements; wherein the first intermediate position precedes the second intermediate position in the ranking result.

20. The method of claim 18, wherein the obtaining the stable region of the transmission rate based on the sorting result comprises:

calculating P_bTo P_eMiddle position P of the region_m＝(P_e–P_b) /2 if P_mRounding it up or down if not an integer;

based on P_bTo P_eSequencing result of region, traverse queue Q_fIn the slave P_bTo P_m-10 queue elements, calculating the variance of the transmission time of each queue element and the transmission time of the following 9 data, obtaining a group of queue elements with the smallest variance;

based on P_bTo P_eSequencing result of region, traverse queue Q_fIn the slave P_m+1 to P_e-10 queue elements, calculating the variance of the transmission time of each queue element and the transmission time of the following 9 data, obtaining a group of queue elements with the smallest variance;

the two sets of queue elements obtained constitute a stable region of transmission rate.

21. The method of claim 4, wherein the determining the matching target code rate based on the real-time bandwidth comprises:

predicting a target bandwidth according to the real-time bandwidth;

determining a matched switching strategy according to the target bandwidth and the optimal characteristics corresponding to the current scene;

and determining the current target code rate according to the switching strategy.

22. The method of claim 1, further comprising:

monitoring the real-time playing delay of the streaming media according to a preset delay monitoring period;

and determining a corresponding playing speed according to the real-time playing delay, and playing according to the playing speed.

23. The method of claim 22, wherein determining the corresponding playback rate according to the real-time playback delay comprises:

if the real-time playing delay exceeds a specified tolerable delay range, determining the corresponding playing rate as a fast playing rate;

and if the real-time playing delay is within the tolerable delay range, determining the corresponding playing rate as a normal rate.

24. The method of claim 22, wherein determining the corresponding playback rate according to the real-time playback delay comprises:

if the real-time playing delay exceeds a specified tolerable delay range and the consumption rate of the current streaming media cache data is not greater than the downloading rate, determining the corresponding playing rate as a fast playing rate;

if the real-time playing delay is within the tolerable delay range and the consumption rate of the current streaming media cache data is not greater than the downloading rate, determining that the corresponding playing rate is a normal rate;

and if the difference between the consumption rate of the current streaming media cache data and the downloading rate is greater than a preset difference threshold, determining that the corresponding playing rate is a slow playing rate, wherein the difference threshold is greater than or equal to zero.

25. The method according to claim 23 or 24, wherein the determining of the fast playing rate comprises:

inquiring a mapping relation between a difference value between preset real-time playing delay and target delay and a fast playing rate to obtain a fast playing rate corresponding to the difference value between the real-time playing delay and the target delay; wherein the difference value in the mapping relation is in direct proportion to the fast playing rate.

26. The method of claim 24, wherein the determining of the slow play rate comprises:

inquiring a mapping relation between a preset difference value between the consumption rate and the download rate and the slow play rate to obtain the slow play rate corresponding to the current difference value between the consumption rate and the download rate; wherein the difference in the mapping relationship is inversely proportional to the slow-play rate.

27. A streaming media playing apparatus, comprising:

the downloading module is used for determining the downloading granularity of the current segment to be downloaded in the streaming media playing process and requesting to download the segment according to the downloading granularity;

and the playing module is used for playing the streaming media based on the received streaming media data.

28. An electronic device for streaming media playback, comprising a processor and a memory;

the memory stores an application program executable by the processor for causing the processor to execute the streaming media playing method according to any one of claims 1 to 26.

29. A computer-readable storage medium having stored therein computer-readable instructions for executing the streaming media playing method according to any one of claims 1 to 26.

Technical Field

The present invention relates to mobile communication technologies, and in particular, to a method and an apparatus for playing streaming media.

Background

With the development of network technology in recent years, the use of streaming media is spread in all aspects of people's work and life, and various terminals such as televisions, mobile phones, notebooks, etc. use streaming media to play videos. Correspondingly, with the wide application of streaming media playing, the requirements of people on the video playing quality, including the requirements on definition, fluency, instantaneity and the like, are higher and higher.

Currently, to meet the high quality requirement of video playback, 4K and 8K high-resolution streaming media are increasingly popular. The inventor finds that the high-quality requirement of video playing cannot be met when the existing streaming media playing method is used for playing high-resolution streaming media such as 4K and 8K.

Disclosure of Invention

In view of the above, the present invention provides a method and an apparatus for playing a streaming media, which can improve the playing quality.

In order to achieve the purpose, the technical scheme provided by the invention is as follows:

a streaming media playing method, comprising:

in the process of streaming media playing, determining a downloading granularity for a segment to be downloaded currently, and requesting to download the segment according to the downloading granularity;

and playing the streaming media based on the received streaming media data.

A streaming media playing apparatus, comprising:

the downloading module is used for determining the downloading granularity of the current segment to be downloaded in the streaming media playing process and requesting to download the segment according to the downloading granularity;

and the playing module is used for playing the streaming media based on the received streaming media data.

An electronic device for streaming media playback includes a processor and a memory;

the memory stores an application program executable by the processor for causing the processor to execute the streaming media playing method as described above.

A computer readable storage medium having stored therein computer readable instructions for performing the streaming media playing method as described above.

In summary, in the streaming media playing scheme provided in the embodiment of the present invention, when downloading a streaming media segment, a download granularity needs to be determined for the segment first, and according to the download granularity, the streaming media segment is requested to be downloaded, so as to refine the granularity of data download requested from the server each time. Therefore, the transmission time of the data downloaded by a single request can be shortened, the synchronism of the audio and video data downloading is enhanced, and the fluency and the instantaneity of the streaming media playing based on the downloaded data can be further improved. Therefore, under a high-definition streaming media playing scene, the streaming media playing scheme provided by the application can be applied to effectively improve the streaming media playing quality.

Drawings

Fig. 1 is a flow chart of a streaming media playing method according to an embodiment of the present invention;

FIG. 2 is a flow chart of a code rate switching control method according to an embodiment of the present invention;

FIG. 3 is a schematic flow chart of a real-time bandwidth monitoring method according to an embodiment of the present invention;

FIG. 4 is a schematic flow chart of a second method for monitoring a real-time bandwidth according to an embodiment of the present invention;

FIG. 5 is a schematic flow chart of one embodiment of step r1 in FIG. 4;

FIG. 6 is a schematic flow chart of one embodiment of step r2 in FIG. 4;

fig. 7 to 9 show comparison between a real-time bandwidth monitoring method two according to an embodiment of the present invention and bandwidth measurement results in the prior art under different live broadcast stream bit rates and network bandwidth conditions;

FIG. 10 is a schematic structural diagram of an apparatus according to an embodiment of the present invention;

fig. 11 is a schematic diagram illustrating comparison between the effect of the fragment splitting request downloading according to the embodiment of the present invention and the effect of the whole fragment request downloading according to the prior art.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.

Aiming at the problem that the playing quality cannot be guaranteed when the high-definition video is played by the existing streaming media playing scheme, the inventor finds out through research and analysis that the main reasons are as follows:

in the existing streaming media playing scheme, when streaming media file data is downloaded, a client player finds out an address where a fragment file of each fragment is located according to description of an index file, and applies for downloading the fragment to a server in a whole section according to the address. Thus, a single download unit corresponds to an entire piece of data. For a high-resolution video, the data size of a single segment is large, so that the download duration of the single segment is long, and the total download duration of two audio and video segments which need to be played synchronously is also long. When the streaming media is played, the video segment and the audio segment which are matched in time need to be completely downloaded, and then synchronous playing can be performed, so that after the downloading of one audio or video segment is completed, the downloading of another video or audio segment which is matched in time needs to be completed. Therefore, when the downloading time length of a single segment becomes long, the time length for waiting for the corresponding synchronous segment also becomes long, so that on one hand, more cache space is needed to store the downloaded data which cannot be synchronously played, and on the other hand, due to the overlong synchronous waiting time, the received video segment or audio segment cannot be timely played, so that the synchronism of audio and video downloading is poor, and the smoothness and the real-time performance of playing are influenced.

Based on the analysis, the embodiment of the invention optimizes the data downloading link to overcome the influence of the whole downloading streaming media segment on the streaming media playing quality, thereby improving the streaming media playing quality.

Fig. 1 is a schematic flow chart of a streaming media playing method according to an embodiment of the present invention, and as shown in fig. 1, the streaming media playing method implemented by this embodiment mainly includes the following steps:

step 101, in the process of playing the streaming media, determining a downloading granularity for a segment to be downloaded currently, and requesting to download the segment according to the downloading granularity.

In this step, in the process of playing the streaming media, when each segment is downloaded, the segment needs to be downloaded based on a certain download granularity, so that the segmented downloading of the segment is realized, and thus, the downloading requested each time is not the whole segment, but part of the data in the segment. Fig. 11 is a schematic diagram showing a comparison between a segment segmentation request download according to an embodiment of the present invention and an entire segment request download according to an existing scheme, and as shown in fig. 11, a download granularity according to an embodiment of the present invention is smaller, so that a transmission duration of data downloaded in a single request can be greatly reduced, and a synchronization waiting time for downloading audio and video data is further reduced, so that downloaded audio and video data can be decoded and played in time, synchronization of audio and video data download is effectively enhanced, data in a cache can be played in time, and a cache space requirement is reduced. Therefore, the fluency and the real-time performance of the streaming media playing based on the downloaded data can be improved, and especially the live broadcast starting delay can be effectively reduced.

The above segments can be obtained by using a segment division method of the existing streaming media file, and are not described herein again.

In an embodiment, in order to meet the processing requirements of data with different structure types and enhance the rationality of segmenting and downloading the segment, considering that the segment may contain data with different structure types, the data with different structure types in the segment file may be distinguished in step 101, and multiple download granularities are set.

Preferably, the download granularity may include: a first download granularity and a second download granularity.

Wherein the first download granularity is used for downloading segment description data of a segment, and the second download granularity is used for downloading a content data frame of the segment.

Here, the description information of the segment, that is, the segment description data, and the content data frame, that is, the media data to be decoded and played in the segment. Considering that the basic data unit is bits for the segment description data, and the data frame is the basic processing object for the actual media data, in order to ensure the rationality of the download granularity, the download granularity is separately configured for the segment description data and the content data frame.

In practical application, the first download granularity may be determined according to a preset granularity value or in combination with the current bandwidth, specifically as follows:

the method comprises the following steps: and setting the first downloading granularity as a preset segment description data downloading granularity value.

Here, when the download granularity value of the segment description data is too large, the download data amount of a single request is too large, and the advantage of the segment segmentation download cannot be effectively exerted, whereas when the download granularity value of the segment description data is too small, the download data amount of a single request is too small, and further the download request overhead is too large, and specifically, a user or a person skilled in the art may set an appropriate download granularity value of the segment description data according to experience or actual download requirements based on the above rules.

In consideration of the structural characteristics of the segment description data, it is preferable that the unit of the segment description data download granularity value is Byte in order to facilitate the segmentation and download of the segment description data. For example, 500Byte may be set, but is not limited thereto.

The second method comprises the following steps: and calculating to obtain the first download granularity according to the current bandwidth and the preset first granularity duration.

Specifically, a product of the current bandwidth and the first granularity duration may be calculated to obtain the first download granularity.

Here, the first download granularity is calculated based on the current download bandwidth and the preset granularity duration, so that the download duration of the data requested to be downloaded each time is the same as or close to the first granularity duration, and the first granularity duration can be utilized, so that the download duration of the data requested to be downloaded at a single time has controllability, further the fluency of fragment downloading and processing can be ensured, and the advantage of fine-granularity segmentation downloading of the fragment description data can be effectively exerted.

When the first granularity time is too long, the data amount downloaded by a single request is too large, and the advantage of segmenting and downloading the fragments cannot be effectively exerted, otherwise, when the first granularity time is too short, the data amount downloaded by the single request is too small, and further the downloading request cost is too large. Specifically, the first granularity duration may be set by a user or a person skilled in the art according to the network condition and the playing requirement of the actual application scenario, in combination with the above rule.

In practical applications, the second download granularity may be determined according to a preset granularity value or in combination with the current bandwidth, specifically as follows:

and the first method sets the second downloading granularity as a preset data frame downloading granularity value.

Here, when the data frame download granularity value is too large, the data amount of single request download is too large, and the advantage of segment segmentation download cannot be effectively exerted, whereas when the data frame download granularity value is too small, the data amount of single request download is too small, and further the download request overhead is too large, and specifically, a user or a person skilled in the art may set an appropriate data frame download granularity value according to experience or actual download requirements based on the above rules.

Considering the structural characteristic that the content data frame is composed of a plurality of frames, preferably, in order to simplify the operation overhead, the content data frame is convenient to be segmented and downloaded, and the unit of the data frame downloading granularity value is a frame, namely a plurality of frames. Of course, the unit of the data frame download granularity value may also be set as Byte. When downloading is requested based on the data frame downloading granularity value, the data frame downloading granularity value is converted into the closest data frame quantity based on the data quantity of the current data frame to be downloaded provided in the data frame index table, and the content data is requested to be downloaded based on the quantity. Therefore, the integrity of each downloaded data frame can be ensured, and the subsequent decoding processing is facilitated.

In addition, in one embodiment, in order to facilitate the subsequent decoding of the downloaded data, the data frame download granularity value may also be set based on the number of data frames of a single decoding unit, so that the number of data frames requested to be downloaded at each time is an integer multiple of the number of data frames in the single decoding unit.

And secondly, calculating to obtain a second download granularity according to the current bandwidth and a preset second granularity duration.

Specifically, the product of the current bandwidth and the second granularity duration may be calculated to obtain the second download granularity.

In the second method, the second download granularity is dynamically set based on the current bandwidth, so that the download time length of the data requested to be downloaded each time is the same as or close to the second granularity time length, the second granularity time length can be utilized, the download time length of the data requested to be downloaded at a single time has controllability, the fluency of fragment downloading and processing can be further ensured, and the advantage of performing fine-granularity segmentation downloading on the fragment description data is effectively exerted.

When the duration of the second granularity is too long, the data amount downloaded by a single request is too large, and the advantage of segmenting and downloading the fragments cannot be effectively exerted, whereas when the duration of the second granularity is too small, the data amount downloaded by the single request is too small, and further the downloading request cost is too large. Specifically, the second granularity duration may be set by a user or a person skilled in the art to a suitable value, such as 500ms, according to the network condition and the playing requirement of the actual application scenario, but is not limited thereto.

Further, in step 101, when downloading a segment, in order to facilitate downloading of a content data frame in the segment, in the downloading process of the segment description data, when it is detected that the download of the trun structure data in the segment description data is completed, the data frame index table may be immediately constructed based on the trun structure data. Wherein, the trunk structure (Track Fragment Run Box) is used to describe a set of continuous data frame information of the current Fragment, and it gives specific information of all data frames in the Fragment including: position offset in the file, data frame size, data frame duration, key frame flag, etc. The data frame index table will include the relevant information of each media data frame in the corresponding segment. In this way, when the content data frames of the subsequent downloading segment are downloaded, the data frames in the table can be sequentially downloaded according to the second downloading granularity based on the data frame index table. The specific method for constructing the data frame index table is known to those skilled in the art, and is not described herein again.

In practical application, in order to meet different playing requirements of users, for the same segment, data corresponding to different resolutions is stored in the streaming media server, and the higher the resolution is, the larger the corresponding code rate is. Considering that the network bandwidth is dynamically changed in practical application, in order to further improve the streaming media playing quality, a code rate switching control mechanism can be introduced in the streaming media playing process, and the code rate switching can be triggered in time according to the actually measured bandwidth, so that the code rate of the downloaded streaming media data is matched with the actual transmission bandwidth, that is, a clip file with high code rate can be downloaded under the condition of large bandwidth, and a clip file with low code rate can be downloaded under the condition of small bandwidth. Thus, on the premise of ensuring the downloading speed, a streaming media file with a higher resolution can be obtained to improve the playing effect of the streaming media as much as possible, as shown in fig. 2, the specific code rate switching control method mainly includes the following steps x1 to x 3:

and step x1, monitoring the real-time bandwidth based on the downloading condition of the streaming media data.

Here, during the streaming media playing process, the bandwidth measurement needs to be performed in real time, so as to determine the target bandwidth matching with the actual network transmission condition in the subsequent step based on the bandwidth measurement.

Specifically, this step can be implemented by using the existing real-time bandwidth measurement method.

Further, the inventors found in the process of implementing the present application that: in practical applications, under the influence of network fluctuation, the transmission rate of some data packets may not reflect the real network bandwidth, and in addition, the transmission rate of data packets with too small data amount may not match the real bandwidth. The existing bandwidth measurement scheme often ignores factors affecting accuracy of bandwidth measurement, such as network fluctuation and undersize data packets, and performs bandwidth measurement based on transmission conditions of all received data packets, thereby causing inaccurate bandwidth measurement results.

For this reason, in order to obtain a more accurate real-time bandwidth, several monitoring real-time bandwidths described below may be further employed.

As shown in fig. 3, the first real-time bandwidth monitoring method mainly includes the following steps y 1-y 2:

step y1, when receiving the data packet of the streaming media, judging whether the bandwidth of the data packet needs to be filtered according to a preset bandwidth filtering condition; and if not, updating the current bandwidth measurement parameter according to the downloading condition of the data packet.

In this step, when receiving a data packet returned by the server according to the data download request, the client filters whether the bandwidth of the received data packet can be used for bandwidth measurement according to a preset bandwidth filtering condition, and correspondingly updates the bandwidth measurement parameter in real time, so as to filter out the data packet which can affect the accuracy of bandwidth measurement, enhance the effectiveness of the parameter for bandwidth measurement, and further improve the accuracy of bandwidth measurement.

Here, the fact that the bandwidth of the data packet needs to be filtered means that the bandwidth of the data packet is filtered out and cannot participate in bandwidth measurement.

Preferably, for better filtering effect, the bandwidth filtering condition may be set based on the transmission time and/or transmission size of the received video data.

In one embodiment, the bandwidth filtering condition may be set to include the following:

the data packet belongs to preset small data volume file data, and the small data volume file data comprises index file data and initialization fragment data; and/or the current time interval from the latest request for downloading the data is less than a preset filtering time threshold.

In the bandwidth filtering condition, "the time interval between the current time and the latest data download request is smaller than the preset filtering time threshold," the time interval between the current time and the latest data download request "represents the transmission duration of the currently received data packet, and when the condition is satisfied, it indicates that the transmission duration of the currently received data packet is very short, which may be the case that the data number of the download unit itself is very small, or the remaining data amount of the download unit after resuming the download is very small, or the transmission rate is abnormally fast due to the influence of network fluctuation, and these situations with very short transmission duration usually cannot reflect the real bandwidth situation, so the received data bandwidth under the situation can be considered to be filtered out, so as to improve the accuracy of bandwidth measurement.

For the filtering time threshold, a person skilled in the art can set a reasonable value according to actual needs, so that the data bandwidth which cannot represent the current real bandwidth condition can be filtered by using the filtering time threshold. Such as, but not limited to, 100 ms.

In an embodiment, for the bandwidth measurement parameter that needs to be updated in real time, the method may specifically include: the transmission time to be used for bandwidth sampling and the total amount of data that has been received without the need to filter bandwidth.

Accordingly, the following method may be specifically adopted to update the current bandwidth measurement parameter:

setting the transmission time to be used for bandwidth sampling as a time interval between the current time and the latest request for downloading; and increasing the size of the data packet to the current total data amount.

Here, the transmission time to be used for bandwidth sampling is set as the time interval between the current time and the latest requested data, where the latest requested data may be a data unit requested to be downloaded for the first time, or a data download request triggered by resuming downloading after the downloading is suspended. Therefore, the transmission time to be used for bandwidth sampling can always record the transmission time of the latest request data through the updating, so that the idle time after the downloading is suspended can be eliminated, and the influence of the idle time on the accuracy of bandwidth measurement is avoided.

In the above updating method, for data that does not need to be filtered, the data size of the data is added to the current total data amount, so that when the bandwidth is measured, the difference between the parameter and the total data amount currently participating in the bandwidth measurement can be calculated, the data amount that does not need to be filtered and does not participate in the bandwidth measurement is obtained, and then bandwidth sampling can be performed based on the data amount.

Accordingly, in an embodiment, if the bandwidth filtering condition includes that the "time interval between the current time and the latest request for downloading the video data is less than the preset filtering time threshold", in step y1, it is detected that the bandwidth of the data packet needs to be filtered currently, and the downloading unit to which the data packet belongs has a downloading pause operation, and the following conditions are satisfied: when the time interval between the current time and the latest request for downloading is less than the preset filtering time threshold, setting the total data amount which is currently involved in bandwidth measurement as the total data amount which is currently received and does not need to filter the bandwidth, and timing the current sampling period to zero.

In the above method, it is considered that if the download unit to which the data packet belongs has a pause download operation, and the following conditions are satisfied: the time interval between the current time and the latest time for requesting downloading the data is smaller than the preset filtering time threshold, which indicates that the currently received data needs to be filtered because the residual data of the downloading unit where the data is located is smaller when the downloading is suspended and the data volume transmitted after the downloading is resumed is smaller. In this case, the download time of the download unit cannot be accurately used to calculate the bandwidth, and therefore, all data information downloaded and not downloaded by the download unit needs to be filtered. Therefore, in the above method, it is necessary to set the current "amount of data that does not need filtering and does not participate in bandwidth measurement" to zero, that is, the "total amount of video data that currently participates in bandwidth measurement" is set to "total amount of video data that is currently received and does not need filtering, so that the difference between the two amounts is zero. In this way, it can be ensured that the "transmission time to be used for bandwidth sampling" and the "amount of data which does not need filtering and does not participate in bandwidth measurement" counted thereafter are consistent, and they both correspond to the newly received unfiltered and sampled data, so that it can be ensured that the subsequent sampling result is not affected by the current data filtering.

And y2, judging whether the current bandwidth meets the preset bandwidth measurement condition in real time according to the bandwidth measurement parameter and the preset sampling period, if so, triggering bandwidth sampling according to the bandwidth measurement parameter, acquiring the number of current bandwidth sampling points, and calculating the current bandwidth measurement value according to the sampled bandwidth value and the number of the bandwidth sampling points.

The difference between the step and the prior art is that when the bandwidth measurement condition is judged to be satisfied according to the current bandwidth measurement parameter and the preset sampling period, the bandwidth measurement needs to be triggered without waiting for the completion of downloading of one or more downloading units and then triggering the bandwidth measurement, so that the real-time performance of the bandwidth measurement can be improved.

Specifically, in order to improve the real-time performance of the bandwidth, when calculating the current bandwidth measurement value, it is necessary to perform calculation based on the M bandwidth sample values sampled most recently. And if the bandwidth value sampled currently reaches the number of the bandwidth sampling points, M is the number of the bandwidth sampling points acquired currently, otherwise, M is the number of all the bandwidth values actually acquired currently.

In particular, when the current bandwidth measurement value is calculated by using the M bandwidth sample values of the new sample, an average value may be calculated, but the method is not limited to this, and other calculation methods may be used to make the measurement result more match with the actual bandwidth.

In one embodiment, to further improve the timeliness of bandwidth measurement, the bandwidth measurement condition may be set to include the following:

the sampling period is currently reached; or the number of video data which do not participate in bandwidth measurement in the video data which are currently received and do not need to be subjected to bandwidth filtering is larger than a preset threshold value of the number of bandwidth measurements.

Wherein, the monitoring whether the sampling period is reached can be realized by a timer.

Based on the bandwidth measurement condition, when a preset sampling period is reached, or the number of received video data which do not need to filter bandwidth and do not participate in bandwidth measurement reaches a certain number, triggering needs to be considered for bandwidth measurement calculation.

The sampling period and the bandwidth measurement quantity threshold may be specifically set by those skilled in the art according to actual needs, as long as timeliness and accuracy of bandwidth measurement can be ensured.

In an embodiment, after determining that the preset bandwidth measurement condition is currently satisfied, the following method may be specifically adopted to trigger bandwidth sampling according to the bandwidth measurement parameter:

and if the transmission time to be used for bandwidth sampling is within a preset time range, calculating the bandwidth value of the current sampling according to the transmission time to be used for bandwidth sampling and the quantity of data which does not participate in bandwidth measurement in the data which is currently received and does not need bandwidth filtering.

In the above method, considering that the short transmission time of the data sample based on the bandwidth sampling may affect the authenticity of the sampling result, in order to further improve the effectiveness of the bandwidth sampling, the above time range is set to limit: bandwidth sampling can only be performed if the transmission time currently to be used for bandwidth sampling is within the time range. Specifically, in one embodiment, the time range is greater than or equal to a preset minimum sampling transmission time and less than or equal to the sampling period. Thus, the following can be avoided as much as possible: the transmission time corresponding to the sampling data is too short, so that the accuracy of the sampling result is influenced. For the minimum sampling transmission time, a suitable value can be set by a person skilled in the art according to the needs of an actual application scenario.

In an embodiment, in step y2, the current number of bandwidth sampling points may also be obtained based on the playing duration of the currently received and undisplayed data, so as to improve the accuracy of bandwidth measurement by dynamically obtaining the current number of bandwidth sampling points, and specifically, the following steps may be adopted to obtain the current number of bandwidth sampling points:

and step z1, dividing the playing time length of the currently received and undisplayed data by the sampling period to obtain the number of initial sampling points.

The method comprises the step of determining the number of initial sampling points based on the playing time corresponding to the currently cached undisplayed data, so that the number of the sampling points finally used for bandwidth measurement is obtained in the subsequent steps based on the number of the initial sampling points. Here, when the initial sampling point number is determined, the play duration of the current cache data is considered, so that the number of the sampling points finally used for bandwidth measurement can be matched with the duration of the current cache data, and thus the continuity and the real-time performance of bandwidth sampling can be guaranteed.

Step z2, if the playing time of the maximum download unit of the currently played file is less than the preset time threshold, determining the number of the bandwidth sampling points matched with the number of the initial sampling points according to the mapping relation between the number of the preset initial sampling points and the number of the bandwidth sampling points, and setting the number as the current number of the bandwidth sampling points, otherwise, setting the current number as the preset maximum threshold.

Here, in order to match the determined number of bandwidth sampling points with the transmission requirement of the currently played file, when the number of current bandwidth sampling points is set, the playing time length of the maximum download unit of the currently played file is considered, and when the playing time length is not too long, the number of current bandwidth sampling points is determined based on the number of initial sampling points obtained in step z1 according to the mapping relationship between the preset number of initial sampling points and the number of bandwidth sampling points. When the playing time of the maximum downloading unit of the currently played file is longer, the required bandwidth is larger, at this time, the number of the current bandwidth sampling points is set as the preset maximum sampling point number threshold value, and the accuracy of bandwidth measurement can be improved through the number of the sampling points as many as possible.

The mapping relation between the initial sampling point quantity and the bandwidth sampling point quantity can be set according to the proportional relation between the initial sampling point quantity and the bandwidth sampling point quantity, so that when the time length of data to be played in a cache region is long, the large bandwidth sampling point quantity can be set, and further the accuracy of bandwidth measurement can be improved on the basis of more sampling data as much as possible.

For the maximum sampling point number threshold, a suitable value can be specifically set by a person skilled in the art according to practical experience, as long as the requirement on the accuracy of bandwidth measurement can be met as far as possible.

In the first real-time bandwidth monitoring method, whether the bandwidth under the current data downloading condition can be used for bandwidth measurement is screened in real time to filter data which can influence the accuracy of bandwidth measurement, and the parameters for bandwidth measurement are updated based on the video data which is not filtered, so that the effectiveness and the accuracy of the bandwidth measurement parameters can be improved, factors which easily influence the accuracy of bandwidth measurement are excluded from the bandwidth measurement, and the parameters for bandwidth measurement can accurately reflect the real bandwidth, thereby improving the accuracy of bandwidth measurement. And when the method meets the preset bandwidth measurement condition, the bandwidth measurement is triggered immediately, so that the real-time performance of the bandwidth measurement can be improved. The real-time bandwidth monitoring method is particularly suitable for the http transmission scene adopting non-chunk coding.

As shown in fig. 4, the second real-time bandwidth monitoring method mainly includes the following steps r 1-r 2:

and r1, filtering the data packets which are not sent in saturation when the data packets of the streaming media are received.

In one embodiment, this step may specifically filter the data packets sent without saturation by using the following method:

calculating the preset time T_oAnd if the data size of the current data packet is smaller than or equal to the data size average value, filtering the current data packet. As shown in fig. 5, in one embodiment, the method specifically includes the following steps:

step r11, calculating the transmission time of the current data packet, namely: the time taken for the packet network to transmit.

Step r12, inserting the data size and transmission time of the current data packet into a bidirectional queue Q_oThe tail of (2).

In one embodiment, this step can be specifically realized by the following method:

if the data is received for the first time after the request is sent, the transmission time is as follows: time T of currently received data packet_nowTime T of requesting data_begin；

If the data is not the first packet (i.e. it does not belong to the first time data is received after the request is sent), the transmission time is: time T of currently received data_nowTime T of last data reception_previous。

Step r13, if insert queue Q_oThe time of the queue element at the head is greater than the preset time T from the current time_oThen indicate queue Q_oThe duration of the accumulated queue element is longer than the set time T_oAt this point, some old queue elements should be discarded, i.e., the queue element at the head of the queue is discarded, and step r14 is performed; otherwise, step r15 is performed directly.

The preset time T_oSuitable values, such as, but not limited to, 1.5s, can be set empirically by one skilled in the art.

Step r14, delete queue Q_oA queue element at the head.

Step r15, calculating the queue Q_oAverage value of medium packet size: s_avg＝S_totalAnd/count. Wherein S is_totalIs queue Q_oThe total number of all data packet sizes in the queue, i.e., the total data size, count is the queue Q_oNumber of middle queue elements.

Step r16, if the data size of the data packet in step r11 is larger than the queue Q_oMean value of median data size S_avgThen step r17 is executed to insert the data size and transmission time of the data packet as a queue element into the bidirectional queue Q_fThe tail of (2). Similar to step r13, if queue Q is inserted_fThe time of the queue element at the head is more than the preset time T from the current time_fThen indicate queue Q_fThe duration of the medium accumulation queue element is greater than the preset time T_fAt this point, the queue element at the head of the queue should be discarded, i.e., queue Q is deleted_fA queue element at the head.

The preset time T_fSuitable values, such as, but not limited to, 1.5s, can be set empirically by one skilled in the art.

Through the above steps r11 to r17, the client can determine whether the received data packet is an unsaturated transmission data packet, and the data size and transmission time information of the unsaturated transmission data packet cannot enter the queue Q_fThus, queue Q_fThe data packets which are sent in an unsaturated way are filtered from the buffered data packets.

And step r2, finding a stable area of the data size and the transmission rate in the filtered data packet, and calculating the current bandwidth based on the stable area of the data size and the transmission rate.

This step is used to calculate the current bandwidth based on the filtered data packet currently cached. Specifically, the above calculation may be triggered in real time according to a certain sampling period. Specifically, the sampling period may be set to a suitable value according to a real-time measurement requirement, such as 250ms, 500ms, and the like, but is not limited thereto.

In one embodiment, step r2 can be implemented by the following method:

step r21, for the queue Q_fSorting by data size.

Step r22, traversing the sorted queue Q_fAcquiring a group of queue elements with the same data size and the maximum number, and recording the queue elements in a queue Q_fPosition (2): starting position P_bEnd position P_eAnd the number of elements C_max。

Step r23, if said C_maxGreater than or equal to a predetermined value C, indicating that this is foundA set of queue elements constitutes a stable region of data size, based on which bandwidth can be calculated, step r24 is performed; otherwise, if C_maxIf the value is less than the preset value C, step r27 is executed.

The preset value C can be set to a suitable value, such as 30, by a person skilled in the art according to experience, but is not limited thereto.

Step r 24: pair queue Q_fMiddle P_bTo P_eThe queue elements in the region are ordered by the size of the transmission time.

Step r 25: and acquiring a stable region of the transmission rate based on the sorting result of the step r 24.

The transmission time reflects the transmission rate, and based on the sorting result of step r24, the following two preferable methods for obtaining the stable region of the transmission rate can be specifically adopted:

the stable region acquisition method comprises the following steps: and (5) simple calculation. For example, directly taking N1 queue elements at the first intermediate position and N2 queue elements at the second intermediate position of the sorting result, the N1+ N2 queue elements form a stable region of the transmission rate; wherein the first intermediate position precedes the second intermediate position in the ranking result.

Here, the first intermediate position and the second intermediate position may be more reasonable positions obtained through experimental analysis. N1 and N2 can be set by those skilled in the art according to the actual application scenario.

Considering that the impact values with too high or too low rate may occur at the initial and final positions, the impact values should be removed, and the sudden change of the bandwidth at the middle 1/2 may occur and is not suitable for calculation; 1/4 and 3/4 are two regions where the bandwidth rate is stable, and these values can be used to make the bandwidth measurement more accurate, so it is preferable to set the first middle position as 1/4 of the queue and the second middle position as 3/4 of the queue, but not limited to this, and will not be described herein.

And a second stable region acquisition method: and (4) performing complex calculation, and acquiring a stable region of the transmission rate by calculating the variance value. The method comprises the following specific steps:

step 1) calculating P_bTo P_eMiddle position P of the region_m＝(P_e–P_b) /2 if P_mInstead of being an integer, rounding may be up or down.

Step 2) P based on step r24_bTo P_eSequencing result of region, traverse queue Q_fIn the slave P_bTo P_m-10 queue elements, calculating the variance of the transmission time of each queue element and the transmission time of the next 9 data, and obtaining a group of queue elements with the minimum variance.

Step 3) P based on step r24_bTo P_eSequencing result of region, traverse queue Q_fIn the slave P_m+1 to P_e-10 queue elements, calculating the variance of the transmission time of each queue element and the transmission time of the next 9 data, and obtaining a group of queue elements with the minimum variance.

And the two groups of queue elements obtained in the step 2) and the step 3) form a stable region of the transmission rate.

Step r 26: and based on the stable region of the transmission rate obtained in the step r25, accumulating the data size to obtain S, accumulating the transmission time to obtain T, returning to the state where B is S/T as the current bandwidth, and ending the current flow.

Step r 27: using queue Q_oAnd calculating the bandwidth.

In particular, the accumulation queue Q_oThe total data size S is obtained from the data size in the queue, and a queue Q is accumulated_oThe total transmission time T is obtained from the transmission time in (1), and B ═ S/T is calculated as the current bandwidth.

This concludes the current flow.

In the second real-time bandwidth monitoring method, when the client receives the data packet, the data packet sent by the client is filtered out at first; then, in the filtered data packet, a stable region of the data size and the transmission rate is searched, and the current bandwidth is calculated based on the stable region of the data size and the transmission rate, so that the influence of a large amount of data fragments and network fluctuation or instable instantaneous transmission rate caused by server performance can be effectively avoided, and the accuracy of bandwidth measurement can be improved. The method is particularly suitable for http transmission scenes adopting chunk coding.

Fig. 7 to 9 show that, under different live stream bit rates and network bandwidths, comparing the real-time bandwidth monitoring method two with the bandwidth measurement result in the prior art shows that the real-time bandwidth monitoring method two has an obvious performance improvement compared with the prior bandwidth measurement method. Wherein:

dark lines indicate: the existing measurement method directly obtains a bandwidth value through data size/transmission time;

the light line indicates: and (5) calculating the bandwidth value by using a real-time bandwidth monitoring method II.

In fig. 7, the bit rate of the live stream is: 1500kbps, network bandwidth: 800 kbps.

The measurement result of the conventional measurement method is about 700kbps, and the measurement result of the second real-time bandwidth monitoring method is about 750 kbps. The difference between the two measurement results is not much, but the real-time bandwidth monitoring method II is closer to the actual network bandwidth.

In fig. 8, the bit rate of the live stream is: 2000kbps, network bandwidth: 5000 kbps.

The measurement result of the existing measurement method is about 2000kbps, which is close to the code rate of the live stream, and the network bandwidth can not be accurately measured; and the measurement result of the second real-time bandwidth monitoring method is 4700kbps, which is close to the actual network bandwidth.

In fig. 9, the bit rate of the live stream is: 1000kbps, network bandwidth: cycling from 200kbps to 8 Mbps.

The maximum bandwidth measured by the existing measuring method is about 3Mbps, and the maximum network bandwidth cannot be accurately measured; the maximum bandwidth measured by the real-time bandwidth monitoring method two is about 8Mbps and is close to the actual maximum bandwidth value.

And step x2, determining the matched target code rate based on the real-time bandwidth.

In an embodiment, the following method may be adopted to determine the matched target bitrate based on the real-time bandwidth in combination with the actual scene application requirement:

and step x21, predicting the target bandwidth according to the real-time bandwidth.

In one embodiment, in order to match the target bandwidth predicted in this step with the playing requirement of the actual application scene, a matching bandwidth prediction policy may be selected based on the current scene type, and the matching target bandwidth may be predicted according to the bandwidth prediction policy and the current real-time bandwidth.

And step x22, determining a matched switching strategy according to the target bandwidth and the optimal characteristics corresponding to the current scene.

In this step, an appropriate switching strategy needs to be selected in combination with the optimal characteristics corresponding to the current scene, so that code rate switching can be performed in time according to the playing needs of the actual application scene.

In particular, the preferred characteristics may include performance parameters such as resolution, playout delay, and/or fluency. Wherein the fluency can be characterized by the number of times of caching within a certain time.

And step x23, determining the current target code rate according to the switching strategy.

Step x3, when the target code rate is different from the code rate of the current downloading segment, triggering code rate switching to request the data to be downloaded according to the segment corresponding to the target code rate.

In consideration of the fact that in practical applications, in the downloading process of step 101, there may be a case that the server has not generated the requested data, in an embodiment, at this time, the server may return a message of downloading failure to the client, and the client re-requests to download the corresponding data after receiving the message of downloading failure.

And 102, playing the streaming media based on the received streaming media data.

The step may specifically be performed by the existing method based on the requested download data.

In one embodiment, considering that when a rate switch occurs, the decoder needs to reset the pre-processing for the new rate, the decoding delay is increased, and the playback pause is generated. Especially, when the code rate is switched frequently, the influence of the pause on the playing quality is more serious. Therefore, when code rate switching occurs, the code rate switching time can be reduced by using a mode of cooperative processing of a plurality of decoders, and the following method can be specifically adopted when streaming media is played:

when code rate switching occurs, loading the downloaded data corresponding to the switched code rate to a first decoder which is currently in a decoding stop state, and switching a second decoder which is currently in a decoding start state to the decoding stop state when the second decoder finishes decoding a currently played decoding unit so as to stop the second decoder from decoding, and switching the first decoder to the decoding start state so as to start the first decoder to decode.

By adopting the method, when code rate switching occurs, the downloaded decoding unit data with the new code rate is preloaded into other decoders which do not start decoding, so that the preprocessing is reset aiming at the new code rate in advance, and when the current decoder finishes processing the decoding unit data with the old code rate, the decoder which finishes data preloading is started to decode.

In an actual streaming media playing scene, when a server side inserts new streaming media data (such as an advertisement stream) into a live stream, the method can be used for realizing the non-pause insertion of the new streaming media by utilizing the cooperative processing of a plurality of decoders. The specific process is as follows:

when a client receives an index file (such as MPD) of a new streaming media file inserted in a live stream by a server, the MPD is analyzed to obtain a corresponding code rate, the new streaming media file is downloaded according to the code rate, if the code rate is different from the code rate of currently decoded data, the downloaded new streaming media file is loaded to a first decoder which is currently in a decoding stop state, and when a second decoder which is currently in a decoding start state completes decoding of a currently played decoding unit, the second decoder is switched to the decoding stop state to stop the second decoder from decoding, and the first decoder is switched to the decoding start state to start the first decoder to decode.

In practical application, before playing streaming media, the cache parameters may be configured according to a certain initial configuration policy (e.g., an upper limit of a cache that is as small as possible) based on streaming media initial information (e.g., manifest) downloaded from a server, and during playing streaming media, the cache parameters may be dynamically configured according to a real-time target code rate, that is, when code rate switching occurs, the cache parameters need to be configured according to a new target code rate, so that the cache configuration matches with actual scene application needs, and thus, a storage space is not wasted during playing streaming media, and an actual playing performance requirement can be satisfied.

In an actual application scenario of streaming media playing, the situation that playing delay is large due to the influence of network speed is often considered, so that the real-time performance of streaming media playing is influenced. For this reason, in order to further enhance the real-time performance of streaming media playing, in an embodiment, in the playing process of step 102, a mechanism for automatically triggering double-speed playing based on real-time playing delay is introduced to achieve a target playing delay, so that the playing requirement of low delay and ultra-low delay can be met, and the specific method is as follows:

step 1021, monitoring the real-time playing delay of the streaming media according to a preset delay monitoring period.

Here, the monitoring of the real-time playing delay can be implemented by using the existing method, and is not described herein again.

The delay monitoring period can be specifically set according to the accuracy requirement of actual real-time playing delay.

Step 1022, determining a corresponding playing rate according to the real-time playing delay, and playing according to the playing rate.

Specifically, the following methods may be adopted to determine the corresponding playing rate according to the real-time playing delay:

the first method for determining the playing speed comprises the following steps:

and if the real-time playing delay exceeds a specified tolerable delay range, determining the corresponding playing rate as the fast playing rate.

And if the real-time playing delay is within the tolerable delay range, determining the corresponding playing rate as a normal rate.

Specifically, the tolerable delay range may be specifically set based on the current target delay setting, for example, the tolerable delay range may be set to be 0.95 times to 1.05 times of the target delay, but is not limited thereto.

In an embodiment, the fast playing rate may specifically be determined by the following method:

inquiring a mapping relation between a difference value between preset real-time playing delay and target delay and a fast playing rate to obtain a fast playing rate corresponding to the difference value between the real-time playing delay and the target delay; wherein the difference value in the mapping relation is in direct proportion to the fast playing rate.

The mapping relation needs to satisfy the relationship that the difference value between the real-time playing delay and the target delay is in direct proportion to the fast playing rate, so that when the difference value between the real-time playing delay and the target delay is larger, the real-time playing delay can be quickly close to the target delay by adjusting to the fast playing rate; otherwise, when the difference between the real-time playing delay and the target delay is small, fine adjustment can be performed by using a small fast playing rate.

In practical applications, the mapping relationship can be set by those skilled in the art according to practical needs. For example, it is possible to set: when the difference between the real-time playing delay and the target delay is large (for example, the real-time playing delay is more than or equal to 10 times of the target delay), playing at the speed of 2.1-16 times; when the difference between the real-time playing delay and the target delay is large (for example, the real-time playing delay is greater than 2 times of the target delay and less than or equal to 10 times of the target delay), playing at the speed of 1.6-2 times; when the difference between the real-time playing delay and the target delay is small (for example, the real-time playing delay is greater than 1.05 times of the target delay and less than or equal to 2 times of the target delay), the speed of 1.1-1.5 times is used for playing, but not limited to the above.

In an embodiment, the slow play rate may be determined by specifically using the following method:

inquiring a mapping relation between a preset difference value between the consumption rate and the download rate and the slow play rate to obtain the slow play rate corresponding to the current difference value between the consumption rate and the download rate; wherein the difference in the mapping relationship is inversely proportional to the slow-play rate.

In practical applications, a mapping relationship between the difference between the consumption rate and the download rate and the slow-play rate may be set by those skilled in the art according to practical needs. In the mapping relationship, the slow playing rate may range from 0.1 to 0.9 times, but is not limited thereto.

And in the second playing speed determining method, the playing speed is controlled by utilizing the real-time playing delay, and the playing speed is controlled based on the comparison relation between the consumption speed of the cache data and the downloading speed, so that when the amount of the cache data is less, the playing speed is reduced, and the consumption speed of the cache data is slowed down, thereby avoiding the condition that the cache data is empty, further avoiding the playing jam caused by the empty cache, and enhancing the playing fluency. Preferably, the method can be specifically realized by the following steps:

if the real-time playing delay exceeds a specified tolerable delay range and the consumption rate of the current streaming media cache data is not greater than the downloading rate, determining the corresponding playing rate as a fast playing rate;

if the real-time playing delay is within the tolerable delay range and the consumption rate of the current streaming media cache data is not greater than the downloading rate, determining that the corresponding playing rate is a normal rate;

and if the difference value between the consumption rate of the current streaming media cache data and the downloading rate is greater than a preset difference threshold value, determining that the corresponding playing rate is a slow playing rate. The difference threshold is greater than or equal to zero.

Here, different from the first play rate determining method, when the consumption rate of the current streaming media cache data is greater than the download rate, the corresponding play rate is determined to be a slow play rate. Therefore, the playing speed is set to be the slow playing speed under the condition that the consumption speed of the current streaming media cache data is larger than the downloading speed, so that the playing fluency can be preferentially ensured.

Corresponding to the above embodiment of the streaming media playing method, an embodiment of a streaming media playing apparatus is further provided in the embodiment of the present invention, as shown in fig. 10, and the method mainly includes:

a downloading module 1001, configured to determine a downloading granularity for a segment to be downloaded currently in a streaming media playing process, and request to download the segment according to the downloading granularity;

the playing module 1002 is configured to play the streaming media based on the received streaming media data.

In one embodiment, the download granularity includes: a first download granularity for downloading segment description data for a segment and a second download granularity for downloading content data frames for a segment.

In an embodiment, the downloading module 1001 is specifically configured to determine the downloading granularity, and includes:

setting the first downloading granularity as a preset segment description data downloading granularity value; or, the first download granularity is calculated according to the current bandwidth and a preset first granularity duration;

setting the second download granularity as a preset data frame download granularity value; or, the second download granularity is calculated according to the current bandwidth and a preset second granularity duration.

In an embodiment, the downloading module is further configured to monitor a real-time bandwidth based on a downloading condition of the streaming media data, and determine a matched target bitrate based on the real-time bandwidth; and when the target code rate is different from the code rate of the current downloading segment, triggering code rate switching to request the data to be downloaded according to the segment corresponding to the target code rate.

In an embodiment, the playing module 1002 is further configured to, when code rate switching occurs, load downloaded data corresponding to the switched code rate to a first decoder currently in a decoding stop state, and when a second decoder currently in a decoding start state completes decoding of a currently played decoding unit, switch the second decoder to the decoding stop state to stop the second decoder from decoding, and switch the first decoder to the decoding start state to start the first decoder to decode.

In an embodiment, the playing module 1002 is further configured to configure the buffering parameter according to a new target bitrate when bitrate switching occurs.

In an embodiment, the downloading module 1001 is specifically configured to monitor a real-time bandwidth based on a downloading condition of the streaming media data, and includes:

when a data packet of the streaming media is received, judging whether the bandwidth of the data packet needs to be filtered according to a preset bandwidth filtering condition; if not, updating the current bandwidth measurement parameter according to the downloading condition of the data packet;

and judging whether the current bandwidth meets the preset bandwidth measurement condition in real time according to the bandwidth measurement parameter and a preset sampling period, if so, triggering bandwidth sampling according to the bandwidth measurement parameter, acquiring the number of current bandwidth sampling points, and calculating the current bandwidth measurement value according to the sampled bandwidth value and the number of the bandwidth sampling points.

In one embodiment, the bandwidth filtering condition includes:

the data packet belongs to preset small data volume file data, and the small data volume file data comprises index file data and initialization fragment data;

and/or the current time interval from the last request for downloading is less than a preset filtering time threshold.

In one embodiment, the bandwidth measurement parameters include a transmission time to be used for bandwidth sampling and a total amount of data that has been received without requiring bandwidth filtering.

In an embodiment, the downloading module 1001 is specifically configured to update the current bandwidth measurement parameter, and includes:

setting the transmission time to be used for bandwidth sampling as a time interval between the current time and the latest request for downloading;

and increasing the size of the data packet to the current total data amount.

In an embodiment, the downloading module 1001 is specifically configured to, based on the downloading condition of the streaming media data, monitor a real-time bandwidth further includes:

when the bandwidth of the data packet needs to be filtered and the downloading unit to which the data packet belongs has the downloading pause operation, the following conditions are met: when the time interval between the current time and the latest request for downloading is less than the preset filtering time threshold, setting the total data amount which is currently involved in bandwidth measurement as the total data amount which is currently received and does not need to filter the bandwidth, and timing the current sampling period to zero.

In one embodiment, the bandwidth measurement condition includes:

the sampling period is currently reached;

or the quantity of data which does not participate in bandwidth measurement in the data which is currently received and does not need to filter the bandwidth is larger than a preset threshold value of the quantity of bandwidth measurement.

In an embodiment, the downloading module 1001 is specifically configured to trigger bandwidth sampling according to the bandwidth measurement parameter, and includes:

and if the transmission time to be used for bandwidth sampling is within a preset time range, calculating the bandwidth value of the current sampling according to the transmission time to be used for bandwidth sampling and the quantity of data which does not participate in bandwidth measurement in the data which is currently received and does not need bandwidth filtering.

In an embodiment, the downloading module 1001 is specifically configured to obtain the current number of bandwidth sampling points, and includes:

dividing the playing time length of the currently received and undisplayed data by the sampling period to obtain the number of initial sampling points;

if the playing time of the maximum downloading unit of the currently played file is less than a preset time threshold, determining the number of the bandwidth sampling points matched with the number of the initial sampling points according to the mapping relation between the number of the preset initial sampling points and the number of the bandwidth sampling points, and setting the number of the bandwidth sampling points as the number of the current bandwidth sampling points, otherwise, setting the number of the current bandwidth sampling points as the preset maximum sampling point number threshold.

In an embodiment, the downloading module 1001 is specifically configured to monitor a real-time bandwidth based on a downloading condition of the streaming media data, and includes:

when receiving the data packet of the streaming media, filtering the data packet which is not sent in saturation;

in the filtered data packet, a stable region of the data size and the transmission rate is found, and the current bandwidth is calculated based on the data size and the stable region of the transmission rate.

In an embodiment, the downloading module 1001 is specifically configured to filter packets sent without saturation, and includes:

calculating the preset time T_oAnd if the data size of the current data packet is smaller than or equal to the data size average value, filtering the current data packet.

In an embodiment, the downloading module 1001 is specifically configured to calculate an average data size of a data packet received within a preset time, and if the data size of a current data packet is smaller than or equal to the average data size, filter the current data packet, and includes:

calculating the transmission time of the current data packet;

inserting the data size and transmission time of the current data packet into a bidirectional queue Q_oThe tail of (a);

if inserted into queue Q_oThe time of the queue element at the head is greater than the preset time T from the current time_oThen the queue Q is deleted_oA queue element of the head;

calculate the queue Q_oAverage of median data size;

if the data size of the current data packet is larger than the average value of the data sizes, inserting the data size and the transmission time of the current data packet into a bidirectional queue Q_fThe tail of (2).

In an embodiment, the downloading module 1001 is specifically configured to find a stable region of the data size and the transmission rate in the filtered data packet, and calculate the current bandwidth based on the stable region of the data size and the transmission rate, and includes:

for the queue Q_fSorting according to the data size;

traversing the sorted queue Q_fAcquiring a group of queue elements with the same data size and the maximum number, and recording the queue elements in a queue Q_fPosition (2): starting position P_bEnd position P_eAnd the number of elements C_max；

If said C is_maxIf the value is greater than or equal to the preset value C, the queue Q is aligned_fMiddle P_bTo P_eSequencing queue elements in the region according to the transmission time, acquiring a stable region of the transmission rate based on a sequencing result, accumulating the data size of the stable region of the transmission rate to obtain S, accumulating the transmission time to obtain T, and calculating B as S/T as the current bandwidth;

if said C is_maxIf the value is less than the preset value C, accumulating the queue Q_oThe total data size S is obtained from the data size in the queue, and a queue Q is accumulated_oThe total transmission time T is obtained from the transmission time in (1), and B ═ S/T is calculated as the current bandwidth.

In an embodiment, the downloading module 1001 is specifically configured to obtain a stable region of a transmission rate based on a sorting result, and includes:

taking N1 queue elements at the first middle position and N2 queue elements at the second middle position of the sorting result, and forming a stable region of the transmission rate by the N1+ N2 queue elements; wherein the first intermediate position precedes the second intermediate position in the ranking result.

In an embodiment, the downloading module 1001 is specifically configured to obtain a stable region of a transmission rate based on a sorting result, and includes:

calculating P_bTo P_eMiddle position P of the region_m＝(P_e–P_b) /2 if P_mRounding it up or down if not an integer;

based on P_bTo P_eSequencing result of region, traverse queue Q_fIn the slave P_bTo P_m-10 queue elements, calculating the variance of the transmission time of each queue element and the transmission time of the following 9 data, obtaining a group of queue elements with the smallest variance;

based on P_bTo P_eSequencing result of region, traverse queue Q_fIn the slave P_m+1 to P_e-10 queue elements, calculating the variance of the transmission time of each queue element and the transmission time of the following 9 data, obtaining a group of queue elements with the smallest variance;

the two sets of queue elements obtained constitute a stable region of transmission rate.

In an embodiment, the downloading module 1001 is specifically configured to determine the matched target bitrate based on the real-time bandwidth, and includes:

predicting a target bandwidth according to the real-time bandwidth;

determining a matched switching strategy according to the target bandwidth and the optimal characteristics corresponding to the current scene;

and determining the current target code rate according to the switching strategy.

In an embodiment, the playing module 1002 is further configured to: monitoring the real-time playing delay of the streaming media according to a preset delay monitoring period; and determining a corresponding playing speed according to the real-time playing delay, and playing according to the playing speed.

In an embodiment, the playing module 1002 is specifically configured to determine a corresponding playing rate according to the real-time playing delay, and includes:

if the real-time playing delay exceeds a specified tolerable delay range, determining the corresponding playing rate as a fast playing rate;

and if the real-time playing delay is within the tolerable delay range, determining the corresponding playing rate as a normal rate.

In an embodiment, the playing module 1002 is specifically configured to determine a corresponding playing rate according to the real-time playing delay, and includes:

if the real-time playing delay exceeds a specified tolerable delay range and the consumption rate of the current streaming media cache data is not greater than the downloading rate, determining the corresponding playing rate as a fast playing rate;

if the real-time playing delay is within the tolerable delay range and the consumption rate of the current streaming media cache data is not greater than the downloading rate, determining that the corresponding playing rate is a normal rate;

and if the difference between the consumption rate of the current streaming media cache data and the downloading rate is greater than a preset difference threshold, determining that the corresponding playing rate is a slow playing rate, wherein the difference threshold is greater than or equal to zero.

In an embodiment, the playing module 1002 is specifically configured to determine the fast playing rate, and includes:

inquiring a mapping relation between a difference value between preset real-time playing delay and target delay and a fast playing rate to obtain a fast playing rate corresponding to the difference value between the real-time playing delay and the target delay; wherein the difference value in the mapping relation is in direct proportion to the fast playing rate.

In an embodiment, the playing module 1002 is specifically configured to determine the slow playing rate, and includes:

inquiring a mapping relation between a preset difference value between the consumption rate and the download rate and the slow play rate to obtain the slow play rate corresponding to the current difference value between the consumption rate and the download rate; wherein the difference in the mapping relationship is inversely proportional to the slow-play rate.

According to the embodiment of the streaming media playing method, the embodiment of the application realizes the electronic equipment for playing the streaming media, and the electronic equipment comprises a processor and a memory; the memory stores an application program executable by the processor for causing the processor to execute the streaming media playing method as described above. Specifically, a system or an apparatus equipped with a storage medium on which a software program code that realizes the functions of any of the embodiments described above is stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program code stored in the storage medium. Further, part or all of the actual operations may be performed by an operating system or the like operating on the computer by instructions based on the program code. The program code read out from the storage medium may be written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then based on instructions of the program code, a CPU or the like mounted on the expansion board or the expansion unit may be caused to perform part or all of the actual operations, thereby implementing the functions of any of the above-described embodiments of the streaming media playing method.

The memory may be embodied as various storage media such as an Electrically Erasable Programmable Read Only Memory (EEPROM), a Flash memory (Flash memory), and a Programmable Read Only Memory (PROM). The processor may be implemented to include one or more central processors or one or more field programmable gate arrays, wherein the field programmable gate arrays integrate one or more central processor cores. In particular, the central processor or central processor core may be implemented as a CPU or MCU.

Embodiments of the present application implement a computer program product comprising computer programs/instructions which, when executed by a processor, implement the streaming media playing steps as described above.

It should be noted that not all steps and modules in the above flows and structures are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The division of each module is only for convenience of describing adopted functional division, and in actual implementation, one module may be divided into multiple modules, and the functions of multiple modules may also be implemented by the same module, and these modules may be located in the same device or in different devices.

The hardware modules in the various embodiments may be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (e.g., a special purpose processor such as an FPGA or ASIC) for performing specific operations. A hardware module may also include programmable logic devices or circuits (e.g., including a general-purpose processor or other programmable processor) that are temporarily configured by software to perform certain operations. The implementation of the hardware module in a mechanical manner, or in a dedicated permanent circuit, or in a temporarily configured circuit (e.g., configured by software), may be determined based on cost and time considerations.

"exemplary" means "serving as an example, instance, or illustration" herein, and any illustration, embodiment, or steps described as "exemplary" herein should not be construed as a preferred or advantageous alternative. For the sake of simplicity, the drawings are only schematic representations of the parts relevant to the invention, and do not represent the actual structure of the product. In addition, in order to make the drawings concise and understandable, components having the same structure or function in some of the drawings are only schematically illustrated or only labeled. In this document, "a" does not mean that the number of the relevant portions of the present invention is limited to "only one", and "a" does not mean that the number of the relevant portions of the present invention "more than one" is excluded. In this document, "upper", "lower", "front", "rear", "left", "right", "inner", "outer", and the like are used only to indicate relative positional relationships between relevant portions, and do not limit absolute positions of the relevant portions.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.