阅读说明：本技术 一种基于协同过滤推荐算法的hls缓存方法及系统 (HLS caching method and system based on collaborative filtering recommendation algorithm ) 是由 朱博 刘军 陈宇 于 2021-08-23 设计创作，主要内容包括：本发明公开了一种基于协同过滤推荐算法的HLS缓存方法及系统,包括,基于视频播放时长,对每个用户的视频感兴趣程度评分,形成点播评分表；根据所述点播评分表,利用余弦相似度计算公式,为每个用户挑选出五个兴趣最相似的用户；结合相似用户点播评分,挑选所述用户可能感兴趣的视频；根据挑选结果,在后台提前打开视频。本发明通过后台服务提前生成TS文件大大加快视频点播速度,根据用户点播记录,只针对每个用户选取20路热点视频提前点播缓存,生成TS文件,既减少了网络带宽的占用,又能兼顾到热点的命中率,提高播放速度,系统会根据每天的播放情况,调整历史记录矩阵,动态更新缓存策略,不需要手动配置干预,对用户透明,简单易用。(The invention discloses an HLS caching method and system based on a collaborative filtering recommendation algorithm, which comprises the steps of scoring the video interest degree of each user based on the video playing time length to form an on-demand scoring table; selecting five users with the most similar interests for each user by utilizing a cosine similarity calculation formula according to the on-demand rating table; selecting videos which are possibly interested by the user by combining the similar user on-demand scores; and according to the selection result, opening the video in the background in advance. The invention generates the TS file in advance through the background service, greatly accelerates the video on demand speed, selects 20 paths of hot video on demand cache in advance only aiming at each user according to the user on demand record, generates the TS file, reduces the occupation of network bandwidth, considers the hit rate of the hot spot, improves the playing speed, and the system can adjust a historical record matrix according to the playing condition every day, dynamically updates the cache strategy, does not need manual configuration intervention, is transparent to the user, and is simple and easy to use.)

1. An HLS caching method based on a collaborative filtering recommendation algorithm is characterized in that: comprises the steps of (a) preparing a mixture of a plurality of raw materials,

grading the video interest degree of each user based on the video playing time to form an on-demand grading table;

selecting five users with the most similar interests for each user by utilizing a cosine similarity calculation formula according to the on-demand rating table;

selecting videos which are possibly interested by the user by combining the similar user on-demand scores;

and according to the selection result, opening the video in the background in advance.

2. The HLS caching method based on the collaborative filtering recommendation algorithm according to claim 1, wherein: comprises the steps of (a) preparing a mixture of a plurality of raw materials,

for each path of video, recording the video as one minute when the user plays the video for 0.5 hour, and processing the video according to one minute when the video is less than 0.5 hour;

and the score is fully recorded for 48 points, and the score is recorded again the next day after the statistics on the score recording day is completed.

3. The HLS caching method based on the collaborative filtering recommendation algorithm according to claim 1 or 2, wherein: the cosine similarity calculation formula includes the following formulas,

wherein A is_iFor the on-demand rating of each video for each user, the closer the calculation result to 1, the more similar the calculation result is.

4. The HLS caching method based on the collaborative filtering recommendation algorithm according to claim 3, wherein: comprises the steps of (a) preparing a mixture of a plurality of raw materials,

removing videos which are already requested by the target user from the calculated request records of the similar users, and calculating the possible interest degree of the target user in the remaining point locations, wherein the calculation formula is as follows:

wherein p (u, i) represents the interest degree of the target u in the video i, and S (u, K) represents K users most similar to the target user, where K is 5; r is_viAnd the like degree of the user v to i is represented, namely the grade of the user v to i is obtained.

5. The HLS caching method based on the collaborative filtering recommendation algorithm according to claim 4, wherein: and the first ten videos are selected according to the calculation result of each video in a descending order.

6. The HLS caching method based on the collaborative filtering recommendation algorithm according to claim 5, wherein: the method comprises the steps of selecting ten most interesting videos from the ordering statistical results of target users, selecting ten videos from the third step of calculation results, obtaining a twenty video list by a system, and opening the twenty videos in the background in advance.

7. The HLS caching method based on the collaborative filtering recommendation algorithm according to claim 6, wherein: comprises the steps of (a) preparing a mixture of a plurality of raw materials,

the on-demand scoring table stores records in a matrix form, each row of the matrix represents on-demand record scoring of each user, and each column represents one path of video.

8. An HLS cache system based on a collaborative filtering recommendation algorithm is characterized in that: the system comprises a management service module and a streaming media module;

the management service module comprises a video receiving module, a video requesting module and a streaming media playing module, wherein the video requesting module receives a video command requested by a client, the video command is transmitted by using an HTTP (hyper text transport protocol), records a user on-demand record, calculates and scores the data after collecting data of one day, requests a video from a streaming media, transmits the video requesting command by using a WebSocket protocol, and provides m3u8 and ts files needing to be downloaded to the client;

the streaming media module receives a video request from the management service module, the request command is transmitted by adopting a WebSocket protocol, a video is requested from a video platform/camera, the request mode is provided by a video platform/camera manufacturer, and an m3u8 and ts file are generated according to the requested video code stream.

Technical Field

The invention relates to the technical field of HLS caching and data filtering, in particular to an HLS caching method and system based on a collaborative filtering recommendation algorithm.

Background

The HLS protocol is a streaming media transfer protocol based on HTTP proposed by apple inc, and is currently widely applied in the fields of video on demand and live broadcast. The HLS protocol realizes the effect of actually playing a stream by cutting a video stream into small TS files that can be downloaded via HTTP, then providing a matching TS list file to the client, and allowing the client to sequentially pull the TS files for playing.

Due to the design principle of the HLS, when requesting for the HLS video, the client only needs to generate a TS with a certain size to start downloading the media file to the local for playing. Compared with the traditional streaming media transmission, the mode adds a process of generating the TS file, after the client successfully requests, the client can download and play the TS file only after the server generates a complete TS file, generally, one TS file contains about 5s of video data, and then the client can download the first TS media file for playing after five seconds, so that the video opening speed is generally more than five seconds.

Disclosure of Invention

This section is for the purpose of summarizing some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. In this section, as well as in the abstract and the title of the invention of this application, simplifications or omissions may be made to avoid obscuring the purpose of the section, the abstract and the title, and such simplifications or omissions are not intended to limit the scope of the invention.

The present invention has been made in view of the above-mentioned conventional problems.

Therefore, the technical problem solved by the invention is as follows: and predicting the video which the user is interested in, and caching the HLS video in advance to accelerate the opening speed of the HLS video.

In order to solve the technical problems, the invention provides the following technical scheme: the method comprises the steps of scoring the video interest degree of each user based on the video playing time length to form an on-demand scoring table; selecting five users with the most similar interests for each user by utilizing a cosine similarity calculation formula according to the on-demand rating table; selecting videos which are possibly interested by the user by combining the similar user on-demand scores; and according to the selection result, opening the video in the background in advance.

As a preferred scheme of the collaborative filtering recommendation algorithm-based HLS caching method according to the present invention, wherein: the method comprises the steps that each path of video is recorded as one minute when the user plays 0.5 hour, and the video is processed according to one minute when the time is less than 0.5 hour; and the score is fully recorded for 48 points, and the score is recorded again the next day after the statistics on the score recording day is completed.

As a preferred scheme of the collaborative filtering recommendation algorithm-based HLS caching method according to the present invention, wherein: the cosine similarity calculation formula includes the following formulas,

wherein A is_iFor the on-demand rating of each video for each user, the closer the calculation result to 1, the more similar the calculation result is.

As a preferred scheme of the collaborative filtering recommendation algorithm-based HLS caching method according to the present invention, wherein: the method comprises the steps of removing videos which are already requested by a target user from calculated on-demand records of similar users, and calculating the possible interest degree of the target user in the remaining point positions, wherein the calculation formula is as follows:

wherein p (u, i) represents the interest degree of the target u in the video i, and S (u, K) represents K users most similar to the target user, where K is 5; r is_viAnd the like degree of the user v to i is represented, namely the grade of the user v to i is obtained.

As a preferred scheme of the collaborative filtering recommendation algorithm-based HLS caching method according to the present invention, wherein: and the first ten videos are selected according to the calculation result of each video in a descending order.

As a preferred scheme of the collaborative filtering recommendation algorithm-based HLS caching method according to the present invention, wherein: the method comprises the steps of selecting ten most interesting videos from the ordering statistical results of target users, selecting ten videos from the third step of calculation results, obtaining a twenty video list by a system, and opening the twenty videos in the background in advance.

As a preferred scheme of the collaborative filtering recommendation algorithm-based HLS caching method according to the present invention, wherein: the method comprises the steps that records are stored in the on-demand rating table in a matrix mode, each row of the matrix represents on-demand record rating of each user, and each column represents one path of video.

As a preferred scheme of the collaborative filtering recommendation algorithm based HLS caching system according to the present invention, wherein: the system comprises a management service module and a streaming media module; the management service module comprises a video receiving module, a video requesting module and a streaming media playing module, wherein the video requesting module receives a video command requested by a client, the video command is transmitted by using an HTTP (hyper text transport protocol), records a user on-demand record, calculates and scores the data after collecting data of one day, requests a video from a streaming media, transmits the video requesting command by using a WebSocket protocol, and provides m3u8 and ts files needing to be downloaded to the client; the streaming media module receives a video request from the management service module, the request command is transmitted by adopting a WebSocket protocol, a video is requested from a video platform/camera, the request mode is provided by a video platform/camera manufacturer, and an m3u8 and ts file are generated according to the requested video code stream.

The invention has the beneficial effects that: the invention generates the TS file in advance through the background service, greatly accelerates the video on demand speed, selects 20 paths of hot video on demand cache in advance only aiming at each user according to the user on demand record, generates the TS file, reduces the occupation of network bandwidth, considers the hit rate of the hot spot, improves the playing speed, and the system can adjust a historical record matrix according to the playing condition every day, dynamically updates the cache strategy, does not need manual configuration intervention, is transparent to the user, and is simple and easy to use.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise. Wherein:

fig. 1 is a schematic structural diagram of an HLS caching method and system based on a collaborative filtering recommendation algorithm according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating an execution flow of the HLS caching method and system based on the collaborative filtering recommendation algorithm according to an embodiment of the present invention;

fig. 3 is a schematic view of an on-demand rating table of the HLS caching method and system based on the collaborative filtering recommendation algorithm according to an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, specific embodiments accompanied with figures are described in detail below, and it is apparent that the described embodiments are a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present invention, shall fall within the protection scope of the present invention.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.

Furthermore, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.

The present invention will be described in detail with reference to the drawings, wherein the cross-sectional views illustrating the structure of the device are not enlarged partially in general scale for convenience of illustration, and the drawings are only exemplary and should not be construed as limiting the scope of the present invention. In addition, the three-dimensional dimensions of length, width and depth should be included in the actual fabrication.

Meanwhile, in the description of the present invention, it should be noted that the terms "upper, lower, inner and outer" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of describing the present invention and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation and operate, and thus, cannot be construed as limiting the present invention. Furthermore, the terms first, second, or third are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

The terms "mounted, connected and connected" in the present invention are to be understood broadly, unless otherwise explicitly specified or limited, for example: can be fixedly connected, detachably connected or integrally connected; they may be mechanically, electrically, or directly connected, or indirectly connected through intervening media, or may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.

Example 1

Referring to fig. 2 and 3, a first embodiment of the present invention provides a collaborative filtering recommendation algorithm-based HLS caching method, which includes:

s1: and scoring the video interest degree of each user based on the video playing time length to form an on-demand scoring table.

S2: and selecting five users with the most similar interests for each user by utilizing a cosine similarity calculation formula according to the on-demand rating table.

S3: and combining the similar user on-demand scores to select videos which are possibly interested by the user.

S4: and according to the selection result, opening the video in the background in advance.

Referring to fig. 3, the embodiment further needs to be specifically described as follows:

the on-demand scoring table stores records in a matrix form, each row of the matrix represents on-demand record scoring of each user, and each column represents one path of video;

each path of video is recorded as one point when the user plays for 0.5 hour, and the video which is less than 0.5 hour is processed according to one point;

and the score is fully recorded for 48 points, and the score is recorded again the next day after the statistics on the score recording day is completed.

Further, the cosine similarity calculation formula includes:

wherein A is_iFor the on-demand rating of each video for each user, the closer the calculation result to 1, the more similar the calculation result is.

Specifically, the calculated video that the target user has requested is removed from the requested video records of the similar users, and the degree of possible interest of the target user is calculated from the remaining point locations, with the following calculation formula:

wherein p (u, i) represents the interest degree of the target u in the video i, and S (u, K) represents K users most similar to the target user, where K is 5; r is_viAnd the like degree of the user v to i is represented, namely the grade of the user v to i is obtained.

Further, the method also comprises the following steps:

sorting the first ten videos from large to small according to the calculation result of each video;

and selecting ten paths of most interesting videos from the ordering statistical results of the target users, selecting ten paths of videos from the third step of calculation results, obtaining a twenty-path video list by the system, and opening the twenty paths of videos in the background in advance.

Preferably, in order to better verify and explain the technical effects adopted in the method of the present invention, in the embodiment, a comparison test is performed between the conventional HLS video playing system and the method of the present invention, and the test results are compared by a scientific demonstration method to verify the real effects of the method of the present invention.

The traditional HLS video playing system cannot cache HLS videos in advance, so that when a user requests the videos, the average video opening time is about 5 seconds, and the user experience is very poor; compared with the traditional method, the method has the advantages that the occupied bandwidth is small, the hit rate of interest prediction of the user is high, the hit video is predicted to be started in seconds, the user experience is greatly improved, and the on-demand video opening speed is measured and compared in real time.

And (3) testing environment: testing in an HLS on-demand system of a certain city information center, testing samples in a real user on-demand mode, and respectively ordering videos and obtaining test result data by using a non-cache HLS system of a traditional method; by adopting the method, the HLS cache on-demand system based on the collaborative filtering recommendation algorithm is used for realizing the test of the method, the data is obtained according to the experimental result, 10 groups of data are tested in each method, the average video opening time of each group of data is calculated, and the comparison error is calculated. The results are shown in the following table.

Table 1: the traditional method does not cache the average open time of the HLS on-demand system.

Day one

The next day

The third day

The fourth day

The fifth day

Day six

The seventh day

Day eight

The ninth day

The tenth day

User one

6.3s

5.7s

6.1s

5.5s

6.0s

5.6s

5.8s

5.2s

5.5s

5.4s

User two

5.7s

5.8s

5.5s

5.1s

5.4s

5.1s

6.1s

5.7s

5.6s

5.1s

User three

5.3s

5.2s

5.2s

5.6s

6.0s

5.2s

5.6s

5.4s

6.1s

5.7s

User four

6.4s

5.8s

5.4s

5.6s

6.1s

5.4s

5.9s

5.9s

5.3s

6.5s

User five

5.7s

5.7s

6.7s

5.8s

5.3s

5.4s

5.7s

5.4s

6.0s

5.2s

Table 2: the HLS cache on-demand system average opening time is obtained by the method.

Day one

The next day

The third day

The fourth day

The fifth day

Day six

The seventh day

Day eight

The ninth day

The tenth day

User one

0.9s

1.2s

1.5s

1.2s

1.2s

1.0s

0.9s

1.4s

1.1s

1.4s

User two

1.3s

1.1s

0.7s

1.0s

1.3s

1.6s

1.3s

1.3s

1.3s

1.2s

User three

1.1s

1.4s

1.3s

1.5s

1.2s

0.8s

1.2s

1.5s

1.6s

0.9s

User four

1.5s

1.0s

1.3s

1.2s

0.8s

0.9s

1.0s

1.2s

1.0s

1.3s

User five

1.1s

1.0s

1.3s

1.0s

1.4s

1.4s

1.4s

1.1s

1.1s

1.3s

Referring to tables 1 and 2, it can be seen intuitively that the method of the present invention represents higher efficiency for the same test sample compared to the conventional method, i.e., the technical effect proposed by the method of the present invention is verified.

Example 2

Referring to fig. 1, a second embodiment of the present invention is different from the first embodiment in that, an HLS caching system based on a collaborative filtering recommendation algorithm is provided, which specifically includes:

the management service module comprises:

and receiving a client request video command, wherein the video command is transmitted by using an HTTP protocol.

User on-demand records are recorded and after a day of data is collected, the data is scored computationally.

And requesting video from the streaming media, wherein the video requesting command is transmitted by adopting a WebSocket protocol.

And providing the m3u8 and ts files needing to be downloaded to the client.

The streaming media module comprises:

and receiving a video request from the management service module, wherein the request command is transmitted by adopting a WebSocket protocol. And requesting the video from the video platform/camera, wherein the request mode is provided by a video platform/camera manufacturer.

And generating an m3u8 and ts file according to the requested video code stream.

It should be recognized that embodiments of the present invention can be realized and implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.

Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.

Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein. A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.

As used in this application, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being: a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of example, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and/or thread of execution and a component can be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the internet with other systems by way of the signal).

It should be noted that the above-mentioned embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.