阅读说明：本技术 一种多对象存储的云视频Timeline存储方法和系统 (Multi-object storage cloud video Timeline storage method and system ) 是由 周莹 方泽梁 陈帅斌 蒋泽飞 夏虹 于 2021-08-31 设计创作，主要内容包括：本发明公开了一种多对象存储的云视频Timeline存储方法和系统,所述方法包括如下步骤：获取对象存储,计算所述对象存储的质量分数；选取最优的质量分数的对象存储作为目标对象存储；生成raw文件和所述raw文件key,将所述raw文件和raw文件key保存于目标对象存储中；判断当前目标对象存储是否变更,若变更则保存对象存储变更记录。所述方法和系统在存储时,可以将raw文件元信息编码进对象存储key,使得所述对象存储key可以实现自我描述的功能,从而在查询时通过raw文件key规则匹配raw文件列表,从匹配的所述raw文件列表反向生成Timeline,从而无需单独设置的Timeline索引,显著降低了存储成本和运维复杂度。(The invention discloses a cloud video Timeline storage method and a cloud video Timeline storage system with multi-object storage, wherein the method comprises the following steps: acquiring an object storage, and calculating the quality score of the object storage; selecting an object storage with the optimal quality score as a target object storage; generating a raw file and a raw file key, and storing the raw file and the raw file key in a target object storage; and judging whether the current target object storage is changed or not, and if so, saving the object storage change record. According to the method and the system, raw file meta information can be coded into the object storage key during storage, so that the object storage key can realize a self-description function, a raw file list is matched through a raw file key rule during query, and the Timeline is reversely generated from the matched raw file list, so that a separately set Timeline index is not needed, and the storage cost and the operation and maintenance complexity are remarkably reduced.)

1. A cloud video Timeline storage method for multi-object storage is characterized by comprising the following steps:

acquiring an object storage, and calculating the quality score of the object storage;

selecting an object storage with the optimal quality score as a target object storage;

acquiring a raw file, generating a raw file key, and storing the raw file and the raw file key in a target object storage;

and judging whether the current target object storage is changed or not, and if so, saving the object storage change record.

2. The method for storing the cloud video Timeline stored in the multi-object manner according to claim 1, wherein the method for generating the raw file key comprises the following steps:

acquiring a unique identification of a camera, and acquiring a video starting timestamp and a video ending timestamp of a raw file;

converting the raw file video starting timestamp into time in a year, month and day format;

combining the unique camera identification, the start timestamp of the raw file video, the end timestamp of the raw file video and the time converted into the year, month and day format to generate a raw file key in a character string form;

and saving the generated raw file key.

3. The method according to claim 1, wherein an expiration time of a raw file is set when the generated raw file key is acquired, and if the time exceeds the expiration time, the raw file in the target object storage is automatically cleaned.

4. The method of claim 1, wherein if it is determined whether the current object store is the same as the last object store, if not, generating an object store change record, and storing the object store change record in a database.

5. The method of claim 1, wherein the method comprises a query phase comprising: and acquiring change records of the camera, filtering according to a query time range preset by a user, and acquiring the change records in a target change range.

6. The method of claim 5, wherein a key prefix is generated according to the raw file key generation rule, and the key prefix comprises a string in days and a string in hours.

7. The method of claim 6, wherein the query phase further comprises: calling an object storage service to acquire all raw files matched with the key prefixes; and traversing all change records of the object storage service, calling all object storages in the change records, summarizing all raw files meeting the query time range, generating a raw file list, analyzing the raw file list, and reversely generating timeline.

8. The method for multi-object stored cloud video Timeline storage according to claim 5, wherein said query phase further comprises: and acquiring the attribute dimension of object storage, and performing duplicate removal query according to the attribute dimension.

9. A multi-object-storage cloud video Timeline storage system, wherein the system executes the multi-object-storage cloud video Timeline storage method according to any of claims 1 to 8.

10. A computer-readable storage medium storing a computer program executable by a processor to perform a multi-object stored cloud video Timeline storage method according to any of claims 1-8.

Technical Field

The invention relates to the technical field of storage, in particular to a cloud video Timeline storage method and system for multi-object storage.

Background

With the development of cloud computing and cloud storage technologies, video data storage of the camera gradually turns to a cloud from local, and a cloud video platform provides a service for video cloud storage. With the rapid increase of the access of a cloud video platform to a camera, how to reliably store and efficiently retrieve video data becomes an urgent problem to be solved. Video data can be divided into raw files and time indices. The original video is generally divided into raw files by time (about one minute), and the raw files are stored in an object storage service provided by a cloud service provider through encoding, encryption and the like. the time index records the storage path, the starting time, the ending time and other meta information of the raw file, and the raw file of the target time can be quickly searched through the time index.

At present, the timeline index is generally stored in a MySQL traditional relational database in the industry, and horizontal database partitioning and table partitioning are performed to construct a MySQL cluster so as to meet the capacity expansion to a certain degree. However, this memory architecture has three drawbacks:

one is the introduction of additional storage costs. raw file storage is necessary, and the storage of the timeline index is additionally generated on the basis of the raw file, and the storage cost of the timeline index is high.

Secondly, the cluster load cannot be balanced dynamically. Once the rules of the cluster database partitioning and table partitioning are determined, the rules cannot be changed any more, so that the load difference between database instances in the cluster is large over time until a hot spot database is generated, the performance of the whole cluster is limited, and great negative influence is brought to the service quality of the platform.

And thirdly, deleting the overdue timeline index consumes excessive resources. Since conventional databases typically do not have an automatic expiration policy, a special service is required to periodically clean up expired timeline indexes. Clean-up services consume considerable service resources while competing with normal traffic for database resources.

Disclosure of Invention

One of the purposes of the present invention is to provide a method and a system for storing a cloud video Timeline stored in multiple objects, where in the method and the system, raw file meta information may be encoded into an object storage key during storage, so that the object storage key may implement a self-description function, and thus a raw file list is matched through a raw file key rule during query, and a Timeline is reversely generated from the matched raw file list, so that a Timeline index set separately is not needed, and storage cost and operation and maintenance complexity are significantly reduced.

Another object of the present invention is to provide a method and a system for storing a cloud video Timeline with multi-object storage, where the method and the system implement automatic cleaning of raw files by means of a life cycle of an object storage service, and do not need to deploy a cleaning service separately, so that service resources can be saved, and competition between the cleaning service and normal services for storage resources is avoided.

The invention also aims to provide a cloud video Timeline storage method and system for multi-object storage, wherein the method and system store a plurality of raw files into different objects, the cloud video of one camera can be stored into a plurality of object storages through the storage of the plurality of objects, the storage resources can be dynamically expanded through the combined query of the plurality of object storages, the factors such as the availability, the storage load, the regions, the network and the like of the object storage are comprehensively considered, the preferred storage is realized, and the flexibility and the reliability of the storage are improved.

In order to achieve at least one of the above objects, the present invention further provides a cloud video Timeline storage method for multi-object storage, including the following steps:

acquiring an object storage, and calculating the quality score of the object storage;

selecting an object storage with the optimal quality score as a target object storage;

generating a raw file and a raw file key, and storing the raw file and the raw file key in a target object storage;

and judging whether the current target object storage is changed or not, and if so, saving the object storage change record.

According to one preferred embodiment of the present invention, the method for generating raw file keys includes the following steps:

acquiring a unique identification of a camera, and acquiring a video starting timestamp and a video ending timestamp of a raw file;

converting the raw file video starting timestamp into time in a year, month and day format;

combining the unique camera identification, the start timestamp of the raw file video, the end timestamp of the raw file video and the time converted into the year, month and day format to generate a raw file key in a character string form;

and saving the generated raw file key.

According to another preferred embodiment of the invention, the generated raw file key is acquired, the expiration time of the raw file is set, and if the time exceeds the expiration time, the raw file in the target object storage is automatically cleaned.

According to another preferred embodiment of the present invention, if the currently used object storage is determined to be the same as the last time, if the currently used object storage is different from the last time, an object storage change record is generated, and the object storage change record is stored in the database.

According to another preferred embodiment of the present invention, the method includes a query phase, the query phase including: and acquiring change records of the camera, filtering according to a query time range preset by a user, and acquiring the change records in a target change range.

According to another preferred embodiment of the present invention, a key prefix is generated according to the raw file key generation rule, and the key prefix includes a character string in days and a character string in hours.

According to another preferred embodiment of the present invention, the query phase further comprises: calling an object storage service to acquire all raw files matched with the key prefixes; traversing all change records of the object storage service, calling all object storages in the change records, summarizing all raw files conforming to the query time range, generating a raw file list, analyzing the raw file list, reversely generating timeline,

according to another preferred embodiment of the present invention, the query phase further comprises: and acquiring the attribute dimension of object storage, and performing duplicate removal query according to the attribute dimension.

In order to achieve at least one of the above objects, the present invention further provides a cloud video Timeline storage system with multi-object storage, where the system executes the above cloud video Timeline storage method with multi-object storage.

The present invention further provides a computer-readable storage medium storing a computer program that can be executed by a processor to perform the above-mentioned cloud video Timeline storage method for multi-object storage.

Drawings

Fig. 1 is a schematic flow chart showing a cloud video Timeline storage method for multi-object storage according to the present invention.

Fig. 2 is a schematic flow chart showing a query phase in the cloud video Timeline storage method for multi-object storage according to the present invention.

Detailed Description

The following description is presented to disclose the invention so as to enable any person skilled in the art to practice the invention. The preferred embodiments in the following description are given by way of example only, and other obvious variations will occur to those skilled in the art. The basic principles of the invention, as defined in the following description, may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the invention.

It is understood that the terms "a" and "an" should be interpreted as meaning that a number of one element or element is one in one embodiment, while a number of other elements is one in another embodiment, and the terms "a" and "an" should not be interpreted as limiting the number.

Referring to fig. 1 and fig. 2, the present invention discloses a method and a system for storing a cloud video Timeline stored in multiple objects, where the method mainly includes the following steps: first, in the storage phase, since the video of each camera can be stored in the multi-object storage, the multi-object storage may be different types of object storage provided by different cloud service providers, or may be object storage in a plurality of different areas provided by the same cloud service provider. Through the combined query according to different dimensions, the query results stored by different objects are collected and then reversely generated to the timeline, so that the timeline service is not required to be set, the service maintenance cost is reduced, the object storage resources can be dynamically expanded, the object storage can be dynamically adjusted when a node fails, and the storage stability is improved.

Specifically, a raw file of the camera is obtained first, the raw file is a video storage file of the camera, and the raw file can be generated when the camera starts video shooting. Calling a storage service in a storage phase, wherein the storage service acquires attributes of a plurality of object storages, and determines an optimal object storage to store according to the attributes of the object storages, wherein the object storage attributes include, but are not limited to, availability, load, storage location, network condition, and the like, wherein the object storage attributes may be quantized to specific values, for example, the availability may be quantized to a ratio of the available storage quantity to the total object storage quantity, and the load may be quantized to: the storage positions can be local storage and different cloud storages, quantitative assignment is carried out according to the storage effect, and the network condition can calculate network delay amount to carry out quantitative assignment. Further, the quality score stored in the current object is calculated according to the attribute stored in the current object, wherein the value of the quality score may be weighted and summed according to the attribute stored in different objects, and the weight value stored in each different object may be configured according to requirements, which is not described in detail herein.

Executing a calling operation of related object storage by taking the mass fraction as a raw storage strategy of the camera, and when a raw file is generated at the head end of the camera, performing key assignment on the raw file, wherein the key assignment method comprises the following steps:

acquiring and generating a unique identifier of a camera corresponding to the raw file; acquiring a video starting timestamp and a video ending timestamp of a current raw file; converting the raw file video starting timestamp into a time character string taking days as a unit, converting the raw file video starting timestamp into a time character string taking hours as a unit, and further combining the character strings into a raw file key, wherein the form of the raw file key can be as follows:

raw_file/{deviceId}/{yyyyMMdd}/{HH}/{startTime}_{endTime}；

wherein raw _ file represents a raw file, deviceId represents a unique identifier of a corresponding camera, yyyMMdd represents a time string in days, HH represents a time string in hours, startTime represents a raw file video start timestamp, and endTime represents a raw file video end timestamp. Wherein the brackets in the form of the raw file key are filled with corresponding string values. For example:

the camera deviceId is: ab28965dc491, the timestamp generated is: 1629343440000-1629343500000, the corresponding time strings are respectively: 2021-08-1911: 24: 00-2021-08-1911: 25:00, and the value of the raw file key may be:

raw_file/ab28965dc491/20210819/11/1629343440000_1629343500000。

and further storing the object with the optimal storage value of the raw file and the raw file key, setting the storage expiration time of the raw file and the raw file key, and if the storage time of the stored object is greater than the storage expiration time, automatically clearing the raw file and the raw file key, so that the regular clearing operation of timeline can be realized by means of a life cycle management strategy of object storage service, the clearing service does not need to be separately deployed, and the non-service storage resource competition is reduced. And if the raw file and the raw file key are called currently and last time, the change record of the object storage call is indicated.

It is worth mentioning that the storage method further comprises a query phase, wherein the query phase comprises:

calling query service of object storage, and acquiring storage change records of the camera;

performing a deduplication operation according to dimensions of an object store, the deduplication operation comprising: judging whether the inquired object storage is the same object storage according to the dimensions of the storage position, the change record and the like, and deleting the same object storage if the inquired object storage is the same object storage;

obtaining raw files in a specified time range required to be inquired by a user in inquiry service

Generating a key prefix for matching query according to a generation rule of the raw file key;

calling an object storage service to match all raw files according to the key prefix;

traversing all change records, calling object storage of all related raw files, and summarizing the raw files in the object storage according to the key prefixes to generate a raw file list;

analyzing the raw file list and reversely generating timeline;

the reverse generated timeline;

the form of the key prefix respectively comprises a query form taking day as a unit and a query form taking hour as a unit, and the query form taking day as a unit is as follows: raw _ file/{ deviceId }/{ yyymmdd }, the query form in hours is: raw _ file/{ deviceId }/{ yyymmdd }/{ HH }.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. The computer program, when executed by a Central Processing Unit (CPU), performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wire segments, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless section, wire section, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It will be understood by those skilled in the art that the embodiments of the present invention described above and illustrated in the drawings are given by way of example only and not by way of limitation, the objects of the invention having been fully and effectively achieved, the functional and structural principles of the present invention having been shown and described in the embodiments, and that various changes or modifications may be made in the embodiments of the present invention without departing from such principles.