阅读说明：本技术 一种降低图片存储空间的方法 (Method for reducing picture storage space ) 是由 苏伟 于 2021-09-27 设计创作，主要内容包括：本发明公开了一种降低图片存储空间的方法,包括以下步骤：步骤S1、客户端将图片上传至图片接入服务单元；步骤S2、客户端上传图片时获取图片元数据；步骤S3、图片接入服务单元将图片缓存到本地高速缓单元,将所述图片元数据录入数据库并返回步骤S1；步骤S4、元数据管理服务单元将图片元数据根据预设的图片分类规则进行分类,并将符合所述图片分类规则的图片对应的图片元数据生成任务列表；步骤S5、转码服务单元从元数据管理服务单元获取任务列表,并执行任务；步骤S6、转码服务单元完成转码任务后将转码后的视频流chunk文件发送至存储节点单元,并更新数据库中的图片元数据。(The invention discloses a method for reducing picture storage space, which comprises the following steps: step S1, the client side uploads the picture to the picture access service unit; step S2, acquiring picture metadata when the client uploads the picture; step S3, the picture access service unit caches the picture to a local cache unit, records the picture metadata into a database and returns to step S1; step S4, the metadata management service unit classifies the picture metadata according to preset picture classification rules, and generates a task list from the picture metadata corresponding to the pictures according with the picture classification rules; step S5, the transcoding service unit acquires a task list from the metadata management service unit and executes the task; and step S6, after finishing the transcoding task, the transcoding service unit sends the transcoded video stream chunk file to the storage node unit and updates the picture metadata in the database.)

1. A method for reducing picture storage space is characterized by comprising the following steps:

step S1, the client side uploads the picture to the picture access service unit;

step S2, acquiring picture metadata when the client uploads the picture;

step S3, the picture access service unit caches the picture to a local cache unit, records the picture metadata into a database and returns to step S1;

step S4, the metadata management service unit classifies the picture metadata according to preset picture classification rules, and generates a task list from the picture metadata corresponding to the pictures according with the picture classification rules;

step S5, the transcoding service unit acquires a task list from the metadata management service unit and executes the task;

and step S6, after finishing the transcoding task, the transcoding service unit sends the transcoded video stream chunk file to the storage node unit and updates the picture metadata in the database.

2. The method of claim 1, wherein the method further comprises:

when downloading the picture, the client side initiates a request to the access service unit;

the access service unit reads the picture metadata to obtain a video stream chunk ID, inquires the video stream chunk file metadata according to the chunk ID, and obtains a download address of the video stream chunk file and the size and position of the picture ID in the video stream chunk file;

after downloading the video stream chunk file, sending the video stream chunk file data and the index information to a transcoding service unit to obtain picture data, and sending the picture data to a client.

3. The method according to claim 2, wherein the chunk ID comprises a snapshot machine ID and a preset time.

4. The method according to claim 2, wherein the chunk file metadata comprises chunk IDs, a list of picture IDs, and locations and sizes of video frames corresponding to the picture IDs.

5. The method according to claim 1, wherein the picture metadata in step S2 includes a picture ID, a snapshot ID, a chunk ID, a picture name, a picture size, and a picture type.

6. The method of claim 1, wherein the picture classification rule preset in step S4 includes:

and transcoding the pictures captured by the capturing machine ID into video stream chunk files, and classifying the pictures captured by the capturing machine ID according to the time period.

7. The method for reducing the storage space of pictures according to claim 1, wherein the task in step S5 includes a snapshot machine ID and pictures that the snapshot machine takes in a preset time period.

8. The method for reducing the storage space of pictures according to any one of claims 1 to 7, wherein said step S6 includes:

sending the video stream chunk file to a storage node unit, and simultaneously storing metadata of the chunk file into a database and updating metadata information of a picture;

the downloading of pictures is read in a video stream chunk file.

Technical Field

The invention relates to the technical field of cloud storage, in particular to a method for reducing a picture storage space, which is used for a system with a large demand on the storage space of a self-built private cloud.

Background

With the gradual maturity of artificial intelligence technology, the industry of "AI + security protection" is rapidly developing under the drive of policies and technologies, and the most mature application of AI in the security protection field is image recognition and big data analysis based on video monitoring. The number of the snapshot pictures generated by the number of the cameras at a city level every day is huge, and with the popularization of high-definition cameras, even 4K and 8K cameras, the picture data is larger and larger, and a large amount of picture data provides a great challenge to the storage space. Therefore, a technical solution is needed to store more data in an effective storage space so as to achieve the purpose of saving cost.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a method for reducing the picture storage space, which can store more data in an effective storage space so as to achieve the purpose of saving cost.

The technical scheme adopted by the invention is as follows:

a method for reducing picture storage space comprises the following steps:

step S1, the client side uploads the picture to the picture access service unit;

step S2, acquiring picture metadata when the client uploads the picture;

step S3, the picture access service unit caches the picture to a local cache unit, records the picture metadata into a database and returns to step S1;

step S4, the metadata management service unit classifies the picture metadata according to preset picture classification rules, and generates a task list from the picture metadata corresponding to the pictures according with the picture classification rules;

step S5, the transcoding service unit acquires a task list from the metadata management service unit and executes the task;

and step S6, after finishing the transcoding task, the transcoding service unit sends the transcoded video stream chunk file to the storage node unit and updates the picture metadata in the database.

In the technical scheme, the client uploads the picture to the picture access service unit, and the picture access service unit writes data to the local cache unit, writes the picture metadata into the database and then returns the picture metadata to the client downloading address. The picture metadata includes a picture ID, a snapshot machine ID, a picture name, a picture size, a picture type, and the like. The metadata management service unit classifies pictures in a time range according to the ID of the snapshot machine, so the reason that the pictures are classified by the ID of the snapshot machine is that the difference between the pictures and the pictures belonging to one snapshot machine is small, and when the pictures are transcoded into a video stream chunk file, the data volume is greatly compressed. The transcoding service unit acquires a task list from the metadata management service unit and executes the operation of converting a plurality of pictures into video, wherein one task comprises the ID of the snapshot machine and a plurality of pictures in a period of time of the snapshot machine. The transcoded video stream chunk file is uniquely identified by chunk ID, the chunk ID consists of a snapshot machine ID and time, chunk metadata comprises the chunk ID and a picture ID list, and the picture ID corresponds to the position and the size of a video frame. And after transcoding is finished, writing the chunk file into a storage node, simultaneously writing chunk metadata into a database, updating metadata information of the picture, and then downloading the picture and reading the picture from the chunk file. When downloading the picture, the client initiates a request to the access service, the access service queries the picture metadata to obtain the chunk ID, and queries the chunk metadata through the chunk ID to obtain the download address of the chunk and the position and the size of the picture ID in the chunk file. And after downloading the chunk file, sending the chunk data and the index information to a transcoding service unit to obtain picture data, and finally returning the picture data to the client.

In a further technical scheme, the method further comprises the steps that when the client downloads the picture, a request is sent to the access service unit;

the access service unit reads the picture metadata to obtain a video stream chunk ID, inquires the video stream chunk file metadata according to the chunk ID, and obtains a download address of the video stream chunk file and the size and position of the picture ID in the video stream chunk file;

after downloading the video stream chunk file, sending the video stream chunk file data and the index information to a transcoding service unit to obtain picture data, and sending the picture data to a client.

In a further technical scheme, the chunk ID comprises a snapshot machine ID and preset time.

In a further technical solution, the chunk file metadata includes chunk IDs, a picture ID list, and positions and sizes of video frames corresponding to the picture IDs.

In a further technical solution, the picture metadata in the step S2 includes a picture ID, a snapshot machine ID, a picture name, a picture size, and a picture type.

In a further technical solution, the picture classification rule preset in step S4 includes:

and transcoding the pictures captured by the capturing machine ID into video stream chunk files, and classifying the pictures captured by the capturing machine ID according to the time period.

In a further technical solution, the task in step S5 includes a snapshot machine ID and a picture that the snapshot machine takes in a preset time period.

In a further aspect, step S6 includes:

sending the video stream chunk file to a storage node unit, and simultaneously storing metadata of the chunk file into a database and updating metadata information of a picture;

the downloading of pictures is read in a video stream chunk file.

Drawings

Fig. 1 is a flowchart of a method for reducing a picture storage space according to the present invention.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Example (b):

as shown in fig. 1, a method for reducing a picture storage space includes the following steps:

step S1, the client side uploads the picture to the picture access service unit;

step S2, acquiring picture metadata when the client uploads the picture;

step S3, the picture access service unit caches the picture to a local cache unit, records the picture metadata into a database and returns to step S1;

step S4, the metadata management service unit classifies the picture metadata according to preset picture classification rules, and generates a task list from the picture metadata corresponding to the pictures according with the picture classification rules;

step S5, the transcoding service unit acquires a task list from the metadata management service unit and executes the task;

and step S6, after finishing the transcoding task, the transcoding service unit sends the transcoded video stream chunk file to the storage node unit and updates the picture metadata in the database.

In this embodiment, the client uploads the picture to the picture access service unit, and the picture access service unit writes data to the local cache unit and writes the picture metadata into the database, and then returns the picture metadata to the client download address. The picture metadata includes a picture ID, a snapshot machine ID, a picture name, a picture size, a picture type, and the like. The metadata management service unit classifies pictures in a time range according to the ID of the snapshot machine, so the reason that the pictures are classified by the ID of the snapshot machine is that the difference between the pictures and the pictures belonging to one snapshot machine is small, and when the pictures are transcoded into a video stream chunk file, the data volume is greatly compressed. The transcoding service unit acquires a task list from the metadata management service unit and executes the operation of converting a plurality of pictures into video, wherein one task comprises the ID of the snapshot machine and a plurality of pictures in a period of time of the snapshot machine. The transcoded video stream chunk file is uniquely identified by chunk ID, the chunk ID consists of a snapshot machine ID and time, chunk metadata comprises the chunk ID and a picture ID list, and the picture ID corresponds to the position and the size of a video frame. And after transcoding is finished, writing the chunk file into a storage node, simultaneously writing chunk metadata into a database, updating metadata information of the picture, and then downloading the picture and reading the picture from the chunk file. When downloading the picture, the client initiates a request to the access service, the access service queries the picture metadata to obtain the chunk ID, and queries the chunk metadata through the chunk ID to obtain the download address of the chunk and the position and the size of the picture ID in the chunk file. And after downloading the chunk file, sending the chunk data and the index information to a transcoding service unit to obtain picture data, and finally returning the picture data to the client.

In another embodiment, the method further comprises the steps that when the client downloads the picture, a request is sent to the access service unit;

the access service unit reads the picture metadata to obtain a video stream chunk ID, inquires the video stream chunk file metadata according to the chunk ID, and obtains a download address of the video stream chunk file and the size and position of the picture ID in the video stream chunk file;

after downloading the video stream chunk file, sending the video stream chunk file data and the index information to a transcoding service unit to obtain picture data, and sending the picture data to a client.

In this embodiment, the client mainly initiates a request for uploading a picture, and may be a snapshot machine or a third-party application. The picture access service unit provides an interface for uploading and downloading pictures, after the pictures are collected and placed in the local cache unit, on one hand, concurrence of picture uploading can be improved, on the other hand, the local cache can be used as a temporary cache for subsequent picture-to-video processing, the picture access service can be deployed in a distributed mode, load balancing can be achieved by deploying nginx configuration in the front, and different picture access requests are balanced and targeted.

In another embodiment, the chunk ID includes a snapshot machine ID and a preset time.

In another embodiment, the chunk file metadata includes chunk IDs, a list of picture IDs, and the location and size of the video frames to which the picture IDs correspond.

The chunk ID in this embodiment is written when the metadata of the picture is updated after the subsequent transcoding task is completed, and the chunk ID is used when the picture is downloaded subsequently. When downloading the picture, the client initiates a request to the access service, the access service queries the picture metadata to obtain the chunk ID, and queries the chunk metadata through the chunk ID to obtain the download address of the chunk and the position and the size of the picture ID in the chunk file.

In another embodiment, the picture metadata in step S2 includes a picture ID, a camera ID, a picture name, a picture size, and a picture type.

In another embodiment, the picture classification rules preset in step S4 include:

and transcoding the pictures captured by the capturing machine ID into video stream chunk files, and classifying the pictures captured by the capturing machine ID according to the time period.

In this embodiment, a picture stream of a certain snapshot machine within a certain time range is transcoded into a video stream chunk file, a task is derived from the metadata management service unit, and a picture list of the snapshot machine ID is classified according to a time period, for example, a picture list of each half hour corresponds to a video stream chunk file, that is, the picture list of the half hour is transcoded into a video file.

In another embodiment, the task in step S5 includes a camera ID and a picture that the camera takes in a preset time period.

In another embodiment, step S6 includes:

sending the video stream chunk file to a storage node unit, and simultaneously storing metadata of the chunk file into a database and updating metadata information of a picture;

the downloading of pictures is read in a video stream chunk file.

In this embodiment, the metadata management service unit pre-inputs metadata written by the picture access service, classifies the metadata according to the ID of the snapshot machine, sorts the pictures belonging to a certain snapshot machine ID according to the time sequence, and then encapsulates the pictures belonging to a certain snapshot machine ID into a transcoding task according to the time period, for example, encapsulates the pictures of a certain snapshot machine for one hour into a transcoding task.

The above-mentioned embodiments only express the specific embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.