阅读说明：本技术 一种在客户端实现分级存储的方法及装置 (Method and device for realizing hierarchical storage at client ) 是由 李旭东 于 2021-07-30 设计创作，主要内容包括：本发明提供一种在客户端实现分级存储的方法及装置,所述方法包括如下步骤：S1.在客户端节点设置客户端读写模块和数据管理模块,并配置客户端读写模块和数据管理模块功能；S2.设置数据管理模块采用分布式架构,并获取数据管理模块对接的目标存储系统,建立数据管理块与目标存储系统的接口；S3.客户端读写模块接收用户读写请求,并向一级存储进行读写操作,数据管理模块实现冷数据从一级存储到二级存储的下刷,以及热数据从二级存储到一级存储的提取。本发明提供的在客户端实现分级存储的方法及装置,实现分级存储的基础上,可兼容客户端原有一二级存储架构,并能兼容对中一二级存储系统型号,还可兼容各类私有云及公有云。(The invention provides a method and a device for realizing hierarchical storage at a client, wherein the method comprises the following steps: s1, a client read-write module and a data management module are arranged on a client node, and functions of the client read-write module and the data management module are configured; s2, setting a data management module to adopt a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management module and the target storage system; and S3, the client read-write module receives a user read-write request and performs read-write operation on the primary storage, and the data management module realizes the refreshing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage. The method and the device for realizing the hierarchical storage at the client can be compatible with the original secondary storage framework of the client on the basis of realizing the hierarchical storage, can be compatible with the models of the secondary storage system in the middle and the secondary storage system, and can be compatible with various private clouds and public clouds.)

1. A method for realizing hierarchical storage at a client is characterized by comprising the following steps:

s1, a client read-write module and a data management module are arranged on a client node, and functions of the client read-write module and the data management module are configured;

s2, setting a data management module to adopt a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management module and the target storage system;

and S3, the client read-write module receives a user read-write request and performs read-write operation on the primary storage, and the data management module realizes the refreshing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage.

2. The method for implementing hierarchical storage at a client end according to claim 1, wherein the step S1 includes the following steps:

s11, a client read-write module and a data management module are arranged on a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

s12, configuring a user interface for a client read-write module to receive a read-write request of a user;

s13, configuring a read-write interface for the client read-write module, and performing corresponding read-write operation on the primary storage through the read-write interface according to a read-write request of a user;

s14, detecting cold data from the primary storage by the configuration data management module, and printing the cold data down to the secondary storage;

and S15, detecting thermal data from the secondary storage by the configuration data management module, and extracting the thermal data to the primary storage.

3. The method for implementing hierarchical storage at a client end according to claim 1, wherein the step S2 includes the following steps:

s21, setting a distributed architecture for the data management module;

s22, acquiring a target storage system in butt joint with the data management module;

and S23, establishing an interface which is in butt joint with the target storage system in the data management module, and establishing an interface specification.

4. The method for implementing tiered storage at a client as recited in claim 3 wherein the target storage systems comprise an optical disk library, a magnetic disk library, a public cloud, and a private cloud.

5. The method for implementing hierarchical storage at a client end according to claim 1, wherein the step S3 includes the following steps:

s31, when a user requests to write in file data, the client read-write module writes the file data into a primary storage, and meanwhile, the data management module brushes the file data which the user requests to write in from a background to a secondary storage;

s32, when a user requests to access the file data, the client read-write module searches the file data requested to be accessed from the primary storage and judges whether the file data is searched;

if yes, the client read-write module returns the file data searched from the primary storage to the user, and the step S34 is carried out;

if not, go to step S33;

s33, the client read-write module enters the secondary storage to search the requested file data, returns the searched file data from the secondary storage to the user, meanwhile, the data management module extracts the requested file data into the primary storage, and the step S34 is carried out;

and S34, the data management module detects cold data in the primary storage, writes the cold data into the secondary storage, detects hot data in the secondary storage, and extracts the hot data to the primary storage.

6. The method for implementing hierarchical storage at a client as claimed in claim 5, wherein the step S34 comprises the following steps:

s341, the data management module detects the access frequency of the data in the primary storage, and sets the data with the access frequency lower than a first threshold value as cold data;

s342, the data management module brushes the cold data from the primary storage to the secondary storage in a stripe mode;

s343, the data management module detects the access frequency of the data in the secondary storage, and sets the data with the access frequency higher than a second threshold value as hot data;

s344, the data management module extracts the hot data from the secondary storage to the primary storage;

s345, judging whether the migration speed in the distributed storage system meets the requirement;

if yes, ending;

if not, deploying a plurality of data management modules according to the required migration speed, and setting the data management modules to process data migration between the primary storage and the secondary storage in parallel.

7. An apparatus for implementing hierarchical storage at a client, comprising:

the module adding unit (1) is used for setting a client read-write module and a data management module at a client node and configuring the functions of the client read-write module and the data management module;

the data management module interface setting unit (2) is used for setting that the data management module adopts a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management block and the target storage system;

and the data migration unit (3) is used for receiving the read-write request of the user by the client read-write module and performing read-write operation on the primary storage, and the data management module realizes the down-flushing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage.

8. The apparatus for implementing a hierarchical storage at a client according to claim 7, wherein the module adding unit (1) comprises:

the read-write module data management module setting subunit (1.1) is used for setting a client read-write module and a data management module at a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

the read-write module configuration subunit (1.2) is used for configuring a user interface for the client read-write module to receive a read-write request of a user;

the read-write interface configuration subunit (1.3) is used for configuring a read-write interface for the client read-write module and performing corresponding read-write operation to the primary storage through the read-write interface according to a read-write request of a user;

the data brushing-down configuration subunit (1.4) is used for configuring the data management module to detect cold data from the primary storage and brush down the cold data to the secondary storage;

and the data extraction configuration subunit (1.5) is used for configuring the data management module to detect the thermal data from the secondary storage and extract the thermal data to the primary storage.

9. The apparatus for implementing a hierarchical storage at a client according to claim 7, wherein the data management module interface setting unit (2) comprises:

the data management module architecture setting subunit (2.1) is used for setting the data management module to adopt a distributed architecture;

the target storage system acquisition subunit (2.2) is used for acquiring the target storage system butted by the data management module;

an interface creation subunit (2.3) for creating an interface at the data management module to interface with the target storage system and creating an interface specification.

10. The apparatus for implementing a hierarchical storage at a client according to claim 7, wherein the data migration unit (3) comprises:

the data writing subunit (3.1) is used for writing the file data into the primary storage by the client read-write module when a user requests to write the file data, and brushing the file data requested to be written by the user from the background to the secondary storage by the data management module;

the data reading subunit (3.2) is used for searching the file data requested to be accessed from the primary storage by the client reading and writing module when the user requests to access the file data, and judging whether the file data is searched;

the first data return subunit (3.3) is used for returning the file data searched from the primary storage to the user by the client read-write module when the data is read from the primary storage;

the data extraction subunit (3.4) is used for enabling the client read-write module to enter the file data of the secondary storage search request when the data is not read from the primary storage, returning the file data searched from the secondary storage to a user, and enabling the data management module to extract the file data of the request into the primary storage;

and the data migration subunit (3.5) is used for detecting cold data in the primary storage, brushing the cold data to the secondary storage, detecting hot data in the secondary storage and extracting the hot data to the primary storage by the data management module.

Technical Field

The invention belongs to the technical field of hierarchical storage of distributed storage, and particularly relates to a method and a device for realizing hierarchical storage at a client.

Background

The application system of the enterprise processes a large amount of data on line, and along with the continuous increase of the data amount, if the traditional online storage mode is adopted, large-capacity local primary storage is needed. This results in a relatively large investment on the one hand; on the other hand, most of the data access rate of the primary storage is not high, but still occupies the hard disk space, which results in the reduction of the access speed. In this case, the hierarchical storage management software, or data migration software, may make the best balance between performance and price. Hierarchical Storage Management (HSM) has been developed.

Hierarchical storage management is a technology that merges offline storage with online storage. The method takes a high-speed and high-capacity off-line storage device as the next level device of a disk device, and then automatically migrates the data which is not commonly used in the disk to a secondary mass storage device such as a tape library according to a specified strategy. When the data is needed, the hierarchical storage system will automatically call the data from the next storage device back to the previous disk. The data migration operation described above is completely transparent to the user, only slightly slowed in the speed of accessing the disk, while perceived as a significant increase in the capacity of the logical disk. In general, data migration is a technique that can store a large amount of data that is not accessed frequently on an off-line medium such as a tape library or an optical disk library, and only store a small amount of data with high access frequency on a disk array. When data on those media, such as magnetic tape, is accessed, the system automatically migrates the data back to the disk array; also, data in the disk array that has not been accessed for a long time is automatically migrated to the tape media, thereby greatly reducing investment and management costs.

However, the traditional hierarchical storage management system also has the defects of poor compatibility to a secondary storage architecture, few supported secondary storage models and the like.

This is a disadvantage of the prior art, and therefore, it is very necessary to provide a method and an apparatus for implementing hierarchical storage at a client to solve the above-mentioned disadvantages in the prior art.

Disclosure of Invention

Aiming at the defects that the traditional hierarchical memory management system in the prior art is poor in compatibility to a secondary memory architecture and few in supported secondary memory models, the invention provides a method and a device for realizing hierarchical memory at a client side so as to solve the technical problems.

In a first aspect, the present invention provides a method for implementing hierarchical storage at a client, including the following steps:

s1, a client read-write module and a data management module are arranged on a client node, and functions of the client read-write module and the data management module are configured;

s2, setting a data management module to adopt a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management module and the target storage system;

and S3, the client read-write module receives a user read-write request and performs read-write operation on the primary storage, and the data management module realizes the refreshing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage.

Further, the step S1 specifically includes the following steps:

s11, a client read-write module and a data management module are arranged on a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

s12, configuring a user interface for a client read-write module to receive a read-write request of a user;

s13, configuring a read-write interface for the client read-write module, and performing corresponding read-write operation on the primary storage through the read-write interface according to a read-write request of a user;

s14, detecting cold data from the primary storage by the configuration data management module, and printing the cold data down to the secondary storage;

and S15, detecting thermal data from the secondary storage by the configuration data management module, and extracting the thermal data to the primary storage.

Further, the step S2 specifically includes the following steps:

s21, setting a distributed architecture for the data management module;

s22, acquiring a target storage system in butt joint with the data management module;

and S23, establishing an interface which is in butt joint with the target storage system in the data management module, and establishing an interface specification.

Further, the target storage system comprises an optical disk library, a magnetic disk library, a public cloud and a private cloud.

Further, the step S3 specifically includes the following steps:

s31, when a user requests to write in file data, the client read-write module writes the file data into a primary storage, and meanwhile, the data management module brushes the file data which the user requests to write in from a background to a secondary storage;

s32, when a user requests to access the file data, the client read-write module searches the file data requested to be accessed from the primary storage and judges whether the file data is searched;

if yes, the client read-write module returns the file data searched from the primary storage to the user, and the step S34 is carried out;

if not, go to step S33;

s33, the client read-write module enters the secondary storage to search the requested file data, returns the searched file data from the secondary storage to the user, meanwhile, the data management module extracts the requested file data into the primary storage, and the step S34 is carried out;

and S34, the data management module detects cold data in the primary storage, writes the cold data into the secondary storage, detects hot data in the secondary storage, and extracts the hot data to the primary storage.

Further, the step S34 specifically includes the following steps:

s341, the data management module detects the access frequency of the data in the primary storage, and sets the data with the access frequency lower than a first threshold value as cold data;

s342, the data management module brushes the cold data from the primary storage to the secondary storage in a stripe mode;

s343, the data management module detects the access frequency of the data in the secondary storage, and sets the data with the access frequency higher than a second threshold value as hot data;

s344, the data management module extracts the hot data from the secondary storage to the primary storage;

s345, judging whether the migration speed in the distributed storage system meets the requirement;

if yes, ending;

if not, deploying a plurality of data management modules according to the required migration speed, and setting the data management modules to process data migration between the primary storage and the secondary storage in parallel.

In a second aspect, the present invention provides an apparatus for implementing hierarchical storage at a client, including:

the module adding unit is used for setting a client read-write module and a data management module at a client node and configuring the functions of the client read-write module and the data management module;

the data management module interface setting unit is used for setting the data management module to adopt a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management block and the target storage system;

and the data migration unit is used for receiving the read-write request of the user by the client read-write module and performing read-write operation on the primary storage, and the data management module realizes the down-flushing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage.

Further, the module adding unit includes:

the read-write module data management module setting subunit is used for setting a client read-write module and a data management module at a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

the read-write module configuration subunit is used for configuring a user interface for the client read-write module to receive a read-write request of a user;

the read-write interface configuration subunit is used for configuring a read-write interface for the client read-write module and performing corresponding read-write operation on the primary storage through the read-write interface according to a read-write request of a user;

the data brushing-down configuration subunit is used for configuring the data management module to detect cold data from the primary storage and brush down the cold data to the secondary storage;

and the data extraction configuration subunit is used for configuring the data management module to detect the thermal data from the secondary storage and extract the thermal data to the primary storage.

Further, the data management module interface setting unit includes:

the data management module framework setting subunit is used for setting the data management module to adopt a distributed framework;

the target storage system acquisition subunit is used for acquiring the target storage system butted by the data management module;

and the interface creating subunit is used for creating an interface which is in butt joint with the target storage system in the data management module and creating an interface specification.

Further, the data migration unit includes:

the data writing subunit is used for writing the file data into the primary storage by the client read-write module when a user requests to write the file data, and brushing the file data requested to be written by the user from the background to the secondary storage by the data management module;

the data reading sub-unit is used for searching the file data requested to be accessed from the primary storage by the client reading and writing module when a user requests to access the file data, and judging whether the file data is searched;

the first data return subunit is used for returning the file data searched from the primary storage to the user by the client read-write module when the data is read from the primary storage;

the data extraction subunit is used for enabling the client read-write module to enter the secondary storage to search the requested file data when the data is not read from the primary storage, returning the searched file data from the secondary storage to a user, and enabling the data management module to extract the requested file data into the primary storage;

and the data migration subunit is used for detecting cold data in the primary storage, brushing the cold data into the secondary storage, detecting hot data in the secondary storage and extracting the hot data to the primary storage by the data management module.

Further, the data migration subunit includes:

the cold data detection component is used for detecting the access frequency of the data in the primary storage by the data management module and setting the data with the access frequency lower than a first threshold value as cold data;

the cold data down-brushing component is used for the data management module to brush down the cold data from the primary storage to the secondary storage in a stripe mode;

the thermal data detection component is used for detecting the access frequency of the data in the secondary storage by the data management module and setting the data with the access frequency higher than a second threshold value as the thermal data;

the hot data extraction component is used for extracting the hot data from the secondary storage to the primary storage by the data management module;

the migration speed judging component is used for judging whether the migration speed in the distributed storage system meets the requirement or not;

the parallel data management module setting component is used for deploying a plurality of data management modules according to the required migration speed when the migration speed does not meet the requirement, and setting the data management modules to process data migration between the primary storage and the secondary storage in parallel.

The beneficial effect of the invention is that,

the method and the device for realizing the hierarchical storage at the client can be compatible with the original secondary storage framework of the client on the basis of realizing the hierarchical storage, can be compatible with the models of the secondary storage system in the middle and the secondary storage system, and can be compatible with various private clouds and public clouds.

In addition, the invention has reliable design principle, simple structure and very wide application prospect.

Therefore, compared with the prior art, the invention has prominent substantive features and remarkable progress, and the beneficial effects of the implementation are also obvious.

Drawings

In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.

Fig. 1 is a first flowchart illustrating a method for implementing hierarchical storage at a client according to the present invention.

Fig. 2 is a schematic flow chart of a method for implementing hierarchical storage at a client according to the present invention.

FIG. 3 is a schematic diagram of an apparatus for implementing hierarchical storage at a client according to the present invention.

In the figure, 1-module adds unit; 1.1-the data management module of the read-write module is provided with a subunit; 1.2-read-write module configuration subunit; 1.3-read-write interface configuration subunit; 1.4-data lower brush configuration subunit; 1.5-data extraction configuration subunit; 2-data management module interface setting unit; 2.1-data management module architecture setup subunit; 2.2-target storage system acquisition subunit; 2.3-interface creation subunit; 3-a data migration unit; 3.1-data write subunit; 3.2-data read subunit; 3.3-first data return subunit; 3.4-data extraction subunit; 3.5-data migration subunit.

Detailed Description

In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Example 1:

as shown in fig. 1, the present invention provides a method for implementing hierarchical storage at a client, including the following steps:

s1, a client read-write module and a data management module are arranged on a client node, and functions of the client read-write module and the data management module are configured;

s2, setting a data management module to adopt a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management module and the target storage system;

and S3, the client read-write module receives a user read-write request and performs read-write operation on the primary storage, and the data management module realizes the refreshing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage.

Example 2:

as shown in fig. 2, the present invention provides a method for implementing hierarchical storage at a client, including the following steps:

s1, a client read-write module and a data management module are arranged on a client node, and functions of the client read-write module and the data management module are configured; the method comprises the following specific steps:

s11, a client read-write module and a data management module are arranged on a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

s12, configuring a user interface for a client read-write module to receive a read-write request of a user;

s13, configuring a read-write interface for the client read-write module, and performing corresponding read-write operation on the primary storage through the read-write interface according to a read-write request of a user;

s14, detecting cold data from the primary storage by the configuration data management module, and printing the cold data down to the secondary storage;

s15, detecting thermal data from the secondary storage by the configuration data management module, and extracting the thermal data to the primary storage;

s2, setting a data management module to adopt a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management module and the target storage system; the method comprises the following specific steps:

s21, setting a distributed architecture for the data management module;

s22, acquiring a target storage system in butt joint with the data management module;

s23, an interface which is in butt joint with a target storage system is established in a data management module, and an interface specification is established;

s3, a client read-write module receives a user read-write request and performs read-write operation on primary storage, and a data management module realizes the refreshing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage; the step S3 includes the following steps:

s31, when a user requests to write in file data, the client read-write module writes the file data into a primary storage, and meanwhile, the data management module brushes the file data which the user requests to write in from a background to a secondary storage;

s32, when a user requests to access the file data, the client read-write module searches the file data requested to be accessed from the primary storage and judges whether the file data is searched;

if yes, the client read-write module returns the file data searched from the primary storage to the user, and the step S34 is carried out;

if not, go to step S33;

s33, the client read-write module enters the secondary storage to search the requested file data, returns the searched file data from the secondary storage to the user, meanwhile, the data management module extracts the requested file data into the primary storage, and the step S34 is carried out;

and S34, the data management module detects cold data in the primary storage, writes the cold data into the secondary storage, detects hot data in the secondary storage, and extracts the hot data to the primary storage.

Example 3:

as shown in fig. 2, the present invention provides a method for implementing hierarchical storage at a client, including the following steps:

s1, a client read-write module and a data management module are arranged on a client node, and functions of the client read-write module and the data management module are configured; the method comprises the following specific steps:

s11, a client read-write module and a data management module are arranged on a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

s12, configuring a user interface for a client read-write module to receive a read-write request of a user;

s13, configuring a read-write interface for the client read-write module, and performing corresponding read-write operation on the primary storage through the read-write interface according to a read-write request of a user;

s14, detecting cold data from the primary storage by the configuration data management module, and printing the cold data down to the secondary storage;

s15, detecting thermal data from the secondary storage by the configuration data management module, and extracting the thermal data to the primary storage;

s2, setting a data management module to adopt a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management module and the target storage system; the method comprises the following specific steps:

s21, setting a distributed architecture for the data management module;

s22, acquiring a target storage system in butt joint with the data management module; the target storage system comprises an optical disk library, a magnetic disk library, a common cloud and a private cloud;

s23, an interface which is in butt joint with a target storage system is established in a data management module, and an interface specification is established;

s3, a client read-write module receives a user read-write request and performs read-write operation on primary storage, and a data management module realizes the refreshing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage; the step S3 includes the following steps:

s31, when a user requests to write in file data, the client read-write module writes the file data into a primary storage, and meanwhile, the data management module brushes the file data which the user requests to write in from a background to a secondary storage;

s32, when a user requests to access the file data, the client read-write module searches the file data requested to be accessed from the primary storage and judges whether the file data is searched;

if yes, the client read-write module returns the file data searched from the primary storage to the user, and the step S34 is carried out;

if not, go to step S33;

s33, the client read-write module enters the secondary storage to search the requested file data, returns the searched file data from the secondary storage to the user, meanwhile, the data management module extracts the requested file data into the primary storage, and the step S34 is carried out;

s34, the data management module detects cold data in the primary storage, writes the cold data into the secondary storage, detects hot data in the secondary storage, and extracts the hot data into the primary storage; the method comprises the following specific steps:

s341, the data management module detects the access frequency of the data in the primary storage, and sets the data with the access frequency lower than a first threshold value as cold data;

s342, the data management module brushes the cold data from the primary storage to the secondary storage in a stripe mode;

s343, the data management module detects the access frequency of the data in the secondary storage, and sets the data with the access frequency higher than a second threshold value as hot data;

s344, the data management module extracts the hot data from the secondary storage to the primary storage;

s345, judging whether the migration speed in the distributed storage system meets the requirement;

if yes, ending;

if not, deploying a plurality of data management modules according to the required migration speed, and setting the data management modules to process data migration between the primary storage and the secondary storage in parallel.

Example 4:

as shown in fig. 3, the present invention provides an apparatus for implementing hierarchical storage at a client, including:

the module adding unit 1 is used for setting a client read-write module and a data management module at a client node and configuring the functions of the client read-write module and the data management module;

the data management module interface setting unit 2 is used for setting that the data management module adopts a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management block and the target storage system;

and the data migration unit 3 is used for receiving a user read-write request by the client read-write module and performing read-write operation on the primary storage, and the data management module realizes the down-flushing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage.

Example 5:

as shown in fig. 3, the present invention provides an apparatus for implementing hierarchical storage at a client, including:

the module adding unit 1 is used for setting a client read-write module and a data management module at a client node and configuring the functions of the client read-write module and the data management module; the module adding unit 1 includes:

the read-write module data management module is provided with a subunit 1.1, which is used for arranging a client read-write module and a data management module at a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

the read-write module configuration subunit 1.2 is used for configuring a user interface for the client read-write module to receive a read-write request of a user;

the read-write interface configuration subunit 1.3 is used for configuring a read-write interface for the client read-write module and performing corresponding read-write operation to the primary storage through the read-write interface according to a read-write request of a user;

the data downloading and brushing configuration subunit 1.4 is used for configuring the data management module to detect cold data from the primary storage and download and brush the cold data to the secondary storage;

the data extraction configuration subunit 1.5 is used for configuring the data management module to detect the thermal data from the secondary storage and extract the thermal data to the primary storage;

the data management module interface setting unit 2 is used for setting that the data management module adopts a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management block and the target storage system; the data management module interface setting unit 2 includes:

the data management module framework setting subunit 2.1 is used for setting the data management module to adopt a distributed framework;

the target storage system acquisition subunit 2.2 is used for acquiring the target storage system butted by the data management module;

the interface creating subunit 2.3 is used for creating an interface which is butted with the target storage system in the data management module and creating an interface specification;

the data migration unit 3 is used for receiving a user read-write request by the client read-write module and performing read-write operation on the primary storage, and the data management module realizes the down-brushing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage; the data migration unit 3 includes:

the data writing subunit 3.1 is configured to, when a user requests to write in file data, write the file data into the primary storage by the client read-write module, and brush the file data requested to be written by the user from the background to the secondary storage by the data management module;

the data reading subunit 3.2 is used for searching the file data requested to be accessed from the primary storage by the client reading and writing module when the user requests to access the file data, and judging whether the file data is searched;

the first data return subunit 3.3 is used for returning the file data searched from the primary storage to the user by the client read-write module when the data is read from the primary storage;

the data extraction subunit 3.4 is used for entering the file data of the secondary storage search request by the client read-write module when the data is not read from the primary storage, returning the file data searched from the secondary storage to the user, and extracting the file data of the request into the primary storage by the data management module;

and the data migration subunit 3.5 is used for detecting cold data in the primary storage, brushing the cold data into the secondary storage, detecting hot data in the secondary storage, and extracting the hot data to the primary storage by the data management module.

Example 6:

as shown in fig. 3, the present invention provides an apparatus for implementing hierarchical storage at a client, including:

the module adding unit 1 is used for setting a client read-write module and a data management module at a client node and configuring the functions of the client read-write module and the data management module; the module adding unit 1 includes:

the read-write module data management module is provided with a subunit 1.1, which is used for arranging a client read-write module and a data management module at a client node; the client read-write module and the data management module are compatible with client nodes of different operating systems;

the read-write module configuration subunit 1.2 is used for configuring a user interface for the client read-write module to receive a read-write request of a user;

the read-write interface configuration subunit 1.3 is used for configuring a read-write interface for the client read-write module and performing corresponding read-write operation to the primary storage through the read-write interface according to a read-write request of a user;

the data downloading and brushing configuration subunit 1.4 is used for configuring the data management module to detect cold data from the primary storage and download and brush the cold data to the secondary storage;

the data extraction configuration subunit 1.5 is used for configuring the data management module to detect the thermal data from the secondary storage and extract the thermal data to the primary storage;

the data management module interface setting unit 2 is used for setting that the data management module adopts a distributed architecture, acquiring a target storage system which is butted by the data management module, and establishing an interface between the data management block and the target storage system; the data management module interface setting unit 2 includes:

the data management module framework setting subunit 2.1 is used for setting the data management module to adopt a distributed framework;

the target storage system acquisition subunit 2.2 is used for acquiring the target storage system butted by the data management module;

the interface creating subunit 2.3 is used for creating an interface which is butted with the target storage system in the data management module and creating an interface specification;

the data migration unit 3 is used for receiving a user read-write request by the client read-write module and performing read-write operation on the primary storage, and the data management module realizes the down-brushing of cold data from the primary storage to the secondary storage and the extraction of hot data from the secondary storage to the primary storage; the data migration unit 3 includes:

the data writing subunit 3.1 is configured to, when a user requests to write in file data, write the file data into the primary storage by the client read-write module, and brush the file data requested to be written by the user from the background to the secondary storage by the data management module;

the data reading subunit 3.2 is used for searching the file data requested to be accessed from the primary storage by the client reading and writing module when the user requests to access the file data, and judging whether the file data is searched;

the first data return subunit 3.3 is used for returning the file data searched from the primary storage to the user by the client read-write module when the data is read from the primary storage;

the data extraction subunit 3.4 is used for entering the file data of the secondary storage search request by the client read-write module when the data is not read from the primary storage, returning the file data searched from the secondary storage to the user, and extracting the file data of the request into the primary storage by the data management module;

the data migration subunit 3.5 is used for the data management module to detect cold data in the primary storage, to brush the cold data down to the secondary storage, to detect hot data in the secondary storage, and to extract the hot data to the primary storage; the data migration subunit 3.5 includes:

the cold data detection component is used for detecting the access frequency of the data in the primary storage by the data management module and setting the data with the access frequency lower than a first threshold value as cold data;

the cold data down-brushing component is used for the data management module to brush down the cold data from the primary storage to the secondary storage in a stripe mode;

the thermal data detection component is used for detecting the access frequency of the data in the secondary storage by the data management module and setting the data with the access frequency higher than a second threshold value as the thermal data;

the hot data extraction component is used for extracting the hot data from the secondary storage to the primary storage by the data management module;

the migration speed judging component is used for judging whether the migration speed in the distributed storage system meets the requirement or not;

the parallel data management module setting component is used for deploying a plurality of data management modules according to the required migration speed when the migration speed does not meet the requirement, and setting the data management modules to process data migration between the primary storage and the secondary storage in parallel.

Although the present invention has been described in detail by referring to the drawings in connection with the preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made on the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and these modifications or substitutions are within the scope of the present invention/any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.