PL reverse metadata storage method, system, device and storage medium

文档序号：970845 发布日期：2020-11-03 浏览：12次中文

阅读说明：本技术 一种pl反向元数据的存储方法、系统、设备及存储介质 (PL reverse metadata storage method, system, device and storage medium ) 是由意刘如孙京本李佩于 2020-07-28 设计创作，主要内容包括：本申请公开了一种PL反向元数据的存储方法,应用于基于LSA的全闪存储系统中,包括：在生成PL反向元数据并置入缓存时,按照预设的分配规则为PL反向元数据分配对应的树目录以及在树目录下的树；针对任意一棵树,当该颗树满足设定的刷盘条件时,将该颗树中的全部PL反向元数据从缓存下刷至磁盘中；分配规则中设定了M个树目录,全闪存储系统中具有N个CPU核心,每个CPU核心用于负责该CPU核心对应的各个树目录的数据处理,且每个CPU核心至少与一个树目录对应,M和N均为不小于2的正整数。应用本申请的方案,有利于提高下刷效率,提升系统性能。本申请还提供了一种PL反向元数据的存储系统、设备及存储介质,具有相应效果。(The application discloses a storage method of PL reverse metadata, which is applied to a full flash storage system based on LSA, and comprises the following steps: when generating PL reverse metadata and placing the PL reverse metadata into a cache, distributing a corresponding tree directory and a tree under the tree directory for the PL reverse metadata according to a preset distribution rule; for any tree, when the tree meets a set disk-brushing condition, brushing all PL reverse metadata in the tree from a buffer to a disk; m tree directories are set in the allocation rule, the full flash memory system is provided with N CPU cores, each CPU core is used for data processing of each tree directory corresponding to the CPU core, each CPU core at least corresponds to one tree directory, and M and N are positive integers not less than 2. By the scheme, lower brushing efficiency is improved, and system performance is improved. The application also provides a storage system, equipment and a storage medium of the PL reverse metadata, and the storage system, the equipment and the storage medium have corresponding effects.)

1. A storage method of PL reverse metadata is applied to a full flash storage system based on LSA, and comprises the following steps:

when generating PL reverse metadata and placing the PL reverse metadata into a cache, distributing a corresponding tree directory and a tree under the tree directory for the PL reverse metadata according to a preset distribution rule;

for any tree, when the tree meets a set disk-brushing condition, brushing all PL reverse metadata in the tree from a buffer to a disk;

the distribution rule sets M tree directories, the full flash memory system is provided with N CPU cores, each CPU core is used for data processing of each tree directory corresponding to the CPU core, each CPU core at least corresponds to one tree directory, and M and N are positive integers not less than 2.

2. The method for storing PL reverse metadata according to claim 1, wherein the allocating a corresponding tree directory and a tree under the tree directory for the PL reverse metadata according to a preset allocation rule comprises:

and allocating a corresponding tree directory and a tree under the tree directory for the PL reverse metadata based on the PBAs in the PL reverse metadata, so that two PL reverse metadata with adjacent PBAs can be allocated to different tree directories.

3. The method for storing PL reverse metadata according to claim 1, wherein the allocating a corresponding tree directory and a tree under the tree directory for the PL reverse metadata according to a preset allocation rule comprises:

using the value formed by bit A to bit B of the PBA in the PL reverse metadata as a tree directory distributed for the PL reverse metadata; wherein A and B are preset positive integers, and A is less than B;

and allocating a corresponding tree under the tree directory for the PL reverse metadata based on the PBA in the PL reverse metadata.

4. The method of claim 3, wherein the allocating the corresponding tree under the tree directory for the PL reverse metadata based on the PBA in the PL reverse metadata comprises:

taking a numerical value formed by bit C to bit D and bit E to bit F of the PBA in the PL reverse metadata as a tree under the tree directory distributed for the PL reverse metadata; wherein C, D, E and F are all preset positive integers, and D is more than or equal to C and E is more than or equal to F.

5. The PL reverse metadata storage method according to any one of claims 1 to 4, wherein for any tree, when the tree meets a set disk-flushing condition, flushing all PL reverse metadata in the tree from cache to disk comprises:

and for any tree, when the number of the PL reverse metadata in the tree reaches a lower brushing threshold configured for the tree, all the PL reverse metadata in the tree is flushed from the buffer to the disk.

6. The method for storing PL reverse metadata according to claim 5, wherein for any tree under the same tree directory, the lower-brushing thresholds of other trees are different from the lower-brushing threshold of the tree under the tree directory.

7. A storage system of PL reverse metadata, which is applied to an LSA-based full flash storage system, and comprises:

the system comprises a tree splitting module, a cache module and a cache module, wherein the tree splitting module is used for distributing a corresponding tree directory and a tree under the tree directory for PL reverse metadata according to a preset distribution rule when the PL reverse metadata is generated and is placed in the cache;

the flash module is used for flashing all PL reverse metadata in any tree into a disk from the cache when the tree meets a set disk-flashing condition;

8. The system for storing PL reverse metadata as claimed in claim 7, wherein said treeing module is specifically configured to:

when generating PL reverse metadata and placing the PL reverse metadata into a cache, allocating a corresponding tree directory and a tree under the tree directory for the PL reverse metadata based on PBAs in the PL reverse metadata, so that two PL reverse metadata with adjacent PBAs can be allocated to different tree directories.

9. A storage device for PL reverse metadata, comprising:

a memory for storing a computer program;

a processor for executing the computer program to implement the steps of the method of storage of PL reverse metadata according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that a computer program is stored thereon, which when executed by a processor, implements the steps of the storage method of PL reverse metadata according to any one of claims 1 to 6.

Technical Field

The present invention relates to the field of storage technologies, and in particular, to a method, a system, a device, and a storage medium for storing PL reverse metadata.

Background

With the popularization of SSD hard disks, full flash memory systems are becoming a trend. The LSA is a key technology in the full flash memory storage system, and can convert random small block writing into continuous large block writing through metadata, thereby improving the performance of the storage system and the service life of the SSD disk. The metadata is data describing data, where the LP metadata is used to map an LBA (Logical Block Address) issued by a client to a PBA (physical Block Address), that is, L of the LP metadata refers to the LBA, P refers to the PBA.

In large-scale storage systems, the amount of metadata is very large, and the memory cannot be completely stored, so the metadata needs to be finally landed. Currently, no special PL reverse metadata landing scheme is designed, that is, a conventional cache data landing scheme is adopted, for example, PL reverse metadata is flushed to a disk through a set thread, and the flushing performance of such a scheme still needs to be improved.

In summary, how to effectively store the PL reverse metadata and improve the brushing efficiency is a technical problem that needs to be solved by those skilled in the art.

Disclosure of Invention

The invention aims to provide a storage method, a system, equipment and a storage medium of PL reverse metadata, so as to improve the brushing efficiency and the system performance.

In order to solve the technical problems, the invention provides the following technical scheme:

a storage method of PL reverse metadata is applied to a full flash storage system based on LSA, and comprises the following steps:

for any tree, when the tree meets a set disk-brushing condition, brushing all PL reverse metadata in the tree from a buffer to a disk;

Preferably, the allocating, according to a preset allocation rule, a corresponding tree directory and a tree under the tree directory for the PL reverse metadata includes:

and allocating a corresponding tree under the tree directory for the PL reverse metadata based on the PBA in the PL reverse metadata.

Preferably, the allocating a corresponding tree under the tree directory for the PL reverse metadata based on the PBA in the PL reverse metadata includes:

Preferably, for any tree, when the tree satisfies the set flushing condition, flushing all PL reverse metadata in the tree from the buffer to the disk includes:

Preferably, for any tree in the same tree directory, the lower-brushing thresholds of other trees are different from the lower-brushing threshold of the tree in the tree directory.

A storage system of PL reverse metadata, which is applied to an LSA-based full flash storage system, and comprises:

the flash module is used for flashing all PL reverse metadata in any tree into a disk from the cache when the tree meets a set disk-flashing condition;

Preferably, the tree splitting module is specifically configured to:

A storage device for PL reverse metadata, comprising:

a memory for storing a computer program;

a processor for executing the computer program to implement the steps of any one of the above-described storage methods of PL reverse metadata.

A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any one of the above-described methods of storage of PL reverse metadata.

By applying the technical scheme provided by the embodiment of the invention, the concurrency can be improved and the system performance can be improved by multiple times under the multi-core CPU environment, so that the PL reverse metadata can be landed based on N CPU cores. In order to enable multiple CPU cores to work in parallel, the present application divides the PL inverse metadata into two levels, the level of the tree directory and the level of the tree. When generating PL reverse metadata and placing it in a cache, a corresponding tree directory and a tree under the tree directory may be allocated for the PL reverse metadata according to a preset allocation rule. Each CPU core corresponds to at least one tree directory, and each CPU core is used for being responsible for data processing of each tree directory corresponding to the CPU core, so that the partition of PL reverse metadata enables the application to utilize N CPU cores to work in parallel, thereby being beneficial to improving the lower brushing efficiency and improving the system performance.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a flow chart of an embodiment of a storage method of PL reverse metadata according to the present invention;

FIG. 2 is a format diagram of PBA of PL reverse metadata in an embodiment of the present invention;

FIG. 3 is a schematic diagram of a storage system for PL reverse metadata according to the present invention;

FIG. 4 is a schematic diagram of a storage device for PL reverse metadata according to the present invention.

Detailed Description

The core of the invention is to provide a storage method of PL reverse metadata, which is beneficial to improving the brushing efficiency and the system performance.

In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, fig. 1 is a flowchart illustrating an implementation of a PL reverse metadata storage method according to the present invention, which can be applied to an LSA-based full flash storage system, and includes the following steps:

step S101: when generating PL reverse metadata and placing the PL reverse metadata into a cache, distributing a corresponding tree directory and a tree under the tree directory for the PL reverse metadata according to a preset distribution rule;

step S102: for any tree, when the tree meets a set disk-brushing condition, brushing all PL reverse metadata in the tree from a buffer to a disk;

Specifically, the specific values of the CPU cores N and the total number M of the tree directories can be set according to actual needs, and the purpose of the application is to enable the N CPU cores to work in parallel, so that the system performance is improved, and each CPU core at least corresponds to one tree directory. Of course, in practical applications, generally the same number of tree directories may be allocated to each CPU core, for example, in a specific case, 128 tree directories are provided, and when 8 CPU cores are provided, each CPU core may correspond to 16 tree directories. Of course, in actual application, the number of tree directories corresponding to each CPU core may be appropriately adjusted according to actual needs.

The tree directory described in the present application corresponds to a CPU, and means: each tree directory has a corresponding thread, and each thread has a CPU core corresponding to the thread, that is, for each tree directory, a uniquely determined CPU core is used for data processing in charge of the tree directory, that is, processing of the thread corresponding to the tree directory, so that the tree directory is referred to as corresponding to the CPU core.

When distributing the corresponding tree directory and the tree under the tree directory for the PL inverse metadata according to the preset distribution rule, the specific form of the distribution rule may be set and adjusted according to actual needs, but it is understood that, in order to make the N CPU cores work in parallel as much as possible and to take balance of each CPU core into consideration, the generated PL inverse metadata should be equally distributed to each tree directory as much as possible.

For example, in one embodiment of the present invention, step S101 describes: allocating a corresponding tree directory and a tree under the tree directory for the PL reverse metadata according to a preset allocation rule may include:

based on the PBAs in the PL reverse metadata, the PL reverse metadata is assigned a corresponding tree directory and a tree under the tree directory, so that two PL reverse metadata with adjacent PBAs will be assigned to different tree directories.

In this embodiment, considering that LSA is a key technology in the full flash storage system, random small block writing can be converted into continuous large block writing, and the feature of converting random writing into sequential writing determines that the PBA allocation is sequential within a certain range. That is, among the generated PL reverse metadata, PBAs are allocated in order. Thus, in such an embodiment, PL reverse metadata may be assigned a corresponding tree directory and a tree under the tree directory based on the PBAs in the PL reverse metadata, such that two PL reverse metadata with adjacent PBAs would be assigned to different tree directories.

For example, 128 tree directories are provided, PL inverse metadata is assigned to tree directory 0 when the value of PBA is 0, PL inverse metadata is assigned to tree directory 1 when the value of PBA is 1, and so on, PL inverse metadata is assigned to tree directory 127 when the value of PBA is 127. When the value of PBA is 128, PL reverse metadata is assigned to tree directory 0. It can be seen that, in this embodiment, since two pieces of PL reverse metadata having adjacent PBAs are distributed to different tree directories, the PL reverse metadata can be uniformly distributed in M tree directories, so that N CPU cores can be used to take charge of data processing of the M tree directories, and the balance of each CPU core is ensured.

In a specific embodiment of the present invention, step S101 may specifically include:

the method comprises the following steps: using the value formed by bit A to bit B of PBA in the PL reverse metadata as a tree directory allocated for the PL reverse metadata; wherein A and B are preset positive integers, and A is less than B.

Step two: based on the PBA in the PL reverse metadata, the PL reverse metadata is assigned a tree under the corresponding tree directory.

In this embodiment, it is considered that in the above-described embodiment, two pieces of PL inverse metadata having adjacent PBAs are allocated to different tree directories, and although it is realized that the PL inverse metadata are equally allocated to different tree directories as much as possible, such an embodiment generally requires the allocation of the tree directories in accordance with the numerical values of the entire PBAs. In this embodiment, the tree directory can be determined only according to the partial bits, i.e., bitA to bitB, of the PBA in the PL reverse metadata, so that implementation of the scheme is simpler.

For example, in the embodiment of fig. 2, a PBA format of 64 bits in total is set in consideration of the compression specification of 8 byte alignment in order to support 1PB capacity. Specifically, in the embodiment of fig. 2, bits 0 to bit 2, and these 3 bits are reserved for expansion, so as to maintain the extensibility of the scheme. bit 3 is used to indicate whether data is compressed, i.e. this embodiment of the application can support compression of PBA. The 10 bits from bit 4 to bit 13 can represent the value range from 0 to 1023, and the length range of PBA can reach 0 to 8K by combining the convention of 8-byte alignment.

In the embodiment of fig. 2, bit 14 to bit 63, the 50 bits, may represent the range of 0 to 1PB, and the PBA may represent the range of maximum 8PB by combining the convention of 8 byte alignment.

The specific values of a and B may be set and adjusted according to actual needs, for example, in the embodiment of fig. 2, bit a is bit 32, bit B is bit 38, that is, 128 tree directories are selected according to the 7 bits. I.e. root page is selected. In the embodiment, according to the numerical value formed by bit a to bit B of the PBA in the PL reverse metadata, the tree directory is allocated to the PL reverse metadata, and in consideration of the characteristic that random writing is changed into sequential writing, the allocation of PBAs in a certain range is sequential, a part of bits in the PBAs are selected to allocate the tree directory, and adjacent PBAs can be divided into different tree directories as much as possible, that is, each PL reverse metadata is substantially uniformly distributed in each tree directory, so as to ensure the balance of each CPU core.

For the distribution of the trees in the tree directory, the specific rules may also be set and adjusted as required, for example, the rules may be uniformly distributed to the trees in the tree directory. In such an implementation, the PL reverse metadata may be assigned a tree under the corresponding tree directory based on the PBA in the PL reverse metadata.

The specific manner of allocating the corresponding tree under the tree directory to the PL reverse metadata based on the PBA in the PL reverse metadata may also be set as needed, for example, a tree under the tree directory may be selected according to the values of a plurality of bits in the PBA.

Further, in an embodiment of the present invention, the step two may specifically include:

taking the value formed by bitC to bitD and bitE to bitF of PBA in the PL reverse metadata as a tree under a tree directory distributed for the PL reverse metadata; wherein C, D, E and F are all preset positive integers, and D is more than or equal to C and E is more than or equal to F.

In this embodiment, considering that the allocation of the fruit trees is the same as the allocation of the tree directory, and the allocation of a plurality of continuous bits is also specified in the PBA, the trees in the tree directory can be uniformly used. In this embodiment, since D < E, i.e. the present application allocates trees according to the value formed by two bits together, the present application can use different numbers of trees in different PBA ranges.

For example, in the embodiment of fig. 2, C is 39, D is 44, E is 61, and F is 63, for a total of 9 bits, which are used to select 512 trees in each tree directory. Due to the division into two parts, instead of using all 512 trees, a different number of trees can be used in the range of different PBAs, which is advantageous for further improving the efficiency of data destaging.

For any tree, when the tree meets the set disk-flushing condition, all PL reverse metadata in the tree can be flushed from the cache to the disk.

The set brushing condition may be set according to actual needs, for example, in an embodiment of the present invention, step S102 may specifically include:

And the brushing is carried out based on the brushing threshold, so that batch brushing of PL reverse metadata can be realized, and the brushing efficiency is improved.

Further, the present application considers that, after applying the foregoing solution of the present application, PL reverse metadata is more evenly distributed to different trees under different tree directories, and if a uniform lower-brushing threshold is set, when step S102 is executed, a situation that many trees simultaneously reach the lower-brushing threshold is likely to occur, resulting in that a large number of trees are instantaneously brushed at the same time, which causes performance fluctuation of the system.

Therefore, in the implementation mode, a gradient threshold strategy is used, and the brushing thresholds of different trees are different, so that brushing is relatively balanced, a certain concurrency degree is kept, and the overall performance of the system is improved. Specifically, in this embodiment, for any tree in the same tree directory, the lower-brushing thresholds of other trees are different from the lower-brushing threshold of the tree in the tree directory.

Corresponding to the above method embodiment, the embodiment of the invention also provides a storage system of PL reverse metadata, which can be referred to in correspondence with the above.

Referring to fig. 3, a schematic structural diagram of a PL reverse metadata storage system according to the present invention is applied to an LSA-based full flash storage system, and includes:

a tree splitting module 301, configured to, when generating PL reverse metadata and placing the PL reverse metadata in a cache, allocate a corresponding tree directory and a tree under the tree directory for the PL reverse metadata according to a preset allocation rule;

a flushing module 302, configured to flush, for any tree, when the tree meets a set flushing condition, all PL reverse metadata in the tree from a buffer to a disk;

In an embodiment of the present invention, the tree splitting module 301 is specifically configured to:

when generating and placing the PL reverse metadata into the cache, allocating a corresponding tree directory and a tree under the tree directory for the PL reverse metadata based on the PBA in the PL reverse metadata, so that two PL reverse metadata with adjacent PBAs can be allocated to different tree directories.

In an embodiment of the present invention, the tree splitting module 301 includes:

a tree directory allocation unit, configured to use the value formed by bit a to bit B of the PBA in the PL reverse metadata as a tree directory allocated for the PL reverse metadata; wherein A and B are preset positive integers, and A is less than B;

and the tree allocation unit is used for allocating a tree under the corresponding tree directory for the PL reverse metadata based on the PBA in the PL reverse metadata.

In an embodiment of the present invention, the tree allocation unit is specifically configured to:

using a numerical value formed by bit C to bit D and bit E to bit F of the PBA in the PL reverse metadata as a tree under a tree directory distributed for the PL reverse metadata; wherein C, D, E and F are all preset positive integers, and D is more than or equal to C and E is more than or equal to F.

In an embodiment of the present invention, the flash module 302 is specifically configured to:

In an embodiment of the present invention, for any tree in the same tree directory, the lower-brushing thresholds of other trees are different from the lower-brushing threshold of the tree in the tree directory.

Corresponding to the above method and system embodiments, the embodiments of the present invention also provide a storage device for PL reverse metadata and a computer readable storage medium, which can be referred to in correspondence with the above. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the method of storing PL reverse metadata in any of the embodiments. A computer-readable storage medium as referred to herein may include Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

FIG. 4 is a schematic structural diagram of a storage device for PL reverse metadata, which includes:

a memory 401 for storing a computer program;

a processor 402 for executing a computer program to implement the steps of the storage method of PL inverse metadata in any of the above embodiments.

It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The principle and the implementation of the present invention are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.

12页详细技术资料下载

PL reverse metadata storage method, system, device and storage medium

相关技术

网友询问留言