阅读说明：本技术 一种解决存储磁盘阵列Write Hole的处理方法 (Processing method for solving Write Hole of storage disk array ) 是由 史建才 于 2021-08-26 设计创作，主要内容包括：本发明涉及一种解决存储磁盘阵列Write Hole的处理方法,于设计RAID时,采用文件系统的Journal日志设计方式进行写请求的原子处理；采用非易失性的快速设备做为写缓存,达到原子写操作的目的。本发明在实现写请求的原子处理过程中,journal盘由SSD固态硬盘与HDD机械硬盘组合而成,并且先通过对io块的大小进行解析,再根据io块的大小不同而分别写入不同journal盘,若io块为小块,则将小块io并写入journal-SSD；若io块为大块,则将大块io并写入journal-HDD；若io块为小块且为连续状态,则写入journal-HDD,从而避免Write Hole问题的出现。(The invention relates to a processing method for solving the problem of Write Hole of a storage disk array, which adopts a Journal log design mode of a file system to perform atomic processing of a Write request when designing RAID; and nonvolatile fast equipment is used as a write cache to achieve the purpose of atomic write operation. In the process of realizing the atomic processing of the write request, the journal disk is formed by combining an SSD solid state disk and an HDD mechanical hard disk, the size of an io block is analyzed, then different journal disks are respectively written into according to the different sizes of the io block, and if the io block is a small block, the small block io is written into a journal-SSD; if the io block is a large block, writing the large block io into a journal-HDD; if the io block is a small block and is in a continuous state, the journal-HDD is written, thereby avoiding the Write Hole problem.)

1. A processing method for solving the problem of the Write Hole of the storage disk array is characterized by comprising the following specific processing steps:

when designing RAID, adopting a Journal log design mode of a file system to perform atomic processing of a write request;

adopting nonvolatile fast equipment as a write cache so as to achieve the purpose of atomic write operation;

further, the non-volatile fast device is formed by combining an SSD solid state disk and an HDD mechanical hard disk, and the size of the io block is analyzed, and then the io block is written into different journal disks according to the size of the io block, including:

if the io block is a small block, writing the small block io into a journal-SSD;

if the io block is a large block, writing the large block io into a journal-HDD;

if the io block is a small block and is in a continuous state, the journal-HDD is written.

2. The processing method for solving the problem of Write Hole of the storage disk array of claim 1, wherein: the journal disk is a combination of at least two hard disks so as to simultaneously ensure the service life of the hard disks and improve the performance of the hard disks.

3. The processing method for solving the problem of Write Hole of the storage disk array of claim 1, wherein: the adopted mode of the file system Journal log comprises that a data updating operation is regarded as a transaction, namely an atomic operation.

4. The processing method for solving the problem of Write Hole of the storage disk array of claim 3, wherein: the one atomic operation forms an atomic write operation.

5. The processing method for solving the problem of Write Hole of the storage disk array of claim 4, wherein: for atomic write operations, when a fault occurs in the atomic write intermediate process, the system can continue to operate after restarting, and it is ensured that data is either completely updated or completely written without an intermediate state.

6. The process of resolving a storage disk array Write Hole of any of claims 1 to 5, wherein: when the RAID carries out stripe data updating, firstly, a log is recorded, the event type and relevant data are stored, and then the data are submitted into the stripe.

7. The processing method for solving the problem of Write Hole of the storage disk array of claim 6, wherein: and when the strip data is completely updated, clearing the log data.

8. The data writing method according to claim 1, wherein: the atomic processing operation of the write request is introduced into the nonvolatile memory, when the RAID data is updated, the data is firstly written into the nonvolatile memory, and when the data in the memory is aggregated, the write stripe is updated.

9. The process of claim 8 for resolving a Write Hole of a storage disk array, comprising: in the process of updating the stripe data, if the system is abnormally restarted, the data in the stripe will be updated again because the data which is not updated completely still exists in the memory, and no intermediate data state exists.

10. A data reading method according to claim 1, wherein: the RAID is matched with a file system and an upper application system.

Technical Field

The invention relates to the technical field of computer storage, in particular to a processing method for solving the problem of Write Hole of a storage disk array.

Background

Those skilled in the art know that a problem called Write Hole exists in a RAID system, where Write Hole is a very visual description, and when an abnormality (such as power failure, reset, etc.) occurs in a system, a Write Hole phenomenon easily occurs in a disk array. Specifically, for a system which adopts the stripe interior for data protection through check data, when an application program is updating the stripe interior data, if the system is suddenly powered off, the stripe data can not be normally updated, and then the data in the stripe is partially updated, in this case, after the system is restarted, the data in the stripe is incomplete, the check data and the valid data in the stripe can not be matched, and the stripe is in a condition of data inconsistency.

To further analyze the Write Hole problem, the designer of the solution of the present application performs a detailed analysis by taking the listed fig. 1 as an example:

since RAID stripes the disks, that is, each disk takes continuous sectors of PAGE _ SIZE (usually 4k, 64k), and makes a stripe, where multiple stripes are combined to form a chunk, and multiple chunks are combined to form the whole array.

The array is assumed to be raid6 composed of five disks, three pieces of inventory data D1-D3, two inventory checks P and Q, and one stripe constitutes a chunk; before the system generates crash, strip data D1- > D1' are updated, however, P and Q of the strip are not updated rapidly, and at the moment, the system is restarted when the system generates crash; in this case, the data of the strip is in an inconsistent state, and after the system is restarted, the RAID cannot sense the strip with the inconsistent data.

In this case, if D2 fails again at this time, D1 '+ D3+ P + Q calculation is used to generate D2, and it is obvious that D1' is used in the calculation process, and the recovered data error is obviously no longer the original D2 data. Therefore, the strip with inconsistent data further causes the problem of data correctness. It can be seen that the Write Hole problem not only destroys the updated data, but also potentially affects the correctness of the non-updated data, and in extreme cases, the entire stripe of data, which is also a prominent manifestation of the Write Hole problem.

Although the skilled person has made a series of improvements in dealing with the Write Hole problem in the past, for example, consider that when the Write Hole problem occurs, i.e. when the stripe data is updated, the system is suddenly powered off, and several stripes with inconsistent data appear in the RAID system. After the system is restarted, for the stripes with inconsistent data, the stripes with inconsistent data can be repaired in a full system scanning manner, which is also a method commonly used by many RAID systems at present, and by this means, the problem of correctness of the data that is not updated due to the stripes with inconsistent data can be prevented.

Even if the SYNC function is usually present in the RAID system, after the system is illegally restarted, all stripes in the RAID can be checked through the SYNC function, and the data inconsistent stripes are repaired, however, it can be obtained from actual verification that this way can prevent the spread of the Write Hole problem, but can still cause complete loss of part of data in the RAID system.

The technical scheme of the invention is to develop a corresponding solution for the above problems, and to thoroughly solve the problem of Write Hole, the invention provides a processing method for solving the Write Hole of a storage disk array, which has the main design concept that when RAID4, 5 and 6 are designed, the atomic processing of Write requests is realized by adopting a design mode of a file system journal, and the purpose of atomic processing is usually achieved by writing and caching by means of a nonvolatile storage medium capable of rapidly processing io, so that the problem of Write Hole is avoided. Of course, practical applications show that the technical scheme provided by the invention can relieve, partially solve or completely solve the problems in the prior art.

Disclosure of Invention

To overcome the above problems or at least partially solve the above problems, the present invention provides a processing method for Write Hole of a storage disk array, which adopts a design manner of a file system journal to implement atomic processing of Write requests, and usually performs Write caching by means of a nonvolatile storage medium capable of rapidly processing io, so as to achieve the purpose of atomic processing, thereby avoiding the occurrence of the Write Hole problem.

In order to achieve the purpose, the invention adopts the following technical scheme:

a processing method for solving the problem of Write Hole of a storage disk array comprises the following specific processing steps:

when designing RAID, adopting a Journal log design mode of a file system to perform atomic processing of a write request;

adopting nonvolatile fast equipment as a write cache so as to achieve the purpose of atomic write operation;

further, the non-volatile fast device is formed by combining an SSD solid state disk and an HDD mechanical hard disk, and different journal disks are respectively written according to different sizes of different io blocks by analyzing sizes of the io blocks, including:

if the io block is a small block, writing the small block io into a journal-SSD;

if the io block is a large block, writing the large block io into a journal-HDD;

if the io block is a small block and is in a continuous state, the journal-HDD is written.

By implementing the above method, the obtained technical scheme can be further applied with different technical means, including:

wherein, the journal disk is a combination of at least two hard disks so as to simultaneously ensure the service life of the hard disks and improve the performance of the hard disks;

the adopted mode of the Journal log of the file system comprises that one data updating operation is regarded as one transaction, namely one atomic operation; accordingly, one atomic operation forms a atomic write operation.

For atomic write operations, when a fault occurs in the atomic write intermediate process, the system can continue to operate after restarting, and it is ensured that data is either completely updated or completely written without an intermediate state.

Furthermore, when the stripe data is updated by the RAID, firstly, a log is recorded, the event type and the related data are stored, and then the data are submitted to the stripe; and when the strip data is completely updated, clearing the log data.

The invention also comprises other technical schemes based on the same conception of solving the processing method of the memory disk array Write Hole:

the atomic processing operation of the write request is introduced into a nonvolatile memory, when the RAID data is updated, the data is firstly written into the nonvolatile memory, and when the data in the memory is aggregated, the write-in stripe is updated; in the process of updating the stripe data, if the system is abnormally restarted, the data in the stripe will be updated again because the data which is not updated completely still exists in the memory, and no intermediate data state exists.

In addition, RAID can be matched with a file system and an upper application system

The method is based on the design idea of the file system, the atomic processing of the write request is realized, the journal disk is formed by combining an SSD solid state disk and an HDD mechanical hard disk, the size of an io block is analyzed, and then different journal disks are respectively written according to the different sizes of the io block, for example, if the io block is a small block, the small block io is written into a journal-SSD; if the io block is a large block, writing the large block io into a journal-HDD; if the io block is a small block and is in a continuous state, writing into the journal-HDD, thereby achieving the purpose of atomic processing and avoiding the occurrence of the Write Hole problem.

Drawings

The invention is explained in further detail below with reference to the drawing.

FIG. 1 is a schematic diagram illustrating the principle of the Write Hole problem of the present-day storage disk array;

FIG. 2 is a flow chart illustrating a processing method for solving the problem of Write Hole of a storage disk array according to the present invention, which employs the Journal design concept of a file system to perform atomic processing;

FIG. 3 is a schematic flow chart of the processing method for solving the problem of Write Hole of the storage disk array according to the present invention, which employs a suitable medium as a journal disk for atomic processing;

FIG. 4 is a schematic flow chart of a process for writing to a journal disk according to a processing method for solving the Write Hole of the storage disk array implemented by the present invention.

Detailed Description

The invention discloses a processing method for solving the Write Hole of a storage disk array, which is to be implemented and aims to achieve the aim of achieving atomic Write operation by adopting the design idea of Journal of a file system, selecting an effective combination mode of ssd and hdd disks and using nonvolatile fast equipment as a Write cache, thereby avoiding the occurrence of the Write Hole problem.

The processing method implemented by the designer of the technical scheme of the invention mainly adopts the design idea of the Journal log of the file system when designing the RAID and combines the effective combination mode of two disks to realize the atomic processing of the write request. However, the technical solution is not limited by the specific model of each disc or the algorithm used in the method, and the technical solution of the present invention does not need to unnecessarily limit these aspects, and if a technician needs to optimize the aspect in accordance with the actual application requirements when applying the method, the various means can be implemented by conventional technical means. Therefore, the implemented technical solution is a processing method that can be referred and implemented by a person skilled in the art by combining with a conventional technical means, and the person skilled in the art can construct a final processing method for solving the Write Hole of the storage disk array according to different application requirements and design requirements, and a series of advantages brought by the constructed processing method for solving the Write Hole of the storage disk array can be actually obtained during application, such as improving data accuracy, implementing atomic processing of Write requests, avoiding the occurrence of Write Hole problems, and the like, and the advantages will be gradually embodied in the following analysis of method steps.

As shown in fig. 1-4, in the processing method for solving the Write Hole of the storage disk array implemented by the present invention, when designing RAID, the design idea of the Journal log of the file system is used to implement atomic processing of the Write request, and a nonvolatile fast device is used as a Write cache to achieve the purpose of atomic Write operation, and before adopting corresponding method steps, the design idea and the design process of the method are further analyzed as follows:

through repeated verification of designers, if a nonvolatile fast device is used as a write cache, the steps of the atomic processing method for the write request can be optimized practically.

Firstly, when a file system updates data, in order to ensure atomicity of the data update, a log mode can be adopted, and one data update operation is regarded as one transaction, namely no matter how many times IO the data update operation has, the transaction is regarded as one atomic operation from the outside, and the transaction can be called as atomic write; when the atomic write intermediate process fails, the system can continue to operate after being restarted, and the data is ensured to be either completely updated or completely not written, so that an intermediate state cannot exist; the RAID system wants to solve the problem of Write Hole, and also needs to introduce the Atomic Write mechanism, and the most direct way is to record a log at first and store the event type and related data when the RAID performs stripe data update; then, submitting the data to the stripe; and when the strip data is completely updated, clearing the log data. The mode can ensure atomic write, even if system failure occurs in the updating process of the strip data, after the system is restarted, the writing process can be continued through the log data, and the strip data is ensured to be completely updated and to be in a state of consistent data.

Accordingly, while the approach of using logs seems to be perfect, performance is also severely impacted. In a disk storage system, log updating is time-consuming, multiple disk IO operations are required for one-time updating, and if a log storage space and a stripe storage space are physically discontinuous, a disk jitter problem is introduced, so that the performance is seriously influenced; for a flash memory storage system, after a log system is introduced, double data writing operations are introduced into all data operations, and the problem of writing amplification becomes very troublesome; therefore, in the open source RAID system, such a log operation mode is not provided, and an upper file system is generally required to solve such a Write Hole problem.

Further, on the basis of introducing the nonvolatile memory, when updating RAID data, data is first written into the nonvolatile memory, for example, a nonvolatile memory NVRAM (nonvolatile random access memory, a RAM that can retain data after power failure), and when data aggregation in the NVRAM is completed, a Write stripe is updated, and in the process of updating stripe data, if an abnormal restart occurs in the system, because the data that has not been updated still exists in the NVRAM, the data in the stripe will be updated again, there will be no intermediate data state, and there will be no Write Hole problem.

In addition, in soft RAID, PCIe NVRAM or NVDIMM may be used in a system to implement a nonvolatile data cache of RAID, and in a storage system, the soft RAID cooperates with a file system and an application system on an upper layer, so the NVRAM also shares with the file system, thereby solving the problem of Write Hole. Write Hole is a difficult problem to solve in RAID, but with the help of nonvolatile memory technology, this kind of problem is easy to avoid and solve effectively.

As shown in fig. 2-4, the processing method for solving the Write Hole of the storage disk array according to the present invention summarizes the design concept and the design process, and the steps thereof are mainly as follows:

firstly, when designing RAID, carrying out atomic processing of a write request on the basis of Journal log design of a file system;

then, by using a nonvolatile fast device as a write cache, the purpose of atomic write operation is achieved, and for the selection of the journal disk, the following analysis is performed:

the selection of the SSD solid state disk and the mechanical HDD hard disk have corresponding advantages and disadvantages, for example, the respective storage principles are different, all disks of the mechanical HDD hard disk are mounted on a rotating shaft, each disk is parallel, a magnetic head is disposed on a storage surface of each disk, all magnetic heads are connected to a magnetic head controller, the magnetic head controller is responsible for the movement of each magnetic head, and the SSD solid state disk is a hard disk made of an array of solid electronic storage chips. For another example, the hard disk drive has different shock and fall resistances, the hard disk drive is of a disk type, data are stored in a disk sector, the shock and fall resistances are poor, and the SSD solid state disk is manufactured by using flash memory particles, so that no mechanical part exists in the SSD solid state disk, normal use cannot be influenced even if the SSD solid state disk is moved at high speed and even is tilted along with overturning, and the possibility of data loss can be reduced to the minimum when collision and oscillation occur; for another example, the data storage speeds are different, and from the evaluation data, the performance of the SSD solid state disk is improved by at least two times compared with the mechanical hard disk of the HDD.

Aiming at the technical scheme implemented by the invention, if only the SSD is selected as the journal disk, the SSD has better performance, but has limited erasing times and relatively higher price, and can not recover data; if only the HDD is selected as the journal disk, the HDD has no limit on erasing times, is low in price, and has relatively low writing performance.

In view of the respective advantages and disadvantages of the SSD solid state disk and the HDD mechanical hard disk, the technical scheme of the invention adopts the mode of SSD + HDD, and eliminates the problems of the service life of the SSD magnetic disk and the low performance of the HDD by effectively combining the two disks.

For the effective combination mode of the two disks, the corresponding algorithm is designed by utilizing the conventional technical means by monitoring the size of the data io block, the io size is analyzed through the algorithm, if the io size is a small block, the small block io is written into the journal-SSD, if the io size is a large block, the large block io is written into the journal-HDD, and if the io size is continuous, the large block io is also written into the journal-HDD.

In the description herein, the appearances of the phrases "embodiment one," "this embodiment," "specific implementation," and the like in this specification are not necessarily all referring to the same embodiment or example, but rather to the same embodiment or example. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example; furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

In the description of the present specification, the terms "disposed", "having", and the like are to be understood in a broad sense, and the specific meanings of the above terms in the present invention can be understood according to specific situations.

The foregoing description of the embodiments is provided to enable any person skilled in the art to make and use the embodiments, and it is to be understood that various modifications may be readily apparent to those skilled in the art, and that the generic principles defined herein may be applied to other embodiments without the use of the inventive faculty. Therefore, the present disclosure is not limited to the above embodiments, and modifications to the following cases should be included within the scope of the present disclosure: firstly, based on the technical scheme of the invention and combined with the new technical scheme implemented by the prior common general knowledge, for example, on the basis of the Journal design, the SSD and the HDD are combined to form a disk for distinguishing the writing of different io blocks, and the expected effect of the formed technical scheme is not beyond the invention; secondly, equivalent replacement of part of characteristics of the technical scheme of the invention by adopting a known technology, such as an algorithm designed according to a conventional technical means, and the like, wherein the generated technical effect is the same as the technical effect of the invention; expanding on the basis of the technical scheme of the invention, wherein the substantial content of the expanded technical scheme does not exceed the technical scheme of the invention; and fourthly, the technical means obtained by utilizing the equivalent transformation carried out by the text record content of the invention is applied to other related technical fields.