Method, apparatus and computer program product for managing metadata at a control device
阅读说明:本技术 在控制设备处管理元数据的方法、设备和计算机程序产品 (Method, apparatus and computer program product for managing metadata at a control device ) 是由 韩耕 刘友生 李雄成 杨利锋 康剑斌 于 2018-07-27 设计创作,主要内容包括:本公开的实施例涉及在控制设备处管理元数据的方法、设备和计算机程序产品。该方法包括:从与用户数据相对应的高速缓冲存储器页中确定用于存储该用户数据的原始元数据的第一区域,该原始元数据包括存储系统中的被用于存储该用户数据的地址信息;响应于该用户数据被修改,确定经修改的该用户数据的更新元数据,以更新该第一区域中的该原始元数据;以及将该更新元数据复制到该控制设备与另一控制设备共享的高速存储器中。本公开的实施例能够降低高速存储器的使用频度,从而延长高速存储器的使用寿命并降低成本。(Embodiments of the present disclosure relate to methods, devices and computer program products for managing metadata at a control device. The method comprises the following steps: determining a first region for storing original metadata of user data from a cache page corresponding to the user data, the original metadata including address information in a storage system used to store the user data; in response to the user data being modified, determining updated metadata for the modified user data to update the original metadata in the first region; and copying the update metadata to a high-speed memory shared by the control device and another control device. The embodiment of the disclosure can reduce the use frequency of the high-speed memory, thereby prolonging the service life of the high-speed memory and reducing the cost.)
1. A method for managing metadata at a control device, comprising:
determining a first region for storing original metadata of user data from a cache page corresponding to the user data, the original metadata including address information in a storage system used to store the user data;
in response to the user data being modified, determining updated metadata for the modified user data to update the original metadata in the first region; and
the update metadata is copied to a high-speed memory shared by the control device and another control device.
2. The method of claim 1, further comprising:
merging a second region of the cache page into a third region in response to the metadata stored on the first region being updated; and
copying the metadata stored in the third region into the shared high-speed memory.
3. The method of claim 2, wherein merging the first region and the second region into a third region comprises:
determining a first starting location and a first length of the first region in the cache page;
determining a second starting position and a second length of the second region in the cache page;
determining a starting position of the third area based on the first starting position and the second starting position; and
determining a length of the third region based on the first starting position, the first length, the second starting position, and the second length.
4. The method of claim 1, wherein determining the first region comprises:
determining a first region of the cache page for storing the original metadata based on the description information of the cache page.
5. The method of claim 1, wherein determining updated metadata for the user data that is modified comprises:
determining address information that the modified user data is stored in the storage system based on address mapping management information; and
generating the update metadata based on the determined address information.
6. The method of claim 1, further comprising:
sending an update indication message to the other control device to indicate the other control device to use the update metadata that has been stored in the shared high-speed memory.
7. The method of claim 1, further comprising:
in response to receiving an access request for the user data, redirecting the access request to one of the control device and the other control device based on address information of the user data to be accessed.
8. An apparatus for managing metadata at a control device, comprising:
at least one processing unit;
at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which when executed by the at least one processing unit, cause the apparatus to perform acts comprising:
determining a first region for storing original metadata of user data from a cache page corresponding to the user data, the original metadata including address information in a storage system used to store the user data;
in response to the user data being modified, determining updated metadata for the modified user data to update the original metadata in the first region; and
the update metadata is copied to a high-speed memory shared by the control device and another control device.
9. The apparatus of claim 8, the acts further comprising:
merging a second region of the cache page into a third region in response to the metadata stored on the first region being updated; and
copying the metadata stored in the third region into the shared high-speed memory.
10. The apparatus of claim 9, wherein merging the first region and the second region into a third region comprises:
determining a first starting location and a first length of the first region in the cache page;
determining a second starting position and a second length of the second region in the cache page;
determining a starting position of the third area based on the first starting position and the second starting position; and
determining a length of the third region based on the first starting position, the first length, the second starting position, and the second length.
11. The apparatus of claim 8, wherein determining the first region comprises:
determining a first region of the cache page for storing the original metadata based on the description information of the cache page.
12. The apparatus of claim 8, wherein determining updated metadata for the user data that is modified comprises:
determining address information that the modified user data is stored in the storage system based on address mapping management information; and
generating the update metadata based on the determined address information.
13. The apparatus of claim 8, the acts further comprising:
sending an update indication message to the other control device to indicate the other control device to use the update metadata that has been stored in the shared high-speed memory.
14. The apparatus of claim 8, the acts further comprising:
in response to receiving an access request for the user data, redirecting the access request to one of the control device and the other control device based on address information of the user data to be accessed.
15. A computer program product tangibly stored in a non-transitory computer storage medium and comprising machine executable instructions that, when executed by a device, cause the device to perform the method of any of claims 1 to 7.
Technical Field
Embodiments of the present disclosure relate generally to metadata management and, more particularly, relate to a method, apparatus and computer program product for managing metadata at a control device.
Background
With the development of data storage technology, storage servers are often required to manage storage systems containing a large number of disks. In order to provide frequent access to the storage system by users, in storage systems comprising disk arrays, caching is often employed to increase the speed of accessing data in the storage system. The first tier Cache is typically a Cache such as Cache, and the second tier Cache is typically a Cache such as NVMe.
Along with the access request of the user to the data, the metadata corresponding to the user data can be modified while the user data is modified. These modified metadata are typically first copied from the first tier Cache to the second tier Cache, e.g., from Cache pages to NVMe, before being recorded to disk. This can cause the second tier cache to age rapidly. Therefore, how to manage metadata more reasonably is a topic worthy of study.
Disclosure of Invention
Embodiments of the present disclosure provide methods, devices and computer program products for managing metadata at a control device.
In a first aspect of the disclosure, a method for managing metadata at a control device is provided. The method comprises the following steps: determining a first region for storing original metadata of user data from a cache page corresponding to the user data, the original metadata including address information in a storage system used to store the user data; in response to the user data being modified, determining updated metadata for the modified user data to update the original metadata in the first region; and copying the update metadata to a high-speed memory shared by the control device and another control device.
In a second aspect of the present disclosure, a device for managing metadata at a control device is provided. The apparatus comprises at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the apparatus to perform acts comprising: determining a first region for storing original metadata of user data from a cache page corresponding to the user data, the original metadata including address information in a storage system used to store the user data; in response to the user data being modified, determining updated metadata for the modified user data to update the original metadata in the first region; and copying the update metadata to a high-speed memory shared by the control device and another control device.
In a third aspect of the disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine executable instructions. The machine executable instructions, when executed by an apparatus, cause the apparatus to perform any of the steps of the method described according to the first aspect of the disclosure.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the disclosure, nor is it intended to be used to limit the scope of the disclosure.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be apparent from the following more particular descriptions of exemplary embodiments of the disclosure as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the disclosure.
FIG. 1 shows a schematic diagram of an architecture of a storage system according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a schematic diagram of a data access process in a storage system according to an embodiment of the disclosure;
fig. 3 shows a schematic diagram of the structure of address information according to an embodiment of the present disclosure;
FIG. 4 shows a flow diagram of a method for managing metadata at a control device in accordance with an embodiment of the present disclosure;
FIGS. 5A and 5B illustrate schematic diagrams of metadata changes according to embodiments of the present disclosure;
FIG. 6 illustrates a flow diagram of merging updated metadata regions in accordance with an embodiment of the present disclosure;
FIG. 7 shows a schematic diagram of an example of metadata region merging, according to an embodiment of the present disclosure;
FIG. 8 shows a schematic diagram of redirecting an access request to a particular control device, according to an embodiment of the present disclosure; and
FIG. 9 shows a schematic block diagram of an example device that may be used to implement embodiments of the present disclosure.
Like or corresponding reference characters designate like or corresponding parts throughout the several views.
Detailed Description
Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The term "include" and variations thereof as used herein is meant to be inclusive in an open-ended manner, i.e., "including but not limited to". Unless specifically stated otherwise, the term "or" means "and/or". The term "based on" means "based at least in part on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first," "second," and the like may refer to different or the same object. Other explicit and implicit definitions are also possible below.
Fig. 1 shows a schematic diagram of an architecture of a
As shown in fig. 1, a
In fig. 1, the
The
Fig. 2 schematically shows a schematic diagram of a data access process in the
Fig. 3 shows a schematic diagram of a structure of address information according to an embodiment of the present disclosure. As shown in FIG. 3, the addresses of data in the
Each target user data has an address ID comprising a set of addresses, each address in the set of addresses describing a corresponding entry in each level in the tree structure, thereby enabling mapping from logical addresses to virtual addresses.
In the example of fig. 3, the entries of the
When a user performs an operation such as modification on target user data in the user data cache, metadata such as address information corresponding to the target user data is often caused to change. The changed metadata is copied to the high-
Conventionally, when metadata of target user data, particularly address information, changes, the entire contents of a cache page in which the changed metadata is located are copied into the high-
To address, at least in part, the above problems and potentially other problems, embodiments of the present disclosure provide a scheme for managing metadata at a control device.
Fig. 4 shows a flow diagram of a
At 410, a first region for storing original metadata of user data is determined from a cache page corresponding to the user data. The original metadata includes address information in the storage system that is used to store the user data. In some embodiments, the original metadata may be metadata in a cache page used to store user data that has not been modified. The address information may be, for example, a start address, an end address, an address length, a combination thereof, and the like.
In some embodiments, the first region of the cache page for storing the original metadata may be determined based on the description information of the cache page. This will be described in detail below in connection with the specific embodiment of fig. 5A and 5B.
At 420, it is determined whether the user data is modified. If so, at 430, update metadata for the modified user data is determined to update the original metadata in the first region. Here, the update metadata refers to metadata used to store changed user data in a cache page due to a change of the user data.
In some embodiments, address information at which the modified user data is stored in the storage system may be determined based on the address mapping management information. Then, update metadata may be generated based on the determined address information. This will be described in detail below in connection with the specific embodiment of fig. 5A and 5B.
With continued reference to FIG. 1, at 440, the updated metadata is copied to a high speed memory that the control device shares with another control device so that multiple control devices can share changes in the metadata in the high speed memory.
With the scheme in
Additionally, in some embodiments, when metadata on a second region of the cache page is updated, the first region and the second region are merged into a third region, and the metadata stored in the third region is copied into the shared high speed memory. This will be described in further detail below in conjunction with fig. 6 and 7.
Additionally, in some embodiments, upon receiving an access request for user data, the access request is redirected to one of the control device and the other control device based on address information of the user data to be accessed. This will be described in further detail below in conjunction with fig. 8.
Fig. 5A and 5B illustrate schematic diagrams of metadata changes according to embodiments of the present disclosure. The process by which metadata is modified with target user data is described below in conjunction with fig. 5A and 5B.
In fig. 5A, A, B, C, D four pieces of data are schematically stored in the first physical logical block PLB 1. Each piece of data corresponds to an entry in the first virtual logical block VLB 1. In the example of fig. 5A, assume that data a is the target user data to be modified. Wherein the first entry (shaded in the figure) in the first virtual logical block VLB1 points to this data a, and the first entry (shaded in the figure) in
In FIG. 5B, the modified target user data A' would be stored in the second physical logical block PLB2, which now corresponds to the first entry of the second virtual logical block VLB 2. Therefore, the metadata in the first entry in
In an embodiment of the present invention, to modify metadata in a cache page, the following four application program interfaces may be used to perform the corresponding basic operations:
MMDC _ SET (element, val): assigning val to the element;
MMDC _ ADD (element, val): adding a value val to the element;
MMDC _ MEM _ COPY (dst _ ptr, src _ ptr, len): copying data of length len from the src region to the dst region in the cache page;
MMDC _ MEM _ SET (ptr, val, len): val is assigned to the region ptr of length len in the cache page.
While the above four basic operations are performed on the metadata in the cache page to update the metadata, for example, the modified start position and length in the metadata may be recorded to determine the modified area in the metadata.
In some embodiments, in performing the above four basic operations, the description information for the cache page is modified, and the area of the cache page for storing metadata for the modified user data is determined based on the description information.
Wherein the modification of the description information of the cache page can be implemented in various ways. In some embodiments, code may be employed to make the above modifications:
it should be understood that the above code merely illustrates one exemplary implementation. It will be appreciated by those skilled in the art that modifications to the description information of a cache page may also be implemented using other forms of software, hardware, or a combination of software and hardware.
Fig. 6 illustrates a flowchart of merging updated metadata regions according to an embodiment of the present disclosure. When metadata in a cache page is changed at the same time as a plurality of target user data are modified, in order to copy the changed portion of the metadata into the high-speed memory at once, the following operations are performed according to an embodiment of the present invention.
At 610, a determination is made as to whether there are multiple modified regions of data stored on the cache page. This can also be determined, for example, by means of the above-mentioned description information of the cache pages. At 620, the modified first region and the second region of the cache page are merged into an overall third region. The merging process is described in more detail below in conjunction with fig. 7.
Fig. 7 illustrates a schematic diagram of an example of metadata region merging according to an embodiment of the present disclosure. As shown in FIG. 7, there are two regions in the cache page 710 that are modified, namely, a first region 720 and a second region 730. It should be understood that the number of modified regions is merely exemplary, and that a greater number of modified regions is also included within the scope of the present disclosure.
In one embodiment of the present invention, merging the first region 720 and the second region 730 into the third region 740 comprises: determining a first starting position and a first length of the first region 720 in the cache page; determining a second starting position and a second length of the second region 730 in the cache page; determining a starting position of the third region 740 based on the first starting position and the second starting position; and determining a length of the third region 740 based on the first start position, the first length, the second start position, and the second length.
In some embodiments, the extent of the third region may also be determined by means of the start and end positions, or the end position and length, etc. of the first and second regions.
With continued reference to FIG. 6, at 630, the metadata stored in the consolidated third region is copied to high speed memory.
In certain embodiments, the
In some embodiments, the
To avoid the above-described process from occurring frequently, in some embodiments according to the present disclosure, the access request is redirected to one of the
Fig. 8 shows a schematic diagram of redirecting an access request to a specific control device according to an embodiment of the present disclosure. As shown in fig. 8, the
In this way, the frequency of use of the update metadata of the modified user data stored in the shared high-speed memory by the
Fig. 9 illustrates a schematic block diagram of an
Various components in
Various processes and processes described above, such as
The present disclosure may be methods, apparatus, systems, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for carrying out various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
- 上一篇:一种医用注射器针头装配设备
- 下一篇:一种热数据智能识别的UBIFS数据闪存系统