阅读说明：本技术 数据存储系统 (Data storage system ) 是由 张敏 王树 刘宇洋 于 2020-12-28 设计创作，主要内容包括：本申请数据存储系统包括N个处理核模块,M个第一缓存模块、以及第二缓存模块,N个处理核模块集成在FPGA上,并且在前1～N-1号处理核模块中加载有实时系统,将接收到的用户数据高速缓存至多个第一缓存模块中,存储的数据通过处理核模块传递最终归集到第二缓存模块,在N号处理核模块中加载有非实时系统,可以响应对时间明感性不强的用户应用程序操作指令。整个数据存储系统,区分实时性数据和非实时性数据,针对实时性数据采取集成于FPGA多级处理核模块来实现数据高速缓存,针对非实时性数据采取加载非实时系统的N号处理核模块来响应处理,能够有效避免数据存储过程中出现阻塞,可以实现数据的高速缓存。(The data storage system comprises N processing core modules, M first cache modules and a second cache module, wherein the N processing core modules are integrated on an FPGA (field programmable gate array), a real-time system is loaded in the first processing core modules from 1 to N-1, received user data are cached in the first cache modules, the stored data are transmitted through the processing core modules and finally collected to the second cache module, and a non-real-time system is loaded in the N processing core modules, so that the data storage system can respond to user application program operation instructions with weak time sensitivity. The whole data storage system distinguishes real-time data and non-real-time data, realizes data cache by adopting the multi-level processing core module integrated in the FPGA aiming at the real-time data, responds and processes by adopting the N number processing core module loaded on the non-real-time system aiming at the non-real-time data, can effectively avoid the blockage in the data storage process, and can realize the data cache.)

1. The data storage system is characterized by comprising N processing core modules, M first cache modules and a second cache module, wherein N is M +2, N is a positive integer, and M is a positive integer not less than 1;

the N processing core modules are sequentially connected in series and integrated on the same FPGA to form a processing core chain comprising the No. 1-N processing core modules, a real-time system is loaded on the No. 1-N-1 processing core module, a non-real-time system is loaded on the No. N processing core module, two adjacent processing core modules in the No. 1-N-1 processing core modules are connected with the same first cache module, the No. 1 processing core module receives user real-time data, the No. N-1 processing core module and the No. N processing core module are connected with the second cache module, the No. N processing core module receives an operation instruction of a user application program, and data stored in the second cache module are processed corresponding to the operation instruction.

2. The system of claim 1, wherein m memory spaces are partitioned within the first cache module.

3. The system of claim 1, wherein m storage spaces are logically equally sized within the first cache module based on a cache medium.

4. The system according to claim 2 or 3, wherein a completion queue and a free queue are maintained in the first buffer module, wherein the storage spaces filled with data in the m storage spaces constitute the completion queue, and the storage spaces not storing data in the m storage spaces constitute the free queue.

5. The system according to claim 1, further comprising a user data reception control module, wherein the user data reception control module is connected to the number 1 processing core module.

6. The system according to claim 5, wherein the user data reception control module sends a storage space allocation request to the processing core module No. 1 when detecting real-time user data, the processing core module No. 1 allocates a free storage space in the connected first cache module and feeds back an identity corresponding to the allocated storage space to the user data reception control module, and the user data reception control module fills real-time user data in the first cache module connected to the processing core module No. 1 according to the received identity.

7. The system according to claim 6, wherein the user data reception control module is further configured to fill the first buffer module # 1 with user real-time data in a DMA manner according to the received identification.

8. The system of claim 1, wherein the N processing core modules are single cascaded through a shared memory space, and wherein the N processing core modules interact with each other in a messaging manner.

9. The system of claim 1, wherein the first cache module comprises a DDR and the second cache module comprises an SSD or a disk.

10. The system of claim 1, wherein the N processing core modules comprise a hard core supporting deployment of a real-time system and a non-real-time system, or comprise an embedded soft core supporting deployment of a real-time system and a non-real-time system.

Technical Field

The present application relates to the field of data caching technologies, and in particular, to a data storage system.

Background

The information age user data is exponentially increased, and the user puts new requirements on the storage and the export of high-speed data: user data can be stored in a storage medium at high speed without blocking, meanwhile, data can be conveniently and quickly exported, and the exporting of the data is required to be compatible with a general non-real-time system.

In the existing solutions, storage media are mostly directly mounted in a non-real-time system for storage control of user data, so that when the front-end data speed reaches GB level or even higher, the phenomenon of front-end data congestion will be gradually apparent, limiting the overall bandwidth of the system.

Therefore, the traditional high-speed data caching scheme is easy to block and cannot realize high-speed data storage.

Disclosure of Invention

In view of the above, it is necessary to provide a data storage system capable of realizing high-speed data storage in response to the above-described technical problems.

A data storage system comprises N processing core modules, M first cache modules and a second cache module, wherein N is M +2, N is a positive integer, and M is a positive integer not less than 1;

the N processing core modules are sequentially integrated on the same FPGA (Field Programmable Gate Array) in series to form a processing core chain comprising 1-N processing core modules, the 1-N-1 processing core modules are loaded with a real-time system, the N processing core modules are loaded with a non-real-time system, two adjacent processing core modules in the 1-N-1 processing core modules are connected with the same first cache module, the 1 processing core module receives user real-time data, the N-1 processing core module and the N processing core module are connected with the second cache module, and the N processing core module receives operation instructions of user application programs and processes data stored in the second cache module corresponding to the operation instructions.

In one embodiment, m memory spaces are divided in the first cache module.

In one embodiment, the first cache module is internally divided into m storage spaces based on the equal size of cache medium logic.

In one embodiment, a completion queue and a free queue are maintained in the first buffer module, where a storage space filled with data in the m storage spaces constitutes the completion queue, and a storage space not storing data in the m storage spaces constitutes the free queue.

In one embodiment, the data storage system further includes a user data receiving control module, and the user data receiving control module is connected to the processing core module No. 1.

In one embodiment, when the user data receiving control module senses user real-time data, the user data receiving control module sends a storage space allocation request to the processing core module No. 1, the processing core module No. 1 allocates a free storage space in the connected first cache module and feeds back an identity corresponding to the allocated storage space to the user data receiving control module, and the user data receiving control module fills the user real-time data in the first cache module connected with the processing core module No. 1 according to the received identity.

In one embodiment, the user data receiving control module is further configured to fill the user real-time data into the number 1 first cache module in a Direct Memory Access (DMA) manner according to the received identity.

In one embodiment, the N processing core modules are singly cascaded through a shared storage space, and the N processing core modules interact with each other in a message manner.

In one embodiment, the first cache module includes a DDR, and the second cache module includes a Solid State Drive (SSD) or a disk.

In one embodiment, the N processing core modules include a hard core supporting deployment of a real-time system and a non-real-time system, or include an embedded soft core supporting deployment of a real-time system and a non-real-time system.

The data storage system comprises N processing core modules, M first cache modules and a second cache module, wherein the N processing core modules are integrated on an FPGA, a real-time system is loaded in the first 1-N-1 processing core modules, received user data are cached in the first cache modules, the stored data are transmitted through the processing core modules and finally collected to the second cache module, and a non-real-time system is loaded in the N processing core modules, so that the system can respond to user application program operation instructions with weak time sensitivity. The whole data storage system distinguishes real-time data and non-real-time data, realizes data cache by adopting the multi-level processing core module integrated in the FPGA aiming at the real-time data, responds and processes by adopting the N number processing core module loaded on the non-real-time system aiming at the non-real-time data, can effectively avoid the blockage in the data storage process, and can realize the data cache.

Drawings

FIG. 1 is a schematic diagram of a data storage system according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of the data storage system of the present application when N is 3;

fig. 3 is a schematic structural diagram of the data storage system of the present application when N is 4;

FIG. 4 is a schematic diagram of dividing m storage spaces in a first cache module;

FIG. 5 is a schematic diagram of a data storage system according to another embodiment of the present application;

FIG. 6 is an interaction diagram of collecting and storing real-time user data in an application example;

FIG. 7 is a schematic diagram of a shared memory space between processing core modules;

FIG. 8 is a schematic structural diagram of a data storage system according to the present application in an application example.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

As shown in fig. 1, the present application provides a data storage system, which includes N processing core modules 100, M first cache modules 200, and a second cache module 300, where N is M +2, N is a positive integer, and M is a positive integer not less than 1;

the N processing core modules 100 are sequentially connected in series and integrated on the same FPGA to form a processing core chain comprising No. 1-N processing core modules, a real-time system is loaded on the No. 1-N-1 processing core module, a non-real-time system is loaded on the No. N processing core module, two adjacent processing core modules in the No. 1-N-1 processing core modules are connected with the same first cache module 200, the No. 1 processing core module receives user real-time data, the No. N-1 processing core module and the No. N processing core module are connected with a second cache module 300, the No. N processing core module receives an operation instruction of a user application program, and data stored in the second cache module 300 is processed corresponding to the operation instruction.

The processing core module 100 is used for processing the received user real-time data and storing the processed user real-time data to the connected first cache module 200 or the second cache module 300. The first cache module 200 and the second cache module 300 are used for caching data, the processing core module 100 is used for controlling the data to be stored to the corresponding connected first cache modules 200 step by step, and when the processing core module is connected with 2 first cache modules 200, the processing core module 100 is used for reading the data stored in the first cache module 200 of the previous stage and placing the data into the first cache module 200 of the next stage until all the data are finally transferred and collected to the second cache module 300. Optionally, the storage capacity of the first cache module 200 is smaller than the storage capacity of the second cache module 300.

N processing core modules are sequentially connected in series and integrated on the same FPGA to form a processing core chain, the processing core chain is composed of No. 1 to N processing core modules, the processing core modules are sequentially connected in series, wherein the No. 1 processing core module receives real-time data of a user, the No. N processing core module and an operation instruction of a user application program are received, two adjacent processing core modules in the No. 1 to N-1 processing core modules are connected with the same first cache module, the No. N processing core module is connected with a second cache module, the real-time data of the user received by the No. 1 processing core module is stored in the No. 1 first cache module connected with the No. 1 processing core module under the control of the processing core modules, and the No. 2 processing core module is also connected with the No. 1 first cache module, so the No. 2 processing core module can read the data from the processing core modules and then store the data in the No. 2 second cache module connected with the No. 2 processing core module, and the data are collected and stored to the second cache module by analogy, the N processing core module is connected with the second cache module, the N processing core module receives an operation instruction of the user application program, and the operation instruction correspondingly processes the data stored in the second cache module. In addition, the No. 1-N-1 processing core module in the No. 1-N processing core modules is loaded with a real-time system, the No. N processing core module is loaded with a non-real-time system, and the processing core modules are integrated on the same FPGA, the whole data storage system is cached in a second cache module in a cascading way in real time based on a user with high time sensitivity of a real-time system, the data stored in the second cache module is processed based on the user application operation instruction with low time sensitivity of the non-real-time system, the storage of the user data is controlled by the micro real-time system in the FPGA, so that the high-speed storage of the non-inductive control data of a non-real-time system used by a back-end user is ensured, the rate index of the front-end user data provided by the storage system is greatly improved, and the congestion degree of the high-speed storage of the user data is reduced.

In order to explain the technical scheme of the data storage system in the present application in detail, two embodiments of N-3 and N-4 will be described below.

As shown in fig. 2, N is 3, the data storage system of the present application includes a processing core module 1, a processing core module 2, a processing core module 3, a first cache module, and a second cache module, which are sequentially connected in series, where the processing core module 1 is connected to the first cache module, the processing core module 2 is connected to the first cache module and the second cache module, and the processing core module 3 is connected to the second cache module. The No. 1 processing core module acquires user real-time data and stores the acquired user real-time data to the first cache module, the No. 2 processing core module reads the user real-time data stored in the first cache module and stores the read data to the second cache module, and the No. 3 processing core module receives a control instruction of a user application program and responds to the control instruction to correspondingly process the data stored in the second cache module.

As shown in fig. 3, N is 4, the data storage system of the present application includes a processing core module 1, a processing core module 2, a processing core module 3, a processing core module 4, a first cache module 1, a first cache module 2 and a second cache module, the processing core module 1 is connected to the first cache module 1, the processing core module 2 is connected to the first cache module 1 and the first cache module 2 respectively, the processing core module 3 is connected to the first cache module 2 and the second cache module respectively, and the processing core module 4 is connected to the second cache module. The No. 1 processing core module acquires user real-time data, stores the acquired user real-time data into the No. 1 first cache module, the No. 2 processing core module reads the user real-time data stored in the No. 1 first cache module and stores the read data into the No. 2 first cache module, the No. 3 processing core module reads the user real-time data stored in the No. 2 first cache module and stores the read data into the second cache module, the No. 4 processing core module receives a control instruction of a user application program and responds to the control instruction to correspondingly process the data stored in the second cache module.

The data storage system comprises N processing core modules, M first cache modules and a second cache module, wherein the N processing core modules are integrated on an FPGA, a real-time system is loaded in the first 1-N-1 processing core modules, received user data are cached in the first cache modules, the stored data are transmitted through the processing core modules and finally collected to the second cache module, and a non-real-time system is loaded in the N processing core modules, so that the system can respond to user application program operation instructions with weak time sensitivity. The whole data storage system distinguishes real-time data and non-real-time data, realizes data cache by adopting the multi-level processing core module integrated in the FPGA aiming at the real-time data, responds and processes by adopting the N number processing core module loaded on the non-real-time system aiming at the non-real-time data, can effectively avoid the blockage in the data storage process, and can realize the data cache.

As shown in fig. 4, in one embodiment, m memory spaces are divided in the first cache module. In this embodiment, m storage spaces are divided in the first cache module, and these storage spaces can be used one by one, when data needs to be stored each time, a certain amount of storage space is occupied for data filling in each storage, and when data is read by the next-level cache module (including the next-level first cache module or the second cache module), the storage space corresponding to the read data is released. It is to be understood that m storage spaces in the first buffer module may form a queue, wherein a storage space filled with data forms a completion queue, and a storage space not currently storing data forms a free queue, and refer to fig. 4 in detail.

Furthermore, the m storage spaces may specifically be equal-sized storage spaces, and the m storage spaces may be divided based on the logical equal-sized cache medium. In practical application, the first buffer module only needs to maintain the internal completion queue and the idle queue, and the first buffer module can be maintained through the buffer queue control unit, and the buffer queue control unit maintains the internal completion queue and the idle queue under the control of the processing core module, so that data storage and reading are realized, and finally, data transmission in the M first buffer modules and the M second buffer modules is realized.

As shown in fig. 5, in one embodiment, the data storage system further includes a user data receiving control module 400, and the user data receiving control module 400 is connected to the processing core module No. 1.

The user data reception control module 400 is used to listen to the user real-time data. Specifically, as shown in fig. 6, when the user data receiving control module senses the real-time user data, a storage space allocation request is sent to the processing core module No. 1, the processing core module No. 1 allocates a free storage space, for example, m1, in the connected first cache module (first cache module No. 1), an identifier "m 1" corresponding to m1 is fed back to the user data receiving control module, and the user data receiving control module fills the real-time user data into the storage space of m1 in the first cache module No. 1 online with the processing core module No. 1 according to the received identifier "m 1". Furthermore, when data is filled, the data can be filled in a DMA mode or other modes, so that high-speed and accurate filling of the data is realized. DMA transfers, which copy data from one address space to another, are important to high performance embedded system algorithms and networks, when the CPU initiates the transfer, the transfer itself is performed and completed by the DMA controller, typically by moving a block of external memory to a faster memory area within the chip, such as where the processor is not stalled, but rather can be rescheduled to handle other tasks.

In the above 2 embodiments, the data processing procedures and functions of the first cache module and the user data receiving control module in specific applications will be further described below with reference to fig. 6.

In order to reduce the processing logic of the processing core, in the data storage system of the present application, in order to transfer the data transmission function of the cache queue to the FPGA side for implementation, that is, to transfer the software function to the hardware front end for implementation (hardware acceleration), as shown in fig. 6, the core0 first allocates the cache medium into m storage spaces with equal size in logic, and allocates independent numbers for distinguishing which storage space is. Then the core internally maintains two queues, a time complete queue and a free queue, and the m memory spaces are initially powered on and in an unused state (unfilled data), are allocated to the free queue by the core, and are moved into the complete queue once a certain memory space is used (filled with data). Subsequent core1 may then fetch data from the completion queue and once the amount of storage space in the completion queue is fetched, reallocate the storage space to the free queue, thus creating a circular use of the storage space.

As shown in fig. 6, when the user data receiving control module detects front-end user data, it first sends a request to core0 to apply for a storage space, and core0 returns a response according to the current usage of the idle queue, and responds whether the request is successfully carried in the message, and the number of the allocated storage space. After receiving the response, the user data receiving control unit may actively fill the corresponding data into the corresponding storage unit through DMA or other methods, and feed back the completion to the core0 through a message. Therefore, the core0 does not need to control the data transmission process, and only needs to be responsible for maintaining two queues, thereby reducing the processing pressure of the core0 and reducing the risk of front-end data congestion. Meanwhile, only one storage space is requested at a time in the schematic diagram, and a plurality of storage spaces can be requested at a time for the architecture, so that the interaction between the user receiving control unit and the core0 is further reduced.

As shown in fig. 7, in one embodiment, the N processing core modules are singly cascaded through a shared storage space, and the N processing core modules interact with each other in a message manner.

In the embodiment, an architecture of a cascaded core is provided, and a multi-core heterogeneous platform is common nowadays, but mutual communication between cores becomes increasingly complex with the increase of processing core modules (cores). To ensure consistency between the cores, a number of additional operations are required. According to the scheme, the system functions are divided independently according to the flow direction of user data, the control of the functions is realized through the independent hard core or soft core, and due to partial independence among the functions, the system carried on the core can be configured independently according to actual requirements, so that the integration of multiple heterogeneous systems can be realized. Meanwhile, because the direction of the data flow is basically fixed, the cores with corresponding functions can be connected in a single cascade mode of sharing a storage space, and the communication complexity between the cores is simplified.

In one embodiment, the first cache module comprises a DDR and the second cache module comprises an SSD or a disk.

The first cache module and the second cache module may adopt the same storage medium, and the storage capacities of the two storage media may differ, specifically, since the acquired real-time user data is finally collected in the second cache module, the storage capacity of the second cache module may be larger than that of the first cache module. In this embodiment, the first cache module uses DDR to implement data caching, and the second cache module uses SSD or magnetic disk to implement storage of capacity data. Optionally, the second cache module may also employ DDR, which may employ larger capacity DDR. The SSD is called a fixed disk for short, and the hard disk made of a solid electronic storage chip array consists of a control unit and a storage unit. The specification, definition, function and use method of the interface of the solid state disk are completely the same as those of the traditional hard disk, the appearance and size of the product are completely the same as those of the traditional hard disk, and the I/O performance is greatly improved compared with that of the traditional hard disk. The method is widely applied to the fields of military affairs, vehicle-mounted, industrial control, video monitoring, network terminals, electric power, medical treatment, aviation, navigation equipment and the like. The working temperature range of the chip is very wide, and the commercial product (0-70 ℃) is industrial standard product (-40-85 ℃). With the rapid development of the internet, the storage requirements of people on data information are continuously improved, and at present, a plurality of storage manufacturers provide their portable solid state drives, and more particularly, a mobile solid state drive supporting a Type-C interface and a solid state drive supporting fingerprint identification.

In one embodiment, the N processing core modules include a hard core supporting deployment of a real-time system and a non-real-time system, or include an embedded soft core supporting deployment of a real-time system and a non-real-time system.

The processing cores included in the processing core module are not particularly referred to as arm core hard cores or microblaze soft cores, and may be other units or modules capable of carrying simple or complex systems such as real-time systems or non-real-time systems.

To explain the technical solution of the data storage system and its effect in more detail, a specific application example will be used in the following, and will be described in detail with reference to fig. 8.

As shown in fig. 8, in this application example, N is 3, the processing core specifically includes core0, core1, and core2, the cache medium includes a small-capacity storage medium (a first cache module) and a large-capacity storage medium (a second cache module), where core0 hosts a bare engine system (a real-time system), core1 hosts a freertOS (a real-time system), core2 hosts a non-real-time system such as Linux, and the cores use a shared storage space to perform message interaction, and the command format of the message may use a common message format or a custom message format, as shown by a blue double-headed arrow in the figure. In the scheme, the core0 is responsible for controlling the reception of user data and controlling the data to be buffered in a buffer queue (the buffer queue is implemented on a small-capacity storage medium and is described later), and simultaneously, processing structures corresponding to the transactions are interacted with each other through a message and the core1, when the core1 receives the transaction message of the data stored in the buffer, the data is stored in the large-capacity storage medium through a control unit of the large-capacity storage medium (the control unit is arranged inside the large-capacity storage medium), and the recording function of the user data is completed. The corresponding transaction results are then interacted with through message and core 2. The core2 realizes a general non-real-time system, can be conveniently connected with the user application program, and realizes corresponding operation through the core corresponding to the message level one-level notification according to the transaction corresponding to the user application program. Such as the last user data offloaded to a data center in the above figure, etc.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.