阅读说明：本技术 一种热数据智能识别的ubifs数据闪存系统 (UBIFS data flash memory system capable of intelligently identifying thermal data ) 是由 不公告发明人 于 2018-07-27 设计创作，主要内容包括：该系统主要采用哈希函数与LRU算法对热数据进行计算、分析与识别,采用多日志技术对不同热度的日志、数据进行分开存放,并通过热数据日志延迟提交技术来减少闪存系统中的日志提交带来的元数据修改,减少了数据读写的次数,同时也减少了垃圾回收触发的频率,可以有效的提高系统数据的存储效率,减少系统资源的损耗,为软件系统提供更多的数据存储容量与更快的响应速度。(The system mainly adopts a Hash function and an LRU algorithm to calculate, analyze and identify the hot data, adopts a multi-log technology to separately store logs and data with different heat degrees, reduces metadata modification brought by log submission in a flash memory system through a hot data log delayed submission technology, reduces the times of data reading and writing, simultaneously reduces the frequency of garbage recovery triggering, can effectively improve the storage efficiency of system data, reduces the loss of system resources, and provides more data storage capacity and faster response speed for a software system.)

1. A UBIFS data flash memory system of hot data intelligent recognition which characterized in that: the system is developed aiming at the problems of flash memory data storage management, hot data identification, automatic data recovery and the like, the effective classification of stored data is realized mainly by adopting a plurality of software algorithms and related storage technologies, the unified management of the data is realized, and the main design part can be divided into two parts, namely a key technology of the system and a functional module design of the system.

2. According to claim 1, the key technology of the system can be divided into two parts, namely, an LRU algorithm and a hot and cold data identification technology based on a hash table, and an automatic identification mechanism and a data hot degree identification method for hot and cold data can be established through the two software algorithms, so that the hot and cold data can be automatically classified, a reasonable mechanism for data storage and management can be established, and optimal data processing can be realized.

3. According to claim 1, the design of the functional modules of the system can be divided into four parts, namely a cold and hot data identification module, a data hot identification module, a log optimization module and a garbage recovery module, and by developing the functional modules, the functional processes of data identification, classified storage, data analysis, data recovery and the like of the system can be realized, and the data storage process is optimized, so that intelligent data processing and storage management are realized.

Technical Field

A UBIFS data flash memory system for intelligent hot data identification adopts an LRU algorithm and a hot and cold data identification technology based on a hash table to intelligently calculate and analyze hot and cold data and data heat, realizes classified storage of data through a multi-log technology, and establishes an intelligent data storage management system through the internet and the computer and other technologies.

Background

Along with the development and progress of computer technology and internet technology, people pay more and more attention to data storage and management, a flash memory system can store data at any time when equipment is powered off and has the incomparable advantages of other data storage systems, but the types of related data of the current computer system are complex, and the throughput of system data is high, so that the requirements on system data storage and management are continuously increased, and an intelligent data processing mechanism is established through a cold and hot data identification technology, so that the processing efficiency of data storage can be effectively improved, and the utilization rate of system resources is improved.

Disclosure of Invention

The key technology of the system can be divided into two parts, namely an LRU algorithm and a hot and cold data identification technology based on a hash table;

the LRU algorithm mainly screens cold and hot data according to the access history of the data, has the core idea of time locality of a program, can judge the probability of accessing the data in the future according to the latest access record of a data block, uses a linked list to store the block number of the hot data block, can effectively identify the hot data of a system through the linked list, and establishes a data classification mechanism;

2. the technology mainly adopts a hash table to store the heat value of a data block, a system calculates a hash table item corresponding to the number of the data block through a hash function, then adds 1 to the heat value stored in the corresponding hash table item, and then can be matched with the system to identify the heat of the data, thereby realizing the classified storage of cold and hot data.

The functional module design of the system can be divided into a cold and hot data identification module, a data heat identification module, a log optimization module, a garbage recovery module and the like;

1. the cold and hot data identification module is mainly used for identifying the cold and hot states of the written data and storing the data into respective cold and hot logs respectively to realize the automatic separation of the cold and hot data;

2. the data heat identification module is mainly used for filtering data by adopting a bloom filter and counting the cold and hot degrees of the data by using a hash table of a multi-hash function so as to judge the heat value of the data by the system and help the system to establish a method for classified storage and data recovery of the data;

3. the log optimization module is mainly used for preventing the failure of the hot data by a multi-log technology and a hot data log delayed submission technology, optimizing the application and management mode of the logs and optimizing the storage efficiency of the hot and cold data;

4. the garbage recycling module is mainly used for managing the state of the logic block through the logic block attribute tree, judging and analyzing idle data, achieving the recycling function of garbage data, optimizing recycling frequency and recognition and reducing the service pressure of the system.