CN103473180A - Method and system for storing data - Google Patents

Abstract

The embodiment of the present invention discloses a data storage method and system, which acquires the water level value of RAM dirty data, detects the increase of the water level of RAM dirty data, compares the water level value of RAM dirty data with a preset threshold value, and when the water level value of RAM dirty data is less than the preset threshold value, Identify whether dirty data in RAM is hot data, and then write hot data to high-speed storage devices, which can avoid excessive data writing to high-speed storage devices, and solve the problem of hot data being blocked due to non-hot data occupying too many resources of high-speed storage devices. The problem of squeezing out high-speed storage devices; at the same time, it can also solve the problem of reducing the service life of high-speed storage devices due to excessive data writing into high-speed storage devices. Moreover, if the water level of RAM dirty data increases, and the water level value of RAM dirty data is greater than the preset threshold, directly writing RAM dirty data into high-speed storage devices can solve the problem of data writing into low-speed storage devices, which will seriously affect the data storage rate and cause storage Delay problem.

Description

Data storage method and system

technical field

The present invention relates to the technical field of data processing, and more specifically, to a data storage method and system.

Background technique

At present, the existing data storage methods are generally as follows: before storing the data, first identify the hotspot of the data, if it is identified as hotspot data, then write the hotspot data into a high-speed storage device, such as: SSD (Solid State Disk, solid state drive); If it is non-hot data, write the non-hot data to a low-speed storage device, such as: HDD (Hard Disk Drive, hard disk drive).

However, in the above data storage method, when a burst write occurs, since the burst write data is generally non-hot data, it will be written into a low-speed storage device. However, the low performance of low-speed storage devices will seriously affect the data storage rate and cause storage delay.

In order to solve the above problems, there is another data storage method in the prior art, that is, without performing hotspot identification before storing data, but directly writing data into a high-speed storage device, and then writing it into a low-speed storage device. However, this method will cause non-hot data to occupy too many resources of the high-speed storage device, causing hot data to be squeezed out of the high-speed storage device; and writing too much data to the high-speed storage device will also reduce the service life of the high-speed storage device.

Contents of the invention

In view of this, an embodiment of the present invention provides a data storage method to solve the problems in existing data storage methods that non-hot data occupy too many resources of high-speed storage devices, causing hot data to be squeezed out of high-speed storage devices and the problem of excessive Writing a lot of data to a high-speed storage device leads to a problem of reducing the service life of the high-speed storage device.

In order to achieve the above purpose, the proposed scheme is as follows:

The first aspect of the embodiments of the present invention provides a data storage method, including:

Obtain the random access memory RAM dirty data water level value;

According to the obtained RAM dirty data water level value, it is judged whether the RAM dirty data water level has increased;

If the RAM dirty data water level increases, then compare the RAM dirty data water level value with a preset threshold;

If the RAM dirty data water level value is less than the preset threshold, then identify whether the RAM dirty data is hot data;

If identified as hot data, then write the RAM dirty data into a high-speed storage device; otherwise, write it into a low-speed storage device;

If the RAM dirty data water level increases, and the RAM dirty data water level value is greater than the preset threshold, write the RAM dirty data into the high-speed storage device.

In the first implementation manner of the first aspect of the embodiments of the present invention, the data storage method further includes:

If the water level of the dirty data in the RAM has not increased, obtain the value of the storage space occupied by the dirty data in the RAM;

If the storage space occupied by the RAM dirty data is greater than a preset value, write the RAM dirty data into the low-speed storage device.

In the first embodiment of the first aspect of the embodiments of the present invention or the second embodiment of the first aspect of the embodiments of the present invention, the judging whether the RAM dirty data level has increased includes:

Judging whether the promotion rate of the RAM dirty data water level value reaches a preset rate;

Or, it is judged whether the water level value of the RAM dirty data reaches a preset water level value.

In the first embodiment of the first aspect of the embodiments of the present invention or in the second embodiment of the first aspect of the embodiments of the present invention, when the RAM dirty data water level value is less than the preset threshold and the identification The RAM dirty data is hot data, and after writing the RAM dirty data into the high-speed storage device, the method further includes: marking the RAM dirty data stored in the high-speed storage device.

In the first embodiment of the first aspect of the embodiments of the present invention or the second embodiment of the first aspect of the embodiments of the present invention, when the RAM dirty data water level value is greater than the preset threshold, After writing the RAM dirty data into the high-speed storage device, the method further includes: marking the RAM dirty data stored in the high-speed storage device.

In the fourth embodiment of the first aspect of the embodiments of the present invention or the fifth embodiment of the first aspect of the embodiments of the present invention, the data storage method further includes:

After receiving the instruction to eliminate the data stored in the high-speed storage device, query the data stored on the high-speed storage device with the mark;

Writing the data set with the flag from the high-speed storage device to the low-speed storage device.

In the sixth embodiment of the first aspect of the embodiments of the present invention, after querying the data stored on the high-speed storage device with the tag set, it further includes:

Select a preset number of storage blocks corresponding to the data set with the mark;

Merge the selected storage blocks into one storage block.

In the first embodiment of the first aspect of the embodiment of the present invention or in the second embodiment of the first aspect of the embodiment of the present invention, the preset threshold includes a preset low water level threshold and a preset high water level threshold , wherein the preset high water level threshold is greater than the preset low water level threshold.

In the eighth embodiment of the first aspect of the embodiments of the present invention, if the RAM dirty data water level value is less than the preset threshold, then identifying whether the RAM dirty data is hot data is:

If the RAM dirty data water level value is less than the preset low water level threshold, then identify whether the RAM dirty data is hot data;

Alternatively, if the RAM dirty data water level value is smaller than the preset high water level threshold, it is identified whether the RAM dirty data is hot data.

In the eighth embodiment of the first aspect of the embodiments of the present invention, if the RAM dirty data water level increases, and the RAM dirty data water level value is greater than the preset threshold, the RAM dirty data is written to Into the high-speed storage device, for:

If the RAM dirty data water level increases, and the RAM cache data water level value is greater than the preset high water level threshold, write the RAM dirty data into the high-speed storage device;

Or, if the RAM dirty data water level increases, and the RAM dirty data water level value is greater than the preset low water level threshold, write the RAM dirty data into the high-speed storage device.

A data storage system is provided in a first aspect of an embodiment of the present invention, including:

A first acquisition unit, configured to acquire the RAM dirty data water level value;

A judging unit, configured to judge whether the RAM dirty data water level has increased according to the obtained RAM dirty data water level value;

The first comparison unit is used for comparing the RAM dirty data water level value with a preset threshold value when the judging unit judges that the RAM dirty data water level increases;

A hotspot identification unit, used for the first comparison unit to judge that the RAM dirty data water level value is less than a preset threshold, and identify whether the RAM dirty data is hotspot data;

A data writing unit, used for the hotspot identification unit to identify the RAM dirty data as hotspot data, and write the RAM dirty data into a high-speed storage device; the hotspot identification unit identifies the RAM dirty data as non-hotspot data, Writing the RAM dirty data into a low-speed storage device; the first comparison unit judges that the RAM cache data water level value is greater than a preset threshold, and writes the RAM dirty data into the high-speed storage device.

In the first embodiment of the second aspect of the embodiments of the present invention, the data storage system further includes:

The second acquisition unit is used for the judging unit to judge that the water level of the RAM dirty data has not increased, and to acquire the storage space value occupied by the RAM dirty data;

The second comparison unit is used to compare the storage space value occupied by the RAM dirty data with a preset value;

The data writing unit is also used for the second comparison unit to judge that the storage space occupied by the RAM dirty data is greater than a preset value, and write the RAM dirty data into the low-speed storage device.

In the first embodiment of the second aspect of the embodiments of the present invention or in the second embodiment of the second aspect of the embodiments of the present invention, the judging unit includes:

The first judging unit is used to judge whether the raising rate of the RAM dirty data water level reaches a preset rate;

The second judging unit is configured to judge whether the dirty data water level of the RAM reaches a preset water level.

In the first embodiment of the second aspect of the embodiments of the present invention or in the second embodiment of the second aspect of the embodiments of the present invention, the data storage system further includes:

The first marking unit is configured to identify the RAM dirty data as hotspot data in the data writing unit and the hot spot identifying unit, and mark the RAM dirty data written into the high-speed storage device.

In the first embodiment of the second aspect of the embodiments of the present invention or in the second embodiment of the second aspect of the embodiments of the present invention, the data storage system further includes:

The second marking unit is configured to mark the RAM dirty data written into the high-speed storage device when the first comparison unit judges that the RAM dirty data water level value is greater than the preset threshold.

In the fourth embodiment of the second aspect of the embodiments of the present invention or in the fifth embodiment of the second aspect of the embodiments of the present invention, the data storage system further includes:

a receiving unit, configured to receive an instruction to eliminate data stored in the high-speed storage device;

a query unit, configured to query the data stored on the high-speed storage device with the flag after the receiving unit receives an instruction to eliminate the data stored in the high-speed storage device;

A deletion unit, configured to write the data set with the flag from the high-speed storage device into the low-speed storage device.

In the sixth embodiment of the second aspect of the embodiments of the present invention, the data storage system further includes:

A selection unit, configured to select a preset number of storage blocks corresponding to the data with the flag set;

The merging unit is configured to combine the selected storage blocks into one storage block.

In the first embodiment of the second aspect of the embodiments of the present invention or in the second embodiment of the second aspect of the embodiments of the present invention, the first comparison unit includes:

A threshold comparison unit, configured to compare the RAM dirty data water level with a preset low water level threshold and a preset high water level threshold, wherein the preset high water level threshold is greater than the preset low water level threshold.

In the eighth embodiment of the second aspect of the embodiments of the present invention, the hotspot identification unit includes:

The first hotspot identification unit is used for the threshold comparison unit to judge that the RAM dirty data water level value is less than the preset low water level threshold, and to identify whether the RAM dirty data is hotspot data;

Alternatively, the hotspot identification unit includes:

The second hotspot identification unit is used for the threshold comparison unit to determine whether the RAM dirty data water level is smaller than the preset high water level threshold, and to identify whether the RAM dirty data is hotspot data.

In the eighth embodiment of the second aspect of the embodiments of the present invention, the data writing unit includes:

The first data writing unit is used for the threshold comparison unit to judge that the RAM cache data water level value is greater than the preset high water level threshold, and write the RAM dirty data into the high-speed storage device;

Alternatively, the data writing unit includes:

The second data writing unit is used for the threshold comparison unit to judge that the RAM dirty data water level value is greater than the preset low water level threshold, and write the RAM dirty data into the high-speed storage device.

It can be seen from the above technical solutions that in the data storage method disclosed in the embodiment of the present invention, before storing data, the RAM dirty data water level value is obtained first, and when it is detected that the RAM dirty data water level has increased, it indicates that a burst Write situation, at this time, determine the storage mode of RAM dirty data according to the relationship between the RAM dirty data water level value and the preset threshold value, specifically, when the RAM dirty data water level value is less than the preset threshold value, it means that the data to be written is relatively small , identify whether the RAM dirty data is hot data, and then write the hot data into the high-speed storage device, which can avoid excessive data writing into the high-speed storage device, and solve the problem caused by non-hot data occupying too many resources of the high-speed storage device The problem of hot data being squeezed out of high-speed storage devices; at the same time, it can also solve the problem of reducing the service life of high-speed storage devices due to excessive data writing into high-speed storage devices.

Moreover, in the data storage method disclosed in the embodiment of the present invention, if the RAM dirty data water level increases, and the RAM dirty data water level value is greater than the preset threshold value, it means that there is more data to be written. Writing data to high-speed storage devices can solve the problem of data writing to low-speed storage devices. Due to the low performance of low-speed storage devices, it will seriously affect the data storage rate and cause storage delays.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

FIG. 1 is a flowchart of a data storage method disclosed in an embodiment of the present invention;

Fig. 2 is a flow chart of a data storage method disclosed in another embodiment of the present invention;

FIG. 3 is a flow chart of a data storage method disclosed in another embodiment of the present invention;

Fig. 4 is a flowchart of a data storage method disclosed in another embodiment of the present invention;

Fig. 5 is a flowchart of a data storage method disclosed in another embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a data storage system disclosed in an embodiment of the present invention;

Fig. 7 is a schematic structural diagram of a data storage system disclosed in another embodiment of the present invention;

Fig. 8 is an application diagram of a data storage system disclosed by another embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The Chinese and English translations of the key techniques applied in the embodiments of the present invention will be described below.

RAM: random access memory, random access memory;

SSD: Solid State Device/Solid State Drive, solid state drive;

HDD: Hard Disk Drive, hard disk drive.

The embodiment of the present invention provides a data storage method to solve the problem of non-hot data occupying too many resources of high-speed storage devices in existing data storage methods, causing hot data to be squeezed out of high-speed storage devices and excessive data writing A high-speed memory device causes a problem of reducing the service life of the high-speed memory device.

What needs to be explained here is: the high-speed storage device mentioned in all the following embodiments of the present invention is a storage device with a faster storage speed in the prior art, such as: SSD disk, SSD array, etc.; correspondingly, the low-speed storage device is Storage devices with slow storage speed, such as HDD, etc.

Referring to Fig. 1, the data storage method disclosed by the embodiment of the present invention includes steps:

S101. Acquiring the water level value of RAM dirty data;

Wherein: during the data storage process, the data is first cached in the RAM, and then the data in the RAM is written into a high-speed storage device or a low-speed storage device. Data that needs to be written from the RAM to a high-speed storage device or a low-speed storage device is called RAM dirty data.

Moreover, the variable information in the RAM includes identification information, and the identification information reflects whether the data cached in the RAM is dirty data. The RAM dirty data water level reflects the amount of dirty data cached in the RAM. When the cached quantity is large, the RAM dirty data water level value is high, and when the cached quantity is small, the RAM dirty data water level value is low. When a burst write situation occurs, the data cached into RAM will be in large quantities, which will lead to a large quantity of data that needs to be written from RAM to high-speed storage devices or low-speed storage devices. At this time, the dirty data of RAM It will increase, and the water level value of the dirty data in RAM will increase.

S102. According to the obtained RAM dirty data water level value, judge whether the RAM dirty data water level has increased;

Wherein, by judging whether the water level of dirty data in the RAM has increased, it can be known whether a sudden writing situation has occurred.

Specifically, according to the obtained RAM dirty data water level value, the method of judging whether the RAM dirty data water level is increased may be: judging whether the increase rate of the RAM dirty data water level value reaches a preset rate;

Wherein, when the increase rate of the dirty data water level of the RAM reaches a preset rate, it indicates that the dirty data water level of the RAM has increased, and a burst write situation has occurred at this time.

Alternatively, determine whether the RAM dirty data water level has increased by judging whether the RAM dirty data water level reaches a preset water level value.

Wherein, when the dirty data water level of the RAM reaches a preset water level, it means that the dirty data water level of the RAM has increased, and a sudden writing situation has occurred.

In this embodiment, the rate for judging whether the RAM dirty data water level increases and the selection range of the water level value can be determined by those skilled in the art according to actual conditions.

If it increases, then execute S103, compare the dirty data water level of the RAM with the preset threshold value; wherein, when it is determined that the dirty data water level of the RAM increases, it means that a sudden write situation has occurred at this time, and the obtained RAM dirty data water level value and preset Set the threshold for comparison, and select the storage method of RAM dirty data according to the comparison result.

Specifically, the preset threshold is a reasonable water level value of the dirty data water level of the RAM, which reflects that the RAM has accumulated dirty data, but has not accumulated a large amount of dirty data. Wherein, the value range of the threshold may be determined according to the storage space value of the write page of the RAM, and may be any percentage of the storage space value of the write page of the RAM, preferably, the percentage may be 50%.

S104. If the water level value of the RAM dirty data is less than the preset threshold, identify whether the RAM dirty data is hot data;

Wherein, when the water level value of the RAM dirty data is less than the preset threshold value, it means that the corresponding RAM dirty data is relatively small, that is, the burst write data is relatively small, and at this time, before storing the data, first identify whether the data is hot data.

Specifically, the method of identifying whether the data is hot data may include: counting the writing frequency of data per unit time, and when the writing frequency reaches a certain value, the data is determined to be hot data; the written data may also be counted , when the count reaches a certain value, the data is identified as hot data. Wherein, those skilled in the art should know the setting methods and selection ranges of the setting values of the above two methods.

Of course, the methods for identifying whether the RAM dirty data is hot data in this embodiment are not limited to the above two methods, but also include all methods in the prior art, which will not be repeated here.

S105. If identified as hot data, write the RAM dirty data into the high-speed storage device; otherwise, perform S106, write the RAM dirty data into the low-speed storage device;

S107. If the RAM dirty data level increases and the RAM cache data level value is greater than a preset threshold, write the RAM dirty data into the high-speed storage device.

Wherein, when the dirty data water level of the RAM increases, and the dirty data water level value of the RAM is greater than the preset threshold value, it means that the corresponding RAM dirty data is more, that is, there are more data written in a burst, and at this time, the dirty data of the RAM is directly written Entering the high-speed storage device can solve the problem that too much data is written into the low-speed storage device. Due to the low performance of the low-speed storage device, the storage rate of data is seriously affected and the storage delay is formed.

In the data storage method disclosed in this embodiment, before storing data, first obtain the water level value of the dirty data in the RAM. When it is detected that the water level of the dirty data in the RAM increases, it means that a burst write situation occurs at present. At this time, according to the dirty data in the RAM The relationship between the water level value and the preset threshold value determines the storage mode of RAM dirty data. Specifically, when the water level value of the RAM dirty data is less than the preset threshold value, it means that there is less data to be written, and it is possible to identify whether the RAM dirty data is a hotspot Data, the data identified as hotspots are then written into high-speed storage devices, which can avoid excessive data writing into high-speed storage devices, and solve the problem that non-hotspot data occupying too many resources of high-speed storage devices causes hot data to be squeezed out of high-speed storage devices ; At the same time, it can also solve the problem of reducing the service life of the high-speed storage device due to excessive data writing into the high-speed storage device.

Another embodiment of the present invention also discloses a data storage method, see Fig. 2, including steps:

S201. Obtain the water level value of RAM dirty data;

S202. According to the obtained RAM dirty data water level value, determine whether the RAM dirty data water level has increased;

Same as the above-mentioned embodiment, the method for judging whether the dirty data level of the RAM is increased can be as follows:

Judging whether the increase rate of the RAM dirty data water level reaches a preset rate; or judging whether the RAM dirty data water level reaches a preset water level.

If it is judged that the water level of the RAM dirty data has not increased, then perform S203: acquire the value of the storage space occupied by the RAM dirty data;

What needs to be explained here is that when a burst write situation occurs, a large amount of data will be cached in the RAM. At this time, the dirty data level of the RAM will increase, wherein the data corresponding to the burst write situation can be the data itself It can also be that the data itself is not large, but the number is large. Both of these situations will lead to an increase in the level of dirty data in the RAM.

However, if only individual data is cached in the RAM, no matter whether the data itself is large or small, the level of dirty data in the RAM will not increase.

S204. If the storage space value occupied by the RAM dirty data is greater than a preset value, write the RAM dirty data into a low-speed storage device;

Wherein, when individual data is cached in the RAM, the storage space value occupied by the dirty data of the RAM is obtained, and the value is compared with the preset value, and when it is judged to be greater than the preset value, the dirty data of the RAM is written into a low-speed storage device. In this way, the problem of resource occupation by writing large data into a high-speed storage device can be avoided.

Moreover, the preset value can be set according to actual needs, and there is no specific requirement.

If it increases, execute S205, compare the RAM dirty data water level value with a preset threshold;

S206. If the water level value of the RAM dirty data is less than a preset threshold, identify whether the RAM dirty data is hot data;

S207. If identified as hot data, write the RAM dirty data into the high-speed storage device; otherwise, execute S208, write the RAM dirty data into the low-speed storage device;

S209. If the RAM dirty data level increases, and the RAM cache data level value is greater than a preset threshold, write the RAM dirty data into a high-speed storage device.

The specific implementation process of each step in this embodiment is the same as the content disclosed in the above embodiment, please refer to the content disclosed in the above embodiment, and will not be repeated here.

It should be noted here that, in step S204 of the above embodiment, if the storage space occupied by the RAM dirty data is greater than a preset value, the RAM dirty data can be written into a low-speed storage device, and the write-through method can be used to write The RAM dirty data is written into the low-speed storage device.

Specifically, the write-through mode specifically refers to: after the data is cached in the RAM, the write completion command is not fed back, and the write completion command is returned after the data is written into the low-speed storage device.

Alternatively, in step S204, if the storage space value occupied by the RAM dirty data is greater than a preset value, the RAM dirty data is written into the low-speed storage device, and the RAM dirty data can be written into the low-speed storage device in a write-back manner. storage device.

Specifically, the write-back manner specifically refers to: after the data is cached in the RAM, no matter whether the data is written to the low-speed storage device, a write completion command is returned.

Of course, in the case that the storage space occupied by the RAM dirty data is greater than a preset value, other ways of writing the RAM dirty data into the low-speed storage device may also be used, which will not be described here one by one.

Another embodiment of the present invention also discloses a data storage method, as shown in Figure 3, including steps:

S301. Obtain the water level value of RAM dirty data;

S302. According to the obtained RAM dirty data water level value, judge whether the RAM dirty data water level has increased;

If it increases, execute S303, comparing the RAM dirty data water level value with a preset threshold;

S304. If the water level value of the RAM dirty data is less than a preset threshold, identify whether the RAM dirty data is hot data;

S305. If identified as hot data, write the RAM dirty data into the high-speed storage device; otherwise, execute S306, write the RAM dirty data into the low-speed storage device;

S307. If the RAM dirty data water level increases, and the RAM cache data water level value is greater than a preset threshold, write the RAM dirty data into the high-speed storage device;

S308. Mark the RAM dirty data stored in the high-speed storage device.

Wherein: in steps S305 and S307, the RAM dirty data is written into the high-speed storage device, and after the data is written into the high-speed storage device, the data is marked.

What needs to be explained in this embodiment is that writing data to a high-speed storage device is generally called a data cache, and only writing data to a low-speed storage device is called storing data. Therefore, in other embodiments of the present invention, in addition to including the content of the embodiment corresponding to FIG. 3 , this embodiment also includes:

After receiving the instruction to eliminate the data stored in the high-speed storage device, query the data stored on the high-speed storage device with the mark;

Writing the data set with the flag from the high-speed storage device to the low-speed storage device.

Wherein, when corresponding to the high-speed storage device storing marked RAM dirty data, the marked RAM dirty data is data in a burst write situation, and the data is deleted from the high-speed storage device to the low-speed storage device, The storage space of the high-speed storage device can be released; and the data is eliminated from the high-speed storage device to the low-speed storage device, which can avoid the problem of data loss caused by directly deleting data from the high-speed storage device.

Moreover, the marked data is data that is prioritized to be eliminated. After receiving the instruction to eliminate the data stored in the high-speed storage device, the data with the mark in the high-speed storage device is first eliminated. When the elimination is completed Afterwards, it may be further judged whether the received instruction for eliminating data stored in the high-speed storage device is satisfied, and if the requirement is still not met, then the unmarked data in the high-speed storage device is eliminated.

Corresponding to the different methods of eliminating the data stored in the high-speed storage device to the low-speed storage device, another embodiment of the present invention also discloses a data storage method, as shown in FIG. 4 , including steps:

S401. Obtain the water level value of RAM dirty data;

S402. According to the acquired RAM dirty data water level value, determine whether the RAM dirty data water level has increased;

If it increases, execute S403, comparing the RAM dirty data water level with a preset threshold;

S404. If the water level value of the RAM dirty data is less than a preset threshold, identify whether the RAM dirty data is hot data;

S405. If identified as hot data, write the RAM dirty data into the high-speed storage device; otherwise, execute S406, write the RAM dirty data into the low-speed storage device;

S407. If the RAM cache data water level value is greater than a preset threshold, write the RAM dirty data into a high-speed storage device;

S408. Mark the RAM dirty data stored in the high-speed storage device;

Similarly, in steps S405 and S407, the RAM dirty data is written into the high-speed storage device, and after the data is written into the high-speed storage device, the data is marked.

S409. After receiving the instruction to eliminate the data stored in the high-speed storage device, query the data stored on the high-speed storage device with the flag;

S410. Select a preset number of storage blocks corresponding to the data with the flag set;

Among them, the high-speed storage device is divided into multiple storage blocks. Writing data into the high-speed storage device refers to writing data into the storage block. When the data is relatively large and one storage block is not enough to store the data, it needs to be stored in in adjacent storage blocks.

Before the data in the high-speed storage device needs to be eliminated to the low-speed storage device, search for the mark of the data stored in the storage block of the high-speed storage device, if the mark is found, it means that the data stored in the storage block is Needs to be phased out to the low speed storage device.

And, in order to improve the speed of eliminating data from the high-speed storage device to the low-speed storage device, multiple storage blocks with marked data can be stored; wherein, the number of mergers can be two or can be Others can be set according to the actual situation.

It should be noted here that those skilled in the art should know that the merged storage blocks are selected as adjacent storage blocks.

S411. Merge the selected storage blocks into one storage block;

Specifically, take merging two storage blocks as an example for illustration. Two adjacent storage blocks have their own start addresses and lengths. The start address + length of the first storage block is equal to the address of the second storage block. initial address. When two storage blocks need to be merged, the starting address of the first storage block of the two storage blocks to be merged is used as the starting address of the combined storage block, and the length of the two storage blocks is used as the combined storage The length of the block.

S412. Write the data set with the flag from the high-speed storage device to the low-speed storage device.

At this time, after the memory blocks storing the data with the flags are merged, the data of the merged memory blocks are written into the low-speed storage device.

Another embodiment of the present invention also discloses a data storage method, as shown in Figure 5, including:

S501. Acquiring the water level value of RAM dirty data;

S502. According to the obtained RAM dirty data water level value, determine whether the RAM dirty data water level has increased;

If it increases, then execute S503, and compare the dirty data water level value of the RAM with a preset low water level threshold and a preset high water level threshold;

Wherein, in the present embodiment, a low water level threshold and a high water level threshold are set, and the RAM dirty data water level is compared with the low water level threshold and the high water level threshold respectively, so that the corresponding RAM dirty data can be known more accurately Size, that is, the size of the data written in a burst.

Moreover, the high water level threshold and the low water level threshold are respectively arbitrary percentages of the storage space value of the write page of the RAM, which can be set according to the actual situation, as long as the high water level threshold is greater than the low water level threshold.

S504. If the RAM dirty data water level value is less than the preset low water level threshold, identify whether the RAM dirty data is hot data;

S505. If identified as hot data, write the RAM dirty data into the high-speed storage device; otherwise, execute S506, write the RAM dirty data into the low-speed storage device;

S507. If the RAM dirty data water level increases, and the RAM cache data water level value is greater than the preset high water level threshold, write the RAM dirty data into the high-speed storage device.

In this embodiment, the RAM dirty data water level value is less than the preset low water level threshold, and before data storage, it is necessary to identify whether the RAM dirty data is hot data; and when the RAM dirty data water level value is greater than the preset high water level threshold, and directly write the RAM dirty data into the high-speed storage device.

If the comparison result of the RAM dirty data water level with the preset high water level threshold and the preset low water level threshold is: when the RAM dirty data is between the preset high water level threshold and the preset low water level threshold, it can be directly The RAM dirty data is written into the high-speed storage device.

Certainly, when the water level of the RAM dirty data is between a preset high water level threshold and a preset low water level threshold, the RAM dirty data may also be hotspot identified, and the RAM dirty data may be stored according to a hot spot identification result.

Moreover, in this embodiment, the AM dirty data water level value between the preset high water level threshold and the preset low water level threshold may include two endpoint values, the preset high water level threshold and the preset low water level threshold.

In the embodiment corresponding to Fig. 3, Fig. 4 and Fig. 5, the following steps may also be included:

Judging that the water level of the RAM dirty data has not increased, and obtaining the storage space value occupied by the RAM dirty data;

If the storage space occupied by the RAM dirty data is greater than a preset value, write the RAM dirty data into the low-speed storage device.

Moreover, when the storage space occupied by the dirty RAM data is greater than a preset value, the dirty RAM data may be written into the low-speed storage device in a write-through or write-back manner.

Specifically, for the specific implementation process of the above two steps, please refer to the content of the embodiment corresponding to FIG. 2 , which will not be repeated here.

Another embodiment of the present invention also discloses a data storage system, as shown in Figure 6, including:

The first acquiring unit 101 is configured to acquire the RAM dirty data water level value;

A judging unit 102, configured to judge whether the RAM dirty data water level has increased according to the obtained RAM dirty data water level value;

Preferably, the judging unit 102 may include a first judging unit and a second judging unit, wherein:

The first judging unit is used to detect whether the increase rate of the RAM dirty data level value reaches a preset rate; the second judging unit is used to detect whether the RAM dirty data water level value reaches a preset water level value.

Specifically, when the first judging unit detects that the increase rate of the RAM dirty data water level reaches a preset rate or the second judging unit detects that the RAM dirty data water level reaches a preset water level, it means that the RAM is dirty. The data water level increases, and a burst write occurs at this time.

The first comparison unit 103 is used for the judging unit 102 to judge that the RAM dirty data water level has increased, and compare the RAM dirty data water level value with a preset threshold;

The hotspot identification unit 104 is used for the first comparison unit 103 to judge that the RAM dirty data water level value is less than a preset threshold, and to identify whether the RAM dirty data is hotspot data;

The data writing unit 105 is used for the hotspot identification unit 104 to identify the dirty data of the RAM as hotspot data, and write the dirty data of the RAM into a high-speed storage device; the hotspot identification unit 104 identifies the dirty data of the RAM as non-hotspot data, and writes the dirty data to the high-speed storage device. The RAM dirty data is written into the low-speed storage device; the first comparison unit 103 judges that the RAM cache data water level is greater than a preset threshold, and writes the RAM dirty data into the high-speed storage device.

Specifically, in the data storage system disclosed in this embodiment, the first acquiring unit acquires the water level value of the RAM dirty data, and when the judging unit judges that the water level of the dirty data in the RAM has increased, it indicates that a burst write situation occurs currently, and at this time, The data writing unit 105 determines the storage mode of the RAM dirty data according to the comparison result of the first comparison unit 103. Specifically, when the first comparison unit 103 judges that the RAM dirty data water level value is less than a preset threshold, the hotspot identification unit 104 identifies whether the RAM dirty data is hot data, and when the hot spot identification unit 104 identifies it as hot data, the data writing unit 105 writes the data into the high-speed storage device, which can avoid excessive data writing into the high-speed storage device, and is identified as For non-hot data, the data writing unit 105 writes the data into the low-speed storage device, so that the problem that the hot data is squeezed out of the high-speed storage device caused by the excessive resources occupied by the non-hot data of the high-speed storage device is solved; Writing data into a high-speed storage device causes a problem of reducing the service life of the high-speed storage device.

Moreover, when the first comparison unit 103 judges that the RAM dirty data water level value is greater than the preset threshold value, it means that the corresponding RAM dirty data is more, that is, there is more data written in a burst, and at this time, the data writing unit 105 directly writes the RAM dirty data. Writing data into high-speed storage devices can solve the problem of writing too much data into low-speed storage devices. Due to the low performance of low-speed storage devices, it will seriously affect the data storage rate and cause storage delays.

For the specific working process of each unit in the data storage system provided in this embodiment, please refer to the embodiment corresponding to FIG. 1 , which will not be repeated here.

Another embodiment of the present invention also discloses a data storage system, as shown in FIG. In addition, it also includes:

The second acquiring unit 206 is used for the judging unit 202 to judge that the RAM dirty data water level has not increased, and acquire the storage space value occupied by the RAM dirty data;

The second comparison unit 207 is configured to compare the storage space value occupied by the RAM dirty data with a preset value;

Among them, the working process of the first acquiring unit 201, the judging unit 202, the first comparing unit 203, the hotspot identifying unit 204 and the data writing unit 205 are the same as those disclosed in the above-mentioned embodiments. The unit 103, the hotspot identifying unit 104 and the data writing unit 105 are the same, please refer to the content of the above-mentioned embodiment. In addition, in this embodiment, the data writing unit 205 is also used to judge the When the storage space occupied by the dirty RAM data is greater than a preset value, write the dirty RAM data into the low-speed storage device.

Specifically, when individual data is cached in the RAM, no matter whether the data itself is large or small, the water level of the dirty data in the RAM will not increase. At this time, the second acquisition unit 206 acquires the storage space occupied by the dirty data in the RAM. value; the second comparison unit 207 compares the storage space value and the preset value occupied by the RAM dirty data; when the second comparison unit 207 judges that the storage space value occupied by the RAM dirty data exceeds the preset value, the data writing unit 205 Write the RAM dirty data into the low-speed storage device. In this way, the problem of resource occupation by writing large data into the high-speed storage device can be avoided.

It should be noted here that the data writing unit 205 may write the RAM dirty data into the low-speed storage device by using a write-through method to write the RAM dirty data into the low-speed storage device; or, use a write-back method Writing the RAM dirty data into the low-speed storage device.

Certainly, in the case that the storage space value occupied by RAM dirty data is greater than a preset value, the mode of writing the RAM dirty data into the low-speed storage device by the data writing unit 205 can also be other ways, which are not mentioned here one by one. illustrate.

Preferably, in the above-mentioned embodiments, the data storage unit may further include:

The first marking unit is used to identify the RAM dirty data as hotspot data in the hot spot identification unit in the data writing unit, and mark the RAM dirty data written in the high-speed storage device;

The second marking unit is configured to mark the RAM dirty data written into the high-speed storage device when the first comparison unit judges that the RAM dirty data water level value is greater than the preset threshold.

Another embodiment of the present invention also discloses a data storage system, as shown in Figure 8, including:

The first acquiring unit 301, the judging unit 302, the first comparing unit 303, the hot spot identifying unit 304 and the data writing unit 305 also include:

A marking unit 306, configured to mark RAM dirty data stored in the high-speed storage device;

The receiving unit 307 is configured to receive an instruction to eliminate data stored in the high-speed storage device;

The query unit 308 is configured to query the data stored on the high-speed storage device with the mark after the receiving unit 307 receives the instruction to eliminate the data stored in the high-speed storage device;

The deletion unit 309 is configured to write the data set with the flag from the high-speed storage device into the low-speed storage device.

In this embodiment, corresponding to the case where the high-speed storage device stores marked RAM dirty data, the marked RAM dirty data is data in a burst write situation. When the receiving unit 307 receives an instruction from the outside to eliminate data stored in the high-speed storage device, the query unit 308 searches for the marked data stored on the high-speed storage device, and the deleting unit 309 removes the marked data searched by the query unit 308 from Removing the high-speed storage device to the low-speed storage device can release the storage space of the high-speed storage device; and, eliminating data from the high-speed storage device to the low-speed storage device can prevent data from being directly transferred from the high-speed storage device Delete issues that cause data loss.

Preferably, the data storage system disclosed in the above embodiments may further include:

a selection unit, configured to select a preset number of storage blocks corresponding to the marked data;

The merging unit is configured to combine the selected storage blocks into one storage block.

At this time, the high-speed storage device is divided into a plurality of storage blocks, and after the storage blocks storing the data provided with the mark are merged, the data of the merged storage block is written into the low-speed storage device, which can improve the speed of the storage. The speed at which data in the high-speed storage device is written to the low-speed storage device.

And in the above several embodiments of the present invention, preferably, the first comparison unit includes: a threshold comparison unit, configured to compare the RAM dirty data water level with a preset low water level threshold and a preset high water level threshold respectively , wherein the preset high water level threshold is greater than the preset low water level threshold.

At this time, the hot spot identification unit is: a first hot spot identification unit, used for the threshold comparison unit to judge that the RAM dirty data water level value is less than the preset low water level threshold, and the hot spot identifies the RAM dirty data;

Alternatively, the hotspot identification unit is: a second hotspot identification unit, configured for the threshold comparison unit to judge that the RAM dirty data water level is less than the preset high water level threshold, and to identify whether the RAM dirty data is hotspot data.

Correspondingly, the data writing unit is: a first data writing unit, used for the threshold comparison unit to judge that the RAM cache data water level value is greater than the preset high water level threshold, and write the RAM dirty data into The high-speed storage device;

Alternatively, the data writing unit is: a second data writing unit, used for the threshold comparison unit to judge that the RAM dirty data water level value is greater than the preset low water level threshold, and write the RAM dirty data into the high-speed storage devices.

For the specific working process of the units disclosed in this embodiment, please refer to the embodiment corresponding to FIG. 5 , which will not be repeated here.

Finally, it should also be noted that in this text, relational terms such as first and second etc. are only used to distinguish one entity or operation from another, and do not necessarily require or imply that these entities or operations, any such actual relationship or order exists. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (20)

1. a date storage method, is characterized in that, comprising:

Obtain random access memory ram dirty data water level value;

According to the described RAM dirty data water level value obtained, judge whether described RAM dirty data water level improves;

If described RAM dirty data water level improves, more described RAM dirty data water level value and predetermined threshold value;

Whether if described RAM dirty data water level value is less than described predetermined threshold value, identifying described RAM dirty data is hot spot data;

If be identified as hot spot data, described RAM dirty data is write to high speed storing equipment; Otherwise write low speed storage device;

If described RAM dirty data water level improves, and described RAM dirty data water level value is greater than described predetermined threshold value, and described RAM dirty data is write to described high speed storing equipment.

2. method according to claim 1, is characterized in that, also comprises:

If described RAM dirty data water level does not improve, obtain the storage space value that described RAM dirty data takies;

If the storage space value that described RAM dirty data takies is greater than preset value, described RAM dirty data is write to described low speed storage device.

3. method according to claim 1 and 2, is characterized in that, describedly judges whether described RAM dirty data water level improves and comprise:

Whether the lifting speed that judges described RAM dirty data water level value reaches default speed;

Perhaps, judge whether described RAM dirty data water level value reaches the preset water level value.

4. method according to claim 1 and 2, it is characterized in that, being less than described predetermined threshold value and identifying described RAM dirty data at described RAM dirty data water level value is hot spot data, after described RAM dirty data is write to described high speed storing equipment, also comprise: the described RAM dirty data to described high speed storing device storage carries out mark.

5. method according to claim 1 and 2, it is characterized in that, if improve at described RAM dirty data water level, and described RAM dirty data water level value is greater than in described predetermined threshold value situation, after described RAM dirty data is write to described high speed storing equipment, also comprise: the described RAM dirty data to described high speed storing device storage carries out mark.

6. according to the described method of claim 4 or 5, it is characterized in that, also comprise:

After receiving the instruction of the data of eliminating described high speed storing device storage, inquire about the data that are provided with described mark of storing on described high speed storing equipment;

The data that are provided with described mark are write to described low speed storage device by described high speed storing equipment.

7. method according to claim 6, is characterized in that, after the data that are provided with described mark of storing on the described high speed storing equipment of described inquiry, also comprises:

Choose the storage block corresponding to the data that are provided with described mark of predetermined number;

The described storage block of choosing is merged into to a storage block.

8. method according to claim 1 and 2, is characterized in that, described predetermined threshold value comprises default low level threshold and preset high water level threshold value, and wherein, described preset high water level threshold value is greater than described default low level threshold.

9. method according to claim 8, is characterized in that, whether if described RAM dirty data water level value is less than described predetermined threshold value, identifying described RAM dirty data is hot spot data, for:

Whether if described RAM dirty data water level value is less than described default low level threshold, identifying described RAM dirty data is hot spot data;

Perhaps, described RAM dirty data water level value is less than described preset high water level threshold value, and whether identify described RAM dirty data is hot spot data.

10. method according to claim 8, is characterized in that, if described RAM dirty data water level improves, and described RAM dirty data water level value is greater than described predetermined threshold value, and described RAM dirty data is write to described high speed storing equipment, for:

If described RAM dirty data water level improves, and the data cached water level value of described RAM is greater than described preset high water level threshold value, and described RAM dirty data is write to described high speed storing equipment;

Perhaps, if described RAM dirty data water level improves, and described RAM dirty data water level value is greater than described default low level threshold, and described RAM dirty data is write to described high speed storing equipment.

11. a data-storage system, is characterized in that, comprising:

The first acquiring unit, for obtaining described RAM dirty data water level value;

Judging unit, the described RAM dirty data water level value for according to obtaining, judge whether described RAM dirty data water level improves;

The first comparing unit, while improving for the described RAM dirty data of described judgment unit judges water level, more described RAM dirty data water level value and predetermined threshold value;

The focus recognition unit, judge that for described the first comparing unit described RAM dirty data water level value is less than predetermined threshold value, and whether identify described RAM dirty data is hot spot data;

Data write unit, identifying described RAM dirty data for described focus recognition unit is hot spot data, and described RAM dirty data is write to high speed storing equipment; It is non-hot spot data that described focus recognition unit is identified described RAM dirty data, and described RAM dirty data is write to low speed storage device; Described the first comparing unit judges that the data cached water level value of described RAM is greater than predetermined threshold value, writes described high speed storing equipment by described RAM dirty data.

12. system according to claim 11, is characterized in that, also comprises:

Second acquisition unit, do not improve for the described RAM dirty data of described judgment unit judges water level, obtains the storage space value that described RAM dirty data takies;

The second comparing unit, the storage space value and the preset value that for more described RAM dirty data, take;

Described data write unit also judges that for described the second comparing unit the storage space value that described RAM dirty data takies is greater than preset value, writes described low speed storage device by described RAM dirty data.

13. according to the described system of claim 11 or 12, it is characterized in that, described judging unit comprises:

Whether the first judging unit, reach default speed for the lifting speed that judges described RAM dirty data water level value;

Whether the second judging unit, reach the preset water level value for judging described RAM dirty data water level value.

14. according to the described system of claim 11 or 12, it is characterized in that, also comprise:

The first indexing unit, be hot spot data at described data write unit, at described focus recognition unit, identifying described RAM dirty data, and the RAM dirty data that writes described high speed storing equipment is carried out to mark.

15. according to the described system of claim 11 or 12, it is characterized in that, also comprise:

The second indexing unit, be greater than in described predetermined threshold value situation for judge described RAM dirty data water level value at described the first comparing unit, and the RAM dirty data that writes described high speed storing equipment is carried out to mark.

16. according to the described system of claims 14 or 15, it is characterized in that, also comprise:

Receiving element, for receiving the instruction of the data of eliminating described high speed storing device storage;

Query unit, receive the instruction of the data of eliminating described high speed storing device storage for described receiving element after, inquire about the data that are provided with described mark of storing on described high speed storing equipment;

Delete cells, write described low speed storage device for the data that will be provided with described mark by described high speed storing equipment.

17. system according to claim 16, is characterized in that, also comprises:

Choose unit, for choosing the storage block corresponding to the data that are provided with described mark of predetermined number;

Merge cells, for merging into a storage block by the described storage block of choosing.

18. according to the described system of claim 11 or 12, it is characterized in that, described the first comparing unit comprises:

The threshold value comparing unit, for described RAM dirty data water level value is compared with default low level threshold and preset high water level threshold value respectively, wherein, described preset high water level threshold value is greater than described default low level threshold.

19. system according to claim 18, is characterized in that, described focus recognition unit comprises:

The first focus recognition unit, judge that for described threshold value comparing unit described RAM dirty data water level value is less than described default low level threshold, and whether identify described RAM dirty data is hot spot data;

Perhaps, described focus recognition unit comprises:

The second hot area recognition unit, judge that for described threshold value comparing unit described RAM dirty data water level value is less than described preset high water level threshold value, and whether identify described RAM dirty data is hot spot data.

20. system according to claim 18, is characterized in that, described data write unit comprises:

The first data write unit, judge that for described threshold value comparing unit the data cached water level value of described RAM is greater than described preset high water level threshold value, writes described high speed storing equipment by described RAM dirty data;

Perhaps, described data write unit comprises:

The second data write unit, judge that for described threshold value comparing unit described RAM dirty data water level value is greater than described default low level threshold, writes described high speed storing equipment by described RAM dirty data.

Images