CN103473298B - Data archiving method and device and storage system - Google Patents

Abstract

The invention discloses a data archiving method, device and storage system. The method is applied to a storage system including at least two storage levels with different data access speeds, each storage level contains containers, and the method includes: judging the first storage level based on the access reference information of the first container in the first storage level Whether a container needs to be migrated to the second storage level, and if it is determined that the first container needs to be migrated to the second storage level, determine the second container at the second storage level, and transfer the first container The stored data blocks are migrated to the second container, and the container address corresponding to the container number of the first container is changed to the container address of the second container. In the present invention, each storage level of the storage system is used as the same deduplication domain, and the data block migration process avoids time-consuming operations such as repeated data block query, rewriting file metadata, etc., and effectively improves the efficiency of data block migration in the archiving scene.

Description

Data archiving method and device and storage system

technical field

The invention relates to the field of data storage, in particular to a data archiving method and device and a storage system.

Background technique

Generally, devices with deduplication function can be used to reduce storage space and accommodate more data. Deduplication refers to the deduplication of data, that is, each file is cut into multiple data blocks according to a specific algorithm, and it is judged which data blocks are already stored in the storage system and which data blocks are not in the storage system. , and store the unstored data blocks in the container included in the storage system; then modify the metadata of the file to point to the data blocks in the container. Wherein, the metadata of a file refers to a description file corresponding to each file, which describes which data blocks the file consists of and which containers these data blocks are stored in.

In order to save data maintenance costs, data archiving can be carried out, that is, data is stored in containers (containers) of storage levels with different data access speeds according to the degree of activity. For example, active tier containers are used to store data less than 90 days old. The retention layer (Retention tier) container is used to store data with a long time and low access frequency, where the retention layer container can be a storage with low data access speed such as SATA (Serial Advanced Technology Attachment, Serial Advanced Technology Attachment) disk, array, etc. medium. Therefore, in order to implement data archiving, a data migration solution is needed to place data with low access frequency on a storage medium with low data access speed.

In the prior art, in the archiving scenario, when performing deduplication data migration, first find out the expired file stored at a certain level, and then determine whether the data block of the file is stored in the container at the next level, and transfer the unused The stored data blocks are stored in the next-level container, and the metadata of the file is finally modified.

For the above technology, since each level is checked separately, that is, each level is an independent deduplication domain, there may be duplicate data blocks between multiple levels, resulting in the process of data migration requiring repeated data block query, re-duplication Time-consuming operations such as writing file metadata.

Contents of the invention

technical problem

In view of this, the technical problem to be solved by the present invention is how to solve the problem of time-consuming operations such as repeated data block query and rewriting of file metadata during the data block migration process in the archiving scenario.

solution

In order to solve the above technical problems, according to an embodiment of the present invention, a data archiving method is provided, which is applied to a storage system including at least two storage levels with different data access speeds, each storage level includes a container; the container is Refers to a logical storage unit for storing data blocks, and the data blocks in any two containers are different from each other; the storage system includes a container mapping table, and the container mapping table is used to record the container number and A correspondence relationship of container addresses, where the container address refers to an address of the container in the storage system.

The data archiving method includes: judging whether the first container needs to be migrated to the second storage level based on the access reference information of the first container in the first storage level, wherein the access reference information includes reference times and access times At least one of, the number of references indicates the number of times the first container is referenced, and the number of accesses indicates the number of times the first container is accessed within a predetermined period of time; In the case of migration at the second storage level, the second container is determined at the second storage level, the data blocks stored in the first container are migrated to the second container, and the container mapping table and The container address corresponding to the container number of the first container is changed to the container address of the second container.

For the above data archiving method, when the data access speed of the second storage level is lower than the data access speed of the first storage level, in a possible implementation, the Judging whether the first container needs to be migrated to the second storage level based on the access reference information of the first container includes: judging whether the file expires based on the storage time of the file in the storage system; if it is determined that the file When it expires, the container storing the data blocks included in the file is used as the first container; based on the access reference information of the first container, it is judged whether the first container needs to be migrated to the second storage level.

For the above data archiving method, when the data access speed of the second storage level is higher than the data access speed of the first storage level, in a possible implementation, the Judging whether the first container needs to be migrated to the second storage level based on the access reference information of the first container, including: when any of the storage systems is updated according to file writing, deletion, and/or reading In the case of the access reference information of the container, the container whose access reference information has been updated is used as the first container, and based on the updated access reference information of the first container, it is judged whether the first container needs to be transferred to the second storage level Migration; or use each container in each storage level as the first container, and judge whether the first container needs to migrate to the second storage level based on the access reference information of the first container.

In order to solve the above technical problems, according to an embodiment of the present invention, a data archiving device is provided, which is applied to a storage system including at least two storage levels with different data access speeds, each storage level includes a container; the container is Refers to a logical storage unit for storing data blocks, and the data blocks in any two containers are different from each other; the storage system includes a container mapping table, and the container mapping table is used to record the container number and A correspondence relationship of container addresses, where the container address refers to an address of the container in the storage system.

The data archiving device includes: a migration management module, configured to determine whether the first container needs to be migrated to the second storage level based on the access reference information of the first container in the first storage level, wherein the access reference information includes At least one of the number of references and the number of visits, the number of references represents the number of times the first container is referenced, the number of visits represents the number of times the first container is accessed within a predetermined period of time; the migration module, and The migration management module is connected to determine a second container at the second storage level when the migration management module determines that the first container needs to be migrated to the second storage level, and transfer the first container to the second storage level. A data block stored in a container is migrated to the second container, and the container address corresponding to the container number of the first container in the container mapping table is modified to the container address of the second container.

For the above data archiving device, when the data access speed of the second storage level is lower than the data access speed of the first storage level, in a possible implementation manner, the data archiving device further includes A file management module connected to the migration management module, wherein: the file management module is configured to determine whether the file expires based on the storage time of the file in the storage system, and the migration management module is configured to , when the file management module determines that the file has expired, use the container that stores the data blocks included in the file as the first container; and based on the access reference information of the first container, determine that the Whether the first container needs to be migrated to the second storage tier.

For the above data archiving device, when the data access speed of the second storage level is higher than the data access speed of the first storage level, in a possible implementation manner, the migration management module is configured to : when the access reference information of any container in the storage system is updated according to the writing, deletion, and/or reading of files, using the container whose access reference information is updated as the first container, Based on the updated access reference information of the first container, determine whether the first container needs to migrate to the second storage level; or use each container in each storage level as the first container, based on the The access reference information of the first container determines whether the first container needs to be migrated to the second storage level.

In order to solve the above technical problems, according to an embodiment of the present invention, a storage system is provided, including at least two storage levels with different data access speeds, each storage level contains a container; the container refers to the logic for storing data blocks storage unit, and the data blocks in any two containers are different from each other; the container mapping table is used to record the corresponding relationship between the container number and container address of any one of the containers, and the container address refers to the container address in the storage system address; and a data archiving storage device according to any structure in the embodiments of the present invention.

Beneficial effect

In the present invention, firstly, based on the access reference information of the first container in the first storage level, it is judged whether the first container needs to be migrated to the second storage level, and when it is determined that the first container needs to be migrated to the second storage level In the case of layer migration, the second container is determined at the second storage layer, the data blocks stored in the first container are migrated to the second container, and the container mapping table is compared with the first The container address corresponding to the container number of the container is changed to the container address of the second container. In the data archiving method of the embodiment of the present invention, containers of different storage levels can be used as the same deduplication domain, which solves the problem of time-consuming operations in the data migration process such as repeated data block query, rewriting file metadata, etc., and effectively improves the efficiency of archiving scenarios. Data migration efficiency.

Other features and aspects of the present invention will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 shows a schematic structural diagram of a storage system applied to a data archiving method according to an embodiment of the present invention;

FIG. 2 shows a logical structural diagram of the storage system shown in FIG. 1;

Fig. 3 shows a flow chart of a data archiving method according to an embodiment of the present invention;

Fig. 4 shows a schematic diagram of container migration according to an embodiment of the present invention;

Fig. 5 shows a structure diagram of container access reference information according to an embodiment of the present invention;

FIG. 6 shows a flowchart of a data archiving method according to another embodiment of the present invention;

Fig. 7 shows a schematic diagram of container migration according to another embodiment of the present invention;

FIG. 8 shows a flowchart of a data archiving method according to yet another embodiment of the present invention;

Fig. 9 shows a structural block diagram of a data archiving device according to an embodiment of the present invention;

Fig. 10 shows a structural block diagram of a migration management module according to an embodiment of the present invention;

Fig. 11 shows a structural block diagram of a data archiving device according to another embodiment of the present invention; and

Fig. 12 shows a structural block diagram of a storage system according to an embodiment of the present invention.

detailed description

As mentioned in the background technology section, the technical problem to be solved by the present invention is mainly time-consuming operations such as repeated data block query and rewriting of file metadata during the data block migration process. In order to solve this technical problem, as described in the Summary of the Invention, the inventors have an original idea that, for a storage system including multiple storage levels, data block migration is performed in units of containers included in each storage level; the containers are Refers to the logical storage unit that stores data blocks, and the data blocks in the container are different from each other.

Specifically, firstly, based on the access reference information of the first container in the first storage level, it is judged whether the first container needs to be migrated to the second storage level, and when it is determined that the first container needs to be migrated to the second storage level In the case of migration, determine the second container at the second storage level, migrate the data blocks stored in the first container to the second container, and compare the container mapping table with the first container The container address corresponding to the container number is changed to the container address of the second container.

It should be noted that the container migration mentioned here, including the container migration mentioned in the following embodiments, does not refer to the migration of the logical storage unit of the container, but uses the container as a unit, and all data blocks included in the container migration performed.

The storage system provided by the embodiment of the present invention includes at least two storage levels, and the data access speeds of each storage level are different. For example, it includes three storage levels. Solid-state hard disk constitutes the storage level of the storage medium, the storage level of the storage medium is formed of the magnetic disk, and the storage level of the storage medium is formed of the magnetic tape.

Various exemplary embodiments, features, and aspects of the invention will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

In addition, in order to better illustrate the present invention, numerous specific details are given in the specific embodiments below. It will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, means, components and circuits are not described in detail so as to highlight the gist of the present invention.

Example 1

The data archiving method according to an embodiment of the present invention is applied to the storage system shown in Figure 1. The storage system is divided into multiple storage levels according to the data access speed of the storage medium, specifically, it can be divided into active tier (Active tier) and multiple tiers. A retention tier (Retention tier). Wherein, the data access speed of the storage tier as the active tier is higher than that of the storage tier as the reserved tier, and the data access speeds of the storage tiers as the reserved tier are also different. In a possible implementation manner, the active layer stores data written to the storage system within 90 days, and the reserved layer stores data written to the storage system for a longer period of time, for example, data from 90 days to 1 year or 1 year to 7 years. Deduplication data is stored in various storage levels with different data access speeds according to storage time and access times. Each storage tier includes containers.

FIG. 2 is a logical structural diagram of the storage system shown in FIG. 1 . As shown in FIG. 2 , each storage level of the storage system includes three containers (Containers). Among them, the namespace (Namespace) refers to the storage space presented by the user, such as a file system; the file (File) is the file stored in the namespace; the data block (chunk) is the data block divided from the file based on the deduplication function; Containers in each storage level are logical storage units that store data blocks of deduplicated files. Therefore, data blocks in containers are different from each other. At the same time, because the storage system uses a unified deduplication domain, the so-called deduplication domain is Refers to the scope of searching for duplicate data blocks. The storage system uses a unified deduplication domain, which means to search for duplicate data blocks in all storage levels of the entire storage system. Therefore, compared with the traditional use of independent deduplication domains for each storage level, the present invention The data blocks stored in the containers of each storage level of the storage system provided by the embodiment are different from each other.

Fig. 3 is a flowchart of a data archiving method according to an embodiment of the present invention. As shown in Figure 3, the data archiving method mainly includes the following steps:

S310. Based on the storage time of the stored file, determine whether the file expires.

Generally, when files are archived and stored, the upper-layer software or the user decides how long the files should be stored at a certain storage level in the storage system. According to this time, it can be determined whether the file expires.

S320. If it is determined that the file has expired, determine a container for storing each data block of the file based on the metadata of the file.

Wherein, the metadata includes a block number, a container number and an offset, the block number indicates the number of the data block included in the file, the container number indicates the number of the container storing the data block, and the offset The offset represents where the data block is stored on the container.

S330. For the determined container, determine whether the container needs to be migrated to a storage level with a low data access speed based on the access reference information of the container. Wherein, the low data access speed here is relative to the data access speed of the storage level where the container is located.

Wherein, the access reference information includes the number of references and/or the number of visits, the number of references indicates the number of times the container is referenced, and the number of visits indicates the number of times the container is accessed within a predetermined time period.

S340. When it is determined that the container needs to be migrated to a storage level with low data access speed, obtain a new container at the storage level with low data access speed, and migrate the data blocks stored in the container to the new container , and update the storage address corresponding to the container number of the container to the storage address of the new container.

Specifically, the data archiving method of this embodiment is described with reference to the schematic diagram of container migration shown in FIG. 4 . Firstly, arrange a group of expired files according to the storage time of a certain storage level in the storage system configured when the files are archived, and determine the container for storing the data blocks of the expired files according to the metadata of the expired files, As shown in Figure 4, suppose the data blocks of file 1 (F1) are stored in the first container c11 of the active layer La, and the data blocks of file 2 (F2) are stored in the second container c12 and the third container of the active layer La respectively In c13, the data blocks of the file 3 (F3) are respectively stored in the first container c31 and the third container c33 of the reserved layer Lr1. These containers may contain other data blocks that store unexpired files, so for these containers, it is judged whether each container needs to migrate to the retention layer based on their respective access reference information. Assuming that the third container c13 of the active layer La satisfies the migration condition, it is migrated to the reserved layer Lr0.

More specifically, the process of migrating the third container c13 of the active layer La to the reserved layer Lr0 is mainly completed through the container mapping table. The container mapping table is used to record the corresponding relationship between the container number of the container and the container address. The container address is used for Indicates the container's position in the storage hierarchy. All containers take a certain length of space from the address space of the storage level as the storage address of the container. In a possible implementation manner, the structure of the container mapping table is a table whose entry is a container number-container starting storage address. The migration process includes: first apply for a new container c21 in the reserved layer Lr0, assuming that the initial storage address of the new container c21 applied for is 1100; then migrate all the data blocks stored in the container c13 to the new container c21; finally Modify the container storage address corresponding to the container number of container c13 in the container mapping table. For example, assuming that the container number of the third container c13 in the active layer La is 0003, and the initial storage address of the container is 0300, then set the container number to 0003 corresponding to The initial storage address is changed to 1100.

In this way, all storage levels of the storage system serve as the same deduplication domain, and the data block migration process avoids time-consuming operations such as repeated data block queries and rewriting file metadata, effectively improving the efficiency of data block migration in archiving scenarios.

In a possible implementation, the migration conditions satisfied by the third container c13 of the active layer La include any of the following: the number of times the container is referenced by a file with a high activity level is sufficiently small; The number of references is high enough; the number of visits in the recent period is low enough. Wherein, the activity indicates the storage time of the file in the storage system and/or the read frequency, and the higher the file activity, the shorter the storage time of the file in the storage system and/or the higher the read frequency.

In a possible specific implementation, the migration condition satisfied by the third container c13 in the active layer La can be designed to be any of the following: the ratio of the first number of references of the container c13 to the total number of references of the container c13 is less than the first threshold; or the ratio of the second reference times of the container c13 to the total number of references of the container c13 is greater than the second threshold; the number of times the container c13 is accessed within a predetermined time period is sufficiently small. Wherein, the first number of references indicates the number of times the container c13 is referenced by files whose storage time is shorter than the first time length, and the second number of references indicates the number of times the container c13 is referenced by files whose storage time is longer than the second time length.

Specifically, the access reference information of container c13 can be designed as the structure shown in Figure 5, assuming that the storage system is divided into three types: 0-60 days, 60 days-1 year, and 1 year-7 years according to the storage time of the files. Level, the number of references of container c13 is also divided into three items, that is, the number of references of files stored in three time periods of 0-60 days, 60 days-1 year, and 1 year-7 years, and the number of accesses of container c13 is only one item, that is, the number of times the container c13 is accessed within a predetermined period of time.

More specifically, the first time length is set to 60 days, the second time length is set to 1 year, the predetermined time period is set to 30 days, the first threshold is set to 10%, the second threshold is set to 90%, and the third threshold is set to is 100, the first reference count of container c13 is x1, the second reference count is x2, the total reference count is x1+x2+x3, and the visit count is x. If the ratio x1/(x1+x2+x3) of the first number of citations to the total number of citations is less than the first threshold of 10%, or the ratio x2/(x1+x2+x3) of the second number of citations to the total number of citations is greater than the second If the threshold is 90%, or the number of visits x is less than the third threshold 100, the container c13 needs to be migrated to the retention layer Lr0.

It should be noted that although the container access reference information structure shown in FIG. In terms of data access speed at the storage level, but those skilled in the art can understand that the possible container access reference information structure and the method for counting container access reference information in the present invention should not be limited thereto. In fact, users can flexibly set the first time length, second time length, predetermined time period, first threshold, second threshold, and third threshold according to personal preferences and/or actual application scenarios. The reference information is accessed to determine whether the container needs to be migrated to a storage level with a low data access speed.

Example 2

In the case that the access reference information of the container is updated due to the writing, deletion, and/or reading of files in the storage system, the flow chart of the data archiving method is shown in Figure 6, based on the updated access reference information of the container information to determine whether the container needs to migrate to a storage level with high data access speed, where the high data access speed is relative to the data access speed of the storage level where the container is located, and when the In the case of high-speed storage tier migration, the container migration process is completed.

Among them, the so-called situation of updating the access reference information of the container due to the writing, deletion, and/or reading of the file includes: when the file is written into the storage system, the file is first divided into blocks according to a certain algorithm, and divided into multiple Then compare these data blocks with the data blocks already stored in the containers of all storage levels in the storage system to obtain duplicate data blocks and non-duplicate data blocks. Repeated data blocks add one to the number of references to their container, all non-repeated data blocks together form a container, and each data block adds one to the number of references to this container; when a file is deleted in the entire storage system, The referenced container is determined according to the metadata of the file, and the number of references to these containers for each data block of the file is reduced by one; and, when the file is read, the referenced container is determined according to the metadata of the file, the The number of accesses to these containers is incremented by one for each data block of the file.

As shown in Figure 6, the data archiving method according to another embodiment of the present invention mainly includes the following steps:

S610. Based on the updated access reference information of the container, it is judged whether the container needs to be migrated to a storage layer with a high data access speed. Wherein, the high data access speed here is relative to the data access speed of the storage level where the container is located.

S620. When it is determined that the container needs to be migrated to a storage level with high data access speed, obtain a new container at the storage level with high data access speed, and migrate the data blocks stored in the container to the new container , and update the storage address corresponding to the container number of the container to the storage address of the new container.

Specifically, the data archiving method of this embodiment is described in conjunction with the schematic diagram of container migration shown in FIG. 7 . As shown in Figure 7, the data blocks of file 1 are stored in the first container c11 of the active layer La, the data blocks of file 2 are respectively stored in the second container c12 and the third container c13 of the active layer La, and the data of file 3 Chunks are stored in the first container c21, the second container c22, and the third container c23 of the reserved layer Lr0, respectively. Assuming that the first container c21 of the reserved layer Lr0 satisfies the migration condition, it is migrated to the active layer La.

More specifically, the process of migrating the first container c21 of the reserved layer Lr0 to the active layer La is mainly completed through the container mapping table. The structure of the container mapping table is still described in Embodiment 1 as an example. The migration process includes: First apply for a new container c13 in the active layer La, assuming that the initial storage address of the application is 1100; then migrate all the data blocks stored in the container c21 to the new container c13; finally modify the container number of the container c21 in the container The corresponding container storage address in the mapping table, for example, assuming that the container number of the first container c21 of the reserved layer Lr0 is 0001, and the initial storage address of the container is 0100, then the initial storage address corresponding to the container number 0001 is modified to 1100.

In a possible implementation, the migration conditions satisfied by the first container c21 of the retention layer Lr0 include any of the following: the container is referenced by a file with a high activity degree enough times; the container is referenced by a file with a low activity degree The number of references is small enough; the number of times visited in the recent period is enough. Wherein, the activity indicates the storage time of the file in the storage system and/or the read frequency, and the higher the file activity, the shorter the storage time of the file in the storage system and/or the higher the read frequency.

In a possible specific implementation, the migration condition satisfied by the first container c21 of the retention layer Lr0 can be designed as any of the following: the ratio of the third reference times of the container c21 to the total number of references of the container c21 is greater than the first threshold; or the ratio of the fourth number of references of the container c21 to the total number of references of the container c21 is greater than the second threshold; the number of times the container c21 is accessed within a predetermined time period is sufficiently small. Wherein, the third number of references indicates the number of times the container c21 is referenced by files whose storage time is less than the third time length, and the fourth number of references indicates the number of times the container c21 is referenced by files whose storage time is longer than the fourth time length.

Specifically, the access reference information of the container c21 can also be designed as the structure shown in Figure 5. In a possible implementation, the third time length is set to 60 days, the fourth time length is set to 1 year, and the predetermined time The segment is set to 30 days, the fourth threshold is set to 90%, the fifth threshold is set to 10%, the sixth threshold is set to 100, the third citation count of the container is x1, the fourth citation count is x2, the total citation count It is x1+x2+x3, and the number of visits is x. If the ratio x1/(x1+x2+x3) of the third number of citations to the total number of citations is greater than the fourth threshold of 90%, or the ratio x2/(x1+x2+x3) of the fourth number of citations to the total number of citations is less than the fifth If the threshold is 10%, or the number of visits x is greater than the sixth threshold 100, the container c21 needs to migrate to the active layer La.

It should be noted that although the container access reference information structure shown in FIG. In terms of data access speed at the storage level, but those skilled in the art can understand that the possible container access reference information structure and the method for counting container access reference information in the present invention should not be limited thereto. In fact, users can flexibly set the third time length, the fourth time length, the predetermined time period, the fourth threshold, the fifth threshold and the sixth threshold according to personal preferences and/or actual application scenarios. The reference information is accessed to determine whether the container needs to be migrated to a storage level with a high data access speed.

The data archiving method in the embodiment of the present invention judges whether the container needs to be migrated to a storage level with a high data access speed based on the updated access reference information of the container. If the container migrated to a storage level with a low data access speed is frequently accessed and References can be moved back in time, which can improve the user's access speed. In addition, containers at different storage levels can be used as the same deduplication domain. The data block migration process avoids time-consuming operations such as repeated data block queries and rewriting file metadata, and improves the efficiency of data block migration.

Example 3

Fig. 8 shows a flowchart of a data archiving method according to yet another embodiment of the present invention. The same reference numerals in FIG. 8 as those in FIG. 6 have the same functions, and for the sake of brevity, detailed descriptions of these reference numerals are omitted. The main difference between the data archiving method shown in Figure 8 and the data archiving method shown in Figure 6 is that judging whether to migrate to a storage layer with high data access speed is not triggered by the update of the access reference information of the container, but is performed periodically .

In a possible implementation, as shown in FIG. 8 , the step before step S620 is S810, that is, for the container, it is regularly judged based on the access reference information of the container whether the container needs to access data at a speed High storage tier migration.

Specifically, the background scanning process of each storage level periodically scans the access reference information of the containers in this level. The scanning period can be 30 days or 45 days, which can be flexibly set according to personal preferences and/or actual application scenarios , and then it can be judged based on the container migration conditions described in Embodiment 2 of the present invention whether the container meets the migration requirements. If the container meets the migration requirements, the container migration process described in Embodiment 2 of the present invention can be completed. migration. The data access speed of the storage level in the embodiment of the present invention is relative to the data access speed of the storage level where the container is currently located.

In this way, even if the storage system does not set the scanning of the access reference information of the container whose access reference information is updated due to the writing, deletion, and/or reading of files, it can be accessed regularly based on the access of the containers of each storage level. The reference information judges whether the container needs to be migrated to a storage layer with a high data access speed. If the container migrated to a storage layer with a low data access speed is frequently accessed and referenced, it can still be moved back in time, which can improve user access speed.

Example 4

FIG. 9 is a structural block diagram of a data archiving device according to an embodiment of the present invention. The data archiving device is applied to the storage system shown in FIG. 1 . Let me repeat. As shown in FIG. 9 , the data archiving device includes: a migration management module 920 and a migration module 930 .

Wherein, the migration management module 920 is configured to judge whether the first container needs to be migrated to the second storage level based on the access reference information of the first container in the first storage level, wherein the access reference information includes the number of references and the number of visits In at least one of the above, the number of references indicates the number of times the first container is referenced, and the number of accesses indicates the number of times the first container is accessed within a predetermined period of time. The migration module 930 is connected to the migration management module 920, and is configured to determine a second container at the second storage level when the migration management module 920 determines that the container needs to be migrated to the second storage level, and transfer the first The data blocks stored in the container are migrated to the second container, and the container address corresponding to the container number of the first container in the container mapping table is modified to the container address of the second container.

In a possible implementation manner, the data archiving device further includes a file management module 910 connected to the migration management module 920, and the file management module 910 is used to access data at the second storage level lower than the second storage level. In the case of a data access speed of a storage level, it is determined whether the file expires based on the storage time of the file in the storage system. When the file management module 910 determines that the file has expired, the migration management module 920 is configured to use the container that stores the data blocks included in the file as the first container; and the access reference based on the first container information to determine whether the first container needs to be migrated to the second storage level.

Generally, when files are archived and stored, the upper layer software or the user decides the storage time of the files at a certain storage level in the storage system, and the storage time is saved in the file management module 910 or other places in the storage system. Based on this time, the file management module 910 can determine whether the file has expired.

In a possible implementation, the migration management module 920 further includes three judging units 921, 922, 923 for judging whether the first container needs to be migrated to the second storage level, as shown in FIG. 10 , If the judgment result of any judging unit is yes, it is judged that the first container needs to be migrated to the second storage level:

The first judging unit 921 is configured to judge whether the ratio of the first number of references of the first container to the total number of references is less than a first threshold, wherein the first number of references represents the number of times the first file references the first container , the total number of references indicates the total number of times the first container is referenced, and the first file is a file whose storage time is shorter than a first length of time.

The second judging unit 922 is configured to judge whether the ratio of the second number of references of the first container to the total number of references is greater than a second threshold, wherein the second number of references represents the number of times the second file references the first container , the second file is a file whose storage time is shorter than a second time length.

The third judging unit 923 is configured to judge whether the number of visits is less than a third threshold.

The first judging unit 921, the second judging unit 922, and the third judging unit 923 can be realized by three comparators, and the migration management module 920 executes a specific process of judging whether the first container needs to be migrated to the second storage level, wherein, The data access speed of the second storage level is lower than the data access speed of the storage level where the first container is located, and reference may be made to the related description of Embodiment 1.

In a possible implementation, when the data access speed of the second storage tier is higher than the data access speed of the first storage tier, the migration management module 920 is configured to: , delete, and/or read and update the access reference information of any container in the storage system, use the container whose access reference information has been updated as the first container, and update based on the first container After access reference information, determine whether the first container needs to migrate to the second storage level; or use each container in each storage level as the first container, based on the access reference information of the first container , judging whether the first container needs to be migrated to the second storage level.

In a possible implementation, the migration management module 920 further includes three judging units 924, 925, and 926, as shown in FIG. Second storage tier migration:

The fourth judging unit 924 is configured to judge whether the ratio of the third number of references of the first container to the total number of references is greater than a fourth threshold, wherein the third number of references represents the number of times a third file references the first container , the total number of references indicates the total number of times the first container is referenced, and the third file is a file whose storage time is less than a third time length;

The fifth judging unit 925 is configured to judge whether the ratio of the fourth number of references of the first container to the total number of references is less than a fifth threshold, wherein the fourth number of references represents the number of times the fourth file references the first container , the fourth file is a file whose storage time is less than the fourth time length; and

The sixth judging unit 926 is configured to judge whether the number of visits is greater than a sixth threshold.

The fourth judging unit 924, the fifth judging unit 925, and the sixth judging unit 926 can be realized by three comparators. The migration management module 920 performs a specific process of judging whether the container needs to be migrated to the second storage level, wherein the second The data access speed of the storage level is higher than the data access speed of the storage level where the first container is located, and reference may be made to the related description of Embodiment 2.

In this way, the data archiving device in the embodiment of the present invention uses the migration management module 920 to judge whether the container needs to be migrated based on the access reference information. When the migration management module 920 determines that the container needs to be migrated, the migration module 930 can transfer the All data blocks included in the container are migrated to an appropriate storage level. During the data block migration process, each storage level of the storage system is used as the same deduplication domain, which avoids time-consuming operations such as repeated data block queries and rewriting file metadata, and effectively improves Improves the efficiency of data block migration in archiving scenarios.

In addition, if the storage system updates the access reference information of the container due to file writing, deletion, and/or reading, or the storage system regards each container in each of the storage levels as the concerned container, the embodiment of the present invention Based on the updated access reference information of the container or the access reference information of the concerned container, the data archiving device judges whether the container needs to be migrated to a storage layer with high data access speed through the migration management module 920. Containers at the storage level are frequently accessed and referenced, and can be migrated back in time through the migration module 930, which can improve the user's access speed.

Example 5

Fig. 11 is a structural block diagram of a data archiving device according to another embodiment of the present invention. The data archiving device 1100 may be a host server with computing capabilities, a personal computer PC, or a portable portable computer or terminal. The specific embodiments of the present invention do not limit the specific implementation of the computing nodes.

The data archiving device 1100 includes a processor (processor) 1110 , a communication interface (Communications Interface) 1120 , a memory (memory) 1130 and a bus 1140 . Wherein, the processor 1110 , the communication interface 1120 , and the memory 1130 communicate with each other through the bus 1140 .

The communication interface 1120 is used for communicating with network elements, where the network elements include, for example, a virtual machine management center, shared storage, and the like.

The processor 1110 is used to execute programs. The processor 1110 may be a central processing unit CPU, or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

The memory 1130 is used to store files. The memory 1130 may include a high-speed RAM container, and may also include a non-volatile memory (non-volatile memory), such as at least one disk container. Storage 1130 may also be an array of containers. The storage 1130 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

In a possible implementation manner, the above program may be a program code including computer operation instructions. The program can specifically be used to: judge whether the first container needs to be migrated to the second storage level based on the access reference information of the first container in the first storage level, wherein the access reference information includes the number of references and the number of accesses At least one, the number of references indicates the number of times the first container is referenced, and the number of visits indicates the number of times the first container is accessed within a predetermined period of time; when it is determined that the first container needs to be referred to the In the case of migration at the second storage level, determine the second container at the second storage level, migrate the data blocks stored in the first container to the second container, and compare the container mapping table with the The container address corresponding to the container number of the first container is changed to the container address of the second container.

In a possible implementation manner, when the data access speed of the second storage tier is lower than the data access speed of the first storage tier, the access based on the first container in the first storage tier Judging whether the first container needs to be migrated to the second storage level based on reference information includes: judging whether the file expires based on the storage time of the file in the storage system; if it is determined that the file expires, storing A container of data blocks included in the file is used as the first container; and based on the access reference information of the first container, it is judged whether the first container needs to be migrated to the second storage level.

In a possible implementation manner, when the data access speed of the second storage tier is higher than the data access speed of the first storage tier, the access based on the first container in the first storage tier The reference information judging whether the first container needs to migrate to the second storage level includes: when the access reference information of any container in the storage system is updated according to the writing, deletion, and/or reading of files In this case, the container whose access reference information has been updated is used as the first container, and based on the updated access reference information of the first container, it is judged whether the first container needs to be migrated to the second storage level; Each container in the storage level is respectively used as the first container, and based on the access reference information of the first container, it is judged whether the first container needs to migrate to the second storage level.

The data archiving mechanism in the data archiving device according to the above-mentioned embodiments of the present invention is similar to that explained in the data archiving device in Embodiment 4, and those skilled in the art should understand that the aforementioned possible implementation methods can all be applied to this embodiment and The same beneficial effect can be obtained, and will not be repeated here.

Example 6

FIG. 12 is a structural block diagram of a storage system according to an embodiment of the present invention. As shown in FIG. 12 , the storage system 1200 may include: three storage levels 1210 , 1220 , and 1230 , a container mapping table, and a data archiving device 1240 .

Among them, the data access speeds of storage tiers 1210, 1220, and 1230 are different. The data access speed of storage tier 1210 is the highest, storing data within 90 days, and the data access speed of storage tier 1220 is second, storing data from 90 days to 1 year , the storage tier 1230 has the lowest data access speed, and stores data from 1 to 7 years. Deduplication data is stored in various storage levels with different data access speeds according to storage time and access times. The storage levels 1210, 1220, and 1230 each include three containers. The containers refer to logical storage units for storing data blocks, and the data blocks in any two containers are different from each other. The container mapping table is used to record the corresponding relationship between the container number of any one of the containers and the container address, and the container address refers to the address of the container in the storage system.

It should be noted that, in order to more clearly introduce the data archiving method, device, and storage system, this embodiment of the present invention only takes the storage level including three containers as an example, but those skilled in the art can understand that one storage level in an actual storage system It can include containers on the order of hundreds of thousands or even millions of orders.

In a possible implementation manner, the data archiving device 1240 can adopt the structure described in any one of Embodiment 4 or 5 of the present invention, and the data archiving device 1240 is mainly used for the access reference of the first container in the first storage level information to determine whether the first container needs to be migrated to the second storage tier, and if it is determined that the first container needs to be migrated to the second storage tier, determine the second container at the second storage tier, and The data blocks stored in the first container are migrated to the second container, and the container address corresponding to the container number of the first container in the container mapping table is modified to the container address of the second container.

The containers in each storage level of the storage system in the embodiment of the present invention are logical storage units that store data blocks of files after deduplication, so the data blocks in the containers are different from each other, and because the storage system uses a unified deduplication domain Therefore, different from the traditional deduplication domains used for each storage level, the data blocks stored in the containers of each storage level of the storage system provided by the embodiment of the present invention are different from each other. In this way, it can ensure that the data block migration process uses each storage level of the storage system as the same deduplication domain, avoiding time-consuming operations such as repeated data block queries and rewriting file metadata, and effectively improving the efficiency of data block migration in archiving scenarios .

Those skilled in the art can appreciate that each exemplary unit and algorithm steps in the embodiments described herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in the form of hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may choose different methods to implement the described functions for specific applications, but this implementation should not be regarded as exceeding the scope of the present invention.

If the functions are implemented in the form of computer software and sold or used as an independent product, it can be considered that all or part of the technical solution of the present invention (such as the part that contributes to the prior art) is In the form of computer software products. The computer software product is usually stored in a computer-readable non-volatile storage medium, and includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all the methods of the various embodiments of the present invention. or partial steps. The aforementioned storage medium includes various media capable of storing program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk, or an optical disk.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (7)

1. A data archiving method, characterized in that the method is applied to a storage system comprising at least two storage levels with different data access speeds, each storage level includes a container; the container refers to a logical storage for storing data blocks unit, and the data blocks in any two containers are different from each other; the storage system includes a container mapping table, and the container mapping table is used to record the correspondence between the container number and the container address of any one of the containers, so The container address refers to the address of the container in the storage system; the data archiving method includes: Judging whether the first container needs to be migrated to the second storage level based on the access reference information of the first container in the first storage level, wherein the access reference information includes at least one of the number of references and the number of accesses, the The number of references indicates the number of times the first container is referenced, and the number of visits indicates the number of times the first container is accessed within a predetermined period of time; If it is determined that the first container needs to be migrated to the second storage level, determine a second container at the second storage level, and migrate the data blocks stored in the first container to the second container , and modify the container address corresponding to the container number of the first container in the container mapping table to the container address of the second container. 2. The data archiving method according to claim 1, wherein when the data access speed of the second storage level is lower than the data access speed of the first storage level, the first storage based The access reference information of the first container in the hierarchy determines whether the first container needs to be migrated to the second storage hierarchy, including: judging whether the file expires based on the storage time of the file in the storage system; If it is determined that the file expires, the container storing the data blocks included in the file is used as the first container; Based on the access reference information of the first container, it is judged whether the first container needs to be migrated to the second storage level. 3. The data archiving method according to claim 1, wherein when the data access speed of the second storage level is higher than the data access speed of the first storage level, the first storage based The access reference information of the first container in the hierarchy determines whether the first container needs to be migrated to the second storage hierarchy, including: When the access reference information of any container in the storage system is updated according to the writing, deletion, and/or reading of files, the container whose access reference information is updated is used as the first container, based on The updated access reference information of the first container determines whether the first container needs to be migrated to the second storage level; or Each container in each storage level is used as the first container, and based on the access reference information of the first container, it is judged whether the first container needs to migrate to the second storage level. 4. A data archiving device, characterized in that the device is applied to a storage system including at least two storage levels with different data access speeds, each storage level includes a container; the container refers to a logical storage for storing data blocks unit, and the data blocks in any two containers are different from each other; the storage system includes a container mapping table, and the container mapping table is used to record the correspondence between the container number and the container address of any one of the containers, so The container address refers to the address of the container in the storage system; the data archiving device includes: A migration management module, configured to judge whether the first container needs to be migrated to the second storage level based on the access reference information of the first container in the first storage level, wherein the access reference information includes reference times and access times At least one, the number of references indicates the number of times the first container is referenced, and the number of accesses indicates the number of times the first container is accessed within a predetermined period of time; a migration module, connected to the migration management module, configured to determine a second container at the second storage level when the migration management module determines that the first container needs to be migrated to the second storage level, Migrating the data block stored in the first container to the second container, and modifying the container address corresponding to the container number of the first container in the container mapping table to the container address of the second container . 5. The data archiving device according to claim 4, characterized in that, when the data access speed of the second storage level is lower than the data access speed of the first storage level, further comprising The file management module connected to the management module, where: The file management module is used to determine whether the file expires based on the storage time of the file in the storage system, and The migration management module is configured to, when the file management module determines that the file has expired, use the container storing the data blocks included in the file as the first container; and based on the first container access reference information to determine whether the first container needs to be migrated to the second storage level. 6. The data archiving device according to claim 4, wherein when the data access speed of the second storage level is higher than the data access speed of the first storage level, the migration management module is Configured as: When the access reference information of any container in the storage system is updated according to the writing, deletion, and/or reading of files, the container whose access reference information is updated is used as the first container, based on The updated access reference information of the first container determines whether the first container needs to be migrated to the second storage level; or Each container in each storage level is used as the first container, and based on the access reference information of the first container, it is judged whether the first container needs to migrate to the second storage level. 7. A storage system, characterized in that, comprising: At least two storage levels with different data access speeds, each storage level includes a container; the container refers to a logical storage unit for storing data blocks, and the data blocks in any two containers are different from each other; A container mapping table, configured to record the correspondence between the container number of any of the containers and the container address, where the container address refers to the address of the container in the storage system; and A data archiving device according to any one of claims 4 to 6.