WO2020011038A1 - 一种数据迁移方法及装置 - Google Patents

一种数据迁移方法及装置 Download PDF

Info

Publication number
WO2020011038A1
WO2020011038A1 PCT/CN2019/094110 CN2019094110W WO2020011038A1 WO 2020011038 A1 WO2020011038 A1 WO 2020011038A1 CN 2019094110 W CN2019094110 W CN 2019094110W WO 2020011038 A1 WO2020011038 A1 WO 2020011038A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage system
data
bucket
request
access request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/094110
Other languages
English (en)
French (fr)
Inventor
徐丰
张宇
林灵
凌晨
黄磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP19833427.8A priority Critical patent/EP3800538B1/en
Publication of WO2020011038A1 publication Critical patent/WO2020011038A1/zh
Priority to US17/144,999 priority patent/US11914881B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0635—Configuration or reconfiguration of storage systems by changing the path, e.g. traffic rerouting, path reconfiguration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system

Definitions

  • the present application relates to the field of storage technology, and in particular, to a data migration method and device.
  • one solution is a cross-region (region) replication technology, which is a cross-region backup for new services of customers.
  • the principle is that the user creates a target bucket in the target area, and then synchronizes the stock data or incremental data in the source bucket in the source area to the target bucket through configuration.
  • This cross-region replication technology is a copy technology between buckets Requires two buckets to be distributed in different regions, which requires additional storage costs.
  • data synchronization between the two buckets is a data replication technology, not a data migration.
  • Customers need to In order to access data in the target bucket, the customer needs to decide whether to access the source bucket or the target bucket.
  • the other solution is a solution for migrating third-party data.
  • step 1 First, you need to fully migrate the historical data before T1.
  • step 2 Open the Object Storage Service (OSS) image back to the source, and read and write Switch to OSS, no new data is added to the migration source.
  • step 3 Quickly migrate new data between T1 and T2.
  • the operation steps are relatively complicated and require a strong technical background to achieve. For customers, it is also data migration between two buckets.
  • API Application Programming Interface
  • This application provides a data migration method and device, which can solve the problems that affect the normal business of customers during data migration.
  • a data migration method including: a first storage system sends a location update request to a location server, and the location update request is used to instruct the location server to update the location information of the first bucket from the second storage system to the first storage system; A storage system; migrating the data of the first bucket from the second storage system and storing the data in the second bucket of the first storage system, the identifier of the second bucket is the same as the identifier of the first bucket; receiving the data access request, the data The access request is used to access the data in the first bucket; the first storage system determines that the first storage system or the second storage system processes the data access request according to the type of the data access request and the data migration status.
  • the data access request will be Locate to the first storage system for processing. If the first storage system can process the request according to the data that has been migrated, it directly processes the request. If the data accessed by the data access request has not been migrated to the first storage system, the first storage system can The second storage system obtains feedback to the customer. For the customer, the bucket identity remains unchanged, the API remains unchanged, the customer's business department is damaged during the migration process, the customer does not perceive the migration of the data, does not increase the customer's storage cost, and can solve the problem. Issues that affect customers' normal business during data migration.
  • the first storage system migrates the data in the first bucket from the second storage system, including: the first storage system first migrates the metadata of the objects in the first bucket, and then migrates the first bucket to remove the metadata The rest of the data.
  • the metadata of the objects in the bucket has nothing to do with the object size, it is basically a fixed size, and the metadata of the migrated objects is migrated quickly.
  • some data access requests can be processed directly in the second bucket of the first storage system, and there is no need to forward the data access request to the second storage system. For example, the data access request is used to list the buckets. Objects, counting objects, or life cycle management of objects.
  • receiving the data access request by the first storage system includes: the first storage system receives the data access request from the second storage system, or the first storage system receives the data access request sent by the virtual host, and the data access request Include the logo of the first bucket. If the first storage system receives a data access request from the second storage system, it can be understood that the client first sends the data access request to the migrated second storage system, which is a fixed IP access method and sends data access to the second storage system. The request carries the IP address of the second storage system and the identifier of the first bucket. The second storage system learns from the location server that the location information of the first bucket points to the first storage system, and the second storage system sends a data access request to the first storage system. .
  • the data access request is sent directly to the first storage system.
  • the data access request is first sent to the first storage system for processing, so that the first storage system can directly process or save the uploaded data to the local according to the migrated data.
  • the first storage system determines that the first storage system or the second storage system processes the data access request according to the type of the data access request and the data migration status, including: when the type of the data access request is a download type Request, the first storage system determines whether the data requested by the download request has been migrated to the second bucket; when the data requested by the download request has been migrated to the second bucket, the first storage system obtains the download request request from the second bucket The requested data; when the data requested by the download class request is not migrated to the second bucket, and the data requested by the download class request is not metadata, the first storage system obtains the data requested by the download class request from the first bucket and sends it to Client.
  • the first storage system migrates the metadata from the first bucket and saves it to the second bucket, and sends the metadata to Client.
  • the migrated second storage system and the migrated first storage system simultaneously provide external services online.
  • the data access request will be sent to the first storage system for processing.
  • the identity and API of the bucket remain unchanged. Data migration is not perceived by customers and does not increase customer storage costs.
  • the first storage system determines that the first storage system or the second storage system processes the data access request according to the type of the data access request and the data migration status, including: when the type of the data access request is an upload class Request, the first storage system saves the data uploaded by the upload request to the second bucket.
  • a storage system as a first storage system, includes a communication module for sending a location update request to a location server.
  • the location update request is used to instruct the location server to change the location information of the first bucket from
  • the second storage system is updated to be located in the first storage system;
  • the communication module is further configured to migrate the data in the first bucket from the second storage system and store the data in the second bucket of the first storage system.
  • the identifier is the same as the identifier of the first bucket; the communication module is also used to receive a data access request, the data access request is used to access the data in the first bucket; the processing module is used to determine according to the type of the data access request and the migration status of the data
  • the data storage request is processed by the first storage system or the second storage system.
  • the communication module is used to: first migrate the metadata of the objects in the first bucket, and then migrate the remaining data in the first bucket except the metadata.
  • the communication module is configured to receive a data access request from the second storage system or a first storage system to receive a data access request sent by a virtual host, and the data access request includes an identifier of the first bucket.
  • the processing module is configured to: when the type of the data access request is a download request, determine whether the data requested by the download request has been migrated to the second bucket; when the data requested by the download request has been migrated To the second bucket, the data requested by the download class request is obtained from the second bucket; when the data requested by the download class request is not migrated to the second bucket, and the data requested by the download class request is not metadata, the first bucket is obtained from Get the data requested by the download class request and send it to the client; when the data requested by the download class request is not migrated to the second bucket, and the data requested by the download class request is metadata, the first storage system migrates the metadata from the first bucket The data is saved to the second bucket, and the metadata is sent to the client.
  • the processing module is configured to: when the type of the data access request is an upload request, save the data uploaded by the upload request to the second bucket.
  • an embodiment of the present application provides a computer storage medium for storing computer software instructions used by the foregoing first storage system, which includes a program designed to execute the foregoing first aspect.
  • an embodiment of the present application provides a computer program product including instructions, which when executed on a computer, causes the computer to execute the method described in the first aspect above.
  • the embodiments of the present application provide a data migration method and device, which can be applied to a first storage system.
  • the first storage system sends a location update request to a location server.
  • the location update request is used to instruct the location server to delete the location information of the first bucket from the location.
  • the second storage system is updated to be located in the first storage system; the data in the first bucket is migrated from the second storage system, and the data is stored in the second bucket of the first storage system.
  • the identifier of the second bucket is the same as that of the first bucket. Identical identification; Receives a data access request, which is used to access the data in the first bucket; the first storage system determines that the first storage system or the second storage system will process the data access according to the type of data access request and the data migration status request.
  • the data access request will be Locate to the first storage system for processing. If the first storage system can process the request according to the data that has been migrated, it directly processes the request. If the data accessed by the data access request has not been migrated to the first storage system, the first storage system can The second storage system obtains feedback to the customer. For the customer, the bucket identity remains unchanged, the API remains unchanged, the customer's business department is damaged during the migration process, the customer does not perceive the migration of the data, does not increase the customer's storage cost, and can solve the problem. Issues that affect customers' normal business during data migration.
  • FIG. 1 is a schematic diagram of a migration scheme of OssImport + mirror back to source
  • FIG. 2 is a schematic diagram of a network architecture according to an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a first storage system according to an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a data migration method according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of processing a data access request during a data migration process according to an embodiment of the present application
  • FIG. 6 is a schematic diagram of processing a data access request during a data migration process according to an embodiment of the present application
  • FIG. 7 is a schematic structural diagram of a first storage system according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a first storage system according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a first storage system according to an embodiment of the present application.
  • the data migration method provided by the embodiment of the present application can be applied to the scenario where data in the same cloud service provider bucket is migrated in different locations or systems.
  • the client Before and after the migration, the client does not need any changes. After the migration starts, it can be immediately in the new location or The storage system accesses the data in the bucket, and the newly added data is also stored directly to the new location or new storage system.
  • the network architecture of the present application may include a first storage system 20, a second storage system 21, a domain name system (DNS) 22, and a location server 23.
  • the first storage system is a storage system after data migration
  • the second storage system is a storage system before data migration
  • a domain name system is used for domain name resolution
  • a location server is used to query the location of data storage, for example, used to query the buckets where data is stored.
  • the location is on the first storage system or the second storage system.
  • the storage system includes a foreground request processing module and multiple storage areas, and each storage area may be called a bucket.
  • each bucket stores at least one object and metadata corresponding to each object.
  • the storage system may be the first storage system 20 or the second storage system 21 described above.
  • This application is used to solve the problem of data migration in buckets in the field of object storage, including the migration of existing data and new data.
  • the bucket identifier or bucket name
  • the API does not change.
  • the customer's business is not damaged during the migration process, the customer does not perceive the data migration, does not increase the customer's storage cost, does not require the customer to have a strong technical background, and can also specify multiple buckets of concurrent migration at the same time.
  • the basic principle of this application may be: data migration is performed according to the bucket level. When it is determined that data migration is required, new services of the migrated bucket are isolated first, and new services are accessed to the bucket to which the migration is performed. The data remains stable, and then the data of the migrated bucket is gradually migrated to the migrated bucket. After the migration is completed, the data of the migrated bucket is automatically recovered without additional billing. Before, during, and during bucket migration, customers do not need to make any adaptations.
  • An embodiment of the present application provides a data migration method, as shown in FIG. 4, including:
  • a second bucket is created in the first storage system according to the metadata of the first bucket of the migrated second storage system.
  • the identifier of the second bucket and the identifier of the first bucket the same.
  • the customer determines that bucket migration is required, for example, to take the second storage system offline, and data in the second storage system needs to be migrated to the first storage system
  • manual operations can be performed in the first storage system so that the first storage
  • the system begins to migrate buckets.
  • the metadata of the buckets of the second storage system may be obtained from the second storage system first, so as to create corresponding ones in the new area of the first storage system according to the metadata of the buckets of the second storage system.
  • the second storage system also learns that data will be migrated from this system to the first storage system.
  • at least one bucket can be migrated in the second storage system, that is, multiple buckets can be migrated at the same time.
  • This application uses the first bucket in at least one bucket as an example for description.
  • the first storage system when the bucket being migrated by the second storage system includes the first bucket, when the first storage system obtains metadata of the first bucket, the first storage system creates a second bucket based on the metadata of the first bucket in the first storage system.
  • the metadata of the first bucket may include the creation time of the first bucket, the identifier of the first bucket, and the user.
  • the metadata of the second bucket is the same as the metadata of the first bucket.
  • the identifier of the second bucket is the same as that of the first bucket.
  • the identity of a bucket is the same.
  • the first storage system sends a location update request to the location server, and the location update request is used to instruct the location server to update the location information of the first bucket from the second storage system to the first storage system.
  • the first storage system may instruct the location server to modify the location information of the bucket in the second storage system.
  • the location update request may include an identifier of the second storage system to be migrated and an identifier of the first storage system to be migrated to instruct the location server to update the location information of the bucket of the second storage system from the second storage system to the first Storage System.
  • the location server updates the location information of the first bucket from the second storage system to the first storage system.
  • the location server may change the location of the bucket of the second storage system in the location server according to the identifier of the second storage system being migrated and the identifier of the first storage system to be migrated in the location update request.
  • the value of the information field is changed from the identifier of the second storage system to the identifier of the first storage system.
  • the location information field of the first bucket in the second storage system is represented by bkt2
  • the value of this field before modification is the identifier of the second storage system, cluster1, indicating that the location information of the first bucket is the second storage system.
  • the value of the field is the identifier cluster2 of the first storage system, and then the location information of the first bucket is updated by the second storage system to the first storage system.
  • the location server sends a response message to the first storage system to notify that the modification of the location information of the first storage system is completed.
  • the first storage system migrates data in the first bucket from the second storage system.
  • the first storage system may start to obtain data of the bucket in the second storage system from the second storage system.
  • the first storage system obtains the data of the first bucket from the second storage system through the REST interface of the object storage.
  • the REST interface is used for data migration, and the internal architecture and storage model of the system can not be sensed.
  • the timing for the second storage system to start migrating data to the first storage system may be when the second storage system receives an instruction from the first storage system to obtain bucket metadata, if the second storage system has When the data access request has not been processed, the second storage system does not start to migrate data to the first storage system.
  • the second storage system After the second storage system completes processing the data access request being processed, the second storage system starts to the first storage system.
  • the storage system migrates data. For example, the first storage system starts to receive data migrated from the first bucket of the second storage system, and writes the data into the second bucket corresponding to the first storage system. If there is a conflict when data is merged, the first storage system is subject to its own system.
  • the data storage request received by the second storage system after receiving the instruction to acquire the metadata of the bucket will be first sent to the first storage system for processing, which will be specifically described in step 406 below.
  • the metadata of the objects in each bucket may be migrated from the second storage system first, and then the rest of the data in each bucket except the metadata of the object is migrated.
  • the first storage system first migrates the metadata of the object in the first bucket, stores the metadata of the object in the second bucket, and then migrates the remaining data in the first bucket other than the metadata of the object, and stores the remaining data.
  • the metadata of the objects in the bucket has nothing to do with the size of the objects, the metadata of the objects is basically a fixed size and the migration speed is fast.
  • the data access request is used to enumerate objects in a bucket, calculate the number of objects, or manage the life cycle of an object.
  • the first storage system receives a data access request, and the data access request is used to access data in the first bucket.
  • the data access request is to access the data in a fixed Internet Protocol (IP) mode, and in another case, the data access request is sent through a virtual host.
  • IP Internet Protocol
  • the data access request carries the IP address of the second storage system being migrated, the identifier of the bucket to be accessed, and the identifier of the object, that is, the customer still accesses the data using the original IP address. It is said that the bucket name does not change, the API does not change, and data migration is not perceived by the customer.
  • the data access request is first sent to the second storage system, and the second storage system requests the location server to know the location information of the bucket to be accessed. The location information of the bucket has been updated to the first storage system in the location server.
  • the second storage system learns that the bucket location information is the first storage system and obtains the IP address of the first storage system
  • the second storage system An IP address of a storage system sends a data access request to the first storage system.
  • the data access request includes the IP address of the first storage system, the identifier of the bucket to be accessed, and the identifier of the object.
  • the second storage system may also cache the updated location information of the bucket when accessing the bucket for the first time, and cache it in the cache. When not expired, directly send a data access request to the first storage system to obtain data to be accessed from the first storage system.
  • the second data storage system culster1 receives the first data access request sent by the client to be accessed in a fixed IP manner, and the Uniform Resource Locator (URL) corresponding to the first data access request is http: // 8.40.229.12/bkt2/obj1, where 8.40.229.12 is the fixed IP address of the second storage system, bkt2 is the identifier of the first bucket, obj1 is the identifier of the object, and when the second storage system receives the first data access request, Ask the location server for the location information of the first bucket.
  • URL Uniform Resource Locator
  • the second storage system learns that the location information of the first bucket bkt2 points to the first storage system culster2, and obtains the first storage
  • the fixed IP address of system culster2 is 8.40.229.32
  • the second storage system sends a second data access request to the first storage system, and the second data access request is the data access request in step 405, and carries the data storage request of the first storage system.
  • a data access request is sent to the DNS.
  • the URL corresponding to the data access request carries the identifier of the bucket to be accessed and the identifier of the object.
  • the DNS stores the identifier of each storage system and the corresponding IP address. After the DNS resolves the URL to obtain the bucket identifier, it requests the location server to obtain the bucket's location information. After the location server obtains the identifier of the first storage system corresponding to the bucket according to the bucket identifier, it feeds back the DNS to the identifier of the first storage system.
  • the DNS obtains the IP address corresponding to the first storage system according to the identifier of the first storage system and sends it back to the customer. The customer sends a data access request to the first storage system according to the IP address corresponding to the first storage system. It includes the ID of the bucket to be accessed and the ID of the object.
  • the second storage system is the system being migrated
  • the first storage system is the system after data migration.
  • the corresponding URL is http: // bkt2. hwsclouds.com/obj1
  • the URL is first sent to the DNS.
  • the DNS stores the identifier culster1 of the second storage system and its corresponding IP address 8.40.229.12, and the identifier culster2 of the first storage system and its corresponding IP address 8.40.
  • the DNS resolves the URL to obtain the identifier bkt2 of the first bucket to be accessed, and requests the location server to obtain the location information of the first bucket according to the identifier bkt2 of the first bucket. If the location information of the first bucket after the location update points to the For a storage system, the DNS obtains the IP address corresponding to the identity of the first storage system culster2 from the local and returns it to the customer. The customer sends a data access request to the first storage system according to the fixed IP address 8.402.229.32 of the first storage system culster2. The access request includes the identifier bkt2 of the first bucket to be accessed and the identifier obj1 of the object.
  • the first storage system determines that the first storage system or the second storage system processes the data access request according to the type of the data access request and the data migration status.
  • step 406 it can be known from step 406 that no matter which way the customer accesses, the data access request will eventually be sent to the first storage system for processing. Because the first storage system is still in the process of data migration, that is, before the data is migrated, the first storage system cannot process the access request for the data that is not migrated. Therefore, the first storage system needs the second storage system to assist in processing. During the migration process, the processing of the data access request by the first storage system needs to be determined by the first storage system or the second storage system to assist in processing the data access request according to the type of the data access request and the data migration status.
  • the first storage system determines whether the data requested by the download request has been migrated to the second of the first storage system. Bucket; when the data requested by the download request has been migrated to the second bucket, the first storage system obtains the data requested by the download request from the second bucket; when the data requested by the download request is not migrated to the second bucket, and The data requested by the download request is not metadata. The first storage system obtains the data requested by the download request from the first bucket and sends it to the client, and then saves the data requested by the download request to the second bucket.
  • the first storage system may obtain the requested data from the second bucket of the second storage system and store it in the first storage system. Memory, feedback the requested data to the client, and then migrate the data stored in the memory to the first bucket in the background, so for the client, the delay in requesting data to get feedback is small; when the download class requests The requested data is not migrated to the second bucket, and the data requested by the download request is metadata.
  • the first storage system migrates the metadata from the first bucket and saves it to the second bucket, sends the metadata to the client, and then backstage Migrate data other than metadata from the first bucket to the second bucket. Because the metadata of the objects in the bucket has nothing to do with the object size, it is basically a fixed size and the migration speed is fast. Therefore, when the client requests metadata, the metadata can be migrated to the second bucket first, and then the metadata is fed back to the client.
  • the first storage system saves the data uploaded by the upload request to the second bucket.
  • step 406 Combining the different access methods in step 406 and the type of data access request in step 407, if the client accesses by a fixed IP method, the transfer of the data access request between the two storage systems can be as shown in Figure 5.
  • the data access request can be passed between the two storage systems as shown in Figure 6.
  • the first bucket in the second storage system and the second bucket in the first storage system simultaneously provide external services online.
  • the first bucket Line no longer provides business. Therefore, in the data migration method of this application, even if the data is still being migrated, the data access request will be transmitted to the new storage system for processing. If the new storage system can process it, it will be processed directly, such as accessing the migrated Data or upload new data. If the new storage system cannot handle it, get data from the old storage system and feed it back to the client. Before and after the data migration, the client does not need to change anything, does not increase customer storage costs, and does not require the customer to have a strong Technical background, can solve the problems that affect the normal business of customers during data migration.
  • each network element such as the first storage system, includes a hardware structure and / or a software module corresponding to each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein. Whether a certain function is performed by hardware or computer software-driven hardware depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
  • the first storage system may be divided into functional modules according to the foregoing method example.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a logical function division. In actual implementation, there may be another division manner.
  • FIG. 7 shows a possible structural schematic diagram of the storage system involved in the foregoing embodiment.
  • the storage system serves as the first storage system.
  • the storage system 70 includes : Sending unit 7001, receiving unit 7002, and processing unit 7003.
  • the processing unit 7003 is configured to support the first storage system 70 to execute the process 401 in FIG. 4, and the sending unit 7001 is configured to support the first storage system to execute the process 402 in FIG. 4; the receiving unit 7002 is configured to execute the process 405 in FIG. 4 and 406.
  • All relevant content of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.
  • FIG. 8 shows a possible structure diagram of the storage system involved in the foregoing embodiment, and the storage system serves as the foregoing first storage system.
  • the storage system 80 includes a processing module 8002 and a communication module 8003.
  • the processing module 8002 is configured to control and manage actions of the first storage system.
  • the communication module 8003 is used to send a location update request to the location server.
  • the location update request is used to instruct the location server to update the location information of the first bucket from the second storage system to the first storage system.
  • the communication module 8003 also uses For migrating the data in the first bucket from the second storage system and storing the data in the second bucket of the first storage system, the identifier of the second bucket is the same as the identifier of the first bucket; the communication module 8003 is also used for receiving A data access request.
  • the data access request is used to access the data in the first bucket.
  • the processing module 8002 is configured to determine that the first storage system or the second storage system processes the data access request according to the type of the data access request and the data migration status.
  • the communication module 8003 may be used to: first migrate the metadata of the objects in the first bucket, and then migrate the remaining data except the metadata in the first bucket.
  • the communication module 8003 may be configured to receive a data access request from a second storage system or a first storage system to receive a data access request sent by a virtual host, where the data access request includes the first bucket.
  • logo a data access request from a second storage system or a first storage system to receive a data access request sent by a virtual host, where the data access request includes the first bucket.
  • the processing module 8002 may be configured to: when the type of the data access request is a download request, determine whether the data requested by the download request has been migrated to the second bucket; when the download request The requested data has been migrated to the second bucket, and the data requested by the download category request is obtained from the second bucket; when the data requested by the download category request is not migrated to the second bucket, and the data requested by the download category request is not metadata
  • the first storage system obtains the data requested by the download class request from the first bucket and sends it to the client; when the data requested by the download class request is not migrated to the second bucket, and the data requested by the download class request is metadata, the first A storage system migrates metadata from the first bucket and saves it to the second bucket, and sends the metadata to the client.
  • the processing module 8002 may be configured to: when the type of the data access request is an upload request, save the data uploaded by the upload request to the second bucket.
  • the processing module 8002 is configured to support the first storage system to execute the processes 401 and 407 in FIG. 4, and / or other processes used in the technology described herein.
  • the communication module 8003 is configured to support communication between the first storage system and other network entities, for example, communication with the functional module or network entity shown in FIG. 2.
  • the communication module 8003 may be used to support the first storage system to execute the operations shown in FIG. 4. Processes 402, 405, and 406.
  • the first storage system may further include a storage module 8001 for storing program code and data of the first storage system.
  • the processing module 8002 may be a processor or a controller.
  • the processing module 8002 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), and an application-specific integrated circuit (Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application.
  • the processor may also be a combination that realizes computing functions, for example, a combination including one or more microprocessors, a combination of a DSP and a microprocessor, and so on.
  • the communication module 8003 may be a transceiver, a transceiver circuit, or a communication interface.
  • the storage module 8001 may be a memory.
  • the processing module 8002 is a processor
  • the communication module 8003 is a communication interface
  • the storage module 8001 is a memory
  • the first storage system involved in the embodiment of the present application may be the storage system shown in FIG. 9.
  • the storage system 9010 includes a processor 9012, a communication interface 9013, a memory 9011, and a bus 9014.
  • the communication interface 9013, the processor 9012, and the memory 9011 are connected to each other through a bus 9014.
  • the bus 9014 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Wait.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • Wait The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only a thick line is used in FIG. 9, but it does not mean that there is only one bus or one type of bus.
  • the steps of the method or algorithm described in combination with the disclosure of this application may be implemented in a hardware manner, or may be implemented in a manner that a processor executes software instructions.
  • Software instructions can be composed of corresponding software modules.
  • Software modules can be stored in Random Access Memory (RAM), flash memory, Read Only Memory (ROM), and erasable programmable read-only memory (ROM). Erasable (Programmable ROM, EPROM), electrically erasable programmable read-only memory (EPROM), registers, hard disks, mobile hard disks, read-only optical disks (CD-ROMs), or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium.
  • the storage medium may also be an integral part of the processor.
  • the processor and the storage medium may reside in an ASIC.
  • the ASIC can be located in a core network interface device.
  • the processor and the storage medium can also exist as discrete components in the core network interface device.
  • Computer-readable media includes computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
  • a storage media may be any available media that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

提供了一种数据迁移方法及装置，涉及存储技术领域，能够解决在数据迁移时影响客户正常业务的问题。上述方法为：第一存储系统向位置服务器发送位置更新请求，位置更新请求用于指示位置服务器将第一桶的位置信息由位于第二存储系统更新为第一存储系统（402）；第一存储系统从第二存储系统迁移第一桶中的数据（405）；第一存储系统接收数据访问请求，数据访问请求用于访问第一桶中的数据（406）；第一存储系统根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求（407）。上述方法用于存储系统间桶内数据的迁移。

Description

一种数据迁移方法及装置 技术领域
本申请涉及存储技术领域，尤其涉及一种数据迁移方法及装置。
背景技术
随着对象存储业务更新换代，从客户的角度希望使用性能更快，价格更合理的新存储系统，从企业管理的角度也希望新存储系统能够收编老存储系统，老存储系统下线，以减少维护系统的管理成本。因此，新老系统间的数据迁移是对象存储业务发展的必然诉求。
目前，一种方案为跨区域(region)复制技术，是针对客户新增业务的跨区域备份。其原理为：用户在目标区域创建一个目标桶，然后通过配置将源区域的源桶中的存量数据或增量数据同步到目标桶，这种跨区域复制技术是一种桶之间的复制技术，要求两个桶分布于不同区域，需要花费额外的存储成本，对客户来说，这两个桶之间的数据同步，是一种数据复制技术，并非数据迁移，客户需要在数据复制完成后才能在目标桶访问数据，且需要客户决定访问源桶还是目标桶。另一种方案为通过迁移第三方数据的解决方案，这种方案包括两种技术，存量数据的导入和增量数据的回源，需要组合使用才能完成迁移。例如OssImport+镜像回源迁移方案中，如图1所示，步骤1：首先需要全量迁移T1之前的历史数据，步骤2：打开对象存储系统(Object Storage Service，OSS)镜像回源，并将读写切换到OSS，迁移源不再新增数据，步骤3：快速迁移T1至T2间的新增数据。这种方案中，操作步骤相对复杂，需要很强的技术背景才能实现，对客户来说，也是两个桶之间的数据迁移，客户需要修改应用程序编程接口(Application Programming Interface，API)来访问目标桶，需要花费额外的存储成本，且需要客户手动保证迁移桶之间的属性一致，迁移历史数据时新增数据还是会上传至源桶，需要再次迁移。因此，如何在迁移数据的同时不影响客户的正常业务，当前业界并没有相应的技术。
发明内容
本申请提供一种数据迁移方法及装置，能够解决在数据迁移时影响客户正常业务的问题。
第一方面，提供一种数据迁移方法，包括：第一存储系统向位置服务器发送位置更新请求，位置更新请求用于指示位置服务器将第一桶的位置信息由位于第二存储系统更新为位于第一存储系统；从第二存储系统迁移第一桶的数据，并将数据存储于第一存储系统的第二桶中，第二桶的标识与第一桶的标识相同；接收数据访问请求，数据访问请求用于访问第一桶中的数据；第一存储系统根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求。这样一来，由于被迁移的第二存储系统中桶的标识与迁移至的第一存储系统中桶的标识相同，位置服务器中更新了被迁移的桶的位置信息，那么数据访问请求就会被定位给第一存储系统进行处理，第一存储系统如果根据已经迁移的数据可以处理该请求则直接进行处理，如果数据访问请求访问的数据还未迁移至第一存储系统，第一存储系统可以从第二存储系统获取反馈给客户，对于客户来说，桶的标识不变，API不变，迁移过程中客户的业务部受损，客户不感知数据的迁移，不增加客户的存储成本，能够解决在数据迁移时影响客户正常业务的问题。
在一种可能的设计中，第一存储系统从第二存储系统迁移第一桶中的数据，包括：第一存储系统先迁移第一桶中对象的元数据，再迁移第一桶中除元数据以外的其余数据。这样一来，由于桶内对象的元数据和对象大小无关，基本是固定大小，迁移对象的元数据的迁移速度快。对象的元数据迁移完成后，一些数据访问请求就可以直接在第一存储系统的第二桶中直接处理，不需要将数据访问请求转发给第二存储系统，例如数据访问请求用于列举桶内对象、计算对象数或对象的生命周期管理等。
在一种可能的设计中，第一存储系统接收数据访问请求，包括：第一存储系统从第二存储系统接收数据访问请求，或第一存储系统接收虚拟主机发送的数据访问请求，数据访问请求包括第一桶的标识。如果第一存储系统从第二存储系统接收数据访问请求，可以理解为客户端先将数据访问请求发送给被迁移的第二存储系统，为固定IP访问方式，发送给第二存储系统的数据访问请求携带第二存储系统的IP地址以及第一桶的标识，第二存储系统从位置服务器获知第一桶的位置信息指向第一存储系统，第二存储系统再向第一存储系统发送数据访问请求。若客户以虚拟主机发送数据访问请求，则该数据访问请求直接发送给第一存储系统。这样，无论两种方式中哪种访问方式，数据访问请求都先发送给第一存储系统进行处理，以便于第一存储系统根据已迁移的数据可以直接处理或者保存上传的数据至本地。
在一种可能的设计中，第一存储系统根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求，包括：当数据访问请求的类型为下载类请求，第一存储系统确定下载类请求所请求的数据是否已迁移至第二桶；当下载类请求所请求的数据已迁移至第二桶，第一存储系统从第二桶获取下载类请求所请求的数据；当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据不为元数据，第一存储系统从第一桶获取下载类请求所请求的数据发送给客户端。当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据为元数据，第一存储系统从第一桶迁移元数据并保存至第二桶，将元数据发送给客户端。这样一来，数据迁移过程中，被迁移的第二存储系统和迁移至的第一存储系统同时在线对外提供服务，期间的数据访问请求会先发送给第一存储系统进行处理，对于客户来说，桶的标识和API不变，数据迁移对客户不感知，不增加客户存储成本。
在一种可能的设计中，第一存储系统根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求，包括：当数据访问请求的类型为上传类请求，第一存储系统将上传类请求所上传的数据保存至第二桶。
第二方面，提供一种存储系统，该存储系统作为第一存储系统，包括：通信模块，用于向位置服务器发送位置更新请求，位置更新请求用于指示位置服务器将第一桶的位置信息由第二存储系统更新为位于第一存储系统；通信模块，还用于从第二存储系统迁移第一桶中的数据，并将数据存储于第一存储系统的第二桶中，第二桶的标识与第一桶的标识相同；通信模块，还用于接收数据访问请求，数据访问请求用于访问第一桶中的数据；处理模块，用于根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求。
在一种可能的设计中，通信模块用于：先迁移第一桶中对象的元数据，再迁移第一桶 中除元数据以外的其余数据。
在一种可能的设计中，通信模块用于：从第二存储系统接收数据访问请求，或第一存储系统接收虚拟主机发送的数据访问请求，数据访问请求包括第一桶的标识。
在一种可能的设计中，处理模块用于：当数据访问请求的类型为下载类请求，确定下载类请求所请求的数据是否已迁移至第二桶；当下载类请求所请求的数据已迁移至第二桶，从第二桶获取下载类请求所请求的数据；当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据不为元数据，从第一桶获取下载类请求所请求的数据发送给客户端；当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据为元数据，第一存储系统从第一桶迁移元数据并保存至第二桶，将元数据发送给客户端。
在一种可能的设计中，处理模块用于：当数据访问请求的类型为上传类请求，将上传类请求所上传的数据保存至第二桶。
又一方面，本申请实施例提供了一种计算机存储介质，用于储存为上述第一存储系统所用的计算机软件指令，其包含用于执行上述第一方面所设计的程序。
又一方面，本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面所述的方法。
本申请实施例提供一种数据迁移方法及装置，可以应用于第一存储系统，第一存储系统向位置服务器发送位置更新请求，位置更新请求用于指示位置服务器将第一桶的位置信息由位于第二存储系统更新为位于第一存储系统；从第二存储系统迁移第一桶中的数据，并将数据存储于第一存储系统的第二桶中，第二桶的标识与第一桶的标识相同；接收数据访问请求，数据访问请求用于访问第一桶中的数据；第一存储系统根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求。这样一来，由于被迁移的第二存储系统中桶的标识与迁移至的第一存储系统中桶的标识相同，位置服务器中更新了被迁移的桶的位置信息，那么数据访问请求就会被定位给第一存储系统进行处理，第一存储系统如果根据已经迁移的数据可以处理该请求则直接进行处理，如果数据访问请求访问的数据还未迁移至第一存储系统，第一存储系统可以从第二存储系统获取反馈给客户，对于客户来说，桶的标识不变，API不变，迁移过程中客户的业务部受损，客户不感知数据的迁移，不增加客户的存储成本，能够解决在数据迁移时影响客户正常业务的问题。
附图说明
图1为一种OssImport+镜像回源的迁移方案的示意图；
图2为本申请实施例提供的一种网络架构的示意图；
图3为本申请实施例提供的一种第一存储系统的结构示意图；
图4为本申请实施例提供的一种数据迁移方法的流程示意图；
图5为本申请实施例提供的一种数据迁移过程中数据访问请求处理示意图；
图6为本申请实施例提供的一种数据迁移过程中数据访问请求处理示意图；
图7为本申请实施例提供的一种第一存储系统的结构示意图；
图8为本申请实施例提供的一种第一存储系统的结构示意图；
图9为本申请实施例提供的一种第一存储系统的结构示意图。
具体实施方式
本申请实施例提供的数据迁移方法可以应用于同一云服务商桶内数据在不同位置或系统间迁移的场景，迁移前后，客户端不需要任何改变，开始迁移后，可以立即在新的位置或存储系统访问桶内数据，新增数据也直接存储到新的位置或新的存储系统。
如图2所示，本申请的网络架构可以包括第一存储系统20、第二存储系统21、域名系统(Domain Name System，DNS)22以及位置服务器(location server)23。第一存储系统为数据迁移后的存储系统，第二存储系统为数据迁移前的存储系统，域名系统用于域名解析，位置服务器用于查询数据存储的位置，例如用于查询存放数据的桶的位置在第一存储系统或第二存储系统。
如图3所示，在存储系统中，包括前台请求处理模块以及多个存储区域，每个存储区域可以称为桶。在一种存储模型中，每个桶中存储有至少一个对象以及每个对象对应的元数据。该存储系统可以为上述第一存储系统20或第二存储系统21。
本申请用于解决对象存储领域桶内数据迁移问题，包括存量数据和新增数据的迁移。迁移完成只是桶和桶内数据的位置信息发生变化，对于客户来说桶的标识(或称为桶名)不变，API不变。迁移过程中客户的业务不受损，客户不感知数据的迁移，不增加客户的存储成本，不需要客户有很强的技术背景，也可以同时指定多桶并发迁移。
本申请的基本原理可以为：按照桶的级别进行数据迁移，在确定需进行数据迁移时，先将被迁移桶的新业务进行隔离，让新业务接入迁移至的桶，这样被迁移桶的数据保持稳定，而后再逐步将被迁移桶的数据迁移到迁移至的桶，迁移完成后被迁移桶的数据自动回收，不额外计费。桶迁移前后和迁移过程中，客户不需要进行任何适配。
下面对本申请实施例进行阐述。
本申请实施例提供一种数据迁移方法，如图4所示，包括：
401、第一存储系统确定要进行桶迁移时，根据被迁移的第二存储系统的第一桶的元数据在第一存储系统中创建第二桶，第二桶的标识与第一桶的标识相同。
当客户确定需要进行桶迁移，例如要将第二存储系统下线，需要将第二存储系统中的数据迁移至第一存储系统时，可以在第一存储系统中进行人工操作，以便第一存储系统开始进行桶的迁移。第一存储系统确定开始桶迁移时，可以先从第二存储系统获取第二存储系统的桶的元数据，以根据第二存储系统的桶的元数据在第一存储系统新的区域创建对应的新桶，第二存储系统也获知到将要从本系统迁移数据至第一存储系统。这里第二存储系统中被迁移的桶可以为至少一个，也就是可以同时多桶迁移，本申请以至少一个桶中的第一桶为例进行说明。
例如第二存储系统被迁移的桶包括第一桶时，第一存储系统在第二存储系统获取到第一桶的元数据时，在第一存储系统中根据第一桶的元数据创建第二桶，该第一桶的元数据可以包括第一桶的创建时间、第一桶的标识以及用户等，第二桶的元数据与第一桶的元数据相同，当然第二桶的标识与第一桶的标识也相同。
402、第一存储系统向位置服务器发送位置更新请求，位置更新请求用于指示位置服务器将第一桶的位置信息由位于第二存储系统更新为位于第一存储系统。
在第一存储系统新创桶完成后，第一存储系统可以指示位置服务器对第二存储系统中 桶的位置信息进行修改。该位置更新请求可以包括被迁移的第二存储系统的标识以及将要迁移至的第一存储系统的标识，以指示位置服务器将第二存储系统的桶的位置信息由第二存储系统更新为第一存储系统。
403、位置服务器将第一桶的位置信息由第二存储系统更新为第一存储系统。
在一种可能的实现中，位置服务器可以按照位置更新请求中被迁移的第二存储系统的标识以及将要迁移至的第一存储系统的标识，在位置服务器中将第二存储系统的桶的位置信息字段的值由第二存储系统的标识修改为第一存储系统的标识。例如第二存储系统中第一桶的位置信息字段以bkt2表示时，修改前该字段的值为第二存储系统的标识cluster1，表示第一桶的位置信息为第二存储系统，如果修改后该字段的值为第一存储系统的标识cluster2，那么第一桶的位置信息就由第二存储系统更新为了第一存储系统。
404、位置服务器向第一存储系统发送响应信息，以通知第一存储系统位置信息修改完成。
405、第一存储系统从第二存储系统迁移第一桶中的数据。
第一存储系统在获知位置服务器修改第一桶的位置信息完成后，可以开始从第二存储系统获取第二存储系统中桶的数据。例如第一存储系统通过对象存储的REST接口从第二存储系统获取第一桶的数据。这里利用REST接口进行数据迁移，可以不感知系统内部的架构和存储模型。
需要说明的是，第二存储系统开始向第一存储系统迁移数据的时机可以是第二存储系统在接收到第一存储系统获取桶的元数据的指令时，若第二存储系统有正在处理但还未处理完的数据访问请求时，第二存储系统先不开始向第一存储系统迁移数据，等到第二存储系统将该正在处理的数据访问请求处理完成后，第二存储系统开始向第一存储系统迁移数据，例如第一存储系统开始接收从第二存储系统的第一桶迁移的数据，写入第一存储系统与第一桶对应的第二桶中。如果数据合并时有冲突，第一存储系统以自身系统为准。第二存储系统在接收到获取桶的元数据的指令之后接收到的数据访问请求会被首先发送给第一存储系统进行处理，下面在步骤406中进行具体说明。
其中，第一存储系统在从第二存储系统迁移数据时，可以先从第二存储系统迁移各个桶中对象的元数据，再迁移各个桶中除对象的元数据以外的其余数据。例如第一存储系统先迁移第一桶中对象的元数据，将该对象的元数据存储到第二桶中，再迁移第一桶中除对象的元数据以外的其余数据，并将该其余数据存储到第二桶中。这样一来，由于桶中对象的元数据和对象大小无关，对象的元数据基本是固定大小，迁移速度快，这样对象的元数据在迁移完成后，一些访问对象的元数据的数据访问请求就可以直接在第一存储系统中处理，不需要将数据访问请求转发给第二存储系统。例如该数据访问请求用于列举桶内对象、计算对象数或对象的生命周期管理等。
406、第一存储系统接收数据访问请求，数据访问请求用于访问第一桶中的数据。
这里分两种情况进行说明，一种情况下，该数据访问请求是以固定网际协议(Internet Protocol，IP)方式访问数据的，另一种情况下，该数据访问请求是通过虚拟主机发送的。
如果以固定IP方式访问数据，该数据访问请求中携带被迁移的第二存储系统的IP地址、要访问的桶的标识以及对象的标识，即客户还是以原先的IP地址访问数据，对客户来 说桶名不变，API不变，数据迁移对客户不感知，这样一来，数据访问请求先发送到第二存储系统，第二存储系统向位置服务器请求获知要访问的桶的位置信息，由于位置服务器中已经将桶的位置信息更新为第一存储系统，第二存储系统在得知桶的位置信息为第一存储系统并获取到第一存储系统的IP地址时，第二存储系统根据第一存储系统的IP地址向第一存储系统发送数据访问请求，此时的数据访问请求中包括第一存储系统的IP地址、要访问的桶的标识以及对象的标识。这里需要说明的是，如果对桶更新位置信息后，该桶不是第一次被访问，第二存储系统中也可以在第一次访问该桶时缓存该桶更新后的位置信息，并在缓存未过期时，直接向第一存储系统发送数据访问请求，以从第一存储系统获取要访问的数据。
举例来说，第二存储系统culster1接收到客户发送的第一数据访问请求是以固定IP方式访问，该第一数据访问请求对应的统一资源定位符(Uniform Resource Locator，URL)为http://8.40.229.12/bkt2/obj1,其中8.40.229.12为第二存储系统的固定IP地址，bkt2为要第一桶的标识，obj1为对象的标识，第二存储系统接收到第一数据访问请求时，向位置服务器询问第一桶的位置信息，如果位置服务器中存在字段bkt1：culster1,bkt2：culster2,第二存储系统得知第一桶bkt2的位置信息指向第一存储系统culster2,并获取第一存储系统culster2的固定IP地址8.40.229.32，那么第二存储系统向第一存储系统发送第二数据访问请求，该第二数据访问请求即为步骤405中的数据访问请求，并携带第一存储系统的固定IP地址、第一桶的标识bkt2以及对象的标识obj1。
如果以虚拟主机访问数据，数据访问请求发送至DNS，该数据访问请求对应的URL中携带有要访问的桶的标识以及对象的标识，DNS中保存有每个存储系统的标识以及对应的IP地址，DNS解析URL得到桶的标识后，向位置服务器请求获取该桶的位置信息，位置服务器根据桶的标识得到桶对应的第一存储系统的标识后，向DNS反馈该第一存储系统的标识，DNS便根据第一存储系统的标识获取第一存储系统对应的IP地址并反馈给客户，客户根据第一存储系统对应的IP地址向第一存储系统发送数据访问请求，此时的数据访问请求中包括有要访问的桶的标识以及对象的标识。
举例来说，第二存储系统为被迁移系统，第一存储系统为数据迁移后的系统，如果通过虚拟主机要访问第一桶bkt2中的对象obj1，其对应的URL为http://bkt2.hwsclouds.com/obj1,该URL首先发送至DNS，DNS中保存有第二存储系统的标识culster1及其对应的IP地址8.40.229.12，和第一存储系统的标识culster2及其对应的IP地址8.40.229.32，DNS解析该URL得到要访问的第一桶的标识bkt2，并根据第一桶的标识bkt2向位置服务器请求获取该第一桶的位置信息，如果位置更新后第一桶的位置信息指向第一存储系统，那么DNS从本地获取第一存储系统culster2的标识对应的IP地址并反馈给客户，客户根据第一存储系统culster2的固定IP地址8.40.229.32向第一存储系统发送数据访问请求，数据访问请求中包括要访问的第一桶的标识bkt2以及对象的标识obj1。
这样看来，桶迁移前后和迁移过程中，客户不需要进行任何适配。在开启迁移后，如果客户以虚拟主机方式访问桶，访问请求会直接定位到数据迁移后的桶，如果客户以固定IP访问被迁移的桶，被迁移的桶也会将访问请求转发给数据迁移后的桶进行处理。
407、第一存储系统根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求。
由步骤406可知，无论客户以哪种方式访问，数据访问请求会最终先发送给第一存储系统进行处理。由于还处于数据迁移过程中，即数据还未迁移完时，第一存储系统还不能处理对未迁移的数据的访问请求，因此，第一存储系统需要第二存储系统协助处理，这样，在数据迁移过程中，第一存储系统处理数据访问请求需要根据数据访问请求的类型和数据迁移状况决定由第一存储系统处理还是由第二存储系统协助处理数据访问请求。
具体来说，以客户访问第一桶中的数据为例，当数据访问请求的类型为下载类请求，第一存储系统确定下载类请求所请求的数据是否已迁移至第一存储系统的第二桶；当下载类请求所请求的数据已迁移至第二桶，第一存储系统从第二桶获取下载类请求所请求的数据；当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据不为元数据，第一存储系统从第一桶获取下载类请求所请求的数据发送给客户端，再将下载类请求所请求的数据保存至第二桶。具体的，若下载类请求所请求的数据不为元数据且还未迁移至第二桶中，第一存储系统可以从第二存储系统的第二桶获取所请求的数据存入第一存储系统内存，将所请求的数据反馈给客户端，后台再将内存中存储的数据迁移至第一桶中，这样对于客户端来说，其请求数据得到反馈的时延较小；当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据为元数据，第一存储系统从第一桶迁移元数据并保存至第二桶，将元数据发送给客户端，后台再从第一桶迁移元数据以外的数据至第二桶。由于桶中对象的元数据和对象大小无关，基本为固定大小，迁移速度快，因此客户端请求元数据时，可以将元数据先迁移至第二桶中，再将元数据反馈给客户端。
当数据访问请求的类型为上传类请求，则第一存储系统将上传类请求所上传的数据保存至第二桶。
当第二存储系统中第一桶中的数据迁移完至第二桶中时，所有后续接收到的用于访问第一桶的数据访问请求就只在第一存储系统中进行处理，第一桶就可以下线。当第二存储系统各桶的数据都迁移至第一存储系统时，第二存储系统就可以下线。
结合步骤406中访问方式的不同以及步骤407中数据访问请求的类型，如果客户端以固定IP方式访问时，数据访问请求在两个存储系统间的传递可以如图5所示，如果客户端以虚拟主机访问时，数据访问请求在两个存储系统间的传递可以如图6所示。
通过图5和图6所示，在数据迁移未完成时，第二存储系统中的第一桶和第一存储系统中的第二桶同时在线对外提供服务，当迁移完成后，第一桶下线不再提供业务。因此，本申请数据迁移方法中，即使数据还在迁移过程中，数据访问请求会先被传送给新的存储系统进行处理，如果新的存储系统可以处理时则直接进行处理，例如访问已迁移的数据或上传新数据，如果新的存储系统不能处理则从老的存储系统获取数据反馈给客户端，数据迁移前后，客户端不需要任何改变，不增加客户存储成本，不需要客户有很强的技术背景，能够解决在数据迁移时影响客户正常业务的问题。
上述主要从各个网元之间交互的角度对本申请实施例提供的方案进行了介绍。可以理解的是，各个网元，例如第一存储系统为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例 描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对第一存储系统进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图7示出了上述实施例中所涉及的存储系统的一种可能的结构示意图，该存储系统作为上述第一存储系统，该存储系统70包括：发送单元7001，接收单元7002以及处理单元7003。处理单元7003用于支持第一存储系统70执行图4中的过程401，发送单元7001用于支持第一存储系统执行图4中的过程402；接收单元7002用于执行图4中的过程405以及406。其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
在采用集成的单元的情况下，图8示出了上述实施例中所涉及的存储系统的一种可能的结构示意图，该存储系统作为上述第一存储系统。该存储系统80包括：处理模块8002和通信模块8003。处理模块8002用于对第一存储系统的动作进行控制管理。其中，通信模块8003用于向位置服务器发送位置更新请求，位置更新请求用于指示位置服务器将第一桶的位置信息由位于第二存储系统更新为位于第一存储系统；通信模块8003，还用于从第二存储系统迁移第一桶中的数据，并将数据存储于第一存储系统的第二桶中，第二桶的标识与第一桶的标识相同；通信模块8003，还用于接收数据访问请求，数据访问请求用于访问第一桶中的数据；处理模块8002，用于根据数据访问请求的类型以及数据的迁移状况确定由第一存储系统或第二存储系统处理数据访问请求。
在本申请实施例中，可选的，通信模块8003可以用于：先迁移第一桶中对象的元数据，再迁移第一桶中除元数据以外的其余数据。
在本申请实施例中，可选的，通信模块8003可以用于：从第二存储系统接收数据访问请求，或第一存储系统接收虚拟主机发送的数据访问请求，数据访问请求包括第一桶的标识。
在本申请实施例中，可选的，处理模块8002可以用于：当数据访问请求的类型为下载类请求，确定下载类请求所请求的数据是否已迁移至第二桶；当下载类请求所请求的数据已迁移至第二桶，从第二桶获取下载类请求所请求的数据；当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据不为元数据，第一存储系统从第一桶获取下载类请求所请求的数据发送给客户端；当下载类请求所请求的数据没有迁移至第二桶，且下载类请求所请求的数据为元数据，第一存储系统从第一桶迁移元数据并保存至第二桶，将元数据发送给客户端。
在本申请实施例中，可选的，处理模块8002可以用于：当数据访问请求的类型为上传 类请求，将上传类请求所上传的数据保存至第二桶。
结合图4所示的方法实施例，例如，处理模块8002用于支持第一存储系统执行图4中的过程401以及407，和/或用于本文所描述的技术的其它过程。通信模块8003用于支持第一存储系统与其他网络实体的通信，例如与图2中示出的功能模块或网络实体之间的通信，通信模块8003可以用于支持第一存储系统执行图4中的过程402，405以及406。第一存储系统还可以包括存储模块8001，用于存储第一存储系统的程序代码和数据。
其中，处理模块8002可以是处理器或控制器，例如可以是中央处理器(Central Processing Unit，CPU)，通用处理器，数字信号处理器(Digital Signal Processor，DSP)，专用集成电路(Application-Specific Integrated Circuit，ASIC)，现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。所述处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。通信模块8003可以是收发器、收发电路或通信接口等。存储模块8001可以是存储器。
当处理模块8002为处理器，通信模块8003为通信接口，存储模块8001为存储器时，本申请实施例所涉及的第一存储系统可以为图9所示的存储系统。
参阅图9所示，该存储系统9010包括：处理器9012、通信接口9013、存储器9011以及总线9014。其中，通信接口9013、处理器9012以及存储器9011通过总线9014相互连接；总线9014可以是外设部件互连标准(Peripheral Component Interconnect，PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图9中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
结合本申请公开内容所描述的方法或者算法的步骤可以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于核心网接口设备中。当然，处理器和存储介质也可以作为分立组件存在于核心网接口设备中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (13)

  1. 一种数据迁移方法，其特征在于，包括：
    第一存储系统向位置服务器发送位置更新请求，所述位置更新请求用于指示所述位置服务器将第一桶的位置信息由位于第二存储系统更新为位于所述第一存储系统；
    所述第一存储系统从所述第二存储系统迁移所述第一桶的数据，并将所述数据存储于所述第一存储系统的第二桶中，所述第二桶的标识与所述第一桶的标识相同；
    所述第一存储系统接收数据访问请求，所述数据访问请求用于访问所述第一桶中的数据；
    所述第一存储系统根据所述数据访问请求的类型以及所述数据的迁移状况确定由所述第一存储系统或所述第二存储系统处理所述数据访问请求。
  2. 根据权利要求1所述的数据迁移方法，其特征在于，所述第一存储系统从所述第二存储系统迁移所述第一桶的数据，包括：
    所述第一存储系统先迁移所述第一桶中对象的元数据，再迁移所述第一桶中除所述元数据以外的其余数据。
  3. 根据权利要求1或2所述的数据迁移方法，其特征在于，所述第一存储系统接收数据访问请求，包括：
    所述第一存储系统从所述第二存储系统接收所述数据访问请求，或所述第一存储系统接收虚拟主机发送的所述数据访问请求，所述数据访问请求包括所述第一桶的标识。
  4. 根据权利要求1-3任一项所述的数据迁移方法，其特征在于，所述第一存储系统根据所述数据访问请求的类型以及所述数据的迁移状况确定由所述第一存储系统或所述第二存储系统处理所述数据访问请求，包括：
    当所述数据访问请求的类型为下载类请求，所述第一存储系统确定所述下载类请求所请求的数据是否已迁移至所述第二桶；
    当所述下载类请求所请求的数据已迁移至所述第二桶，所述第一存储系统从所述第二桶获取所述下载类请求所请求的数据；
    当所述下载类请求所请求的数据没有迁移至所述第二桶，且所述下载类请求所请求的数据不为元数据，所述第一存储系统从所述第一桶获取所述下载类请求所请求的数据发送给客户端；
    当所述下载类请求所请求的数据没有迁移至所述第二桶，且所述下载类请求所请求的数据为元数据，所述第一存储系统从所述第一桶迁移所述元数据并保存至所述第二桶，将所述元数据发送给客户端。
  5. 根据权利要求1-3任一项所述的数据迁移方法，其特征在于，所述第一存储系统根据所述数据访问请求的类型以及所述数据的迁移状况确定由所述第一存储系统或所述第二存储系统处理所述数据访问请求，包括：
    当所述数据访问请求的类型为上传类请求，所述第一存储系统将所述上传类请求所上传的数据保存至所述第二桶。
  6. 一种存储系统，其特征在于，所述存储系统作为第一存储系统，包括：
    通信模块，用于向位置服务器发送位置更新请求，所述位置更新请求用于指示所述位置服务器将第一桶的位置信息由位于第二存储系统更新为位于所述第一存储系统；
    所述通信模块，还用于从所述第二存储系统迁移所述第一桶的数据，并将所述数据存储于所述第一存储系统的第二桶中，所述第二桶的标识与所述第一桶的标识相同；
    所述通信模块，还用于接收数据访问请求，所述数据访问请求用于访问所述第一桶中的数据；
    处理模块，用于根据所述数据访问请求的类型以及所述数据的迁移状况确定由所述第一存储系统或所述第二存储系统处理所述数据访问请求。
  7. 根据权利要求6所述的存储系统，其特征在于，所述通信模块用于：
    先迁移所述第一桶中对象的元数据，再迁移所述第一桶中除所述元数据以外的其余数据。
  8. 根据权利要求6或7所述的存储系统，其特征在于，所述通信模块用于：从所述第二存储系统接收所述数据访问请求，或所述第一存储系统接收虚拟主机发送的所述数据访问请求，所述数据访问请求包括所述第一桶的标识。
  9. 根据权利要求6-8任一项所述的存储系统，其特征在于，所述处理模块用于：
    当所述数据访问请求的类型为下载类请求，确定所述下载类请求所请求的数据是否已迁移至所述第二桶；
    当所述下载类请求所请求的数据已迁移至所述第二桶，从所述第二桶获取所述下载类请求所请求的数据；
    当所述下载类请求所请求的数据没有迁移至所述第二桶，且所述下载类请求所请求的数据不为元数据，从所述第一桶获取所述下载类请求所请求的数据发送给客户端；
    当所述下载类请求所请求的数据没有迁移至所述第二桶，且所述下载类请求所请求的数据为元数据，从所述第一桶迁移所述元数据并保存至所述第二桶，将所述元数据发送给客户端。
  10. 根据权利要求6-8任一项所述的存储系统，其特征在于，所述处理模块用于：
    当所述数据访问请求的类型为上传类请求，将所述上传类请求所上传的数据保存至所述第二桶。
  11. 一种通信装置，包括存储器，所述存储器存储有计算机指令，当所述计算机指令被执行时，使得所述通信装置执行如权利要求1-5中任一项的数据迁移方法。
  12. 一种计算机存储介质，所述计算机存储介质存储有计算机指令，当所述计算机指令被计算机执行时，使得所述计算机执行如权利要求1-5中任一项的数据迁移方法。
  13. 一种存储系统，其特征在于，所述存储系统作为第一存储系统，包括处理器和接口，所述处理器和所述接口通信，所述处理器用于执行如权利要求1-5中任一项的数据迁移方法。
PCT/CN2019/094110 2018-07-09 2019-07-01 一种数据迁移方法及装置 Ceased WO2020011038A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19833427.8A EP3800538B1 (en) 2018-07-09 2019-07-01 Method and apparatus for data migration
US17/144,999 US11914881B2 (en) 2018-07-09 2021-01-08 Data migration method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810748632.4A CN109189324B (zh) 2018-07-09 2018-07-09 一种数据迁移方法及装置
CN201810748632.4 2018-07-09

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/144,999 Continuation US11914881B2 (en) 2018-07-09 2021-01-08 Data migration method and apparatus

Publications (1)

Publication Number Publication Date
WO2020011038A1 true WO2020011038A1 (zh) 2020-01-16

Family

ID=64936332

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/094110 Ceased WO2020011038A1 (zh) 2018-07-09 2019-07-01 一种数据迁移方法及装置

Country Status (4)

Country Link
US (1) US11914881B2 (zh)
EP (1) EP3800538B1 (zh)
CN (1) CN109189324B (zh)
WO (1) WO2020011038A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112131128A (zh) * 2020-09-29 2020-12-25 网易（杭州）网络有限公司 数据测试方法、装置、存储介质和电子装置

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109189324B (zh) * 2018-07-09 2021-01-08 华为技术有限公司 一种数据迁移方法及装置
US11567840B2 (en) * 2020-03-09 2023-01-31 Rubrik, Inc. Node level recovery for clustered databases
CN111800497A (zh) * 2020-06-30 2020-10-20 北京金山云网络技术有限公司 请求响应方法、热迁移系统及装置
CN112511634A (zh) * 2020-12-02 2021-03-16 北京邮电大学 一种数据获取方法、装置、电子设备及存储介质
CN117729217B (zh) * 2020-12-14 2024-08-27 华为云计算技术有限公司 云平台及其提供的对象存储服务的桶管理方法
CN113312147B (zh) * 2021-06-11 2022-12-30 中国科学技术大学 一种对象存储跨集群海量数据迁移方法及系统
CN116266839A (zh) * 2021-12-16 2023-06-20 华为云计算技术有限公司 一种对象存储桶的数据访问方法以及云管理平台
CN115346050A (zh) * 2022-06-27 2022-11-15 武汉光庭信息技术股份有限公司 一种基于oss数据集在线预处理方法及系统
CN115473815B (zh) * 2022-08-23 2024-01-19 浪潮通信信息系统有限公司 基于设备变更的业务迁移系统及方法
CN117749813A (zh) * 2022-09-13 2024-03-22 华为云计算技术有限公司 基于云计算技术的数据迁移方法及云管理平台
CN115543967A (zh) * 2022-11-02 2022-12-30 中国建设银行股份有限公司 一种数据迁移方法、装置、电子设备及计算机可读介质
CN118733526A (zh) * 2023-03-31 2024-10-01 华为技术有限公司 一种存储访问方法及系统
US20240394231A1 (en) * 2023-05-26 2024-11-28 Pure Storage, Inc. Migrating data from disparate object storage environments
US20250123966A1 (en) * 2023-10-15 2025-04-17 Nvidia Corporation Application programming interface to update information
CN118118728B (zh) * 2024-04-30 2024-07-05 四川天邑康和通信股份有限公司 Iptv机顶盒的数据处理方法和装置、机顶盒及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191389A1 (en) * 2005-07-28 2011-08-04 Vmware, Inc. Distributed data management system
CN104603774A (zh) * 2012-10-11 2015-05-06 株式会社日立制作所 迁移目的地文件服务器和文件系统迁移方法
CN104778179A (zh) * 2014-01-14 2015-07-15 阿里巴巴集团控股有限公司 一种数据迁移测试方法和系统
CN105549904A (zh) * 2015-12-08 2016-05-04 华为技术有限公司 一种应用于存储系统中的数据迁移方法及存储设备
CN109189324A (zh) * 2018-07-09 2019-01-11 华为技术有限公司 一种数据迁移方法及装置

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3410010B2 (ja) * 1997-12-24 2003-05-26 株式会社日立製作所 サブシステムの移行方法および情報処理システム
KR100662290B1 (ko) * 2000-12-27 2007-01-02 엘지전자 주식회사 광 기록매체 상의 데이터 복사 및 이동 방법
US7707186B2 (en) * 2004-06-18 2010-04-27 Emc Corporation Method and apparatus for data set migration
JP4387261B2 (ja) * 2004-07-15 2009-12-16 株式会社日立製作所 計算機システム、および、記憶装置システムの移行方法
JP2006146476A (ja) * 2004-11-18 2006-06-08 Hitachi Ltd ストレージシステム及びストレージシステムのデータ移行方法
JP4900784B2 (ja) * 2006-04-13 2012-03-21 株式会社日立製作所 ストレージシステム及びストレージシステムのデータ移行方法
CN100538617C (zh) * 2006-07-11 2009-09-09 任永坚 存储介质处理方法及系统
US8407436B2 (en) * 2009-02-11 2013-03-26 Hitachi, Ltd. Methods and apparatus for migrating thin provisioning volumes between storage systems
US8122213B2 (en) * 2009-05-05 2012-02-21 Dell Products L.P. System and method for migration of data
CN103067433B (zh) * 2011-10-24 2016-04-20 阿里巴巴集团控股有限公司 一种分布式存储系统的数据迁移方法、设备和系统
US8965847B1 (en) * 2011-10-28 2015-02-24 Oxygen Cloud, Inc. Independent synchronization of file data and file metadata
US8904133B1 (en) * 2012-12-03 2014-12-02 Hitachi, Ltd. Storage apparatus and storage apparatus migration method
US10860529B2 (en) * 2014-08-11 2020-12-08 Netapp Inc. System and method for planning and configuring a file system migration
US20150355862A1 (en) * 2014-06-04 2015-12-10 Pure Storage, Inc. Transparent array migration
CN106294387B (zh) * 2015-05-19 2020-06-02 深圳市腾讯计算机系统有限公司 数据迁移方法和装置
CN107577425A (zh) * 2017-08-23 2018-01-12 杭州宏杉科技股份有限公司 一种数据迁移方法及装置
CN107704206B (zh) * 2017-10-09 2020-09-18 苏州浪潮智能科技有限公司 在线迁移异构系统数据的方法、装置、设备和存储介质
CN107656705B (zh) * 2017-10-25 2020-10-23 苏州浪潮智能科技有限公司 一种计算机存储介质和一种数据迁移方法、装置及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191389A1 (en) * 2005-07-28 2011-08-04 Vmware, Inc. Distributed data management system
CN104603774A (zh) * 2012-10-11 2015-05-06 株式会社日立制作所 迁移目的地文件服务器和文件系统迁移方法
CN104778179A (zh) * 2014-01-14 2015-07-15 阿里巴巴集团控股有限公司 一种数据迁移测试方法和系统
CN105549904A (zh) * 2015-12-08 2016-05-04 华为技术有限公司 一种应用于存储系统中的数据迁移方法及存储设备
CN109189324A (zh) * 2018-07-09 2019-01-11 华为技术有限公司 一种数据迁移方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3800538A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112131128A (zh) * 2020-09-29 2020-12-25 网易（杭州）网络有限公司 数据测试方法、装置、存储介质和电子装置
CN112131128B (zh) * 2020-09-29 2023-08-22 网易（杭州）网络有限公司 数据测试方法、装置、存储介质和电子装置

Also Published As

Publication number Publication date
CN109189324A (zh) 2019-01-11
US20210132849A1 (en) 2021-05-06
EP3800538B1 (en) 2024-06-12
EP3800538A1 (en) 2021-04-07
US11914881B2 (en) 2024-02-27
EP3800538A4 (en) 2021-08-04
CN109189324B (zh) 2021-01-08

Similar Documents

Publication Publication Date Title
WO2020011038A1 (zh) 一种数据迁移方法及装置
US20200336455A1 (en) Hierarchical namespace service with distributed name resolution caching and synchronization
CN112306993B (zh) 基于Redis的数据读取方法、装置、设备及可读存储介质
EP3811596A1 (en) Hierarchical namespace with strong consistency and horizontal scalability
US11537575B1 (en) Real-time database performance tuning
CN105528368A (zh) 一种数据库迁移方法及装置
US10620871B1 (en) Storage scheme for a distributed storage system
CN111200634B (zh) 缓存资源联动更新方法、系统及服务器
US20120324040A1 (en) Local networked storage linked to remote networked storage system
CN107483250A (zh) 分布式配置管理方法、装置及实现分布式配置管理的系统
US11240302B1 (en) Live migration of log-based consistency mechanisms for data stores
CN111722909A (zh) 一种虚拟机迁移方法、系统、设备及存储介质
KR20140143775A (ko) 캐시 관리 기법
CN110442307A (zh) 一种Linux系统中磁盘的绑定方法、设备以及存储介质
CN113746641B (zh) 一种基于分布式存储的odx协议处理方法
CN108279924A (zh) 程序发布方法及装置
CN110445841A (zh) 一种云物理机挂载云盘的方法、设备以及存储介质
CN118502907A (zh) 请求处理方法、装置、设备和存储介质
US9658935B1 (en) Processing file modifications in a networked storage system
CN115988067A (zh) 一种应用注册方法、装置、电子设备及存储介质
CN114338645A (zh) 一种整机镜像制作方法、镜像安装方法、装置及电子设备
CN113946550B (zh) 基于局域网去中心化实现文件共享的方法、装置、处理器及其计算机可读存储介质
WO2019143967A1 (en) Methods for automated artifact storage management and devices thereof
JP5525740B2 (ja) 仮想アプライアンスサーバ管理方法及びシステム
JP7580501B2 (ja) イメージ配布方法、電子機器及び記憶媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19833427

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019833427

Country of ref document: EP

Effective date: 20201228