JP6255496B2 - ボリュームコーホート内の小ボリュームの照合調整 - Google Patents

ボリュームコーホート内の小ボリュームの照合調整 Download PDF

Info

Publication number
JP6255496B2
JP6255496B2 JP2016540658A JP2016540658A JP6255496B2 JP 6255496 B2 JP6255496 B2 JP 6255496B2 JP 2016540658 A JP2016540658 A JP 2016540658A JP 2016540658 A JP2016540658 A JP 2016540658A JP 6255496 B2 JP6255496 B2 JP 6255496B2
Authority
JP
Japan
Prior art keywords
data
cohort
storage
stored
small volumes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2016540658A
Other languages
English (en)
Japanese (ja)
Other versions
JP2017504885A (ja
Inventor
ドンラン，ブライアン・ジェームズ
フランクリン，ポール・デイヴィッド
Original Assignee
アマゾン・テクノロジーズ・インコーポレーテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/133,575 external-priority patent/US10620830B2/en
Priority claimed from US14/133,522 external-priority patent/US10685037B2/en
Application filed by アマゾン・テクノロジーズ・インコーポレーテッド filed Critical アマゾン・テクノロジーズ・インコーポレーテッド
Publication of JP2017504885A publication Critical patent/JP2017504885A/ja
Application granted granted Critical
Publication of JP6255496B2 publication Critical patent/JP6255496B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2094—Redundant storage or storage space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/1658—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
    • G06F11/1662—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit the resynchronized component or unit being a persistent storage device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G06F16/137—Hash-based
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G06F16/184—Distributed file systems implemented as replicated file system
    • G06F16/1844—Management specifically adapted to replicated file systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Retry When Errors Occur (AREA)
JP2016540658A 2013-12-18 2014-12-18 ボリュームコーホート内の小ボリュームの照合調整 Active JP6255496B2 (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US14/133,522 2013-12-18
US14/133,575 2013-12-18
US14/133,575 US10620830B2 (en) 2013-12-18 2013-12-18 Reconciling volumelets in volume cohorts
US14/133,522 US10685037B2 (en) 2013-12-18 2013-12-18 Volume cohorts in object-redundant storage systems
PCT/US2014/071159 WO2015095521A1 (en) 2013-12-18 2014-12-18 Reconciling volumelets in volume cohorts

Publications (2)

Publication Number Publication Date
JP2017504885A JP2017504885A (ja) 2017-02-09
JP6255496B2 true JP6255496B2 (ja) 2017-12-27

Family

ID=53403692

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2016540658A Active JP6255496B2 (ja) 2013-12-18 2014-12-18 ボリュームコーホート内の小ボリュームの照合調整

Country Status (5)

Country Link
EP (1) EP3084647A4 (de)
JP (1) JP6255496B2 (de)
CN (1) CN105830063B (de)
CA (1) CA2934041C (de)
WO (1) WO2015095521A1 (de)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107968809B (zh) * 2016-10-20 2021-06-04 北京金山云网络技术有限公司 一种副本放置方法及装置
CN115917523A (zh) * 2020-08-24 2023-04-04 富士胶片株式会社 信息处理装置、信息处理方法及信息处理程序
CN115380506B (zh) 2021-03-03 2025-05-09 谷歌有限责任公司 隐私保护的活动聚合机制的方法、系统

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5548724A (en) * 1993-03-22 1996-08-20 Hitachi, Ltd. File server system and file access control method of the same
JPH06332782A (ja) * 1993-03-22 1994-12-02 Hitachi Ltd ファイルサーバシステム及びそのファイルアクセス制御方法
US6952737B1 (en) * 2000-03-03 2005-10-04 Intel Corporation Method and apparatus for accessing remote storage in a distributed storage cluster architecture
JP3617632B2 (ja) * 2001-07-19 2005-02-09 富士通株式会社 Ｒａｉｄ制御装置及びその制御方法
JP4843976B2 (ja) * 2005-03-25 2011-12-21 日本電気株式会社 レプリケーションシステムと方法
US20080065704A1 (en) * 2006-09-12 2008-03-13 Microsoft Corporation Data and replica placement using r-out-of-k hash functions
JP2009187141A (ja) * 2008-02-04 2009-08-20 Brother Ind Ltd 情報配信システム及び同システムにおける重複数調整方法
JP2009245089A (ja) * 2008-03-31 2009-10-22 Fujitsu Ltd 分散オブジェクト・プログラム及びレプリケーション処理方法
JP5484470B2 (ja) * 2008-09-19 2014-05-07 オラクル・インターナショナル・コーポレイション オフロードされたブルームフィルタを伴うインテリジェントストレージにおける協調並列フィルタ処理を用いるハッシュジョイン
US8682903B2 (en) * 2009-06-30 2014-03-25 International Business Machines Corporation System and method for synchronized content directories on cluster devices
US20100332506A1 (en) * 2009-06-30 2010-12-30 International Business Machines Corporation Marking sought after content items on network media devices
US8458287B2 (en) * 2009-07-31 2013-06-04 Microsoft Corporation Erasure coded storage aggregation in data centers
US8504535B1 (en) * 2010-12-20 2013-08-06 Amazon Technologies, Inc. Erasure coding and redundant replication
US9047331B2 (en) * 2011-04-21 2015-06-02 International Business Machines Corporation Scalable row-store with consensus-based replication
US9146937B2 (en) * 2011-06-23 2015-09-29 Cohort Fs, Llc Client-based data replication
GB2494867A (en) * 2011-09-19 2013-03-27 Cloudtran Inc Scalable distributed transaction processing method
CN108388632B (zh) * 2011-11-15 2021-11-19 起元科技有限公司 数据分群、分段、以及并行化
US9449014B2 (en) * 2011-11-29 2016-09-20 Dell Products L.P. Resynchronization of replicated data

Also Published As

Publication number Publication date
CN105830063B (zh) 2020-12-15
CA2934041C (en) 2021-04-13
CA2934041A1 (en) 2015-06-25
EP3084647A4 (de) 2017-11-29
CN105830063A (zh) 2016-08-03
WO2015095521A1 (en) 2015-06-25
EP3084647A1 (de) 2016-10-26
JP2017504885A (ja) 2017-02-09

Similar Documents

Publication Publication Date Title
US10620830B2 (en) Reconciling volumelets in volume cohorts
US10685037B2 (en) Volume cohorts in object-redundant storage systems
CN111090645B (zh) 基于云存储的数据传输方法、装置及计算机设备
US10095581B2 (en) Transmission time refinement in a storage system
US8977660B1 (en) Multi-level distributed hash table for data storage in a hierarchically arranged network
JP5850044B2 (ja) 情報処理装置、分散ファイルシステム、クライアント装置、情報処理方法、および、コンピュータ・プログラム
US20130191523A1 (en) Real-time analytics for large data sets
US8255366B1 (en) Segment-based method for efficient file restoration
NL2011817C2 (en) A method of generating a reference index data structure and method for finding a position of a data pattern in a reference data structure.
US8010648B2 (en) Replica placement in a distributed storage system
US11553023B2 (en) Abstraction layer for streaming data sources
CN109271545B (zh) 一种特征检索方法及装置、存储介质和计算机设备
CN114936188A (zh) 数据处理方法、装置、电子设备及存储介质
CN112912870A (zh) 租户标识符的转换
JP6255496B2 (ja) ボリュームコーホート内の小ボリュームの照合調整
US20130275992A1 (en) Distributed processing system, distributed processing method, and distributed processing program
JP6113816B1 (ja) 情報処理システム、情報処理装置、及びプログラム
CN107465706B (zh) 一种基于无线通信网络的分布式数据对象存储装置
US20180246666A1 (en) Methods for performing data deduplication on data blocks at granularity level and devices thereof
CN119166079A (zh) 数据处理方法、装置、存储介质及计算机设备
JP4891657B2 (ja) データ記憶システム、ファイル検索装置およびプログラム
CN115344422B (zh) 纠删块副本到三副本双向映射方法、装置以及电子设备
US9916372B1 (en) Folded-hashtable synchronization mechanism
JP2015088109A (ja) ストレージシステム、ストレージ制御方法、およびストレージ制御プログラム
Poonthottam et al. A dynamic data placement scheme for hadoop using real-time access patterns

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20160616

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20170704

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20170725

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20171025

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20171121

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20171204

R150 Certificate of patent or registration of utility model

Ref document number: 6255496

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250