CA2934041A1 - Rapprochement de fractions de volume dans des cohortes de volume - Google Patents

Rapprochement de fractions de volume dans des cohortes de volume Download PDF

Info

Publication number
CA2934041A1
CA2934041A1 CA2934041A CA2934041A CA2934041A1 CA 2934041 A1 CA2934041 A1 CA 2934041A1 CA 2934041 A CA2934041 A CA 2934041A CA 2934041 A CA2934041 A CA 2934041A CA 2934041 A1 CA2934041 A1 CA 2934041A1
Authority
CA
Canada
Prior art keywords
data
volumelets
cohort
storage
stored
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA2934041A
Other languages
English (en)
Other versions
CA2934041C (fr
Inventor
Bryan James Donlan
Paul David Franklin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Amazon Technologies Inc
Original Assignee
Amazon Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/133,575 external-priority patent/US10620830B2/en
Priority claimed from US14/133,522 external-priority patent/US10685037B2/en
Application filed by Amazon Technologies Inc filed Critical Amazon Technologies Inc
Publication of CA2934041A1 publication Critical patent/CA2934041A1/fr
Application granted granted Critical
Publication of CA2934041C publication Critical patent/CA2934041C/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2094—Redundant storage or storage space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/1658—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
    • G06F11/1662—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit the resynchronized component or unit being a persistent storage device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G06F16/137—Hash-based
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G06F16/184—Distributed file systems implemented as replicated file system
    • G06F16/1844—Management specifically adapted to replicated file systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Retry When Errors Occur (AREA)
CA2934041A 2013-12-18 2014-12-18 Rapprochement de fractions de volume dans des cohortes de volume Active CA2934041C (fr)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US14/133,522 2013-12-18
US14/133,575 2013-12-18
US14/133,575 US10620830B2 (en) 2013-12-18 2013-12-18 Reconciling volumelets in volume cohorts
US14/133,522 US10685037B2 (en) 2013-12-18 2013-12-18 Volume cohorts in object-redundant storage systems
PCT/US2014/071159 WO2015095521A1 (fr) 2013-12-18 2014-12-18 Rapprochement de fractions de volume dans des cohortes de volume

Publications (2)

Publication Number Publication Date
CA2934041A1 true CA2934041A1 (fr) 2015-06-25
CA2934041C CA2934041C (fr) 2021-04-13

Family

ID=53403692

Family Applications (1)

Application Number Title Priority Date Filing Date
CA2934041A Active CA2934041C (fr) 2013-12-18 2014-12-18 Rapprochement de fractions de volume dans des cohortes de volume

Country Status (5)

Country Link
EP (1) EP3084647A4 (fr)
JP (1) JP6255496B2 (fr)
CN (1) CN105830063B (fr)
CA (1) CA2934041C (fr)
WO (1) WO2015095521A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107968809B (zh) * 2016-10-20 2021-06-04 北京金山云网络技术有限公司 一种副本放置方法及装置
CN115917523A (zh) * 2020-08-24 2023-04-04 富士胶片株式会社 信息处理装置、信息处理方法及信息处理程序
CN115380506B (zh) 2021-03-03 2025-05-09 谷歌有限责任公司 隐私保护的活动聚合机制的方法、系统

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5548724A (en) * 1993-03-22 1996-08-20 Hitachi, Ltd. File server system and file access control method of the same
JPH06332782A (ja) * 1993-03-22 1994-12-02 Hitachi Ltd ファイルサーバシステム及びそのファイルアクセス制御方法
US6952737B1 (en) * 2000-03-03 2005-10-04 Intel Corporation Method and apparatus for accessing remote storage in a distributed storage cluster architecture
JP3617632B2 (ja) * 2001-07-19 2005-02-09 富士通株式会社 Ｒａｉｄ制御装置及びその制御方法
JP4843976B2 (ja) * 2005-03-25 2011-12-21 日本電気株式会社 レプリケーションシステムと方法
US20080065704A1 (en) * 2006-09-12 2008-03-13 Microsoft Corporation Data and replica placement using r-out-of-k hash functions
JP2009187141A (ja) * 2008-02-04 2009-08-20 Brother Ind Ltd 情報配信システム及び同システムにおける重複数調整方法
JP2009245089A (ja) * 2008-03-31 2009-10-22 Fujitsu Ltd 分散オブジェクト・プログラム及びレプリケーション処理方法
JP5484470B2 (ja) * 2008-09-19 2014-05-07 オラクル・インターナショナル・コーポレイション オフロードされたブルームフィルタを伴うインテリジェントストレージにおける協調並列フィルタ処理を用いるハッシュジョイン
US8682903B2 (en) * 2009-06-30 2014-03-25 International Business Machines Corporation System and method for synchronized content directories on cluster devices
US20100332506A1 (en) * 2009-06-30 2010-12-30 International Business Machines Corporation Marking sought after content items on network media devices
US8458287B2 (en) * 2009-07-31 2013-06-04 Microsoft Corporation Erasure coded storage aggregation in data centers
US8504535B1 (en) * 2010-12-20 2013-08-06 Amazon Technologies, Inc. Erasure coding and redundant replication
US9047331B2 (en) * 2011-04-21 2015-06-02 International Business Machines Corporation Scalable row-store with consensus-based replication
US9146937B2 (en) * 2011-06-23 2015-09-29 Cohort Fs, Llc Client-based data replication
GB2494867A (en) * 2011-09-19 2013-03-27 Cloudtran Inc Scalable distributed transaction processing method
CN108388632B (zh) * 2011-11-15 2021-11-19 起元科技有限公司 数据分群、分段、以及并行化
US9449014B2 (en) * 2011-11-29 2016-09-20 Dell Products L.P. Resynchronization of replicated data

Also Published As

Publication number Publication date
CN105830063B (zh) 2020-12-15
CA2934041C (fr) 2021-04-13
JP6255496B2 (ja) 2017-12-27
EP3084647A4 (fr) 2017-11-29
CN105830063A (zh) 2016-08-03
WO2015095521A1 (fr) 2015-06-25
EP3084647A1 (fr) 2016-10-26
JP2017504885A (ja) 2017-02-09

Similar Documents

Publication Publication Date Title
US10620830B2 (en) Reconciling volumelets in volume cohorts
US10685037B2 (en) Volume cohorts in object-redundant storage systems
US9846540B1 (en) Data durability using un-encoded copies and encoded combinations
US10387382B2 (en) Estimating a number of entries in a dispersed hierarchical index
US8977660B1 (en) Multi-level distributed hash table for data storage in a hierarchically arranged network
US10430433B2 (en) Systems and methods for data conversion and comparison
US10176044B2 (en) Fallback delegates for modification of an index structure
US10423626B2 (en) Systems and methods for data conversion and comparison
US10067832B2 (en) Imposter slices
US10282440B2 (en) Prioritizing rebuilding of encoded data slices
US20180336083A1 (en) Dispersed b-tree directory trees
CN102708165A (zh) 分布式文件系统中的文件处理方法及装置
US10852957B2 (en) Migration agent employing moveslice request
US20140052706A1 (en) Archival storage and retrieval system
CN112148797B (zh) 基于区块链的分布式数据存取方法、装置及存储节点
US20160224638A1 (en) Parallel and transparent technique for retrieving original content that is restructured in a distributed object storage system
US11553023B2 (en) Abstraction layer for streaming data sources
US20160205190A1 (en) Parallel transparent restructuring of immutable content in a distributed object storage system
AU2014353667A1 (en) A method of generating a reference index data structure and method for finding a position of a data pattern in a reference data structure
US20190004727A1 (en) Using a namespace to augment de-duplication
CA2934041C (fr) Rapprochement de fractions de volume dans des cohortes de volume
US9639549B2 (en) Hybrid of proximity and identity similarity based deduplication in a data deduplication system
US20170310754A1 (en) Selecting memory for data access in a dispersed storage network
CN109478125B (zh) 操纵分布式一致性协议以识别期望的存储单元集
US10958731B2 (en) Indicating multiple encoding schemes in a dispersed storage network

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20160615

MPN Maintenance fee for patent paid

Free format text: FEE DESCRIPTION TEXT: MF (PATENT, 10TH ANNIV.) - STANDARD

Year of fee payment: 10

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20241213

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20241213

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-4-4-W10-W00-W100 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: LETTER SENT

Effective date: 20260226