WO2018141304A1 - 一种闪存文件系统及其数据管理方法 - Google Patents

一种闪存文件系统及其数据管理方法 Download PDF

Info

Publication number
WO2018141304A1
WO2018141304A1 PCT/CN2018/075376 CN2018075376W WO2018141304A1 WO 2018141304 A1 WO2018141304 A1 WO 2018141304A1 CN 2018075376 W CN2018075376 W CN 2018075376W WO 2018141304 A1 WO2018141304 A1 WO 2018141304A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
flash
flash buffer
buffer
page
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/075376
Other languages
English (en)
French (fr)
Inventor
舒继武
罗圣美
陆游游
张佳程
杨洪章
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to US16/483,608 priority Critical patent/US20200034340A1/en
Priority to EP18747430.9A priority patent/EP3579111A4/en
Priority to JP2019542554A priority patent/JP6920448B2/ja
Publication of WO2018141304A1 publication Critical patent/WO2018141304A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G06F12/0868—Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/1847—File system types specifically adapted to static storage, e.g. adapted to flash memory or SSD
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0877—Cache access modes
    • G06F12/0882—Page mode
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/162—Delete operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/172—Caching, prefetching or hoarding of files
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/178—Techniques for file synchronisation in file systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0656—Data buffering arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0893—Caches characterised by their organisation or structure
    • G06F12/0897—Caches characterised by their organisation or structure with two or more cache hierarchy levels
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1016—Performance improvement
    • G06F2212/1024—Latency reduction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1032—Reliability improvement, data loss prevention, degraded operation etc
    • G06F2212/1036—Life time enhancement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/21—Employing a record carrier using a specific recording technology
    • G06F2212/214—Solid state disk
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/22—Employing cache memory using specific memory technology
    • G06F2212/225—Hybrid cache memory, e.g. having both volatile and non-volatile portions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/28—Using a specific disk cache architecture
    • G06F2212/283—Plural cache memories
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/46—Caching storage objects of specific type in disk cache
    • G06F2212/463—File
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7203—Temporary buffering, e.g. using volatile buffer or dedicated buffer blocks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7204—Capacity control, e.g. partitioning, end-of-life degradation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/4401—Bootstrapping
    • G06F9/4418—Suspend and resume; Hibernate and awake
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • This document relates to but not limited to the field of storage technology, and in particular relates to a flash file system and a data management method thereof.
  • Flash Memory is an electronic erasable programming memory. Compared with traditional disk media, flash memory has the characteristics of high read/write bandwidth, low access latency, low power consumption and high stability.
  • flash memory has begun. Popularized in data centers, personal computers, and mobile devices. The flash memory is read and written in units of pages. Before the flash memory rewrites a page, it needs to be erased first. Flash memory is erased in blocks, and a flash block contains hundreds of flash pages. The cells of the flash have a limited number of erase operations, ie each flash cell has a limited lifetime.
  • the page cache is used to cache the most recently manipulated data to speed up the read and write process.
  • you need to read the data first find in the page cache whether the part of the content resides in the memory, if found, directly return the data; if not found, then read it in the flash memory.
  • the data is no longer written directly to the device. Instead, the data is written to the page cache and the corresponding page is marked as dirty and then returned directly. Dirty pages in the page cache are written to the flash device when the user issues a synchronous call or the operating system background thread initiates a synchronous operation.
  • Embodiments of the present invention provide a flash file system and a data management method thereof, which can avoid unnecessary data writing.
  • An embodiment of the present invention provides a flash file system, including: a create module, a markup module, a synchronization module, and a backfill module;
  • Marking module configured to mark the write data as dirty data in the memory cache when data is written and the amount of data written is less than or equal to a preset mark threshold, wherein the mark threshold is used to indicate write to the memory The amount of data that needs to be cached by the data granularity;
  • the synchronization module is configured to merge all the dirty data in the memory cache or the dirty data of the file to be synchronized into the flash buffer when the data synchronization is needed, and notify the backfill module when the flash buffer is full;
  • the backfill module is set to receive notification of the synchronization module, read out dirty data in the flash buffer and write it to the file system area, and erase the flash buffer.
  • the flash buffer includes a first flash buffer and a second flash buffer; wherein
  • the synchronization module is configured to: when all the dirty data in the memory cache or the dirty data of the file to be synchronized is merged, write the dirty data into the first flash buffer when the data synchronization is needed; when the first flash buffer has been When full, the first notification is sent to the backfill module, and when data synchronization is required, all dirty data in the memory cache or dirty data of the file to be synchronized is merged and written into the second flash buffer; the second flash buffer When it is full, the second notification is sent to the backfill module, and when the data synchronization is needed, all the dirty data in the memory cache or the dirty data of the file to be synchronized is merged and written into the first flash buffer;
  • the backfill module is configured to: when receiving the first notification of the synchronization module, read out dirty data in the first flash buffer and write it to the file system area, and erase the first flash buffer; When the second notification of the synchronization module is received, the dirty data in the second flash buffer is read and written to the file system area, and the second flash buffer is erased.
  • the marking module is configured to: when there is data writing and the amount of data written is less than or equal to the marking threshold, the index node number and the data segment of the file corresponding to the written data are located The page number, the offset within the page, the length of the data segment, and the data segment contents are encapsulated into records, the records are added to the preset dirty data list, and the reference count of the memory cache page corresponding to the write data is incremented by one.
  • the synchronization module is set to:
  • system further includes: a recovery module, configured to detect whether there is dirty data in the flash buffer when the flash file system is restarted;
  • the embodiment of the invention further provides a data management method for a flash file system, comprising:
  • the write data is marked as dirty data in the memory cache, wherein the mark threshold is used to indicate that the write to the memory cache needs to be granular according to the data.
  • the amount of data to be marked is used to indicate that the write to the memory cache needs to be granular according to the data.
  • the dirty data in the flash buffer is read out and written to the file system area, and the flash buffer is erased.
  • the flash buffer includes a first flash buffer and a second flash buffer; wherein
  • the second flash buffer When the first flash buffer is full, the second flash buffer is set to the current buffer for writing data when data synchronization is required; and the dirty data in the first flash buffer is read out at the same time It is written to the file system area and erases the first flash buffer;
  • the first flash buffer When the second flash buffer is full, the first flash buffer is set to the current buffer for writing data when data synchronization is required; at the same time, the dirty data in the second flash buffer is read and It is written to the file system area and erases the second flash buffer.
  • the marking the write data in the memory cache as dirty data includes: an index node number of the file corresponding to the write data, a page number where the data segment is located, an offset within the page, and data
  • the length of the segment and the content of the data segment are encapsulated into records, and the records are added to the preset dirty data list; and the reference count of the memory cache page corresponding to the written data is incremented by one.
  • the dirty data in the dirty data list is combined and written into the flash buffer, including:
  • the data management method further includes:
  • the flash file system and the data management method thereof according to the embodiment of the present invention, by marking the dirty data and merging the dirty data into the flash memory, avoid unnecessary data writing, thereby reducing the delay of the synchronization operation and improving The life of the flash memory;
  • FIG. 1 is a schematic structural diagram of a flash file system according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of a data structure of a flash file system according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a data structure of a merge record according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a data structure of a write flash buffer according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a data structure of a backfill operation according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of another flash file system according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a data structure of fault recovery according to an embodiment of the present invention.
  • FIG. 8 is a schematic flowchart of a data management method of a flash file system according to an embodiment of the present invention.
  • the terms “installation”, “connected”, and “connected” are to be understood broadly, and may be, for example, a fixed connection or a Removable connection, or integral connection; may be mechanical connection or electrical connection; may be directly connected, or may be indirectly connected through an intermediate medium, and may be internal communication between the two elements.
  • installation may be, for example, a fixed connection or a Removable connection, or integral connection; may be mechanical connection or electrical connection; may be directly connected, or may be indirectly connected through an intermediate medium, and may be internal communication between the two elements.
  • an embodiment of the present invention provides a flash file system, including: a creating module 11, a marking module 12, a synchronization module 13, and a backfill module 14;
  • Creating a module 11 configured to divide the flash into a file system area and a flash buffer when the file system is created;
  • the marking module 12 is configured to mark the write data as dirty data in the memory cache when the amount of data written and written is less than or equal to the preset mark threshold, wherein the mark threshold is used to indicate writing The amount of data that needs to be marked by the data granularity of the memory cache;
  • the synchronization module 13 is configured to merge all the dirty data in the memory cache or the dirty data of the file to be synchronized into the flash buffer when the data synchronization is needed, and notify the backfill module when the flash buffer is full;
  • the backfill module 14 is configured to receive the notification of the synchronization module, read the dirty data in the flash buffer and write it to the file system area, and erase the flash buffer.
  • the dirty data in the embodiment of the present invention refers to the data in the memory cache modified by the process.
  • the file system uses pages as the unit of memory cache.
  • the page is marked as dirty.
  • the write data is marked as dirty data in a granularity to avoid unnecessary data writing.
  • the size of the flash buffer is specified by a user or preset by the system.
  • the size of the flash buffer is specified by the user, when creating and mounting the file system, a separate area is divided on the flash device as a buffer according to the buffer size parameter passed by the user.
  • the file system performs physical space allocation, the allocated space is not within this flash buffer, so the flash buffer is not indexed by the file system.
  • the flash buffer includes a first flash buffer and a second flash buffer; wherein
  • the synchronization module is configured to: after all the dirty data in the memory cache or the dirty data of the file to be synchronized is merged and written into the first flash buffer when the data synchronization is needed; the first flash buffer is full Sending the first notification to the backfill module, and when data synchronization is required, merging all the dirty data in the memory cache or the dirty data of the file to be synchronized into the second flash buffer; the second flash buffer has When full, send the second notification to the backfill module, and when the data synchronization is needed, merge all the dirty data in the memory cache or the dirty data of the file to be synchronized into the first flash buffer;
  • the backfill module is configured to: when receiving the first notification of the synchronization module, read out the dirty data in the first flash buffer and write it to the file system area, and erase the first flash buffer; when receiving Upon the second notification of the synchronization module, the dirty data in the second flash buffer is read and written to the file system area, and the second flash buffer is erased.
  • the memory cache is a page cache.
  • the marking module 12 is further configured to perform processing according to an existing input/output (IO) path when data is written and the amount of data written is greater than a preset flag threshold.
  • IO input/output
  • the processing according to the existing IO path includes: writing the write data to the page cache, marking the page corresponding to the data as a dirty page, and then returning.
  • the marking module 12 is configured to: when the amount of data written and written is less than or equal to the preset marking threshold, the inode number of the file corresponding to the data to be written, and the data segment
  • the page number, the page offset, the length of the data segment, and the data segment are encapsulated into records, ie ⁇ inode number, page number, page offset , in the form of length, data>, adds the record to the preset dirty data list; and increments the reference count of the corresponding page cache page by one.
  • the marking module 12 of the embodiment of the present invention may mark dirty data by using a preset dirty data list, or may use other methods to mark dirty data.
  • the dirty data list can use any data structure, such as an array, a tree list, a linked list, and the like.
  • the markup module 12 adds the write data to the dirty data list, the corresponding page cache page is not marked as a dirty page, but the reference count of the corresponding page cache page is forced to be increased by one, thereby The write data in the page cache is not written to the flash device, forcing the portion of the page cache page to be for fast read.
  • the data segment content in the record may be a specific content of the data segment, or may be a pointer to a page cache corresponding page.
  • the tagging module 12 uses the base tree and linked list to organize and manage all records of the same file.
  • the base tree is used for easy retrieval, and the linked list is used to facilitate traversal.
  • the radix_tree is a less common data structure. It is a storage method of the Linux file system.
  • the tree structure mainly contains three pointers: the root pointer (root): the root node of the tree; the free pointer (free) : Point to the free node list; start pointer: points to the free memory block, each node in use is connected to each other using the parent, left, and right pointers, and the idle nodes are connected into a linked list by the right pointer.
  • An inode is a data structure in many Unix-like file systems. Each index node holds meta-information data for a file system object in the file system, but does not include data content or file names.
  • the tagging module 12 maintains a base tree indexed by an index node number, and the corresponding node represents a file; in order to facilitate traversal, all leaf nodes in the base tree are linked by using a linked list. stand up.
  • Each node in the base tree also maintains a base tree, indexed by page number, and each node represents a record of a page.
  • Each record consists of five elements: the inode number, the page number, the in-page offset value, the length of the data segment, and a pointer to the page corresponding to the page cache, ie ⁇ inode number, page number, offset, length, data pointer>
  • all records of the same file are also linked by a linked list.
  • the marking module 12 Upon receiving the write request, the marking module 12 is arranged to perform a search lookup in the base tree in FIG. 2 according to the index node number of the current write operation, and if a corresponding node is not found, create a new node and insert it In the linked list of the base tree and the link node; then in the base tree of the node, the page number involved in the write operation is indexed, and the corresponding record is searched; if the corresponding record is not found, a new record is created, and the index node is added.
  • New length max(old offset+old length,current offset+current length)
  • new offset indicates the in-page offset value of the new record
  • old offset indicates the in-page offset value of the original record
  • current offset indicates the in-page offset value of the current write operation
  • new length indicates the length of the data segment of the new record
  • old Length represents the length of the data segment of the original record
  • current length represents the length of the data segment of the current write operation
  • the synchronization module 13 is configured to: find all records of the file according to the index node number of the file corresponding to the write data, apply for a new memory page, and sequentially copy the contents of the multiple records to the new memory page. , the content of the new memory page is sequentially written to the flash buffer.
  • the synchronization module 13 finds a corresponding node according to the index node number in the base tree shown in FIG. 2, applies for a new memory page, and then traverses all of the nodes. Recording, for each page of the page cache data segment pointed to by the record, copy them from the page cache to the new memory page, and the current record information, including the index node number, page number, page offset The data segment length equivalent is also copied to the new memory page, and then the record structure is deleted from the data structure of FIG. 2; then, referring to FIG. 4, the contents of the new memory page are sequentially written. Go to the flash buffer; then repeat the above process until all the records belonging to the file have been processed.
  • the meaningless data is filled so that the content of the new memory page is full or memory An integer multiple of the page size.
  • the backfill module 14 is configured to: when receiving the notification of the synchronization module 13, first, the backfill module 14 traverses the data structure shown in FIG. 2, and traverses all of the nodes thereof. Record, each record points to a memory page in the page cache, marks all memory pages pointed to by the record as dirty, and decrements the reference count by one (forms a record when a write request is received, and records The reference count of the pointed memory page is incremented by one to force the memory page to be reserved; then the record is deleted from the data structure of Figure 2, and when all the records of a node are processed, the node is deleted from the base tree; When all nodes in the tree have been processed, the entire buffer is erased.
  • a recovery module 15 configured to detect whether there is dirty data in the flash buffer when the flash file system is restarted
  • the reply module 15 detects whether there is dirty data in the flash buffer; if there is dirty data in the flash buffer, all the records are read from the flash buffer. For each of the records, according to the index node number and the page number, read the corresponding data from the file system area (the data is stale data), and then copy the contents of the record to the page according to the offset value within the page. Cache the corresponding page; repeat the above process until all the records have been processed. At this point, the entire system has been restored to the latest state, and the failure recovery process ends.
  • the embodiment of the present invention further provides a data management method for a flash file system, including:
  • the flash memory is divided into a file system area and a flash buffer
  • S802 marks the write data as dirty data in the memory cache when the amount of data written and written is less than or equal to the preset mark threshold, wherein the mark threshold is used to indicate that the write to the memory cache is required to be pressed.
  • S803 combines all the dirty data in the memory cache or the dirty data of the file to be synchronized into the flash buffer when data synchronization is required;
  • the dirty data in the embodiment of the present invention refers to the data in the memory cache modified by the process.
  • the file system uses pages as the unit of memory cache.
  • the page is marked as dirty.
  • the write data is marked as dirty data in a granularity to avoid unnecessary data writing.
  • the size of the flash buffer is specified by a user or preset by the system.
  • the size of the flash buffer is specified by the user, when creating and mounting the file system, a separate area is divided on the flash device as a buffer according to the buffer size parameter passed by the user.
  • the file system performs physical space allocation, the allocated space is not within this flash buffer, so the flash buffer is not indexed by the file system.
  • the flash buffer includes a first flash buffer and a second flash buffer; wherein
  • the second flash buffer When the first flash buffer is full, the second flash buffer is set to the current buffer for writing data when data synchronization is required; and the dirty data in the first flash buffer is read and Write to the file system area and erase the first flash buffer;
  • the first flash buffer When the second flash buffer is full, the first flash buffer is set to the current buffer for writing data when data synchronization is required; and the dirty data in the second flash buffer is read and Write to the file system area and erase the second flash buffer.
  • another buffer acts as the current buffer, and the synchronization operations during the period are sequentially written to another buffer, so the entire buffer is not caused by the backfill.
  • the system stops and waits; the two buffers alternately use to ensure the normal operation of the system.
  • the data management method further comprises processing according to an existing input/output (IO) path when the amount of data written and written is greater than the mark threshold.
  • IO input/output
  • the processing according to the existing IO path includes: writing the write data to the page cache, marking the page corresponding to the data as a dirty page, and then returning.
  • the memory cache is a page cache.
  • the marking the write data in the memory cache as dirty data including: an inode number of a file corresponding to the write data, a page number where the data segment is located, and an in-page
  • the offset (page offset), the length of the data segment (length), and the data segment content (data) are encapsulated into a record, that is, the form of ⁇ inode number, page number, page offset, length, data>, which will be recorded.
  • dirty data may be marked by using a preset dirty data list, and other methods may be used to mark dirty data.
  • the dirty data list can use any data structure, such as an array, a tree list, a linked list, and the like.
  • the data management method of the embodiment of the present invention does not mark the corresponding page cache page as a dirty page when the write data is added to the dirty data list, but forces the corresponding page to cache the page.
  • the reference count is incremented by one so that the write data in the page cache is not written to the flash device, forcing the portion of the page cache page to be for fast read.
  • the data segment content in the record may be a specific content of the data segment, or may be a pointer to a page cache corresponding page.
  • the data management method uses a base tree and a linked list to organize and manage all records of the same file.
  • the base tree is used for easy retrieval, and the linked list is used to facilitate traversal.
  • the radix_tree is a less common data structure. It is a storage method of the Linux file system.
  • the tree structure mainly contains three pointers: the root pointer (root): the root node of the tree; the free pointer (free) : Point to the free node list; start pointer: points to the free memory block, each node in use is connected to each other using the parent, left, and right pointers, and the idle nodes are connected into a linked list by the right pointer.
  • An inode is a data structure in many Unix-like file systems. Each index node holds meta-information data for a file system object in the file system, but does not include data content or file names.
  • the file system of the embodiment of the present invention maintains a base tree A, and the base tree A is indexed by an index node number.
  • Node 101 represents file 1
  • node 102 represents file 2
  • node 103 represents file 3.
  • File 1 file 2, and file 3 each maintain a base tree, which is called base tree B1, base tree B2, and base tree B3.
  • the base tree B1 is indexed by page number, node 1011 represents record 1, node 1012 represents record 2, node 1013 represents record 3, node 1014 represents record 4, and node 1015 represents record 5.
  • Each record consists of five elements: the inode number, the page number, the in-page offset value, the length of the data segment, and a pointer to the corresponding page of the page cache, ie ⁇ inode number, page number, offset, length, data pointer>
  • the file system of the embodiment of the present invention maintains a base tree, where the base tree is indexed by an index node number, and the corresponding node 101 represents a file; in order to facilitate traversal, all the base trees are included. Node 101 is linked using a linked list.
  • Each node in the base tree also maintains a base tree, indexed by page number, and each node represents a record of a page.
  • Each record consists of five elements: the inode number, the page number, the in-page offset value, the length of the data segment, and a pointer to the page corresponding to the page cache, ie ⁇ inode number, page number, offset, length, data pointer>
  • all records of the same file are also linked by a linked list.
  • the search node number of the current write operation is searched and searched in the base tree in FIG. 2. If the corresponding node is not found, a new node is created and inserted into the base tree and the link node. In the linked list; then in the base tree of the node, the page number involved in the write operation is indexed, and the corresponding record is searched; if the corresponding record is not found, a new record is created, and the index node number, page number, and page are The internal offset value, the data segment length, and the memory page pointer are assigned accordingly; if the corresponding record is found, the two records need to be merged, the inode number, the page number, and the memory page pointer are unchanged, and the offset value within the page is The length of the data segment has the following updates:
  • New length max(old offset+old length,current offset+current length)– new offset
  • new offset indicates the in-page offset value of the new record
  • old offset indicates the in-page offset value of the original record
  • current offset indicates the in-page offset value of the current write operation
  • new length indicates the length of the data segment of the new record
  • old Length represents the length of the data segment of the original record
  • current length represents the length of the data segment of the current write operation
  • the dirty data of the file to be synchronized is merged and written into the flash buffer, including: searching all records of the file according to the index node number of the file corresponding to the written data, applying for a new memory page, and recording multiple records.
  • the content is sequentially copied to the new memory page, and the content of the new memory page is sequentially written to the flash buffer.
  • the corresponding node when data synchronization is required, the corresponding node is found according to the index node number in the base tree shown in FIG. 2, a new memory page is requested, and then all records of the node are traversed.
  • Each page records a page cache data segment, copies them from the page cache to the new memory page, and records the current record, including the inode number, page number, in-page offset, data segment
  • the length equivalent is also copied to the new memory page, and then the record structure is deleted from the data structure of FIG. 2; then, referring to FIG. 4, the contents of the new memory page are sequentially written to the flash buffer. Zone; then repeat the above process until all records belonging to the file have been processed.
  • the meaningless data is filled so that the content of the new memory page is full or memory An integer multiple of the page size.
  • the data structure shown in FIG. 2 is traversed, and for each of the nodes, all the records are traversed, and each record points to the page cache.
  • To force the memory page to be written write the dirty page to the file system area; then delete the record from the data structure of Figure 2, when all the records of a node are processed, delete the node from the base tree; When all nodes in the base tree have been processed, the entire buffer is erased.
  • the data management method further includes: detecting whether there is dirty data in the flash buffer when the flash file system is restarted;
  • the flash file system and the data management method thereof according to the embodiment of the present invention, by marking the dirty data and merging the dirty data into the flash memory, avoid unnecessary data writing, thereby reducing the delay of the synchronization operation and improving The life of the flash memory;
  • another buffer acts as the current buffer, and the synchronous operations during the period are sequentially written to another buffer.
  • the two buffers are alternately used to ensure the normal operation of the system.
  • the embodiment of the invention further provides a computer readable storage medium storing computer executable instructions, which are implemented by the processor to implement the method described in the foregoing embodiments.
  • computer storage medium includes volatile and nonvolatile, implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules, or other data. , removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or may Any other medium used to store the desired information and that can be accessed by the computer.
  • communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media.
  • the embodiment of the invention avoids unnecessary data writing, thereby reducing the delay of the synchronous operation and improving the service life of the flash memory; further, the two buffers are alternately used to ensure the normal operation of the system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Memory System Of A Hierarchy Structure (AREA)
  • Memory System (AREA)

Abstract

一种闪存文件系统及其数据管理方法，包括创建模块(11)，设置为在创建文件系统时，将闪存划分成文件系统区和闪存缓冲区；标记模块(12)，设置为在有数据写入且写入的数据量小于或等于预设标记阈值时，在内存缓存中将写入数据标记为脏数据，标记阈值用于表示写入内存缓存的需要按数据粒度进行标记的数据量大小；同步模块(13)，设置为在需要进行数据同步时，将内存缓存中的脏数据合并后写入闪存缓冲区，并当闪存缓冲区已满时，通知回填模块(14)；回填模块(14)，设置为接收到同步模块的通知，读出闪存缓冲区中的脏数据并将其写入文件系统区，并擦除闪存缓冲区。

Description

一种闪存文件系统及其数据管理方法 技术领域
本文涉及但不限于存储技术领域，具体涉及一种闪存文件系统及其数据管理方法。
背景技术
闪存(Flash Memory)是一种电子式可擦除编程存储器，与传统的磁盘介质相比，闪存具有读写带宽高、访问延迟低、功耗低、稳定性强的特点，目前，闪存已经开始在数据中心、个人电脑、移动设备上普及。闪存以页为单位进行读写，当闪存重写一个页之前，需要先进行擦除操作。闪存以块为单位进行擦除，一个闪存块中包含几百个闪存页。闪存的单元具有有限次的擦写操作，即每个闪存单元具有有限的寿命。
在文件系统中，使用页高速缓存来缓存最近操作的数据，以加速读写过程。当需要读取数据时，首先在页高速缓存中查找该部分内容是否驻留在内存中，如果找到，则直接返回数据；如果没找到，再到闪存中进行读取。当需要进行写操作时，不再直接把数据写入到设备中，而是把数据写入到页高速缓存后将相应页面标记为脏页，然后直接返回。当用户发出同步调用或操作系统后台线程启动同步操作时，将页高速缓存中的脏页写入到闪存设备中。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本发明实施例提供一种闪存文件系统及其数据管理方法，能够避免不必要的数据写入。
本发明实施例的技术方案是这样实现的：
本发明实施例提供了一种闪存文件系统，包括：创建模块、标记模块、同步模块和回填模块；其中，
创建模块，设置为在创建文件系统时，将闪存划分成文件系统区和闪存缓冲区；
标记模块，设置为在有数据写入且写入的数据量小于或等于预设标记阈值时，在内存缓存中将写入数据标记为脏数据，其中，所述标记阈值用于表示写入内存缓存的需要按数据粒度进行标记的数据量大小；
同步模块，设置为在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入闪存缓冲区，并当闪存缓冲区已满时，通知回填模块；
回填模块，设置为接收到同步模块的通知，读出闪存缓冲区中的脏数据并将其写入文件系统区，并擦除闪存缓冲区。
可选地，所述闪存缓冲区包括第一闪存缓冲区和第二闪存缓冲区；其中，
所述同步模块是设置为：在需要进行数据同步时，将所述内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；当第一闪存缓冲区已满时，发送第一通知至回填模块，并在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第二闪存缓冲区；第二闪存缓冲区已满时，发送第二通知至回填模块，并在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；
所述回填模块是设置为：当接收到所述同步模块的第一通知时，读出第一闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第一闪存缓冲区；当接收到同步模块的第二通知时，读出第二闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第二闪存缓冲区。
可选地，所述标记模块是设置为：在有数据写入且写入的数据量小于或等于所述标记阈值时，将所述写入数据对应的文件的索引节点号、数据段所在的页面号、页面内的偏移量、数据段的长度以及数据段内容 封装成记录，将记录添加至预设的脏数据列表；并将写入数据对应的内存缓存页面的引用计数加一。
可选地，所述同步模块是设置为：
根据所述写入数据对应的文件的索引节点号查找该文件的所有记录，申请新的内存页面，将多个记录的内容顺序拷贝至所述新的内存页面，将所述新的内存页面的内容顺序写入到闪存缓冲区。
可选地，所述系统还包括：恢复模块，设置为在闪存文件系统重启时，检测闪存缓冲区中是否有脏数据；
如果闪存缓冲区中有脏数据，读出闪存缓冲区的所有脏数据，根据每条脏数据更新所述内存缓存的内容。
本发明实施例还提供了一种闪存文件系统的数据管理方法，包括：
在创建文件系统时，将闪存划分成文件系统区和闪存缓冲区；
在有数据写入且写入的数据量小于等于预设标记阈值时，在内存缓存中将写入数据标记为脏数据，其中，所述标记阈值用于表示写入内存缓存的需要按数据粒度进行标记的数据量大小；
在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入闪存缓冲区；
当闪存缓冲区已满时，读出闪存缓冲区中的脏数据并将其写入文件系统区，并擦除闪存缓冲区。
可选地，所述闪存缓冲区包括第一闪存缓冲区和第二闪存缓冲区；其中，
在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；
当第一闪存缓冲区已满时，将第二闪存缓冲区设成当前缓冲区，用于在需要进行数据同步时，写入数据使用；同时读出第一闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第一闪存缓冲区；
当第二闪存缓冲区已满时，将第一闪存缓冲区设成当前缓冲区，用 于在需要进行数据同步时，写入数据使用；同时读出第二闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第二闪存缓冲区。
可选地，所述在内存缓存中将写入数据标记为脏数据，包括：将所述写入数据对应的文件的索引节点号、数据段所在的页面号、页面内的偏移量、数据段的长度以及数据段内容封装成记录，将记录添加至预设的脏数据列表；并将写入数据对应的内存缓存页面的引用计数加一。
可选地，所述将脏数据列表中的脏数据合并后写入闪存缓冲区，包括：
根据写入数据对应的文件的索引节点号查找该文件的所有记录，申请新的内存页面，将多个记录的内容顺序拷贝至所述新的内存页面，将所述新的内存页面的内容顺序写入到闪存缓冲区。
可选地，所述的数据管理方法，还包括：
在闪存文件系统重启时，检测闪存缓冲区中是否有脏数据；
如果闪存缓冲区中有脏数据，读出闪存缓冲区的所有脏数据，根据每条脏数据更新所述内存缓存的内容。
本发明实施例的闪存文件系统及其数据管理方法，通过对脏数据进行标记，并将脏数据合并后写入闪存，避免了不必要的数据写入，从而降低了同步操作的延时，提高了闪存的使用寿命；
进一步地，通过设置第一闪存缓冲区和第二闪存缓冲区，在系统回填其中一个闪存缓冲区期间，由另一缓冲区充当当前缓冲区，期间的同步操作均顺序写入到另一缓冲区中，因此不会因为回填造成整个系统停下来等待的情况；两块缓冲区轮流交替使用，保证了系统的正常运转。
在阅读并理解了附图和详细描述后，可以明白其他方面。
附图概述
图1是本发明实施例提供的一种闪存文件系统的结构示意图；
图2是本发明实施例提供的闪存文件系统的数据结构示意图；
图3是本发明实施例的合并记录的数据结构示意图；
图4是本发明实施例的写入闪存缓冲区的数据结构示意图；
图5是本发明实施例的回填操作的数据结构示意图；
图6是本发明实施例提供的另一种闪存文件系统的结构示意图；
图7是本发明实施例的故障恢复的数据结构示意图；
图8是本发明实施例提供的闪存文件系统的数据管理方法的流程示意图。
本发明的实施方式
在本发明实施例的描述中，需要理解的是，术语“中心”、“纵向”、“横向”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明实施例和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性。
在本发明实施例的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以依据具体情况理解上述术语在本发明实施例中的含义。
参照下面的描述和附图，将清楚本发明的实施例的这些和其他方面。在这些描述和附图中，公开了本发明的实施例中的一些特定实施方式，来表示实施本发明的实施例的原理的一些方式，但是应当理解，本发明的实施例的范围不受此限制。相反，本发明的实施例包括落入所附加权利要求书的精神和内涵范围内的所有变化、修改和等同物。
由于一次写操作将整个页面标记为脏页，这样，即使本次写操作仅仅涉 及到该页面的很少一部分，当执行同步操作时，仍会将整个页面写入到闪存设备中。这使得写入数据量大大增加，不仅增加了同步操作的延时，降低了系统的性能，而且增加了闪存设备的磨损，大大降低了其寿命。
基于此，如图1所示，本发明实施例提出了一种闪存文件系统，包括：创建模块11、标记模块12、同步模块13和回填模块14；其中，
创建模块11，设置为在创建文件系统时，将闪存划分成文件系统区和闪存缓冲区；
标记模块12，设置为在有数据写入且写入的数据量小于或等于预设标记阈值时，在内存缓存中将写入数据标记为脏数据，其中，所述标记阈值用于表示写入内存缓存的需要按数据粒度进行标记的数据量大小；
同步模块13，设置为在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入闪存缓冲区，并当闪存缓冲区已满时，通知回填模块；
回填模块14，设置为接收到同步模块的通知，读出闪存缓冲区中的脏数据并将其写入文件系统区，并擦除闪存缓冲区。
需要说明的是，本发明实施例所述的脏数据，指的是被进程修改了的内存缓存里的数据。文件系统以页作为内存缓存的单位，当进程修改了内存缓存里的数据时，该页就被标记为脏页。本发明实施例以字节为粒度，将所述写入数据标记为脏数据，以避免不必要的数据写入。
可选地，所述闪存缓冲区的大小，由用户指定或者系统预先设置。
可选地，如果由用户指定闪存缓冲区的大小，在创建和挂载文件系统时，根据用户传递的缓冲区大小参数，在闪存设备上划分出一块单独的区域充当缓冲区。文件系统进行物理空间分配时，分配的空间都不在此闪存缓冲区之内，因此闪存缓冲区不会被文件系统所索引。
可选地，所述闪存缓冲区包括第一闪存缓冲区和第二闪存缓冲区；其中，
所述同步模块是设置为：在需要进行数据同步时，将所述内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；第 一闪存缓冲区已满时，发送第一通知至回填模块，并在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第二闪存缓冲区；第二闪存缓冲区已满时，发送第二通知至回填模块，并在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；
所述回填模块是设置为：当接收到同步模块的第一通知时，读出第一闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第一闪存缓冲区；当接收到同步模块的第二通知时，读出第二闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第二闪存缓冲区。
通过设置两块闪存缓冲区，在系统回填其中一个闪存缓冲区期间，由另一缓冲区充当当前缓冲区，期间的同步操作均顺序写入到另一缓冲区中，因此不会因为回填造成整个系统停下来等待的情况；两块缓冲区轮流交替使用，保证了系统的正常运转。
可选地，所述内存缓存为页高速缓存。
可选地，所述标记模块12还设置为，在有数据写入且写入的数据量大于预设标记阈值时，按现有的输入输出(IO)路径进行处理。
在本发明的一实施例中，所述按现有的IO路径进行处理，包括：将写入数据写入页高速缓存，将数据对应的页面标记为脏页，然后返回。
可选地，所述标记阈值的大小，可以根据具体的加速读取过程设置。例如，可以将标记阈值设置为内存页面大小的一半(4096*50％＝2048字节)，或者为内存页面大小的80％(4096*80％＝3276.8字节)。
可选地，标记模块12是设置为：在有数据写入且写入的数据量小于或等于预设标记阈值时，将写入数据对应的文件的索引节点号(inode number)、数据段所在的页面号(page number)、页面内的偏移量(page offset)、数据段的长度(length)以及数据段内容(data)封装成记录(record)，即<inode number，page number，page offset，length，data>的形式，将记录添加至预设的脏数据列表；并将相应的页高速缓存页面的引用计数加一。
需要说明的是，本发明实施例的标记模块12既可以使用预设的脏数据 列表来标记脏数据，也可以使用其他的方法来标记脏数据。所述的脏数据列表可以使用任意的数据结构，如数组、树形列表、链表等。
值得注意的是，标记模块12将写入数据添加至脏数据列表时，并不将相应的页高速缓存页面标记为脏页，而是强制将相应的页高速缓存页面的引用计数加一，从而使得页高速缓存中的写入数据不被写入到闪存设备上，强制保留这部分页高速缓存页面，以供快速读取。
可选地，所述记录中的数据段内容可以是数据段的具体内容，也可以是一个指向页高速缓存对应页面的指针(data pointer)。
可选地，标记模块12使用基树和链表来组织和管理同一个文件的所有记录。使用基树是为了方便检索，使用链表是为了方便遍历。基树(radix_tree)是一种不太常用的数据结构，是Linux文件系统的一种存储方式，树结构主要包含三个指针：根指针(root)：指向树的根节点；空闲指针(free)：指向空闲的节点链表；开始指针(start)：指向空闲的内存块，每个使用中的节点使用parent、left、right指针相互连接，空闲的节点通过right指针连成一个链表。索引节点(inode)是在许多类Unix文件系统中的一种数据结构，每个索引节点保存了文件系统中的一个文件系统对象的元信息数据，但不包括数据内容或者文件名。
如图2所示，标记模块12维护了一棵基树，所述基树以索引节点号为索引，相应的节点代表了一个文件；为了方便遍历，将基树中的所有叶子节点使用链表链接起来。该基树中的每一个节点也维持了一棵基树，以页面号为索引，每个节点表示某个页面的记录。每条记录包括五个元素：索引节点号、页面号、页内偏移值、数据段长度和一个指向页高速缓存对应页面的指针，即<inode number，page number，offset，length，data pointer>的形式，为了方便遍历，同一文件的所有记录也用链表链接起来。
在接收到写入请求时，标记模块12设置为根据本次写操作的索引节点号在图2中的基树中进行搜索查找，如果没有找到相应的节点，则新建一个节点，并将其插入基树和链接节点的链表中；然后在该节点的基树中以本次写操作涉及的页面号为索引，查找相应的记录；如果没有找到相应的记录，则新建一条记录，将其索引节点号、页面号、页内偏移值、数据段长度和内 存页面指针相应赋值；如果找到了相应的记录，则两条记录需要进行合并，索引节点号、页面号和内存页面指针不变，对页内偏移值和数据段长度，有如下更新：
new offset＝min(old offset，current offset)
new length＝max(old offset+old length，current offset+current length)–new offset
其中new offset表示新纪录的页内偏移值，old offset表示原纪录的页内偏移值，current offset表示当前写入操作的页内偏移值，new length表示新纪录的数据段长度，old length表示原纪录的数据段长度，current length表示当前写入操作的数据段长度。根据以上情况计算得到新纪录的一个或多个项值以后，将其插入基树和链表中。然后仍然对页高速缓存中的内存页面进行更新，但是为了防止文件系统将该内存页面整页写入闪存设备，系统不再将该页面标记为脏。由于页面不再标记为脏，随时有被文件系统回收的危险，为了维护数据的高效读取和一致性，强制将内存页面的引用计数加一，从而强制保证其不被回收。这样，后续的读取操作仍然通过页高速缓存中的页面读取。
可选地，同步模块13是设置为：根据写入数据对应的文件的索引节点号查找该文件的所有记录，申请新的内存页面，将多个记录的内容顺序拷贝至所述新的内存页面，将所述新的内存页面的内容顺序写入到闪存缓冲区。
可选地，如图3所示，在需要进行数据同步时，同步模块13在图2所示的基树中根据索引节点号找到相应的节点，申请新的内存页面，然后遍历该节点的所有记录，对于每一条记录所指向的页高速缓存数据段，把它们从页高速缓存中拷贝到所述新的内存页面，并把当前记录的信息，包括索引节点号、页面号、页面内偏移、数据段长度等值也一并拷贝到所述新的内存页面，然后将记录结构从图2的数据结构中删除；然后，参照图4，将所述新的内存页面中的内容顺序写入到闪存缓冲区；接着重复上述过程，直到属于该文件的所有记录处理完毕。
可选地，当写入到闪存缓冲区的新的内存页面的内容不满一页或不是内存页面大小的整数倍时，填充无意义数据，以使得新的内存页面的内容满一页或是内存页面大小的整数倍。
可选地，参照图5，回填模块14是设置为：当接收到同步模块13的通知时，首先，回填模块14遍历图2所示的数据结构，对其中的每一个节点，再遍历其所有的记录，每一个记录指向页高速缓存中的一个内存页面，将这些由记录指向的内存页面全都标记为脏，并将引用计数减一(在接收到写入请求时，形成记录，并将记录指向的内存页面的引用计数加一，以强制保留内存页面)；然后从图2的数据结构中删除该记录，当一个节点的所有记录处理完毕时，将该节点从基树中删除；当基树中的所有节点都处理完毕时，将整个缓冲区擦除。
可选地，参照图6，还包括恢复模块15，设置为在闪存文件系统重启时，检测闪存缓冲区中是否有脏数据；
如果闪存缓冲区中有脏数据，读出闪存缓冲区的所有脏数据，根据每条脏数据更新所述内存缓存的内容。
当发生突然掉电等意外事件时，需要进行系统故障恢复。可选地，如图7所示，在闪存文件系统重启时，回复模块15检测闪存缓冲区中是否有脏数据；如果闪存缓冲区中有脏数据，从闪存缓冲区中读出所有的记录，对于其中的每一条记录，根据其中的索引节点号和页面号，从文件系统区读出相应的数据(该数据为陈旧数据)，然后根据页内偏移值，将记录中的内容拷贝到页高速缓存相应的页面；重复上述过程直到所有的记录都处理完毕。此时整个系统已经恢复到最新状态，故障恢复过程结束。
如图8所示，本发明实施例还提供了一种闪存文件系统的数据管理方法，包括：
S801在创建文件系统时，将闪存划分成文件系统区和闪存缓冲区；
S802在有数据写入且写入的数据量小于或等于预设标记阈值时，在内存缓存中将写入数据标记为脏数据，其中，所述标记阈值用于表示写入内存缓存的需要按数据粒度进行标记的数据量大小；
S803在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入闪存缓冲区；
S804当闪存缓冲区已满时，读出闪存缓冲区中的脏数据并将其写入文件系统区，并擦除闪存缓冲区。
需要说明的是，本发明实施例所述的脏数据，指的是被进程修改了的内存缓存里的数据。文件系统以页作为内存缓存的单位，当进程修改了内存缓存里的数据时，该页就被标记为脏页。本发明实施例以字节为粒度，将所述写入数据标记为脏数据，以避免不必要的数据写入。
可选地，所述闪存缓冲区的大小，由用户指定或者系统预先设置。
可选地，如果由用户指定闪存缓冲区的大小，在创建和挂载文件系统时，根据用户传递的缓冲区大小参数，在闪存设备上划分出一块单独的区域充当缓冲区。文件系统进行物理空间分配时，分配的空间都不在此闪存缓冲区之内，因此闪存缓冲区不会被文件系统所索引。
可选地，所述闪存缓冲区包括第一闪存缓冲区和第二闪存缓冲区；其中，
在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；
当第一闪存缓冲区已满时，将第二闪存缓冲区设成当前缓冲区，用于需要进行数据同步时，写入数据使用；同时读出第一闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第一闪存缓冲区；
当第二闪存缓冲区已满时，将第一闪存缓冲区设成当前缓冲区，用于需要进行数据同步时，写入数据使用；同时读出第二闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第二闪存缓冲区。通过设置两块闪存缓冲区，在系统回填其中一个闪存缓冲区期间，由另一缓冲区充当当前缓冲区，期间的同步操作均顺序写入到另一缓冲区中，因此不会因为回填造成整个系统停下来等待的情况；两块缓冲区轮流交替使用，保证了系统的正常运转。
可选地，所述数据管理方法还包括，在有数据写入且写入的数据量大 于标记阈值时，按现有的输入输出(IO)路径进行处理。
在本发明的一实施例中，所述按现有的IO路径进行处理，包括：将写入数据写入页高速缓存，将数据对应的页面标记为脏页，然后返回。
可选地，所述内存缓存为页高速缓存。
可选地，所述标记阈值的大小，可以根据具体的加速读取过程设置。例如，可以将标记阈值设置为内存页面大小的一半(4096*50％＝2048字节)，或者为内存页面大小的80％(4096*80％＝3276.8字节)。
可选地，所述在内存缓存中将写入数据标记为脏数据，包括：将写入数据对应的文件的索引节点号(inode number)、数据段所在的页面号(page number)、页面内的偏移量(page offset)、数据段的长度(length)以及数据段内容(data)封装成记录(record)，即<inode number，page number，page offset，length，data>的形式，将记录添加至预设的脏数据列表；并将相应的页高速缓存页面的引用计数加一。
需要说明的是，本发明实施例既可以使用预设的脏数据列表来标记脏数据，也可以使用其他的方法来标记脏数据。所述的脏数据列表可以使用任意的数据结构，如数组、树形列表、链表等。
值得注意的是，本发明实施例的数据管理方法，将写入数据添加至脏数据列表时，并不将相应的页高速缓存页面标记为脏页，而是强制将相应的页高速缓存页面的引用计数加一，从而使得页高速缓存中的写入数据不被写入到闪存设备上，强制保留这部分页高速缓存页面，以供快速读取。
可选地，所述记录中的数据段内容可以是数据段的具体内容，也可以是一个指向页高速缓存对应页面的指针(data pointer)。
可选地，所述数据管理方法使用基树和链表来组织和管理同一个文件的所有记录。使用基树是为了方便检索，使用链表是为了方便遍历。基树(radix_tree)是一种不太常用的数据结构，是Linux文件系统的一种存储方式，树结构主要包含三个指针：根指针(root)：指向树的根节点；空闲指针(free)：指向空闲的节点链表；开始指针(start)：指向空闲的内存块，每个使用中的节点使用parent、left、right指针相互连接，空闲的节点通过 right指针连成一个链表。索引节点(inode)是在许多类Unix文件系统中的一种数据结构，每个索引节点保存了文件系统中的一个文件系统对象的元信息数据，但不包括数据内容或者文件名。
如图2所示，本发明实施例的文件系统维护了一棵基树A，所述基树A以索引节点号为索引。节点101代表文件1，节点102代表文件2，节点103代表文件3。文件1、文件2、文件3各自维护了一棵基树，分别称为基树B1、基树B2、基树B3。以基树B1为例：所述基树B1以页面号为索引，节点1011代表记录1，节点1012代表记录2，节点1013代表记录3，节点1014代表记录4，节点1015代表记录5。
每条记录包括5个元素：索引节点号、页面号、页内偏移值、数据段长度和一个指向页高速缓存对应页面的指针，即<inode number，page number，offset，length，data pointer>的形式，为了方便遍历，同一文件的所有记录也用链表链接起来。如图2所示，本发明实施例的文件系统维护了一棵基树，所述基树以索引节点号为索引，相应的节点101代表了一个文件；为了方便遍历，将基树中的所有节点101使用链表链接起来。该基树中的每一个节点也维持了一棵基树，以页面号为索引，每个节点表示某个页面的记录。每条记录包括五个元素：索引节点号、页面号、页内偏移值、数据段长度和一个指向页高速缓存对应页面的指针，即<inode number，page number，offset，length，data pointer>的形式，为了方便遍历，同一文件的所有记录也用链表链接起来。
在有数据写入时，根据本次写操作的索引节点号在图2中的基树中进行搜索查找，如果没有找到相应的节点，则新建一个节点，并将其插入基树和链接节点的链表中；然后在该节点的基树中以本次写操作涉及的页面号为索引，查找相应的记录；如果没有找到相应的记录，则新建一条记录，将其索引节点号、页面号、页内偏移值、数据段长度和内存页面指针相应赋值；如果找到了相应的记录，则两条记录需要进行合并，索引节点号、页面号和内存页面指针不变，对页内偏移值和数据段长度，有如下更新：
new offset＝min(old offset，current offset)
new length＝max(old offset+old length，current offset+current length)– new offset
其中new offset表示新纪录的页内偏移值，old offset表示原纪录的页内偏移值，current offset表示当前写入操作的页内偏移值，new length表示新纪录的数据段长度，old length表示原纪录的数据段长度，current length表示当前写入操作的数据段长度。根据以上情况计算得到新纪录的一个或多个项值以后，将其插入基树和链表中。然后仍然对页高速缓存中的内存页面进行更新，但是为了防止文件系统将该内存页面整页写入闪存设备，系统不再将该页面标记为脏。由于页面不再标记为脏，随时有被文件系统回收的危险，为了维护数据的高效读取和一致性，强制将内存页面的引用计数加一，从而强制保证其不被回收。这样，后续的读取操作仍然通过页高速缓存中的页面读取。
可选地，将要同步的文件的脏数据合并后写入闪存缓冲区，包括：根据写入数据对应的文件的索引节点号查找该文件的所有记录，申请新的内存页面，将多个记录的内容顺序拷贝至所述新的内存页面，将所述新的内存页面的内容顺序写入到闪存缓冲区。
可选地，如图3所示，在需要进行数据同步时，在图2所示的基树中根据索引节点号找到相应的节点，申请新的内存页面，然后遍历该节点的所有记录，对于每一条记录所指向的页高速缓存数据段，把它们从页高速缓存中拷贝到所述新的内存页面，并把当前记录的信息，包括索引节点号、页面号、页面内偏移、数据段长度等值也一并拷贝到所述新的内存页面，然后将记录结构从图2的数据结构中删除；然后，参照图4，将所述新的内存页面中的内容顺序写入到闪存缓冲区；接着重复上述过程，直到属于该文件的所有记录处理完毕。
可选地，当写入到闪存缓冲区的新的内存页面的内容不满一页或不是内存页面大小的整数倍时，填充无意义数据，以使得新的内存页面的内容满一页或是内存页面大小的整数倍。
可选地，当当前缓冲区已满时，如图5所示，首先，遍历图2所示的数据结构，对其中的每一个节点，再遍历其所有的记录，每一个记录指向页高速缓存中的一个内存页面，将这些由记录指向的内存页面全都标记为脏页， 并将引用计数减一(在接收到写入请求时，形成记录，并将记录指向的内存页面的引用计数加一，以强制保留内存页面)，将脏页写入文件系统区中；然后从图2的数据结构中删除该记录，当一个节点的所有记录处理完毕时，将该节点从基树中删除；当基树中的所有节点都处理完毕时，将整个缓冲区擦除。
可选地，所述数据管理方法还包括，在闪存文件系统重启时，检测闪存缓冲区中是否有脏数据；
如果闪存缓冲区中有脏数据，读出闪存缓冲区的所有脏数据，根据每条脏数据更新所述内存缓存的内容。
当发生突然掉电等意外事件时，需要进行系统故障恢复。可选地，如图7所示，在闪存文件系统重启时，检测闪存缓冲区中是否有脏数据；如果闪存缓冲区中有脏数据，从闪存缓冲区中读出所有的记录，对于其中的每一条记录，根据其中的索引节点号和页面号，从文件系统索引区读出相应的数据(该数据为陈旧数据)，然后根据页内偏移值，将记录中的内容拷贝到页高速缓存相应的页面；重复上述过程直到所有的记录都处理完毕。此时整个系统已经恢复到最新状态，故障恢复过程结束。
本发明实施例的闪存文件系统及其数据管理方法，通过对脏数据进行标记，并将脏数据合并后写入闪存，避免了不必要的数据写入，从而降低了同步操作的延时，提高了闪存的使用寿命；
可选地，通过设置第一闪存缓冲区和第二闪存缓冲区，在系统回填其中一个闪存缓冲区期间，由另一缓冲区充当当前缓冲区，期间的同步操作均顺序写入到另一缓冲区中，因此不会因为回填造成整个系统停下来等待的情况；两块缓冲区轮流交替使用，保证了系统的正常运转。
本发明实施例还提供了一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现上述实施例所述的方法。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步 骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理单元的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些组件或所有组件可以被实施为由处理器，如数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
以上所述仅为本发明的可选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。本领域普通技术人员可以理解上述方法中的全部或部分步骤可通过程序来指令相关硬件完成，所述程序可以存储于计算机可读存储介质中，如只读存储器、磁盘或光盘等。可选地，上述实施例的全部或部分步骤也可以使用一个或多个集成电路来实现，相应地，上述实施例中的各模块/模块可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。本申请不限制于任何特定形式的硬件和软件的结合。
工业实用性
本发明实施例避免了不必要的数据写入，从而降低了同步操作的延时， 提高了闪存的使用寿命；进一步地，两块缓冲区轮流交替使用，保证了系统的正常运转。

Claims (11)

  1. 一种闪存文件系统，包括：创建模块(11)、标记模块(12)、同步模块(13)和回填模块(14)；其中，
    创建模块(11)，设置为在创建文件系统时，将闪存划分成文件系统区和闪存缓冲区；
    标记模块(12)，设置为在有数据写入且写入的数据量小于或等于预设标记阈值时，在内存缓存中将写入数据标记为脏数据，其中，所述标记阈值用于表示写入内存缓存的需要按数据粒度进行标记的数据量大小；
    同步模块(13)，设置为在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入闪存缓冲区，并当闪存缓冲区已满时，通知回填模块(14)；
    回填模块(14)，设置为接收到同步模块(13)的通知，读出闪存缓冲区中的脏数据并将其写入文件系统区，并擦除闪存缓冲区。
  2. 根据权利要求1所述的闪存文件系统，其中：所述闪存缓冲区包括第一闪存缓冲区和第二闪存缓冲区；其中，
    所述同步模块(13)是设置为：在需要进行数据同步时，将所述内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；当第一闪存缓冲区已满时，发送第一通知至回填模块(14)，并在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第二闪存缓冲区；第二闪存缓冲区已满时，发送第二通知至回填模块(14)，并在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；
    所述回填模块(14)是设置为：当接收到所述同步模块(13)的第一通知时，读出第一闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第一闪存缓冲区；当接收到同步模块(13)的第二通知时，读出第二闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第二闪存缓冲区。
  3. 根据权利要求1所述的闪存文件系统，其中：所述标记模块(12)是设置为：在有数据写入且写入的数据量小于或等于所述标记阈值时，将所 述写入数据对应的文件的索引节点号、数据段所在的页面号、页面内的偏移量、数据段的长度以及数据段内容封装成记录，将记录添加至预设的脏数据列表；并将写入数据对应的内存缓存页面的引用计数加一。
  4. 根据权利要求3所述的闪存文件系统，其中：所述同步模块(13)是设置为：
    根据所述写入数据对应的文件的索引节点号查找该文件的所有记录，申请新的内存页面，将多个记录的内容顺序拷贝至所述新的内存页面，将所述新的内存页面的内容顺序写入到闪存缓冲区。
  5. 根据权利要求1所述的闪存文件系统，所述系统还包括：
    恢复模块(15)，设置为在闪存文件系统重启时，检测闪存缓冲区中是否有脏数据；如果闪存缓冲区中有脏数据，读出闪存缓冲区的所有脏数据，根据每条脏数据更新所述内存缓存的内容。
  6. 一种闪存文件系统的数据管理方法，包括：
    在创建文件系统时，将闪存划分成文件系统区和闪存缓冲区(S801)；
    在有数据写入且写入的数据量小于等于预设标记阈值时，在内存缓存中将写入数据标记为脏数据，其中，所述标记阈值用于表示写入内存缓存的需要按数据粒度进行标记的数据量大小(S802)；
    在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入闪存缓冲区(S803)；
    当闪存缓冲区已满时，读出闪存缓冲区中的脏数据并将其写入文件系统区，并擦除闪存缓冲区(S804)。
  7. 根据权利要求6所述的数据管理方法，其中：所述闪存缓冲区包括第一闪存缓冲区和第二闪存缓冲区；其中，
    在需要进行数据同步时，将内存缓存中的所有脏数据或要同步的文件的脏数据合并后写入第一闪存缓冲区；
    当第一闪存缓冲区已满时，将第二闪存缓冲区设成当前缓冲区，用于在需要进行数据同步时，写入数据使用；同时读出第一闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第一闪存缓冲区；
    当第二闪存缓冲区已满时，将第一闪存缓冲区设成当前缓冲区，用于在需要进行数据同步时，写入数据使用；同时读出第二闪存缓冲区中的脏数据并将其写入文件系统区，并擦除第二闪存缓冲区。
  8. 根据权利要求6所述的数据管理方法，其中：所述在内存缓存中将写入数据标记为脏数据，包括：将所述写入数据对应的文件的索引节点号、数据段所在的页面号、页面内的偏移量、数据段的长度以及数据段内容封装成记录，将记录添加至预设的脏数据列表；并将写入数据对应的内存缓存页面的引用计数加一。
  9. 根据权利要求8所述的数据管理方法，其中：所述将脏数据列表中的脏数据合并后写入闪存缓冲区，包括：
    根据写入数据对应的文件的索引节点号查找该文件的所有记录，申请新的内存页面，将多个记录的内容顺序拷贝至所述新的内存页面，将所述新的内存页面的内容顺序写入到闪存缓冲区。
  10. 根据权利要求6所述的数据管理方法，还包括：
    在闪存文件系统重启时，检测闪存缓冲区中是否有脏数据；
    如果闪存缓冲区中有脏数据，读出闪存缓冲区的所有脏数据，根据每条脏数据更新所述内存缓存的内容。
  11. 一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现权利要求6至10中任一项所述的方法。
PCT/CN2018/075376 2017-02-06 2018-02-06 一种闪存文件系统及其数据管理方法 Ceased WO2018141304A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US16/483,608 US20200034340A1 (en) 2017-02-06 2018-02-06 Flash file system and data management method therof
EP18747430.9A EP3579111A4 (en) 2017-02-06 2018-02-06 FLASH FILES SYSTEM AND PROCESS FOR MANAGING DATA THEREOF
JP2019542554A JP6920448B2 (ja) 2017-02-06 2018-02-06 フラッシュメモリファイルシステム及びそのデータ管理方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710066027.4 2017-02-06
CN201710066027.4A CN108399047B (zh) 2017-02-06 2017-02-06 一种闪存文件系统及其数据管理方法

Publications (1)

Publication Number Publication Date
WO2018141304A1 true WO2018141304A1 (zh) 2018-08-09

Family

ID=63039351

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/075376 Ceased WO2018141304A1 (zh) 2017-02-06 2018-02-06 一种闪存文件系统及其数据管理方法

Country Status (5)

Country Link
US (1) US20200034340A1 (zh)
EP (1) EP3579111A4 (zh)
JP (1) JP6920448B2 (zh)
CN (1) CN108399047B (zh)
WO (1) WO2018141304A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112925759A (zh) * 2021-03-31 2021-06-08 北京金山云网络技术有限公司 数据文件的处理方法和装置、存储介质、电子装置
US20220100417A1 (en) * 2020-09-29 2022-03-31 International Business Machines Corporation Buffer pool contention optimization

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110895515B (zh) * 2018-09-12 2024-11-05 南京中兴新软件有限责任公司 内存缓存管理方法、多媒体服务器及计算机存储介质
CN110245121A (zh) * 2019-05-08 2019-09-17 深圳市战音科技有限公司 文件管理方法、系统以及电子设备
CN110704468A (zh) * 2019-10-17 2020-01-17 武汉微派网络科技有限公司 数据更新方法、装置及控制器
CN113377684B (zh) * 2020-03-09 2024-03-08 瑞昱半导体股份有限公司 数据写入系统与方法
CN112506442A (zh) * 2020-12-22 2021-03-16 深圳市时创意电子有限公司 一种闪存芯片数据处理方法、装置、电子设备及存储介质
CN116107503A (zh) * 2022-12-26 2023-05-12 长春吉大正元信息技术股份有限公司 数据传输方法、装置及电子设备
CN116301602B (zh) * 2023-02-20 2025-06-24 重庆长安汽车股份有限公司 数据记录或读取方法、装置、采集设备、车辆及介质
CN117854553B (zh) * 2024-03-06 2024-06-25 北京云豹创芯智能科技有限公司 一种数据整形电路、方法和芯片
CN120122882B (zh) * 2025-02-20 2025-12-09 中科方德软件有限公司 一种数据写入的控制方法、装置、电子设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063271A (zh) * 2010-12-17 2011-05-18 曙光信息产业(北京)有限公司 一种磁盘外置Cache基于状态机的写回方法
CN102725752A (zh) * 2011-10-20 2012-10-10 华为技术有限公司 处理脏数据的方法及装置
CN102841851A (zh) * 2012-07-19 2012-12-26 深圳市江波龙电子有限公司 闪存管理方法和闪存设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7975109B2 (en) * 2007-05-30 2011-07-05 Schooner Information Technology, Inc. System including a fine-grained memory and a less-fine-grained memory
KR101543431B1 (ko) * 2008-11-20 2015-08-11 삼성전자주식회사 불휘발성 메모리 시스템 및 그것의 액세스 방법
JP2012008651A (ja) * 2010-06-22 2012-01-12 Toshiba Corp 半導体記憶装置、その制御方法および情報処理装置
WO2012116369A2 (en) * 2011-02-25 2012-08-30 Fusion-Io, Inc. Apparatus, system, and method for managing contents of a cache
US9135123B1 (en) * 2011-12-28 2015-09-15 Emc Corporation Managing global data caches for file system
CN104102695B (zh) * 2014-06-26 2017-11-10 晨星半导体股份有限公司 智能设备启动过程的数据处理方法及智能设备
AU2014403638B2 (en) * 2014-08-15 2020-06-25 Microsoft Technology Licensing, Llc Flushing in file system
CN105573918A (zh) * 2015-12-17 2016-05-11 深圳市新国都支付技术有限公司 一种轻量级闪存系统和方法
CN105740334A (zh) * 2016-01-22 2016-07-06 中国科学院计算技术研究所 一种文件系统中异步批量创建文件的系统及方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063271A (zh) * 2010-12-17 2011-05-18 曙光信息产业(北京)有限公司 一种磁盘外置Cache基于状态机的写回方法
CN102725752A (zh) * 2011-10-20 2012-10-10 华为技术有限公司 处理脏数据的方法及装置
CN102841851A (zh) * 2012-07-19 2012-12-26 深圳市江波龙电子有限公司 闪存管理方法和闪存设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3579111A4 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220100417A1 (en) * 2020-09-29 2022-03-31 International Business Machines Corporation Buffer pool contention optimization
US11762578B2 (en) * 2020-09-29 2023-09-19 International Business Machines Corporation Buffer pool contention optimization
CN112925759A (zh) * 2021-03-31 2021-06-08 北京金山云网络技术有限公司 数据文件的处理方法和装置、存储介质、电子装置
CN112925759B (zh) * 2021-03-31 2024-05-31 北京金山云网络技术有限公司 数据文件的处理方法和装置、存储介质、电子装置

Also Published As

Publication number Publication date
US20200034340A1 (en) 2020-01-30
EP3579111A1 (en) 2019-12-11
JP2020510905A (ja) 2020-04-09
CN108399047B (zh) 2022-11-29
JP6920448B2 (ja) 2021-08-18
EP3579111A4 (en) 2020-11-25
CN108399047A (zh) 2018-08-14

Similar Documents

Publication Publication Date Title
WO2018141304A1 (zh) 一种闪存文件系统及其数据管理方法
US11799959B2 (en) Data processing method, apparatus, and system
US11301379B2 (en) Access request processing method and apparatus, and computer device
US11030092B2 (en) Access request processing method and apparatus, and computer system
CN106951375B (zh) 在存储系统中删除快照卷的方法及装置
WO2017041654A1 (zh) 用于分布式存储系统的写入数据、获取数据的方法和设备
US12141106B2 (en) File system cloning method and apparatus
CN105045850B (zh) 云存储日志文件系统中垃圾数据回收方法
CN117951094A (zh) 存储空间的回收方法、文件系统、介质和计算设备
US12093232B2 (en) Method for storing time series data
KR101618999B1 (ko) 네트워크 부트 시스템
CN115269448B (zh) 一种存储空间回收方法、设备及介质
CN114968946A (zh) 一种存储设备和对应的操作方法及装置
CN105183385A (zh) 一种闪存及其拷贝方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18747430

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019542554

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2018747430

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 2018747430

Country of ref document: EP