WO2024066496A1 - 一种数据的访问方法及装置 - Google Patents

一种数据的访问方法及装置 Download PDF

Info

Publication number
WO2024066496A1
WO2024066496A1 PCT/CN2023/100809 CN2023100809W WO2024066496A1 WO 2024066496 A1 WO2024066496 A1 WO 2024066496A1 CN 2023100809 W CN2023100809 W CN 2023100809W WO 2024066496 A1 WO2024066496 A1 WO 2024066496A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
target
address
virtual address
storage system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/100809
Other languages
English (en)
French (fr)
Inventor
钟刊
崔文林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP23869745.2A priority Critical patent/EP4579447A4/en
Publication of WO2024066496A1 publication Critical patent/WO2024066496A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24552—Database cache management
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G06F12/1027—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
    • G06F12/1045—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB] associated with a data cache
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G06F12/1027—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
    • G06F12/1045—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB] associated with a data cache
    • G06F12/1063—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB] associated with a data cache the data cache being concurrently virtually addressed
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects
    • G06F3/0667—Virtualisation aspects at data level, e.g. file, record or object virtualisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/65—Details of virtual memory and virtual address translation
    • G06F2212/657—Virtual address space management

Definitions

  • the present invention relates to the field of computer technology, and in particular to a data access method and device.
  • the software stack refers to the collection of programs or components required for the application to run.
  • Mmap is a method of memory mapping files, that is, mapping a file or other object to the address space of a process, and establishing a one-to-one mapping relationship between the address of the file or other object and a virtual address in the virtual address space of the process.
  • mmap can reduce the physical space occupied during file access, it is limited to memory mapping of files, objects, block devices, etc. within a single node, and does not have the ability to map across nodes.
  • the present application provides a data access method, device, computer-readable storage medium, and computer program product.
  • a global shared memory pool in the distributed storage system is constructed, which can also be referred to as a global memory pool.
  • a global data block is constructed based on the global memory pool, and the global data block is mapped to the process address space of the client.
  • the client establishes a local cache for caching the global data block in the distributed system, and establishes a mapping from the process virtual address to the cache page through the client's local page table. That is, after using Distributed mmap to map the remote file in the distributed storage system, the DVA of the file is recorded locally. When the local cache does not hit, the remote DVA space is accessed through Page Fault, saving the latency overhead of the end-to-end software stack.
  • the present application provides a data access method, the method comprising: during the running of a target process, obtaining a first virtual address of target data in a target process; based on the first virtual address, querying a first address mapping relationship table to determine whether the target data is stored in a local cache, the first address mapping relationship table comprising: a correspondence between the virtual address of data in a first data set in the process and the physical address of data in the first data set in the local cache, wherein the data in the first data set is obtained in advance from a distributed storage system; if the target data is stored in the local cache, based on the first virtual address, querying the first address mapping relationship table to obtain the physical address of the target data in the local cache; and obtaining the target data according to the physical address of the target data in the local cache.
  • a global memory pool formed by distributed storage nodes and a global data block of a distributed storage system can be constructed based on the global memory pool.
  • the client processor can map part of the data in the global data block to the virtual address space of the target process and generate a first address mapping relationship table.
  • the client processor establishes a cache page similar to Page Cache in the local memory of the client to cache the global data block, and establishes a mapping relationship between the local cache page and the virtual address of the target process. Then the client processor updates the first address mapping relationship table according to the mapping relationship.
  • the client processor determines whether the target data is stored in the cache page of the local cache according to the first virtual address of the target data in the target process.
  • the client processor can directly read the target data from the cache page of the local cache. That is, in a distributed storage system.
  • the client process can quickly access the global data block, saving the latency overhead of the client to the storage node software stack.
  • the first address mapping relationship table also includes: the correspondence between the virtual address of the data in the second data set in the process and the virtual address of the data in the second data set in the distributed storage system
  • the method also includes: if the target data is not stored in the local cache, based on the first virtual address, querying the first address mapping relationship table to obtain the second virtual address of the target data in the distributed storage system; based on the second virtual address, querying the second address mapping relationship table to obtain the second virtual address of the target data in the distributed storage system the physical address of the target data in the distributed storage system
  • the second address mapping relationship table includes: the correspondence between the virtual address of the data in the third data set stored in the distributed storage system and the physical address of the data in the third data set in the distributed storage system; wherein the data in the third data set includes: the data in the first data set and the data in the second data set; based on the physical address of the target data in the distributed storage system, the target data is obtained from the distributed storage system.
  • the client processor needs to read the target data into the cache page from the storage node of the distributed storage system. Specifically, the client processor can query the first address mapping relationship table according to the first virtual address of the target data to obtain the global virtual address of the target data in the distributed storage system, that is, the second virtual address. Then the client processor can query the second virtual address table in the distributed storage system according to the second virtual address, obtain the physical address of the target data in the distributed storage system, and obtain the target data from the distributed storage system according to the physical address.
  • the second virtual address table is the address mapping table obtained when the distributed storage system constructs global data.
  • the method after obtaining the target data from the distributed storage system based on the physical address of the target data in the distributed storage system, the method also includes: storing the target data in a local cache; establishing a first correspondence between a first virtual address of the target data in the target process and a physical address of the target data in the local cache; replacing a second correspondence in the first address mapping relationship table with the first correspondence, the second correspondence being a correspondence between the virtual address of the target data in the target process and the virtual address of the target data in the distributed storage system.
  • the client processor after the client processor obtains the target data from the distributed storage system, it also needs to cache the target data in the local cache and establish a mapping relationship between the physical address of the target data in the local cache and the virtual address of the target data in the target process. Then, the client processor saves the mapping relationship in the first address mapping relationship table. When the target process needs to obtain the target data again, it can directly obtain it from the local cache. This saves the latency overhead of the target process obtaining the target data.
  • the method also includes: during the running of the target process, obtaining the third virtual address of the data to be written in the target process; based on the third virtual address, querying the first address mapping relationship table to determine whether data corresponding to the third virtual address is stored in the local cache; if data corresponding to the third virtual address is stored in the local cache, replacing the data corresponding to the third virtual address in the local cache with the data to be written.
  • the client process can directly cache the data to be written in the client's cache page. Then, the client processor can mark the cache page storing the data written by the client process as a dirty page, and periodically write the dirty page back to the storage node. This simplifies the process of writing to the storage node by the client process.
  • the present application provides a data processing device, comprising:
  • An acquisition module used for acquiring the first virtual address of the target data in the target process during the running of the target process
  • a processing module configured to query a first address mapping relationship table according to a first virtual address to determine whether the target data is stored in a local cache, wherein the first address mapping relationship table includes: a correspondence between a virtual address of the data in the first data set in the process and a physical address of the data in the first data set in the local cache, wherein the data in the first data set is obtained in advance from a distributed storage system; if the target data is stored in the local cache, query the first address mapping relationship table based on the first virtual address to obtain a physical address of the target data in the local cache;
  • the acquisition module is also used to acquire the target data according to the physical address of the target data in the local cache.
  • the first address mapping relationship table further includes: a correspondence between a virtual address of data in the second data set in the process and a virtual address of data in the second data set in the distributed storage system, and the processing module is further configured to:
  • the target data is not stored in the local cache, based on the first virtual address, query the first address mapping relationship table to obtain the second virtual address of the target data in the distributed storage system; query the second address mapping relationship table according to the second virtual address to obtain the physical address of the target data in the distributed storage system; the second address mapping relationship table includes: the correspondence between the virtual address of the data in the third data set stored in the distributed storage system and the physical address of the data in the third data set in the distributed storage system; wherein the data in the third data set includes: the data in the first data set and the data in the second data set;
  • the acquisition module is also used to acquire the target data from the distributed storage system according to the physical address of the target data in the distributed storage system.
  • processing module is further configured to:
  • the second corresponding relationship in the first address mapping relationship table is replaced by the first corresponding relationship, where the second corresponding relationship is the corresponding relationship between the virtual address of the target data in the target process and the virtual address of the target data in the distributed storage system.
  • the acquisition module is further used to, during the running of the target process, acquire a third virtual address of the data to be written in the target process;
  • the processing module is also used to query the first address mapping relationship table according to the third virtual address to determine whether data corresponding to the third virtual address is stored in the local cache; if data corresponding to the third virtual address is stored in the local cache, replace the data corresponding to the third virtual address in the local cache with the data to be written.
  • the present application provides a computer storage medium, in which instructions are stored.
  • the instructions When the instructions are executed on a computer, the computer executes the method provided in the first aspect.
  • the present application provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the method provided in the first aspect.
  • FIG1 is a schematic diagram of data access based on mmap provided in an embodiment of the present application.
  • FIG2 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • FIG3 is a flow chart of a data access method provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of a process of accessing data in a distributed storage system provided by an embodiment of the present application.
  • FIG5 is a schematic diagram of another process of accessing data in a distributed storage system provided in an embodiment of the present application.
  • FIG6 is a schematic diagram of another process of accessing data in a distributed storage system provided in an embodiment of the present application.
  • FIG7 is a schematic diagram showing a comparison between a traditional file access path and a global mmap access path provided in an embodiment of the present application
  • FIG8 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application.
  • FIG. 9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application.
  • any embodiment or design scheme of "exemplary”, “for example” or “for example” should not be understood as being more preferred or more advantageous than other embodiments or designs.
  • the use of words such as “exemplary”, “for example” or “for example” is intended to present related concepts in a specific way.
  • first and second are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as “first” and “second” may explicitly or implicitly include one or more of the features.
  • the terms “include”, “comprises”, “has” and their variations all mean “including but not limited to”, unless otherwise specifically emphasized.
  • mmap is a method of memory mapping files, that is, mapping files or other objects in the file disk to the address space (user space) of the application process, realizing a one-to-one mapping relationship between the file disk address and a virtual address in the process virtual address space. After realizing such a relationship, the process can use pointers to read and write this section of memory, and the system will automatically write back the dirty page to the corresponding file disk, that is, the operation on the file is completed without calling read, write and other system call functions. On the contrary, the modification of this area by the kernel space can also be directly reflected in the user space, so that file sharing between different processes can be realized.
  • Bypass means that two networks can be physically connected directly without passing through the network security device system through a specific trigger state (power failure or crash). Therefore, with Bypass, when the network security device fails, the networks connected to this device can still be connected to each other. At this time, the network device will no longer process the packets in the network.
  • the global memory pool refers to logically combining multiple physically separated memory pools into one memory pool.
  • FIG1 shows a schematic diagram of data access based on mmap.
  • the operating system on the electronic device allocates an independent virtual address space to each process of the application.
  • the target process (which can be any process in the application) calls the library function mmap in the user space, and searches for a free continuous virtual address that meets the requirements in the virtual address space of the target process.
  • the continuous virtual address in the target process is associated with the file stored in the file space (disk).
  • the mmap function it is necessary to indicate the identifier (file) of the mapped file, the starting point (offset) of the mapped file on the file space (disk), and the length (len) of the mapped file.
  • the operating system After establishing the mapping relationship between the virtual address of the target process and the file in the user space, the operating system establishes the mapping relationship between the virtual address of the process and the physical address (Page in Page Cache) in the memory of the electronic device through the memory management unit (Memory Management Unit, MMU), and stores the mapping relationship in the Page Table.
  • the Page in the Page Cache can be used to cache the file content in the file space. It is understandable that each process has its own Page Table, which stores the mapping relationship between the virtual address and the physical address in the process.
  • the process sends the virtual address of the data to be read to the MMU through the CPU of the electronic device.
  • the MMU After receiving the virtual address sent by the CPU, the MMU starts to query the Page Table of the process to determine whether there is a mapping in the Page Table of the process and whether the read and write permissions of the process are normal.
  • the CPU when there is no physical address corresponding to the virtual address of the data to be read in the Page Table (i.e., the local cache does not hit), the CPU generates a page fault interrupt (Page Fau lt). At this time, the process will switch from user mode to kernel mode and hand over the Page Fault to the kernel's Page Fault Handler for processing.
  • the CPU can read data from the file space through rede/prefetch and store the read data in the Page in the Page Cache. Then, the CPU triggers the MMU to establish a mapping relationship between the virtual address of the data to be read and the physical address in the Page Cache, and saves the mapping relationship in the Page Table. In another possible example, when there is a physical address corresponding to the virtual address of the data to be read in the Page Cache (i.e., the local cache hits), the CPU can directly read the data in the Page Cache.
  • the modified file data will not be synchronized to the disk (or file space) immediately, and the modified data will be cached in the Page Cache of the memory first.
  • the page that is inconsistent with the disk data can be called a dirty page, and the dirty page will be synchronized to the disk at the appropriate time.
  • data access based on mmap enables multiple processes to access the same file through mmap, and the accessed file is only stored in the Page Cache, reducing the physical space occupied.
  • data access based on mmap can realize direct data interaction between user space and kernel space, eliminating the tedious process of different spaces and different data.
  • mmap is limited to memory mapping of files, objects, block devices, etc. within a single node, and has no cross-node mapping capability. That is, based on the mmap data access method shown in Figure 1, global data access in a distributed storage system cannot be realized.
  • an embodiment of the present application provides a data access method that can be applied to a distributed storage system.
  • the physical memory of each storage node in the distributed storage system is used to form a global address space, and a distributed physical address space (DPAS) is provided to the outside.
  • the distributed physical address is mapped to a distributed virtual address (DVA) through a distributed page table (DPT).
  • DVA distributed virtual address
  • DPT distributed page table
  • the global data block in the distributed storage system can be constructed based on DVA. Among them, the global data block includes: user files or objects.
  • the application program in the client (Client) end of the distributed storage system can map the global data block to the address space of the local process through distributed mmap, and access it through load/store. This allows the client-side application program in the distributed storage system to access the data in the distributed storage node faster.
  • FIG2 shows a schematic diagram of an application scenario provided by an embodiment of the present invention.
  • the distributed storage system includes multiple storage nodes (Node).
  • the storage node can be used to store data.
  • the storage node may include: dynamic random access memory (DRAM), PM and other physical storage media.
  • the physical storage media in multiple storage nodes constitute the global address space of the distributed storage system and provide a distributed physical address space to the outside.
  • the distributed physical address (DPA) in the distributed physical address space can be mapped to the DVA through the DPT.
  • the DVA based on the distributed storage system can construct a global data block in the distributed storage system. Among them, the DPA can access the data in the storage node through the global memory bus, and bypass the CPU of the storage node.
  • An application is installed on the client.
  • the application can map the data in the global data block built based on DVA in the distributed storage system to the address space of the local process on the client by calling the Distributedmmap function. And read and write data mapped to the address space of the local process through load/store operations.
  • an embodiment of the present application provides a flow chart of a method for accessing data.
  • the method can be applied to the distributed storage system shown in Figure 2. Referring to Figure 3, the method includes: S301-S306.
  • the management software in the distributed storage system may first construct a global data block of the distributed storage system.
  • the storage medium (e.g., AEP, DRAM, XL-Flash) in each of the multiple storage nodes in the distributed storage system constitutes the global address space of the distributed storage system, providing a distributed physical address space to the outside.
  • the distributed physical address space of the distributed storage system can be accessed through the DPA. That is, after obtaining the DPA, the data in the storage node can be directly accessed through the global memory data bus according to the obtained DPA.
  • the management software in the distributed storage system After the management software in the distributed storage system obtains the DPA in the distributed storage system, in order to allow all users using the distributed storage system to share the storage space of the entire distributed storage system, it is also necessary to map the DPA of the distributed storage system to DVA.
  • mapping the DPA of the distributed storage system to DVA it is necessary to generate a DPT in the distributed storage system.
  • the DPT can be stored in any storage node of the distributed storage system, and the DPT is used to store the mapping relationship from the DVA to the DPT of the distributed storage system.
  • the management software in the distributed storage system may construct a global data block of the distributed storage system based on the obtained DVA of the distributed storage system.
  • the global data block of the distributed storage system may include: user files or objects.
  • the operating system of the client side allocates a virtual address space of a fixed size for each process.
  • the virtual address space of the process is composed of multiple virtual memory areas.
  • the virtual memory area is a homogeneous interval in the virtual address space of the process, that is, a continuous address range with the same characteristics.
  • the virtual address space of the process of the client side in Figure 4 includes: a stack area, a file mapping area, a heap, a bss data segment, an initial data segment, and a text data segment.
  • the file mapping area is the free space between the heap and the stack.
  • the target process can call the library function mmap in user space.
  • the prototype of the library function mmap is:
  • the parameter start points to the starting address of the memory to be mapped;
  • the parameter length represents how much of the global data block in the distributed storage system needs to be mapped to the memory;
  • the parameter port represents the protection method of the mapping area;
  • the parameter flags represents the various characteristics that affect the mapping area;
  • the parameter fd represents the file descriptor of the global data block to be mapped to the memory;
  • the parameter offset represents the offset when mapping the global data block, which is usually set to 0, indicating that the mapped global data block corresponds to the front of the entire data block, where offset must be an integer multiple of the paging size.
  • the client-side CPU When the target process calls the library function mmap to map the global data block stored in the distributed storage node to the virtual address space of the target process, the client-side CPU also needs to save the mapping relationship between the DVA of the global data block and the virtual address (virt) of the global data block in the virtual address space of the target process to the local page table maintained by the MMU of the client, such as the mapping relationship saved in the Page Table in Figure 4. And the client-side CPU also needs to save the mapping relationship between the DVA of the global data block and the virtual address (virt) of the global data block in the virtual address space of the target process to the mmap management software.
  • the client CPU can use the client physical memory as a local cache.
  • the client operating system allocates a virtual address space for the process, it also needs to map the virtual address in the virtual space to the client local cache through the MMU.
  • the virtual address can be mapped to the local cache through a page table, which is maintained by the client operating system and referenced by the processor.
  • the virtual address space of the process may be divided according to a fixed size, and the local cache may also be set to different page sizes.
  • the size of the cache page in the local cache is the same as the size of the virtual page.
  • the target process on the client can trigger the CPU to read the data stored in the distributed storage node into the local cache page on the client. Specifically, after the CPU on the client obtains the virtual address of the data to be read in the target process, the CPU on the client can find the local page table according to the virtual address of the data to be read in the target process. Get the DVA of the data in the distributed storage system, and then the CPU on the client side sends a data read request command to the distributed storage system, which carries the DVA of the data to be read. After the CPU in the distributed storage system obtains the DVA, it searches the DPT in the distributed storage system according to the DVA to obtain the DPA of the data to be read.
  • the CPU in the distributed storage system obtains the data to be read according to the DPT, and sends the data to be read to the CPU on the client side.
  • the CPU on the client side After receiving the data to be read, the CPU on the client side stores the data to be read in the local cache page. Then, the CPU on the client side also needs to establish a mapping relationship between the cache page and the virtual address of the data to be read in the process, and use the mapping relationship to replace the mapping relationship between the virtual address of the data to be read in the process and the DVA of the data to be read in the local page table.
  • a data processing unit may be deployed on the client side.
  • the DPU may pre-cache some DPTs in the distributed storage system.
  • the CPU may obtain the DVA of the global data block that needs to be read into the local cache page, i.e., the target DVA, according to the local page table maintained by the MMU. Then, the CPU sends the target DVA to the DPU.
  • the DPU obtains data corresponding to the target DVA from the distributed storage system based on the received target DVA.
  • the DPU may be a PCIE device plugged into the client side.
  • the DPU may have its own operating system, CPU, storage medium, network interface, etc.
  • a portion of the DPT maintained by the distributed storage system is pre-cached in the DPU.
  • the DPU can search in the DPT cached in the DPU for a DPA corresponding to the target DVA. If the DPU can find the DPA corresponding to the target DVA in the DPT cached by the DPU, the DPU can directly read the data stored at the DPA from the storage node of the distributed storage system according to the DPA, and send the data to the CPU on the Client side. If the DPA corresponding to the target DVA does not exist in the DPT cached in the DPU, the DPU needs to obtain the DPA corresponding to the target DVA from the DPT maintained by the distributed storage system.
  • the DPU After the DPU obtains the DPA corresponding to the target DVA from the DPT maintained by the distributed storage system, the DPU can first save the correspondence between the target DVA and the DPA in the DPT cached by the DPU. Then, the DPU can read the data stored at the DPA from the storage node of the distributed storage system according to the obtained DPA corresponding to the target DVA, and send the data to the CPU on the Client side. After the CPU on the client side receives the data corresponding to the target DVA sent by the DPU, the CPU caches the data in the cache page of the local cache on the client side and triggers the MMU to update the local page table. Among them, the MMU updates the local page table to update the mapping relationship between the target DVA and the target virtual address of the target process stored in the local page table to the mapping relationship between the target virtual address and the physical address in the local memory.
  • the page table is also called a mapping table.
  • the operating system wants to map the virtual address of the target process to the physical address of the memory. The most intuitive way is to create a mapping table. This mapping table can realize the mapping of the page in the virtual address of the target process to the page in the physical memory.
  • the target process when the target process needs to read a certain segment of data in the target process virtual address space during operation, the target process will initiate a read request to the CPU.
  • the read request carries the virtual address of the target data block to be read.
  • the target process When the target process needs to modify a certain segment of data in the target process virtual address space during operation, the target process will initiate a write request to the CPU.
  • the write request carries the virtual address of the target data block to be modified.
  • S305 Read target data according to the virtual address of the target process carried in the first message.
  • the CPU when the first message sent by the target process is used to indicate that data is to be read from the cache page, the CPU needs to send the virtual address of the target data to be read carried in the first message to the MMU.
  • the MMU After the MMU receives the virtual address of the target data block, the MMU needs to determine whether the local page table stores the physical address in the local cache corresponding to the virtual address of the target data.
  • the MMU can trigger the CPU to read the target data directly from the local cache according to the physical address corresponding to the virtual address of the target data.
  • the MMU determines that the local page table does not contain the physical address in the local cache corresponding to the virtual address of the target data block
  • the MMU triggers the CPU of the Client to generate a page fault, i.e., Page Fault.
  • the CPU obtains the target data block from the storage node of the distributed storage system and caches it in the local cache page.
  • the client side memory is used as a local cache to cache global data blocks in the distributed storage system.
  • the client side also includes: the client side CPU and DPU.
  • the target process will switch from user mode to kernel mode and hand the Page Fault to the kernel terminal processor (Page Fault Handler) in the kernel for processing.
  • the DPU can perform Page Fault acceleration.
  • the MMU when the MMU determines that there is no physical address in the local cache corresponding to the virtual address of the target data block in the local page table, the MMU triggers the CPU to generate a Page Fault.
  • the Page Fault Handler in the CPU on the Client side obtains the DVA corresponding to the virtual address of the target data block from the local page table, that is, the target DVA.
  • the Page Fault Handler then sends the obtained target DVA to the DPU.
  • the operating system on the Client side needs to perform a context switch, let the CPU go, and the DPU performs subsequent Page Fault processing.
  • the "virtual address of the target data block" refers to the virtual address of the target data block in the virtual address space of the target process.
  • the CPU in the DPU can use the memory management unit (DPU-MMU) inside the DPU to find out whether there is a DPA corresponding to the target DVA in the DPT cached inside the DPU.
  • DPU-MMU memory management unit
  • the CPU inside the DPU can directly read the data at the DPA from the distributed storage node through the network interface controller (NIC) of the DPU.
  • NIC network interface controller
  • the CPU inside the DPU needs to obtain the DPA corresponding to the target DVA from the DPT on the distributed storage node through the NIC of the DPU.
  • the CPU inside the DPU needs to trigger the DPU-MMU to add the mapping relationship between the target DVA and the DPA in the DPT cached by the DPU.
  • the DPU can read the data at the DPA from the distributed storage node through the NIC of the DPU.
  • the DPU After the DPU obtains the data corresponding to the target DVA, it sends the data to the Page Fault Handler.
  • the Page Fault Handler stores the data corresponding to the target DVA in the cache page in the local cache of the client. Then, the Page Fault Handler needs to establish a mapping relationship between the cache page and the virtual address of the target process corresponding to the target DVA. At this point, the Page Fault Handler completes the processing of the Page Fault generated by the client.
  • the target process can read the data in the client's local cache normally.
  • the CPU on the client side can generate a Page Fault.
  • the Page Fault generated by the CPU on the client side will be processed by the DPU process.
  • the memory of the DPU can be used as a page cache (i.e., a local cache page) to cache global data blocks in the distributed storage system.
  • the CPU on the client side obtains the virtual address of the data that the target process needs to read
  • the CPU on the client side obtains the DVA corresponding to the virtual address of the target data block from the local page table through the MMU, that is, the target DVA.
  • the MMU uses the obtained target address to directly access the DPU.
  • the DPU-MMU in the DPU After the DPU-MMU in the DPU receives the target DVA sent by the MMU, the DPU-MMU queries the local cache page in the DPU and performs address translation to obtain the physical address corresponding to the target DVA (that is, the page address in the cache page).
  • the CPU in the DPU can directly read data from the cache page corresponding to the target DVA.
  • the DPU-MMU triggers the CPU inside the DPU to generate a Page Fault.
  • the Page Fault Handler in the DPU can use the memory management unit (DPU-MMU) inside the DPU to check whether there is a DPA corresponding to the target DVA in the DPT of the DPU's internal cache.
  • the Page Fault Handler in the DPU determines that there is a DPA corresponding to the target DVA in the DPT of the DPU cache, the Page Fault Handler in the DPU can directly read the data at the DPA from the distributed storage node through the NIC of the DPU.
  • the Page Fault Handler in the DPU determines that there is no DPA corresponding to the target DVA in the DPT of the DPU cache
  • the Page Fault Handler in the DPU needs to obtain the DPA corresponding to the target DVA from the DPT on the distributed storage node through the NIC of the DPU.
  • the Page Fault Handler in the DPU needs to trigger the DPU-MMU to add the mapping relationship between the target DVA and the DPA in the DPT of the DPU cache.
  • the Page Fault Handler in the DPU can read the data at the DPA from the distributed storage node through the NIC of the DPU.
  • the Page Fault Handler in the DPU After the Page Fault Handler in the DPU reads the data corresponding to the target DVA, it also needs to write the data corresponding to the target DVA to the cache page in the local cache inside the DPU. Then, the DPU sends a message to the MMU, triggering the MMU to establish a mapping relationship between the cache page in the DPU and the virtual address of the target process corresponding to the target DVA. At this point, the Page Fault processing generated inside the DPU is completed. The target process can read the data in the local cache inside the GPU normally.
  • writing the data to be written into the cache page refers to overwriting, that is, the newly written data needs to overwrite the original data.
  • the MMU determines whether the local page table contains the physical address in the local cache corresponding to the virtual address of the data to be written.
  • the MMU can trigger the CPU on the client side to write the data to be written into the cache page of the local cache according to the physical address corresponding to the virtual address of the data to be written.
  • the MMU When the MMU determines that the local page table does not contain the physical address in the local cache corresponding to the virtual address of the target data block, the MMU triggers the CPU on the client side to generate a page fault, i.e., Page Fault.
  • Page Fault the CPU in the DPU obtains the target data block from the storage node of the distributed storage system and caches it in the local cache page.
  • the CPU on the client side establishes a mapping relationship between the virtual address of the data to be written and the local cache, and writes the data to be written into the local cache page according to the mapping relationship.
  • the process in which the CPU in the DPU obtains the target data block from the storage node of the distributed storage system in response to the Page Fault and caches it to the local cache page can refer to the description of FIG. 5 in S306, which will not be repeated here.
  • the CPU of the client After the CPU of the client writes the data to be written into the local cache of the client.
  • the CPU of the client will mark the cache page of the written data as a dirty page and add it to the dirty list.
  • the mmap management software of the client periodically writes the page in the dirty list back to the distributed storage node, so that the data in the distributed storage node is consistent with the data cached in the memory of the client. It can be understood that the mmap management software stores the mapping relationship between the DVA of the global data block and the virtual address (virt) of the global data block in the virtual address space of the target process.
  • the mmap management software when the mmap management software writes the page in the dirty list back to the distributed storage node, the mmap management software can obtain the virtual address in the target process corresponding to the page in the dirty list through the local page table. Then, the mmap management software can write the data stored in the page in the dirty list back to the storage node in the distributed storage system according to the virtual address in the target process corresponding to the page in the dirty list and the mapping relationship stored in the mmap management software.
  • the mmap management software on the client side obtains the dirty page in the local cache and calls mcommit() to initiate the process of synchronizing the dirty page to the storage node in the distributed storage system. After the dirty page is synchronized to the storage node in the distributed storage system, mcommit() returns success.
  • the mmap management software on the client side obtains the dirty pages in the local cache and writes the dirty pages back to the storage nodes in the distributed storage system by calling the refresh change function msync().
  • the CPU on the client side needs to send the write request to the DPU.
  • the DPU-MMU in the DPU determines whether the local page table contains the physical address in the local cache corresponding to the virtual address of the data to be written according to the virtual address of the data to be written.
  • the virtual address of the data to be written refers to the "virtual address in the address space of the target process". If the DPU-MMU determines that the page table in the DPU contains the physical address in the local cache of the DPU corresponding to the virtual address of the data to be written.
  • the MMU can trigger the CPU in the DPU to write the data to be written into the cache page in the local cache of the DPU according to the physical address corresponding to the virtual address of the data to be written. If the DPU-MMU determines that the page table in the DPU does not contain the physical address in the local cache of the DPU corresponding to the virtual address of the data to be written. The DPU-MMU triggers the CPU inside the DPU to generate a Page Fault. In response to the Page Fault, the Page Fault Handler in the DPU obtains the target data block from the storage node of the distributed storage system and caches it in the cache page of the DPU.
  • the CPU in the DPU establishes a mapping relationship between the virtual address of the data to be written and the cache page in the local cache of the DPU, and writes the data to be written into the local cache page according to the mapping relationship.
  • the process in which the Page Fault Handler in the DPU obtains the target data block from the storage node of the distributed storage system in response to the Page Fault and caches it into the local cache page of the DPU can refer to the description of FIG. 6 in S306, which will not be repeated here.
  • the CPU in the DPU After the CPU in the DPU writes the data that the target process needs to write to the cache page in the local cache in the DPU.
  • the CPU in the DPU will mark the cache page where the data is written as a dirty page and add it to the dirty list.
  • the mmap management software in the GPU will periodically write the page in the dirty list back to the distributed storage node, so that the data in the distributed storage node is consistent with the data cached in the client's memory.
  • the mmap management software in the DPU obtains the dirty page in the local cache and calls mcommit() to initiate the process of synchronizing the dirty page to the storage node in the distributed storage system. After the dirty page is synchronized to the storage node in the distributed storage system, mcommit() returns success.
  • the mmap management software in the DPU obtains the dirty page in the local cache and writes the dirty page back to the storage node in the distributed storage system by calling the refresh change function msync().
  • the embodiment of the present application also provides a comparative diagram of the traditional file access path and the global mmap access path.
  • the traditional file access path is long and needs to pass through VFS, the file system client, the file system server, and finally access PM, which has a large software stack overhead.
  • Distributed mmap is used. After mapping the remote file, the DVA of the file is recorded locally (on the client side) (which can be recorded in the process page table). If the local cache misses, the remote DVA space is accessed through Page Fault.
  • the NIC directly accesses the PM medium, bypassing the software stack overhead on the server side, and the end-to-end software stack latency is reduced from 150us to 20us.
  • a global shared memory pool in the distributed storage system is constructed. Then, a global data block is constructed based on the global memory pool, and the global data block is mapped to the process address space of the client. Then, the client establishes a local cache for caching the global data block in the distributed system, and establishes a mapping from the process virtual address to the cache page through the local page table of the client. And in the distributed storage system, the files stored in the storage node are mmap mapped so that the client's application can directly access the data in the storage node through load/store.
  • a global data block based on the global memory pool is constructed in the distributed storage system, so that multiple clients belonging to the same distributed storage system can mmap the same global data block, thereby realizing shared access to the global data block among various clients.
  • the DVA of the file is recorded locally (which can be recorded in the process page table).
  • the remote DVA space is accessed through Page Fault to save the latency overhead of the end-to-end software stack.
  • the present application embodiment further provides a data processing device, see Figure 8.
  • Figure 8 is a schematic diagram of the structure of a data processing device provided by the present application embodiment.
  • the data processing device includes: an acquisition module 810, a processing module 820, and a storage module 830.
  • the storage module 830 is used to store part of the data required during the operation of the target process, and the storage module 830 also stores a local page table, which can also be called a first address mapping relationship table.
  • the local page table stores the mapping relationship between the DVA of the global data block in the distributed storage system and the virtual address in the target process, and the mapping relationship between the virtual address in the target process and the physical address of the local cache in the data processing device.
  • the acquisition module 810 is used to acquire the virtual address of the target data in the target process during the target process, which may also be referred to as the first virtual address.
  • the processing module 820 is used to query the local page table to determine whether the local cache physical address corresponding to the virtual address of the target data is stored in the local page table according to the obtained virtual address of the target data. When the processing module 820 determines that the local cache physical address corresponding to the virtual address of the target data is stored in the local page table, the processing module 820 obtains the physical address.
  • the acquisition module 810 acquires the target data in the local cache according to the physical address acquired by the processing module 820 .
  • the processing module 820 determines that the local page table stores the physical address of the local cache corresponding to the virtual address of the target data
  • the processing module 820 queries the local page table according to the virtual address of the target data to obtain the DVA of the target data.
  • the processing module 810 determines the DPA of the target data according to the DVA of the target data.
  • the acquisition module acquires the target data from the distributed storage system according to the DPA of the target data obtained by the processing module.
  • a computing device is also provided in an embodiment of the present application.
  • the computing device includes at least a processor and a memory, and a program is stored in the memory.
  • the processor executes the program, it can implement the units or modules of each step in the method shown in Figure 3.
  • FIG9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application.
  • the computing device 900 includes at least one processor 910, a memory 920, and a communication interface 930.
  • the processor 910, the memory 920, and the communication interface 930 are communicatively connected, and the communication connection can be achieved by wired means (such as a bus) or by wireless means.
  • the communication interface 930 is used to receive data sent by other devices (such as the IR corresponding to the DUT); the memory 920 stores computer instructions, and the processor 910 executes the computer instructions to execute the method in the aforementioned method embodiment.
  • the processor 910 may be a central processing unit CPU, and the processor 910 may also be other general-purpose processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field programmable gate arrays (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or any conventional processor, etc.
  • the memory 920 may include a read-only memory and a random access memory, and provides instructions and data to the processor 910.
  • the memory 920 may also include a nonvolatile random access memory.
  • the memory 920 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.
  • the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
  • the volatile memory may be a random access memory (RAM), which is used as an external cache.
  • RAM random access memory
  • DRAM dynamic random access memory
  • SDRAM synchronous DRAM
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous link dynamic random access memory
  • DR RAM direct rambus RAM
  • computing device 900 can execute the method shown in Figure 3 in the embodiment of the present application.
  • the detailed description of the implementation of the method is shown above, and for the sake of brevity, it will not be repeated here.
  • An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon.
  • the computer program is executed by a processor, the above-mentioned method is implemented.
  • An embodiment of the present application provides a chip, which includes at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.
  • An embodiment of the present application provides a computer program or a computer program product, wherein the computer program or the computer program product comprises instructions, and when the instructions are executed, the computer is caused to execute the above-mentioned method.
  • the method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions.
  • the software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium.
  • the storage medium can also be a component of the processor.
  • the processor and the storage medium can be located in an ASIC.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium.
  • the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated.
  • the available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种数据访问方法，包括：在目标进程运行过程中，获取目标数据在目标进程中的第一虚拟地址；基于所述第一虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有目标数据，若本地缓存中存储有目标数据，基于所述第一虚拟地址，查询所述第一地址映射关系表，得到所述目标数据在本地缓存中的物理地址；根据所述目标数据在本地缓存中的物理地址，获取所述目标数据。在分布式存储系统中使用Distributed mmap映射分布式存储系统中的文件之后，在本地记录该文件在分布式存储系统中的分布式虚拟地址。在目标进程运行过程中，若本地缓存不命中，通过Page Fault访问远端DVA空间，节省端到端软件栈的时延开销。

Description

一种数据的访问方法及装置
本申请要求在2022年9月30日提交中国国家知识产权局、申请号为202211213882.0，发明名称为“一种数据的访问方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及计算机技术领域,尤其涉及一种数据的访问方法及装置。
背景技术
传统的文件访问路径长，在文件访问过程中，需要经过文件虚拟系统(virtual file system，VFS)、文件系统客户端、文件系统服务端以后，最终访问持久内存(persistent memory，PM)中的文件。在整个的访问过程中，软件栈的开销较大。其中，软件堆栈是指应用程序运行所需的程序或组件的集合。
mmap是一种内存映射文件的方法，即将一个文件或者其他对象映射到进程的地址空间，建立文件或者其他对象的地址与进程的虚拟地址空间中的一段虚拟地址的一对一映射关系。mmap虽然能够在文件访问过程中，减少物理空间占用。但是，mmap局限于单节点内部对文件、对象、块设备等实现内存映射，没有跨节点映射的能力。
发明内容
本申请提供了一种数据的访问方法、装置、计算机可读存储介质、计算机程序产品。针对分布式存储系统，构建分布式存储系统中的全局共享内存池，也可以称之为全局内存池。然后，基于该全局内存池构建全局数据块，并将该全局数据块映射到客户端的进程地址空间中。然后，客户端建立用于缓存分布式系统中的全局数据块的本地缓存，并通过客户端的本地页表，建立进程虚拟地址到缓存页面的映射。即在分布式存储系统中使用Distributed mmap映射远端文件之后，在本地记录文件的DVA。当本地缓存不命中，通过Page Fault访问远端DVA空间，节省端到端软件栈的时延开销。
第一方面，本申请提供了一种数据访问方法，该方法包括：在目标进程运行过程中，获取目标数据在目标进程中的第一虚拟地址；基于第一虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有目标数据，第一地址映射关系表中包括：第一数据集合中的数据在进程中的虚拟地址和第一数据集合中的数据在本地缓存中的物理地址间的对应关系，其中，第一数据集合中的数据预先从分布式存储系统中获取得到；若本地缓存中存储有目标数据，基于第一虚拟地址，查询第一地址映射关系表，得到目标数据在本地缓存中的物理地址；根据目标数据在本地缓存中的物理地址，获取目标数据。
也就是说，在分布式存储系统中，可以基于分布式存储节点构成的全局内存池，以及基于全局内存池构建分布式存储系统的全局数据块。客户端处理器可以将全局数据块中的部分数据映射到目标进程的虚拟地址空间中，并生成第一地址映射关系表。以及客户端处理器在客户端的本地内存中建立类似于Page Cache的缓存页面，用于缓存全局数据块，并建立本地缓存页面与目标进程的虚拟地址间的映射关系。然后客户端处理器根据该映射关系更新第一地址映射关系表。在目标进程的运行过程中，客户端处理器根据目标数据在目标进程中的第一虚拟地址，确定本地缓存的缓存页面中是否存储有目标数据。当本地缓存的缓存页面中存储有目标数据时，客户端处理器可以直接从本地缓存的缓存页面中读取目标数据。即在分布式存储系统中。通过建立全局数据块的虚拟地址到目标进程虚拟地址间的映射，以及进程虚拟地址到缓存页面的映射，可以实现客户端进程对全局数据块的访问快速访问，节省了客户端到存储节点软件栈的时延开销。
在一个可能的实现方式中，第一地址映射关系表中还包括：第二数据集合中的数据在进程中的虚拟地址和第二数据集合中的数据在分布式存储系统中的虚拟地址间的对应关系，方法还包括：若本地缓中没有存储有目标数据，基于第一虚拟地址，查询第一地址映射关系表，得到目标数据在分布式存储系统中的第二虚拟地址；基于第二虚拟地址，查询第二地址映射关系表，得到目标数据在分布式存储系统中 的物理地址；第二地址映射关系表中包括：在分布式存储系统中存储的第三数据集合中的数据的虚拟地址和第三数据集合中的数据在分布式存储系统中的物理地址间的对应关系；其中，第三数据集合中的数据包括：第一数据集合中的数据和第二数据集合中的数据；基于目标数据在分布式存储系统中的物理地址，从分布式存储系统中获取目标数据。
也就是说，在本地缓存中没有缓存目标数据的情况下。客户端处理器需要从分布式存储系统的存储节点中，将目标数据读取到缓存页面中。具体地，客户端处理器可以根据目标数据的第一虚拟地址查询第一地址映射关系表得到目标数据在分布式存储系统中的全局虚拟地址，即第二虚拟地址。然后客户端处理器可以根据第二虚拟地址，查询分布式存储系统中的第二虚拟地址表，得到目标数据在分布式存储系统中的物理地址，并根据该物理地址从分布式存储系统中获取目标数据。其中，第二虚拟地址表是分布式存储系统构建全局数据时，得到的地址映射表。
在一个可能的实现方式中，在基于目标数据在分布式存储系统中的物理地址，从分布式存储系统中获取目标数据以后，该方法还包括：将目标数据存储到本地缓存中；建立目标数据在目标进程中的第一虚拟地址和目标数据在本地缓存中的物理地址间的第一对应关系；用第一对应关系替换第一地址映射关系表中的第二对应关系，第二对应关系为目标数据在目标进程中的虚拟地址和目标数据在分布式存储系统中的虚拟地址间的对应关系。
也就是的说，客户端处理器在从分布式存储系统中获取到目标数据以后，还需要将该目标数据缓存在本地缓存中，并建立该目标数据在本地缓存中的物理地址与该目标数据在目标进程中的虚拟地址间的映射关系。然后，客户端处理器将该映射关系保存在第一地址映射关系表中。使得当目标进程需要再次获取目标数据时，可以直接从本地缓存中进行获取。节省了目标进程获取目标数据的时延开销。
在一个可能的实现方式中，该方法还包括：在目标进程运行过程中，获取待写入数据在目标进程中的第三虚拟地址；基于第三虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有与第三虚拟地址对应的数据；若本地缓存中存储有与第三虚拟地址对应的数据，将本地缓存中与第三虚拟地址对应的数据替换为待写入数据。
也就是说，客户端进程需要向分布式存储系统中的存储节点写入数据时，客户端进程可以直接将需要写入的数据缓存在客户端的缓存页面中。然后，客户端处理器可以将存储有客户端进程写入数据的缓存页面标记为脏页面，并周期性的将脏页面写回到存储节点中。简化了客户端进程对存储节点的写入过程。
第二方面，本申请提供了一种数据处理装置，包括：
获取模块，用于在目标进程运行过程中，获取目标数据在目标进程中的第一虚拟地址；
处理模块，用于根据第一虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有目标数据，第一地址映射关系表中包括：第一数据集合中的数据在进程中的虚拟地址和第一数据集合中的数据在本地缓存中的物理地址间的对应关系，其中，第一数据集合中的数据预先从分布式存储系统中获取得到；若本地缓存中存储有目标数据，基于第一虚拟地址，查询第一地址映射关系表，得到目标数据在本地缓存中的物理地址；
获取模块，还用于根据目标数据在本地缓存中的物理地址，获取目标数据。
在一个可能的实现方式中，第一地址映射关系表中还包括：第二数据集合中的数据在进程中的虚拟地址和第二数据集合中的数据在分布式存储系统中的虚拟地址间的对应关系，处理模块还用于：
若本地缓中没有存储有目标数据，基于第一虚拟地址，查询第一地址映射关系表，得到目标数据在分布式存储系统中的第二虚拟地址；根据第二虚拟地址，查询第二地址映射关系表，得到目标数据在分布式存储系统中的物理地址；第二地址映射关系表中包括：在分布式存储系统中存储的第三数据集合中的数据的虚拟地址和第三数据集合中的数据在分布式存储系统中的物理地址间的对应关系；其中，第三数据集合中的数据包括：第一数据集合中的数据和第二数据集合中的数据；
获取模块还用于，根据目标数据在分布式存储系统中的物理地址，从分布式存储系统中获取目标数据。
在一个可能的实现方式中，处理模块还用于：
将目标数据存储到本地缓存中；
建立目标数据在目标进程中的第一虚拟地址和目标数据在本地缓存中的物理地址间的第一对应关 系；
用第一对应关系替换第一地址映射关系表中的第二对应关系，第二对应关系为目标数据在目标进程中的虚拟地址和目标数据在分布式存储系统中的虚拟地址间的对应关系。
在一个可能的实现方式中，获取模块还用于，在目标进程运行过程中，获取待写入数据在目标进程中的第三虚拟地址；
处理模块还用于，根据第三虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有与第三虚拟地址对应的数据；若本地缓存中存储有与第三虚拟地址对应的数据，将本地缓存中与第三虚拟地址对应的数据替换为待写入数据。
第三方面，本申请提供了一种计算机存储介质，计算机存储介质中存储有指令，当指令在计算机上运行时，使得计算机执行第一方面所提供的方法。
第四方面，本申请提供了一种包含指令的计算机程序产品，当指令在计算机上运行时，使得计算机执行第一方面所提供的方法。
可以理解的是，上述第二方面至第四方面的有益效果可以参见上述第一方面中的相关描述，在此不再赘述。
附图说明
图1为本申请实施例提供的一种基于mmap的数据访问示意图；
图2为本申请实施例提供的一种应用场景示意图；
图3为本申请实施例提供的一种数据访问方法的流程示意图；
图4为本申请实施例提供的一种访问分布式存储系统中的数据的过程示意图；
图5为本申请实施例提供的又一种访问分布式存储系统中的数据的过程示意图；
图6为本申请实施例提供的又一种访问分布式存储系统中的数据的过程示意图；
图7为本申请实施例提供的一种传统文件访问路径与全局mmap访问路径的比较示意图；
图8为本申请实施例提供的一种数据处理装置的结构示意图；
图9为本申请实施例提供的一种计算设备的结构示意图。
具体实施方式
为了使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图，对本申请实施例中的技术方案进行描述。
在本申请实施例中的描述中，“示例性的”、“例如”或者“举例来说”的任何实施例或设计方案不应该被理解为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”、“例如”或者“举例来说”等词旨在以具体方式呈现相关概念。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
在介绍本发明申请的方案之前，首先对本发明申请实施例中需要用到的关键术语进行解释。
1、mmap是一种内存映射文件的方法，即将文件磁盘中的文件或者其他对象映射到应用程序的进程的地址空间(用户空间)，实现文件磁盘地址和进程虚拟地址空间中一段虚拟地址的一一对映关系。实现这样的关系后，进程就可以采用指针的方式读写操作这一段内存，而系统会自动回写脏页面到对应的文件磁盘上，即完成了对文件的操作而不必再调用read，write等系统调用函数。相反，内核空间对这段区域的修改也可以直接反映到用户空间，从而可以实现不同进程间的文件共享。
2、Bypass，指可以通过特定的触发状态(断电或死机)让两个网络不通过网络安全设备的系统，而直接物理上导通。所以有了Bypass后，当网络安全设备故障以后，还可以让连接在这台设备上的网络相互导通，此时，网络设备不会再对网络中的封包做处理。
3、全局内存池，是指将在物理上分离的多个内存池，在逻辑上组合为一个内存池。
接着对本申请涉及的技术方案进行介绍。
示例性的，图1示出了一种基于mmap的数据访问示意图。如图1所示，电子设备上的操作系统为应用程序的每一个进程分配独立的虚拟地址空间。然后，目标进程(可以是应用程序中的任意一个进程)在用户空间中调用库函数mmap，在目标进程的虚拟地址空间中，寻找一段空闲的满足要求的连续的虚拟地址。然后，将寻找到的目标进程中的连续的虚拟地址与存储在文件空间(磁盘)中的文件相关联。其中，在调用mmap函数时，需要标明被映射的文件的标识(file)、被映射的文件在文件空间(磁盘)上的起点(offset)以及被映射的文件的长度(len)。在建立目标进程的虚拟地址与用户空间中的文件的映射关系以后，操作系统通过内存管理单元(Memory Management Unit，MMU)建立进程的虚拟地址与电子设备内存中的物理地址(Page Cache中的Page)的映射关系，并将该映射关系存储在Page Table中。其中，Page Cache中的Page可以用于对文件空间中的文件内容进行缓存。可以理解的是，每一个进程都会有自己的Page Table，Page Table中存储了该进程中虚拟地址到物理地址的映射关系。
当应用程序的进程需要读取该进程地址空间中的数据时，进程通过电子设备的CPU向MMU发送该需要读取的数据的虚拟地址。MMU接收到CPU发送的虚拟地址之后，开始查询该进程的Page Table，确定该进程的Page Table中是否存在映射，以及该进程的读写权限是否正常。在一个可能的示例中，当Page Table中不存在与需要读取的数据的虚拟地址对应的物理地址(即本地缓存不命中)，CPU产生缺页中断(Page Fau lt)。此时，进程会从用户态切换到内核态，并将Page Fault交给内核的Page Fault Handler进行处理。具体地，CPU可以通过rede/prefetch从文件空间中读取数据，并将读取的数据存储到Page Cache中的Page中。然后，CPU触发MMU建立需要读取的数据的虚拟地址与Page Cache中的物理地址之间的映射关系，并将该映射关系保存在Page Table中。在另一个可能的示例中，当Page Cache中存在与需要读取的数据的虚拟地址对应的物理地址(即本地缓存命中)，CPU可以直接读取Page Cache中的数据。
当应用程序的进程在执行过程中，需要修改文件数据时，一般来说修改的文件数据不会马上同步到磁盘(或者说文件空间)，修改后的数据会先缓存在内存的Page Cache中。此时，可以把这种和磁盘数据不一致的页称为脏页，脏页会在合适的时机同步到磁盘。
虽然，基于mmap的数据访问，使得多进程可以通过mmap访问同一个文件，而被访问的文件在Page Cache中只存放一份，减少了物理空间的占用。以及基于mmap的数据访问能够实现用户空间和内核空间的数据直接交互而省去空间不同，数据不同的繁琐过程。但是，mmap局限于单节点内部对文件、对象、块设备等实现内存映射，没有跨节点映射能力。即基于图1所示的mmap数据访问方式，不能实现分布式存储系统中的全局数据访问。
有鉴于此，本申请实施例提供了一种数据访问方法，可以应用于分布式存储系统。利用分布时存储系统中各个存储节点的物理存储器，组成全局地址空间，并对外提供分布式物理地址空间(distributed physical address space，DPAS)，分布式物理地址通过分布式页表(distributed page table，DPT)映射到分布式虚拟地址(distributed virtual address，DVA)。分布式存储系统中的全局数据块可以基于DVA进行构建。其中，全局数据块包括：用户文件或者对象。进一步地，分布式存储系统中的客户(Client)端中的应用程序可以通过distributed mmap将全局数据块映射到本地进程的地址空间中，并通过load/store对其进行访问。使得分布式存储系统中的Client端的应用程序更快的访问分布式存储节点中的数据。
示例性的，图2示出了本发明申请实施例提供的一种应用场景示意图。如图2所示，包括分布式存储系统和Client端。分布式存储系统中包括多个存储节点(Node)。存储节点可以用于存储数据。在一些可能的示例中，存储节点中可以包括:动态随机存取存储器(dynamic random access memory，DRAM)、PM等物理存储介质。多个存储节点中的物理存储介质，组成了分布式存储系统的全局地址空间，并对外提供分布式物理地址空间。分布式物理地址空间中的分布式物理地址(distributed physical address，DPA)可以通过DPT映射到DVA。基于分布式存储系统的DVA可以构建分布式存储系统中的全局数据块。其中，DPA可以通过全局内存总线访问存储节点中的数据，以及Bypass存储节点的CPU。
Client端上安装有应用程序。当应用程序运行时，应用程序可以通过调用Distributedmmap函数，将分布式存储系统中基于DVA构建的全局数据块中的数据映射到Client端的本地进程的地址空间中， 并通过load/store对映射到本地进程的地址空间中的数据进行读写操作。
示例性的，本申请实施例中提供了一种数据的访问方法的流程示意图。该方法可以应用于图2所示的分布式存储系统。参见图3，该方法包括：S301-S306。
S301，建立分布式存储系统中的全局数据块。
在本实施例中，由于针对的是分布式存储系统中存储的数据。所以，在对分布式存储系统中的数据进行访问之前，分布式存储系统中的管理软件可以先构建分布式存储系统的全局数据块。
在一个可能的示例中，如图4所示，分布式存储系统中的多个存储节点(Storage node)中的每一个存储节点中的存储介质(比如，AEP、DRAM、XL-Flash)构成了分布式存储系统的全局地址空间，对外提供分布式物理地址地址空间。其中，分布式存储系统的分布式物理地址空间可以通过DPA进行访问。即获取到DPA以后，根据获取的DPA可以直接通过全局内存数据总线直接访问到存储节点中的数据。
分布式存储系统中的管理软件在获取到分布分布式存储系统中的DPA以后，为了让使用该分布式存储系统的用户都可以共享整个分布式存储系统的存储空间，还需要将分布式存储系统的DPA映射为DVA。在将分布式存储系统的DPA映射为DVA时，需要在分布式存储系统中生成DPT。该DPT可以存储在分布式存储系统的任意一个存储节点，且该DPT用于存放分布存储系统的DVA到DPT的映射关系。
在得到分布式存储系统的DVA以后，分布式存储系统中的管理软件可以基于得到的分布式存储系统的DVA构建分布式存储系统的全局数据块。在一个可能的示例中，分布式存储系统的全局数据块可以包括：用户文件或者对象。
S302，将分布式存储系统中的全局数据块映射到客户端的目标进程的虚拟地址空间。
在本实施例中，Client端的操作系统会为每一个进程都分配一个固定大小的虚拟地址空间。其中，进程的虚拟地址空间由多个虚拟内存区域构成。虚拟内存区域是进程的虚拟地址空间中的一个同质区间，即具有同样特性的连续地址范围。在一个可能的示例中，如图4所示，图4中的Client端的进程的虚拟地址空间中包括：栈区(stack)、文件映射区、堆(heap)、bss数据段、初始数据段、text数据段。其中，文件映射区为堆、栈之间的空余部分。
在启动映射的过程中，目标进程可以在用户空间调用库函数mmap。库函数mmap的原型为：
void*mmap(viod*start,size_t length,intprot,int flags,intfd,off_t offset)。
其中，参数start指向欲映射的内存的起始地址；参数length代表需要将分布式存储系统中的全局数据块中的多大部分映射到内存；参数port表示映射区域的保护方式；参数flags表示影响映射区域的各种特性；参数fd表示要映射到内存中的全局数据块的文件描述符；参数offset表示全局数据块映射时的偏移量，通常设置为0，代表被映射的全局数据块从整个数据块的最前方开始对应，其中，offset必须是分页大小的整数倍。
当目标进程调用库函数mmap将存储在分布式存储节点中的全局数据块，映射到目标进程的虚拟地址空间以后，Client端的CPU还需要将全局数据块的DVA，与全局数据块在目标进程的虚拟地址空间中的虚拟地址(virt)之间的映射关系，保存到Client端的MMU维护的本地页表中，比如图4中的Page Table中保存的映射关系。以及Client端的CPU还需要将全局数据块的DVA，与全局数据块在目标进程的虚拟地址空间中的虚拟地址(virt)之间的映射关系，保存到mmap管理软件中。
S303，在Client端的本地缓存中建立缓存页面，以及该缓存页面与目标进程的虚拟地址之间的映射关系，其中，缓存页面用于缓存分布式存储系统中的全局数据块。
在本实施例中，Client端的CPU可以将Client端的物理内存作为本地缓存。Client端的操作系统在为进程分配虚拟的地址空间以后，还需要通过MMU将该虚拟空间中的虚拟地址映射到client端的本地缓存中。其中，虚拟地址可以通过页表映射到本地缓存，页表由Client端的操作系统维护，并被处理器引用。
在一个可能的示例中，进程的虚拟地址空间可以根据一个固定的大小进行划分，本地缓存也可以设置成不同的页面大小，通常情况下，本地缓存中的缓存页面的大小和虚拟页的大小是一样的。
在Client端建立了本地缓存页面以后，Client端的目标进程可以触发CPU将分布存储节点中存储的数据读取到Client端的本地缓存页面。具体地，Client端的CPU获取到待读取数据在目标进程中的虚拟地以后，Client端的CPU可以根据待读取数据在目标进程中的虚拟地址，查找本地页表得到待读 取数据在分布式存储系统中的DVA，然后Client端的CPU向分布式存储系统发送读取数据请求命令，该命令中携带待读取数据的DVA。分布式存储系统中的CPU获取到该DVA以后，根据该DVA查找分布式存储系统中的DPT，得到待读取数据的DPA。然后，分布式存储系统中的CPU根据该DPT获取到待读取数据，并将该待读取数据发送给Client端的CPU。Client端的CPU接收到该待读取数据以后，将该待读取数据存储在本地缓存页面中。然后，Client端的CPU还需要，建立该缓存页面与待读取数据在进程中的虚拟地址间的映射关系，并用该映射关系替换本地页表中的待读取数据在进程中的虚拟地址与待读取数据的DVA间的映射关系。
在一个可能的示例中，在Client端可以部署有数据处理装置(data processing unit，DPU)。DPU中可以预先缓存部分分布式存储系统中的DPT。当CPU需要将分布式存储系统中的全局数据块读取到本地缓存页面中时，CPU可以根据MMU维护的本地页表获取需要被读取到本地缓存页面中的全局数据块的DVA，即目标DVA。然后，CPU将该目标DVA发送给DPU。由DPU根据接收的目标DVA从分布式存储系统中获取与该目标DVA对应的数据。可以理解是的是，DPU可以是插在Client端的PCIE设备。DPU可以具有自己的操作系统、CPU、存储介质和网络接口等。
在一个可能的示例中，DPU中预先缓存有部分由分布式存储系统维护的DPT。当DPU接收到目标DVA后，DPU可以在DPU中缓存的DPT中查找是否有与该目标DVA对应的DPA。若DPU能够在DPU缓存的DPT中找到与该目标DVA对应的DPA，DPU可以直接根据该DPA从分布式存储系统的存储节点中读取存储在该DPA处的数据，并将该数据发送给Client端的CPU。若DPU中缓存的DPT中不存在与该目标DVA对应的DPA，DPU需要从分布式存储系统维护的DPT中获取与该目标DVA对应的DPA。当DUP从分布式存储系统维护的DPT中获取到与目标DVA对应的DPA后，DPU可以先将该目标DVA与DPA的对应关系保存到DPU缓存的DPT中。然后，DPU可以根据获取到的与该目标DVA对应的DPA，从分布式存储系统的存储节点中读取存储在该DPA处的数据，并将该数据发送给Client端的CPU。Client端的CPU接收到DPU发送的与目标DVA对应的数据以后，CPU将该数据缓存到Client侧的本地缓存的缓存页面中，并触发MMU更新本地页表。其中，MMU更新本地页表是指，将本地页表中保存的目标DVA与目标进程的目标虚拟地址的映射关系，更新为目标虚拟地址与本地内存中的物理地址的映射关系。
在本实施例中，页表也叫做映射表。操作系统想要把目标进程的虚拟地址映射到内存的物理地址，最直观的方法就是建立一张映射表。这个映射表能够实现目标进程的虚拟地址里边的页，到物理内存里边的页的映射。
S304，接收目标进程发送的第一消息，当第一消息用于指示从缓存页面中读数据时，执行S305，当第一消息用于指示向缓存页面中写入数据时，执行S306。
在本实施例中，当目标进程在运行过程中，需要读取目标进程虚拟地址空间中的某一段数据时，目标进程会向CPU发起一个读请求。该读请求携带了需要读取的目标数据块的虚拟地址。当目标进程在运行过程中，需要修改目标进程虚拟地址空间中的某一段数据时，目标进程会向CPU发起一个写请求。该写请求中携带了需要修改的目标数据块的虚拟地址。
S305，根据第一消息中携带的目标进程的虚拟地址读取目标数据。
在本实施例中，当目标进程发送的第一消息用于指示从缓存页面中读取数据时，CPU需要将第一消息中携带的需要读取的目标数据的虚拟地址发送给MMU。MMU接收到目标数据块的虚拟地址以后，MMU需要确定本地页表中是否保存有与该目标数据的虚拟地址对应的本地缓存中的物理地址。当MMU确定本地页表中保存有与该目标数据的虚拟地址对应的本地缓存中的物理地址时，MMU可以触发CPU根据该目标数据的虚拟地址对应的物理地址，直接从本地缓存中读取目标数据。当MMU确定本地页表中不存在与目标数据块的虚拟地址对应的本地缓存中的物理地址时，MMU会触发Client端的CPU产生一个缺页错误，即Page Fault。响应于该缺页错误，CPU从分布式存储系统的存储节点中，获取目标数据块，并缓存到本地缓存页面中。
在一个可能的示例中，如图5所示，在Client端，将Client端的内存作为本地缓存，用于缓存分布式存储系统中的全局数据块。Client端还包括：client端的CPU以及DPU。当目标程序运行过程中产生了Page Fault以后，目标进程会从用户态切换到内核态，并将Page Fault交给内核中的内核终端处理器(Page Fault Handler)进行处理。在Page Fault Handler处理Page Fault的过程中，DPU可以进行Page Fault加速。
参照图5，MMU确定本地页表中不存在与目标数据块的虚拟地址对应的本地缓存中的物理地址时，MMU触发CPU产生Page Fault。响应于该Page Fault，Client侧的CPU中的Page Fault Handler从本地页表中获取目标数据块的虚拟地址对应的DVA，即目标DVA。然后Page fault Handler将获取的目标DVA发送给DPU。当Client侧的Page Fault Handler将目标DVA发送给DPU以后，Client端的操作系统需要进行上下文切换，让出CPU，由DPU进行后续的Page Fault处理。可以理解的是，在本实施例中，“目标数据块的虚拟地址”指的是目标数据块在目标进程的虚拟地址空间中所具有的虚拟地址。
DPU接收到Page fault Handler发送的目标DVA以后，DPU中的CPU可以通过DPU内部的内存管理单元(DPU-MMU)查找DPU内部缓存的DPT中，是否有与目标DVA对应的DPA。当DPU中的CPU确定DPU缓存的DPT中存在有与目标DVA对应的DPA时，DPU内部的CPU可以直接通过DPU的网络接口控制器(network interface controller，NIC)，从分布式存储节点中读取DPA处的数据。当DPU中的CPU确定DPU缓存的DPT中不存在与目标DVA对应的DPA时，DPU内部的CPU需要通过DPU的NIC从分布式存储节点上的DPT中获取与目标DVA对应的DPA。DPU内部的CPU获取到与目标DVA对应的DPA以后，DPU内部的CPU需要触发DPU-MMU在DPU缓存的DPT中增加目标DVA与DPA的映射关系。以及DPU内部的CPU获取到与目标DVA对应的DPA以后，DPU可以通过DPU的NIC，从分布式存储节点中读取DPA处的数据。
DPU获取到目标DVA对应的数据以后，DPU将该数据发送给Page Fault Handler。Page Fault Handler将目标DVA对应的数据存储到Client端的本地缓存中的缓存页面上。然后，Page Fault Handler需要建立该缓存页面，与目标DVA对应的目标进程的虚拟地址之间的映射关系。此时，Page Fault Handler对Client端产生的Page Fault处理完成。目标进程可以正常读取client本地缓存中的数据。
在一个可能额示例中，如图6所示，在Client端，Client端的CPU可以产生Page Fault。但是Client端CPU产生的Page Fault会由DPU进程处理此时，可以使用DPU的内存作为页面缓存(即本地缓存页面)，用于缓存分布式存储系统中的全局数据块。
参照图6，当Client侧的CPU获取到目标进程需要读取的数据的虚拟地址以后，Client侧的CPU通过MMU从本地页表中获取目标数据块的虚拟地址对应的DVA，即目标DVA。然后MMU使用获取的目标地址直接访问DPU。
DPU中的DPU-MMU接收到MMU发送的目标DVA以后，DPU-MMU查询DPU中的本地缓存页面，并进行地址翻译，获取目标DVA对应的物理地址(即缓存页面中的页面地址)。当目标DVA对应的缓存页面在DPU的本地缓存中时，DPU中的CPU可以直接从目标DVA对应的缓存页面中读取数据。当目标DVA对应的缓存页面不在DPU的本地缓存中时，DPU-MMU触发DPU内部的CPU产生Page Fault。响应于该Page Fault，DPU内部的的CPU或者DPU中的其他硬件单元需要通知Client端的操作系统需要进行上下文切换，让出系统CPU。然后，DPU中的Page Fault Handler可以通过DPU内部的内存管理单元(DPU-MMU)查找DPU内部缓存的DPT中，是否有与目标DVA对应的DPA。当DPU中的Page Fault Handler确定DPU缓存的DPT中存在有与目标DVA对应的DPA时，DPU中的Page Fault Handler可以直接通过DPU的NIC，从分布式存储节点中读取DPA处的数据。当DPU中的Page Fault Handler确定DPU缓存的DPT中不存在与目标DVA对应的DPA时，DPU中的Page Fault Handler需要通过DPU的NIC从分布式存储节点上的DPT中获取与目标DVA对应的DPA。Page Fault Handler获取到与目标DVA对应的DPA以后，DPU中的Page Fault Handler需要触发DPU-MMU在DPU缓存的DPT中增加目标DVA与DPA的映射关系。以及DPU中的Page Fault Handler获取到与目标DVA对应的DPA以后，DPU中的Page Fault Handler可以通过DPU的NIC，从分布式存储节点中读取DPA处的数据。
DPU中的Page Fault Handler读取到目标DVA对应的数据以后，还需要将该目标DVA对应的数据写入到DPU内部的本地缓存中的缓存页面上。然后，DPU向MMU发送消息，触发MMU建立DPU中的缓存页面，与目标DVA对应的目标进程的虚拟地址之间的映射关系。此时，DPU内部产生的Page Fault处理完成。目标进程可以正常读取GPU内部本地缓存中的数据。
S306，获取第一消息中携带的待写入数据，并将该待写入数据写入到缓存页面中。
在本实施例中，将待写入数据写入到缓存页面中，指的是覆盖性写入，即新写入的数据需要覆盖原始数据。
在一个可能的示例中，针对图5所示的分布式存储系统。当目标进程向Client端的CPU发送写请 求以后，Client端的CPU获取待写入数据的虚拟地址。该虚拟地址是指“目标进程地址空间中的虚拟地址”。然后，MMU根据待写入数据的虚拟地址确定本地页表中是否保存有与该数据的虚拟地址本地缓存中的物理地址。当MMU确定本地页表中保存有与待写入数据的虚拟地址对应的本地缓存中的物理地址时，MMU可以触发Client端的CPU根据该待写入数据的虚拟地址对应的物理地址，将待写入数据写入到本地缓存的缓存页面中。当MMU确定本地页表中不存在与目标数据块的虚拟地址对应的本地缓存中的物理地址时，MMU会触发Client端的CPU产生一个缺页错误，即Page Fault。响应于该Page Fault，DPU中的CPU从分布式存储系统的存储节点中，获取目标数据块，并缓存到本地缓存页面中。然后，Client端CPU建立待写入数据的虚拟地址与本地缓存之间的映射关系，并根据该映射关系将待写入数据写入到本地缓存页面中。其中，DPU中的CPU响应于Page Fault从分布式存储系统的存储节点中，获取目标数据块，并缓存到本地缓存页面的过程可以参照S306中关于图5的描述，在此不再赘述。
Client端的CPU将待写入数据写入到Client端的本地缓存中以后。Client端的CPU会将写入数据的缓存页面标记为脏页面(dirty page)，并将其加入到dirty list中。然后，Client端的mmap管理软件周期性的将dirty list中的page，写回到分布式存储节点中，使得分布式存储节点中的数据和Client端的内存中缓存的数据一致。可以理解的是，mmap管理软件中保存有全局数据块的DVA与全局数据块在目标进程的虚拟地址空间中的虚拟地址(virt)之间的映射关系。因此，mmap管理软件在将dirty list中的page，写回到分布式存储节点中时，mmap管理软件可以通过本地页表获取dirty list中的page对应的目标进程中的虚拟地址。然后，mmap管理软件可以根据dirty list中的page对应的目标进程中的虚拟地址，以及mmap管理软件中保存的映射关系，将dirty list中的page中存储的数据写回到分布式存储系统中的存储节点上。
在一个可能的示例中，client端的mmap管理软件获取本地缓存中的脏页面，并调用mcommit()发起脏页面同步到分布式存储系统中的存储节点的流程。当脏页面同步到分布式存储系统中的存储节点以后，mcommit()返回成功。
在另一个可能的示例中，client端的mmap管理软件获取本地缓存中的脏页面，通过调用刷新变化函数msync(),将脏页面写回到分布式存储系统中的存储节点上。
在一个可能的示例中，针对图6所示的分布式存储系统。目标进程向Client端的CPU发送写请求以后，Client端的CPU需要将该写请求发送给DPU。DPU中的DPU-MMU根据待写入数据的虚拟地址确定本地页表中是否保存有与该数据的虚拟地址本地缓存中的物理地址。待写入数据的虚拟地址是指“目标进程地址空间中的虚拟地址”。若DPU-MMU确定DPU中的页表中保存有，与待写入数据的虚拟地址对应的DPU的本地缓存中的物理地址。MMU可以触发DPU中的CPU根据该待写入数据的虚拟地址对应的物理地址，将待写入数据写入到DPU中的本地缓存中的缓存页面中。若DPU-MMU确定DPU中的页表中未保存与待写入数据的虚拟地址对应的DPU的本地缓存中的物理地址。DPU-MMU触发DPU内部的CPU产生PageFault。响应于该Page Fault，DPU中的Page Fault Handler从分布式存储系统的存储节点中，获取目标数据块，并缓存到DPU的缓存页面中。然后，DPU中的CPU建立待写入数据的虚拟地址与DPU中的本地缓存中的缓存页面的映射关系，并根据该映射关系将待写入数据写入到本地缓存页面中。其中，DPU中的Page Fault Handler响应于Page Fault从分布式存储系统的存储节点中，获取目标数据块，并缓存到DPU的本地缓存页面的过程，可以参照S306中关于图6的描述，在此不再赘述。
DPU中的CPU将目标进程需要写的数据写入到DPU中的本地缓存中的缓存页面以后。DPU中的CPU会将写入数据的缓存页面标记为脏页面(dirty page)，并将其加入到dirty list中。然后，GPU中的mmap管理软件会周期性的将dirty list中的page写回到分布式存储节点中，使得分布式存储节点中的数据和Client端的内存中缓存的数据一致。在一个可能的示例中，DPU中的mmap管理软件获取本地缓存中的脏页面，并调用mcommit()发起脏页面同步到分布式存储系统中的存储节点的流程。当脏页面同步到分布式存储系统中的存储节点以后，mcommit()返回成功。在另一个可能的示例中，DPU中的mmap管理软件获取本地缓存中的脏页面，通过调用刷新变化函数msync(),将脏页面写回到分布式存储系统中的存储节点上。
针对上述实施例例中的方法，本申请实施例还提供了一种传统文件访问路径与全局mmap访问路径的比较示意图。参照图7可知，传统文件访问路径长，需要经过VFS，文件系统客户端，文件系统服务端，最终访问PM，软件栈开销大。而在本申请实施例中，在分布式存储系统中，使用Distributed mmap  映射远端文件之后，在本地(Client端)记录文件的DVA(可记录在进程页表中)，本地缓存不命中，通过Page Fault访问远端DVA空间。NIC直通访问PM介质，Bypass Server端的软件栈开销，端到端软件栈时延从150us降20us。
在本申请实施例中，针对分布式存储系统，构建分布式存储系统中的全局共享内存池。然后，基于该全局内存池构建全局数据块，并将该全局数据块映射到客户端的进程地址空间中。然后，客户端建立用于缓存分布式系统中的全局数据块的本地缓存，并通过客户端的本地页表，建立进程虚拟地址到缓存页面的映射。及在分布式存储系统中，对存储节点中存储的文件进行mmap映射，使得客户端的应用程序可以直接通过load/store访问，存储节点中的数据。在分布式存储系统中构建基于全局内存池的全局数据块，使得属于同一个分布式存储系统中的多个客户端可mmap同一个全局数据块，实现全局数据块在各个客户端之间的共享访问。以及在分布式存储系统中使用Distributed mmap映射远端文件之后，在本地记录文件的DVA(可记录在进程页表中)。当本地缓存不命中，通过Page Fault访问远端DVA空间，节省端到端软件栈的时延开销。
示例性的，本申请实施例还提供了一种数据处理装置，请参阅图8。图8是本申请实施例提供的一种数据处理装置的结构示意图。如图8所示，该数据处理装置中包括：获取模块810、处理模块820、存储模块830。
存储模块830用于存储目标进程运行过程中所需要的部分数据，以及存储模块830中还存储有本地页表也可以称之为第一地址映射关系表。本地页表中存储有分布式存储系统中的全局数据块的DVA与目标进程中的虚拟地址间的映射关系、以及目标进程中的虚拟地址与数据处理装置中的本地缓存的物理地址间的映射关系。
获取模块810用于在目标进程过程中，获取目标数据在目标进程中的虚拟地地址，也可以称之为第一虚拟地址。
处理模块820用于根据获取的目标数据的虚拟地址，查询本地页表确定本地页表中是否存储有与该目标数据的虚拟地址对应的本地缓存的物理地址。当处理模块820确定本地页表中是存储有与该目标数据的虚拟地址对应的本地缓存的物理地址时，处理模块820获取该物理地址。
获取模块810根据处理模块820获取的物理地址，在本地缓存中获取目标数据。
在一个可能的示例中，当处理模块820确定本地页表中是存储有与该目标数据的虚拟地址对应的本地缓存的物理地址时，处理模块820根据目标数据的虚拟地址查询本地页表，得到目标数据的DVA。然后，处理模块810根据目标数据的DVA确定目标数据的DPA。获取模块根据处理模块得到的目标数据DPA从分布式存储系统中获取目标数据。
其中，在数据处理装置中，访问目标进程所需要的目标数据的具体过程，可以参照S302-S306。在此，不再赘述。
示例性的，本申请实施例中还提供了一种计算设备，该计算设备至少包括处理器和存储器，存储器上存储有程序，处理器该程序时，可以实现图3所示的方法中的各个步骤的单元或模块。
图9为本申请实施例提供的一种计算设备的结构示意图。如图9所示，所述计算设备900包括至少一个处理器910、存储器920和通信接口930。其中，处理器910、存储器920和通信接口930通信连接，可以通过有线(例如总线)的方式实现通信连接，也可以通过无线的方式实现通信连接。该通信接口930用于接收其他设备发送的数据(例如DUT对应的IR)；存储器920存储有计算机指令，处理器910执行该计算机指令，执行前述方法实施例中的方法。
应理解，在本申请实施例中，该处理器910可以是中央处理单元CPU，该处理器910还可以是其他通用处理器、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者是任何常规的处理器等。
该存储器920可以包括只读存储器和随机存取存储器，并向处理器910提供指令和数据。存储器920还可以包括非易失性随机存取存储器。
该存储器920可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其 中，非易失性存储器可以是只读存储器(read-only memory,ROM)、可编程只读存储器(programmable ROM,PROM)、可擦除可编程只读存储器(erasable PROM,EPROM)、电可擦除可编程只读存储器(electrically EPROM,EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory,RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM,SRAM)、动态随机存取存储器(DRAM)、同步动态随机存取存储器(synchronous DRAM,SDRAM)、双倍数据速率同步动态随机存取存储器(double data date SDRAM,DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM,ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM,SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM,DR RAM)。
应理解，根据本申请实施例的计算设备900可以执行实现本申请实施例中图3所示方法，该方法实现的详细描述参见上文，为了简洁，在此不再赘述。
本申请的实施例提供了一种计算机可读存储介质，其上存储有计算机程序，当所述计算机指令在被处理器执行时，使得上文提及的方法被实现。
本申请的实施例提供了一种芯片，该芯片包括至少一个处理器和接口，所述至少一个处理器通过所述接口确定程序指令或者数据；该至少一个处理器用于执行所述程序指令，以实现上文提及的方法。
本申请的实施例提供了一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品包括指令，当该指令执行时，令计算机执行上文提及的方法。
本申请的实施例中的方法步骤可以通过硬件的方式来实现，也可以由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(random access memory，RAM)、闪存、只读存储器(read-only memory，ROM)、可编程只读存储器(programmablerom，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、CD-ROM或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者通过所述计算机可读存储介质进行传输。所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
可以理解的是，在本申请的实施例中涉及的各种数字编号仅为描述方便进行的区分，并不用来限制本申请的实施例的范围。

Claims (10)

  1. 一种数据访问方法，其特征在于，所述方法包括：
    在目标进程运行过程中，获取目标数据在所述目标进程中的第一虚拟地址；
    基于所述第一虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有所述目标数据，所述第一地址映射关系表中包括：第一数据集合中的数据在进程中的虚拟地址和所述第一数据集合中的数据在本地缓存中的物理地址间的对应关系，其中，所述第一数据集合中的数据预先从分布式存储系统中获取得到；
    若本地缓存中存储有所述目标数据，基于所述第一虚拟地址，查询所述第一地址映射关系表，得到所述目标数据在本地缓存中的物理地址；
    根据所述目标数据在本地缓存中的物理地址，获取所述目标数据。
  2. 根据权利要求1所述的方法，其特征在于，所述第一地址映射关系表中还包括：第二数据集合中的数据在进程中的虚拟地址和所述第二数据集合中的数据在所述分布式存储系统中的虚拟地址间的对应关系，所述方法还包括：
    若本地缓中没有存储有所述目标数据，基于所述第一虚拟地址，查询所述第一地址映射关系表，得到所述目标数据在所述分布式存储系统中的第二虚拟地址；
    基于所述第二虚拟地址，查询第二地址映射关系表，得到所述目标数据在所述分布式存储系统中的物理地址；所述第二地址映射关系表中包括：在分布式存储系统中存储的第三数据集合中的数据的虚拟地址和所述第三数据集合中的数据在所述分布式存储系统中的物理地址间的对应关系；其中，所述第三数据集合中的数据包括：所述第一数据集合中的数据和所述第二数据集合中的数据；
    基于所述目标数据在所述分布式存储系统中的物理地址，从所述分布式存储系统中获取所述目标数据。
  3. 根据权利要求2所述的方法，其特征在于，在基于所述目标数据在所述分布式存储系统中的物理地址，从所述分布式存储系统中获取所述目标数据以后，所述方法还包括：
    将所述目标数据存储到本地缓存中；
    建立所述目标数据在所述目标进程中的第一虚拟地址和所述目标数据在本地缓存中的物理地址间的第一对应关系；
    用所述第一对应关系替换所述第一地址映射关系表中的第二对应关系，所述第二对应关系为所述目标数据在所述目标进程中的虚拟地址和所述目标数据在所述分布式存储系统中的虚拟地址间的对应关系。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    在所述目标进程运行过程中，获取待写入数据在所述目标进程中的第三虚拟地址；
    基于所述第三虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有与第三虚拟地址对应的数据；
    若所述本地缓存中存储有与所述第三虚拟地址对应的数据，将所述本地缓存中与所述第三虚拟地址对应的数据替换为所述待写入数据。
  5. 一种数据处理装置，其特征在于，包括：
    获取模块，用于在目标进程运行过程中，获取目标数据在所述目标进程中的第一虚拟地址；
    处理模块，用于根据所述第一虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有所述目标数据，所述第一地址映射关系表中包括：第一数据集合中的数据在进程中的虚拟地址和所述第一数据集合中的数据在本地缓存中的物理地址间的对应关系，其中，所述第一数据集合中的数据预先从分布式存储系统中获取得到；若本地缓存中存储有所述目标数据，基于所述第一虚拟地址，查询所述第一地址映射关系表，得到所述目标数据在本地缓存中的物理地址；
    所述获取模块，还用于根据所述目标数据在本地缓存中的物理地址，获取所述目标数据。
  6. 根据权利要求5所述的装置，其特征在于，所述第一地址映射关系表中还包括：第二数据 集合中的数据在进程中的虚拟地址和所述第二数据集合中的数据在所述分布式存储系统中的虚拟地址间的对应关系，所述处理模块还用于：
    若本地缓中没有存储有所述目标数据，基于所述第一虚拟地址，查询所述第一地址映射关系表，得到所述目标数据在所述分布式存储系统中的第二虚拟地址；根据所述第二虚拟地址，查询第二地址映射关系表，得到所述目标数据在所述分布式存储系统中的物理地址；所述第二地址映射关系表中包括：在分布式存储系统中存储的第三数据集合中的数据的虚拟地址和所述第三数据集合中的数据在所述分布式存储系统中的物理地址间的对应关系；其中，所述第三数据集合中的数据包括：所述第一数据集合中的数据和所述第二数据集合中的数据；
    所述获取模块还用于，根据所述目标数据在所述分布式存储系统中的物理地址，从所述分布式存储系统中获取所述目标数据。
  7. 根据权利要求6所述的装置，其特征在于，所述处理模块还用于：
    将所述目标数据存储到本地缓存中；
    建立所述目标数据在所述目标进程中的第一虚拟地址和所述目标数据在本地缓存中的物理地址间的第一对应关系；
    用所述第一对应关系替换所述第一地址映射关系表中的第二对应关系，所述第二对应关系为所述目标数据在所述目标进程中的虚拟地址和所述目标数据在所述分布式存储系统中的虚拟地址间的对应关系。
  8. 根据权利要求5所述的装置，其特征在于，所述获取模块还用于，在目标进程运行过程中，获取待写入数据在所述目标进程中的第三虚拟地址；
    所述处理模块还用于，根据所述第三虚拟地址，查询第一地址映射关系表，确定本地缓存中是否存储有与第三虚拟地址对应的数据；若所述本地缓存中存储有与所述第三虚拟地址对应的数据，将所述本地缓存中与所述第三虚拟地址对应的数据替换为所述待写入数据。
  9. 一种计算机可读介质，所述计算机存储介质中存储有指令，当所述指令在计算机上运行时，使得计算机执行如权利要求1-4任一所述的方法。
  10. 一种包含指令的计算机程序产品，当所述指令在计算机上运行时，使得所述计算机执行如权利要求1-4任一所述的方法。
PCT/CN2023/100809 2022-09-30 2023-06-16 一种数据的访问方法及装置 Ceased WO2024066496A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP23869745.2A EP4579447A4 (en) 2022-09-30 2023-06-16 METHOD AND APPARATUS FOR ACCESSING DATA

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211213882.0A CN117851281A (zh) 2022-09-30 2022-09-30 一种数据的访问方法及装置
CN202211213882.0 2022-09-30

Publications (1)

Publication Number Publication Date
WO2024066496A1 true WO2024066496A1 (zh) 2024-04-04

Family

ID=90475894

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/100809 Ceased WO2024066496A1 (zh) 2022-09-30 2023-06-16 一种数据的访问方法及装置

Country Status (3)

Country Link
EP (1) EP4579447A4 (zh)
CN (1) CN117851281A (zh)
WO (1) WO2024066496A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119829620A (zh) * 2024-12-31 2025-04-15 深圳计算科学研究院 一种数据缓冲区的流量控制方法、装置、设备及介质
CN121166043A (zh) * 2025-11-24 2025-12-19 阿里云计算有限公司 数据处理方法、电子设备、系统、存储介质和程序产品

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119718216B (zh) * 2025-02-27 2025-05-06 苏州元脑智能科技有限公司 元数据的落盘方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180232320A1 (en) * 2016-07-29 2018-08-16 Advanced Micro Devices, Inc. Controlling Access by IO Devices to Pages in a Memory in a Computing Device
CN109032963A (zh) * 2017-06-12 2018-12-18 Arm有限公司 访问控制
CN113986137A (zh) * 2021-10-28 2022-01-28 英韧科技(上海)有限公司 存储装置和存储系统
CN114968539A (zh) * 2021-02-22 2022-08-30 华为技术有限公司 数据处理方法、计算机系统和中间设备

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8499121B2 (en) * 2011-08-31 2013-07-30 Hewlett-Packard Development Company, L.P. Methods and apparatus to access data in non-volatile memory
US9507729B2 (en) * 2013-10-01 2016-11-29 Synopsys, Inc. Method and processor for reducing code and latency of TLB maintenance operations in a configurable processor

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180232320A1 (en) * 2016-07-29 2018-08-16 Advanced Micro Devices, Inc. Controlling Access by IO Devices to Pages in a Memory in a Computing Device
CN109032963A (zh) * 2017-06-12 2018-12-18 Arm有限公司 访问控制
CN114968539A (zh) * 2021-02-22 2022-08-30 华为技术有限公司 数据处理方法、计算机系统和中间设备
CN113986137A (zh) * 2021-10-28 2022-01-28 英韧科技(上海)有限公司 存储装置和存储系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4579447A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119829620A (zh) * 2024-12-31 2025-04-15 深圳计算科学研究院 一种数据缓冲区的流量控制方法、装置、设备及介质
CN121166043A (zh) * 2025-11-24 2025-12-19 阿里云计算有限公司 数据处理方法、电子设备、系统、存储介质和程序产品

Also Published As

Publication number Publication date
CN117851281A (zh) 2024-04-09
EP4579447A4 (en) 2025-09-10
EP4579447A1 (en) 2025-07-02

Similar Documents

Publication Publication Date Title
EP4579447A1 (en) Data access method and apparatus
US7206915B2 (en) Virtual space manager for computer having a physical address extension feature
WO2024230779A1 (zh) 一种文件访问方法、系统、电子设备及机器可读存储介质
US20050198441A1 (en) Multiprocessor system
WO2021008425A1 (zh) 一种系统启动方法以及相关设备
CN107341114B (zh) 一种目录管理的方法、节点控制器和系统
WO2024260039A1 (zh) 数据访问方法及装置、非易失性可读存储介质及电子装置
CN109901909B (zh) 用于虚拟化系统的方法及虚拟化系统
CN116414563A (zh) 内存控制装置、缓存一致性系统和缓存一致性方法
CN116028455A (zh) 一种数据处理方法、装置、存储介质及电子设备
CN112130747A (zh) 分布式对象存储系统及数据读写方法
CN116684385A (zh) 一种基于eBPF在内核层面的DNS缓存方法
CN103297490A (zh) 信息处理装置、分布式处理系统和分布式处理方法
WO2022262623A1 (zh) 一种数据交换方法及装置
CN117806526A (zh) 数据迁移方法、装置、芯片以及计算机可读存储介质
CN111273860A (zh) 一种基于网络和页粒度管理的分布式内存管理方法
CN117312004A (zh) 一种数据库的访问方法及装置
CN110018879A (zh) 应用于分布式系统的延迟加载方法及装置
US20230088344A1 (en) Storage medium management method and apparatus, device, and computer-readable storage medium
CN113806389A (zh) 一种数据处理方法、装置、计算设备与存储介质
WO2023217255A1 (zh) 数据处理方法、装置、处理器及计算机系统
CN115904625A (zh) 跨节点多虚拟机内存管理方法、系统、终端及介质
CN116866429A (zh) 一种数据访问方法及相关装置
CN107402818A (zh) 一种客户端读写缓存分离的方法及系统
WO2024082702A1 (zh) 数据处理方法、装置、芯片以及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23869745

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023869745

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023869745

Country of ref document: EP

Effective date: 20250325

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 2023869745

Country of ref document: EP