WO2016119618A1 - Procédé, dispositif et système d'attribution de mémoire déportée - Google Patents

Procédé, dispositif et système d'attribution de mémoire déportée Download PDF

Info

Publication number
WO2016119618A1
WO2016119618A1 PCT/CN2016/071483 CN2016071483W WO2016119618A1 WO 2016119618 A1 WO2016119618 A1 WO 2016119618A1 CN 2016071483 W CN2016071483 W CN 2016071483W WO 2016119618 A1 WO2016119618 A1 WO 2016119618A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
node
remote
computing node
storage space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/071483
Other languages
English (en)
Chinese (zh)
Inventor
李辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of WO2016119618A1 publication Critical patent/WO2016119618A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a remote memory allocation method, apparatus, and system.
  • Physical memory is one of the important components in a computer, and it is a bridge to communicate with the CPU. All programs in the computer run in memory, so the performance of physical memory has a huge impact on the computer.
  • the first terminal device when the physical memory of the first terminal device is too small to meet the requirement, the first terminal device usually requests remote memory from other terminal devices to expand the physical memory of the first terminal device.
  • other terminal devices for the sake of convenience, the following "second terminal device” replaces “other terminal device"
  • a terminal device to serve as the remote memory of the first terminal device A terminal device to serve as the remote memory of the first terminal device.
  • the disadvantage of this scheme is that since the physical memory size of the second terminal device is fixed, the remote memory allocated to the computing node cannot exceed the upper limit of the free physical memory in the second terminal device.
  • the present invention provides a remote memory allocation method, which is used to limit the allocation of remote memory to a computing node without being limited by the upper limit of physical memory in the memory node.
  • an embodiment of the present invention provides a remote memory allocation method, where the method includes:
  • the memory node receives a request sent by the computing node, the request instructing the computing node to apply for remote memory of the XB to the memory node, where the memory node and the computing node are connected by network communication, the X Greater than zero and less than 2 n , the n is the data width of the memory node CPU general purpose register;
  • the memory node allocates the said node from the free kernel memory of the memory node The first kernel memory of XB;
  • the method After the memory node allocates the first kernel memory of the XB from the free kernel memory of the memory node for the computing node, and before the computing node reads and writes data to the remote memory, the method also includes:
  • the memory node maps the first kernel memory to the first storage space of the memory node, so as to allocate the remote memory to the computing node, so that the computing node is to the first storage space Read and write data, the size of the first storage space is the XB; the first storage space is physical memory, or the first storage space is virtual memory, or the first storage space is physical memory And virtual memory.
  • the request further indicates that the remote memory is a remote physical memory
  • the request further indicates that the remote memory is a remote virtual memory
  • the request further indicates that the remote memory is remote physical memory and remote virtual memory.
  • the memory node maps the first kernel memory to the first storage space of the memory node, so as to allocate the remote memory to the computing node, so that the computing node is to the first storage space Reading and writing data, the size of the first storage space is the XB, and specifically includes:
  • the memory node maps the first kernel memory to a first physical memory of the memory node to implement allocating the remote physical memory to the computing node, thereby causing the computing node to the first physical
  • the memory reads and writes data, and the size of the first physical memory is the XB.
  • the first embodiment of the first aspect or the second embodiment of the first aspect in a third embodiment of the first aspect,
  • the request further indicates a first start address, where the first start address refers to an address of the remote memory start location facing the computing node;
  • the method further includes:
  • the memory node establishes a mapping relationship between the first start address and a second start address, where the second start address refers to an address of the first kernel memory start position in the free core memory.
  • the first embodiment of the first aspect or the third embodiment of the first aspect in a fourth implementation manner of the first aspect,
  • the memory node allocates the first kernel memory of the XB to the computing node from the free kernel memory of the memory node, and specifically includes:
  • the memory node allocates the first kernel memory from the free core memory to the computing node.
  • an embodiment of the present invention provides a remote memory allocation method, where the method includes:
  • the memory node receives a request sent by the computing node, where the request indicates that the computing node requests the remote memory of the XB from the memory node, where the X is greater than zero and less than 2 n , and the n is the memory node CPU a data width of the general purpose register, wherein the memory node and the computing node are connected by a network communication;
  • the memory node allocates, according to the request, the first logical memory of the XB from the idle logical memory of the first target process to the computing node, where the first target process refers to running in the memory node, a first interface that interacts with the computing node and can allocate a logical memory to the computing node through the first interface, where the idle logical memory of the first target process is greater than or equal to the XB;
  • the method also includes:
  • the memory node Mapping, by the memory node, the first logical memory to a first storage space of the memory node, to implement allocating the remote memory to the computing node, thereby causing the computing node to move to the first storage space Reading and writing data; wherein the size of the first storage space is the XB, the first storage space is physical memory and virtual memory, or the first storage space is physical memory, or the first The storage space is virtual memory.
  • the method After the memory node receives the request sent by the computing node, and the memory node allocates the first logical memory of the XB from the idle logical memory of the first target process to the computing node according to the request, the method also includes:
  • the memory node selects the first target process from among all processes running on the memory node.
  • the method After the memory node receives the request sent by the computing node, and the memory node allocates the first logical memory of the XB from the idle logical memory of the first target process to the computing node according to the request, the method also includes:
  • the memory node creates the first target process and allocates logical memory for the first target process.
  • the request further indicates that the remote memory is a remote physical memory
  • the request further indicates that the remote memory is a remote virtual memory
  • the request further indicates that the remote memory is remote physical memory and remote virtual memory.
  • a fourth implementation manner of the second aspect in the case that the request further indicates that the remote memory is a remote physical memory,
  • Mapping by the memory node, the first logical memory to a first storage space of the memory node, to implement allocating the remote memory to the computing node, thereby causing the computing node to move to the first storage space
  • Read and write data including:
  • the memory node maps the first logical memory to a first physical memory of the memory node to implement allocating the remote physical memory to the computing node, thereby causing the computing node to the first physical
  • the memory reads and writes data, and the size of the first physical memory is the XB.
  • the request further indicates a first start address, where the first start address refers to an address of the remote memory start location facing the computing node;
  • the method further includes:
  • the memory node establishes a mapping relationship between the first start address and a second start address, where the second start address refers to an address of the first logical memory start position in the idle logical memory .
  • the number of cores of the memory node is greater than the number of processes running the memory node before the first target process is created
  • the number of processes running on the memory node does not exceed The number of cores of the memory node.
  • an embodiment of the present invention provides a remote memory allocation device, where the device includes:
  • a receiving unit configured to receive a request sent by the computing node, where the request instructs the computing node to apply for a remote memory of the XB to the memory node, where the X is greater than zero and less than 2 n , and the n is the CPU of the memory node
  • the data width of the register
  • An allocating unit configured to allocate, by the free kernel memory of the memory node, the first kernel memory of the XB to the computing node;
  • the allocating unit allocates the first kernel memory of the XB from the free kernel memory of the memory node to the computing node, and before the computing node reads and writes data to the remote memory,
  • mapping unit configured to map the first kernel memory to a first storage space of the memory node, to implement allocating the remote memory to the computing node, thereby causing the computing node to the first storage Spatially reading and writing data, the size of the first storage space is the XB; the first storage space is physical memory, or the first storage space is virtual memory, or the first storage space is physical Memory and virtual memory.
  • the request further indicates that the remote memory is a remote physical memory
  • the request further indicates that the remote memory is a remote virtual memory
  • the request further indicates that the remote memory is remote physical memory and remote virtual memory.
  • the mapping unit is specifically configured to map the first kernel memory to the first physical memory of the memory node, so as to implement allocating the remote physical memory to the computing node, thereby causing the computing node to
  • the first physical memory reads and writes data, and the size of the first physical memory is the XB.
  • the request further indicates a first start address, where A starting address is an address of the remote memory starting location facing the computing node;
  • the device also includes:
  • the establishing unit is configured to establish a mapping relationship between the first starting address and the second starting address, where the second starting address is an address of the first kernel memory starting position in the free core memory.
  • an embodiment of the present invention provides a remote memory allocation device, where the device includes:
  • a receiving unit configured to receive a request sent by the computing node, where the request instructs the computing node to apply for a remote memory of the XB to the memory node, where the X is greater than zero and less than 2 n , and the n is the memory node a data width of a CPU general purpose register, wherein the memory node and the computing node are connected by a network communication;
  • An allocating unit configured to allocate, according to the request, the first logical memory of the XB from the idle logical memory of the first target process, where the first target process refers to running in the memory node, a process having a first interface that interacts with the computing node and capable of allocating logical memory to the computing node through the first interface, the idle logical memory of the first target process is greater than or equal to the XB;
  • the allocating unit allocates the first logical memory of the XB from the idle logical memory of the first target process to the computing node, and before the computing node reads and writes data to the remote memory,
  • mapping unit configured to map the first logical memory to a first storage space of the memory node, to implement allocating the remote memory to the computing node, so that the computing node is to the first storage Spatially reading and writing data, the size of the first storage space is the XB; the first storage space is physical memory and virtual memory, or the first storage space is physical memory, or the first storage The space is virtual memory.
  • the device After the receiving unit receives the request sent by the computing node, and the allocating unit allocates the first logical memory of the XB from the idle logical memory of the first target process to the computing node according to the request,
  • the device also includes:
  • a selection unit for selecting the first target process from among all processes running on the memory node.
  • the device After the receiving unit receives the request sent by the computing node, and the allocating unit allocates the first logical memory of the XB from the idle logical memory of the first target process to the computing node according to the request,
  • the device also includes:
  • the request further indicates that the remote memory is a remote physical memory
  • the request further indicates that the remote memory is a remote virtual memory
  • the request further indicates that the remote memory is remote physical memory and remote virtual memory.
  • the mapping unit is specifically configured to map the first logical memory to the first physical memory of the memory node, so as to implement allocating the remote physical memory to the computing node, thereby causing the computing node to
  • the first physical memory reads and writes data, and the size of the first physical memory is the XB.
  • the request further indicates a first start address, where the first start address refers to an address of the remote memory start location facing the computing node;
  • the device also includes:
  • a establishing unit configured to establish a mapping relationship between the first starting address and a second starting address, where the second starting address refers to a starting position of the first logical memory in the idle logical memory address.
  • the number of cores of the memory node is greater than the number of processes running the memory node before the first target process is created
  • the number of processes running on the memory node does not exceed the number of cores of the memory node.
  • an embodiment of the present invention provides a remote memory allocation system, where the system includes a computing node, a memory node, and a third implementation manner of the first to third aspects of the third aspect or the third aspect.
  • the computing node is configured to send the request received by the remote memory allocation device
  • the memory node is configured to provide the first storage space
  • the computing node is further configured to acquire the first storage space as the remote memory.
  • the remote memory allocation device is integrated in the memory node.
  • an embodiment of the present invention provides a remote memory allocation system, where the system includes a computing node, a memory node, and a sixth implementation manner of the first to fourth aspects of the fourth aspect or the fourth aspect.
  • the computing node is configured to send the request received by the remote memory allocation device
  • the memory node is configured to provide the first storage space
  • the computing node is further configured to acquire the first storage space as the remote memory.
  • the remote memory allocation device is integrated in the memory node. It can be seen that, in the remote memory allocation method provided by the embodiment of the present invention, the memory node allocates the first kernel memory from the free kernel memory of the memory node according to the request of the computing node, and then maps the first kernel memory to The first storage space is configured to allocate remote memory for the compute node.
  • the kernel memory is the memory allocated by the operating system for the kernel object, and is the logical memory.
  • the size of the kernel memory is 2 n B, where n is the data width of the general-purpose register of the memory node CPU. That is, the kernel memory is much larger than the physical memory of the memory node.
  • the kernel memory can be mapped not only to physical memory but also to virtual memory (ie, virtual memory expanded from external memory). Therefore, with the technical solution provided by the embodiment of the present invention, when the computing node applies for the remote memory to the memory node, the computing node can apply for a larger remote memory to the memory node without being limited by the upper limit of the free physical memory in the memory node.
  • FIG. 1 is a schematic diagram of an application scenario of a remote memory allocation method according to an embodiment of the present disclosure
  • FIG. 1b is a schematic flowchart of a remote memory allocation method according to an embodiment of the present invention.
  • 1c is a schematic flowchart of a remote memory allocation method according to an embodiment of the present invention.
  • FIG. 2a is a schematic structural diagram of a remote memory allocation device according to an embodiment of the present invention.
  • FIG. 2b is a schematic structural diagram of another remote memory allocation device according to an embodiment of the present invention.
  • 2c is a schematic structural diagram of still another remote memory allocation device according to an embodiment of the present invention.
  • FIG. 2d (1) is a schematic structural diagram of still another remote memory allocation device according to an embodiment of the present invention.
  • 2d (2) is a schematic structural diagram of still another remote memory allocation device according to an embodiment of the present invention.
  • 2 e is a schematic structural diagram of still another remote memory allocation device according to an embodiment of the present invention.
  • FIG. 3a is a schematic structural diagram of a remote memory allocation system according to an embodiment of the present invention.
  • FIG. 3b is a schematic structural diagram of another remote memory allocation system according to an embodiment of the present invention.
  • each process can obtain an independent logical address space, thereby accessing the physical address through a logical address, and can provide protection of the logical address space.
  • the general processor includes a Memeory Management Unit (MMU) and a Translation Lookside Buffer (TLB) to optimize memory access performance.
  • MMU Memeory Management Unit
  • TLB Translation Lookside Buffer
  • virtual memory It refers to the dynamic expansion of physical memory based on virtual memory technology. Specifically, the expansion of physical memory is realized by using an external storage device (HDD/SDD/NVM, etc.).
  • an external storage device HDD/SDD/NVM, etc.
  • a schematic diagram of an application scenario of a remote memory allocation method includes a memory node 11 and a computing node 13, and a high-speed interconnect interface between the memory node 11 and the computing node 13 Connected to implement information interaction between the memory node 11 and the compute node 13.
  • FIG. 1b a flowchart of a remote memory allocation method according to an embodiment of the present invention is applied to the application scenario shown in FIG. 1a. Specifically, the following steps are included:
  • the memory node receives a request sent by a computing node, where the request indicates that the computing node requests the remote memory of the XB from the memory node, where the memory node and the computing node are connected by using a network communication.
  • X is greater than zero and less than 2 n
  • n is the data width of the memory node CPU general purpose register
  • the memory node in the embodiment of the present invention refers to a node having a physical storage space and a network communication function.
  • the memory node is referred to as a memory node in the embodiment of the present invention for convenience of description, and should not be construed as a limitation. Provisions.
  • the computing node described in the embodiment of the present invention refers to a node having a computing capability and a network communication function.
  • the computing node is also referred to as a computing node in the embodiment of the present invention, and should not be construed as a limitation.
  • the request further indicates that the remote memory is a remote physical memory; or the request further indicates that the remote memory is a remote virtual memory; or the request further indicates The remote memory is remote physical memory and remote virtual memory.
  • the request further indicates that the first start address is further indicated, and the first start address refers to an address of the remote memory start location facing the computing node.
  • the memory node allocates the first kernel memory of the XB from the free core memory of the memory node to the computing node.
  • the technical solution described in the embodiment of the present invention is directed to the case where the CPU of the memory node operates in a kernel mode.
  • the memory node allocates the first kernel memory of the XB to the computing node from the free kernel memory of the memory node, and specifically includes: the memory node determining whether the free kernel memory of the memory node is If the free memory is greater than or equal to the remote memory, and the memory node allocates the XB to the computing node from the free core memory.
  • a kernel memory It should be understood that in the case where the free kernel memory is smaller than the remote memory, the following steps are not continued, that is, the memory node does not allocate the remote memory for the computing node.
  • the operating system of the memory node provides sufficient kernel memory for the kernel of the memory node when setting parameters for the kernel.
  • the kernel memory can be configured to 8EB, which provides sufficient kernel memory for allocation to compute nodes.
  • the method also includes:
  • the memory node maps the first kernel memory to a first storage space of the memory node to
  • the computing device allocates the remote memory, and the computing node reads and writes data to the first storage space, where the size of the first storage space is the XB; the first storage space is Physical memory, or the first storage space is virtual memory, or the first storage space is physical memory and virtual memory.
  • the computing node After the memory node allocates the first kernel memory of the XB from the free kernel memory of the memory node to the computing node, the computing node is allocated a logical memory of size XB because Before the real storage space of the first kernel memory map, it is only a piece of logical memory. Since the logical memory cannot be used to read data, the computing node cannot read data into the first kernel memory. Referring to step S105, after the computing node is allocated the first kernel memory, and before the first kernel memory is mapped to the first storage space, the computing node does not have the first kernel memory. Read and write data. The computing node reads and writes data to the first storage space after the first kernel is memory mapped to the first storage space. It should be understood that the "memory node maps the first kernel memory to the first storage space of the memory node" in step S105 is a mapping of a logical address to a physical address based on a memory paging mechanism.
  • the memory node maps the first kernel memory to a first storage space of the memory node to implement Allocating the remote memory to the computing node, so that the computing node reads and writes data to the first storage space, where the size of the first storage space is the XB, specifically: the memory node will The first core memory is mapped to the first physical memory of the memory node, so as to allocate the remote physical memory to the computing node, so that the computing node reads and writes data to the first physical memory.
  • the size of the first physical memory is the XB.
  • the method of the embodiment of the present invention further includes: the memory node determining the memory node Whether the free physical memory is greater than or equal to the remote memory, and if the free physical memory is greater than or equal to the remote memory, the memory node allocates the calculated node from the free physical memory.
  • the first physical memory It should be understood that, in the case that the free physical memory is smaller than the remote memory, the following steps are not continued, that is, the memory node does not allocate the remote memory to the computing node.
  • the request further indicates that the remote memory is a remote physical memory and a remote virtual memory, and the size of the remote physical memory is KB, and the size of the remote virtual memory is XB.
  • the memory node maps the first kernel memory to the first storage space of the memory node to implement allocation of the remote memory to the computing node, thereby causing the computing node to The first storage space reads and writes data, and the size of the first storage space is the XB, specifically: the memory node maps a portion of the first kernel memory with a size of KB to the memory node.
  • the memory node mapping a portion of the first kernel memory having an size of XB-KB to the virtual memory of the memory node, thereby causing the computing node to the physical memory of the KB and the XB- Read and write data in KB's virtual memory.
  • the memory node further determines whether the free physical memory of the memory node is greater than or equal to The KB, and in a case where the free physical memory is greater than or equal to the KB, the memory node maps a portion of the first kernel memory having a size of KB to a physical memory of the memory node.
  • the technical solution provided by the embodiment of the present invention further includes: the memory node establishing between the first start address and the second start address Mapping relationship, the second starting address is an address of the first kernel memory starting position in the free core memory.
  • the address of the computing node A 1 is sent to the memory a node, the memory node determining, according to the mapping relationship, an address A 2 in the free kernel memory of the starting position of the data A, and mapping the A 2 to the storage space A of the memory node 3, to write the data to the A 3 in A.
  • the A 3 may be physical memory or virtual memory, or part of the A 3 is physical memory, and the other part is virtual memory.
  • the memory node allocates the first kernel memory from the free kernel memory of the memory node according to the request of the computing node, and then maps the first kernel memory to the first storage space.
  • the kernel memory is the memory allocated by the operating system for the kernel object, the logical memory, the kernel memory is much larger than the physical memory of the memory node, and based on the virtual memory technology, the kernel memory can not only map.
  • virtual memory that is, virtual memory that is extended from external memory
  • FIG. 1c a flowchart of a remote memory allocation method according to an embodiment of the present invention is applied to the application scenario shown in FIG. 1a. Specifically, the following steps are included:
  • the memory node receives a request sent by the computing node, where the request indicates that the computing node requests the remote memory of the XB from the memory node, where the X is greater than zero and less than 2 n , and the n is the memory.
  • a data width of a node CPU general purpose register wherein the memory node and the computing node are connected by a network communication;
  • the memory node in the embodiment of the present invention refers to a node having a physical storage space and a network communication function.
  • the memory node is referred to as a memory node in the embodiment of the present invention for convenience of description, and should not be construed as a limitation. Provisions.
  • the computing node described in the embodiment of the present invention refers to a node having a computing capability and a network communication function.
  • the computing node is also referred to as a computing node in the embodiment of the present invention, and should not be construed as a limitation.
  • the request further indicates that the remote memory is a remote physical memory; or the request further indicates that the remote memory is a remote virtual memory; or the request further indicates The remote memory is remote physical memory and remote virtual memory.
  • the request further indicates a first start address, where the first start address refers to an address of the remote memory start location facing the computing node.
  • the memory node allocates, according to the request, the first logical memory of the XB from the idle logical memory of the first target process, where the first target process refers to running in the memory node. a process having a first interface that interacts with the computing node and capable of allocating logical memory to the computing node through the first interface, the idle logical memory of the first target process is greater than or equal to the XB;
  • the operating system allocates certain resources (such as logical memory, etc.) to the program, and then performs a series of complicated operations to make the program become a process for the system to call.
  • resources such as logical memory, etc.
  • the relationship between its logical memory and the number of bits of the CPU is usually: the logical memory of the process is 2 r B, where r is the number of bits of the CPU.
  • the logical memory of a process is much larger than physical memory, and a large part of the logical memory of a process is often idle.
  • the first target process described in the embodiment of the present invention is selected from multiple processes running on the memory node, or the first target process is created by the memory node. That is, after the memory node receives the request sent by the computing node, and the memory node allocates the first logical memory of the XB from the idle logical memory of the first target process to the computing node according to the request, The method further includes: the memory node selecting the first target process from all processes running in the memory node; or the memory node creating the first target process and being the first The target process allocates logical memory.
  • the memory node creates the first target process in a case where the first target process does not exist in all processes running in the memory node.
  • the method further includes: after the first target process is created, the number of processes running on the memory node does not exceed The number of cores of the memory node. This is to ensure the speed of the process. It should be known that if the number of processes running on the memory node exceeds the number of cores of the memory node, the process runs slower.
  • the method further includes:
  • the memory node Mapping, by the memory node, the first logical memory to a first storage space of the memory node, to implement allocating the remote memory to the computing node, thereby causing the computing node to move to the first storage space Reading and writing data; wherein the size of the first storage space is the XB, the first storage space is physical memory and virtual memory, or the first storage space is physical memory, or the first The storage space is virtual memory.
  • the computing node is allocated a logical memory of size XB. Because the virtual memory is only a piece of logical memory before the real storage space of the first logical memory map, since the logical memory cannot be used to read data, the computing node cannot read data to the first logical memory. . Referring to step S115, the computing node is assigned the first After a logical memory, and before the first logical memory is mapped to the first storage space, the computing node does not read or write data to the first logical memory.
  • the computing node reads and writes data to the first storage space after the first logical memory is mapped to the first storage space.
  • the “memory node maps the first logical memory to the first storage space of the memory node” in step S115 is a mapping of a logical address to a physical address based on a memory paging mechanism.
  • the memory node maps the first logical memory to a first storage space of the memory node to implement Allocating the remote memory to the computing node, so that the computing node reads and writes data to the first storage space, specifically, the memory node mapping the first logical memory to the memory node
  • the first physical memory is configured to allocate the remote physical memory to the computing node, so that the computing node reads and writes data to the first physical memory, and the size of the first physical memory is the XB.
  • the method of the embodiment of the present invention further includes: the memory node determining the memory node Whether the free physical memory is greater than or equal to the remote memory, and if the free physical memory is greater than or equal to the remote memory, the memory node allocates the calculated node from the free physical memory.
  • the first physical memory It should be understood that, in the case that the free physical memory is smaller than the remote memory, the following steps are not continued, that is, the memory node does not allocate the remote memory to the computing node.
  • the technical solution provided by the embodiment of the present invention further includes: the memory node establishing between the first start address and the second start address a mapping relationship, where the second starting address refers to an address of the first logical memory starting position in the idle logical memory.
  • the computing node to the distal end of the memory read data A the data will be a starting position for the A and the data A
  • the address of the computing node A 1 is sent to the memory a node
  • the memory node determining, according to the mapping relationship, an address A 2 in the free kernel memory of the starting position of the data A, and mapping the A 2 to the storage space A of the memory node 3, to write the data to the A 3 in A.
  • the A 3 may be physical memory or virtual memory, or part of the A 3 is physical memory, and the other part is virtual memory.
  • the memory node is requested according to the computing node. Allocating first logical memory to the computing node from idle logical memory of the first target process, the first target process running in the memory node; mapping the first logical memory to the first of the memory node Storage space to enable allocation of remote memory to the compute node.
  • the logical memory of a process is much larger than the physical memory of a memory node. Based on the virtual memory technology, the logical memory of a process can be mapped not only to physical memory but also to virtual memory (ie, extended from external memory). Virtual memory).
  • the computing node when the computing node applies for the remote memory to the memory node, the computing node can apply for a larger remote memory to the memory node without being limited by the upper limit of the free physical memory in the memory node.
  • FIG. 2a is a schematic structural diagram of a remote memory allocation device 200 according to an embodiment of the present invention.
  • the remote memory allocation device 200 according to the embodiment of the present invention is integrated into the memory node shown in FIG. 1a.
  • the remote memory allocation device 200 of the embodiment of the present invention is an execution body of the method described in the first embodiment, and can be used to perform the method described in the first embodiment.
  • the remote memory allocation device 200 includes:
  • the receiving unit 201 is configured to receive a request sent by the computing node, where the request instructs the computing node to apply for a remote memory of the XB to the memory node, where the X is greater than zero and less than 2 n , and the n is the memory node CPU
  • the request further indicates that the remote memory is a remote physical memory; or the request further indicates that the remote memory is a remote virtual memory; or the request further indicates The remote memory is the remote physical memory and the remote virtual memory.
  • the request further indicates a first start address, where the first start address refers to an address of the remote memory start location facing the computing node.
  • the allocating unit 203 is configured to allocate, by the free kernel memory of the memory node, the first kernel memory of the XB to the computing node;
  • the allocating unit allocates the first kernel memory of the XB from the free kernel memory of the memory node to the computing node, and before the computing node reads and writes data to the remote memory,
  • mapping unit 205 configured to map the first kernel memory to the first storage space of the memory node And allocating the remote memory to the computing node, so that the computing node reads and writes data to the first storage space; wherein the size of the first storage space is the XB,
  • the first storage space is physical memory, or the first storage space is virtual memory, or the first storage space is physical memory and virtual memory.
  • the mapping unit 205 is specifically configured to map the first kernel memory to the first physical memory of the memory node.
  • the remote physical memory is allocated to the computing node, so that the computing node reads and writes data to the first physical memory, and the size of the first physical memory is the XB.
  • the remote memory allocation device 210 further includes an establishing unit 212, wherein the establishing unit 212 is configured to establish a mapping relationship between the first start address and the second start address, where the second start address refers to an address of the first kernel memory start position in the free core memory.
  • the memory node allocates the first kernel memory from the free kernel memory of the memory node according to the request of the computing node, and then maps the first kernel memory to The first storage space is configured to allocate remote memory for the compute node.
  • the kernel memory is the memory allocated by the operating system for the kernel object, the logical memory, the kernel memory is much larger than the physical memory of the memory node, and based on the virtual memory technology, the kernel memory can not only map. To physical memory, you can also map to virtual memory (that is, virtual memory that is extended from external memory).
  • the remote memory allocation device when the computing node applies for the remote memory to the memory node, is not limited by the upper limit of the free physical memory in the memory node, and the computing node can apply for a greater distance to the memory node. Side memory.
  • FIG. 2c is a schematic structural diagram of a remote memory allocation device 220 according to an embodiment of the present invention.
  • the remote memory allocation device 220 according to the embodiment of the present invention is integrated into the memory node shown in FIG. 1a.
  • the remote memory allocation device 220 of the embodiment of the present invention is an execution body of the method described in the second embodiment, and can be used to perform the method described in the second embodiment.
  • the remote memory allocation device 220 includes:
  • the receiving unit 221 is configured to receive a request sent by the computing node, where the request instructs the computing node to apply to the memory node for the remote memory of the XB, where the X is greater than zero and less than 2 n , and the n is the memory a data width of a node CPU general purpose register, wherein the memory node and the computing node are connected by a network communication;
  • the request further indicates that the remote memory is a remote physical memory; or the request further indicates that the remote memory is a remote virtual memory; or the request further indicates The remote memory is the remote physical memory and the remote virtual memory.
  • the request further indicates a first start address, where the first start address refers to an address of the remote memory start location facing the computing node.
  • the allocating unit 223 is configured to allocate, according to the request, the first logical memory of the XB from the idle logical memory of the first target process, where the first target process refers to running in the memory node. a process having a first interface that interacts with the computing node and capable of allocating logical memory to the computing node through the first interface, the idle logical memory of the first target process is greater than or equal to the XB;
  • the first target process described in the embodiment of the present invention is selected from multiple processes running on the memory node, or the first target process is created by the memory node.
  • the remote memory allocation device 230 further includes a selecting unit 232, after the receiving unit 231 receives the request sent by the computing node, and the allocating unit 233 according to the Requesting, before the first logical memory of the first target process allocates the first logical memory of the XB to the computing node, the selecting unit 232 is configured to select the first target from all processes running in the memory node process.
  • the remote memory allocation device 240 further includes a creating unit 242, after the receiving unit 241 receives the request sent by the computing node, and the allocating unit 243 according to the request Before the first logical memory of the first target process allocates the first logical memory of the XB to the computing node, the creating unit 242 is configured to create the first target process and allocate logical memory for the first target process .
  • the creating unit 242 creates the first target process and runs in the The number of processes on the memory node does not exceed the number of cores of the memory node. This is to ensure the speed of the process, it should be known that if the process running on the memory node The number of threads exceeds the number of cores in the memory node, and the process runs slower.
  • the allocating unit allocates the first logical memory of the XB from the idle logical memory of the first target process to the computing node, and before the computing node reads and writes data to the remote memory,
  • the mapping unit 225 is configured to map the first logical memory to the first storage space of the memory node, so as to implement allocating the remote memory to the computing node, so that the computing node is to the first
  • the first storage space is the physical memory and the virtual memory, or the first storage space is physical memory, or the first The storage space is virtual memory.
  • the mapping unit 225 is specifically configured to map the first logical memory to the first physical memory of the memory node, The remote physical memory is allocated to the computing node, so that the computing node reads and writes data to the first physical memory, and the size of the first physical memory is the XB.
  • the remote memory allocation device 250 further includes an establishing unit 252.
  • the establishing unit 252 is configured to use Establishing a mapping relationship between the first start address and a second start address, where the second start address refers to an address of the first logical memory start position in the idle logical memory.
  • the memory node allocates the first logical memory to the computing node from the idle logical memory of the first target process according to the request of the computing node, where the first target process Running in the memory node; mapping the first logical memory to the first storage space of the memory node to implement allocating remote memory to the computing node.
  • the logical memory of a process is much larger than the physical memory of a memory node. Based on the virtual memory technology, the logical memory of a process can be mapped not only to physical memory but also to virtual memory (ie, extended from external memory). Virtual memory).
  • the remote memory allocation device when the computing node applies for the remote memory to the memory node, is not limited by the upper limit of the free physical memory in the memory node, and the computing node can apply for a greater distance to the memory node. Side memory.
  • a computing node 301 a memory node 305, and a remote memory allocation device 303 as described in the third embodiment are included.
  • the computing node 301 is configured to send the request received by the remote memory allocation device 303;
  • the memory node 305 is configured to provide the first storage space
  • the computing node 301 is further configured to acquire the first storage space as the remote memory.
  • the remote memory allocation device 303 is integrated in the memory node 305.
  • the remote memory allocation system according to the embodiment of the present invention is used, and the computing node 301 is not required to apply for the remote memory to the memory node 305.
  • the memory node 305 can limit the upper limit of the free physical memory, and the compute node 301 can apply for a larger remote memory to the memory node 305.
  • a computing node 311, a memory node 315, and a remote memory allocation device 313 as described in the fourth embodiment are included.
  • the computing node 311 is configured to send the request received by the remote memory allocation device 313.
  • the memory node 315 is configured to provide the first storage space
  • the computing node 311 is further configured to acquire the first storage space as the remote memory.
  • the remote memory allocation device 313 is integrated in the memory node 315.
  • the remote memory allocation system according to the embodiment of the present invention is used.
  • the computing node 311 requests the remote memory from the memory node 315, the memory is not affected by the memory.
  • the limit of the upper limit of the free physical memory in the node 315 the compute node 311 can apply for a larger remote memory to the memory node 315.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in the network device provided by each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Memory System Of A Hierarchy Structure (AREA)
  • Memory System (AREA)

Abstract

L'invention concerne un procédé d'attribution de mémoire déportée, ainsi qu'un dispositif et un système correspondants, le procédé comportant les étapes consistant à: faire recevoir, par un nœud (305) de mémoire, une demande émise par un nœud (301) de calcul; faire attribuer, par le nœud (305) de mémoire, une première mémoire de noyau de Xo au nœud (301) de calcul à partir d'une mémoire de noyau inoccupée du nœud (305) de mémoire; après que le nœud (305) de mémoire a attribué la première mémoire de noyau de Xo au nœud (301) de calcul à partir de la mémoire de noyau inoccupée du nœud (305) de mémoire, et le procédé comportant en outre, avant que le nœud (301) de calcul ne lise et n'écrive des données dans la mémoire déportée, les étapes consistant à: faire associer, par le nœud (305) de mémoire, la première mémoire de noyau à un premier espace de stockage du nœud (305) de mémoire, de façon à attribuer la mémoire déportée au nœud (301) de calcul. La solution technique s'affranchit d'une limite supérieure de mémoire physique du nœud (305) de mémoire lors de l'attribution de mémoire déportée au nœud (301) de calcul, et attribue une plus grande mémoire déportée au nœud (301) de calcul.
PCT/CN2016/071483 2015-01-27 2016-01-20 Procédé, dispositif et système d'attribution de mémoire déportée Ceased WO2016119618A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510041110.7A CN105988871B (zh) 2015-01-27 2015-01-27 一种远端内存分配方法、装置和系统
CN201510041110.7 2015-01-27

Publications (1)

Publication Number Publication Date
WO2016119618A1 true WO2016119618A1 (fr) 2016-08-04

Family

ID=56542395

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/071483 Ceased WO2016119618A1 (fr) 2015-01-27 2016-01-20 Procédé, dispositif et système d'attribution de mémoire déportée

Country Status (2)

Country Link
CN (1) CN105988871B (fr)
WO (1) WO2016119618A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107729145A (zh) * 2017-10-11 2018-02-23 广东欧珀移动通信有限公司 应用程序运行控制方法及装置
CN111367461B (zh) * 2018-12-25 2024-02-20 兆易创新科技集团股份有限公司 一种存储空间管理方法及装置
CN112948149A (zh) * 2021-03-29 2021-06-11 江苏为是科技有限公司 一种远端内存共享方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102110196A (zh) * 2009-12-25 2011-06-29 中国长城计算机深圳股份有限公司 并行运行多用户操作系统间的数据安全传输方法及系统
CN103077120A (zh) * 2012-12-31 2013-05-01 东软集团股份有限公司 程序共享内存的地址转换方法和装置
CN103942087A (zh) * 2014-03-31 2014-07-23 华为技术有限公司 虚拟机热迁移方法及相关装置和集群系统
CN104216835A (zh) * 2014-08-25 2014-12-17 杨立群 一种实现内存融合的方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6785888B1 (en) * 1997-08-29 2004-08-31 International Business Machines Corporation Memory allocator for a multiprocessor computer system
CN101594309B (zh) * 2009-06-30 2011-06-08 华为技术有限公司 集群系统中内存资源的管理方法、设备及网络系统
WO2012106908A1 (fr) * 2011-07-20 2012-08-16 华为技术有限公司 Procédé de simulation et simulateur pour l'accès à distance à la mémoire dans un système multiprocesseur
KR101355105B1 (ko) * 2012-01-03 2014-01-23 서울대학교산학협력단 캐시 일관성 보장을 위한 공유 가상 메모리 관리 장치
CN104239227B (zh) * 2013-06-20 2017-06-06 新华三技术有限公司 一种虚拟化存储管理装置、本地虚拟存储管理装置及方法
CN104102460A (zh) * 2014-07-23 2014-10-15 浪潮(北京)电子信息产业有限公司 一种基于云计算的内存管理方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102110196A (zh) * 2009-12-25 2011-06-29 中国长城计算机深圳股份有限公司 并行运行多用户操作系统间的数据安全传输方法及系统
CN103077120A (zh) * 2012-12-31 2013-05-01 东软集团股份有限公司 程序共享内存的地址转换方法和装置
CN103942087A (zh) * 2014-03-31 2014-07-23 华为技术有限公司 虚拟机热迁移方法及相关装置和集群系统
CN104216835A (zh) * 2014-08-25 2014-12-17 杨立群 一种实现内存融合的方法及装置

Also Published As

Publication number Publication date
CN105988871A (zh) 2016-10-05
CN105988871B (zh) 2020-06-02

Similar Documents

Publication Publication Date Title
CN110597451B (zh) 一种虚拟化缓存的实现方法及物理机
US11644994B2 (en) Data migration method, host, and solid state disk
CN111708717B (zh) 数据拷贝方法、直接内存访问控制器及计算机系统
EP3214553B1 (fr) Procédé d'accès à des ressources de stockage prenant en charge la technologie sr-iov, contrôleur de stockage, et dispositif de stockage
US9547535B1 (en) Method and system for providing shared memory access to graphics processing unit processes
CN107250995B (zh) 存储器管理设备
CN104461735B (zh) 一种虚拟化场景下分配cpu资源的方法和装置
US20160085450A1 (en) System for providing remote memory and temporary page pool operating method for providing remote memory
WO2015180598A1 (fr) Procédé, appareil et système de traitement d'informations d'accès d'un dispositif de stockage
CN111124951A (zh) 管理数据访问的方法、设备和计算机程序产品
WO2016019566A1 (fr) Système, dispositif et procédé de gestion de mémoire et réseau sur puce
TW202349214A (zh) 輸入／輸出裝置的位址變換預取出
CN120353729B (zh) 系统内存管理的方法及系统、电子设备和存储介质
CN110383254B (zh) 优化与网络节点相关联的内存映射
CN106155910B (zh) 一种实现内存访问的方法、装置和系统
WO2016127807A1 (fr) Procédé d'écriture de multiples copies dans un dispositif de stockage et dispositif de stockage
JP2016522915A (ja) 共有メモリシステム
WO2022262623A1 (fr) Procédé et appareil d'échange de données
WO2016119618A1 (fr) Procédé, dispositif et système d'attribution de mémoire déportée
JP2013539111A (ja) キャッシングに対応したストレージ装置上における効率的なシーケンシャルロギングのためのシステム及び方法
CN117857253A (zh) 一种基于jailhouse smmu的域间通信方法
CN112395220B (zh) 共享存储控制器的处理方法、装置、系统及存储控制器
CN106201921A (zh) 一种缓存分区容量的调整方法及装置
WO2015161804A1 (fr) Procédé et dispositif de partitionnement de mémoire cache
WO2021120843A1 (fr) Procédé d'attribution de mémoire d'hôte en nuage, hôte en nuage, dispositif et support de stockage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16742680

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16742680

Country of ref document: EP

Kind code of ref document: A1