WO2016066073A1 - 支持sr-iov的存储资源访问方法、存储控制器及存储设备 - Google Patents

支持sr-iov的存储资源访问方法、存储控制器及存储设备 Download PDF

Info

Publication number
WO2016066073A1
WO2016066073A1 PCT/CN2015/092877 CN2015092877W WO2016066073A1 WO 2016066073 A1 WO2016066073 A1 WO 2016066073A1 CN 2015092877 W CN2015092877 W CN 2015092877W WO 2016066073 A1 WO2016066073 A1 WO 2016066073A1
Authority
WO
WIPO (PCT)
Prior art keywords
resource
storage
destination
host
storage sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/092877
Other languages
English (en)
French (fr)
Inventor
胡海燕
沈绍锋
唐苗
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP15855076.4A priority Critical patent/EP3214553B1/en
Priority to BR112017009045-7A priority patent/BR112017009045B1/pt
Publication of WO2016066073A1 publication Critical patent/WO2016066073A1/zh
Priority to US15/581,095 priority patent/US10534552B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/062—Securing storage systems
    • G06F3/0622—Securing storage systems in relation to access
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/16—Handling requests for interconnection or transfer for access to memory bus
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/32—Handling requests for interconnection or transfer for access to input/output bus using combination of interrupt and burst mode transfer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects
    • G06F3/0664—Virtualisation aspects at device level, e.g. emulation of a storage device or system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45579—I/O management, e.g. providing access to device drivers or storage
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45583—Memory management, e.g. access or allocation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7202—Allocation control and policies
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0026—PCI express

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a storage resource access method, a storage controller, and a storage device that support a single root input/output virtualized SR-IOV.
  • SR-IOV Single Root I/O Virtualization
  • PCIe Peripheral Component Interconnect
  • the PCIe Solid State Driver features high bandwidth, high IOPS (Input/Output Operations Per Second), low latency, and high hardware potential.
  • the current technology does not provide an implementation method for supporting storage devices (such as PCIe SSD) to support SR-IOV, so that existing storage devices cannot support SR-IOV, and hardware resource sharing cannot be better realized to improve storage resources in storage devices. Utilization.
  • storage devices such as PCIe SSD
  • the embodiment of the present invention provides a storage resource access method, a storage controller, and a storage device that support the SR-IOV, and is used to solve the problem that the storage device existing in the prior art cannot implement hardware resource sharing to improve storage resources in the storage device.
  • the problem of utilization is used to solve the problem that the storage device existing in the prior art cannot implement hardware resource sharing to improve storage resources in the storage device. The problem of utilization.
  • an embodiment of the present invention provides a controller implementation method for supporting single input/output virtualization, which is performed by a storage controller in a storage device, where the storage device further includes a storage medium, and the storage controller is used by the storage controller. Connecting the host and the storage medium respectively, wherein the host runs a virtual machine monitor and one or more virtual machines, wherein the storage controller includes a physical function PF and one or more virtual functions VF associated with the PF, the PF directly communicating with the virtual machine monitor Each of the VFs is in direct communication with one of the virtual machines, the method comprising:
  • the child resource performs an operation corresponding to the host command.
  • the host includes a memory
  • the memory includes at least one queue
  • at least one of the PF or the VF respectively corresponds to one or more different queues ;
  • the receiving host command sent by the virtual machine to the destination VF or sent by the virtual machine monitor to the destination PF includes:
  • the host learns to put the host command into the queue in the host memory, initiate a direct memory access DMA operation, and the destination VF or the destination PF corresponds to the queue in the host memory. Get the host command in the queue.
  • the storage controller further includes a plurality of interrupt resources, where the multiple interrupt resources are allocated to a queue in the host memory And causing one or more of the queues to correspond to one interrupt resource, the method further includes:
  • the destination queue of the destination VF or the destination PF is: in the host memory The destination VF or the queue corresponding to the destination PF in the queue.
  • the interrupt resource is an MSI-x interrupt, and each of the queues in the host memory corresponds to one MSI-x interrupt.
  • the host command includes a storage identifier for identifying the storage sub-resource a resource relative identifier ID, where the storage sub-resource in the resource allocation table is identified by an absolute identifier of the storage sub-resource;
  • the searching for the resource allocation table according to the destination VF or the destination PF, and corresponding to the destination VF or the destination VF according to the mapping relationship between the storage sub-resource in the resource allocation table and the PF or the VF includes:
  • the correspondence between the absolute IDs finds the storage sub-resource absolute ID corresponding to the storage sub-resource relative ID carried in the host command, and operates on the storage sub-resource corresponding to the found storage sub-resource absolute ID.
  • the storing the sub-resource according to the preset allocation principle At least one of the PF and the VF, and maintaining a resource allocation table includes:
  • the PF is used to perform a global storage controller function
  • the global storage controller function includes: Firmware Download or At least one of the implementation of the Firmware Active command, the implementation of the Format command, or the shutdown function of the full disk reset.
  • an embodiment of the present invention provides a storage controller that supports a single root input/output virtualization, where the storage controller is configured to respectively connect a host and a storage medium, where the host runs a virtual machine monitor and One or more virtual machines, the storage controller including a physical function PF and one or more virtual functions VF associated with the PF, the VF directly communicating with the virtual machine monitor, each of the The VF communicates directly with one of the virtual machines, including:
  • An integration module configured to integrate the storage medium into a unified storage resource, and divide the unified storage resource into multiple storage sub-resources;
  • an allocation module configured to allocate the storage sub-resource divided by the integration module to at least one of the PF and the VF according to a preset allocation principle, and maintain a resource allocation table, where the resource allocation
  • the table includes a mapping relationship between the storage sub-resource and the PF and at least one of the VFs;
  • a receiving module configured to receive a host command sent by the virtual machine to the destination VF or sent by the virtual machine monitor to the destination PF;
  • An operation module configured to search the resource allocation table according to the destination VF or the destination PF, and according to the mapping relationship between the storage sub-resource in the resource allocation table and the PF or the VF, the destination PF or the The storage sub-resource corresponding to the destination VF performs an operation corresponding to the host command received by the receiving module.
  • the host includes a memory
  • the memory includes at least one queue
  • at least one of the PF or the VF respectively corresponds to one or more different queues ;
  • the receiving module is specifically configured to:
  • the host learns to put the host command into the queue in the host memory, initiate a direct memory access DMA operation, and the destination VF or the destination PF corresponds to the queue in the host memory. Get the host command in the queue.
  • the storage controller further includes a plurality of interrupt resources, where the multiple interrupt resources are allocated to a queue in the host memory So that one or more of the queues correspond to one interrupt resource;
  • the storage controller further includes:
  • a transmission module configured to transmit, by the DMA operation, a controller command that needs to be sent to the host to a memory in the host;
  • a notification module configured to notify the host to acquire a controller command by using an interrupt resource corresponding to the destination VF or the destination queue of the destination PF;
  • the destination queue of the destination VF or the destination PF is: the destination VF in the queue in the host memory or the queue corresponding to the destination PF.
  • the interrupt resource is an MSI-x interrupt
  • each of the queues in the host memory corresponds to one MSI-x interrupt.
  • the host command includes a storage identifier for identifying the storage sub-resource a resource relative identifier ID, where the storage sub-resource in the resource allocation table is identified by an absolute identifier of the storage sub-resource;
  • the operation module is specifically configured to:
  • the correspondence between the absolute IDs finds the storage sub-resource absolute ID corresponding to the storage sub-resource relative ID carried in the host command, and operates on the storage sub-resource corresponding to the found storage sub-resource absolute ID.
  • the allocating module is specifically configured to:
  • the PF is used to perform a global storage controller function
  • the global storage controller function includes: implementation of a Firmware Download or Firmware Active command, implementation of a Format command, or a shutdown function of a full disk reset.
  • an embodiment of the present invention provides a storage device, including: a storage controller and a storage medium;
  • the storage controller is connected to the storage medium and is configured to be connected to an external host, so that the host reads and writes data in the storage medium through the storage controller, where the host runs virtual machine monitoring And one or more virtual machines, the storage controller including a physical function PF and one or more virtual functions VF associated with the PF, the PF directly communicating with the virtual machine monitor, each The VF performs direct communication with one of the virtual machines;
  • the storage controller is used to:
  • the child resource performs an operation corresponding to the host command.
  • the host includes a memory
  • the memory includes at least one queue
  • at least one of the PF or the VF respectively corresponds to one or more different queues ;
  • the storage controller is configured to: when receiving the host command sent by the virtual machine to the destination VF or the virtual machine monitor to send to the destination PF, specifically:
  • the storage controller further includes a plurality of interrupt resources, where the multiple interrupt resources are allocated to a queue in the host memory, such that one or more of the The queue corresponds to an interrupt resource, and the storage controller is further configured to:
  • the destination queue of the destination VF or the destination PF is: the destination VF in the queue in the host memory or the queue corresponding to the destination PF.
  • the interrupt resource is an MSI-x interrupt, and each of the queues in the host memory corresponds to one MSI-x interrupt.
  • the host command includes a storage identifier for identifying the storage sub-resource a resource relative identifier ID, where the storage sub-resource in the resource allocation table is identified by an absolute identifier of the storage sub-resource;
  • the storage controller is configured to search the resource allocation table according to the destination VF or the destination PF, and use the mapping relationship between the storage sub-resource in the resource allocation table and the PF or the VF to target the PF or the When the storage sub-resource corresponding to the destination VF performs an operation corresponding to the host command, it is specifically used to:
  • the correspondence between the absolute IDs finds the storage sub-resource absolute ID corresponding to the storage sub-resource relative ID carried in the host command, and operates on the storage sub-resource corresponding to the found storage sub-resource absolute ID.
  • the storage controller is configured to use the storage sub-resource according to a preset allocation principle When at least one of the PF and the VF is allocated and a resource allocation table is maintained, Specifically used for:
  • an embodiment of the present invention provides a storage system, where the storage system includes a host, and the storage device in any one of the possible implementations of the third aspect or the third aspect.
  • the storage medium in the storage device is integrated and allocated, and the storage sub-resource corresponding to the VF or PF receiving the command is operated according to the command sent by the virtual machine or the virtual machine monitor.
  • the implementation of how the storage device implements the SR-IOV support is provided, thereby improving the utilization of the storage resources in the storage device and improving the performance of the storage device.
  • FIG. 1 is a schematic structural diagram of a system of a PCIe SSD according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of an embodiment of a method for accessing a storage resource supporting SR-IOV according to an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart diagram of another embodiment of a method for accessing an SR-IOV-enabled storage resource according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of mapping of NAND Flash resources according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of an embodiment of a storage controller supporting an SR-IOV according to an embodiment of the present disclosure
  • FIG. 6 is a schematic structural diagram of another embodiment of a storage controller supporting an SR-IOV according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of still another embodiment of a storage controller according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of a storage device according to an embodiment of the present invention.
  • the controller implementation method supporting single root input/output virtualization described in the embodiments of the present invention may be performed by a storage controller in a storage device.
  • the storage device may be a solid state hard disk SSD, and the SSD includes a storage controller (SSD controller) and a storage medium (also referred to as “storage resource” or “physical storage resource”, specifically, NAND Flash.
  • the storage controller is connected to the storage medium, and is also connected to the host (can be connected to the host through a high-speed interface such as PCIe), so that the host reads and writes data in the storage medium through the storage controller.
  • the host in the embodiment of the present invention may be a server, or a PC, or another computing unit that needs to connect to the storage device.
  • the host includes a virtual machine monitor VMM, one or more virtual machine VMs (including VM0, VM1, VM2, ... VM n, where n is a natural number).
  • the above storage controller includes a PF and one or more VFs associated with the PF (such as VF0, VF1, VF2, ..., VFn, etc. as shown in the figure, where n is a natural number).
  • the host and the storage controller can communicate based on the NVMe protocol, and the PF directly communicates with the virtual machine monitor, and each VF directly communicates with a virtual machine, for example, the PF can be connected to the virtual machine.
  • the monitor VMM performs direct communication, VF1 can communicate directly with VM1, and VF2 can communicate directly with VM2.
  • the direct communication technology between the above PF and the virtual machine monitor, and between the VF and the VM is a technique well known to those skilled in the art, and will not be specifically described herein.
  • the method for accessing a storage resource supporting SR-IOV described in the embodiment of the present invention includes:
  • the storage device in the embodiment of the present invention may be a PCIe SSD (that is, an SSD with an interface of a PCIe interface), and the storage medium included in the storage device may be a NAND Flash.
  • the storage controller in order to better allocate the storage medium in the PCIe SSD to the PF and each VF, can integrate the storage medium in the PCIe SSD into a unified storage resource, and divide the unified storage resource into multiple storage sub-resources. To allocate the plurality of storage sub-resources to the PF and the respective VFs.
  • the memory controller can map the NAND Flash into a single logical space (ie, SSD logical space) through a Flash Translation Layer (FTL) technology, and use an absolute logical block address (Logical Block Address, LBA), for example: Abs_Start_LBA ⁇ Abs_End_LAB; wherein the single logical space is a unified storage resource obtained by integrating NAND Flash.
  • LBA Logical Block Address
  • the single logical space is divided into different segments, that is, the single logical space is divided into multiple LBA ranges, and each segment (ie, each LBA range)
  • the size of the NS may be the same or different, and is not limited herein.
  • the NS obtained by dividing the single logical space is a storage sub-resource obtained by dividing the unified storage resource.
  • the storage controller can record the absolute LBA range of each NS in the above single logical space (ie, the global logical space), and thus can allocate the above NSs to the PF or VF.
  • the foregoing may be performed according to a preset allocation principle.
  • the storage sub-resource is allocated to the PF or VF and maintains a resource allocation table.
  • the storage controller may allocate the storage sub-resource to the PF and the VF according to the allocation command sent by the virtual machine monitor, where the resource allocation table includes a mapping relationship between the storage sub-resource and the PF or the VF.
  • the resource allocation table further includes identifier information of the PF, and identifier information of each VF associated with the PF.
  • the storage controller can establish the identification information of the PF and assign it to the PF.
  • the mapping relationship of the storage medium is to query the storage medium allocated to the PF by using the identification information of the PF.
  • the storage controller may also establish a mapping relationship between the identification information of the VF and the storage medium allocated to the VF, so as to query the storage medium allocated to the VF by using the identification information of the VF.
  • the storage controller may allocate the storage sub-resource to the PF and the VF according to an allocation command sent by the virtual machine monitor.
  • the “host command” is a command sent by the host, such as an I/O command. Specifically, it can be sent by a virtual machine running in the host or a virtual machine monitor.
  • the host described in this embodiment includes a memory.
  • the host includes at least one queue in the memory, and the PF or VF corresponds to one or more different queues.
  • a virtual machine or a virtual machine monitor sends a host command, it can be sent by using a queue.
  • a virtual machine or a virtual machine monitor sends a host command, which queue is used, the data corresponding to the host command is written to the storage space corresponding to the queue.
  • the virtual machine or the virtual machine monitor in the host can put the host command into the queue corresponding to the destination VF or the destination PF in the host memory, and then notify the storage controller to obtain the host command from the queue of the host memory.
  • the storage controller After the storage controller learns that the host puts the host command in the queue in the host memory, it can initiate a direct memory access (DMA) operation, and obtain the queue corresponding to the destination VF or the destination PF in the queue of the host memory.
  • DMA direct memory access
  • the virtual machine monitor sends a command to the PF in a direct mode.
  • the storage controller receives the host command through the queue corresponding to the destination PF, the storage device obtains the identifier information of the PF.
  • the function module for implementing PCIe communication in the memory controller adopts the SR-IOV pass-through mode, and can know which VF (for example, VF1), that is, the destination VF, is directly connected to the I/O command sent by the virtual machine, and thus can be in the PCIe message.
  • the control field adds the identification information of the VF1 (for example, the ID of the VF1), and sends the packet carrying the identification information of the VF1 to the processing module in the storage controller.
  • the processing module in the storage controller can parse the PCIe packet carrying the VF1 identification information from the PCIe. Obtain the identification information of VF1 in the packet.
  • the resource allocation table may be queried according to the identifier information of the PF or the identifier information of the VF, and the PF is obtained by searching from the resource allocation table.
  • the storage sub-resource of the VF may further perform operations corresponding to the host command on the storage sub-resource corresponding to the PF or the storage sub-resource corresponding to the VF.
  • the storage controller obtains the storage sub-resource corresponding to the PF (specifically, it can be NS), the data in the NS can be transferred to the physical memory address of the host through DMA.
  • the storage controller may generate a controller command in response to the host command, and transfer the controller command to the memory in the host through the DMA, and send the interrupt through the interrupt resource corresponding to the destination PF or the destination queue of the destination VF. Inform the host to get the controller command.
  • the foregoing PF may be used to perform a global storage controller function in the foregoing NVMe protocol.
  • the above-mentioned global storage controller functions include: implementation of Firmware Download or Firmware Active command, implementation of Format command, or shutdown function of full disk reset.
  • the VF is used to implement a lightweight SSD controller function, including a disk IO function and a small portion of the SSD control function, and the above global functions can only be implemented in the PF.
  • the host interface protocol NVMe is used as an example.
  • the commands such as Firmware Download and Firmware Active are global commands. They are only implemented in PF. VF does not support.
  • VF will fail.
  • both PF and VF can support the secure erase of the Namespace, and the corresponding Lhang Format for the LBA Format is only supported by the PF.
  • the VF resets only its own related resources, including the IO queue. After receiving the shutdown command, the PF must notify the entire disk to reset, in addition to resetting its own resources, including the backend NAND Flash. Management table items are saved.
  • the PF in the storage controller can also be used to implement the function of single root I/O virtualization, the query, the allocation of the storage medium in the storage device, and the maintenance of the resource allocation table.
  • the virtual machine monitor in the host can load the PF driver, enable the SR-IOV function, create a management queue, and send a resource query command to the PF through the PF driver management queue. After receiving the query command, the PF can The storage medium, the interrupt resource, and the status of the queue resource included in the storage device are returned to the virtual machine monitor.
  • the virtual machine monitor receives the storage medium interrupt resource returned by the PF and After the status of the queue resource, the allocation command may be sent to the PF, and the PF may parse the allocation command, and divide the unified storage resource into multiple storage sub-resources according to the foregoing allocation command, thereby allocating the storage sub-resource, the queue resource, and the interrupt resource allocation.
  • the PF may parse the allocation command, and divide the unified storage resource into multiple storage sub-resources according to the foregoing allocation command, thereby allocating the storage sub-resource, the queue resource, and the interrupt resource allocation.
  • the storage controller may integrate the storage medium in the storage device into a unified storage resource, divide the unified storage resource into multiple storage sub-resources, and then, the storage sub-resources according to the allocation command sent by the host. Assigned to the PF or VF, saves the mapping relationship between the storage sub-resource and the PF or VF to the resource allocation table.
  • the resource allocation table may be queried according to the identifier of the PF or the VF that receives the command, and the storage sub-resource is operated according to the mapping relationship between the sub-resource and the PF or the VF.
  • the method described in the embodiment of the present invention can support a single input/output virtualization technology to improve the I/O performance of the storage device and the utilization of the storage resources in the storage device.
  • an embodiment of the present invention provides a storage resource access method supporting SR-IOV, including:
  • the controller implementation method for supporting single root input/output virtualization described in the embodiment of the present invention may be performed by a storage controller, where, as shown in FIG. 1 , the structural relationship between the foregoing storage controller and the host and the storage medium may be The foregoing description of the first embodiment of the controller implementation method for supporting a single input/output virtualization provided by the embodiment of the present invention is not described herein again.
  • the specific implementation process of the storage medium in the storage device is integrated into a unified storage resource and the unified storage resource is divided into multiple storage sub-resources in the embodiment of the present invention. Step S101 will not be repeated here.
  • FIG. 4 is a schematic diagram of mapping of NAND Flash resources described in the embodiment of the present invention.
  • the storage controller maps the NAND Flash into a single logical space (SSD logical space) through the FTL technology function, which is denoted as Abs_NS, and represents the range of a single logical space (ie, unified storage resources) with the starting point and the end point of the absolute LBA, for example, :Abs_Start_LBA ⁇ Abs_End_LAB.
  • the storage controller may divide the above single logical space into N according to a preset division rule (N is self
  • N is self
  • the LBA range of different sizes or the LBA range of the same size, the LBA range of different sizes will be exemplified below
  • the segment NS which can be recorded as Abs_NS1, Abs_NS2, Abs_NS3, ..., Abs_NSn, etc.
  • a single logical space may be first divided into N NSs, and the position of each NS in the above single logical space (ie, an absolute logical space address), including a starting position and a ending position, as shown in Table 1:
  • Abs_NS absolute space
  • Start_LBA starting position
  • End_LAB terminating position
  • the storage controller may divide the single logical space into N (for example, 6) sizes according to a preset division rule. Different NS, and record the starting position and ending position of each NS, as shown in Table 2 below:
  • Abs_NS Start_LBA starting position
  • End_LAB terminating position
  • Abs_NS1 0x0000000 0x7ffffff Abs_NS2 0x8000000 0xfffff Abs_NS3 0x1000000 0x17fffff Abs_NS4 0x1800000 0x1ffffff Abs_NS5 0x2000000 0x2fffff Abs_NS6 0x3000000 0x3ffffffffff
  • the storage controller may also associate a queue in the host memory with the foregoing PF or VF, so that the PF or the VF corresponds to at least one NS and at least one queue (that is, at least one of the queues in the host memory), and each The queues correspond to a different interrupt resource.
  • the NS or queue corresponding to the PF or VF may be marked with NS IDs or Queue IDs.
  • S202 Assign the storage sub-resource to the PF and the VF according to a preset allocation principle. At least one of, and maintaining a resource allocation table, wherein the resource allocation table includes a mapping relationship between the storage sub-resource and the PF and at least one of the VFs.
  • the storage controller allocates the storage sub-resource to at least one of the PF or the VF, and the specific implementation process of the maintenance of the resource allocation table can be referred to step S102 in the first embodiment of the present invention, and details are not described herein again.
  • the resource allocation table may further include: a mapping relationship between the identifier information of the PF or the VF and the queue resource and the interrupt resource allocated to the PF or the VF, where the queue resource is specifically PCIe.
  • the identifier information (specifically, the queue number) of the queue in the host memory corresponding to the PF or VF in the SSD.
  • each queue corresponds to a different interrupt resource, so that each PF corresponds to at least one interrupt resource, and each VF corresponds to at least one interrupt resource.
  • the correspondence between the PF or the VF and the queue in the host memory may be specifically represented by the correspondence between the PF or the VF and the queue number of the queue in the host memory, and the corresponding relationship may also be changed according to the specific application scenario.
  • the queue number of the queue in the host memory can be redistributed according to the specific application scenario, and the corresponding PF or VF is changed.
  • the PF in the storage controller can view all storage media and interrupt resources in the PCIe SSD, and resource information such as queues in the host memory corresponding to the PF or the VF.
  • the interrupt resource may be an extended MSI-x interrupt of a Message Signaled Interrupt (MSI), and each queue in the host memory corresponds to an MSI-x interrupt.
  • MSI Message Signaled Interrupt
  • the foregoing interrupt resources may also be an MSI interrupt, and each queue in the host memory corresponds to an MSI interrupt, and the specific manifestation of the foregoing interrupt resources is not limited herein.
  • the storage controller allocates the storage sub-resource and the interrupt resource, and establishes a correspondence between the queue (specifically, the I/O queue) in the host memory and the PF or the VF, and the PF can be used according to the virtual machine.
  • the allocation command sent by the monitor allocates the above storage sub-resource, I/O queue resource (such as the queue number of the I/O queue) or the interrupt resource to the PF and the VF.
  • the foregoing allocation command may include a preset allocation principle, and the PF may allocate an NS to a PF according to the foregoing preset allocation principle, and save the mapping relationship between the NS and the PF in the resource allocation table.
  • the PF may also allocate one or more NSs to one VF according to a preset allocation principle (the NS assigned to the VF is an absolute logical space of the VF), and save the mapping relationship between the NS and the VF to the resource.
  • the allocation table assign an NS to multiple VFs according to the preset allocation principle, that is, Different VFs can share the same NS and save the mapping relationship between the NS and each VF in the resource allocation table.
  • the storage controller can allocate an I/O queue resource and its corresponding interrupt resource in the PCIe SSD to the PF or VF according to the preset allocation principle, and map the I/O queue to the PF or VF.
  • the storage controller may allocate a queue number of one or more queues in the host memory and its corresponding interrupt resource to a VF according to a preset allocation principle, and one or more I/Os in the host memory.
  • the mapping relationship between the queue number of the queue and the VF, or the mapping relationship between the one or more interrupt resources and the VF is saved in the resource allocation table.
  • the resource allocation table further includes identifier information of the PF and identifier information of each VF associated with the PF.
  • the storage controller may establish a mapping relationship between the identifier information of the PF and the queue number or the interrupt resource of the NS, the I/O queue allocated to the PF, and query the NS and I/O queues allocated to the PF by using the identifier information of the PF. Queue number or interrupt resource.
  • the storage controller may also establish a mapping relationship between the identification information of the VF and the queue number or the interrupt resource of the NS, the I/O queue allocated to the PF, and query the NS and the I/O allocated to the VF by using the identifier information of the VF.
  • the queue number or interrupt resource of the queue As shown in Table 3 below:
  • the foregoing destination VF is a VF that directly communicates with the virtual machine
  • the PF is a PF that directly communicates with the virtual machine monitor.
  • the storage controller receives the host command sent by the virtual machine or the virtual machine monitor.
  • step S104 For a specific implementation process of the storage controller to query the resource allocation table according to the identification information of the destination PF or the destination VF, refer to step S104 in the first embodiment of the present invention, and details are not described herein.
  • the virtual machine may first send a query command to the storage controller, and request the storage controller to query the state information of the physical resource of the PCIe SSD.
  • the above query command carries the identification information of the destination PF or the destination VF.
  • the storage controller can query the resource allocation table according to the identification information of the destination PF or the destination VF, and determine the capability entry of the destination PF or the destination VF, including the storage medium, I/O queue resources, and interrupt resources that can be occupied by the destination PF or the destination VF.
  • the virtual machine 1 can query the storage controller for the capability entry of the VF1, as shown in Table 4 below:
  • the capability table of the VF1 may be used. After the item is sent to the VF1, the VF1 can obtain the resource information such as the NS, the I/O queue, and the interrupt, and the resource information such as the NS, the I/O queue, and the interrupt can be recorded to establish the NS and I.
  • the relative ID of the /O queue and interrupt resources may be used.
  • VF1 can establish the relative ID of the NS (ie, storage sub-resource) that it can occupy, the relative ID of the I/O queue, and the relative ID of the interrupt resource, such as: VF1: [NS1, NS2, NS3], [IO1, IO2, IO3], [IRQ1, IRQ2, IRQ3], where [NS1, NS2, NS3] is the relative logical space of VF1.
  • NS storage sub-resource
  • VF2 can request the PF to query its capability table entry, and establish its relative ID of the NS that can be occupied, the relative ID of the I/O queue, and the relative ID of the interrupt resource, such as: VF2: [NS 1, NS2] , [IO1, IO2], [IRQ1, IRQ2].
  • the I/O command may be sent to the storage controller, where the I/O command includes the identifier information of the VF requested by the virtual machine, and the corresponding VF.
  • the NS relative ID (NS1 or NS2 or NS3) corresponding to VF1 the relative ID (IO1 or IO2 or IO3) of the I/O queue corresponding to VF1
  • the storage controller may query the resource allocation table according to the identification information of the VF, and obtain all NSs (Abs_NS2, Abs_NS3, Abs_NS4) corresponding to the VF (for example, VF1) from the resource allocation table, and all I/ O queue resources (Abs_Q2, Abs_Q3, Abs_Q4), and all interrupt resources (Abs_IRQ2, Abs_IRQ3, Abs_IRQ4). Further, based on the correspondence between the storage sub-resource relative ID and the storage sub-resource absolute ID, the NS corresponding to the NS1 (Abs_NS2) is obtained from all the NSs corresponding to the VF1, and the same reason.
  • the storage controller may obtain the I/O queue resource (Abs_Q2) corresponding to the IO1 from all the I/O queue resources corresponding to the VF1 according to the correspondence between the relative ID of the I/O queue and the absolute ID of the I/O queue, according to the interrupt resource.
  • the correspondence between the ID and the absolute ID of the interrupt resource is obtained from the interrupt resources corresponding to the VF1, and the interrupt resource (Abs_IRQ2) corresponding to the IRQ1 is obtained.
  • the data of the NS may be transferred to the physical memory address of the host through DMA.
  • the controller command that responds to the host command can be sent to the host through the I/O queue resource corresponding to the PF.
  • the controller command can be generated, and the controller command is DMA-into the host memory, and the interrupt is sent to the host through the interrupt resource corresponding to the queue, and the host can extract the controller command from the host queue.
  • the storage controller may integrate the storage medium in the PCIe SSD into a unified storage resource, divide the unified resource into multiple storage sub-resources, and then allocate the storage sub-resource to the PF according to the allocation information sent by the host. And VF, save the mapping relationship between the storage sub-resource and the PF or VF to the resource allocation table.
  • the resource allocation table may be queried according to the identification information of the destination PF or the destination VF, according to the mapping relationship between the storage sub-resource and the PF or the VF.
  • the PF and the storage sub-resource corresponding to the destination VF operate.
  • the method described in the embodiment of the present invention can support a single input/output virtualization technology to improve the I/O performance of the storage device and the utilization of the storage resources in the storage device.
  • an embodiment of the present invention provides a storage controller supporting SR-IOV, including:
  • the integration module 10 is configured to integrate the storage medium in the storage device into a unified storage resource, and divide the unified storage resource into multiple storage sub-resources.
  • the allocating module 20 is configured to allocate the storage sub-resource divided by the integration module to at least one of the PF and the VF according to a preset allocation principle, and maintain a resource allocation table, where the resource is configured
  • the allocation table includes a mapping relationship between the storage sub-resource and at least one of the PF and the VF.
  • the receiving module 30 is configured to receive a host command sent by the virtual machine to the destination VF or sent by the virtual machine monitor to the destination PF.
  • the operation module 40 is configured to search the resource allocation table according to the destination VF or the destination PF, and use the mapping relationship between the storage sub-resource in the resource allocation table and the PF or the VF to target the PF or the The storage sub-resource corresponding to the destination VF performs an operation corresponding to the host command received by the receiving module.
  • the storage controller supporting the single input/output virtualization provided by the embodiment of the present invention is the storage controller 200 described in the foregoing embodiment, the storage controller 200 and the host 100,
  • the storage controller 200 described in the foregoing embodiment, the storage controller 200 and the host 100,
  • the connection relationship of the storage medium 300 refer to the foregoing description, and details are not described herein again.
  • the storage device described in the embodiment of the present invention may be a PCIe SSD, and the storage medium included in the storage device may be a NAND Flash.
  • the specific implementation manners of the modules in the storage controller described in the embodiments of the present invention can be referred to the steps S101-S104 in the first embodiment of the present invention, and details are not described herein again.
  • the storage controller may integrate the storage medium in the PCIe SSD into a unified storage resource, divide the unified storage resource into multiple storage sub-resources, and then, multiple storage sub-resources according to the allocation information sent by the host. Assigned to the PF or VF, saves the mapping relationship between the storage sub-resource and the PF or VF to the resource allocation table.
  • the resource allocation table may be queried according to the identifier of the PF or the VF that receives the command, and the storage sub-resource is operated according to the mapping relationship between the sub-resource and the PF or the VF.
  • the method described in the embodiment of the present invention can support a single input/output virtualization technology to improve the I/O performance of the storage device and the utilization of the storage resources in the storage device.
  • an embodiment of the present invention provides a storage controller supporting SR-IOV.
  • the storage controller described in the embodiment of the present invention includes:
  • the integration module 10 is configured to integrate the storage medium in the storage device into a unified storage resource, and divide the unified storage resource into multiple storage sub-resources.
  • the allocating module 50 is configured to allocate the storage sub-resource divided by the integration module to at least one of the PF and the VF according to a preset allocation principle, and maintain a resource allocation table, where the resource is configured
  • the allocation table includes a mapping relationship between the storage sub-resource and at least one of the PF and the VF.
  • the receiving module 60 is configured to receive a host command sent by the virtual machine to the destination VF or sent by the virtual machine monitor to the destination PF.
  • the operation module 80 is configured to search the resource allocation table according to the destination VF or the destination PF, and use the mapping relationship between the storage sub-resource in the resource allocation table and the PF or the VF to target the PF or the The storage sub-resource corresponding to the destination VF performs an operation corresponding to the host command received by the receiving module.
  • the foregoing receiving module 60 may be implemented by the receiving module 30 described in the foregoing embodiment, and may also be specifically used to:
  • the host learns to put the host command into the queue in the host memory, initiate a direct memory access DMA operation, and the destination VF or the destination PF corresponds to the queue in the host memory. Get the host command in the queue.
  • the storage controller provided by the embodiment of the present invention further includes:
  • the transmission module 90 is configured to transmit, by the DMA operation, a controller command that needs to be sent to the host to a memory in the host;
  • the notification module 70 is configured to notify the host to acquire a controller command by using an interrupt resource corresponding to the destination VF or the destination queue of the destination PF.
  • the foregoing operation module 80 may be implemented by using the operation module 40 described in the foregoing embodiment, and may also be specifically used to:
  • the correspondence between the absolute IDs finds the storage sub-resource absolute ID corresponding to the storage sub-resource relative ID carried in the host command, and operates on the storage sub-resource corresponding to the found storage sub-resource absolute ID.
  • the foregoing allocation module 50 may be implemented by the distribution module 20 described in the foregoing embodiment, and may also be specifically used to:
  • the storage controller described in the embodiment of the present invention is specifically the storage controller as described in FIG. 1 , and the structure relationship between the foregoing storage controller and the host and the storage medium may be referred to the support single root provided by the embodiment of the present invention.
  • the first embodiment of the controller implementation method of input/output virtualization is described in the foregoing Rong, no longer repeat here.
  • each module in the storage controller described in the embodiment of the present invention may perform the specific implementation manner of each module described in the foregoing Embodiment 3. Further, the implementation described in the foregoing Embodiment 2 may also be performed. For the specific implementation, refer to steps S201-S205 in the foregoing Embodiment 2, and details are not described herein again.
  • the storage controller may integrate the storage medium in the PCIe SSD into a unified storage resource, divide the unified resource into multiple storage sub-resources, and then allocate the storage sub-resource to the PF according to the allocation information sent by the host. And VF, save the mapping relationship between the storage sub-resource and the PF or VF to the resource allocation table.
  • the resource allocation table may be queried according to the identification information of the destination PF or the destination VF, according to the mapping relationship between the storage sub-resource and the PF or the VF.
  • the PF and the storage sub-resource corresponding to the destination VF operate.
  • the storage controller described in the embodiments of the present invention can support a single input/output virtualization technology to improve the I/O performance of the storage device and the utilization of storage resources in the storage device.
  • an embodiment of the present invention provides a memory controller 1000 including a processing chip 1001 and its accessory circuit 1002.
  • the processing chip 1001 may include a Field-Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC).
  • the FPGA or the ASIC may have a built-in storage medium 1003 or may be externally connected to the storage medium 1003.
  • the FPGA chip or the ASIC chip may be programmed to complete the receiving and receiving request of the host, and perform various access and control on the Flash chipset.
  • the above-mentioned storage controller 1000 can be used to implement the method for implementing the SR-IOV-supporting controller provided by the embodiment of the present invention.
  • the specific implementations described in the second embodiment are not described herein again.
  • an embodiment of the present invention provides a storage device 800, including a storage controller 801 and a storage medium 802, where the storage controller 801 is configured to:
  • the resource allocation table is searched according to the destination VF or the destination PF, and the operation corresponding to the host command is performed on the storage sub-resource corresponding to the destination PF or the destination VF according to the mapping relationship between the storage sub-resource and the PF or the VF in the resource allocation table.
  • the host includes a memory
  • the memory includes at least one queue, and at least one of the PF or the VF respectively corresponds to one or more different queues;
  • the storage controller When the storage controller is used to receive a host command sent by the virtual machine to the destination VF or the virtual machine monitor and sent to the destination PF, the storage controller is specifically configured to:
  • the host After the host learns to put the host command into the queue in the host memory, it initiates a direct memory access DMA operation, and obtains a host command from the queue corresponding to the destination VF or the destination PF in the queue in the host memory.
  • the storage controller further includes multiple interrupt resources, and multiple interrupt resources are allocated to the queue in the host memory, so that one or more queues correspond to one interrupt resource, and the storage controller is further used for :
  • the destination queue of the destination VF or the destination PF is the destination VF or the queue corresponding to the destination PF in the queue in the host memory.
  • the interrupt resource is an MSI-x interrupt
  • each queue in the host memory corresponds to one MSI-x interrupt
  • the host command includes a storage sub-resource relative identifier ID for identifying the storage sub-resource, and the storage sub-resource in the resource allocation table is identified by the storage sub-resource absolute ID;
  • the storage controller is configured to search the resource allocation table according to the destination VF or the destination PF, according to the resource allocation.
  • the mapping between the storage sub-resource and the PF or the VF in the table is performed on the storage sub-resource corresponding to the destination PF or the destination VF, the operation is specifically performed on:
  • the storage sub-resource corresponding to the destination VF or the destination PF is found according to the mapping relationship between the storage sub-resource and the PF or the VF in the resource allocation table, and is found and carried according to the correspondence between the storage sub-resource relative ID and the storage sub-resource absolute ID.
  • the storage sub-resource absolute ID corresponding to the storage sub-resource relative ID in the host command operates on the storage sub-resource corresponding to the found storage sub-resource absolute ID.
  • the storage controller when the storage controller allocates the storage sub-resource to at least one of the PF and the VF according to a preset allocation principle, and maintains a resource allocation table, the storage controller is specifically configured to:
  • a storage sub-resource is allocated to a plurality of VFs according to a preset allocation principle, and a mapping relationship between the storage sub-resources and the respective VFs is saved to the resource allocation table.
  • the embodiment of the present invention provides a storage system, including a host, and a storage device provided in Embodiment 6.
  • the structure of the device may be referred to FIG. 1 and FIG. 8. A detailed description is given in the embodiment, and details are not described herein.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

本发明实施例公开了一种支持SR-IOV的存储资源访问方法，包括：将存储介质整合为统一存储资源，并将统一存储资源划分为多个存储子资源；根据预设分配原则将存储子资源分配给PF和VF中的至少一个，并维护一个资源分配表，其中，资源分配表包括存储子资源与PF以及VF中的至少一个的映射关系；接收虚拟机发送给目的VF或者虚拟机监控器发送给目的PF的主机命令；根据目的VF或目的PF查找资源分配表，根据资源分配表中的存储子资源与PF或者VF的映射关系对目的PF或者目的VF对应的存储子资源进行与主机命令对应的操作。

Description

支持SR-IOV的存储资源访问方法、存储控制器及存储设备
本申请要求于2014年10月31日提交中国专利局、申请号为201410607052.5，发明名称为“支持SR-IOV的存储资源访问方法、存储控制器及存储设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及通信技术领域，尤其涉及一种支持单根输入/输出虚拟化SR-IOV的存储资源访问方法、存储控制器及存储设备。
背景技术
单根输入/输出(Input/Output，I/O)虚拟化(Single Root I/O Virtualization，SR-IOV)是对快速外设互连总线接口(Peripheral Component Interconnect，PCIe)规范的延伸，它促使各类系统镜像或用户随机存取物理I/O资源上的子集，从而获得更好的数据移动，达到基础硬件资源的共享。
PCIe固态硬盘(Solid State Driver，SSD)具备高带宽、高IOPS(Input/Output Operations Per Second，每秒输入/输出操作)、低延时的特点，硬件潜力大。
然而，现在技术并没有提供让存储设备(如PCIe SSD)支持SR-IOV的实现方法，从而使得现有存储设备无法支持SR-IOV，无法更好地实现硬件资源共享以提升存储设备中存储资源的利用率。
发明内容
本发明实施例提供一种支持SR-IOV的存储资源访问方法、存储控制器及存储设备，用于解决现有技术存在着的存储设备无法更好地实现硬件资源共享以提升存储设备中存储资源的利用率的问题。
第一方面，本发明实施例提供了一种支持单根输入/输出虚拟化的控制器实现方法，由存储设备中的存储控制器执行，所述存储设备还包括存储介质，所述存储控制器用于分别连接主机以及所述存储介质，其中，所述主机运行有 虚拟机监控器以及一个或多个虚拟机，所述存储控制器中包括物理功能PF以及与所述PF关联的一个或多个虚拟功能VF，所述PF与所述虚拟机监控器进行直接通信，每个所述VF与一个所述虚拟机进行直接通信，所述方法包括：
将所述存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源；
根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系；
接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令；
根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作。
在第一方面的第一种可能的实现方式中，所述主机包括内存，所述内存包括至少一个队列，所述PF或者所述VF中的至少一个分别对应一个或多个不同的所述队列；
所述接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令包括：
当获知所述主机将所述主机命令放到所述主机内存中的所述队列后，发起直接内存存取DMA操作，从所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列中获取所述主机命令。
结合第一方面第一种可能的实现方式，在第二种可能的实现方式中，所述存储控制器还包括多个中断资源，所述多个中断资源被分配给所述主机内存中的队列，使得一个或多个所述队列对应一个中断资源，所述方法还包括：
通过所述DMA操作将需要发送给所述主机的控制器命令传输到主机中的内存；
通过与所述目的VF或者所述目的PF的目的队列对应的中断资源通知所述主机去获取控制器命令；
其中，所述目的VF或者所述目的PF的目的队列为：所述主机内存中的 队列中所述目的VF或者所述目的PF对应的队列。
结合第一方面第二种可能的实现方式，在第三种可能的实现方式中：
所述中断资源为MSI-x中断，所述主机内存中每个所述队列对应一个MSI-x中断。
结合第一方面，或者第一方面第一至第三种任意一种可能的实现方式，在第四种可能的实现方式中，所述主机命令中包括用于标识所述存储子资源的存储子资源相对标识ID，所述资源分配表中的存储子资源用存储子资源绝对ID来标识；
所述根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的VF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作包括：
根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系找到与所述目的VF或所述目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
结合第一方面，或者第一方面第一至第四种任意一种可能的实现方式，在第五种可能的实现方式中，所述根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表包括：
根据预设分配原则将至少一个所述存储子资源分配给一个所述PF，并将所述存储子资源与所述PF的映射关系保存至所述资源分配表；或者
根据预设分配原则将至少一个所述存储子资源分配给一个所述VF，并将所述至少一个所述存储子资源与所述VF的映射关系保存至所述资源分配表；或者
根据预设分配原则将一个所述存储子资源分配给多个所述VF，并将所述存储子资源与各个所述VF的映射关系保存至所述资源分配表。
结合第一方面，或者第一方面第一至第五种任意一种可能的实现方式，在第六种可能的实现方式中，所述PF用于执行全局性的存储控制器功能；
其中，所述全局性的存储控制器功能包括：Firmware Download或者 Firmware Active命令的实现、Format命令的实现，或者全盘复位的关机功能中的至少一种。
第二方面，本发明实施例提供了一种支持单根输入/输出虚拟化的存储控制器，所述存储控制器用于分别连接主机以及存储介质，其中，所述主机运行有虚拟机监控器以及一个或多个虚拟机，所述存储控制器中包括物理功能PF以及与所述PF关联的一个或者多个虚拟功能VF，所述VF与所述虚拟机监控器进行直接通信，每个所述VF与一个所述虚拟机进行直接通信，包括：
整合模块，用于将所述存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源；
分配模块，用于根据预设分配原则将所述整合模块所划分的所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系；
接收模块，用于接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令；
操作模块，用于根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述接收模块接收到的所述主机命令对应的操作。
在第二方面的第一种可能的实现方式中，所述主机包括内存，所述内存包括至少一个队列，所述PF或者所述VF中的至少一个分别对应一个或多个不同的所述队列；
所述接收模块具体用于：
在获知所述主机将所述主机命令放到所述主机内存中的所述队列后，发起直接内存存取DMA操作，从所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列中获取所述主机命令。
结合第二方面第一种可能的实现方式，在第二种可能的实现方式中，所述存储控制器还包括多个中断资源，所述多个中断资源被分配给所述主机内存中的队列，使得一个或多个所述队列对应一个中断资源；
所述存储控制器还包括：
传输模块，用于通过所述DMA操作将需要发送给所述主机的控制器命令传输到主机中的内存；
通知模块，用于通过与所述目的VF或者所述目的PF的目的队列对应的中断资源通知所述主机去获取控制器命令；
其中，所述目的VF或者所述目的PF的目的队列为：所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列。
结合第二方面第二种可能的实现方式，在第三种可能的实现方式中，所述中断资源为MSI-x中断，所述主机内存中每个所述队列对应一个MSI-x中断。
结合第二方面，或者第二方面第一至第三种任意一种可能的实现方式，在第四种可能的实现方式中，所述主机命令中包括用于标识所述存储子资源的存储子资源相对标识ID，所述资源分配表中的存储子资源用存储子资源绝对ID来标识；
所述操作模块具体用于：
根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系找到与所述目的VF或所述目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
结合第二方面，或者第二方面第一至第四种任意一种可能的实现方式，在第五种可能的实现方式中，所述分配模块具体用于：
根据预设分配原则将至少一个所述存储子资源分配给一个所述PF，并将所述存储子资源与所述PF的映射关系保存至所述资源分配表；或者
根据预设分配原则将至少一个所述存储子资源分配给一个所述VF，并将所述至少一个所述存储子资源与所述VF的映射关系保存至所述资源分配表；或者
根据预设分配原则将一个所述存储子资源分配给多个所述VF，并将所述存储子资源与各个所述VF的映射关系保存至所述资源分配表。
结合第二方面，或者第二方面第一至第五种任意一种可能的实现方式，在 第六种可能的实现方式中，所述PF用于执行全局性的存储控制器功能；
其中，所述全局性的存储控制器功能包括：Firmware Download或者Firmware Active命令的实现、Format命令的实现，或者全盘复位的关机功能中的至少一种。
第三方面，本发明实施例提供了一种存储设备，包括：存储控制器以及存储介质；
所述存储控制器与所述存储介质相连，并且用于与外部的主机相连，以使所述主机通过所述存储控制器读写所述存储介质中的数据，所述主机运行有虚拟机监控器以及一个或多个虚拟机，所述存储控制器中包括物理功能PF以及与所述PF关联的一个或多个虚拟功能VF，所述PF与所述虚拟机监控器进行直接通信，每个所述VF与一个所述虚拟机进行直接通信；
所述存储控制器用于：
将所述存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源；
根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系；
接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令；
根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作。
在第三方面的第一种可能的实现方式中，所述主机包括内存，所述内存包括至少一个队列，所述PF或者所述VF中的至少一个分别对应一个或多个不同的所述队列；
所述存储控制器用于所述接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令时，具体用于：
当获知所述主机将所述主机命令放到所述主机内存中的所述队列后，发起直接内存存取DMA操作，从所述主机内存中的队列中所述目的VF或者所述 目的PF对应的队列中获取所述主机命令。
在第三方面的第一种可能的实现方式中，所述存储控制器还包括多个中断资源，所述多个中断资源被分配给所述主机内存中的队列，使得一个或多个所述队列对应一个中断资源，所述存储控制器还用于：
通过所述DMA操作将需要发送给所述主机的控制器命令传输到主机中的内存；
通过与所述目的VF或者所述目的PF的目的队列对应的中断资源通知所述主机去获取控制器命令；
其中，所述目的VF或者所述目的PF的目的队列为：所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列。
结合第三方面第二种可能的实现方式，在第三种可能的实现方式中，
所述中断资源为MSI-x中断，所述主机内存中每个所述队列对应一个MSI-x中断。
结合第三方面，或者第三方面第一至第三种任意一种可能的实现方式，在第四种可能的实现方式中，所述主机命令中包括用于标识所述存储子资源的存储子资源相对标识ID，所述资源分配表中的存储子资源用存储子资源绝对ID来标识；
所述存储控制器用于根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作时，具体用于：
根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系找到与所述目的VF或所述目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
结合第三方面，或者第三方面第一至第四种任意一种可能的实现方式，在第五种可能的实现方式中，所述存储控制器用于根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表时， 具体用于：
根据预设分配原则将至少一个所述存储子资源分配给一个所述PF，并将所述存储子资源与所述PF的映射关系保存至所述资源分配表；或者
根据预设分配原则将至少一个所述存储子资源分配给一个所述VF，并将所述至少一个所述存储子资源与所述VF的映射关系保存至所述资源分配表；或者
根据预设分配原则将一个所述存储子资源分配给多个所述VF，并将所述存储子资源与各个所述VF的映射关系保存至所述资源分配表
第四方面，本发明实施例提供了一种存储系统，所述存储系统包括主机以及第三方面或者第三方面任一种可能的实现方式中的存储设备。
在本发明实施例中，本发明实施例通过对存储设备中的存储介质进行整合、分配，并根据虚拟机或者虚拟机监控器发送的命令对接收命令的VF或者PF对应的存储子资源进行操作，给出了存储设备如何具体实现对SR-IOV支持的实现方式，从而可以提高存储设备中存储资源的利用率以及提高存储设备的性能。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的PCIe SSD的系统结构示意图；
图2是本发明实施例提供的支持SR-IOV的存储资源访问方法的一实施例流程示意图；
图3是本发明实施例提供的支持SR-IOV的存储资源访问方法的另一实施例流程示意图；
图4是本发明实施例提供的NAND Flash资源的映射示意图；
图5是本发明实施例提供的支持SR-IOV的存储控制器的一实施例结构示意图；
图6是本发明实施例提供的支持SR-IOV的存储控制器的另一实施例结构示意图；
图7是本发明实施例提供的存储控制器的又一实施例结构示意图；
图8是本发明实施例提供的存储设备结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
实施例一
本发明实施例中所描述的支持单根输入/输出虚拟化的控制器实现方法可由存储设备中的存储控制器执行。例如，如图1所示，该存储设备可以是固态硬盘SSD，SSD包括存储控制器(SSD控制器)以及存储介质(也可以称“存储资源”或者“物理存储资源”，具体可以为NAND Flash)，存储控制器与存储介质相连，同时，也与主机相连(可通过PCIe等高速接口与主机相连)，以使主机通过存储控制器来读写存储介质中的数据。
本发明实施例中的主机可以是服务器，或者PC，或者其他需要连接存储设备的计算单元。主机中包括虚拟机监控器VMM、一个或者多个虚拟机VM(包括VM0、VM1、VM 2……VM n，其中，n为自然数)。上述存储控制器中包括PF，以及与PF关联的一个或者多个VF(如图中所示的VF0、VF1、VF2，…，VFn等，其中，n为自然数)。在本发明实施例中，主机与存储控制器之间可以基于NVMe协议进行通信，上述PF与虚拟机监控器进行直接通信，每个VF与一个虚拟机进行直接通信，例如，PF可与虚拟机监控器VMM进行直接通信，VF1可与VM1直接通信，VF2可与VM2直接通信等。上述PF与虚拟机监控器之间、以及VF与VM之间的直接通信技术为本领域技术人员所公知的技术，这里不进行具体描述。
参见图2，本发明实施例中所描述的支持SR-IOV的存储资源访问方法包括：
S101，将存储设备中的存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源。
可选地，本发明实施例中所描述的存储设备具体可为PCIe SSD(即接口为PCIe接口的SSD)，上述存储设备中包括的存储介质具体可为NAND Flash。具体实现中，为了将PCIe SSD中的存储介质更好地分配给PF和各个VF，存储控制器可将PCIe SSD中的存储介质整合为统一存储资源，将统一存储资源划分为多个存储子资源，以将上述多个存储子资源分配给PF和各个VF。
具体的，如图1，存储控制器可通过闪存转换层(Flash Translation Layer，FTL)技术将NAND Flash映射成单一的逻辑空间(即SSD逻辑空间)，用绝对逻辑区块地址(Logical Block Address，LBA)来表示，例如：Abs_Start_LBA～Abs_End_LAB；其中，上述单一的逻辑空间即为将NAND Flash进行整合得到的统一存储资源。将上述NAND Flash映射为单一的逻辑空间之后，再将单一的逻辑空间划分为不同的区段，即，将上述单一的逻辑空间划分为多个LBA范围，每个区段(即每个LBA范围)为一个命名空间Namespace(简称NS)，其中，上述各个NS的大小可相同，也可不同，在此不做限制。其中，上述由单一的逻辑空间划分得到的NS即为有统一存储资源划分得到的存储子资源。存储控制器可记录每个NS在上述单一的逻辑空间(即全局逻辑空间)的绝对LBA范围，进而可将上述各个NS分配给PF或者VF使用。
S102，根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系。
在一些可行的是实施方式中，存储控制器通过PF将PCIe SSD中的存储介质整合成统一存储资源，并将统一存储资源划分为多个存储子资源之后，则可根据预设分配原则将上述存储子资源分配给PF或者VF，并维护一个资源分配表。具体的，存储控制器可根据虚拟机监控器发送的分配命令将上述存储子资源分配给PF和VF，其中，上述资源分配表中包括存储子资源和PF或者VF的映射关系。具体的，上述资源分配表还包括PF的标识信息、与上述PF关联的每个VF的标识信息。存储控制器可建立PF的标识信息与分配给该PF 的存储介质的映射关系，以通过上述PF的标识信息查询分配给该PF的存储介质。存储控制器还可建立VF的标识信息与分配给该VF的存储介质的映射关系，以通过上述VF的标识信息查询分配给该VF的存储介质。其中，存储控制器可根据虚拟机监控器发送的分配命令将上述存储子资源分配给PF和VF。
S103，接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令。
本实施例中，“主机命令”即为主机发送的命令，如I/O命令。具体可以由主机中运行的虚拟机或者虚拟机监控器来发送。
可选地，在具体实现中，本实施例所描述的主机包括内存，在运行时，主机的内存中包括至少一个队列，上述PF或者VF对应一个或者多个不同的队列。虚拟机或者虚拟机监控器发送主机命令时可使用队列来发送，虚拟机或者虚拟机监控器发送主机命令时具体使用哪个队列，则往该队列对应的存储空间写该主机命令对应的数据。本实施例中，主机中的虚拟机或者虚拟机监控器可将主机命令放到主机内存中的目的VF或者目的PF对应的队列，再通知存储控制器从主机内存的队列中获取主机命令。
存储控制器获知主机将主机命令放到主机内存中的队列后，则可发起直接内存存取(Direct Memory Access，DMA)操作，从主机内存的队列中上述目的VF或者目的PF对应的队列中获取上述主机命令。
S104，根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作。
可选地，具体实现中，上述虚拟机监控器给PF发送命令采用的是直通的方式，存储控制器通过上述目的PF对应的队列接收到主机命令时可获得上述PF的标识信息。此外，存储控制器中实现PCIe通信的功能模块采用SR-IOV直通模式，可得知虚拟机发送的I/O命令直通到哪个VF(例如VF1)，即目的VF，进而可在PCIe报文中控制字段添加上VF1的标识信息(例如VF1的ID)，将上述携带VF1的标识信息的报文发送至存储控制器中的处理模块。存储控制器中的处理模块可解析上述携带VF1的标识信息的PCIe报文，从上述PCIe 报文中获取VF1的标识信息。
具体实现中，存储控制器获取得到PF的标识信息或者VF的标识信息之后，则可根据上述PF的标识信息或者VF的标识信息查询资源分配表，从上述资源分配表中查找得到分配给PF或者VF的存储子资源，进而可对PF对应的存储子资源或者VF对应的存储子资源进行与上述主机命令对应的操作。存储控制器获取得到PF对应的存储子资源(具体可为NS)之后，则可将上述NS中的数据通过DMA的方式传输到主机物理内存地址中。数据传输完成之后，存储控制器可生成响应主机命令的控制器命令，通过DMA的方式将控制器命令传输到主机中的内存，并通过上述目的PF或者目的VF的目的队列对应的中断资源发送中断通知主机去获取控制器命令。
可选地，在本发明实施例中，上述PF可以用于执行上述NVMe协议中的全局性的存储控制器功能。其中，上述全局性的存储控制器功能包括：Firmware Download或者Firmware Active命令的实现、Format命令的实现，或者全盘复位的关机功能等。在本发明实施例中，VF用于实现轻量级的SSD控制器功能，包括磁盘IO功能以及少部分SSD控制功能，对于上述全局性的功能，只能在PF中实现。例如，以主机接口协议NVMe为例，类似Firmware Download，Firmware Active等命令，因为是全局性的命令，仅在PF实现，VF不支持，虚拟机如果下发Firmware Download或者Firmware Active，VF将返回失败。对于Format命令，PF和VF均可支持Namespace的secure erase，对应Chang LBA Format则只有PF支持。对于关机策略，VF接收到关机指令之后，仅对自身相关资源进行复位，包括IO队列等；PF接收到关机指令之后，除了复位自身资源之外，还得通知全盘进行复位，包括后端NAND Flash管理表项的保存等。
在本发明实施例中，存储控制器中的PF还可用于实现单根I/O虚拟化的使能，存储设备中的存储介质的查询、分配以及资源分配表的维护等功能。具体的，主机中的虚拟机监控器可加载PF驱动，使能SR-IOV功能，创建管理队列，并通过PF驱动管理队列下发资源查询命令给PF，PF接收到上述查询命令之后，则可将存储设备中包括的存储介质、中断资源以及队列资源的状态返回给虚拟机监控器。虚拟机监控器接收到PF返回的存储介质中断资源以及 队列资源的状态之后，则可向PF发送分配命令，PF可对分配命令进行解析，根据上述分配命令将统一存储资源划分为多个存储子资源，进而将存储子资源、队列资源和中断资源分配给PF或者VF。
在本发明实施例中，存储控制器可将存储设备中的存储介质整合为统一存储资源，将上述统一存储资源划分为多个存储子资源，进而根据主机发送的分配命令将多个存储子资源分配给PF或者VF，将存储子资源与PF或者VF的映射关系保存至资源分配表。当存储控制器接收到虚拟机或者虚拟机监控器发送的命令时，则可根据接收命令的PF或者VF的标识查询资源分配表，根据子资源与PF或者VF的映射关系对存储子资源进行操作。本发明实施例所描述的方法，可支持单根输入/输出虚拟化技术，提高存储设备的I/O性能和存储设备中存储资源的利用率。
实施例二
参见图3，基于上述实施例，本发明实施例提供了一种支持SR-IOV的存储资源访问方法，包括：
S201，将存储设备中的存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源。
可选地，本发明实施例中所描述的支持单根输入/输出虚拟化的控制器实现方法可由存储控制器执行，其中，如图1，上述存储控制器与主机、存储介质的结构关系可参见本发明实施例提供的支持单根输入/输出虚拟化的控制器实现方法的第一实施例前面的描述内容，在此不再赘述。
具体实现中，本发明实施例中所描述的将存储设备中的存储介质整合为统一存储资源，并将统一存储资源划分为多个存储子资源的具体实现过程可参见上述本发明实施例一中的步骤S101，在此不再赘述。
进一步的，如图4，图4是本发明实施例中描述的NAND Flash资源的映射示意图。存储控制器通过FTL技术功能将NAND Flash映射成单一的逻辑空间(SSD逻辑空间)，记为Abs_NS，并以绝对LBA的起点和终点来表示单一的逻辑空间的范围(即统一存储资源)，例如：Abs_Start_LBA～Abs_End_LAB。存储控制器可根据预设划分规则将上述单一的逻辑空间划分为N个(N为自 然数)大小不同的LBA范围(或者大小相同的LBA范围，下面将以大小不同的LBA范围为例进行具体说明)(即区段NS，可记为Abs_NS1、Abs_NS2、Abs_NS3，…，Abs_NSn等)。具体的，可先将单一的逻辑空间划分为N个NS，并记录每个NS在上述单一的逻辑空间中的位置(即绝对逻辑空间地址)，包括起始位置和终止位置，如下表1：
表1
Abs_NS(绝对逻辑空间) Start_LBA(起始位置) End_LAB(终止位置)
Abs_NS1 Abs_Start_LBA1 Abs_End_LAB1
Abs_NS2 Abs_Start_LBA2 Abs_End_LAB2
Abs_NS3 Abs_Start_LBA3 Abs_End_LAB3
…… ...... ……
具体实现中，假设上述单一的逻辑空间的范围为0x0000000-0x3fffffff，即Abs_NS：0x0000000-0x3fffffff，存储控制器可根据预设的划分规则将上述单一的逻辑空间划分为N个(例如6个)大小不同的NS，并记录每个NS的起始位置和终止位置，如下表2：
表2
Abs_NS Start_LBA(起始位置) End_LAB(终止位置)
Abs_NS1 0x0000000 0x7ffffff
Abs_NS2 0x8000000 0xfffffff
Abs_NS3 0x1000000 0x17ffffff
Abs_NS4 0x1800000 0x1fffffff
Abs_NS5 0x2000000 0x2ffffff
Abs_NS6 0x3000000 0x3ffffff
可选地，存储控制器还可将主机内存中的队列与上述PF或者VF建立对应关系，使得PF或者VF至少对应一个NS和至少一个队列(即主机内存中的队列中的至少一个)，每个队列对应一个不同的中断资源。其中，如图1，上述PF或者VF对应的NS或者队列可用NS IDs或者Queue IDs进行标记。
S202，根据预设分配原则将所述存储子资源分配给所述PF和所述VF中 的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系。
具体实现中，存储控制器将存储子资源分配给PF或者VF中的至少一个，并维护一个资源分配表的具体实现过程可参见上述本发明实施例一中的步骤S102，在此不再赘述。
进一步的，在本发明实施例中，上述资源分配表中还可包括：PF或者VF的标识信息与分配给该PF或者VF的队列资源和中断资源的映射关系，其中，上述队列资源具体为PCIe SSD中的PF或者VF对应的主机内存中的队列的标识信息(具体可为队列号)等。在本发明实施例中，每个队列对应一个不同的中断资源，使得每个PF对应至少一个中断资源，每个VF对应至少一个中断资源。具体实现中，上述PF或者VF与主机内存中的队列的对应关系具体可表现为PF或者VF与主机内存中的队列的队列号的对应关系，并且上述对应关系还可根据具体应用场景进行变化，即可根据具体应用场景对主机内存中的队列的队列号进行重新分配，改变其对应的PF或者VF。
可选的，存储控制器中的PF可见PCIe SSD中的所有存储介质和中断资源，以及PF或者VF对应的主机内存中的队列等资源信息。其中，上述中断资源具体可为消息信号中断(Message Signaled Interrupt，MSI)的扩展MSI-x中断，主机内存中每个队列对应一个MSI-x中断。具体实现中，上述中断资源具体还可为MSI中断，主机内存中每个队列对应一个MSI中断，上述中断资源的具体表现形式在此不做限制。
进一步的，可选地，存储控制器将存储子资源和中断资源进行分配，建立主机内存中的队列(具体可为I/O队列)与PF或者VF的对应关系时，可通过PF根据虚拟机监控器发送的分配命令将上述存储子资源、I/O队列资源(例如I/O队列的队列号)或者中断资源分配给PF和VF。其中，上述分配命令中可包括预设分配原则，PF可根据上述预设分配原则将一个NS分配给一个PF，并将上述NS与PF的映射关系保存至资源分配表中。
可选的，PF还可根据预设分配原则将一个或者多个NS分配给一个VF(分配给VF的NS即为VF的绝对的逻辑空间)，并将上述NS与VF的映射关系保存至资源分配表中。或者根据预设分配原则将一个NS分配给多个VF，即 不同的VF可共用相同的NS，并将上述NS与各个VF的映射关系保存至资源分配表中。同理，存储控制器还可根据预设分配原则将PCIe SSD中的一个I/O队列资源及其对应的中断资源分配给PF或者VF，并将上述I/O队列与PF或者VF的映射关系，或者上述中断资源与PF或者VF的映射关系保存至所述资源分配表。此外，存储控制器还可根据预设分配原则将主机内存中的一个或者多个队列的队列号，以及其对应的中断资源分配给一个VF，并将主机内存中的一个或者多个I/O队列的队列号与VF的映射关系，或者一个或者多个中断资源与VF的映射关系保存至资源分配表中。
具体实现中，上述资源分配表中还包括PF的标识信息、与上述PF关联的每个VF的标识信息。存储控制器可建立PF的标识信息与分配给该PF的NS、I/O队列的队列号或者中断资源的映射关系，以通过上述PF的标识信息查询分配给该PF的NS、I/O队列的队列号或者中断资源。存储控制器还可建立VF的标识信息与分配给该PF的NS、I/O队列的队列号或者中断资源的映射关系，以通过上述VF的标识信息查询分配给该VF的NS、I/O队列的队列号或者中断资源。如下表3：
表3
Figure PCTCN2015092877-appb-000001
S203，接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的 PF的主机命令。
可选地，上述目的VF为与虚拟机直接通信的VF，上述PF为与虚拟机监控器直接通信的PF。具体实现中，存储控制器接收虚拟机或者虚拟机监控器发送的主机命令的具体实现过程可参见上述本发明实施例一中的步骤S103，在此不再赘述。
S204，根据所述目的VF或者所述目的PF的标识信息查找所述资源分配表，从所述资源分配表中获取所述目的VF或者所述目的PF对应的存储子资源。
具体实现中，存储控制器根据目的PF或者目的VF的标识信息查询资源分配表的具体实现过程可参见上述本发明实施例一中的步骤S104，在此不再赘述。
S205，根据存储子资源相对ID与存储子资源绝对ID之间的对应关系，从所述目的VF或者所述目的PF对应的存储子资源中，找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
具体实现中，虚拟机向存储控制器请求物理资源之前，虚拟机可先向存储控制器发送查询命令，向存储控制器请求查询PCIe SSD的物理资源的状态信息。上述查询命令中携带目的PF或者目的VF的标识信息。存储控制器可根据目的PF或者目的VF的标识信息查询资源分配表，确定目的PF或者目的VF的能力表项，包括目的PF或者目的VF可占用的存储介质、I/O队列资源和中断资源等。例如，虚拟机1可向存储控制器查询VF1的能力表项，如下表4：
表4
资源 数量
NS 3
I/O队列 3
中断 3
虚拟机1获取得到上述VF1的能力表项之后，则可将上述VF1的能力表 项发送给VF1，VF1获知自身可占用的NS、I/O队列和中断等资源信息之后，则可将上述NS、I/O队列和中断等资源信息进行记录，建立用于标识上述NS、I/O队列和中断资源的相对ID。例如，VF1可建立其可占用的NS(即存储子资源)的相对ID、I/O队列的相对ID和中断资源的相对ID等，如：VF1：[NS1、NS2、NS3]，[IO1、IO2、IO3]，[IRQ1、IRQ2、IRQ3]，其中，[NS1、NS2、NS3]则为VF1的相对的逻辑空间。同理，VF2可向PF请求查询其能力表项，并建立其可占用的NS的相对ID、I/O队列的相对ID和中断资源的相对ID等，如：VF2：[NS 1、NS2]，[IO1、IO2]，[IRQ1、IRQ2]。
各个虚拟机记录了直接与其通信的VF的能力表项之后，则可向存储控制器发送I/O命令，上述I/O命令中包括虚拟机所请求的VF的标识信息，以及VF对应的多个NS中的NS相对ID，VF对应的多个I/O队列中的I/O队列相对ID，VF对应的多个中断资源中的中断资源相对ID等。例如，VF1对应的NS相对ID(NS1或者NS2或者NS3)，VF1对应的I/O队列的相对ID(IO1或者IO2或者IO3)，VF1对应的中断资源的相对ID(IRQ1或者IRQ2或者IRQ3)等。
存储控制器接收到上述I/O命令之后，可根据VF的标识信息查询资源分配表，从上述资源分配表中获取VF(例如VF1)对应的所有NS(Abs_NS2，Abs_NS3，Abs_NS4)、所有I/O队列资源(Abs_Q2，Abs_Q3，Abs_Q4)，以及所有中断资源(Abs_IRQ2，Abs_IRQ3，Abs_IRQ4)。进而根据存储子资源相对ID与存储子资源绝对ID之间的对应关系，从上述VF1对应的所有NS中获取NS1对应的NS(Abs_NS2)，同理。存储控制器可根据I/O队列相对ID与I/O队列绝对ID的对应关系，从VF1对应的所有I/O队列资源中获取IO1对应的I/O队列资源(Abs_Q2)，根据中断资源相对ID与中断资源绝对ID的对应关系，从VF1对应的所有中断资源中获取IRQ1对应的中断资源(Abs_IRQ2)等。
可选地，存储控制器获取得到PF对应的NS、I/O队列和中断资源之后，则可将NS的数据通过DMA的方式传输到主机物理内存地址中。所有数据DMA到主机物理内存之后，则可通过PF对应的I/O队列资源向主机发送响应主机命令的控制器命令。存储控制器将所有数据DMA到主机物理内存之后， 可生成控制器命令，并将控制器命令DMA到主机内存中，并通过上述队列对应的中断资源发送中断通知主机有命令来了，主机可从主机队列中提取控制器命令。
在本发明实施例中，存储控制器可将PCIe SSD中的存储介质整合为统一存储资源，将上述统一资源划分为多个存储子资源，进而根据主机发送的分配信息将存储子资源分配给PF和VF，将存储子资源与PF或者VF的映射关系保存至资源分配表。当存储控制器接收到虚拟机或者虚拟机监控器发送的I/O命令时，则可根据目的PF或者目的VF的标识信息查询资源分配表，根据存储子资源与PF或者VF的映射关系对目的PF以及目的VF对应的存储子资源进行操作。本发明实施例所描述的方法，可支持单根输入/输出虚拟化技术，提高存储设备的I/O性能和存储设备中存储资源的利用率。
实施例三
参见图5，基于上述各实施例，本发明实施例提供了一种支持SR-IOV的存储控制器，包括：
整合模块10，用于将存储设备中的存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源。
分配模块20，用于根据预设分配原则将所述整合模块所划分的所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系。
接收模块30，用于接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令。
操作模块40，用于根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述接收模块接收到的所述主机命令对应的操作。
具体实现中，本发明实施例提供的支持单根输入/输出虚拟化的存储控制器，即为上述实施例中所描述的存储控制器200，存储控制器200与主机100、 存储介质300的连接关系可参见上述描述内容，在此不再赘述。
可选地，本发明实施例中所描述的存储设备具体可为PCIe SSD，上述存储设备中包括的存储介质具体可为NAND Flash。具体实现中，本发明实施例中所描述的存储控制器中各个模块的具体实现方式可参见上述本发明实施例一中的步骤S101-S104，在此不再赘述。
在本发明实施例中，存储控制器可将PCIe SSD中的存储介质整合为统一存储资源，将上述统一存储资源划分为多个存储子资源，进而根据主机发送的分配信息将多个存储子资源分配给PF或者VF，将存储子资源与PF或者VF的映射关系保存至资源分配表。当存储控制器接收到虚拟机或者虚拟机监控器发送的命令时，则可根据接收命令的PF或者VF的标识查询资源分配表，根据子资源与PF或者VF的映射关系对存储子资源进行操作。本发明实施例所描述的方法，可支持单根输入/输出虚拟化技术，提高存储设备的I/O性能和存储设备中存储资源的利用率。
实施例四
参见图6，基于上述各实施例，本发明实施例提供了一种支持SR-IOV的存储控制器。本发明实施例中所描述的存储控制器，包括：
整合模块10，用于将存储设备中的存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源。
分配模块50，用于根据预设分配原则将所述整合模块所划分的所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系。
接收模块60，用于接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令。
操作模块80，用于根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述接收模块接收到的所述主机命令对应的操作。
可选地，上述接收模块60可执行上述实施例中所描述的接收模块30所执行的实现方式，还可具体用于：
在获知所述主机将所述主机命令放到所述主机内存中的所述队列后，发起直接内存存取DMA操作，从所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列中获取所述主机命令。
可选地，本发明实施例提供的存储控制器还包括：
传输模块90，用于通过所述DMA操作将需要发送给所述主机的控制器命令传输到主机中的内存；
通知模块70，用于通过与所述目的VF或者所述目的PF的目的队列对应的中断资源通知所述主机去获取控制器命令。
可选地，上述操作模块80可执行上述实施例中所描述的操作模块40所执行的实现方式，还可具体用于：
根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系找到与所述目的VF或所述目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
可选地，上述分配模块50可执行上述实施例中所描述的分配模块20所执行的实现方式，还可具体用于：
根据预设分配原则将至少一个所述存储子资源分配给一个所述PF，并将所述存储子资源与所述PF的映射关系保存至所述资源分配表；或者
根据预设分配原则将至少一个所述存储子资源分配给一个所述VF，并将所述至少一个所述存储子资源与所述VF的映射关系保存至所述资源分配表；或者
根据预设分配原则将一个所述存储子资源分配给多个所述VF，并将所述存储子资源与各个所述VF的映射关系保存至所述资源分配表。
可选地，本发明实施例中所描述的存储控制器具体如图1中所描述的存储控制器，上述存储控制器与主机、存储介质的结构关系可参见本发明实施例提供的支持单根输入/输出虚拟化的控制器实现方法的第一实施例前面的描述内 容，在此不再赘述。
具体实现中，本发明实施例中所描述的存储控制器中各个模块可执行上述实施例三中所描述的各个模块的具体实施方式，进一步的，还可执行上述实施例二中所描述的实施方式，其具体实现方式可参见上述实施例二中的步骤S201-S205，在此不再赘述。
在本发明实施例中，存储控制器可将PCIe SSD中的存储介质整合为统一存储资源，将上述统一资源划分为多个存储子资源，进而根据主机发送的分配信息将存储子资源分配给PF和VF，将存储子资源与PF或者VF的映射关系保存至资源分配表。当存储控制器接收到虚拟机或者虚拟机监控器发送的I/O命令时，则可根据目的PF或者目的VF的标识信息查询资源分配表，根据存储子资源与PF或者VF的映射关系对目的PF以及目的VF对应的存储子资源进行操作。本发明实施例所描述的存储控制器可支持单根输入/输出虚拟化技术，提高存储设备的I/O性能和存储设备中存储资源的利用率。
实施例五
参见图7，基于上述各实施例，本发明实施例提供了一种存储控制器1000，包括处理芯片1001及其附属电路1002。其中，上述处理芯片1001具体可包括：现场可编程门阵列(Field－Programmable Gate Array，FPGA)或者专用集成电路(Application Specific Integrated Circuit，ASIC)等。其中，上述FPGA或者ASIC可内置存储介质1003，也可外挂存储介质1003。
具体实现中，可对上述FPGA芯片或者ASIC芯片进行编程来完成主机的读写请求的接收，并对Flash芯片组进行各种访问即控制。具体的，上述存储控制器1000可用于执行本发明实施例提供的支持SR-IOV的控制器实现方法，具体可参见本发明实施例提供的支持SR-IOV的控制器实现方法的第一实施例和第二实施例中所描述的具体实现方式，在此不再赘述。
实施例六
参见图8，基于上述各实施例，本发明实施例提供了一种存储设备800，包括存储控制器801以及存储介质802，其中，存储控制器801用于：
将存储介质整合为统一存储资源，并将统一存储资源划分为多个存储子资源；
根据预设分配原则将存储子资源分配给PF和VF中的至少一个，并维护一个资源分配表，其中，资源分配表包括存储子资源与PF以及VF中的至少一个的映射关系；
接收虚拟机发送给目的VF或者虚拟机监控器发送给目的PF的主机命令；
根据目的VF或目的PF查找资源分配表，根据资源分配表中的存储子资源与PF或者VF的映射关系对目的PF或者目的VF对应的存储子资源进行与主机命令对应的操作。
可选地，本发明实施例中，主机包括内存，内存包括至少一个队列，PF或者VF中的至少一个分别对应一个或多个不同的队列；
存储控制器用于接收虚拟机发送给目的VF或者虚拟机监控器发送给目的PF的主机命令时，具体用于：
当获知主机将主机命令放到主机内存中的队列后，发起直接内存存取DMA操作，从主机内存中的队列中目的VF或者目的PF对应的队列中获取主机命令。
可选地，本发明实施例中，存储控制器还包括多个中断资源，多个中断资源被分配给主机内存中的队列，使得一个或多个队列对应一个中断资源，存储控制器还用于：
通过DMA操作将需要发送给主机的控制器命令传输到主机中的内存；
通过与目的VF或者目的PF的目的队列对应的中断资源通知主机去获取控制器命令；
其中，目的VF或者目的PF的目的队列为：主机内存中的队列中目的VF或者目的PF对应的队列。
可选地，本发明实施例中，中断资源为MSI-x中断，主机内存中每个队列对应一个MSI-x中断。
可选地，本发明实施例中，主机命令中包括用于标识存储子资源的存储子资源相对标识ID，资源分配表中的存储子资源用存储子资源绝对ID来标识；
存储控制器用于根据目的VF或目的PF查找资源分配表，根据资源分配 表中的存储子资源与PF或者VF的映射关系对目的PF或者目的VF对应的存储子资源进行与主机命令对应的操作时，具体用于：
根据资源分配表中的存储子资源与PF或者VF的映射关系找到与目的VF或目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
可选地，本发明实施例中，存储控制器用于根据预设分配原则将存储子资源分配给PF和VF中的至少一个，并维护一个资源分配表时，具体用于：
根据预设分配原则将至少一个存储子资源分配给一个PF，并将存储子资源与PF的映射关系保存至资源分配表；或者
根据预设分配原则将至少一个存储子资源分配给一个VF，并将至少一个存储子资源与VF的映射关系保存至资源分配表；或者
根据预设分配原则将一个存储子资源分配给多个VF，并将存储子资源与各个VF的映射关系保存至资源分配表。
需要说明的是，由于发明实施例基于上述各实施例，因此，本发明实施例上述控制器用于执行的功能具体可以参见前述实施例中的具体描述，这里不再赘述。
实施例七
基于上述各实施例，本发明实施例提供了一种存储系统，包括主机以及实施例六所提供的存储设备，其结构示意图可以参考图1以及图8，其中，主机以及存储设备已在上述各实施例中进行了详细的介绍，这里不再进行赘述。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
以上所揭露的仅为本发明较佳实施例而已，当然不能以此来限定本发明之 权利范围，因此依本发明权利要求所作的等同变化，仍属本发明所涵盖的范围。

Claims (20)

  1. 一种支持SR-IOV的存储资源访问方法，由存储设备中的存储控制器执行，所述存储设备还包括存储介质，所述存储控制器用于分别连接主机以及所述存储介质，其中，所述主机运行有虚拟机监控器以及一个或多个虚拟机，所述存储控制器中包括物理功能PF以及与所述PF关联的一个或多个虚拟功能VF，所述PF与所述虚拟机监控器进行直接通信，每个所述VF与一个所述虚拟机进行直接通信，其特征在于，所述方法包括：
    将所述存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源；
    根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系；
    接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令；
    根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作。
  2. 如权利要求1所述的方法，其特征在于，所述主机包括内存，所述内存包括至少一个队列，所述PF或者所述VF中的至少一个分别对应一个或多个不同的所述队列；
    所述接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令包括：
    当获知所述主机将所述主机命令放到所述主机内存中的所述队列后，发起直接内存存取DMA操作，从所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列中获取所述主机命令。
  3. 如权利要求2所述的方法，其特征在于，所述存储控制器还包括多个 中断资源，所述多个中断资源被分配给所述主机内存中的队列，使得一个或多个所述队列对应一个中断资源，所述方法还包括：
    通过所述DMA操作将需要发送给所述主机的控制器命令传输到主机中的内存；
    通过与所述目的VF或者所述目的PF的目的队列对应的中断资源通知所述主机去获取控制器命令；
    其中，所述目的VF或者所述目的PF的目的队列为：所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列。
  4. 如权利要求3所述的方法，其特征在于：
    所述中断资源为MSI-x中断，所述主机内存中每个所述队列对应一个MSI-x中断。
  5. 如权利要求1-4任一所述的方法，其特征在于：所述主机命令中包括用于标识所述存储子资源的存储子资源相对标识ID，所述资源分配表中的存储子资源用存储子资源绝对ID来标识；
    所述根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的VF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作包括：
    根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系找到与所述目的VF或所述目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
  6. 如权利要求1-5任一所述的方法，其特征在于，所述根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表包括：
    根据预设分配原则将至少一个所述存储子资源分配给一个所述PF，并将 所述存储子资源与所述PF的映射关系保存至所述资源分配表；或者
    根据预设分配原则将至少一个所述存储子资源分配给一个所述VF，并将所述至少一个所述存储子资源与所述VF的映射关系保存至所述资源分配表；或者
    根据预设分配原则将一个所述存储子资源分配给多个所述VF，并将所述存储子资源与各个所述VF的映射关系保存至所述资源分配表。
  7. 如权利要求1-6任一所述的方法，其特征在于：所述PF用于执行全局性的存储控制器功能；
    其中，所述全局性的存储控制器功能包括：Firmware Download或者Firmware Active命令的实现、Format命令的实现，或者全盘复位的关机功能中的至少一种。
  8. 一种支持SR-IOV的存储控制器，所述存储控制器用于分别连接主机以及存储介质，其中，所述主机运行有虚拟机监控器以及一个或多个虚拟机，所述存储控制器中包括物理功能PF以及与所述PF关联的一个或者多个虚拟功能VF，所述VF与所述虚拟机监控器进行直接通信，每个所述VF与一个所述虚拟机进行直接通信，其特征在于，包括：
    整合模块，用于将所述存储介质整合为统一存储资源，并将所述统一存储资源划分为多个存储子资源；
    分配模块，用于根据预设分配原则将所述整合模块所划分的所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系；
    接收模块，用于接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令；
    操作模块，用于根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述接收模块接收到的所述 主机命令对应的操作。
  9. 如权利要求8所述的存储控制器，其特征在于，所述主机包括内存，所述内存包括至少一个队列，所述PF或者所述VF中的至少一个分别对应一个或多个不同的所述队列；
    所述接收模块具体用于：
    在获知所述主机将所述主机命令放到所述主机内存中的所述队列后，发起直接内存存取DMA操作，从所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列中获取所述主机命令。
  10. 如权利要求9所述的方法，其特征在于，所述存储控制器还包括多个中断资源，所述多个中断资源被分配给所述主机内存中的队列，使得一个或多个所述队列对应一个中断资源；
    所述存储控制器还包括：
    传输模块，用于通过所述DMA操作将需要发送给所述主机的控制器命令传输到主机中的内存；
    通知模块，用于通过与所述目的VF或者所述目的PF的目的队列对应的中断资源通知所述主机去获取控制器命令；
    其中，所述目的VF或者所述目的PF的目的队列为：所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列。
  11. 如权利要求10所述的存储控制器，其特征在于，所述中断资源为MSI-x中断，所述主机内存中每个所述队列对应一个MSI-x中断。
  12. 如权利要求8-11任一所述的存储控制器，其特征在于，所述主机命令中包括用于标识所述存储子资源的存储子资源相对标识ID，所述资源分配表中的存储子资源用存储子资源绝对ID来标识；
    所述操作模块具体用于：
    根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系 找到与所述目的VF或所述目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
  13. 如权利要求8-12任一所述的存储控制器，其特征在于，所述分配模块具体用于：
    根据预设分配原则将至少一个所述存储子资源分配给一个所述PF，并将所述存储子资源与所述PF的映射关系保存至所述资源分配表；或者
    根据预设分配原则将至少一个所述存储子资源分配给一个所述VF，并将所述至少一个所述存储子资源与所述VF的映射关系保存至所述资源分配表；或者
    根据预设分配原则将一个所述存储子资源分配给多个所述VF，并将所述存储子资源与各个所述VF的映射关系保存至所述资源分配表。
  14. 如权利要求8-13任一所述的存储控制器，其特征在于，所述PF用于执行全局性的存储控制器功能；
    其中，所述全局性的存储控制器功能包括：Firmware Download或者Firmware Active命令的实现、Format命令的实现，或者全盘复位的关机功能中的至少一种。
  15. 一种存储设备，其特征在于，包括：存储控制器以及存储介质；
    所述存储控制器与所述存储介质相连，并且用于与外部的主机相连，以使所述主机通过所述存储控制器读写所述存储介质中的数据，所述主机运行有虚拟机监控器以及一个或多个虚拟机，所述存储控制器中包括物理功能PF以及与所述PF关联的一个或多个虚拟功能VF，所述PF与所述虚拟机监控器进行直接通信，每个所述VF与一个所述虚拟机进行直接通信；
    所述存储控制器用于：
    将所述存储介质整合为统一存储资源，并将所述统一存储资源划分为多个 存储子资源；
    根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表，其中，所述资源分配表包括存储子资源与所述PF以及所述VF中的至少一个的映射关系；
    接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令；
    根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作。
  16. 如权利要求15所述的存储设备，其特征在于，所述主机包括内存，所述内存包括至少一个队列，所述PF或者所述VF中的至少一个分别对应一个或多个不同的所述队列；
    所述存储控制器用于所述接收所述虚拟机发送给目的VF或者所述虚拟机监控器发送给目的PF的主机命令时，具体用于：
    当获知所述主机将所述主机命令放到所述主机内存中的所述队列后，发起直接内存存取DMA操作，从所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列中获取所述主机命令。
  17. 如权利要求16的存储设备，其特征在于，所述存储控制器还包括多个中断资源，所述多个中断资源被分配给所述主机内存中的队列，使得一个或多个所述队列对应一个中断资源，所述存储控制器还用于：
    通过所述DMA操作将需要发送给所述主机的控制器命令传输到主机中的内存；
    通过与所述目的VF或者所述目的PF的目的队列对应的中断资源通知所述主机去获取控制器命令；
    其中，所述目的VF或者所述目的PF的目的队列为：所述主机内存中的队列中所述目的VF或者所述目的PF对应的队列。
  18. 如权利要求17所存储设备，其特征在于，所述中断资源为MSI-x中断，所述主机内存中每个所述队列对应一个MSI-x中断。
  19. 如权利要求15-18任一所述的存储设备，其特征在于，所述主机命令中包括用于标识所述存储子资源的存储子资源相对标识ID，所述资源分配表中的存储子资源用存储子资源绝对ID来标识；
    所述存储控制器用于根据所述目的VF或目的PF查找所述资源分配表，根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系对所述目的PF或者所述目的VF对应的存储子资源进行与所述主机命令对应的操作时，具体用于：
    根据所述资源分配表中的存储子资源与所述PF或者所述VF的映射关系找到与所述目的VF或所述目的PF对应的存储子资源，并根据存储子资源相对ID与存储子资源绝对ID之间的对应关系找到与携带在所述主机命令中的存储子资源相对ID对应的存储子资源绝对ID，对与找到的存储子资源绝对ID对应的存储子资源进行操作。
  20. 如权利要求15-19任一所述的存储设备，其特征在于，所述存储控制器用于根据预设分配原则将所述存储子资源分配给所述PF和所述VF中的至少一个，并维护一个资源分配表时，具体用于：
    根据预设分配原则将至少一个所述存储子资源分配给一个所述PF，并将所述存储子资源与所述PF的映射关系保存至所述资源分配表；或者
    根据预设分配原则将至少一个所述存储子资源分配给一个所述VF，并将所述至少一个所述存储子资源与所述VF的映射关系保存至所述资源分配表；或者
    根据预设分配原则将一个所述存储子资源分配给多个所述VF，并将所述存储子资源与各个所述VF的映射关系保存至所述资源分配表。
PCT/CN2015/092877 2014-10-31 2015-10-26 支持sr-iov的存储资源访问方法、存储控制器及存储设备 Ceased WO2016066073A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP15855076.4A EP3214553B1 (en) 2014-10-31 2015-10-26 Storage resource access method supporting sr-iov, storage controller and storage device
BR112017009045-7A BR112017009045B1 (pt) 2014-10-31 2015-10-26 Método de acesso de recurso de armazenamento suportado por virtualização de entrada/saída de raiz única sr-iov, controlador de armazenamento suportado por virtualização de entrada/saída de raiz única e dispositivo de armazenamento
US15/581,095 US10534552B2 (en) 2014-10-31 2017-04-28 SR-IOV-supported storage resource access method and storage controller and storage device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410607052.5 2014-10-31
CN201410607052.5A CN104461958B (zh) 2014-10-31 2014-10-31 支持sr-iov的存储资源访问方法、存储控制器及存储设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/581,095 Continuation US10534552B2 (en) 2014-10-31 2017-04-28 SR-IOV-supported storage resource access method and storage controller and storage device

Publications (1)

Publication Number Publication Date
WO2016066073A1 true WO2016066073A1 (zh) 2016-05-06

Family

ID=52908040

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/092877 Ceased WO2016066073A1 (zh) 2014-10-31 2015-10-26 支持sr-iov的存储资源访问方法、存储控制器及存储设备

Country Status (5)

Country Link
US (1) US10534552B2 (zh)
EP (1) EP3214553B1 (zh)
CN (1) CN104461958B (zh)
BR (1) BR112017009045B1 (zh)
WO (1) WO2016066073A1 (zh)

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104461958B (zh) * 2014-10-31 2018-08-21 华为技术有限公司 支持sr-iov的存储资源访问方法、存储控制器及存储设备
US10838852B2 (en) 2015-04-17 2020-11-17 Samsung Electronics Co., Ltd. System and method to extend NVME queues to user space
US11036533B2 (en) 2015-04-17 2021-06-15 Samsung Electronics Co., Ltd. Mechanism to dynamically allocate physical storage device resources in virtualized environments
JP6516860B2 (ja) * 2015-10-26 2019-05-22 株式会社日立製作所 計算機システム、及び、アクセス制御方法
CN106648877B (zh) * 2015-10-28 2020-08-25 阿里巴巴集团控股有限公司 资源申请、释放方法及装置
CN105808167B (zh) * 2016-03-10 2018-12-21 深圳市杉岩数据技术有限公司 一种基于sr-iov的链接克隆的方法、存储设备及系统
CN107515775B (zh) * 2016-06-15 2021-11-19 华为技术有限公司 一种数据传输方法及装置
CN112347012B (zh) * 2016-06-20 2024-07-23 北京忆芯科技有限公司 支持SR-IOV的NVMe控制器及方法
CN107894913B (zh) * 2016-09-30 2022-05-13 超聚变数字技术有限公司 一种计算机系统和存储访问装置
CN108614671B (zh) * 2016-12-12 2023-02-28 北京忆恒创源科技股份有限公司 基于命名空间的键-数据访问方法与固态存储设备
US10318334B2 (en) * 2017-07-07 2019-06-11 Netronome Systems, Inc. Virtio relay
US20190167579A1 (en) * 2017-10-27 2019-06-06 Pulmatrix Operating Company, Inc. Itraconazole dry powders
CN110096355B (zh) * 2018-01-29 2024-04-09 阿里巴巴集团控股有限公司 一种共享资源分配方法、装置和设备
US10592463B2 (en) * 2018-03-05 2020-03-17 Samsung Electronics Co., Ltd. SSD architecture for FPGA based acceleration
KR102755199B1 (ko) * 2018-03-13 2025-01-17 삼성전자주식회사 가상 환경에서의 물리적 스토리지 장치 자원을 동적으로 할당하는 메커니즘
CN108958884B (zh) * 2018-06-22 2022-02-18 郑州云海信息技术有限公司 一种虚拟机管理的方法及相关装置
CN110752937B (zh) * 2018-07-23 2022-04-15 中兴通讯股份有限公司 一种物理功能复用方法、装置和计算机存储介质
US11074013B2 (en) * 2018-08-07 2021-07-27 Marvell Asia Pte, Ltd. Apparatus and methods for providing quality of service over a virtual interface for solid-state storage
US11656775B2 (en) 2018-08-07 2023-05-23 Marvell Asia Pte, Ltd. Virtualizing isolation areas of solid-state storage media
US11010314B2 (en) 2018-10-30 2021-05-18 Marvell Asia Pte. Ltd. Artificial intelligence-enabled management of storage media access
US11194750B2 (en) * 2018-12-12 2021-12-07 Micron Technology, Inc. Memory sub-system with multiple ports having single root virtualization
US11481118B2 (en) 2019-01-11 2022-10-25 Marvell Asia Pte, Ltd. Storage media programming with adaptive write buffer release
CN109902033B (zh) * 2019-02-13 2023-03-14 山东华芯半导体有限公司 应用于NVMe SSD控制器的namespace的LBA分配方法和映射方法
US12271322B2 (en) 2019-06-24 2025-04-08 Samsung Electronics Co., Ltd. Multi-function flexible computational storage device
US11704059B2 (en) 2020-02-07 2023-07-18 Samsung Electronics Co., Ltd. Remote direct attached multiple storage function storage device
CN111651269A (zh) * 2020-05-18 2020-09-11 青岛镕铭半导体有限公司 实现设备虚拟化的方法、装置及计算机可读存储介质
US11323285B1 (en) * 2020-08-28 2022-05-03 Earthsystems Technologies, Inc. Architecture for a multichannel geophysical data acquisition system and method of use
CN113296884B (zh) * 2021-02-26 2022-04-22 阿里巴巴集团控股有限公司 虚拟化方法、装置、电子设备、介质及资源虚拟化系统
US20250291709A1 (en) * 2024-03-12 2025-09-18 Sandisk Technologies Llc Method for supporting increased logical capacity using thin provisioning without increasing dram size
CN120447838B (zh) * 2025-06-27 2025-09-19 苏州元脑智能科技有限公司 存储系统、方法、装置、电子设备、介质及产品

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102801806A (zh) * 2012-08-10 2012-11-28 薛海强 一种云计算系统及云计算资源管理方法
CN104049912A (zh) * 2014-05-23 2014-09-17 上海爱数软件有限公司 基于虚拟化平台的分布式存储管理方法
CN104461958A (zh) * 2014-10-31 2015-03-25 杭州华为数字技术有限公司 支持sr-iov的存储资源访问方法、存储控制器及存储设备

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US1415796A (en) * 1922-05-09 Pbocess of treating lead-zinc sulphide dues
BE794948A (fr) * 1972-02-02 1973-08-02 Raychem Corp Dispositif de connexion pour cables coaxiaux
US9389895B2 (en) 2009-12-17 2016-07-12 Microsoft Technology Licensing, Llc Virtual storage target offload techniques
US8473947B2 (en) * 2010-01-18 2013-06-25 Vmware, Inc. Method for configuring a physical adapter with virtual function (VF) and physical function (PF) for controlling address translation between virtual disks and physical storage regions
US8719817B2 (en) * 2010-03-25 2014-05-06 Vmware, Inc. Virtualization intermediary/virtual machine guest operating system collaborative SCSI path management
US9135044B2 (en) * 2010-10-26 2015-09-15 Avago Technologies General Ip (Singapore) Pte. Ltd. Virtual function boot in multi-root I/O virtualization environments to enable multiple servers to share virtual functions of a storage adapter through a MR-IOV switch
US9135101B2 (en) * 2013-03-01 2015-09-15 Avago Technologies General Ip (Singapore) Pte Ltd Virtual function timeout for single root input/output virtualization controllers
US9003071B2 (en) * 2013-03-13 2015-04-07 Futurewei Technologies, Inc. Namespace access control in NVM express PCIe NVM with SR-IOV
TWI556174B (zh) * 2014-03-05 2016-11-01 威盛電子股份有限公司 虛擬功能分配系統、方法及其管理主機
WO2016026131A1 (zh) * 2014-08-22 2016-02-25 上海交通大学 一种基于 numa 高性能网络缓存资源亲和度的虚拟处理器的调度方法
GB201415796D0 (en) * 2014-09-07 2014-10-22 Technion Res & Dev Foundation Logical-to-physical block mapping inside the disk controller: accessing data objects without operating system intervention

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102801806A (zh) * 2012-08-10 2012-11-28 薛海强 一种云计算系统及云计算资源管理方法
CN104049912A (zh) * 2014-05-23 2014-09-17 上海爱数软件有限公司 基于虚拟化平台的分布式存储管理方法
CN104461958A (zh) * 2014-10-31 2015-03-25 杭州华为数字技术有限公司 支持sr-iov的存储资源访问方法、存储控制器及存储设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3214553A4 *

Also Published As

Publication number Publication date
CN104461958A (zh) 2015-03-25
CN104461958B (zh) 2018-08-21
EP3214553B1 (en) 2019-12-11
US20170235510A1 (en) 2017-08-17
BR112017009045B1 (pt) 2022-12-27
EP3214553A4 (en) 2017-10-25
US10534552B2 (en) 2020-01-14
BR112017009045A2 (zh) 2018-07-03
EP3214553A1 (en) 2017-09-06

Similar Documents

Publication Publication Date Title
WO2016066073A1 (zh) 支持sr-iov的存储资源访问方法、存储控制器及存储设备
US10180843B2 (en) Resource processing method and device for a multi-core operating system
US10423361B2 (en) Virtualized OCSSDs spanning physical OCSSD channels
US9465760B2 (en) Method and apparatus for delivering MSI-X interrupts through non-transparent bridges to computing resources in PCI-express clusters
US9690739B2 (en) Method and apparatus for extending PCIe domain
US20190155548A1 (en) Computer system and storage access apparatus
CN108351829B (zh) 用于输入/输出计算资源控制的系统和方法
CN111880750A (zh) 磁盘读写资源的分配方法、装置、设备及存储介质
WO2017066944A1 (zh) 一种存储设备访问方法、装置和系统
US20140006659A1 (en) Driver-assisted base address register mapping
CN103942087A (zh) 虚拟机热迁移方法及相关装置和集群系统
CN108027642A (zh) 用于隔离输入/输出计算资源的系统和方法
US11016817B2 (en) Multi root I/O virtualization system
WO2015180598A1 (zh) 对存储设备的访问信息处理方法和装置、系统
WO2016119468A1 (zh) 虚拟机内存管理方法、物理主机、pcie设备及其配置方法以及迁移管理设备
AU2015402888B2 (en) Computer device and method for reading/writing data by computer device
US11150928B2 (en) Hypervisor translation bypass
CN106155910B (zh) 一种实现内存访问的方法、装置和系统
WO2023221847A1 (zh) 基于虚拟机设备直通的数据访问方法、设备以及系统
WO2015055117A1 (zh) 一种内存访问的方法、设备和系统
JP2018508869A (ja) 仮想化環境におけるストレージリソース管理
US20250085999A1 (en) Inter-core communication method and apparatus, computer equipment and computer-readable storage medium
WO2016065613A1 (zh) 访问文件的方法、分布式存储系统和网络设备
CN107851062A (zh) 一种主机集群中缓存管理方法及主机
WO2016197947A1 (zh) 一种分页式地址空间管理方法以及控制器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15855076

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2015855076

Country of ref document: EP

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112017009045

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 112017009045

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20170428