WO2014100558A1 - Attribution dynamique d'unités de traitement graphique physiques à des machines virtuelles - Google Patents

Attribution dynamique d'unités de traitement graphique physiques à des machines virtuelles Download PDF

Info

Publication number
WO2014100558A1
WO2014100558A1 PCT/US2013/076854 US2013076854W WO2014100558A1 WO 2014100558 A1 WO2014100558 A1 WO 2014100558A1 US 2013076854 W US2013076854 W US 2013076854W WO 2014100558 A1 WO2014100558 A1 WO 2014100558A1
Authority
WO
WIPO (PCT)
Prior art keywords
gpu
virtual machine
virtual machines
virtual
physical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2013/076854
Other languages
English (en)
Inventor
Salim ABIEZZI
Jose FONSECA
Mark Sheldon
Alan HOURIHANE
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
VMware LLC
Original Assignee
VMware LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/722,998 external-priority patent/US9142004B2/en
Priority claimed from US13/723,035 external-priority patent/US9417929B2/en
Priority claimed from US13/723,021 external-priority patent/US9298490B2/en
Application filed by VMware LLC filed Critical VMware LLC
Publication of WO2014100558A1 publication Critical patent/WO2014100558A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects

Definitions

  • the present disclosure relates to methods, techniques, and systems for virtualizing graphics processing units and, in particular, to methods, techniques, and systems for efficiently allocating graphics processing units dynamically based upon virtual machine workload.
  • a graphics processing unit can be a useful resource to accomplish activities such as accelerating three dimensional (3D) graphics and video processing due to the parallel computing nature of these tasks. More specifically, the highly parallel structure of a GPU makes it more effective than a general-purpose CPU for algorithms that process large blocks of data in parallel such as 3D graphics that involve, for example, transforms, lighting, textures, shading, etc.
  • CPU computer processor unit
  • GPU graphics processing unit
  • computations such as 3D graphics or motion compensation performed for video decoding, that are simple, repetitive, high throughput, and not as latency sensitive can be offloaded to a GPU in order to provide faster computations and leave more room for other tasks to be processed by the CPU.
  • a typical computing system is expected to have a much larger number of CPU cores than GPUs.
  • GPUs come in a variety of forms.
  • a GPU might be integrated with the CPU on the same chip and share system memory or may be made available on a separate card connected to the CPU and memory through, for example, a PCI (Peripheral Controller Interface) bus.
  • PCI Peripheral Controller Interface
  • Figure 1 is a block diagram of an example virtualization server computing system for executing example embodiments of a dynamic GPU allocation system.
  • Figure 2 is an example block diagram of components of an example dynamic GPU allocation system.
  • Figure 3 is an example flow diagram of an overview of operations of an example dynamic GPU allocation system.
  • FIG 4 is a block diagram of an example GPU allocation list (GAL) used with an example dynamic GPU allocation system.
  • GAL GPU allocation list
  • Figure 5 depicts an example allocation of GPUs among an example set of virtual machines.
  • Figures 6A and 6B depict a flow diagram of example logic for managing a GPU allocation list and allocating GPU resources based upon the list.
  • Figure 7 is a flow diagram of example logic for determining a runtime profile of a designated virtual machine.
  • FIG. 8 is a flow diagram of example logic for handling events that cause a GPU benefit factor (GBF) to be set or changed for a designated virtual machine.
  • GPF GPU benefit factor
  • Embodiments described herein provide enhanced computer- and network-based methods, techniques, and systems for dynamically allocating resources of physical graphics processing units among virtual machines.
  • Example embodiments provide a dynamic GPU allocation system (DGAS), which enables efficient allocation of physical GPU resources to one or more virtual machines.
  • DGAS dynamic GPU allocation system
  • the DGAS associates each virtual machine with a GPU benefit factor
  • a GBF which is a measurement of the potential benefit a virtual machine may attain from use of one or more GPUs.
  • the DGAS uses the GBFs assigned to the virtual machines to allocate physical GPU resources as discussed in detail below to a set of those VMs that would seem to gain the greatest benefit. For example, a higher GBF may be assigned to a virtual machine that processes true 3D graphics than a virtual machine that processes mostly 2D graphics or video.
  • Each virtual machine may also be associated with a statically determined (predetermined, designated, pre-assigned, etc.) priority, set for example by an administrator-user or the like. In at least one embodiment, this static priority is used to override the GPU resource allocation that would otherwise occur by the DGAS just taking into account the GBF.
  • the administrator-user can thereby insure that physical GPU resources are allocated to particular users or virtual machines, such as a computer-aided design engineer, mechanical project manager, or architect, who may need such resources to run 3D design applications, for example, as part of his or her job.
  • the DGAS comprises virtualization logic running on one or more server computing systems that computes GPU benefit factors for the virtual machines on a dynamic basis, and combines the computed GBFs with the corresponding static priorities of the virtual machines to determine an initial ranked ordering of virtual machines.
  • a runtime profile of each virtual machine is used to further adjust its GBF and the ranked ordering of the virtual machine adjusted accordingly.
  • the runtime profile provides some measurement of GPU usage effectiveness based upon the workload running on a virtual machine, such as based upon video memory usage, load of the command queue, bus bandwidth usage and/or other measurement. Accordingly, a runtime profile can provide a type of "tie-breaker" when the GBF and priority of two or more virtual machines result in the same ranking.
  • a data structure such as a GPU allocation list (GAL) is used to manage and maintain the rank ordering of the virtual machines for allocation purposes.
  • GAL GPU allocation list
  • the available hardware GPU resources of the one or more server computing systems are then allocated by, for example, by GPU allocation logic, to some subset of these virtual machines in rank order, the subset determined as those virtual machines that will fit on the GPUs, for example based upon matching physical GPU capacity (e.g. , video memory availability) with the requirements of the subset of virtual machines (e.g. , video memory requirements).
  • Hardware GPU resources are then assigned to the determined subset of virtual machines commensurate with the (physical) GPU resource allocations.
  • the allocation of GPU resources can vary over time, resulting in better utilization of GPU resources, and hence a more optimally performing and well balanced system.
  • allocation is described herein based upon a rank ordering of virtual machines, in some embodiments, allocation is based upon a comparison of requirements with some or no regard to ranking (for example, based upon a first-come, first-serve approach).
  • FIG. 1 is a block diagram of an example virtualization server computing system for executing example embodiments of a dynamic GPU allocation system.
  • Virtualization server computing system 100 may be configured on a personal computer, a laptop, or server (host) hardware platform 101 , such as an x86 architecture platform. Note that a general purpose or a special purpose computing system suitably instructed may be used to implement the virtualization server computing system 100.
  • the virtualization server computing system 100 may comprise one or more server computing systems and may span distributed locations.
  • each block shown may represent one or more such blocks as appropriate to a specific embodiment or may be combined with other blocks.
  • host hardware platform 101 may comprise a computer memory 102, one or more central processing units (“CPU”) 103, a frame buffer (“FB”) 104, and one or more network connections, accessible for example via network interface card (“NIC”) 105.
  • the host hardware platform 101 may optionally comprise other components such as one or more displays 109, graphics processing units (“GPU”) 108, input/output (“I/O") devices 1 1 1 (e.g., keyboard, mouse, CRT or LCD display, etc.), or other computer readable media 1 10.
  • Virtualization logic 120 is loaded into memory 102 of host hardware platform 101 and may execute on one or more CPUs 103. Virtualization logic 120 may alternatively be implemented in software, hardware, or firmware, or some combination thereof. Virtualization logic 120, includes one or more virtual machine monitors (VMM) 142a- 142c and VMX processes 151 a- 151c, which can support multiple virtual machines (VM) 141a- 141c that can concurrently be instantiated and executed.
  • VMM virtual machine monitors
  • VMX processes 151 a- 151c which can support multiple virtual machines (VM) 141a- 141c that can concurrently be instantiated and executed.
  • VM virtual machine monitors
  • VMX processes 151 a- 151c which can support multiple virtual machines (VM) 141a- 141c that can concurrently be instantiated and executed.
  • VM virtual machine
  • VM is an abstraction representing the execution space that a guest operating system and applications (the "guest”) may execute within, such as VM 14 la-
  • Each virtual machine 141 a- 141c may include a guest operating system (guest OS), e.g., guest OSes 143a-143c, and one or more corresponding applications, e.g., guest applications 144a-144c, running on each respective guest OSes 143a-143c.
  • guest OS guest operating system
  • applications e.g., guest applications 144a-144c
  • each VM when executing, is made accessible to a different user who is remotely connected from a different client connection.
  • the number of VMs supported by any particular host may vary, for example, based on aspects such as the hardware capabilities, virtualization logic configurations, and desired performance.
  • Other code 161 may also execute on virtualization logic 120.
  • Each VM 141 a- 141c may require virtualization of one or more aspects implemented by the virtualization logic 120 and/or the host hardware platform 101. That is, the virtualization logic 120 may provide emulated hardware and drivers to each VM. For example, through the VMX processes 151 a- 151c and the VMMs 142a- 142c, the virtualization logic 120 may provide one or more of a virtual CPU (“VCPU”), a virtual memory (“VMEM”), virtual device drivers (“VDD”), a virtual file system and virtual disks, virtual network capabilities, and virtual graphics capabilities, such as virtual graphics adaptors drivers and command emulation, and the like.
  • VCPU virtual CPU
  • VMEM virtual memory
  • VDD virtual device drivers
  • Each virtualization environment may function as an equivalent of a standard x86 hardware architecture platform such that any operating system ⁇ e.g., Microsoft Windows®, Linux®, Solaris®86, NetWare, FreeBSD, etc.) may be installed as a guest OS ⁇ e.g., guest OS 143a-143c) to execute applications in an instantiated virtual machine.
  • any operating system ⁇ e.g., Microsoft Windows®, Linux®, Solaris®86, NetWare, FreeBSD, etc.
  • guest OS e.g., guest OS 143a-143c
  • virtualization of other hardware architectures may be supported.
  • the virtualization logic 120 provides virtualized storage support through a distributed VM file system 132, storage stack 131, and device drivers 130 that communicate with the physical data drives 106 and 107.
  • the virtualization logic 120 provides virtualized network support through a virtual switch 133 and network stack 134 to communicate with NIC 105 of the host hardware platform 101. This support may be used to provide TCP/IP connections at the virtualization logic level to connect with other systems, such as to remote user interfaces or send video to client devices.
  • the virtualization logic 120 provides virtualized graphics support through the Super Video Graphics Array (SVGA) or VGA graphics adaptor implementations which use the server graphics API 121 (such as OpenGl, Xserver implementations, etc.) to communicate with graphics drivers 122 that manage and fill frame buffer 104 of the host hardware 101 using graphics commands.
  • server graphics API 121 such as OpenGl, Xserver implementations, etc.
  • Other embodiments may provide virtualized graphics support in other manners using other communications mechanisms.
  • the graphics capabilities of the host hardware platform 101 may be accelerated through the use of one or more GPUs 108.
  • the virtualization execution environments are provided through both a process executing at USER (less privileged mode), referred to as the VMX process (e.g., VMX processes 151 a- 151 c) and the VMM executing in a more privileged state (e.g., VMMs 142a-142c).
  • VMX process e.g., VMX processes 151 a- 151 c
  • VMMs 142a-142c e.g., VMMs 142a-142c.
  • Each VM 141a-141c effectively executes in the process space of its respective VMX process 151 a- 151c (that is its memory is mapped to each respective VMX process).
  • a VMX process may comprise an MKS (mouse, keyboard, screen) thread (e.g., thread 152a) for processing input and output from the respective VM, e.g. , VMs 141 a- 141c.
  • MKS mouse, keyboard, screen
  • a VMX process also includes USER mode graphics level support, such as a virtual SVGA driver 153a.
  • the SVGA driver 153a is used to send graphics commands (through the graphics API 121) to the graphics drivers of the virtualization logic 120. As described herein, these commands may ultimately run on one or more GPUs 108 when the corresponding VM has been allocated one or more GPUs 108.
  • Each VMX process and VMM pair cooperate to provide the effective (and isolated) virtualization execution environment for each VM to run.
  • the virtualization logic 120 receives requests from the virtualized device drivers implemented by the VMMs and VMX processes, translates (or otherwise transfers, forwards, sends, or communicates) these requests to corresponding requests to real device drivers 130 or 122 that communicate with real devices resident in the host hardware platform 101 (such as frame buffer 104, NIC 105, etc.).
  • VMM 142a-142c the virtual components shown as part of virtualization logic 120 that are not included in each VMM 142a-142c (for example, one or more of components 130-134, 121-122, or the like) may be considered in other embodiments to be part of the VMMs 142a- 142c.
  • no VMX process is used and the MKS thread capabilities and virtual graphics adaptor support are integrated instead into the VMMs 142a- 142c or into other parts of the virtualization logic 120.
  • VMMs 142a- 142c may be considered to be separate from or part of the VM 141 a- 141c.
  • Embodiments of a DGAS may be practiced in other virtualized computing environments such as hosted virtual machine systems, where the virtualization logic 120 is implemented on top of an operating system executing on host hardware platform 101 instead of directly on the host hardware.
  • some or all of the components of the virtualization server computing system 100 may be implemented or provided in other manners, such as at least partially in firmware and/or hardware, including, but not limited to one or more application-specific integrated circuits (ASICs), standard integrated circuits, controllers executing appropriate instructions, and including microcontrollers and/or embedded controllers, field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and the like.
  • ASICs application-specific integrated circuits
  • FPGAs field-programmable gate arrays
  • CPLDs complex programmable logic devices
  • Some or all of the components and/or data structures may also be stored as contents (e.g., as executable or other machine-readable software instructions or structured data) on a tangible or non-transitory computer-readable medium (e.g.
  • a hard disk a hard disk
  • memory a hard disk
  • network other computer-readable medium
  • computer readable medium 1 10 to enable the computer-readable medium to execute or otherwise use or provide the contents to perform at least some of the described techniques.
  • FIG. 2 is an example block diagram of components of an example dynamic GPU allocation system.
  • the dynamic GPU allocation system comprises one or more functional components/modules that work together to dynamically allocate physical GPU resources among a set of virtual machines as overviewed above. These components may be implemented in software, hardware, firmware, or a combination.
  • the DGAS comprises server side support that resides on one or more host or server computing systems 201. In other embodiments, there may be some amount of support that resides in different locations to provide up-to-date reporting of runtime profile information such as workload or planned workload factors.
  • the server side support includes GPU allocation logic
  • the 203 for maintaining a ranked list of virtual machines to which physical GPU resources may be allocated (the GAL), for determining which virtual machines should be allocated some or all of the available GPU resources, and for performing the allocation and de-allocation of hardware GPU support to virtual GPUs (or equivalent communication mechanism) of the virtual machines.
  • the GAL a ranked list of virtual machines to which physical GPU resources may be allocated
  • the virtual GPUs or equivalent communication mechanism
  • other communication mechanisms such as sockets, pipes, and the like
  • the term virtual GPU is used for describing an allocation of physical GPU resources.
  • the GPU allocation logic 203 may execute as part of the virtualization logic 202 or may be provided in whole or in part by other aspects of the computing system running on one or more host/servers 201 such as by distributed logic running on each host/server 201 that communicates to allocate physical GPUs across servers (such as to support VMs that are moved or migrated).
  • distributed logic may be configured to allocate GPUs from multiple host/servers to a single VM.
  • the server side support includes a workload advisor/GBF reporting component for each virtual machine, e.g., advisor/GBF reporting components 206a-206b, for reporting data used to manage profile information, and rendering support, e.g., rendering support logic 205a-205b, for assisting VMs (through API, libraries, commands, etc.) to take advantage of the parallelism available with use of GPUs.
  • these components execute as part of the VM Support 204a-204b, for example, as part of a process (e.g., a VMX process in VMware's virtualization environment) that executes on virtualization logic 202, which is hosted by one or more host/server computing systems 201.
  • the components 204a-206a and 204b-206b may execute as part of an MKS (mouse, keyboard, screen handling) thread 152a, which executes as part of VMX processes 151 a- 151c as described with reference to Figure 1.
  • MKS mouse, keyboard, screen handling
  • these components may be implemented in other parts of the virtualization environment such as part of each VMM (virtual machine monitor, e.g., VMMs 142a- 142c) or as other parts of virtualization logic 202.
  • the rendering support 205a and 205b is responsible for receiving the virtual graphics device commands from guest 210a and 210b (e.g., guest applications 212a and 212b executed from the desktop using the guest operating system 211a and 211b, respectively) and carrying them out through the graphics stack (shown in Figure 1 as graphics API 121 and graphics drivers 122) to the graphics hardware associated with the host 201 , such as frame buffer 104.
  • the workload advisor/GBF reporting logic 206a and 206b is responsible for gathering and communicating information from each respective VM to use in computing the GBF for that VM.
  • the logic 206a and 206b also report factors used to determine a VM's runtime profile (describing or measuring the workload being processed on the VM), such as a measure of the video RAM (VRAM) used, the load of the command queue (e.g. , an SVGA command queue or graphics command queue), the ratio of data transfers to all commands (e.g., the amount of DMA accesses to available bandwidth), and/or other measurements.
  • VRAM video RAM
  • the load of the command queue e.g. , an SVGA command queue or graphics command queue
  • the ratio of data transfers to all commands e.g., the amount of DMA accesses to available bandwidth
  • the components of the server side support for the DGAS are implemented using standard programming techniques.
  • a range of programming languages known in the art may be employed for implementing such example embodiments, including using object-oriented, functional, scripting, and declarative languages.
  • the functionality of the different components may be distributed amongst themselves in different ways or distributed in other ways, yet still achieve the functions of a DGAS.
  • FIG. 2 The components of the DGAS shown in Figure 2 cooperate to dynamically allocate physical GPU resources among a set of virtual machines according to their GPU benefit factors and potentially other considerations.
  • Figure 3 is an example flow diagram of an overview of operations of an example dynamic GPU allocation system. The operations described with reference to Figure 3 are described in further detail with reference to other figures that follow.
  • the DGAS initializes the GPU allocation list (GAL) to insert and rank all running VMs according to their GPU benefit factors and, potentially, static priorities set by someone such as an administrator-user.
  • the GAL can be any suitable data structure, for example a linked list, array, database, file, or the like, and may be stored in temporary or persistent storage.
  • the GAL is a list of VMs, an abstraction of which is described with reference to Figure 4.
  • the GPU benefit factor (GBF) of a virtual machine is calculated based upon the workload (or intended workload) of the virtual machine, such as the types of graphics and/or video commands that are being or intended to be processed.
  • different GPU benefit factors may be associated with VMs that perform or have historically performed 3D graphics, 2.5D graphics, video, 2D graphics, or that will not likely benefit from use of a GPU.
  • Other GBFs may be defined for other situations and accommodated accordingly.
  • other non-graphics and non-video commands that may benefit from GPU acceleration such as for highly parallel and/or repetitive algorithms (e.g. , what is termed GPGPU processing or general purpose GPU processing), may also contribute to a GBF of a virtual machine.
  • the VM may be assigned a GBF based upon the kind of guest operating system and any history available about the applications that will be run or are intended to run on the VM. Over time, the GBF of a particular machine will "self correct" as the workload is better assessed, e.g., by determining a runtime profile of the VM.
  • a GBF may designate six different values that can be assigned to a virtual machine based upon different categorizations (e.g., computational profiles, classifications, and the like).
  • VM that is performing three dimensional (3D) graphics is assigned a GBF of "6”
  • a VM that is performing non-graphics, non-video computing that may benefit from GPU use e.g., general purpose GPU computing, also called GPGPU computing
  • GPGPU computing general purpose GPU computing, also called GPGPU computing
  • GBF of "5 e.g., general purpose GPU computing, also called GPGPU computing
  • a GBF that is performing two and a half dimensional (2.5D) graphics is assigned a GBF of "4”
  • a VM that is encoding/decoding video is assigned a GBF of "3”
  • a VM that is performing two dimensional (2D) graphics is assigned a GBF of "2”
  • a VM that is not performing the type of processing that would generally gain from GPU usage is
  • VMs performing 3D with a GBF of "6" are typically VMs that perform true 3D rendering such as MCAD (Mechanical Computer Aided Design) applications, Google Earth rendered in 3D mode, 3D games (such as Chess Titans), and the like. These VMs typically involve large numbers of 3D triangles that require lighting, texturing, and Z- buffering. Use of a GPU in such instances should make a material difference in improving the performance of these types of applications and their presentation quality.
  • MCAD Mechanismical Computer Aided Design
  • Google Earth rendered in 3D mode such as Chess Titans
  • 3D games such as Chess Titans
  • VMs performing (non-graphics, non-video) computing that may benefit from GPU use are typically VMs that are running high performance computing applications such as those that require highly repetitive or parallel algorithms that contain building blocks capable of running (separately) on multiple processors. Modeling applications such as statistical regressions on very large data are examples of such applications. Additional examples include text search, encryption, mathematical analysis (such as finite element analysis or fluid dynamics, and the like). Some of these applications do not have real-time sensitivity and can be performed in "batch" mode. GPUs can be used to run such workloads in the off-work hours, after the graphics or video intensive VDI sessions terminate.
  • GPUs can be used to run such workloads in the off-work hours, after the graphics or video intensive VDI sessions terminate.
  • VMs performing 2.5D graphics with a GBF of "4" may include, for example, VMs running Windows with the "Aero" user interface turned on, including Flip 3D.
  • Such user interfaces or similar applications may include a few triangles with texturing, however no lighting or Z-buffering.
  • such applications may employ some image processing operations such as Gaussian Blur.
  • VMs performing video with a GBF of "3" may include, for example, VMs engaged in video playback, which are able to leverage the video decode or transcode capabilities of a GPU.
  • VMs performing 2D with a GBF of "2" may include, for example, VMs with desktop user interfaces that do not fall into any of the above categories. They may still benefit from GPU resources, for example, to speed up the encoding for remoting the user interface.
  • VMs that do not fall into any of the above categories with a default GBF of "1" may include VMs with workloads that would not likely gain much advantage from GPU acceleration.
  • VMs for database management, web serving, email serving, and the like may fall into this category.
  • applications that do not lend themselves to parallelization, overly complex, or are latency sensitive may fall into this category.
  • static priorities may also be set by a user, such as an administrator-user to guarantee certain behaviors in spite of the runtime flexibility affording by the GBFs.
  • static priorities may designate one of four values: OVERRIDE, HIGH, DEFAULT, or BOTTOM.
  • the OVERRIDE priority designation is used to indicate that the GBF ranking for a VM is to be overridden - physical GPU resources are to be allocated to the designated VM whenever available. This mechanism can be used, for example, to give preference to users of VMs running 3D applications that are critical to their jobs. It also supports an allocation mechanism that allows users to pay for guaranteed GPU support, such as in a public cloud or shared resource environment.
  • the GPU allocation logic when the GPU allocation logic detects that a VM with an OVERRIDE priority is actually not making use of the GPU resources (such as a VM with a consistently low GBF as determined from its runtime profile) it may issue alerts or warnings to an administrator-user and/or inform the user of the VM that he or she is paying for unused resources.
  • the HIGH priority designation is used to indicate a "tie- breaker" when two or more VMs with the same GBF are contending for the same GPU allocations.
  • a VM with a GBF of 6 and a priority of HIGH is "seated" (allocated GPU resources) before a VM with a GBF of 6 and a priority of DEFAULT.
  • the BOTTOM priority designation is used to indicate that the VM should appear at the bottom of the GBF allocation list to be assigned "left-over" GPU resources. For example, such left-over GPU resources may be used for batch GPGPU workloads.
  • the DEFAULT priority designation is assigned to any remaining VMs not assigned an OVERRIDE, a HIGH, or a BOTTOM priority. In this case VMs with identical GBFs may be randomly ordered.
  • the DGAS determines which subset of VMs on the GPU allocation list can be allocated physical GPU resources based, for example, on which VMs of the ranked list will fit (require an amount of available resources) on the GPUs.
  • An example allocation of a subset of VMs is described with reference to Figure 5.
  • the DGAS processes events and dynamically updates the GBF of running VMs and hence the GPU allocations. More specifically, in block 305, the DGAS process events, such as starting up a new VM, performing a migration of a running VM, etc., that affects the GBF of at least one VM. The DGAS then updates the GPU allocation list. Some events may result in downgrading a GBF of a particular VM. Other events may set an initial GBF for a VM to be initially run, moved, or run again, based, for example, on past history. An example processing of such events is described with reference to Figure 8.
  • the DGAS determines the runtime profiles of the VMs on the GPU allocation list, re -ranks them, and marks any VMs that can be potentially unseated (when a better contender becomes available) because their runtime profiles indicate that they are not really benefiting from GPU acceleration and not benefitting in an amount that is beyond a tolerable threshold (below or above depending upon how the measurement is performed).
  • a tolerable threshold below or above depending upon how the measurement is performed.
  • only unseating a VM when a threshold is reached insures that the potential benefit of seating a new contender VM avoids the cost of the context switch between VMs.
  • the switching between a VM's use of a CPU and use of a GPU needs to occur sufficiently quickly otherwise the potential gain in switching from one to another may be partially or completely compromised.
  • An example determination of runtime profiles is described with reference to Figure 7.
  • the DGAS changes the GPU resource allocation if warranted. That is, if viable contender VMs are available, then one or more of the VMs previously marked as potentially able to be unseated are unseated (have their GPU resource allocations deallocated) and these resources are then given to one or more of the viable contender VMs.
  • the DGAS continues to execute the logic or processing events and handling dynamic GPU allocation by returning to block 305.
  • FIG 4 is a block diagram of an example GPU allocation list (GAL) used with an example dynamic GPU allocation system.
  • the diagram of Figure 4 illustrates an abstraction of a GAL 400 and, as mentioned, may be implemented by any suitable data structure including for example a list, array, table, database, file, etc.
  • the GAL 400 contains a list of VMs in a ranked order, where each row 410-425 indicates a rank 401 , an identifier of a VM 402, an indicator of a priority 403, an indicator of a GPU benefit factor 404, and an indicator of requirements of that VM such as the amount of video random access memory (VRAM) 405.
  • VRAM video random access memory
  • the priority indicator 403 indicates a value of "2" corresponding to a priority of OVERRIDE; a value of "1" corresponding to a priority of HIGH; a value of corresponding to a priority of BOTTOM; and a value of "0" corresponding to a priority of DEFAULT.
  • the indicator and/or values and/or number of discrete priorities may be different or modified.
  • the VM identifier 402 is shown as a number, however this identifier could be of any form, including for example, text, numeric, mixed, or otherwise.
  • the indicator 404 of the GBF is shown as a numeric value from “1" to "6" as described above; however, this indicator and/or values and/or number of discrete GBFs may be different or modified.
  • the indicator 405 of VM requirements may indicate more than VRAM needs, or may not be present an all in the GAL 400 and may be computed or retrieved from an external resource. Again, the figure demonstrates one example of categorizations of GBF and a priority scheme.
  • VMs are allocated GPU resources based, for example, on their position in the ranked ordering 401 in the GAL 400.
  • the GPU resources comprise two GPUs of 4 gigabytes each
  • the VMs occupying "seats" 1-8 in the ranked order 401 can be allocated virtual resources (e.g. , virtual GPU resources) that correspond to their portion of the physical GPU resources (or assigned to existing virtual resources when previously allocated).
  • the subset of VMs 430 represents those VMs which have received an allocation of GPU resources.
  • the subset of VMs 430 contains 4 VMs shown in rows 410-413 with an OVERRIDE priority; thus, they are allocated GPU resources first without regard to their corresponding GBFs.
  • the subset of VMs 430 also contains 2 VMs shown in rows 414-415 with a HIGH priority and the same GBF of "6" (category assigned to VMs with a 3D type workload).
  • the next VM shown in row 416 has the same GBF but a DEFAULT priority and thus appears after the VMs in rows 414-415.
  • the subset of VMs 430 also contains 1 VM in row 417 with a DEFAULT priority and a GBF of "5.”
  • VMs with an OVERRIDE priority will be considered for GPU resource allocations first, followed by those VMs with the highest GBFs, using a "HIGH" priority to break ties between them.
  • the positions of the VMs on the GAL 400 may vary dynamically as the GBFs of these VMs are decreased or increased through evaluating the runtime profile of these VMs as described with reference to Figures 6A-6B and Figure 7.
  • a GBF may be used without a priority. Further, in other embodiments, the priority may be determined at other times other than when initializing, bringing up, or booting the system and may be retrieved by different methods, such as, for example, a property value stored in an initialization file, on a network resource, or the like.
  • GAL 400 as illustrated includes all running VMs - those using GPU resources and those running on one or more CPUs. In some embodiments, separate lists are maintained for those VMs running on CPUs and those running on GPUs. The logic and handlers for maintaining such lists are correspondingly modified.
  • Figure 5 depicts an example allocation of GPUs among an example set of virtual machines. This allocation corresponds to the subset of VMs 430 indicated in Figure 4 and illustrates how the different VMs are allocated to one of the two corresponding GPUs based upon how their resource requirements match the available resources of the two GPUs 510 and 520, available, for example, as part of server/host hardware 101 of Figure 1.
  • the VMs shown in Figure 5 correspond to the rows 410-425 in the GAL 400, thus, some of the VMs, such as the VM represented by VM representation 510, are not allocated GPU resources.
  • VM representations 501-508 correspond to the eight VMs in the subset of VMs 430 listed in the GAL 400 that are allocated GPU resources.
  • the VMs are allocated GPU resources based upon availability - not necessarily in the order that they appear on the GAL 400. Accordingly, VM representations 501, 502, 503, and 507 are allocated GPU resources from the first physical GPU 530 and VM representations 504, 505, 506, and 508 are allocated GPU resources from the second physical GPU 520. In some embodiments, some portion of one or more of the GPUs will be available and not used if there is no VM that fits in the remaining portion. Also, in some embodiments, a VM further down in the ranked list may be allocated the remaining portion before others with higher GBFs if it fits and the other VMs higher on the ranked list do not. In other embodiments, VMs are not allocated GPU resources out of the ranked order.
  • the VM identified by VM representation 509 is the next likely VM to be seated - receive an allocation of VM resources - if and when one of the other VMs identified by VM representations 501-508 is suspended, moved, closed, and/or unseated, for example, by having its VM changed, for example as a result of an assessment of its runtime profile or as a result of a changed priority.
  • the VM representations 501, 503, and 505 are indicated by dashed lines to indicate for one or more reasons that they are available to be unseated.
  • One or more of these VMs may be unseated to allocate resources to a contender VM, such as the VM identified by VM representation 509.
  • a contender VM such as the VM identified by VM representation 509
  • only one of these VMs available to be unseated would be unseated to address the needs of the contender VM identified by VM representation 509 because the contender VM only needs 256 MB, which are more than available from one of the VMs available to be unseated.
  • a VM can be unseated that would not yield sufficient GPU resources (for example, only would yield 128 MB of VRAM as opposed to the needed 256 MB), then the VM that could be unseated may not be unseated.
  • Example embodiments described herein provide applications, tools, data structures and other support to implement a GPU allocation system to be used to dynamically allocate GPU resources to one or more virtual machines.
  • numerous specific details are set forth, such as data formats and code logic sequences, etc., in order to provide a thorough understanding of the described techniques.
  • the embodiments described also can be practiced without some of the specific details described herein, or with other specific details, such as changes with respect to the ordering of the logic, different logic, etc.
  • the scope of the techniques and/or functions described are not limited by the particular order, selection, or decomposition of aspects described with reference to any particular routine, module, component, and the like.
  • the separate handling of the determination of the GBFs and allocation/reallocation as depicted in Figures 6-8 may not be implemented as separate threads in a production implementation, but are so depicted for ease of description.
  • FIGS 6A and 6B depict a flow diagram of example logic for managing a GPU allocation list and allocating GPU resources based upon the list.
  • Logic 600 may be executed, for example, by the GPU allocation logic 203 of virtualization logic 203 running on the host/server 201 in Figure 2.
  • the GPU allocation list handler is responsible for managing the GAL (such as GAL 400 of Figure 4) including updating the ranked list of VMs based upon runtime profiles and performing any reallocations of GPU resources based upon the updated GAL.
  • Blocks 601-607 described example logic to implement updating the ranked list of VMs.
  • Blocks 609-623 describe example logic to implement adjusting the GPU resource allocations.
  • the logic begins a loop to process each running VM in the GPU allocation list beginning with the first.
  • the logic executes additional logic described with reference to Figure 7 to determine the current runtime profile for the current VM being processed.
  • the runtime profiles of the VMs in the GAL are used to determine the most promising contender VMs (to seat next - i.e., to allocate GPU resources) and to identify which VMs have any actual low utilization of the GPU resources previously allocated to them. Since the GAL includes VMs that are potentials for GPU resource allocation, a current runtime profile is computed for each VM that is currently running on a CPU as well as each VM that is currently running on a GPU.
  • a runtime profile may be viewed as a "fractional" component of a GBF (computed once a VM has been executing); whereas the category of workload (3D, 2D, etc.) may be viewed as an "integer" component of a GBF. In this manner the runtime profile can be used to further distinguish two or more VMs with the same category (base) GBF.
  • the logic determines whether it is necessary to downgrade or upgrade the GBF of the current VM being processed, for example, based upon whether the runtime profile determination has indicated that the current VM is using GPU relevant resources or processing a categorization of workload that is indicative that the current VM would benefit more or less from GPU resource allocations. If so, the logic continues in block 604, otherwise the logic continues in block 605. [0055] In block 604, the logic adjusts the GBF of the current VM being processed. In embodiments that use the runtime profile as a fractional component, it can be added to the base component of the GBF to compute a total GBF.
  • a timeout mechanism may be used to downgrade or upgrade the GBF.
  • a downgrade of a GBF can be performed to downgrade a VM's GBF from a 3D type VM to a 2.5D type VM using a "no 3D commands" timeout (e.g., an alert or signal).
  • an upgrade of a GBF can be performed to upgrade a VM's GBF from a 2D type VM to a 2.5D type VM using a "use 2.5D commands" timeout.
  • an upgrade or downgrade of a GBF may be performed at other and/or different times.
  • VM being processed is below a particular threshold (within or above, depending upon how the threshold is defined), and, if yes, the logic continues in block 606, otherwise the logic continues in block 607.
  • the current VM being processed is marked as a candidate for unseating (de-allocation of GPU resources) as illustrated by the dashed line VM representations in Figure 5.
  • a threshold value is used to prevent unnecessary context switches of VMs on a
  • the threshold value is a system configuration parameter that is predetermined. In other embodiments, the threshold value is varied depending upon load and performance of the DGAS.
  • the logic determines whether there are additional VMs to process, for example, more VMs on the GAL, and if so returns to the beginning of the loop in block 601 , otherwise continues processing in block 609.
  • Blocks 609-623 perform adjustments of the GPU resource allocation in view of the updates determined in blocks 601 -607.
  • the logic determines (e.g. , computes) the total physical GPU resources potentially available for allocation based upon the candidate VMs marked for unseating in block 606 and any other available GPU resources. Additional GPU resources may be available, for example, from leftover GPU resource availability that was previously too small to be allocated to any of the VMs on the GAL (with their requirements as listed at the time of prior allocation).
  • the logic determines (e.g. , computes, receives, etc.) the total amount of physical GPU resources required for possible contender VMs such as the contender VM identified by VM representation 509 of Figure 5. [0061] In block 613, the logic determines whether there exist sufficient GPU resources to allocate for all contender VMs, and, if so, continues in block 615, otherwise continues in block 619. In block 615, the logic de-allocates as many GPU resources from VMs that are candidates for unseating as needed for all contender VMs, and updates the GAL with corresponding statuses. In block 617, the logic allocates the resultant GPU resources to all contender VMs by allocating hardware GPU resources to corresponding virtual resources (e.g. , virtual GPUs) of (or allocating virtual resources if not already allocated to) those contender VMs, and updates the GAL with corresponding statuses. The logic 600 then ends.
  • the logic 600 determines (e.g. , computes, receives, etc.) the total amount of physical GPU
  • the logic determines which contender VMs will fit on the potentially available GPU resources of marked VM candidates for unseating (as marked in block 606).
  • the logic de-allocates GPU resources from the VM candidates for unseating identified in the previous block 619, and updates the GAL with corresponding statuses.
  • the logic allocates the GPU resources just de-allocated in block 621 as needed for the contender VMs, and updates the GAL with corresponding statuses.
  • the logic 600 then ends.
  • the GBFs of VMs may be used to load balance across a server cluster.
  • VMs with a high priority and high GBF on a heavily loaded server GPU may be moved to servers with greater GPU resource availability.
  • a single VM may be accelerated on more than one GPU.
  • higher performance and better system utilization may be achieved by processing the workload (e.g., rendering the frames) of the VM in a distributed manner across the multiple GPUs.
  • the GPUs could parallel process alternating scan lines or different chunks of the VM's frame and then combine the results through a primary GPU or the CPU.
  • the rendering support 205x in conjunction with the GPU allocation logic 203 of Figure 2 may be used to accomplish this task. Hooks for the virtualization logic 202 may be provided to determine a GPU's load and available capacity to aid in organizing this distributed approach. Appropriate load balancing and distribution logic may be incorporated into the logic of Figures 6 A and 6B.
  • Figure 7 is a flow diagram of example logic for determining a runtime profile of a designated virtual machine.
  • Logic 700 may be executed, for example, by the GPU allocation logic 203 of virtualization logic 203 running on the host/server 201 in Figure 2 in combination with workload advisor/GBF factor reporting logic 206x and rendering support 205x for the designated VM that reports information used to compute the runtime profile.
  • the logic 700 reflects one technique (algorithm, mechanism, etc.) for determining a runtime profile. Other techniques may be used and similarly incorporated into a DGAS. This logic is computed, for example, for each current VM being processed by the logic that maintains the GAL.
  • the runtime profile is determined based upon some metric of VRAM use, the type or amount of rendering (e.g., SVGA) commands used, and/or the amount of data transfers (e.g., graphics data transfers via DMA-direct memory access or other mechanism).
  • the type or amount of rendering (e.g., SVGA) commands used e.g., graphics data transfers via DMA-direct memory access or other mechanism.
  • other measurements may be substituted, added, or modified. These factors are defined in such a way that they can be measured independently of whether the workload is being served by a GPU or by a CPU.
  • the logic first determines a representation of load/amount of VRAM in use by the designated VM.
  • Large VRAM utilization suggests that the VM has large models in local storage that are rendered repeatedly, which potentially benefits from a GPU resource allocation.
  • a GPU's VRAM bandwidth is usually faster than CPU RAM's bandwidth, so moving workloads that consume more VRAM to a GPU (allocating hardware GPU resources to the VM) should benefit overall performance.
  • One measure of this load is the average fraction of allocated virtual video RAM (VRAM). For example, equation (1) can be used for this determination:
  • vram_ratio vram_workingset / vram_size (1)
  • vram_workingset is the working set size of virtual VRAM used by the guest (e.g. , the size of textures actually being frequently referred to in the command stream) and vram size is the total size of the virtual VRAM allocated to the guest.
  • the logic determines the load of the command queue (e.g., an SVA command queue, rendering command queue, etc.) from the guest OS to the host.
  • the VM Support 204a can determine the load of commands from guest OS 21 1 to the rendering support 205a.
  • the load of the command queue is suggestive of the execution load of the VM. A mostly empty queue suggests that the VM process is waiting for commands. If this is a VM supported by a GPU, then the GPU is not likely being put to good use. If a workload is sporadic (idle a great deal of the time), then it may be advantageous to move the GPU resources from this VM (unseat the VM) to avoid unnecessary consumption of resources like VRAM during this idle period.
  • the processing unit may be a virtual GPU or some other communications mechanism.
  • the logic determines the portion of data transfers (e.g. , DMA accesses, graphics data transfers, and the like) to the total command bandwidth.
  • a high bandwidth use is suggestive that the data being processed is constantly changing and hence is frequently, if not constantly, being communicated from the guest to the VM. For example, textures and vertex data, and the like are constantly getting generated by the application and transmitted to the VM.
  • PCI bus bandwidth (used to communicate between main memory and a GPU when the GPU resides on a separate card) is a limited resource, so workloads that do a lot of graphics data uploads and/or downloads to/from main memory may cause the PCI bandwidth to become a bottleneck if moved to a GPU. Accordingly, it may be preferable to leave such VMs executing on a CPU. Equation (3) can be used for this determination:
  • data_transfer_ratio (cmd_bandwidth - data_transfers) / (cmd_bandwidth) (3) where data_transfers include, for example, textures and vertex/index/constant buffers and cmd bandwidth is the bandwidth of all commands.
  • vram ratio of 1 means the workload is making full use of the VRAM and given that the GPU's VRAM is typically faster than system memory, it is more efficient to move or keep such a workload running on the GPU;
  • • activity ratio near 0 means that the GPU is not heavily utilized (e.g, is not a bottleneck) and it is often idle, therefore, the workload cannot be significantly sped up by moving it to or keeping it on a GPU; • a data transfer ratio of 1 means that the communication to the GPU is dominated by commands, which can be sent very effectively across the PCI bus to the GPU; and
  • data transfer ratio near 0 means that the communication to the GPU is dominated by data transfers such as containing graphics data, which cannot be virtualized well on a GPU as the PCI bus bandwidth is a scarce resource.
  • this measurement may be incorporated into the GBF of the designated VM as a fractional term.
  • the runtime profile may be used to cause the GBF to be decreased or increased via, for example, a timeout mechanism.
  • the logic determines a weighted formula of the three above measurements to adjust the GBF of the designated VM.
  • GBF_frac (vram_ratio A Wl * activity_ratio A W2 *
  • This equation yields a fractional result in [0, 1], which tends to zero if any of its components tends to zero.
  • the weights may start with defaults (for example, based on benchmarks as part of system certification or configuration) and could be calibrated or improved using benchmarks on a specific system configuration.
  • Figure 8 is a flow diagram of example logic for handling events that cause a GPU benefit factor (GBF) to be set or changed for a designated virtual machine.
  • Logic 800 may be executed, for example, by the GPU allocation logic 203 of virtualization logic 203 running on the host/server 201 in Figure 2. Although shown as separate logic from Figures 6 A and 6B, which manages the GAL, it can be appreciated that the logic of Figure 8 may be performed by one or more separate execution paths (e.g. , threads) or the same execution path of the GAL handler.
  • the logic of Figure 8 is currently shown to handle a certain set of events. Other or different events may be incorporated. Specifically, in block 801 , the logic determines whether it has received a timeout event to adjust a GBF of an indicated VM. If so, then the logic continues in block 802, otherwise continues to process a different event in block 804. In block 802, the logic causes the GBF of the VM corresponding to the event to be revised on the GPU allocation list and continues in block 803. In block 803, the logic determines whether the GBF of the VM corresponding to the event has been downgraded and is below a certain threshold (above or outside depending upon how the threshold is defined) and if so continues to block 805 to de-allocate the VM, otherwise continues to block 809.
  • a certain threshold above or outside depending upon how the threshold is defined
  • the logic determines whether it has received notification that a VM with already allocated GPU resources is to be shutdown or suspended. If so, then the logic continues in block 805, otherwise continues to process a different event in block 806. In block 805, the logic de-allocates the VM (but keeps track of the priority and GBF of the VM for potential resumes if the VM is being suspended) and updates the GAL accordingly. In some embodiments this results in a call to another thread responsible for the GAL. The logic continues to block 809.
  • the logic determines whether it has received notification of a new VM launch, a new application launch on a VM, a VM resume, or a VM migrate (e.g., a vMotion event). If so, then the logic continues in block 808, otherwise continues to block 810. In block 808 the logic determines an "initial" GBF for the VM corresponding to the event and then continues to block 809. If the VM is a newly created VM, then the GBF and priority are determined as expressed above with reference, for example, to Figures 3 and 4.
  • the GBF and priority of the VM last in effect when the VM was last running (before it was suspended or live moved) is used to provide an initial seat for the VM.
  • This provides a "predictive" component to the GBF, because the prior history of the GBF is used to predict the needs of the VM when it is resumed or moved.
  • Loading the VM on the GPU is performed as part of the resume/vMotion (live move) process and before its first frame is rendered, to avoid a disruptive context switch shortly after the VM begins serving its user interface.
  • the GBF is universal across servers that run a compatible hypervisor and thus can be used when a VM is moved.
  • This predictive approach also may be used to smooth out the launching of new applications.
  • a 3D application typically loads its models (geometry, textures, and shaders) before rendering its first frame.
  • An analysis of the graphics rendering commands e.g. , SVGA command queue
  • the GBF category e.g. , 3D, 2.5D, etc.
  • the application (its VM) may be switched to use the GPU resources before this context loading phase and before the first frame is rendered. This avoids the disruption that would likely result if such a context switch were to occur shortly after the application's initiation of rendering.
  • the logic causes the GPU allocation list handler to be invoked to recompute the rankings, markings of available VMs to unseat, and changes to allocations, and then continues to block 810.
  • GPU graphics processing unit
  • the method as described above is provided, wherein the list is configured as a table, an array, a file, a linked list, or a portion of a data base.
  • the method as described above is provided, wherein it is determined that no more virtual machines can be accommodated when the processing bandwidth requirements of the virtual machine designated by the next entry on the GPU allocation list exceed the amount of processing capacity of the one or more physical GPU resources remaining after satisfying the processing bandwidth requirements of the virtual machines designated by all of the preceding entries on the GPU allocation list.
  • the method as described above is provided, wherein the GPU benefit factor associated with each virtual machine designated by each entry of the GPU allocation list is based upon predicted usage in the workload of the designated virtual machine of three dimensional graphics constructs, general purpose GPU processing workloads, 2.5 dimensional graphics constructs, video playback, and/or two dimensional graphics constructs.
  • the method as described may further comprise: determining a runtime profile for each virtual machine designated by the GPU allocation list; and using the determined runtime profiles to adjust the allocations of processing capacity of the one or more physical GPU resources to the virtual machines designated by entries on the GPU allocation list.
  • the method as described may also further comprise: for each entry on the GPU allocation list, determining whether the runtime profile of a designated virtual machine indicates to adjust the GPU benefit factor of the virtual machine; and when determined that the runtime profile of a designated virtual machine indicates to adjust the GPU benefit factor, adjusting the GPU benefit factor of the designated virtual machine; reordering the plurality of virtual machine entries on the GPU allocation list taking into account the adjusted GPU benefit factors indicated by one or more GPU allocation list entries; and adjusting the allocations of processing capacity of the one or more physical GPU resources to the virtual machines designated by entries on the GPU allocation list based upon the reordered plurality of virtual machine entries.
  • the method as described may also provide that using the determined runtime profiles to adjust the allocations of processing capacity of the one or more physical GPUs to the virtual machines designated by entries on the GPU allocation list results in de- allocation of processing capacity of the one or more physical GPU resources to at least one virtual machine designated by entries on the GPU allocation list.
  • the method as described above provides that using the determined runtime profiles to adjust the allocations of processing capacity of the one or more physical GPU resources to the virtual machines designated by entries on the GPU allocation list results in an allocation of processing capacity of the one or more physical GPU resources to at least one virtual machine designated by an entry on the GPU allocation list that does not currently having an allocation of processing capacity of the one or more physical GPU resources.
  • the method described above further comprises: determining whether the runtime profile of at least one virtual machine designated by the GPU allocation list indicates that the at least one virtual machine is a candidate for unseating when a contender virtual machine with a GPU benefit factor that indicates more potential benefit becomes available.
  • the method as described further comprises the steps of: identifying one or more candidate virtual machines designated by entries on the GPU allocation list that are candidates for unseating; determining a potential available processing capacity of the one or more physical GPU resources based upon the identified one or more candidate virtual machines; determining one or more contender virtual machines designated by entries on the GPU allocation list having corresponding GPU benefit factors that exceed the GPU benefit factors of the candidate virtual machines by some threshold; and de-allocating allocations of processing bandwidth resources from one or more of the candidate virtual machines and allocating the de-allocated processing bandwidth resources instead to the one or more contender virtual machines.
  • the previously described method is provided, wherein only a portion of the determined one or more contender virtual machines are allocated the de-allocated processing bandwidth resources.
  • the method of as described above is also provided, wherein the determining the runtime profile for each virtual machine designated by the GPU allocation list is determined by a dynamic assessment of workload indicating potential benefit from use of a GPU resource and/or scaling of workloads.
  • the dynamic assessment of workload includes an assessment of use of three dimensional graphics constructs and/or video.
  • the dynamic assessment of workload includes an assessment of use of a portion of three dimensional graphics constructs.
  • the dynamic assessment of workload includes an assessment of whether an application can benefit from processing on a GPU.
  • the previously described method provides that the runtime profile of a virtual machine includes a measure of the allocated virtual video RAM in use, the size of pending SVGA commands in the queue associated with the virtual machine, and/or a measure of DMA access by applications running on the virtual machine.
  • the method as described further comprises the steps of: receiving a priority associated with each of the plurality of virtual machine entries; and causing the rank ordering of the plurality of virtual machine entries to be based additionally upon the determined priority associated with each designated virtual machine.
  • the method as described is provided, wherein the received priority is a statically assigned priority.
  • the method as described is provided, wherein the received priority indicates at least one of a default value, an override high priority value, a tie-breaker value, or an override low priority value.
  • the previously described method is provided, wherein the determining the processing capacity of the one or more physical GPU resources is based upon determining the amount of video memory available.
  • the previously described method further comprises the steps of: receiving notification of an event; adjusting or setting the GPU benefit factor of at least one virtual machine entry on the GPU allocation list; causing the rank ordering of the one or more virtual machine entries on the GPU allocation list to be newly determined based at least upon the GPU benefit factor associated with the virtual machine designated by each entry on the list; and reallocating the portion of the processing bandwidth resources of the one or more physical GPU resources based upon the newly determined rank ordering of the one or more virtual machine entries on the GPU allocation list.
  • the previously described method further comprises causing a new virtual machine entry to be added to the GPU allocation list.
  • the received notification of the event is a notification that a virtual machine designated by at least one virtual machine entry on the GPU allocation list is being shutdown, suspended, or migrated out, and wherein the reallocating the portion of the processing bandwidth resources deallocates processing bandwidth resources of the one or more physical GPU resources from the designated virtual machine.
  • the received notification of the event is a notification that the GPU benefit factor of a virtual machine designated by at least one virtual machine entry on the GPU allocation list is being downgraded and wherein the reallocating the portion of the processing bandwidth resources de-allocates processing bandwidth resources of the one or more physical GPU resources from the designated virtual machine.
  • the previously described method is provided, wherein the received notification of the event is a notification that a virtual machine is being launched, resumed, or migrated in and wherein the reallocating the portion of the processing bandwidth resources of the one or more physical GPU resources causes processing bandwidth resources of the one or more physical GPU resources to be allocated to the launched, resumed, or migrated in virtual machine.
  • the received notification of the event is a notification that a virtual machine that was previously suspended is being resumed, and wherein adjusting or setting the GPU benefit factor of at least one virtual machine entry on the GPU allocation list sets the GPU benefit factor to be the same as the GPU benefit factor when the virtual machine was suspended.
  • the method described above is provided, wherein the received notification of the event is a notification that a virtual machine was migrated in from a different server, and wherein adjusting or setting the GPU benefit factor of at least one virtual machine entry on the GPU allocation list sets the GPU benefit factor to be the GPU benefit factor associated with the virtual machine when the virtual machine was executing at the different server.
  • the method as described above is provided, wherein the received notification of the event is a notification that a virtual machine designated by at least one virtual machine entry on the GPU allocation list is launching a new application, and wherein the reallocating the portion of the processing bandwidth resources of the one or more physical GPU resources causes processing bandwidth resources of the one or more physical GPU resources to be allocated to the virtual machine that is launching the new application.
  • a non-transitory computer- readable medium stored in a server computing system containing content for managing a graphics processing unit (GPU) allocation data structure for allocating physical GPU resources, including processing bandwidth, to one or more virtual machines, the data structure comprising a plurality of virtual machine entries each containing a designation of a virtual machine, an indication of a GPU benefit factor associated with the designated virtual machine, and an indication of processing bandwidth requirements associated with the designated virtual machine, by performing a method comprising: ordering the plurality of virtual machine entries based at least upon the GPU benefit factor associated with each designated virtual machine; determining the processing capacity of one or more physical GPUs associated with the server computing system; determining one or more entries of the GPU allocation data structure to which to assign to the corresponding designated virtual machines a portion of the processing bandwidth resources of the one or more physical GPUs, the determining based upon matching the indicated processing bandwidth requirements associated with each corresponding designated virtual machine to the determined processing capacities of the one or more physical GPUs; and causing processing bandwidth resources of
  • GPU graphics processing unit
  • the computer-readable medium as described above is provided, wherein the medium is a memory of a computing system and the content is computer instructions stored in the memory.
  • the computer-readable medium as described above is provided, wherein the ordering of the plurality of virtual machine entries is also based upon a priority assignment.
  • the computer-readable medium described above is provided, wherein the data structure is implemented as at least one of a list, table, array, file, data base, or cloud repository.
  • a computing system comprising: a memory; a computer processor; and a GPU allocation data structure specifying a plurality of virtual machine entries each containing a designation of a virtual machine, an indication of a GPU benefit factor associated with the designated virtual machine, and an indication of processing bandwidth requirements associated with the designated virtual machine; virtualization logic, stored in the memory, and configured, when executed on the computer processor, to: dynamically allocate physical GPU resources of one or more physical GPUs among the plurality of virtual machines by: rank ordering the plurality of virtual machine entries of the GPU allocation data structure according to the GPU benefit factor associated with each designated virtual machine; determining the processing capacity of the one or more physical GPU resources; determining a subset of the rank ordered entries of the GPU allocation data structure having designated virtual machines that can use the one or more physical GPU resources based upon the determined processing capacities of the one or more physical GPU resources as compared to the indicated processing requirements of virtual machines designated by some of the plurality of virtual machine entries in rank order; and allocating a virtual GPU to
  • the computing system described above provides, in embodiments, that the rank ordering of the plurality of virtual machine entries of the GPU allocation data structure is also based upon a priority, the priority indicating at least one of an override priority, a tie breaker between GPU benefit factors that are the same, a bottom priority, or a default priority.
  • the computing system as described above may further comprise: notification logic, stored in the memory, and configured, when executed on the computer processor, to: receive an event that indicates a change to the status of a virtual machine; add a new entry in the GPU allocation data structure when an entry designating the virtual machine does not yet exist; adjust or set the GPU benefit factor associated with a virtual machine designated by one of the entries in the GPU allocation data structure; and cause the rank ordering of the plurality of virtual machine entries to be reevaluated and the portion of the processing bandwidth resources of the one or more physical GPU resources to be reallocated according to the reevaluated ordering.
  • notification logic stored in the memory, and configured, when executed on the computer processor, to: receive an event that indicates a change to the status of a virtual machine; add a new entry in the GPU allocation data structure when an entry designating the virtual machine does not yet exist; adjust or set the GPU benefit factor associated with a virtual machine designated by one of the entries in the GPU allocation data structure; and cause the rank ordering of the plurality of virtual machine entries to be ree
  • a computer-implemented method in a server computing system for providing runtime profiles of a plurality of virtual machines for use in allocation of physical graphics processing unit (GPU) resources, including processing bandwidth, to one or more of the plurality of virtual machines comprising: dynamically determining a runtime assessment of workload for each virtual machine of the plurality of virtual machines, the workload runtime assessment indicative of a potential benefit that the virtual machine can receive from using a GPU resource; and causing processing bandwidth of the physical GPU resources to be allocated to some of the plurality of virtual machines based upon the dynamically determined runtime assessments of workload.
  • GPU graphics processing unit
  • the method as described above further comprises : forwarding the dynamically determined runtime assessments of workload for the plurality of virtual machines to cause the plurality of virtual machines to be ordered based upon the indicated potential benefit of using a GPU resource determined for each virtual machine; and causing allocating of processing bandwidth resources of the physical GPU resources to some of the plurality of virtual machines according to the ordering by comparing the processing requirements of the some of the virtual machines to the available processing bandwidth of the physical GPU resources.
  • the method as described above is provided, wherein the causing the plurality of virtual machines to be ordered based upon the indicated potential benefit is based in part upon the potential benefit and in part upon other factors.
  • the method as described above is provided, wherein the other factors include a GPU benefit factor assigned to each virtual machine, the GPU benefit factor assigned based upon a predicted type of workload of the virtual machine.
  • the method described above provides that the other factors include a priority assigned to each virtual machine.
  • the previously described method provides that the dynamically determined runtime assessment of workload for each virtual machine determines a combined value of processing parameters that are each indicative of benefit to using a GPU resource.
  • each virtual machine has a video RAM (VRAM) and wherein the processing parameters include a measure of VRAM usage.
  • VRAM video RAM
  • the measure of VRAM usage comprises an average ratio of the amount of the VRAM being used by the virtual machine to the total size of the VRAM allocated for the virtual machine.
  • each virtual machine has a command queue and wherein the processing parameters include a measure of load of the command queue.
  • the measure of load of the command queue comprises an amount indicative of time the command queue is active.
  • command queue is a virtual GPU command queue.
  • the method described above is provided, wherein at least some of the plurality of virtual machines have a processing unit for handling graphics and wherein the processing parameters include a measure of activity on the processing unit for handling graphics.
  • the method as described above is provided, wherein the measure of activity comprises an amount of time the processing unit for handling graphics is idle over a particular time interval.
  • the processing unit for handling graphics is a virtual GPU.
  • processing parameters include a measure of processing performed not using a data transfer mechanism.
  • the measure of processing performed not using the data transfer mechanism comprises a measurement of the amount of total command bandwidth performed not using the data transfer mechanism to the total command bandwidth.
  • the method as described above is provided, wherein the data transfer mechanism comprises a DMA transfer.
  • each virtual machine has VRAM, a command queue, and at least one data transfer mechanism, wherein the combined value of processing parameters comprises a combination of a value indicative of an amount of VRAM in use, a value indicative of an amount the command queue is utilized, and a value indicative of an amount of command bandwidth not using the data transfer mechanism.
  • the method described above is provided, wherein the combined value of processing parameters comprises a weighted combination of processing parameters.
  • the method described above is provided, wherein the combined value yields a fractional component used in ordering the plurality of virtual machines for GPU allocation.
  • the method described above comprises multiplying each processing parameter together.
  • each processing parameter comprises a value between 0 and 1, where values near 1 indicate situations favorable to physical GPU resource allocations and values near 0 indicate situations favorable to CPU resource allocations.
  • a non-transitory computer-readable medium stored in a server computing system containing content for providing runtime profiles of a plurality of virtual machines for use in allocation of physical graphics processing unit (GPU) resources, including processing bandwidth, to one or more of the plurality of virtual machines by performing a method comprising: determining a runtime assessment of GPU favorable workload for each virtual machine of the plurality of virtual machines, the workload runtime assessment indicative of a workable favorable to allocation of a GPU resource; causing ordering of the plurality of virtual machines based upon the determined runtime assessments; and causing physical GPU resources to be allocated to some of the plurality of virtual machines based upon the ordering of the plurality of virtual machines by matching the processing requirements of the some of the virtual machines to available processing bandwidth of the physical GPU resources according to the ordering.
  • GPU graphics processing unit
  • the computer-readable medium described above is provided, wherein the medium is a memory of a computing system and the content is computer instructions stored in the memory.
  • the computer-readable medium as described above wherein the determined runtime assessment of GPU favorable workload for each virtual machine determines a combined value of processing parameters that are each indicative of benefit to using a GPU resource.
  • the computer-readable medium described above provides that the processing parameters include one or more of a measure of usage of VRAM of the virtual machine, a measure of activity in a command queue of the virtual machine, and/or a measure indicative of data transfers within the virtual machine.
  • the computer-readable medium as described above wherein the combined value of processing parameters comprises a weighted combination of processing parameters.
  • a server computing system comprising: a memory; a computer processor; and virtualization logic, stored in the memory, and configured, when executed on the computer processor, to: determine a runtime assessment of workload for each virtual machine of the plurality of virtual machines, the workload runtime assessment indicative of a potential benefit that the virtual machine can receive from using a GPU resource; and cause processing bandwidth of the physical GPU resources to be allocated to some of the plurality of virtual machines based upon the determined runtime assessments of workload.
  • the computing system further comprises: a GPU allocation data structure, including an ordering of the plurality of virtual machines based upon the determined runtime assessments of work for each virtual machine of the plurality of virtual machines.
  • the computing system described above is provided, wherein the determined runtime assessments each virtual machine comprises a combined value of processing parameters that are each indicative of a benefit to using a GPU resource.
  • the computing system just described is provided, wherein the processing parameters include one or more of a measure of usage of VRAM of the virtual machine, a measure of activity in a command queue of the virtual machine, and/or a measure indicative of data transfers within the virtual machine.
  • the computing system as previously described may provide that the combined value of processing parameters comprises a weighted combination of processing parameters.
  • the methods, techniques, and systems for performing video encoding for UI remoting discussed herein are applicable to other architectures other than an x86 architecture.
  • the methods and systems discussed herein are applicable to differing protocols, communication media (optical, wireless, cable, etc.) and devices (such as wireless handsets, electronic organizers, personal digital assistants, portable email machines, tablets, notebooks, game machines, pagers, navigation devices such as GPS receivers, etc.).

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)

Abstract

L'invention concerne des procédés, des techniques et des systèmes permettant d'attribuer de façon dynamique des unités de traitement graphique entre des machines virtuelles. Des modes de réalisation donnés à titre d'exemple concernent un système d'attribution de GPU dynamique (« DGAS ») qui permet d'attribuer efficacement des ressources GPU physiques à une ou plusieurs machines virtuelles. Dans un mode de réalisation, le DGAS comprend une logique de virtualisation s'exécutant sur un système informatique de serveur qui calcule les facteurs bénéfices d'une GPU pour les machines virtuelles de façon dynamique, et combine les GBF calculés avec des priorités statiques pour déterminer un classement des machines virtuelles. Les ressources de la GPU disponibles sont ensuite attribuées à un sous-ensemble de ces machines virtuelles classées lorsque la capacité de la GPU physique est appariée avec les exigences du sous-ensemble. Les ressources de la GPU physique sont ensuite attribuées au sous-ensemble des machines virtuelles qui offrent le meilleur potentiel pour l'utilisation de la GPU.
PCT/US2013/076854 2012-12-20 2013-12-20 Attribution dynamique d'unités de traitement graphique physiques à des machines virtuelles Ceased WO2014100558A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US13/722,998 US9142004B2 (en) 2012-12-20 2012-12-20 Dynamic allocation of physical graphics processing units to virtual machines
US13/723,021 2012-12-20
US13/722,998 2012-12-20
US13/723,035 US9417929B2 (en) 2012-12-20 2012-12-20 Runtime profile determinations of virtual machines for graphics processing unit (GPU) allocation
US13/723,021 US9298490B2 (en) 2012-12-20 2012-12-20 Managing a data structure for allocating graphics processing unit resources to virtual machines
US13/723,035 2012-12-20

Publications (1)

Publication Number Publication Date
WO2014100558A1 true WO2014100558A1 (fr) 2014-06-26

Family

ID=50030449

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/076854 Ceased WO2014100558A1 (fr) 2012-12-20 2013-12-20 Attribution dynamique d'unités de traitement graphique physiques à des machines virtuelles

Country Status (1)

Country Link
WO (1) WO2014100558A1 (fr)

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017027584A1 (fr) * 2015-08-10 2017-02-16 Amazon Technologies, Inc. Virtualisation de traitement graphique dans un réseau de fournisseur
US9904973B2 (en) 2015-11-11 2018-02-27 Amazon Technologies, Inc. Application-specific virtualized graphics processing
US9904975B2 (en) 2015-11-11 2018-02-27 Amazon Technologies, Inc. Scaling for virtualized graphics processing
US9904974B2 (en) 2015-11-11 2018-02-27 Amazon Technologies, Inc. Placement optimization for virtualized graphics processing
CN108347341A (zh) * 2017-01-24 2018-07-31 华为技术有限公司 一种用于调整虚拟机加速能力的加速能力调整方法及装置
US10102605B1 (en) 2016-08-26 2018-10-16 Amazon Technologies, Inc. Graphics library virtualization for virtualized graphics processing
US10169841B1 (en) 2017-03-27 2019-01-01 Amazon Technologies, Inc. Dynamic interface synchronization for virtualized graphics processing
US10181172B1 (en) 2016-06-08 2019-01-15 Amazon Technologies, Inc. Disaggregated graphics asset delivery for virtualized graphics
US10181173B1 (en) 2016-06-08 2019-01-15 Amazon Technologies, Inc. Disaggregated graphics asset management for virtualized graphics
US10200249B1 (en) 2016-12-09 2019-02-05 Amazon Technologies, Inc. Network traffic management for virtualized graphics devices
CN109408449A (zh) * 2017-08-15 2019-03-01 Arm有限公司 数据处理系统
US10255652B2 (en) 2017-01-18 2019-04-09 Amazon Technologies, Inc. Dynamic and application-specific virtualized graphics processing
CN109726008A (zh) * 2017-10-31 2019-05-07 阿里巴巴集团控股有限公司 资源分配方法及设备
US10338847B1 (en) 2017-08-28 2019-07-02 Amazon Technologies, Inc. Coherent buffer mapping for virtualized graphics processing
US10373284B2 (en) 2016-12-12 2019-08-06 Amazon Technologies, Inc. Capacity reservation for virtualized graphics processing
US10423463B1 (en) 2016-06-09 2019-09-24 Amazon Technologies, Inc. Computational task offloading for virtualized graphics
US10430911B1 (en) 2016-12-16 2019-10-01 Amazon Technologies, Inc. Graphics overlays with virtualized graphics processing
US10482561B1 (en) 2017-01-11 2019-11-19 Amazon Technologies, Inc. Interaction monitoring for virtualized graphics processing
US10593009B1 (en) 2017-02-22 2020-03-17 Amazon Technologies, Inc. Session coordination for auto-scaled virtualized graphics processing
US10692168B1 (en) 2017-01-26 2020-06-23 Amazon Technologies, Inc. Availability modes for virtualized graphics processing
US10908940B1 (en) 2018-02-26 2021-02-02 Amazon Technologies, Inc. Dynamically managed virtual server system
CN112732449A (zh) * 2021-01-20 2021-04-30 湖南大学 基于gpu虚拟化技术的显存资源分配方法、装置及设备
CN113703961A (zh) * 2021-07-22 2021-11-26 新华三大数据技术有限公司 一种gpu资源管理方法及装置
US11416300B2 (en) * 2017-06-29 2022-08-16 Intel Corporaton Modular accelerator function unit (AFU) design, discovery, and reuse
DE102020110143B4 (de) 2019-04-26 2025-01-23 Hewlett Packard Enterprise Development Lp Standortbasierte virtualisierungs-workload-platzierung
US12282794B2 (en) 2020-11-04 2025-04-22 Arm Limited Data processing systems
WO2025124208A1 (fr) * 2023-12-14 2025-06-19 天翼云科技有限公司 Procédé de planification de surexploitation de ressources basé sur une virtualisation de gpu
WO2025163417A1 (fr) * 2024-01-29 2025-08-07 云智能资产控股（新加坡）私人股份有限公司 Procédé et appareil d'appel de processeur, et dispositif

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100115510A1 (en) * 2008-11-03 2010-05-06 Dell Products, Lp Virtual graphics device and methods thereof
US20110083131A1 (en) * 2009-10-01 2011-04-07 Fahd Pirzada Application Profile Based Provisioning Architecture For Virtual Remote Desktop Infrastructure
US20120069032A1 (en) * 2010-09-17 2012-03-22 International Business Machines Corporation Optimizing Virtual Graphics Processing Unit Utilization

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100115510A1 (en) * 2008-11-03 2010-05-06 Dell Products, Lp Virtual graphics device and methods thereof
US20110083131A1 (en) * 2009-10-01 2011-04-07 Fahd Pirzada Application Profile Based Provisioning Architecture For Virtual Remote Desktop Infrastructure
US20120069032A1 (en) * 2010-09-17 2012-03-22 International Business Machines Corporation Optimizing Virtual Graphics Processing Unit Utilization

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
VISHAKHA GUPTA ET AL: "Pegasus: Coordinated Scheduling for Virtualized Accelerator-based Systems Niraj Tolia Maginatics Vanish Talwar HP Labs", 1 January 2011 (2011-01-01), XP055108443, Retrieved from the Internet <URL:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.306.9999&rep=rep1&type=pdf> [retrieved on 20140318] *

Cited By (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2016304864B2 (en) * 2015-08-10 2019-12-05 Amazon Technologies, Inc. Virtualizing graphics processing in a provider network
US12361909B2 (en) 2015-08-10 2025-07-15 Amazon Technologies, Inc. Virtualizing graphics processing in a provider network
WO2017027584A1 (fr) * 2015-08-10 2017-02-16 Amazon Technologies, Inc. Virtualisation de traitement graphique dans un réseau de fournisseur
US11145271B2 (en) 2015-08-10 2021-10-12 Amazon Technologies, Inc. Virtualizing graphics processing in a provider network
CN107924335A (zh) * 2015-08-10 2018-04-17 亚马逊科技公司 供应商网络中图形处理的虚拟化
AU2020201660B2 (en) * 2015-08-10 2021-06-24 Amazon Technologies, Inc. Virtualizing graphics processing in a provider network
US11210759B2 (en) 2015-11-11 2021-12-28 Amazon Technologies, Inc. Placement optimization for virtualized graphics processing
US10699367B2 (en) 2015-11-11 2020-06-30 Amazon Technologies, Inc. Placement optimization for virtualized graphics processing
US9904973B2 (en) 2015-11-11 2018-02-27 Amazon Technologies, Inc. Application-specific virtualized graphics processing
US10628908B2 (en) 2015-11-11 2020-04-21 Amazon Technologies, Inc. Application-specific virtualized graphics processing
US9904975B2 (en) 2015-11-11 2018-02-27 Amazon Technologies, Inc. Scaling for virtualized graphics processing
US9904974B2 (en) 2015-11-11 2018-02-27 Amazon Technologies, Inc. Placement optimization for virtualized graphics processing
US10430916B2 (en) 2015-11-11 2019-10-01 Amazon Technologies, Inc. Placement optimization for virtualized graphics processing
US10181172B1 (en) 2016-06-08 2019-01-15 Amazon Technologies, Inc. Disaggregated graphics asset delivery for virtualized graphics
US10181173B1 (en) 2016-06-08 2019-01-15 Amazon Technologies, Inc. Disaggregated graphics asset management for virtualized graphics
US10600144B2 (en) 2016-06-08 2020-03-24 Amazon Technologies, Inc. Disaggregated graphics asset management for virtualized graphics
US10423463B1 (en) 2016-06-09 2019-09-24 Amazon Technologies, Inc. Computational task offloading for virtualized graphics
US10102605B1 (en) 2016-08-26 2018-10-16 Amazon Technologies, Inc. Graphics library virtualization for virtualized graphics processing
US10652094B2 (en) 2016-12-09 2020-05-12 Amazon Technologies, Inc. Network traffic management for virtualized graphics devices
US10200249B1 (en) 2016-12-09 2019-02-05 Amazon Technologies, Inc. Network traffic management for virtualized graphics devices
US10726518B2 (en) 2016-12-12 2020-07-28 Amazon Technologies, Inc. Capacity reservation for virtualized graphics processing
US10373284B2 (en) 2016-12-12 2019-08-06 Amazon Technologies, Inc. Capacity reservation for virtualized graphics processing
US10430911B1 (en) 2016-12-16 2019-10-01 Amazon Technologies, Inc. Graphics overlays with virtualized graphics processing
US10963984B2 (en) 2017-01-11 2021-03-30 Amazon Technologies, Inc. Interaction monitoring for virtualized graphics processing
US10482561B1 (en) 2017-01-11 2019-11-19 Amazon Technologies, Inc. Interaction monitoring for virtualized graphics processing
US10255652B2 (en) 2017-01-18 2019-04-09 Amazon Technologies, Inc. Dynamic and application-specific virtualized graphics processing
US10650484B2 (en) 2017-01-18 2020-05-12 Amazon Technologies, Inc. Dynamic and application-specific virtualized graphics processing
US11379255B2 (en) 2017-01-24 2022-07-05 Huawei Technologies Co., Ltd. Acceleration capacity adjustment method and apparatus for adjusting acceleration capacity of virtual machine
EP3562093A4 (fr) * 2017-01-24 2020-01-22 Huawei Technologies Co., Ltd. Procédé et appareil permettant de régler la capacité d'accélération d'une machine virtuelle
CN108347341A (zh) * 2017-01-24 2018-07-31 华为技术有限公司 一种用于调整虚拟机加速能力的加速能力调整方法及装置
US10692168B1 (en) 2017-01-26 2020-06-23 Amazon Technologies, Inc. Availability modes for virtualized graphics processing
US10593009B1 (en) 2017-02-22 2020-03-17 Amazon Technologies, Inc. Session coordination for auto-scaled virtualized graphics processing
US11710206B2 (en) 2017-02-22 2023-07-25 Amazon Technologies, Inc. Session coordination for auto-scaled virtualized graphics processing
US10169841B1 (en) 2017-03-27 2019-01-01 Amazon Technologies, Inc. Dynamic interface synchronization for virtualized graphics processing
US11416300B2 (en) * 2017-06-29 2022-08-16 Intel Corporaton Modular accelerator function unit (AFU) design, discovery, and reuse
US12112204B2 (en) 2017-06-29 2024-10-08 Intel Corporation Modular accelerator function unit (AFU) design, discovery, and reuse
CN109408449A (zh) * 2017-08-15 2019-03-01 Arm有限公司 数据处理系统
US10338847B1 (en) 2017-08-28 2019-07-02 Amazon Technologies, Inc. Coherent buffer mapping for virtualized graphics processing
CN109726008A (zh) * 2017-10-31 2019-05-07 阿里巴巴集团控股有限公司 资源分配方法及设备
CN109726008B (zh) * 2017-10-31 2022-12-09 阿里巴巴集团控股有限公司 资源分配方法及设备
US10908940B1 (en) 2018-02-26 2021-02-02 Amazon Technologies, Inc. Dynamically managed virtual server system
DE102020110143B4 (de) 2019-04-26 2025-01-23 Hewlett Packard Enterprise Development Lp Standortbasierte virtualisierungs-workload-platzierung
US12282794B2 (en) 2020-11-04 2025-04-22 Arm Limited Data processing systems
CN112732449A (zh) * 2021-01-20 2021-04-30 湖南大学 基于gpu虚拟化技术的显存资源分配方法、装置及设备
CN112732449B (zh) * 2021-01-20 2024-04-09 湖南大学 基于gpu虚拟化技术的显存资源分配方法、装置及设备
CN113703961A (zh) * 2021-07-22 2021-11-26 新华三大数据技术有限公司 一种gpu资源管理方法及装置
WO2025124208A1 (fr) * 2023-12-14 2025-06-19 天翼云科技有限公司 Procédé de planification de surexploitation de ressources basé sur une virtualisation de gpu
WO2025163417A1 (fr) * 2024-01-29 2025-08-07 云智能资产控股（新加坡）私人股份有限公司 Procédé et appareil d'appel de processeur, et dispositif

Similar Documents

Publication Publication Date Title
US9298490B2 (en) Managing a data structure for allocating graphics processing unit resources to virtual machines
US9142004B2 (en) Dynamic allocation of physical graphics processing units to virtual machines
US9417929B2 (en) Runtime profile determinations of virtual machines for graphics processing unit (GPU) allocation
WO2014100558A1 (fr) Attribution dynamique d&#39;unités de traitement graphique physiques à des machines virtuelles
US11094031B2 (en) GPU resource usage display and dynamic GPU resource allocation in a networked virtualization system
US11367160B2 (en) Simultaneous compute and graphics scheduling
US9898795B2 (en) Host-based heterogeneous multi-GPU assignment
EP2622470B1 (fr) Techniques permettant de procéder à un équilibrage de charge sur des machines virtuelles gpu
US8910153B2 (en) Managing virtualized accelerators using admission control, load balancing and scheduling
US8751658B2 (en) Allocating method, allocating apparatus, and computer product
US8667496B2 (en) Methods and systems of managing resources allocated to guest virtual machines
JP5232914B2 (ja) マルチプロセッサにおけるデータパラレルコンピューティングのためのアプリケーションプログラミングインターフェイス
US9727360B2 (en) Optimizing virtual graphics processing unit utilization
CN105843683B (zh) 用于动态优化平台资源分配的方法、系统和设备
WO2019217573A1 (fr) Attribution de tâche dans des systèmes activés par processeur graphique virtuel
US8587594B2 (en) Allocating resources based on a performance statistic
US9176787B2 (en) Preserving, from resource management adjustment, portions of an overcommitted resource managed by a hypervisor
US20140373010A1 (en) Intelligent resource management for virtual machines
JP2007512632A (ja) アクティビティ情報を用いての仮想マシンの管理
AU2008239697A1 (en) Parallel runtime execution on multiple processors
US10360070B2 (en) Application-level dispatcher control of application-level pseudo threads and operating system threads
KR102001641B1 (ko) 가상화 환경에서의 gpu 자원 관리 방법 및 장치
US8402191B2 (en) Computing element virtualization
Gao et al. VirtualGraphGPU: A GPU Virtualization Solution Based on Paravirtualization
HK1187425A (en) Techniques for load balancing gpu enabled virtual machines

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13826684

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13826684

Country of ref document: EP

Kind code of ref document: A1