WO2017133484A1 - 虚拟机部署方法及装置 - Google Patents

虚拟机部署方法及装置 Download PDF

Info

Publication number
WO2017133484A1
WO2017133484A1 PCT/CN2017/071831 CN2017071831W WO2017133484A1 WO 2017133484 A1 WO2017133484 A1 WO 2017133484A1 CN 2017071831 W CN2017071831 W CN 2017071831W WO 2017133484 A1 WO2017133484 A1 WO 2017133484A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual machine
data
node
fragmentation
machine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/071831
Other languages
English (en)
French (fr)
Inventor
阳际荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to EP17746804.8A priority Critical patent/EP3413193A4/en
Priority to JP2018539345A priority patent/JP2019504420A/ja
Publication of WO2017133484A1 publication Critical patent/WO2017133484A1/zh
Priority to US16/054,858 priority patent/US10740194B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45504—Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G06F11/1469—Backup restoration techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1456—Hardware arrangements for backup
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2094—Redundant storage or storage space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45575—Starting, stopping, suspending or resuming virtual machine instances

Definitions

  • the present invention belongs to the field of computer technologies, and in particular, to a virtual machine deployment method and apparatus.
  • Cloud computing adopts an innovative computing model to enable users to obtain near-infinite computing power and rich and diverse information services through the Internet at any time.
  • Cloud computing combines virtualization technology with the scalability and flexibility of virtualization technology to increase resource utilization and save costs.
  • an entity's physical machine has a total of 32 cores, and 30 virtual machines (VMs) can be allocated. Each VM requires 4 cores.
  • the physical machine virtualizes 30 virtual machines.
  • VMs virtual machines
  • all services are not in full load at the same time.
  • the cores in the low-load state are shared in the cloud computing to complete the virtual machine out of 30 virtual machines.
  • a large amount of data may be shared by one virtual machine, which causes a problem that the load pressure of the virtual machine is excessive.
  • the number of products in the subsidiary store is often much higher than that of the ordinary store, and the search heat is higher than that of the ordinary store.
  • the virtualization is based on the above virtual mode, there may be multiple subsidiary store products.
  • the data is deployed to the same virtual machine, when the buyer searches based on the seller, there is a problem that the virtual machine load pressure is high.
  • the present invention provides a virtual machine deployment method and apparatus for solving the problem that the existing virtualization mode causes the virtual machine load pressure to be excessive when a large amount of data is deployed to the same virtual machine.
  • the present invention provides a virtual machine deployment method, including:
  • the present invention provides a virtual machine deployment apparatus, including:
  • Dispatching module used to process the data to perform fragmentation and fragmentation to obtain a fragmentation node
  • An allocating module configured to allocate a target virtual machine to the fragment node according to the data volume of the fragment node
  • a deployment module configured to deploy the fragmentation node to the target virtual machine.
  • the virtual machine deployment method and device of the present invention obtains a fragmentation node by dispersing the fragmentation to be processed, allocates a target virtual machine to the fragmentation node according to the data volume of the fragmentation node, and deploys the fragmentation node to the target virtual machine.
  • the data volume of the fragment node is controlled by data breaking, and the virtual machine matching the data amount is allocated, thereby avoiding the phenomenon that the virtual machine is overloaded. It is good to achieve load balancing.
  • FIG. 1 is a schematic flowchart of a virtual machine deployment method according to Embodiment 1 of the present invention.
  • FIG. 2 is a schematic flowchart of a virtual machine deployment method according to Embodiment 2 of the present invention.
  • FIG. 3 is a schematic diagram of the consistent hash algorithm provided in the second embodiment
  • FIG. 5 is a schematic diagram of application of a virtual machine deployment method provided in Embodiment 2;
  • FIG. 6 is a second schematic diagram of application of the virtual machine deployment method provided in the second embodiment
  • FIG. 7 is a schematic structural diagram of a virtual machine deployment apparatus according to Embodiment 3 of the present invention.
  • FIG. 8 is a schematic structural diagram of a virtual machine deployment apparatus according to Embodiment 4 of the present invention.
  • FIG. 1 it is a schematic flowchart of a virtual machine deployment method according to Embodiment 1 of the present invention.
  • the virtual machine deployment method includes:
  • Step 101 Perform processing on the data to perform fragmentation and fragmentation to obtain a fragmentation node.
  • the data to be processed is fragmented according to a preset breaking algorithm, and the fragment node is obtained.
  • the fragment node is a fragmented fragment data. Specifically, first, according to the data size of the data to be processed, the number of pieces to be broken is set, and after the number of pieces is set, the data to be processed is broken and sliced based on the breaking algorithm.
  • the data to be processed is 2 billion pieces of data, which can be broken up into 256 pieces, that is, 2 billion pieces of data are broken up.
  • the Hash algorithm can be used to break up the product data, and the product data is stored based on the fragmented node obtained by the scattering.
  • Step 102 Assign a target virtual machine to the fragment node according to the data volume of the fragment node.
  • the amount of data obtained by each fragment node is different. After the fragment nodes are acquired, the amount of data of each fragment node is calculated.
  • the target virtual machine After the data volume of the fragment node is obtained, the target virtual machine is allocated to the fragment node.
  • the target virtual machine needs to have the processing capability for processing the data volume included in the fragment node, wherein the processing capability can be used by the CPU. Characterizing at least one of performance parameters such as memory and disk.
  • Step 103 Deploy the fragmentation node to the target virtual machine.
  • the fragmentation node is obtained by de-split fragmentation of the data to be processed, the data volume corresponding to the fragmentation node is calculated, and the target virtual machine is allocated to the fragmentation node according to the data volume of the fragmentation node, The fragmentation node is deployed to the target virtual machine.
  • the data volume of the fragment node is controlled by data breaking, and the virtual machine matching the data amount is allocated, thereby avoiding the phenomenon that the virtual machine is overloaded. Good load balancing is achieved.
  • FIG. 2 it is a schematic flowchart of a virtual machine deployment method according to Embodiment 2 of the present invention.
  • the virtual machine deployment method includes:
  • Step 201 Enter performance parameters of all virtual machines into the machine pool.
  • the performance parameters of all VMs are input into the machine pool in this embodiment.
  • the performance parameters of the VM include the memory size, disk size, and number of cores of the VM.
  • Step 202 Perform processing on the data to perform fragmentation and fragmentation to obtain a fragmentation node.
  • Step 203 Obtain backup data of the fragment node.
  • a consistent hash (Hash) algorithm is used to break down the sliced data, firstly, according to the data size of the data to be processed, the number of broken slices is set, and after the number of slices is set, the consistency is based on the hash.
  • the algorithm performs fragmentation. To ensure the security of the data, the fragment node can be backed up to obtain the backup data of the fragment node.
  • FIG. 3 it is a schematic diagram of the consistent hash algorithm provided in the second embodiment.
  • the Consistent Hash algorithm specifically maps the Value to a 32-bit key value, which is the value space of 0 ⁇ 2 ⁇ 32-1 power, and imagines this space as a first (0) tail (2 ⁇ 32-1) Connected rings. Then map the data to a specific node, such as key%N, key is the key of the data, and N is the number of nodes.
  • key%N key is the key of the data
  • N is the number of nodes.
  • the large circle on the ring represents a node
  • the small circle on the ring represents the node data
  • the main data and backup data are set under each node.
  • the data can be broken up into the results shown in Figure 4 by the consistent hash algorithm.
  • 1 billion pieces of data are broken into 4 pieces, and each piece corresponds to one piece node.
  • the data is backed up, and multiple data are set under each fragment node, and the multiple data under the fragment node are equal, that is, Multiple copies of data are primary backup data.
  • the fragment node 1 includes data 1 to data 4, wherein the data 1, the data 2, the data 3, and the data 4 are mutually primary backup data.
  • the fragment node 2 includes data 1 to data 2, wherein the data 1 and the data 2 are mutually primary backup data.
  • Step 204 Calculate a data amount corresponding to the fragment node.
  • the number of fields included in one piece of data in the fragment node, the field type corresponding to each field, and the number of tables included in one piece of data, and the number of words included in the fragment node are obtained, and the number of the terms is used. Indicates how many pieces of data the fragment node contains. Further, the number of bits occupied by one piece of data can be calculated according to the number of fields included in one piece of data, the type of field corresponding to each field, and the number of tables included. In practice, when the field type is obtained, the number of bits occupied by the field type can be obtained.
  • the integer data int occupies 4 bits (byte); the long integer data (long) occupies 4 bytes; double integer data ( Double) takes up 8 bytes, and the string (char) occupies 2 bytes.
  • the amount of data of the fragmentation node is calculated according to the number of entries and the number of bits occupied by one piece of data in the fragmentation node. Specifically, the number of terms is multiplied by the number of bits occupied by one piece of data to obtain the amount of data of the segmentation node.
  • Step 205 Acquire processing capability required to process the fragmentation node according to the amount of data.
  • the processing capability is characterized by at least one of a CPU, a memory, and a disk performance parameter.
  • the amount of data of the fragmentation node is acquired, and the processing capability required to process the fragmentation node is obtained based on the amount of data.
  • Calculating the disk size required by the segmentation node based on the number of bits of the obtained data and the preset search characteristic expansion coefficient and the combined index coefficient. After calculating the required disk size, the disk size can be obtained according to the disk size.
  • CPU Central Processing Unit
  • a search feature is set in advance.
  • the expansion coefficient, the search characteristic expansion coefficient can be set empirically.
  • the search expansion coefficients of disk and memory are set to 1.17 and 0.85, respectively.
  • the combined index coefficients in vsearch are greater than about 2.
  • Step 206 Select, from the machine pool, a target virtual machine whose performance matches the processing capability for the fragment node.
  • the target virtual machine whose performance matches the processing capability is selected from the machine pool according to the processing capability required to process the fragmentation node.
  • the processing capability includes processing points.
  • the target virtual machine matching the fragment node is obtained from the machine pool.
  • the figure includes M physical machines, and M physical machines virtualize N virtual machines, where M ⁇ N.
  • the physical machine 1 includes a virtual machine 1 and a virtual machine 2, the physical machine 2 includes a virtual machine 3, the physical machine 3 includes a virtual machine 4, and the physical machine M includes a virtual machine N.
  • the fragment node 1 is deployed on the virtual machine 1, and the fragment node 2 is deployed on the virtual machine 2.
  • the fragment node N is deployed on the virtual machine N.
  • Step 207 Select, from the machine pool, a first virtual machine whose performance matches the processing capability for the backup data.
  • the fragment node in order to improve the security of the data and ensure the continuity of the service, the fragment node is backed up, and the target virtual machine is allocated to the fragment node, and the backup data of the fragment node needs to be allocated virtual. machine. Since the backup data is completely consistent with the data in the fragmentation node, when the virtual machine is allocated for the backup data, the first virtual machine whose performance and processing capability are matched is selected as the backup from the machine pool according to the processing capability required by the fragmentation node. The virtual machine corresponding to the data.
  • Step 208 Determine whether the target virtual machine and the first virtual machine belong to the same physical machine.
  • the target virtual machine and the first virtual machine belong to the same physical machine, the data of the fragment node can no longer be served when the physical machine fails and cannot operate normally.
  • step 209 If the result of the determination is yes, go to step 209; otherwise, go to step 210.
  • Step 209 Re-select the first virtual machine that matches the processing capability from the pool of machines for backup data. Until the target virtual machine and the first virtual machine do not belong to the same physical machine.
  • the first virtual machine is reselected for the backup data. Until the selected first virtual machine and the target virtual machine do not belong to the same physical machine. After obtaining the first virtual machine that does not belong to the same physical machine as the target virtual machine, step 210 is performed.
  • Step 210 Deploy the fragment node and the backup data to the target virtual machine and the first virtual machine, respectively.
  • the figure divides 2 billion pieces of data into N pieces, and each piece corresponds to one piece node.
  • Each of the fragment nodes includes a plurality of backup data, for example, the fragment node 1 includes data 1, backup data 1 and backup data 2, the fragment node 2 includes data 1 and backup data 1, and the fragment node 3 includes data 1 and data. Backup 1...
  • the fragment node N includes data 1, backup data 1 to backup data 3.
  • the figure includes M physical machines, and M physical machines virtualize N virtual machines, where M ⁇ N.
  • the physical machine 1 includes a virtual machine 1 and a virtual machine 2
  • the physical machine 2 includes a virtual machine 3
  • the physical machine M includes a virtual machine N.
  • the target virtual machine matching the main data is acquired from the machine pool.
  • the data 1 is deployed on the virtual machine 1 and the data 2 is deployed on the virtual machine 2.
  • the data N is deployed on the virtual machine N.
  • the backup data 1 of the fragment node 1 is deployed on the virtual machine 3
  • the backup data 1 of the fragment node 2 is deployed on the virtual machine 4
  • the backup data 1 of the fragment node 3 is deployed in the virtual machine.
  • the backup data 1 of the fragment node N is deployed on the virtual machine K. Where K is less than N.
  • the backup data 2 of the fragment node 1 is deployed on the virtual machine 4, ... the backup data 1 of the fragment node N is deployed on the virtual machine 1, where I is smaller than N.
  • the backup data 3 of the fragment node N is deployed on the virtual machine 1, where I is less than N.
  • the number of backup data cannot be a condition limiting the present invention, and the number of backup data is not limited.
  • the fragmentation node is obtained by the fragmentation of the data to be processed, the data volume corresponding to the fragmentation node is calculated, and the target virtual machine is allocated to the fragmentation node according to the data volume of the fragmentation node, and Backing up the fragment node and allocating the first virtual machine to the backup data.
  • the target virtual machine and the first virtual machine do not belong to the same physical machine, the fragment node and the backup data are respectively deployed to the target virtual machine and the first virtual machine. on board.
  • the data volume of the fragment node is controlled by data breaking, and the virtual machine matching the data amount is allocated, thereby avoiding the phenomenon that the virtual machine is overloaded. Good load balanced.
  • FIG. 7 it is a schematic structural diagram of a virtual machine deployment apparatus according to Embodiment 3 of the present invention.
  • the virtual machine deployment apparatus includes: a dispersing module 11 , an allocation module 12 , and a deployment module 13 .
  • the breaking module 11 is configured to perform fragmentation and fragmentation to obtain a fragmentation node.
  • the breaking module 11 discards the data to be processed according to the budget breaking algorithm, and obtains the fragment node.
  • the fragment node is a fragmented fragment data.
  • the module 11 is firstly dispersed, and the number of pieces to be broken is set according to the data size of the data to be processed. After the number of pieces is set, the data to be processed is broken and sliced based on the breaking algorithm.
  • the allocating module 12 is configured to allocate a target virtual machine to the fragmentation node according to the data volume of the fragmentation node.
  • the amount of data obtained by each fragment node is different. After the fragment nodes are acquired, the amount of data of each fragment node is calculated. After the data volume of the fragmentation node is obtained, the allocation module 12 allocates a target virtual machine to the fragmentation node.
  • the target virtual machine needs to have the processing capability of processing the data volume included in the fragmentation node, wherein the processing capability It can be characterized by performance parameters such as CPU, memory and disk.
  • the deployment module 13 is configured to deploy the fragmentation node to the target virtual machine.
  • the virtual machine deployment apparatus obtains a fragmentation node by dispersing and fragmenting the data to be processed, assigns a target virtual machine to the fragmentation node according to the data volume of the fragmentation node, and deploys the fragmentation node to the target virtual machine. .
  • the data volume of the fragment node is controlled by data breaking, and the virtual machine matching the data amount is allocated, thereby avoiding the phenomenon that the virtual machine is overloaded. Good load balancing is achieved.
  • FIG. 8 it is a schematic structural diagram of a virtual machine deployment apparatus according to Embodiment 4 of the present invention.
  • the virtual machine deployment apparatus includes: an input module 20, a breaking module 21, a computing module 22, an allocation module 23, and a deployment module 24. Backup module 25 and determination module 26.
  • the input module 20 is configured to input performance parameters of all the virtual machines into the machine pool before the scatter module performs the scatter fragmentation to obtain the shard node.
  • the breaking module 21 is configured to perform a fragmentation to obtain a fragmentation node for the data to be processed.
  • calculation module 22 is configured to calculate the amount of data corresponding to the segmentation node.
  • the calculation module 22 is specifically configured to obtain the number of entries included in the fragmentation node, and obtain the number of bits occupied by one piece of data in the fragmentation node, and calculate the amount of data according to the number of entries and the number of bits occupied by one piece of data. .
  • the allocating module 23 is configured to allocate a target virtual machine to the fragmentation node according to the data amount of the fragmentation node.
  • An optional configuration manner of the distribution module 23 is provided in this embodiment, including: an obtaining unit 231 and a selecting unit 232.
  • the obtaining unit 231 is configured to acquire processing capability required for processing the fragmentation node according to the amount of data; wherein the processing capability is characterized by a CPU, a memory, and a disk performance parameter.
  • the obtaining unit 231 is specifically configured to calculate, according to the preset search characteristic expansion coefficient, the combined search coefficient, and the data amount, the processing capability required to acquire the fragmentation node.
  • the search characteristic expansion coefficient is multiplied by the combined search coefficient and the data amount to obtain the size of the disk required to process the fragment node, and the corresponding memory and CPU are selected according to the size of the disk.
  • the selecting unit 232 is configured to select, from the machine pool, a target virtual machine whose performance and processing capability are matched for the fragment node.
  • the selecting unit 232 is specifically configured to preferentially select the target virtual machine from the low-matching machine pool to the high-matching machine pool to achieve the optimal configuration.
  • the deployment module 24 is configured to deploy the fragmentation node to the target virtual machine.
  • the backup module 25 is configured to perform backup on the fragment node before the calculation module 22 calculates the data amount corresponding to the fragment node, and obtain backup data of the fragment node.
  • the allocating module 23 is further configured to select, from the machine pool, a first virtual machine whose performance and processing capability are matched for the backup data before the deployment module 24 deploys the fragmentation node to the target virtual machine.
  • the determining module 26 is configured to determine whether the target virtual machine and the first virtual machine belong to the same physical machine.
  • the allocating module 23 is further configured to: when the determining module 26 determines that the target virtual machine and the first virtual machine belong to the same physical machine, re-select the first virtual machine that matches the processing capability from the machine pool for the backup data. Until the target virtual machine and the first virtual machine do not belong to the same physical machine.
  • the deployment module 24 is further configured to deploy the backup data to the first virtual machine when the determining module determines that the target virtual machine and the first virtual machine do not belong to the same physical machine.
  • the virtual machine deployment apparatus obtains a fragmentation node by despitting fragmentation of the data to be processed, calculates a data volume corresponding to the fragmentation node, and allocates a target virtual machine to the fragmentation node according to the data volume of the fragmentation node, and Backing up the fragment node and allocating the first virtual machine to the backup data.
  • the target virtual machine and the first virtual machine do not belong to the same physical machine
  • the fragment node and the backup data are respectively deployed to the target virtual machine and the first virtual machine. on board.
  • the data volume of the fragment node is controlled by data breaking, and the virtual machine matching the data amount is allocated, thereby avoiding the phenomenon that the virtual machine is overloaded. Good load balancing is achieved.
  • the aforementioned program can be stored in a computer readable storage medium.
  • the program when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明提供了一种虚拟机部署方法及装置，通过对待处理数据进行打散分片获取分片节点，依据分片节点的数据量为分片节点分配目标虚拟机，将分片节点部署到目标虚拟机上。本实施例中分片节点分配虚拟机时先通过数据打散的方式来控制分片节点的数据量，再按照实际的数据量为与该数据量相匹配的虚拟机，从而可以避免虚拟机出现过载的现象，更好地实现了负载均衡。

Description

虚拟机部署方法及装置
本申请要求2016年02月03日递交的申请号为201610077964.5、发明名称为“虚拟机部署方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明属于计算机技术领域，尤其涉及一种虚拟机部署方法及装置。
背景技术
在互联网高速发展的今天，云计算采用创新的计算模式使用户通过互联网随时获得近乎无限的计算能力和丰富多样的信息服务。云计算融合了以虚拟化技术，借助虚拟化技术的伸缩性和灵活性，提高了资源利用率，节省了成本。具体地，一台实体的物理机共计32核，可以分配出30个虚拟机(Virtual Machine，简称VM)，每个VM需要4个核，此时该物理机虚拟出30个虚拟机，理论上需要120个核，但是实际上物理机只有32核。实际应用中所有业务不会同时处于满负荷状态，为了节省成本，提高资源利用率，云计算中共用处于低负荷状态的核，以完成单台物理机虚拟出30个虚拟机的目的。
通过上述虚拟方式进行虚拟化，大量数据可能会出现共用一个虚拟机的状况，就导致该虚拟机的负载压力过大的问题。例如在淘宝应用中旗舰店的商品数量往往比普通店铺的商品数量高出很多，而且搜索热度也会比普通店铺高，当基于上述虚拟方式进行虚拟化时，有可能出现多个旗舰店的商品数据部署到同一个虚拟机上的情况，在买家基于卖家进行搜索时，就会出现所处虚拟机负载压力大的问题。
发明内容
本发明提供一种虚拟机部署方法及装置，用于解决现有虚拟化方式在大量数据部署到同一虚拟机上时会导致虚拟机负载压力过大的问题。
为了实现上述目的，本发明提供了一种虚拟机部署方法，包括：
对待处理数据进行打散分片获取分片节点；
依据所述分片节点的数据量为所述分片节点分配目标虚拟机；
将所述分片节点部署到所述目标虚拟机上。
为了实现上述目的，本发明提供了一种虚拟机部署装置，包括：
打散模块，用于对待处理数据进行打散分片获取分片节点；
分配模块，用于依据所述分片节点的数据量为所述分片节点分配目标虚拟机；
部署模块，用于将所述分片节点部署到所述目标虚拟机上。
本发明的虚拟机部署方法及装置，通过对待处理数据进行打散分片获取分片节点，依据分片节点的数据量为分片节点分配目标虚拟机，将分片节点部署到目标虚拟机上。本实施例中分片节点分配虚拟机时先通过数据打散的方式来控制分片节点的数据量，再分配与该数据量相匹配的虚拟机，从而可以避免虚拟机出现过载的现象，更好实现了负载均衡。
附图说明
图1为本发明实施例一的虚拟机部署方法的流程示意图；
图2为本发明实施例二的虚拟机部署方法的流程示意图；
图3为本实施例二中提供的一致性Hash算法打散数据的示意图；
图4为本实施例二中提供的数据打散后的结果示意图；
图5为本实施例二中提供的虚拟机部署方法的应用示意图之一；
图6为本实施例二中提供的虚拟机部署方法的应用示意图之二；
图7为本发明实施例三的虚拟机部署装置的结构示意图；
图8为本发明实施例四的虚拟机部署装置的结构示意图。
具体实施方式
下面结合附图对本发明实施例提供的虚拟机部署方法及装置进行详细描述。
实施例一
如图1所示，其为本发明实施例一的虚拟机部署方法的流程示意图，该虚拟机部署方法包括：
步骤101、对待处理数据进行打散分片获取分片节点。
实际应用中，为了对数据进行存储，需要对大数据进行打散分片，将整体的大数据进行分摊存储，这样每个存储数据的设备的数据量相对小一些，降低存储设备的压力。
本实施例中，对待处理数据按照预设的打散算法进行打散分片，获取到分片节点，其实分片节点就是一个打散后的分片数据。具体地，首先按照待处理数据的数据量大小设定打散的片数，设定完片数后，基于打散算法将待处理数据进行打散分片。
例如，待处理数据为20亿条数据，可以打散成256片，也就是将20亿条数据打散 成256个分片节点。在淘宝业务的场景下，可以采用Hash算法对商品数据打散分片，基于打散获取到的分片节点存储商品数据。
步骤102、依据所述分片节点的数据量为所述分片节点分配目标虚拟机。
获取到各分片节点的数据量是不同的，在获取到分片节点后，计算每个分片节点的数据量。
在获取到分片节点的数据量之后，为分片节点分配目标虚拟机，本实施中，该目标虚拟机需要具备处理分片节点所包含的数据量的处理能力，其中，处理能力可以用CPU、内存和磁盘等性能参数中的至少一个进行表征。
步骤103、将所述分片节点部署到所述目标虚拟机上。
为分片节点选取出目标虚拟机后，将分片节点部署到目标虚拟机上。
本实施例提供的虚拟机部署方法，通过对待处理数据进行打散分片获取分片节点，计算分片节点对应的数据量，依据分片节点的数据量为分片节点分配目标虚拟机，将分片节点部署到目标虚拟机上。本实施例中分片节点分配虚拟机时先通过数据打散的方式来控制分片节点的数据量，再分配与该数据量相匹配的虚拟机，从而可以避免虚拟机出现过载的现象，更好地实现了负载均衡。
实施例二
如图2所示，其为本发明实施例二的虚拟机部署方法的流程示意图，该虚拟机部署方法包括：
步骤201、将所有虚拟机的性能参数输入到所述机器池中。
为便于对虚拟机进行分配，本实施例中将所有VM的性能参数输入到机器池中，VM的性能参数包括VM的内存大小、磁盘大小、核数等。
步骤202、对待处理数据进行打散分片获取分片节点。
步骤203、获取分片节点的备份数据。
具体地，采用一致性哈希(Hash)算法来对待处理数据进行打散分片，首先按照待处理数据的数据量大小设定打散的片数，设定完片数后，基于一致性Hash算法进行打散分片。为了保证数据的安全性，可以对分片节点进行备份，获取该分片节点的备份数据。
如图3所示，其为本实施例二中提供的一致性Hash算法打散数据的示意图。一致性Hash算法具体地将Value映射到一个32位的键(key)值，也即是0～2^32-1次方的数值空间，将这个空间想象成一个首(0)尾(2^32-1)相接的圆环。再将数据映射到具体的节点(node)上，如key％N，key是数据的key，N是节点数。图3中所示， 环上的大圆圈代表一个节点，环上的小圆圈代表节点数据，每个节点下面设置有主数据和备份数据。
例如，有10亿条数据需要打散，通过一致性Hash算法可以将数据打散成如图4所示的结果。图中将10亿条数据打散成4片，每个分片对应一个分片节点。为了提高数据的安全性，以及保证服务的延续性，对数据进行备份，每个分片节点下面设置有多份数据，而且分片节点下的多份数据之间为平等关系，也就是说，多份数据之间互为主备份数据。例如，分片节点1下包括数据1～数据4，其中数据1、数据2、数据3以及数据4互为主备份数据。分片节点2下包括数据1～数据2，其中数据1与数据2互为主备份数据。
步骤204、计算分片节点对应的数据量。
具体地，获取分片节点中一条数据所包含的字段个数、每个字段对应的字段类型以及一条数据所包含的表数目，以及分片节点所包含的词条数，该词条数用于指示出分片节点包含多少条数据。进一步地，根据一条数据所包含的字段个数、每个字段对应的字段类型以及所包含的表数目能够计算出一条数据占用的比特数。实际中，当获取到字段类型后，就可以得到该字段类型所占用的比特数，例如，整型数据int占用4比特(byte)；长整型数据(long)占用4byte；双整型数据(double)占用8byte，字符串(char)占用2byte等。进一步地，根据词条数和分片节点中一条数据所占用的比特数，计算得到分片节点的数据量。具体地，将词条数与一条数据所占用的比特数相乘，得到分片节点的数据量。
步骤205、根据数据量获取处理分片节点所需的处理能力。
其中，所述处理能力用CPU、内存和磁盘性能参数中的至少一个进行表征。
在搜索引擎的应用场景下，获取分片节点的数据量，基于数据量获取处理分片节点所需的处理能力。基于获取的一条数据的比特数与预设的搜索特性膨胀系数、合并索引系数，计算该分片节点所需的磁盘大小，在计算出所需的磁盘大小后，根据该磁盘大小可以获取到分片节点所需的内存大小以及所需中央处理器(Central Processing Unit，简称CPU)的性能。
实际应用中，在构建索引时索引级数越多，搜索时所需要的时间就会越长，说明搜索难度就会越高，相当于搜索进行了膨胀，本实施例中，预先设置一个搜索特性膨胀系数，该搜索特性膨胀系数可以根据经验进行设定。在使用Docvalue搜索方式时，磁盘和内存的搜索膨胀系数分别设置为1.17和0.85。在vsearch中合并索引系数约大于2。
步骤206、从机器池中为分片节点选取性能与所述处理能力匹配的目标虚拟机。
具体地，根据处理分片节点所需的处理能力，从机器池中选取性能与该处理能力匹配的目标虚拟机。可选地，根据分片节点所需的处理能力中磁盘和内存与预设系数，计算出目标虚拟机对应的磁盘和内存的大小，例如，将处理能力中磁盘和内存分别与该预设的系数做乘法得到目标虚拟机的磁盘和内存的大小。例如，分片节点所需的内存大小＝目标虚拟机的内存*75％，分片节点所需的磁盘大小＝目标虚拟机的磁盘*75％。从机器池中选取目标虚拟机的过程中，可以优先从低配机器池中选择，逐步往高配机器池中选取，以达到使用最优配置。
下面举例进行说明：如图5所示，图中将20亿条数据打散成N片，每个分片对应一个分片节点。当数据未进行备份时，计算出每分片节点对应的数据量，然后根据获取到数据量能够得到处理分片节点所需的处理能力，其中，如图5中所示，处理能力包括处理分片节点所需的磁盘大小、所需的内存大小以及所需CPU性能等。然后根据该所需的处理能力，从机器池中获取到与分片节点匹配的目标虚拟机。图中包括M个物理机，M个物理机虚拟出N个虚拟机，其中M＜N。物理机1包括虚拟机1和虚拟机2，物理机2包括虚拟机3，物理机3包括虚拟机4……物理机M包括虚拟机N。其中，分片节点1部署在虚拟机1上，分片节点2部署在虚拟机2上……分片节点N部署在虚拟机N上。
步骤207、从机器池中为备份数据选取性能与所述处理能力匹配的第一虚拟机。
本实施例中，为了提高数据的安全性，以及保证服务的延续性，为分片节点进行了备份，在为分片节点分配目标虚拟机的同时，还需要为分片节点的备份数据分配虚拟机。由于备份数据与分片节点中的数据完全一致，在为备份数据分配虚拟机时，也从机器池中按照分片节点所需的处理能力，选取性能与处理能力匹配的第一虚拟机作为备份数据对应的虚拟机。
步骤208、判断目标虚拟机与第一虚拟机是否属于同一物理机。
在分配虚拟机时，如果目标虚拟机和第一虚拟机属于同一物理机时，在物理机出现故障无法正常运行时，该分片节点的数据就不能再提供服务。为了避免主备份数据被分配属于同一物理机的虚拟机上，本实施例中在获取到目标虚拟机和第一虚拟机后，进一步判断该目标虚拟机和第一虚拟机是否属于同一物理机。
如果判断结果为是，执行步骤209；否则，执行步骤210。
步骤209、重新为备份数据从所述机器池中选取与所述处理能力匹配的第一虚拟机 直到目标虚拟机与第一虚拟机不属于同一物理机为止。
如果目标虚拟机和第一虚拟机属于同一个物理机，为了保证当目标虚拟机所在的物理机出现故障后，备份数据可以继续提供服务，本实施例中为备份数据重新选取第一虚拟机，直到选取出的第一虚拟机和目标虚拟机不属于同一物理机上为止。在获取到与目标虚拟机不属于同一物理机的第一虚拟机后，执行步骤210。
步骤210、将分片节点以及备份数据分别部署到目标虚拟机和第一虚拟机上。
下面举例进行说明：如图6所示，图中将20亿条数据打散成N片，每个分片对应一个分片节点。每个分片节点下包括多个备份数据，例如分片节点1包括数据1、备份数据1和备份数据2，分片节点2包括数据1和备份数据1，分片节点3包括数据1和数据备份1……分片节点N包括数据1、备份数据1～备份数据3。图中包括M个物理机，M个物理机虚拟出N个虚拟机，其中M＜N。物理机1包括虚拟机1和虚拟机2，物理机2包括虚拟机3，物理机3包括虚拟机4……物理机M包括虚拟机N。
在对主数据即数据1～数据N部署虚拟机时，从机器池中获取到与各主数据匹配的目标虚拟机。其中，数据1部署在虚拟机1上，数据2部署在虚拟机2上……数据N部署在虚拟机N上。
在对备份数据部署虚拟机时，分片节点1的备份数据1部署在虚拟机3上，分片节点2的备份数据1部署在虚拟机4上，分片节点3的备份数据1部署在虚拟机1上……分片节点N的备份数据1部署在虚拟机K上。其中，K小于N。分片节点1的备份数据2部署在虚拟机4上，……分片节点N的备份数据1部署在虚拟机I上，其中，I小于N。分片节点N的备份数据3部署在虚拟机I上，其中，I小于N。
此时，在为备份数据部署虚拟机时需要考虑备份数据不能部署在与目标虚拟机处于同一物理机上的虚拟机上。
在该示例中，备份数据的数量不能成为限制本发明的条件，备份数据的数量并不进行限定。
本实施例提供的虚拟机部署方法，通过对待处理数据进行打散分片获取分片节点，计算分片节点对应的数据量，依据分片节点的数据量为分片节点分配目标虚拟机，并且为分片节点进行备份，并为备份数据分配第一虚拟机，当目标虚拟机与第一虚拟机不属于同一物理机时，将分片节点和备份数据分别部署到目标虚拟机和第一虚拟机上。本实施例中分片节点分配虚拟机时先通过数据打散的方式来控制分片节点的数据量，再分配与该数据量相匹配的虚拟机，从而可以避免虚拟机出现过载的现象，更好地实现了负载 均衡。
实施例三
如图7所示，其为本发明实施例三的虚拟机部署装置的结构示意图，该虚拟机部署装置包括：打散模块11、分配模块12和部署模块13。
其中，打散模块11，用于对待处理数据进行打散分片获取分片节点。
本实施例中，打散模块11，对待处理数据按照预算的打散算法进行打散分片，获取到分片节点，其实分片节点就是一个打散后的分片数据。具体地，首先打散模块11，按照待处理数据的数据量大小设定打散的片数，设定完片数后，基于打散算法将待处理数据进行打散分片。
分配模块12，用于依据分片节点的数据量为分片节点分配目标虚拟机。
获取到各分片节点的数据量是不同的，在获取到分片节点后，计算每个分片节点的数据量。在获取到分片节点的数据量之后，分配模块12为分片节点分配目标虚拟机，本实施中，该目标虚拟机需要具备处理分片节点所包含的数据量得处理能力，其中，处理能力可以用CPU、内存和磁盘等性能参数进行表征。
部署模块13，用于将分片节点部署到目标虚拟机上。
本实施例提供的虚拟机部署装置，通过对待处理数据进行打散分片获取分片节点，依据分片节点的数据量为分片节点分配目标虚拟机，将分片节点部署到目标虚拟机上。本实施例中分片节点分配虚拟机时先通过数据打散的方式来控制分片节点的数据量，再分配与该数据量相匹配的虚拟机，从而可以避免虚拟机出现过载的现象，更好地实现了负载均衡。
实施例四
如图8所示，其为本发明实施例四的虚拟机部署装置的结构示意图，该虚拟机部署装置包括：输入模块20、打散模块21、计算模块22、分配模块23、部署模块24、备份模块25和判断模块26。
其中，输入模块20，用于在打散模块对待处理数据进行打散分片获取分片节点之前，将所有虚拟机的性能参数输入到机器池中。
进一步地，打散模块21，用于对待处理数据进行打散分片获取分片节点。
进一步地，计算模块22，用于计算分片节点对应的数据量。
其中，计算模块22，具体用于获取分片节点所包含的词条数，以及获取分片节点中一条数据所占用的比特数，根据词条数和一条数据所占用的比特数计算得到数据量。
进一步地，分配模块23，用于依据分片节点的数据量为分片节点分配目标虚拟机。
本实施例中提供了一种分配模块23可选的结构方式，包括：获取单元231和选取单元232。
获取单元231，用于根据数据量获取处理分片节点所需的处理能力；其中，处理能力用CPU、内存和磁盘性能参数进行表征。
其中，获取单元231，具体用于根据预设的搜索特性膨胀系数、合并搜索系数以及所述数据量计算获取处分片节点所需的处理能力。将搜索特性膨胀系数与合并搜索系数以及数据量相乘，得到处理该分片节点所需的磁盘的大小，根据磁盘的大小选取相应的内存和CPU。
选取单元232，用于从机器池中为分片节点选取性能与处理能力匹配的目标虚拟机。
进一步地，选取单元232具体用于优先从低配机器池逐步往高配机器池中选取所述目标虚拟机，以达到使用最优配置。
进一步地，部署模块24，用于将分片节点部署到目标虚拟机上。
进一步地，备份模块25，用于在计算模块22计算分片节点对应的数据量之前，对分片节点进行备份，获取分片节点的备份数据。
进一步地，分配模块23，还用于在部署模块24将分片节点部署到目标虚拟机上之前，从机器池中为备份数据选取性能与处理能力匹配的第一虚拟机。
进一步地，判断模块26，用于判断目标虚拟机与第一虚拟机是否属于同一物理机。
进一步地，分配模块23，还用于在判断模块26判断出目标虚拟机与第一虚拟机属于同一物理机时，重新为备份数据从机器池中选取与所述处理能力匹配的第一虚拟机直到目标虚拟机与第一虚拟机不属于同一物理机为止。
进一步地，部署模块24，还用于在判断模块判断出目标虚拟机与第一虚拟机不属于同一物理机时，将备份数据部署到第一虚拟机上。
本实施例提供的虚拟机部署装置，通过对待处理数据进行打散分片获取分片节点，计算分片节点对应的数据量，依据分片节点的数据量为分片节点分配目标虚拟机，并且为分片节点进行备份，并为备份数据分配第一虚拟机，当目标虚拟机与第一虚拟机不属于同一物理机时，将分片节点和备份数据分别部署到目标虚拟机和第一虚拟机上。本实施例中分片节点分配虚拟机时先通过数据打散的方式来控制分片节点的数据量，再分配与该数据量相匹配的虚拟机，从而可以避免虚拟机出现过载的现象，更好地实现了负载均衡。
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (18)

  1. 一种虚拟机部署方法，其特征在于，包括：
    对待处理数据进行打散分片获取分片节点；
    依据所述分片节点的数据量为所述分片节点分配目标虚拟机；
    将所述分片节点部署到所述目标虚拟机上。
  2. 根据权利要求1所述的虚拟机部署方法，其特征在于，所述依据分片节点的数据量为所述分片节点分配虚拟机，包括：
    根据所述数据量获取处理所述分片节点所需的处理能力；其中，所述处理能力用CPU、内存和磁盘性能参数中的至少一个进行表征；
    从机器池中为所述分片节点选取性能与所述处理能力匹配的目标虚拟机。
  3. 根据权利要求2所述的虚拟机部署方法，其特征在于，所述从机器池中为所述分片节点选取性能与所述处理能力匹配的目标虚拟机，包括：
    优先从低配机器池逐步往高配机器池中选取所述目标虚拟机，以达到使用最优配置。
  4. 根据权利要求3所述的虚拟机部署方法，其特征在于，所述计算所述分片节点对应的数据量之前，还包括：
    获取所述分片节点的备份数据。
  5. 根据权利要求4所述的虚拟机部署方法，其特征在于，所述将所述分片节点存储到所述目标虚拟机上之前，还包括：
    从所述机器池中为所述备份数据选取性能与所述处理能力匹配的第一虚拟机；
    判断所述目标虚拟机与所述第一虚拟机是否属于同一物理机；
    如果判断结果为是，重新为所述备份数据从所述机器池中选取与所述处理能力匹配的所述第一虚拟机直到所述目标虚拟机与所述第一虚拟机不属于同一物理机为止；
    如果判断结果为否，将所述备份数据部署到所述第一虚拟机上。
  6. 根据权利要求5所述的虚拟机部署方法，其特征在于，所述依据所述分片节点的数据量为所述分片节点分配目标虚拟机之前，还包括：
    计算所述分片节点对应的数据量。
  7. 根据权利要求6所述的虚拟机部署方法，其特征在于，所述计算所述分片节点对应的数据量，包括：
    获取所述分片节点所包含的词条数；
    获取所述分片节点中一条数据所占用的比特数；
    根据所述词条数和一条数据所占用的比特数计算得到所述数据量。
  8. 根据权利要求7所述的虚拟机部署方法，其特征在于，所述根据所述数据量获取处理所述分片节点所需的处理能力，包括：
    根据预设的搜索特性膨胀系数、合并搜索系数以及所述数据量计算获取所述处理能力。
  9. 根据权利要求1-8任一项所述的虚拟机部署方法，其特征在于，所述对待处理数据进行打散分片获取分片节点之前，还包括：
    将所有虚拟机的性能参数输入到机器池中。
  10. 一种虚拟机部署装置，其特征在于，包括：
    打散模块，用于对待处理数据进行打散分片获取分片节点；
    分配模块，用于依据所述分片节点的数据量为所述分片节点分配目标虚拟机；
    部署模块，用于将所述分片节点部署到所述目标虚拟机上。
  11. 根据权利要求10所述的虚拟机部署装置，其特征在于，所述分配模块包括：获取单元和选取单元；
    所述获取单元，用于根据所述数据量获取处理所述分片节点所需的处理能力；其中，所述处理能力用CPU、内存和磁盘性能参数中的至少一个进行表征；
    所述选取单元，用于从机器池中为所述分片节点选取性能与所述处理能力匹配的目标虚拟机。
  12. 根据权利要求11所述的虚拟机部署装置，其特征在于，所述选取单元，具体用于优先从低配机器池逐步往高配机器池中选取所述目标虚拟机，以达到使用最优配置。
  13. 根据权利要求12所述的虚拟机部署装置，其特征在于，还包括：备份模块，用于在所述计算模块计算所述分片节点对应的数据量之前，获取所述分片节点的备份数据。
  14. 根据权利要求13所述的虚拟机部署装置，其特征在于，还包括：判断模块；
    所述分配模块，还用于在所述部署模块将所述分片节点部署到所述目标虚拟机上之前，从所述机器池中为所述备份数据选取性能与所述处理能力匹配的第一虚拟机，以及在所述判断模块判断出所述目标虚拟机与所述第一虚拟机属于同一物理机时，重新为所述备份数据从所述机器池中选取与所述处理能力匹配的所述第一虚拟机直到所述目标 虚拟机与所述第一虚拟机不属于同一物理机为止；
    所述判断模块，用于判断所述目标虚拟机与所述第一虚拟机是否属于同一物理机；
    所述部署模块，还用于在所述判断模块判断出所述目标虚拟机与所述第一虚拟机不属于同一物理机时，将所述备份数据部署到所述第一虚拟机上。
  15. 根据权利要求14所述的虚拟机部署装置，其特征在于，还包括：计算模块，用于计算所述分片节点对应的数据量。
  16. 根据权利要求15所述的虚拟机部署装置，其特征在于，所述计算模块，具体用于获取所述分片节点所包含的词条数，以及获取所述分片节点中一条数据所占用的比特数，根据所述词条数和一条数据所占用的比特数计算得到所述数据量。
  17. 根据权利要求16所述的虚拟机部署装置，其特征在于，所述获取单元，具体用于根据预设的搜索特性膨胀系数、合并搜索系数以及所述数据量计算获取所述处理能力。
  18. 根据权利要求10-17任一项所述的虚拟机部署装置，其特征在于，还包括：
    输入模块，用于在所述打散模块对待处理数据进行打散分片获取分片节点之前，将所有虚拟机的性能参数输入到机器池中。
PCT/CN2017/071831 2016-02-03 2017-01-20 虚拟机部署方法及装置 Ceased WO2017133484A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP17746804.8A EP3413193A4 (en) 2016-02-03 2017-01-20 METHOD AND DEVICE FOR USING A VIRTUAL MACHINE
JP2018539345A JP2019504420A (ja) 2016-02-03 2017-01-20 仮想マシン展開の方法および装置
US16/054,858 US10740194B2 (en) 2016-02-03 2018-08-03 Virtual machine deployment method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610077964.5A CN107038059A (zh) 2016-02-03 2016-02-03 虚拟机部署方法及装置
CN201610077964.5 2016-02-03

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/054,858 Continuation US10740194B2 (en) 2016-02-03 2018-08-03 Virtual machine deployment method and apparatus

Publications (1)

Publication Number Publication Date
WO2017133484A1 true WO2017133484A1 (zh) 2017-08-10

Family

ID=59499288

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/071831 Ceased WO2017133484A1 (zh) 2016-02-03 2017-01-20 虚拟机部署方法及装置

Country Status (6)

Country Link
US (1) US10740194B2 (zh)
EP (1) EP3413193A4 (zh)
JP (1) JP2019504420A (zh)
CN (1) CN107038059A (zh)
TW (1) TW201732628A (zh)
WO (1) WO2017133484A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110716698A (zh) * 2019-09-30 2020-01-21 新华三大数据技术有限公司 数据分片副本部署方法及装置
US10740194B2 (en) 2016-02-03 2020-08-11 Alibaba Group Holding Limited Virtual machine deployment method and apparatus

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109741179B (zh) * 2018-12-25 2022-07-08 北京字节跳动网络技术有限公司 全档位行情数据实时计算方法及装置
CN110505276B (zh) * 2019-07-17 2022-05-06 北京三快在线科技有限公司 对象匹配方法、装置及系统、电子设备及存储介质
CN114443057B (zh) * 2022-01-25 2023-03-24 北京百度网讯科技有限公司 对话模型的部署和对话方法、装置、电子设备及存储介质
CN115996228B (zh) * 2023-03-22 2023-05-30 睿至科技集团有限公司 一种基于物联网的能源数据的处理方法及其系统
CN116028930B (zh) * 2023-03-28 2023-08-01 睿至科技集团有限公司 一种物联网中能源数据的防御检测方法及其系统
CN116185571B (zh) * 2023-04-20 2023-07-21 深圳市科力锐科技有限公司 数据迁移方法、设备、存储介质及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080222633A1 (en) * 2007-03-08 2008-09-11 Nec Corporation Virtual machine configuration system and method thereof
CN102981890A (zh) * 2012-11-30 2013-03-20 华南理工大学 一种在虚拟化数据中心内的计算任务及虚拟机部署方法
CN102981910A (zh) * 2012-11-02 2013-03-20 曙光云计算技术有限公司 虚拟机调度的实现方法和装置
CN103970612A (zh) * 2014-05-07 2014-08-06 田文洪 一种基于虚拟机预分割的负载均衡方法和装置

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120054739A1 (en) * 2010-08-27 2012-03-01 Hitachi, Ltd. Method and apparatus for deployment of storage functions on computers having virtual machines
JP5637791B2 (ja) * 2010-09-30 2014-12-10 株式会社日立製作所 計算機システム及びそのデータ処理方法
CN102479217B (zh) * 2010-11-23 2015-07-15 腾讯科技（深圳）有限公司 一种分布式数据仓库中实现计算均衡的方法及装置
US8548790B2 (en) * 2011-01-07 2013-10-01 International Business Machines Corporation Rapidly determining fragmentation in computing environments
CN102662760A (zh) * 2012-03-22 2012-09-12 北京邮电大学 一种基于虚拟机性能向量的任务调度方法
CN103379114B (zh) * 2012-04-28 2016-12-14 国际商业机器公司 用于在MapReduce系统中保护隐私数据的方法和装置
US8972986B2 (en) * 2012-05-25 2015-03-03 International Business Machines Corporation Locality-aware resource allocation for cloud computing
CN103810058B (zh) * 2012-11-12 2017-02-22 华为技术有限公司 虚拟机备份方法、设备及系统
CN103036949B (zh) * 2012-11-27 2015-06-17 华中科技大学 一种虚拟化环境下Cassandra系统的优化方法和系统
US9898339B2 (en) * 2013-03-12 2018-02-20 Itron, Inc. Meter reading data validation
WO2014140697A1 (en) * 2013-03-15 2014-09-18 Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi A method for data partitioning and assignment
US9332071B2 (en) * 2013-05-06 2016-05-03 Microsoft Technology Licensing, Llc Data stage-in for network nodes
CN104516773B (zh) * 2013-09-29 2018-04-20 国际商业机器公司 用于物理机的数据分配方法和数据分配装置
US9489225B2 (en) * 2014-01-15 2016-11-08 Cisco Technology, Inc. Allocating resources for multi-phase, distributed computing jobs
US20150220442A1 (en) * 2014-02-04 2015-08-06 Bluedata Software, Inc. Prioritizing shared memory based on quality of service
US9959142B2 (en) * 2014-06-17 2018-05-01 Mediatek Inc. Dynamic task scheduling method for dispatching sub-tasks to computing devices of heterogeneous computing system and related computer readable medium
US10110911B2 (en) * 2014-11-11 2018-10-23 Cisco Technology, Inc. Parallel media encoding
US9946719B2 (en) * 2015-07-27 2018-04-17 Sas Institute Inc. Distributed data set encryption and decryption
EP3376372A4 (en) * 2015-12-31 2018-12-26 Huawei Technologies Co., Ltd. Data processing method, device, and system
JP2017130705A (ja) * 2016-01-18 2017-07-27 日本電気株式会社 データ管理システム、データ管理方法、及び、データ管理プログラム
US11087234B2 (en) * 2016-01-29 2021-08-10 Verizon Media Inc. Method and system for distributed deep machine learning
CN107038059A (zh) 2016-02-03 2017-08-11 阿里巴巴集团控股有限公司 虚拟机部署方法及装置
US10169027B2 (en) * 2016-05-05 2019-01-01 International Business Machines Corporation Upgrade of an operating system of a virtual machine

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080222633A1 (en) * 2007-03-08 2008-09-11 Nec Corporation Virtual machine configuration system and method thereof
CN102981910A (zh) * 2012-11-02 2013-03-20 曙光云计算技术有限公司 虚拟机调度的实现方法和装置
CN102981890A (zh) * 2012-11-30 2013-03-20 华南理工大学 一种在虚拟化数据中心内的计算任务及虚拟机部署方法
CN103970612A (zh) * 2014-05-07 2014-08-06 田文洪 一种基于虚拟机预分割的负载均衡方法和装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3413193A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10740194B2 (en) 2016-02-03 2020-08-11 Alibaba Group Holding Limited Virtual machine deployment method and apparatus
CN110716698A (zh) * 2019-09-30 2020-01-21 新华三大数据技术有限公司 数据分片副本部署方法及装置

Also Published As

Publication number Publication date
EP3413193A4 (en) 2019-02-13
EP3413193A1 (en) 2018-12-12
TW201732628A (zh) 2017-09-16
US20180341559A1 (en) 2018-11-29
US10740194B2 (en) 2020-08-11
JP2019504420A (ja) 2019-02-14
CN107038059A (zh) 2017-08-11

Similar Documents

Publication Publication Date Title
WO2017133484A1 (zh) 虚拟机部署方法及装置
US11029993B2 (en) System and method for a distributed key-value store
US20200019429A1 (en) Hot-plugging of virtual functions in a virtualized environment
US11693789B2 (en) System and method for mapping objects to regions
US10740016B2 (en) Management of block storage devices based on access frequency wherein migration of block is based on maximum and minimum heat values of data structure that maps heat values to block identifiers, said block identifiers are also mapped to said heat values in first data structure
CN106302702B (zh) 数据的分片存储方法、装置及系统
US11157325B2 (en) System and method for seamless integration of automated orchestrator
CN104081348B (zh) 通过在虚拟数据中心中最佳地放置vm而减少存储器使用的系统和方法
US20200356402A1 (en) Method and apparatus for deploying virtualized network element device
CN104133775B (zh) 一种用于管理存储器的方法和装置
EP3226131B1 (en) Scaling method and device for virtual machine (vm)
US20130275975A1 (en) Resource management server, resource management method and storage medium in which resource management program is stored
CN106296530B (zh) 针对非聚合基础设施的信任覆盖
CN106663012B (zh) 一种硬件加速方法以及相关设备
US11609777B2 (en) System and method for multi-cluster storage
US11809382B2 (en) System and method for supporting versioned objects
WO2018040525A1 (zh) 资源池的处理方法、装置和设备
US20200192678A1 (en) Efficient instantiation of encrypted guests
CN106909556B (zh) 内存集群的存储均衡方法及装置
CN108920613A (zh) 一种元数据管理方法、系统及设备和存储介质
CN108073583B (zh) 一种分图方法及装置
CN105607943A (zh) 一种云环境下虚拟机动态部署机制
WO2016101798A1 (zh) 一种对大数据进行处理的方法和装置
US11042395B2 (en) Systems and methods to manage workload domains with heterogeneous hardware specifications
CN111106945B (zh) 一种vnf实例化方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17746804

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2018539345

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2017746804

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2017746804

Country of ref document: EP

Effective date: 20180903