WO2016107610A1 - 一种数据库集群的资源协调方法、装置及系统 - Google Patents

一种数据库集群的资源协调方法、装置及系统 Download PDF

Info

Publication number
WO2016107610A1
WO2016107610A1 PCT/CN2016/070052 CN2016070052W WO2016107610A1 WO 2016107610 A1 WO2016107610 A1 WO 2016107610A1 CN 2016070052 W CN2016070052 W CN 2016070052W WO 2016107610 A1 WO2016107610 A1 WO 2016107610A1
Authority
WO
WIPO (PCT)
Prior art keywords
coordination node
page data
computing nodes
node
primary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/070052
Other languages
English (en)
French (fr)
Inventor
肖宇雷
叶涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP16732876.4A priority Critical patent/EP3185148B1/en
Publication of WO2016107610A1 publication Critical patent/WO2016107610A1/zh
Priority to US15/639,675 priority patent/US10642822B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
    • G06F11/2043—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant where the redundant components share a common memory address space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G06F11/3433—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment for load management
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2358—Change logging, detection, and notification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/81—Threshold
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/885—Monitoring specific for caches

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a method, device and system for coordinating a database cluster.
  • PNs compute nodes
  • CNs coordination nodes
  • SQL Structured Query Language
  • each computing node is redundant with each other, and a single computing node failure does not affect other computing nodes.
  • coordination node which is used to coordinate the competing public resources when multiple computing nodes are executed concurrently.
  • a typical representative of the existing cluster Share-Disk architecture is the Oracle RAC (Real Application Clusters) solution.
  • each computing node server is directly interconnected through a high-speed private network, and resources on each computing node can share access to each other, and control concurrently through a DML mechanism, which refers to a data lock for protecting data integrity.
  • DML mechanism refers to a data lock for protecting data integrity.
  • Coordination nodes in prior art solutions (such as Oracle RAC) mainly implement the coordination of common resources competing between different nodes, but do not consider the sharing of storage space on the coordination nodes.
  • the data cached by the coordinating node will be very small, so the storage space on the coordination node and the high-speed communication characteristics of the Infiniband (coordination node and computing node communication interface) cannot be fully utilized. Therefore, for the scenario where the computing node fails to restart, or newly adds a computing node, the resource cannot be obtained from the high-speed coordination node, and can only be read from the magnetic array again, and the computing node cannot quickly warm up, and the efficiency is low.
  • the solution provided by the embodiment of the present invention is to solve the problem that the computing node cannot quickly obtain the resource from the high-speed coordination node in the scenario of the faulty restart of the computing node or the newly added node in the prior art.
  • Resource coordination method for database clusters making full use of coordination The idle resources on the node, so that the data generated by the computing node can be stored more on the coordination node, making full use of the idle resources on the coordination node and the high-speed communication characteristics of the Infiniband (coordination node and computing node communication interface), so that the computing node The startup is faster and more efficient.
  • the embodiments of the present invention also provide corresponding devices and systems.
  • an embodiment of the present invention provides a resource coordination method for a database cluster, where the database cluster includes a plurality of computing nodes and a primary coordination node, and the method includes:
  • the respective computing nodes are notified to upload the clean page data generated subsequently to the primary coordination node.
  • the method further includes :
  • the method further includes:
  • the method further includes:
  • the primary coordination node receives the newly generated dirty page data, and the data amount of the newly generated dirty page data is greater than the data amount of the clean page data buffered on the primary coordination node;
  • the clean page data cached on the primary coordination node is preferentially eliminated, and the newly generated dirty page data is stored on the storage device of the database cluster.
  • the cache policy of the primary coordination node includes: each of the plurality of computing nodes a strategy for uploading clean page data by the primary coordination node;
  • the notifying the respective computing nodes to upload the subsequently generated clean page data to the active coordination node includes:
  • the generating the primary coordination according to the state information corresponding to each of the multiple computing nodes includes:
  • the status information includes: a quantity of the unit time transaction TS of the computing node corresponding to the status information, and a reading Write ratio R/M, CPU utilization CP, IO (input/output) utilization IP, and single The bit time generates a dirty page data amount DS.
  • the calculating according to the state information corresponding to each of the plurality of computing nodes The proportion of the current page time dirty page data generated by each compute node in each compute node, including:
  • JP Q1*CP+Q2*IP, where QW is the dirty page data weight generated by the write transaction; where Q1 is the weight of the CPU utilization, and Q2 is the weight of the IO utilization, where Q1+Q2 ⁇ 1.
  • an embodiment of the present invention provides a primary coordination node that is applied to a database cluster, where the primary coordination node includes:
  • An obtaining module configured to acquire state information corresponding to each of the plurality of computing nodes in the database cluster, where the state information is used to indicate an operating load state of the computing node;
  • a first determining module configured to determine, according to status information corresponding to each of the plurality of computing nodes, whether the primary coordination node has an idle resource with a preset threshold X;
  • an execution module configured to notify the respective computing nodes to upload the subsequently generated clean page data to the primary coordination node when the first determination module determines that the result is yes.
  • the active coordination node further includes:
  • a second judging module configured to: when the first judging module determines that the result is YES, determine, according to the state information corresponding to each of the plurality of computing nodes, whether each of the plurality of computing nodes will be Uploading dirty page data smaller than a preset threshold X within a preset time;
  • the executing module is configured to notify the respective computing nodes to upload the subsequently generated clean page data to the active coordination node when the second determining module determines that the result is yes.
  • the primary coordination node further includes a monitoring module, configured to monitor the primary usage in real time. Coordinating the idle resource of the node, when the idle resource of the active coordination node is less than the preset threshold Z, notifying, by the execution module, that each computing node does not upload the generated clean page data to the active coordination node. Where Z is less than X.
  • the execution module notifies the respective computing node to no longer upload the generated clean page data to the After the primary coordination node is described, if the primary coordination node receives the newly generated dirty page data,
  • the execution module is further configured to: preferentially eliminate the clean page data cached by the primary coordination node.
  • a fourth possible implementation manner of the second aspect when the execution module notifies the each computing node to upload the generated clean page data to the After the primary coordination node is received, if the primary coordination node receives the newly generated dirty page data, and the data amount of the newly generated dirty page data is greater than the data of the clean page data cached by the primary coordination node Quantity
  • the execution module is further configured to: preferentially eliminate the clean page data cached by the primary coordination node, and store the newly generated dirty page data on the storage device of the database cluster.
  • the device when the second determining module determines that the result is yes, the device further includes:
  • a cache policy generation module configured to generate a cache policy of the primary coordination node according to state information corresponding to each of the plurality of compute nodes; and the cache policy of the primary coordination node includes: the plurality of compute nodes a strategy for each of the computing nodes to upload clean page data to the primary coordination node;
  • the executing module is configured to: notify the respective computing nodes to upload the subsequently generated clean page data to the active coordination node according to the cache policy of the primary coordination node.
  • the cache policy generating module is specifically configured to:
  • an embodiment of the present invention provides a database cluster, including a plurality of computing nodes and a primary coordination node, where the primary coordination node obtains state information corresponding to each computing node of the multiple computing nodes, and Determining, according to state information corresponding to each of the plurality of computing nodes, whether the primary coordination node has an idle resource of a preset threshold X; if yes, notifying the respective computing nodes to generate clean page data subsequently Uploading to the primary coordination node; wherein the status information is used to indicate an operational load status of the computing node;
  • the respective computing nodes receive the notification sent by the primary coordination node, and when any one of the computing nodes generates clean page data, upload the generated clean page data to the primary coordination node.
  • the primary coordination node acquires state information corresponding to each of the plurality of computing nodes, and according to the multiple computing nodes Determining, by the state information corresponding to each of the computing nodes, whether the primary coordination node has an idle resource of a preset threshold X; if yes, determining the location according to the state information corresponding to each of the plurality of computing nodes Whether each of the plurality of computing nodes will upload dirty page data less than a preset threshold X within a preset time; if yes, notifying the respective computing nodes to upload subsequently generated clean page data to the primary coordination node ;
  • Each of the computing nodes receives the notification sent by the primary coordination node, and when any one of the computing nodes generates clean page data, uploads to the primary coordination node.
  • the primary coordination node determines whether each of the multiple computing nodes is to be in a preset time And uploading the dirty page data that is less than the preset threshold X; if yes, generating a cache policy of the primary coordination node according to the state information corresponding to each of the plurality of computing nodes; and notifying the respective computing nodes to generate the subsequent
  • the clean page data is uploaded to the active coordination node according to the cache policy of the primary coordination node;
  • each of the computing nodes receives the notification sent by the primary coordination node, and when any one of the computing nodes generates clean page data, the generated clean page data is coordinated according to the primary usage.
  • the node's cache policy is uploaded to the primary coordination node.
  • the database cluster further includes an alternate coordination node, Performing data backup for the primary coordination node such that the content on the alternate coordination node is the same as the content on the primary coordination node.
  • the embodiment of the present invention determines whether the primary coordination node has an idle resource with a preset threshold X, and when the determination result is yes, notifies each computing node in the next work. Upload the newly generated clean page data to the primary coordination node. Therefore, the idle resources on the primary coordination node are fully utilized.
  • the computing node restarts or adds a new computing node, resources can be quickly obtained from the high-speed coordination node, and the storage space on the coordination node and the high-speed communication characteristics of Infiniband can be fully utilized. So that the computing node can quickly warm up to improve work efficiency.
  • FIG. 1 is a hardware structural diagram of a database cluster according to an embodiment of the present invention.
  • FIG. 2 is a flowchart of a resource coordination method for a database cluster according to Embodiment 1 of the present invention
  • FIG. 3 is a flowchart of another method for resource coordination of a database cluster according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of still another method for resource coordination of a database cluster according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of still another method for resource coordination of a database cluster according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a primary coordination node applied to a database cluster according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of another primary coordination node applied to a database cluster according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of still another primary coordination node applied to a database cluster according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of still another primary coordination node applied to a database cluster according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of an active coordination node applied to a database cluster according to an embodiment of the present invention.
  • An embodiment of the present invention provides a resource coordination method for a database cluster, and determines, according to state information corresponding to each computing node of the plurality of computing nodes, whether the primary coordination node has an idle resource with a preset threshold X; if yes, notifying each computing node
  • the subsequent generated clean page data is uploaded to the primary coordination node, and the storage space on the coordination node and the high-speed communication characteristics of the Infiniband are fully utilized, so that the computing node can quickly warm up and improve work efficiency.
  • the embodiments of the present invention also provide corresponding devices and systems. The details are described below separately.
  • the database cluster in the embodiment of the present invention includes: a computing node and a primary coordination node.
  • the database cluster has an alternate coordination node, which is used for backing up data of the primary coordination node and assumes primary coordination when the primary coordination node fails.
  • the work of the node There are usually a plurality of computing nodes, which are mainly used to perform specific functions, such as executing specific SQL (Structured Query Language) and specific transactions; there is generally only one primary coordination node, and of course, multiple primary coordination nodes are not excluded.
  • the primary coordination node is used to coordinate the competing public resources when multiple computing nodes are executed concurrently.
  • the 1 can provide hardware support for the resource coordination method of the database cluster provided by the embodiment of the present invention, including the primary coordination node 102, the standby coordination node 104, and the high-speed network channel (eg, Infiniband, IB) 112.
  • Computing nodes 112, 114, 116, 118, Fibre Channel 132, and shared storage disk 142 stores data in database cluster 100, and computing nodes 112, 114, 116, 118 perform specific transactions to perform computational or read-write operations on data stored on shared storage disk 142.
  • the primary coordination node 102 manages communication between the computing nodes 112, 114, 116, 118.
  • the standby coordination node 104 replaces the primary coordination node 102 when the primary coordination node 102 fails, and completes the function of the primary coordination node 102 in the normal state.
  • Communication between computing nodes 112, 114, 116, 118 and shared storage disk 142 is through Fibre Channel 132.
  • the above-mentioned primary coordination node 102, backup coordination node 104, and computing nodes 112, 114, 116, and 118 may each be a computer, and thus the internal hardware structure thereof may be the same as or similar to that of a general-purpose computer, for example, they have
  • the processor has both memory and a hard disk (which can be called external storage).
  • the resource coordination method, device and system of the database cluster provided by the following embodiments of the present invention are performed on the basis of such hardware.
  • FIG. 2 is a flowchart of a method for resource coordination of a database cluster according to an embodiment of the present invention. It can be understood that the flowchart of the method can be run on the hardware foundation of the database cluster 100 as shown in FIG. 1 . on.
  • the primary coordination node acquires corresponding to each of the plurality of computing nodes.
  • Status information where the status information is used to indicate the operational load status of the computing node corresponding to the status information.
  • the standby coordination node can obtain the coordination with the primary management node directly from multiple computing nodes in order to maintain the same data as the primary coordination node.
  • the information obtained by the node with the same information can also be obtained from the primary coordination node that has obtained the information, and the related information is also the same, and will not be described again.
  • the status information herein refers to the situation that each computing node corresponds to the cache load in the process of executing a specific transaction, including but not limited to: the number of transactions per unit time (TS), the ratio of read and write (R) /W), CPU utilization (CP), IO utilization (IP), and data such as dirty page data (DS) per unit time.
  • the primary coordination node determines, according to the state information corresponding to each of the plurality of computing nodes, whether the primary coordination node has an idle resource of a preset threshold X.
  • the idle resource herein may refer to a cache space on the primary coordination node, and the cache space may refer to the memory of the primary coordination node, and may also refer to the memory of the primary coordination node and the external storage (ie, a storage component such as a hard disk).
  • the preset threshold X here may be a threshold set based on an empirical value.
  • step 205 is performed:
  • step 205 the primary coordination node notifies the respective computing nodes to upload subsequently generated clean page data to the primary coordination node.
  • the resource coordination method of the database cluster determines whether the active coordination node has idle resources according to the state information of each computing node according to the running state of the plurality of computing nodes, and if not, notifies the computing node to subsequently generate the generated resources.
  • the clean page data is uploaded to the primary coordination node, so that the primary coordination node can fully utilize its redundant idle resources to help calculate the clean pages of the cached non-competitive resources while coordinating the competing resources of the computing nodes.
  • step 204 is added.
  • the primary coordination node determines, according to the state information corresponding to each of the plurality of computing nodes, whether each of the plurality of computing nodes will upload a dirty less than a preset threshold X within a preset time. Page data.
  • the preset time may be set according to the experience value, and may be in the hour or the day, and there is no limitation here.
  • step 205 is further performed, that is, the primary coordination node notifies the respective computing nodes to upload the subsequently generated clean page data to the primary coordination node.
  • step 204 After the determining operation of step 204 is added, on the basis of the first embodiment, after two judgments of step 203 and step 204, it is determined that the active coordination node has the idle resource of the preset threshold X and each computing node within the preset time. If the uploaded dirty data is smaller than the preset threshold X, it indicates that there will still be idle resources available on the primary coordination node for accommodating the clean page data of the computing node during the preset time, thereby further ensuring the primary coordination.
  • the availability of the idle resources on the node enables the solution provided by the embodiment of the present invention to have more accurate practicability and further improves the working efficiency of the computing node.
  • an embodiment of the present invention may further add step 206 after step 205.
  • the primary coordination node monitors its own idle resource in real time, and when its idle resource is less than the preset threshold Z, it notifies each computing node to no longer upload the generated clean page data to the primary coordination node.
  • Z is less than X.
  • the preset threshold Z can be obtained according to the empirical value or the requirements of the hardware device.
  • Z can be 10% of the total storage space on the primary coordination node, where the total storage space can be the primary coordination node.
  • the total value of the memory space can also be the sum of the memory space on the primary coordination node and the external storage space such as the hard disk. This is not limited by way of example.
  • step 206 another embodiment of the present invention further adds step 207a.
  • step 207a if the primary coordination node receives the newly generated dirty page data, the clean page data cached on the primary coordination node is preferentially eliminated.
  • step 207b may add step 207b after step 206.
  • step 207b if the primary coordination node receives the newly generated dirty page data, and the new production
  • the amount of data of the raw dirty page data is greater than the amount of data of the clean page data buffered on the primary coordination node; then the clean page data cached on the primary coordination node is preferentially eliminated, and the newly generated dirty page is The data is stored on a storage device of the database cluster.
  • step 204' on the basis of the determination in step 204 as YES.
  • a cache policy of the primary coordination node is generated according to the state information corresponding to each of the plurality of computing nodes; the cache policy of the primary coordination node includes: the plurality of computing nodes a strategy for each computing node to upload clean page data to the primary coordination node;
  • the cache policy may be a comprehensive policy consisting of cache policies corresponding to the respective compute nodes, and is used to indicate that each compute node caches clean page data generated by itself on the active coordination node, and cache time (when) The amount of time to start uploading and the amount of cache can be reflected in this caching strategy.
  • step 205 can be further step 205':
  • step 205' the primary coordination node notifies the respective computing nodes to upload the subsequently generated clean page data to the primary coordination node according to the cache policy of the primary coordination node.
  • the method for calculating a cache policy may be used in any of the foregoing manners.
  • the method for calculating a cache policy may be: according to the state information corresponding to each of the plurality of compute nodes. Calculating a proportion of the current page time dirty page data of each of the plurality of computing nodes;
  • the status information herein refers to the situation that each computing node corresponds to the cache load in the process of executing a specific transaction, including but not limited to: the number of transactions per unit time (TS), the ratio of read and write (R/ W), CPU utilization (CP), IO utilization (IP), and data such as dirty page data (DS) are generated per unit time.
  • TS number of transactions per unit time
  • R/ W ratio of read and write
  • CP CPU utilization
  • IP IO utilization
  • DS dirty page data
  • the calculation unit generates a ratio of the current page time of the dirty page data to the T.
  • the specific calculation method may be obtained according to the following method.
  • the following method is only an optional method provided in the embodiment of the present invention. , does not constitute a restriction on this program.
  • JP Q1*CP+Q2*IP, where QW is the dirty page data weight generated by the write transaction; where Q1 is the weight of the CPU utilization, and Q2 is the weight of the IO utilization, where Q1+Q2 ⁇ 1.
  • a resource coordination method for a database cluster is provided.
  • a cache policy of a primary coordination node is generated according to state information corresponding to each computing node of multiple computing nodes, thereby The computing node can cache the clean page data to the primary coordination node according to the cache policy, so that each computing node can cache the clean page data to the primary coordination node in a targeted manner, so that the solution provided by the embodiment of the present invention is provided. It has more precise practicality and further improves the efficiency of computing nodes.
  • Embodiment 2 As shown in FIG. 6, an embodiment of the present invention provides a primary coordination node 600 applied to a database cluster.
  • the primary coordination node 600 can operate on the hardware of the database cluster 100 as provided in FIG. 1, for example, the primary coordination node 600 can be 102 in the database cluster 100.
  • the primary coordination node 600 includes:
  • the obtaining module 602 is configured to acquire state information corresponding to each of the plurality of computing nodes in the database cluster, where the state information is used to indicate an operating load state of the computing node;
  • the first determining module 604 is configured to determine, according to state information corresponding to each of the plurality of computing nodes, whether the primary coordination node has an idle resource with a preset threshold X;
  • the executing module 606 is configured to notify the respective computing nodes to upload the subsequently generated clean page data to the active coordination node when the first determining module determines that the result is yes.
  • the active coordination node provided by the embodiment of the present invention, according to the operation state of multiple computing nodes,
  • the obtaining module 603 obtains the state information of each computing node for determining, by the first determining module 604, whether the primary coordination node has an idle resource, and if yes, the execution module 606 notifies the computing node to upload the subsequently generated clean page data.
  • the primary coordination node 600 can coordinate the spare resources of each computing node, and can fully utilize its redundant resources to help calculate the clean page data of the cached non-competitive resources.
  • the primary coordination node 600 may further include:
  • the second determining module 605 is configured to: when the first determining module 604 determines that the result is YES, determine, according to state information corresponding to each computing node of the plurality of computing nodes, each computing node of the multiple computing nodes Whether dirty page data smaller than the preset threshold X will be uploaded within a preset time;
  • the executing module 606 can be configured to notify the respective computing nodes to upload the subsequently generated clean page data to the active coordination node when the second determining module 605 determines that the result is yes.
  • the primary coordination node 600 may further include:
  • the monitoring module 607 is configured to monitor, in real time, the idle resource of the primary coordination node, and when the idle resource of the primary coordination node is less than a preset threshold Z, notify, by the execution module, that each computing node is no longer generated.
  • the clean page data is uploaded to the primary coordination node, where Z is less than X.
  • the execution module 606 After the execution module 606 notifies the respective computing nodes that the generated clean page data is no longer uploaded to the primary coordination node, if the primary coordination node 600 receives the newly generated dirty page data, the The execution module 606 is further configured to: preferentially retire the clean page data cached on the primary coordination node.
  • the execution module 606 when the execution module 606 notifies the respective computing nodes that the generated clean page data is no longer uploaded to the primary coordination node, if the primary coordination node 600 receives the newly generated dirty page number According to the data volume of the newly generated dirty page data is greater than the data amount of the clean page data cached by the primary coordination node 600; the execution module 606 is further configured to: preferentially phase out the primary coordination node
  • the clean page data cached on 600 and the newly generated dirty page data is stored on a storage device of the database cluster (e.g., shared storage disk 142 in database cluster 100).
  • the primary coordination node 600 further includes: a cache policy generation module 605': Generating a cache policy for the primary coordination node 600 by using state information corresponding to each of the plurality of computing nodes; the cache policy of the primary coordination node 600 includes: each of the plurality of computing nodes to the master A strategy for uploading clean page data by the coordination node 600;
  • the executing module 606 is further configured to: notify the respective computing nodes to upload the subsequently generated clean page data to the active coordination node 600 according to the cache policy of the primary coordination node.
  • cache policy generating module 605' may be specifically configured to:
  • Embodiment 3 as shown in FIG. 1, on the basis of the hardware device of the database cluster 100 of FIG. 1, on the basis of the hardware device of the database cluster 100 of FIG.
  • the embodiment of the invention provides a database cluster, which includes a plurality of computing nodes, such as 122, 124, 126, 128, etc., and a primary coordination node 102:
  • the primary coordination node 102 obtains state information corresponding to each of the plurality of computing nodes, such as 122, 124, 126, 128, and the like, and according to state information corresponding to each of the plurality of computing nodes Determining whether the primary coordination node 102 has a temporary threshold of a predetermined threshold X a source; if yes, notifying the respective computing nodes to upload the clean page data generated subsequently to the primary coordination node 102; wherein the status information is used to indicate an operational load status of the computing node;
  • the respective computing nodes receive the notification sent by the primary coordination node 102, and when any one of the computing nodes generates clean page data, upload the generated clean page data to the primary coordination node. 102.
  • the embodiment of the present invention further provides a database cluster, including multiple computing nodes, such as 122, 124, 126, 128, and the like, and a primary coordination node 102:
  • the primary coordination node 102 obtains state information corresponding to each of the plurality of computing nodes, such as 122, 124, 126, 128, and the like, and according to state information corresponding to each of the plurality of computing nodes Determining whether the primary coordination node 102 has an idle resource of a preset threshold X; if yes, determining, according to the state information corresponding to each of the plurality of computing nodes, each of the plurality of computing nodes Whether the node will upload the dirty page data less than the preset threshold X within the preset time; if yes, notify the respective computing nodes to upload the clean page data generated subsequently to the primary coordination node 102;
  • Each of the computing nodes receives the notification sent by the primary coordination node 102, and uploads to the primary coordination node 102 when any one of the computing nodes generates clean page data.
  • the embodiment of the present invention further provides another database cluster, including multiple computing nodes, such as 122, 124, 126, 128, and the like, and a primary coordination node 102:
  • the primary coordination node 102 determines whether each of the plurality of computing nodes, such as 122, 124, 126, and 128, will upload dirty page data that is less than a preset threshold X within a preset time; Generating the corresponding state information of each computing node in the computing node, generating a cache policy of the primary coordination node 102; and notifying the respective computing nodes to generate subsequent clean page data according to the primary use
  • the coordination node's cache policy is uploaded to the primary coordination node 102;
  • each of the computing nodes receives the notification sent by the primary coordination node 102, and when any one of the computing nodes generates clean page data, the generated clean page data is used according to the primary The caching policy of the coordinating node is uploaded to the primary coordinating node 102.
  • the database cluster may further include an alternate coordination node 104, where the backup coordination node 104 is configured to perform data backup for the primary coordination node 102, so that the standby coordination node The content on 104 is the same as the content on the primary coordination node 102.
  • FIG. 10 is a schematic structural diagram of the primary coordination node 102 of FIG. 1 or the primary coordination node 600 of FIGS. 6 to 9 applied to a database system according to a sixth embodiment of the present invention.
  • the primary coordination node 102 and the primary coordination node 600 are the same objects described at different angles, and their internal structures are the same.
  • the following uses the primary coordination node 102 as an example for description.
  • the primary coordination node 102 includes a transceiver 720, a central processing unit (CPU) 730, a memory 740, and a bus 750.
  • the transceiver 720 is configured to receive a transaction log sent by each computing node in the database cluster system.
  • the transaction log is the database log that records the corresponding transaction.
  • the transceiver 720 is configured to receive, by the high speed network, a transaction log sent by a computing node within the database system.
  • a high speed network refers to a network with a transmission delay of no more than nanoseconds (ns).
  • the high speed network is IB.
  • the transceiver 720 is configured to acquire state information corresponding to each of the plurality of computing nodes, where the state information is used to indicate an operating load state of the computing node;
  • the processor 730 is configured to determine, according to status information corresponding to each of the plurality of computing nodes, whether the primary coordination node has an idle resource with a preset threshold X; if yes, notify the respective calculation by the transceiver 720.
  • the node uploads the clean page data generated subsequently to the primary coordination node.
  • the processor 730 determines, according to state information corresponding to each of the plurality of computing nodes, whether the primary coordination node has an idle resource of a preset threshold X; if And determining, according to state information corresponding to each of the plurality of computing nodes, whether each computing node of the plurality of computing nodes will upload in a preset time Dirty page data smaller than a preset threshold X;
  • the respective computing nodes are notified by the transceiver 720 to upload the subsequently generated clean page data to the active coordination node.
  • the processor 730 is further configured to monitor the real-time monitoring after the respective computing nodes are notified by the transceiver 720 to upload the subsequently generated clean page data to the primary coordination node.
  • the idle resource of the primary coordination node when the idle resource of the primary coordination node is less than the preset threshold Z, the transceiver 720 is notified by the transceiver 720 that the generated clean page data is no longer uploaded to the primary coordination. Node, where Z is less than X.
  • the processor 730 is further configured to: Prioritizing the clean page data cached on the memory 740 of the primary coordination node;
  • the processor 730 further It is used to preferentially eliminate the clean page data cached on the memory 740 of the primary coordination node, and store the newly generated dirty page data on the storage device (shared storage disk) of the database cluster.
  • the cache policy includes: a policy for each of the plurality of compute nodes to upload clean page data to the primary coordination node; and notifying, by the transceiver 720, the respective compute nodes to subsequently generate clean page data according to the The cache policy of the primary coordination node is uploaded to the primary coordination node 102.
  • the bus 750 is used for communication between the processor 730, the transceiver 720, and the memory 740.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Physics (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

一种数据库集群的资源协调方法、设备及系统，方法包括：主用协调节点获取多个计算节点中各个计算节点分别对应的状态信息（201），状态信息用于表示计算节点的运行负载状态；并根据多个计算节点中各个计算节点分别对应的状态信息，判断主用协调节点是否存在预设阈值X的空闲资源（203）；若是则通知各个计算节点将后续产生的干净页数据上传到主用协调节点（205）。通过这样的方法可以使计算节点产生的数据有更多能够存储在协调节点上，充分利用协调节点上的空闲资源和协调节点与计算节点通讯接口高速通讯的特性，使得计算节点的启动更加快速高效。

Description

一种数据库集群的资源协调方法、装置及系统 技术领域
本发明涉及计算机技术领域，具体涉及一种数据库集群的协调方法、装置及系统。
背景技术
在集群Share-Disk(共享磁盘)架构中，所有集群中的服务器共享一个存储。集群中的服务器按照功能，一般可以分为计算节点(PN)和协调节点(CN)。计算节点一般有多个，用来承担执行具体的功能，执行具体的SQL(结构化查询语言)和事务。当数据库性能需要提升时，可以通过增加计算节点来横向扩展，达到提升集群整体计算能力的目的。同时各个计算节点相互冗余，单个计算节点故障不影响其它计算节点。协调节点一般只有一个，是用来当多个计算节点并发执行时，对竞争的公共资源进行协调。
现有集群Share-Disk架构的典型代表是Oracle RAC(Real Application Clusters)方案。该方案中，各个计算节点服务器通过高速私有网络直接互联，每个计算节点上的资源可以相互共享访问，通过DML机制控制并发，该DML是指数据锁，用于保护数据的完整性。现有技术方案(如Oralce RAC)中的协调节点主要实现了不同节点间竞争的公共资源的协调，但是没有考虑到协调节点上存储空间的共享。因此对于只读或者读多写少的场景，协调节点缓存的数据就会非常少，因此无法充分利用协调节点上的存储空间和Infiniband(协调节点与计算节点通讯接口)高速通讯的特性，存在浪费；从而对于计算节点故障重启，或者新添加计算节点等场景，无法从高速的协调节点获取资源，只能重新从磁阵读取，计算节点无法快速预热，效率低下。
发明内容
为了解决现有技术中计算节点故障重启或者新添加节点等场景下，无法从高速的协调节点快速获取资源，从而使得计算节点无法快速预热导致效率地下的问题，本发明实施例提供的一种数据库集群的资源协调方法，充分利用协调 节点上的空闲资源，从而使得计算节点产生的数据有更多能够存储在协调节点上，充分利用协调节点上的空闲资源和Infiniband(协调节点与计算节点通讯接口)高速通讯的特性，使得计算节点的启动更加快速高效。本发明实施例还提供了相应的装置及系统。
第一方面，本发明实施例提供一种数据库集群的资源协调方法，所述数据库集群包括多个计算节点和一个主用协调节点，所述方法包括：
获取所述多个计算节点中各个计算节点分别对应的状态信息，所述状态信息用于表示计算节点的运行负载状态；
根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；若是
则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点；或者，
根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是
则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
结合第一方面，在第一方面的第一种可能的实施方式中，在所述通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点之后，所述方法还包括：
实时监控所述主用协调节点的空闲资源，当所述主用协调节点的空闲资源小于预设阈值Z，则通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点，其中Z小于X。
结合第一方面的第一种可能的实施方式中，在第一方面的第二种可能的实施方式中，在所述通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，所述方法还包括：
若所述主用协调节点收到新产生的脏页数据；
则优先淘汰所述主用协调节点上缓存的干净页数据。
结合第一方面的第一种可能的实施方式中，在第一方面的第三种可能的实施方式中，在所述通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，所述方法还包括：
若所述主用协调节点收到新产生的脏页数据，且所述新产生的脏页数据的数据量大于所述主用协调节点上缓存的干净页数据的数据量；
则优先淘汰所述主用协调节点上缓存的干净页数据，并将所述新产生的脏页数据存储到所述数据库集群的存储设备上。
结合第一方面，在第一方面的第四种可能的实施方式中，在所述根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是之后，所述方法还包括：
则根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略；所述主用协调节点的缓存策略包括：所述多个计算节点中各个计算节点向所述主用协调节点上传干净页数据的策略；
对应地，所述通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点包括：
通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点。
结合第一方面的四种可能的实施方式中，在第一方面的第五种可能的实施方式中，所述根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略包括：
根据所述多个计算节点中各个计算节点分别对应的状态信息，计算所述多个计算节点中每一个计算节点当前单位时间脏页数据的产生比例；
根据所述每一个计算节点当前单位时间脏页数据的产生比例，生成主用协调节点的缓存策略。
结合第一方面的第五种可能的实施方式中，在第一方面的第六种可能的实施方式中，所述状态信息包括：所述状态信息对应的计算节点的单位时间事务数量TS、读写比例R/M、CPU利用率CP、IO(输入/输出)利用率IP、以及单 位时间产生脏页数据量DS。
结合第一方面的第六种可能的实施方式，在第一方面的第七种可能的实施方式中，所述根据所述多个计算节点中各个计算节点分别对应的状态信息，计算所述多个计算节点中每一个计算节点当前单位时间脏页数据的产生比例，包括：
Figure PCTCN2016070052-appb-000001
其中，
Figure PCTCN2016070052-appb-000002
JP＝Q1*CP+Q2*IP，其中QW为写事务产生的脏页数据权值；其中Q1为CPU利用率的权值，Q2为IO利用率的权值，其中Q1+Q2≤1。
第二方面，本发明实施例提供一种应用于数据库集群的主用协调节点，所述主用协调节点包括：
获取模块，用于获取所述数据库集群中多个计算节点中各个计算节点分别对应的状态信息，所述状态信息用于表示计算节点的运行负载状态；
第一判断模块，用于根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；
执行模块，用于当所述第一判断模块判断结果为是时，通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
结合第二方面，在第二方面的第一种可能的实施方式中，所述主用协调节点还包括：
第二判断模块，用于当所述第一判断模块判断结果为是时，根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；
对应地，所述执行模块用于当所述第二判断模块判断结果为是时，通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
结合第二方面或第二方面的第一种可能的实施方式，在第二方面的第二种可能的实施方式中，所述主用协调节点还包括监控模块，用于实时监控所述主用协调节点的空闲资源，当所述主用协调节点的空闲资源小于预设阈值Z，则通过所述执行模块通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点，其中Z小于X。
结合第二方面的第二种可能的实施方式中，在第二方面的第三种可能的实施方式中，当所述执行模块通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，若所述主用协调节点收到新产生的脏页数据，则
所述执行模块还用于：优先淘汰所述主用协调节点上缓存的干净页数据。
结合第二方面的第二种可能的实施方式中，在第二方面的第四种可能的实施方式中，当所述执行模块通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，若所述主用协调节点收到新产生的脏页数据，且所述新产生的脏页数据的数据量大于所述主用协调节点上缓存的干净页数据的数据量；则
所述执行模块还用于：则优先淘汰所述主用协调节点上缓存的干净页数据，并将所述新产生的脏页数据存储到所述数据库集群的存储设备上。
结合第二方面的第一种可能的实施方式中，在第二方面的第五种可能的实施方式中，当所述第二判断模块判断结果为是时，所述装置还包括：
缓存策略生成模块：用于根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略；所述主用协调节点的缓存策略包括：所述多个计算节点中各个计算节点向所述主用协调节点上传干净页数据的策略；
对应地，所述执行模块用于：通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点。
结合第二方面的第五种可能的实施方式中，在第二方面的第六种可能的实施方式中，所述缓存策略生成模块具体用于：
根据所述多个计算节点中各个计算节点分别对应的状态信息，计算所述多个计算节点中每一个计算节点当前单位时间脏页数据的产生比例；
根据所述每一个计算节点当前单位时间脏页数据的产生比例，生成主用协调节点的缓存策略。
第三方面，本发明实施例提供一种数据库集群，包括多个计算节点和一个主用协调节点，所述主用协调节点获取所述多个计算节点中各个计算节点分别对应的状态信息，并根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；若是，则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点；其中，所述状态信息用于表示计算节点的运行负载状态；
所述各个计算节点接收所述主用协调节点发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则将所述产生的干净页数据上传到所述主用协调节点。
结合第三方面，在第三方面的第一种可能的实施方式中，所述主用协调节点获取所述多个计算节点中各个计算节点分别对应的状态信息，并根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；若是之后，再根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点；
所述各个计算节点接收所述主用协调节点发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则上传到所述主用协调节点。
结合第三方面的第一种可能的实施方式，在第三方面的第二种可能的实施方式中，所述主用协调节点判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是则根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略；并通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点；
对应地，所述各个计算节点接收所述主用协调节点发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则将所述产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点。
结合第三方面的第一种可能的实施方式或第三方面的第二种可能的实施方式，在第三方面的第三种可能的实施方式中，所述数据库集群还包括备用协调节点，用于为所述主用协调节点进行数据备份，以使得所述备用协调节点上的内容与所述主用协调节点上的内容相同。
本发明实施例根据多个计算节点中各个计算节点分别对应的状态信息，判断主用协调节点是否存在预设阈值X的空闲资源，当判断结果为是则通知各个计算节点在接下来的工作中将新产生的干净页数据上传到主用协调节点上来。从而将主用协调节点上的空闲资源充分利用，当计算节点重启或添加新的计算节点时，可以从高速的协调节点上迅速获取资源，充分利用协调节点上的存储空间和Infiniband高速通讯的特性，从而使计算节点能够快速预热提高工作效率。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付 出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的数据库集群的硬件结构图；
图2是本发明实施例一提供的一种数据库集群的资源协调方法流程图；
图3是本发明实施例提供的另一种数据库集群的资源协调方法流程图；
图4是本发明实施例提供的又一种数据库集群的资源协调方法流程图；
图5是本发明实施例提供的再一种数据库集群的资源协调方法流程图；
图6是本发明实施例提供的一种应用于数据库集群的主用协调节点的结构示意图；
图7是本发明实施例提供的另一种应用于数据库集群的主用协调节点的结构示意图；
图8是本发明实施例提供的又一种应用于数据库集群的主用协调节点的结构示意图；
图9是本发明实施例提供的再一种应用于数据库集群的主用协调节点的结构示意图；
图10是本发明实施例提供的应用于数据库集群中的主用协调节点的结构示意图。
具体实施方式
本发明实施例提供一种数据库集群的资源协调方法，根据多个计算节点中各个计算节点分别对应的状态信息，判断主用协调节点是否存在预设阈值X的空闲资源；若是则通知各个计算节点将后续产生的干净页数据上传到主用协调节点，充分利用协调节点上的存储空间和Infiniband高速通讯的特性，从而使计算节点能够快速预热提高工作效率。本发明实施例还提供了相应的装置及系统。以下分别进行详细说明。
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所 有其他实施例，都应当属于本发明保护的范围。
本发明实施例中的数据库集群包括：计算节点和主用协调节点，通常这个数据库集群还有备用协调节点，用于备份主用协调节点的数据并当主用协调节点故障的时候来承担主用协调节点的工作。计算节点一般有多个，主要用于承担执行具体的功能，例如执行具体的SQL(结构化查询语言)及具体事务；主用协调节点一般只有一个，当然也不排除多个主用协调节点一起工作的，主用协调节点是用来当多个计算节点并发执行时，对竞争的公共资源进行协调。如图1所示的数据库集群100可以为本发明实施例提供的数据库集群的资源协调方法提供硬件支持，包括主用协调节点102，备用协调节点104，高速网络通道(如：Infiniband，IB)112、计算节点112、114、116、118，光纤通道132，及共享存储磁盘142。在一个实施例中，共享存储磁盘142存储数据库集群100中的数据，计算节点112、114、116、118执行具体的事务，对共享存储磁盘142上存储的数据进行计算或读写操作。主用协调节点102管理计算节点112、114、116、118间的通讯。备用协调节点104在主用协调节点102故障时替代主用协调节点102，完成主用协调节点102在正常状态下的功能。计算节点112、114、116及118之间，主用协调节点102与备用协调节点104之间，以及与计算节点112、114、116、118与主用协调节点102和备用协调节点104之间，均通过高速网络通道112进行通讯，该高速网络通道可以是IB通道。计算节点112、114、116、118与共享存储磁盘142之间通过光纤通道132进行通讯。上述的主用协调节点102、备用协调节点104、计算节点112、114、116、118具体均可以为计算机，因此其内部的硬件结构可以和通用的计算机的硬件结构相同或相似，例如它们都有处理器，都有内存，还可以有硬盘(可以叫做外存)。本发明以下实施例提供的数据库集群的资源协调方法、装置及系统就是在这样的硬件基础上进行的。
实施例一，如图2所示为本发明实施例提供的一种数据库集群的资源协调方法流程图，可以理解的是，该方法流程图可以运行在如图1提供的数据库集群100的硬件基础上。
在步骤201中，主用协调节点获取多个计算节点中各个计算节点分别对应 的状态信息，此处的状态信息用于表示所述状态信息对应的计算节点的运行负载状态。
需要说明的是，当该数据库集群中有备用协调节点时，备用协调节点为了保持和主用协调节点的数据一致，可以和主用协调节点一样，直接从多个计算节点上获取与主用协调节点所获取信息相同的信息，也可以从已经获取这些信息的主用协调节点上去获取这些信息，后面相关处亦是如此，不再赘述。进一步需要说明的是，此处的状态信息指每一个计算节点对应地其本身在执行具体事务过程中的缓存负载等情况，包括但不限于：单位时间事务数量(TS)、读写比例(R/W)、CPU利用率(CP)、IO利用率(IP)和单位时间产生脏页数据量(DS)等数据。
在步骤203中，主用协调节点根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源。
此处的空闲资源可以指主用协调节点上缓存空间，该缓存空间可以是指主用协调节点的内存，也可以指主用协调节点的内存及外存(即硬盘之类的存储部件)一起构成的存储空间。此处的预设阈值X可以是基于经验值设定的阈值。
基于步骤203的判断，若判断结果为是，则执行步骤205：
在步骤205中，主用协调节点通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
即当判断发现该主用协调节点上存在预设阈值X的的空闲资源后便通知各个计算节点在接下来执行具体事务的过程中产生的干净页数据上传到主用协调节点上去。
本发明实施例提供的数据库集群的资源协调方法，根据统筹多个计算节点的运行状态，根据各个计算节点的状态信息判断主用协调节点是否存在空闲资源，如存在则通知计算节点将后续产生的干净页数据上传到主用协调节点上，这样使得主用协调节点在协调各计算节点的竞争资源之余，还能将其多余的空闲资源充分利用来帮助计算几点缓存非竞争资源的干净页数据，从而当计算节点重启或添加新的计算节点时，可以从高速的协调节点上迅速获取数据资源，充分利用协调节点上的存储空间和Infiniband高速通讯的特性，从而使计算节 点能够快速预热提高工作效率。
如图3所示，在图2的基础上，在步骤203判断为是之后又增加步骤204。
在步骤204中，主用协调节点根据多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据。
需要说明的是，此处的预设时间可以是根据经验值来设定，可以是以小时级也可以以天为单位，此处不做限制。
当步骤204判断结果为是，则进一步执行步骤205，即：主用协调节点通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
添加步骤204的判断动作后，在实施例一的基础上，经过步骤203和步骤204的两次判断，在确定主用协调节点存在预设阈值X的空闲资源且在预设时间内各个计算节点将上传的脏也数据小于该预设阈值X，则说明在该预设时间内，主用协调节点上仍将存在空闲资源可以用于容纳计算节点的干净页数据，因此进一步确保了主用协调节点上空闲资源的可用性，使本发明实施例提供的方案具备更精确的实用性，并进一步提高了计算节点的工作效率。
如图4所示，在图3及上述实施例的基础上，本发明的一个实施例还可以在步骤205之后添加步骤206。
在步骤206中，主用协调节点实时监控它自己的空闲资源，当它的空闲资源小于预设阈值Z时，则通知各个计算节点不再将产生的干净页数据上传到所述主用协调节点，其中Z小于X。值得说明的是此处的预设阈值Z可以根据经验值或者硬件设备的要求来取得，例如Z可以为主用协调节点上总存储空间的10％，这里的总存储空间可以为主用协调节点的内存空间总值，还可以为主用协调节点上内存空间与硬盘等外存空间的和值，此处仅为举例不做限定。
可选的，在步骤206之后，本发明的另一个实施例还添加步骤207a。
在步骤207a中，若所述主用协调节点收到新产生的脏页数据；则优先淘汰所述主用协调节点上缓存的干净页数据。
或者，本发明的另一个实施例还可以在步骤206之后添加步骤207b。
在步骤207b中，若所述主用协调节点收到新产生的脏页数据，且所述新产 生的脏页数据的数据量大于所述主用协调节点上缓存的干净页数据的数据量；则优先淘汰所述主用协调节点上缓存的干净页数据，并将所述新产生的脏页数据存储到所述数据库集群的存储设备上。
如图5所示，在图3的基础上，本发明的又一个实施例在步骤204判断为是的基础上，又添加步骤204’。
在步骤204’中，根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略；所述主用协调节点的缓存策略包括：所述多个计算节点中各个计算节点向所述主用协调节点上传干净页数据的策略；
其中所述的缓存策略可以是由各个计算节点分别对应的缓存策略组成的一个综合策略，用于指示各个计算节点将自身产生的干净页数据在主用协调节点上进行缓存，缓存时间(什么时间开始上传什么时间结束上传)和缓存的量的多少都可以在这个缓存策略中体现。
对应地，步骤205可以进一步为步骤205’：
在步骤205’中，主用协调节点通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点。
可选的，关于上述的缓存策略计算方法可以有很多种，本发明实施例提供一种缓存策略的计算方法，具体可以为：根据所述多个计算节点中各个计算节点分别对应的状态信息，计算所述多个计算节点中每一个计算节点当前单位时间脏页数据的产生比例；
根据所述每一个计算节点当前单位时间脏页数据的产生比例，生成主用协调节点的缓存策略。
需要说明的是，此处的状态信息指每一个计算节点对应地其本身在执行具体事务过程中的缓存负载等情况，包括但不限于：单位时间事务数量(TS)、读写比例(R/W)、CPU利用率(CP)、IO利用率(IP)和单位时间产生脏页数据量(DS)等数据。
进一步可选的，所述计算节点当前单位时间脏页数据的产生比例为T，具体的计算方法可以根据下述方法获取，当然，下述方法只是本发明实施例中提供的一种可选方法，不对本方案构成限制。
Figure PCTCN2016070052-appb-000003
其中，
Figure PCTCN2016070052-appb-000004
JP＝Q1*CP+Q2*IP，其中QW为写事务产生的脏页数据权值；其中Q1为CPU利用率的权值，Q2为IO利用率的权值，其中Q1+Q2≤1。
在本发明实施例中提供了一种数据库集群的资源协调方法，在本发明实施例中，根据多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略，从而使计算节点可以根据这个缓存策略向主用协调节点上进行干净页数据的缓存，使得各个计算节点可以有针对性的向主用协调节点进行干净页数据的缓存，从而使本发明实施例提供的方案具备更精确的实用性，并进一步提高了计算节点的工作效率。
实施例二，如图6所示，本发明实施例提供一种应用于数据库集群的主用协调节点600。该主用协调节点600可以运行在如图1提供的数据库集群100的硬件基础上，例如该主用协调节点600可以是数据库集群100中的102。
所述主用协调节点600包括：
获取模块602，用于获取所述数据库集群中多个计算节点中各个计算节点分别对应的状态信息，所述状态信息用于表示计算节点的运行负载状态；
第一判断模块604，用于根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；
执行模块606，用于当所述第一判断模块判断结果为是时，通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
本发明实施例提供的主用协调节点，根据统筹多个计算节点的运行状态， 获取模块603获取各个计算节点的状态信息用于给第一判断模块604进行判断：主用协调节点是否存在空闲资源，如存在则由所述执行模块606通知计算节点将后续产生的干净页数据上传到主用协调节点上，这样使得主用协调节点600在协调各计算节点的竞争资源之余，还能将其多余的空闲资源充分利用来帮助计算几点缓存非竞争资源的干净页数据，从而当计算节点重启或添加新的计算节点时，可以从高速的协调节点上迅速获取数据资源，充分利用主用协调节点上的存储空间和Infiniband高速通讯的特性，从而使计算节点能够快速预热提高工作效率。
如图7所示，在上述实施例二的基础上，可选的，所述主用协调节点600还可以包括：
第二判断模块605，用于当所述第一判断模块604判断结果为是时，根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；
对应地，所述执行模块606可以用于当所述第二判断模块605判断结果为是时，通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
如图8所示，进一步可选的，所述主用协调节点600还可以包括：
监控模块607，用于实时监控所述主用协调节点的空闲资源，当所述主用协调节点的空闲资源小于预设阈值Z，则通过所述执行模块通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点，其中Z小于X。
当所述执行模块606通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，若所述主用协调节点600收到新产生的脏页数据，则所述执行模块606还用于：优先淘汰所述主用协调节点上缓存的干净页数据。
或者，当所述执行模块606通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，若所述主用协调节点600收到新产生的脏页数 据，且所述新产生的脏页数据的数据量大于所述主用协调节点600上缓存的干净页数据的数据量；则所述执行模块606还用于：优先淘汰所述主用协调节点600上缓存的干净页数据，并将所述新产生的脏页数据存储到所述数据库集群的存储设备上(例如数据库集群100中的共享存储磁盘142)。
如图9所示，在上述实施例的基础上，本发明提供另一个实施例，在这个实施例里，所述主用协调节点600还包括：缓存策略生成模块605’：用于根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点600的缓存策略；所述主用协调节点600的缓存策略包括：所述多个计算节点中各个计算节点向所述主用协调节点600上传干净页数据的策略；
对应地，所述执行模块606还用于：通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点600。
进一步可选的，所述缓存策略生成模块605’具体可以用于：
根据所述多个计算节点中各个计算节点分别对应的状态信息，计算所述多个计算节点中每一个计算节点当前单位时间脏页数据的产生比例；
根据所述每一个计算节点当前单位时间脏页数据的产生比例，生成主用协调节点的缓存策略。
实施例三，如图1所示，在图1的数据库集群100的硬件设备基础上，
本发明实施例提供一种数据库集群，包括多个计算节点如122、124、126、128等和一个主用协调节点102：
所述主用协调节点102获取所述多个计算节点如122、124、126、128等中各个计算节点分别对应的状态信息，并根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点102是否存在预设阈值X的空闲资 源；若是，则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点102；其中，所述状态信息用于表示计算节点的运行负载状态；
所述各个计算节点接收所述主用协调节点102发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则将所述产生的干净页数据上传到所述主用协调节点102。
可选的，本发明实施例还提供一种数据库集群，包括包括多个计算节点如122、124、126、128等和一个主用协调节点102：
所述主用协调节点102获取所述多个计算节点如122、124、126、128等中各个计算节点分别对应的状态信息，并根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点102是否存在预设阈值X的空闲资源；若是之后，再根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点102；
所述各个计算节点接收所述主用协调节点102发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则上传到所述主用协调节点102。
在上述实施例的基础上，进一步可选的，本发明实施例还提供另一种数据库集群，包括包括多个计算节点如122、124、126、128等和一个主用协调节点102：
所述主用协调节点102判断所述多个计算节点如122、124、126、128中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是则根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点102的缓存策略；并通知所述各个计算节点将后续产生的干净页数据根据所述主用 协调节点的缓存策略上传到所述主用协调节点102；
对应地，所述各个计算节点接收所述主用协调节点102发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则将所述产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点102。
值得说明的是，如图1所示，所述数据库集群还可以包括备用协调节点104，所述备用协调节点104用于为所述主用协调节点102进行数据备份，以使得所述备用协调节点104上的内容与所述主用协调节点102上的内容相同。
实施例四
图10是本发明第六实施例的应用于数据库系统的图1中的主用协调节点102或图6至图9中的主用协调节点600的结构示意图。值得说明的是，主用协调节点102和主用协调节点600是站在不同角度描述的同一个物体，其内部的结构是相同的。以下以主用协调节点102为例进行说明。主协调节点102包括收发器720，处理器(中央处理单元Central Processing Unit，CPU)730、存储器740以及总线750。收发器720用于接收数据库集群系统内的各个计算节点发送的事务日志。事务日志为记录了对应事务的数据库日志。具体地，收发器720用于通过高速网络接收数据库系统内的计算节点发送的事务日志。高速网络指传输时延不大于纳秒(ns)级的网络。例如，高速网络为IB。
收发器720用于获取所述多个计算节点中各个计算节点分别对应的状态信息，所述状态信息用于表示计算节点的运行负载状态；
处理器730用于根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；若是则通过收发器720通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
在另一个可选的实施例中，所述处理器730在根据多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；若是之后，还用于根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传 小于预设阈值X的脏页数据；
若是，则通过收发器720通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
在又一个可选的实施例中，在通过收发器720通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点之后，所述处理器730还用于实时监控所述主用协调节点的空闲资源，当所述主用协调节点的空闲资源小于预设阈值Z，则通过收发器720通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点，其中Z小于X。
在所述通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，若所述主用协调节点收到新产生的脏页数据，处理器730还用于：优先淘汰所述主用协调节点的存储器740上缓存的干净页数据；
或者若所述主用协调节点收到新产生的脏页数据，且所述新产生的脏页数据的数据量大于所述主用协调节点上缓存的干净页数据的数据量；处理器730还用于优先淘汰所述主用协调节点的存储器740上缓存的干净页数据，并将所述新产生的脏页数据存储到所述数据库集群的存储设备(共享存储磁盘)上。
在再一个可选的实施例中，在根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是之后，所述处理器730还用于根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略；所述主用协调节点的缓存策略包括：所述多个计算节点中各个计算节点向所述主用协调节点上传干净页数据的策略；并通过收发器720通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点102上。
在主用协调节点102中，所述总线750用于处理器730，收发器720以及存储器740之间的通讯。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。 其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
以上所揭露的仅为本发明较佳实施例而已，当然不能以此来限定本发明之权利范围，因此依本发明权利要求所作的等同变化，仍属本发明所涵盖的范围。

Claims (15)

  1. 一种数据库集群的资源协调方法，所述数据库集群包括多个计算节点和一个主用协调节点，其特征在于：
    获取所述多个计算节点中各个计算节点分别对应的状态信息，所述状态信息用于表示计算节点的运行负载状态；
    根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；若是
    则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点；或者，
    根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是
    则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
  2. 根据权利要求1所述的方法，其特征在于，在所述通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点之后，所述方法还包括：
    实时监控所述主用协调节点的空闲资源，当所述主用协调节点的空闲资源小于预设阈值Z，则通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点，其中Z小于X。
  3. 根据权利要求2所述的方法，其特征在于，在所述通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，所述方法还包括：
    若所述主用协调节点收到新产生的脏页数据；
    则优先淘汰所述主用协调节点上缓存的干净页数据。
  4. 根据权利要求2所述的方法，其特征在于，在所述通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，所述方法还包括：
    若所述主用协调节点收到新产生的脏页数据，且所述新产生的脏页数据的数据量大于所述主用协调节点上缓存的干净页数据的数据量；
    则优先淘汰所述主用协调节点上缓存的干净页数据，并将所述新产生的脏 页数据存储到所述数据库集群的存储设备上。
  5. 根据权利要求1所述的方法，其特征在于，在所述根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是之后，所述方法还包括：
    则根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略；所述主用协调节点的缓存策略包括：所述多个计算节点中各个计算节点向所述主用协调节点上传干净页数据的策略；
    对应地，所述通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点包括：
    通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略包括：
    根据所述多个计算节点中各个计算节点分别对应的状态信息，计算所述多个计算节点中每一个计算节点当前单位时间脏页数据的产生比例；
    根据所述每一个计算节点当前单位时间脏页数据的产生比例，生成主用协调节点的缓存策略。
  7. 一种应用于数据库集群的主用协调节点，其特征在于，所述主用协调节点包括：
    获取模块，用于获取所述数据库集群中多个计算节点中各个计算节点分别对应的状态信息，所述状态信息用于表示计算节点的运行负载状态；
    第一判断模块，用于根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；或者，当判断存在预设阈值X的空闲资源时，进一步根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设 时间内上传小于预设阈值X的脏页数据；
    执行模块，用于当所述第一判断模块判断结果为是时，通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。
  8. 根据权利要求7所述的主用协调节点，其特征在于，所述主用协调节点还包括监控模块，用于实时监控所述主用协调节点的空闲资源，当所述主用协调节点的空闲资源小于预设阈值Z，则通过所述执行模块通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点，其中Z小于X。
  9. 根据权利要求8所述的主用协调节点，其特征在于，
    当所述执行模块通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，若所述主用协调节点收到新产生的脏页数据，则
    所述执行模块还用于：优先淘汰所述主用协调节点上缓存的干净页数据。
  10. 根据权利要求8所述的主用协调节点，其特征在于，
    当所述执行模块通知所述各个计算节点不再将产生的干净页数据上传到所述主用协调节点之后，若所述主用协调节点收到新产生的脏页数据，且所述新产生的脏页数据的数据量大于所述主用协调节点上缓存的干净页数据的数据量；则
    所述执行模块还用于：则优先淘汰所述主用协调节点上缓存的干净页数据，并将所述新产生的脏页数据存储到所述数据库集群的存储设备上。
  11. 根据权利要求7所述的主用协调节点，其特征在于，当所述第二判断模块判断结果为是时，所述主用协调节点还包括：
    缓存策略生成模块：用于根据所述多个计算节点中各个计算节点分别对应 的状态信息，生成主用协调节点的缓存策略；所述主用协调节点的缓存策略包括：所述多个计算节点中各个计算节点向所述主用协调节点上传干净页数据的策略；
    对应地，所述执行模块用于：通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点。
  12. 根据权利要求11所述的主用协调节点，其特征在于，所述缓存策略生成模块具体用于：
    根据所述多个计算节点中各个计算节点分别对应的状态信息，计算所述多个计算节点中每一个计算节点当前单位时间脏页数据的产生比例；
    根据所述每一个计算节点当前单位时间脏页数据的产生比例，生成主用协调节点的缓存策略。
  13. 一种数据库集群，包括多个计算节点和一个主用协调节点，其特征在于：
    所述主用协调节点获取所述多个计算节点中各个计算节点分别对应的状态信息，并根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述主用协调节点是否存在预设阈值X的空闲资源；若是则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点；或者，若是则进一步根据所述多个计算节点中各个计算节点分别对应的状态信息，判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是则通知所述各个计算节点将后续产生的干净页数据上传到所述主用协调节点。其中，所述状态信息用于表示计算节点的运行负载状态；
    所述各个计算节点接收所述主用协调节点发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则将所述产生的干净页数据上传到所述主用协调节点。
  14. 根据权利要求13所述的数据库集群，其特征在于，所述主用协调节点判断所述多个计算节点中各个计算节点是否将在预设时间内上传小于预设阈值X的脏页数据；若是则根据所述多个计算节点中各个计算节点分别对应的状态信息，生成主用协调节点的缓存策略；并通知所述各个计算节点将后续产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点；
    对应地，所述各个计算节点接收所述主用协调节点发送的通知，当所述各个计算节点中任意一个计算节点产生干净页数据，则将所述产生的干净页数据根据所述主用协调节点的缓存策略上传到所述主用协调节点。
  15. 根据权利要求13或14所述的数据库集群，其特征在于，所述数据库集群还包括备用协调节点，用于为所述主用协调节点进行数据备份，以使得所述备用协调节点上的内容与所述主用协调节点上的内容相同。
PCT/CN2016/070052 2015-01-04 2016-01-04 一种数据库集群的资源协调方法、装置及系统 Ceased WO2016107610A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP16732876.4A EP3185148B1 (en) 2015-01-04 2016-01-04 Resource coordination method, device, and system for database cluster
US15/639,675 US10642822B2 (en) 2015-01-04 2017-06-30 Resource coordination method, apparatus, and system for database cluster

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510003943.4 2015-01-04
CN201510003943.4A CN104504147B (zh) 2015-01-04 2015-01-04 一种数据库集群的资源协调方法、装置及系统

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/639,675 Continuation US10642822B2 (en) 2015-01-04 2017-06-30 Resource coordination method, apparatus, and system for database cluster

Publications (1)

Publication Number Publication Date
WO2016107610A1 true WO2016107610A1 (zh) 2016-07-07

Family

ID=52945544

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/070052 Ceased WO2016107610A1 (zh) 2015-01-04 2016-01-04 一种数据库集群的资源协调方法、装置及系统

Country Status (4)

Country Link
US (1) US10642822B2 (zh)
EP (1) EP3185148B1 (zh)
CN (1) CN104504147B (zh)
WO (1) WO2016107610A1 (zh)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104504147B (zh) 2015-01-04 2018-04-10 华为技术有限公司 一种数据库集群的资源协调方法、装置及系统
CN107085577B (zh) * 2016-02-15 2021-01-05 华为技术有限公司 一种加载数据页的方法、计算节点以及协调节点
CN106227878B (zh) * 2016-08-03 2020-01-14 杭州数梦工场科技有限公司 一种搜索方法和装置
CN108121618B (zh) * 2016-11-28 2021-02-12 华为技术有限公司 一种修复数据的方法和装置
US10917496B2 (en) * 2017-09-05 2021-02-09 Amazon Technologies, Inc. Networked storage architecture
US11120082B2 (en) 2018-04-18 2021-09-14 Oracle International Corporation Efficient, in-memory, relational representation for heterogeneous graphs
CN109495588A (zh) * 2018-12-25 2019-03-19 鼎信信息科技有限责任公司 前置机的任务处理方法、装置、计算机设备和存储介质
CN112015807B (zh) * 2019-05-31 2024-07-02 阿里巴巴集团控股有限公司 数据同步的处理方法、装置、电子设备及计算机存储介质
CN111045845B (zh) * 2019-11-29 2021-09-17 苏州浪潮智能科技有限公司 一种数据回传方法、装置、设备及计算机可读存储介质
CN116302797A (zh) * 2023-02-03 2023-06-23 安芯网盾(北京)科技有限公司 一种服务端软件健康状态采集方法及系统
CN117407384A (zh) * 2023-10-26 2024-01-16 广州趣研网络科技有限公司 数据清洗方法、装置、计算机设备和存储介质
CN117938955B (zh) * 2024-03-21 2024-06-21 北京火山引擎科技有限公司 用于cdn中的数据预热方法、装置、设备和介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101778002A (zh) * 2010-02-02 2010-07-14 浪潮(北京)电子信息产业有限公司 一种大规模集群系统及其构建方法
CN102591679A (zh) * 2011-11-30 2012-07-18 中国科学院计算机网络信息中心 一种集群系统应用软件快速部署方法
US20130117526A1 (en) * 2011-11-08 2013-05-09 Sybase, Inc. Shared temporary storage management in a shared disk database cluster
CN104239227A (zh) * 2013-06-20 2014-12-24 杭州华三通信技术有限公司 一种虚拟化存储管理装置、本地虚拟存储管理装置及方法
CN104462225A (zh) * 2014-11-12 2015-03-25 华为技术有限公司 一种数据读取的方法、装置及系统
CN104504147A (zh) * 2015-01-04 2015-04-08 华为技术有限公司 一种数据库集群的资源协调方法、装置及系统

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2425045C (en) * 2003-04-08 2013-01-15 Ibm Canada Limited - Ibm Canada Limitee Method and system for executing a database query
US9027030B2 (en) * 2007-11-29 2015-05-05 Red Hat, Inc. Commit-one-phase distributed transactions with multiple starting participants
US20100023564A1 (en) * 2008-07-25 2010-01-28 Yahoo! Inc. Synchronous replication for fault tolerance
US8874961B2 (en) * 2010-03-22 2014-10-28 Infosys Limited Method and system for automatic failover of distributed query processing using distributed shared memory
US9607001B2 (en) * 2012-07-13 2017-03-28 Facebook, Inc. Automated failover of a metadata node in a distributed file system
US9678801B2 (en) * 2012-08-09 2017-06-13 International Business Machines Corporation Service management modes of operation in distributed node service management
CN103023970B (zh) * 2012-11-15 2015-07-22 中国科学院计算机网络信息中心 一种物联网海量数据存储方法及系统
CN103902617B (zh) * 2012-12-28 2017-06-09 华为技术有限公司 分布式数据库同步方法和系统
CN103365987B (zh) 2013-07-05 2017-04-12 北京人大金仓信息技术股份有限公司 一种基于共享磁盘架构的集群数据库系统及数据处理方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101778002A (zh) * 2010-02-02 2010-07-14 浪潮(北京)电子信息产业有限公司 一种大规模集群系统及其构建方法
US20130117526A1 (en) * 2011-11-08 2013-05-09 Sybase, Inc. Shared temporary storage management in a shared disk database cluster
CN102591679A (zh) * 2011-11-30 2012-07-18 中国科学院计算机网络信息中心 一种集群系统应用软件快速部署方法
CN104239227A (zh) * 2013-06-20 2014-12-24 杭州华三通信技术有限公司 一种虚拟化存储管理装置、本地虚拟存储管理装置及方法
CN104462225A (zh) * 2014-11-12 2015-03-25 华为技术有限公司 一种数据读取的方法、装置及系统
CN104504147A (zh) * 2015-01-04 2015-04-08 华为技术有限公司 一种数据库集群的资源协调方法、装置及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3185148A4 *

Also Published As

Publication number Publication date
EP3185148B1 (en) 2021-03-10
CN104504147B (zh) 2018-04-10
US20170308567A1 (en) 2017-10-26
EP3185148A1 (en) 2017-06-28
EP3185148A4 (en) 2017-11-29
US10642822B2 (en) 2020-05-05
CN104504147A (zh) 2015-04-08

Similar Documents

Publication Publication Date Title
CN104504147B (zh) 一种数据库集群的资源协调方法、装置及系统
US20220358106A1 (en) Proxy-based database scaling
CN102694868B (zh) 一种集群系统实现及任务动态分配方法
CN102307206B (zh) 基于云存储的快速访问虚拟机镜像的缓存系统的缓存方法
CN112597202B (zh) 数据查询方法、装置、计算机设备和存储介质
RU2568292C2 (ru) Система и способ выбора синхронного или асинхронного межпроцессного взаимодействия
WO2020134364A1 (zh) 一种虚拟机迁移方法、云计算管理平台和存储介质
CN108900626B (zh) 一种云环境下数据存储方法、装置及系统
CN111787055A (zh) 一种基于Redis且面向事务机制和多数据中心的数据分发方法和系统
US11080207B2 (en) Caching framework for big-data engines in the cloud
EP4492236A1 (en) Data processing method and apparatus for fusion system, device and system
CN102932424A (zh) 一种分布式并行文件系统缓存数据同步的方法及系统
US10033620B1 (en) Partitioned performance adaptive policies and leases
CN110825704A (zh) 一种读数据方法、写数据方法及服务器
TWI498758B (zh) 資料庫引擎調流
CN114296891A (zh) 任务的调度方法、系统、计算设备、存储介质及程序产品
CN110049091A (zh) 数据存储方法及装置、电子设备、存储介质
CN115587118A (zh) 任务数据的维表关联处理方法及装置、电子设备
CN115858423A (zh) 缓存淘汰策略的切换方法、装置、电子设备及存储介质
CN118467243A (zh) 一种数据库备份方法、服务器及共享内存设备
CN107943615B (zh) 基于分布式集群的数据处理方法与系统
CN105868002A (zh) 一种用于在分布式计算中处理重发请求的方法与设备
CN114443686A (zh) 一种基于关系型数据的压缩图构建方法及装置
CN107566341A (zh) 一种基于联邦分布式文件存储系统的数据持久化存储方法及系统
CN117851359A (zh) 基于数据库集群的数据处理方法、装置和计算机设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16732876

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2016732876

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2016732876

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE