WO2024125251A1 - 资源分配的方法及装置 - Google Patents

资源分配的方法及装置 Download PDF

Info

Publication number
WO2024125251A1
WO2024125251A1 PCT/CN2023/133357 CN2023133357W WO2024125251A1 WO 2024125251 A1 WO2024125251 A1 WO 2024125251A1 CN 2023133357 W CN2023133357 W CN 2023133357W WO 2024125251 A1 WO2024125251 A1 WO 2024125251A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
resource
scale
tasks
resource allocation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/133357
Other languages
English (en)
French (fr)
Inventor
孙楚旻
周李
任玉鑫
樊瑞
孙杰
陈东辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP23902450.8A priority Critical patent/EP4625168A4/en
Publication of WO2024125251A1 publication Critical patent/WO2024125251A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/501—Performance criteria
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5017—Task decomposition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5019—Workload prediction

Definitions

  • Embodiments of the present application relate to the field of data processing technology, and more specifically, to a method and device for resource allocation.
  • Computing resource allocation is one of the important factors that affect the performance of a task.
  • the computing resource allocation plan is determined based on a general algorithm.
  • the general algorithm cannot perceive the characteristics of the task, that is, it cannot allocate computing resources according to the characteristics of different tasks, and it is difficult to achieve the best match between tasks and computing resources.
  • the computing resource allocation plan is determined by the user, for example, the user specifies the computing resources required for the task, or the user customizes the computing resource allocation plan for the task.
  • the method of manually determining the resource allocation plan has high requirements for the user, and this method is inefficient and difficult to meet the operation requirements.
  • the embodiments of the present application provide a method and device for resource allocation, which is conducive to allocating more reasonable computing resources to multiple tasks, thereby improving the overall operating performance of multiple tasks.
  • a method for resource allocation comprising: obtaining running performance data of each task among a plurality of tasks, the running performance data of each task among the plurality of tasks including the running performance of each task under a sample resource scale corresponding to each task; determining, based on the running performance data of each task among the plurality of tasks, the running performance of each task among the plurality of tasks when monopolizing a candidate resource scale corresponding to the plurality of tasks; and determining, based on the running performance of each task among the plurality of tasks when monopolizing a candidate resource scale corresponding to each task, a target resource corresponding to each task among the plurality of tasks.
  • tasks can be analyzed based on the running performance data of multiple tasks to identify the behavioral characteristics of each task itself, for example, the running performance of each task when it monopolizes resources, and the target resources corresponding to each task in the multiple tasks can be determined based on the behavioral characteristics of the task itself, which is conducive to improving the matching degree between tasks and resources, thereby improving the overall running performance of the multiple tasks.
  • the running performance of a task may include the running time of the task, the response delay of the task, the resource utilization of the task, or data collected by a hardware performance monitoring unit (PMU) counter.
  • PMU hardware performance monitoring unit
  • the running performance of a task under a resource scale may include at least one of the following: the running performance of the task when the resource scale is exclusively occupied; the running performance of the task when the resource scale is not exclusively occupied.
  • the sample resource scale corresponding to each task may include one sample resource scale or multiple sample resource scales.
  • sample resource sizes corresponding to different tasks can be the same or different.
  • the resource may include a process resource.
  • One unit resource may be one process.
  • the resource scale may be the number of processes.
  • the running performance data of at least one task among the multiple tasks includes the running performance of the at least one task under a sample resource scale that is not exclusive to the at least one task.
  • the operating performance of the task when it monopolizes resources can be determined based on the operating performance of the task when it does not monopolize resources, reducing the restrictions on the operating performance data of the multiple tasks.
  • the operating performance of multiple tasks obtained in the scheme of the embodiment of the present application can include both the operating performance when the task monopolizes resources and the operating performance when the task does not monopolize resources, which increases the amount of data of the acquired operating performance data and lays a data foundation for subsequent processing, thereby facilitating a more accurate analysis of the multiple tasks and achieving reasonable resource allocation.
  • the target resource corresponding to each task includes one or more processes corresponding to each task, and the one or more processes are respectively bound to one or more resource slots.
  • Slots and processor cores can be one-to-one. One or more processes are bound to one or more slots, which can also be understood as a One or more processes are respectively bound to one or more processor cores.
  • the process allocation can be realized globally, that is, the slot allocation can be realized globally, which is conducive to obtaining the optimal resource allocation scheme.
  • the scheme of the embodiment of the present application binds the process to the slot, realizes the refined computing power allocation, and does not require the secondary scheduling of the computing power node kernel, that is, there is no need for the computing power node to schedule the processor core for the process, avoiding the overhead caused by the secondary scheduling.
  • the multiple processes are multiple continuous processes
  • the multiple slots are multiple continuous slots
  • the multiple slots being multiple consecutive slots means that the numbers of the multiple slots are consecutive.
  • allocating continuous slots to each task is beneficial to reducing the communication cost during the execution of each task and further improving the running performance of the task.
  • multiple tasks include a first task
  • determining the operating performance of the first task when exclusively occupying a candidate resource scale corresponding to the first task based on the operating performance data of the first task includes: constructing a first performance model of the first task based on the operating performance data of the first task, wherein the first performance model of the first task is used to predict the operating performance of the first task when exclusively occupying a resource scale input into the performance model of the first task; determining the operating performance of the first task when exclusively occupying a first candidate resource scale corresponding to the first task based on the first performance model of the first task, and the candidate resource scale corresponding to the first task includes the first candidate resource scale corresponding to the first task.
  • the first task may be any one of the multiple tasks.
  • performance modeling of the task is performed through the running performance data of the task, which is conducive to accurate performance analysis of the task, thereby providing a basis for the reasonable allocation of subsequent computing resources.
  • the plurality of tasks include a second task, and determining the operating performance of the second task in a case where the candidate resource scale corresponding to the second task is exclusively occupied according to the operating performance data of the second task includes: determining the operating performance of the second task in a case where the second candidate resource scale corresponding to the second task is exclusively occupied according to the fluctuation coefficient model of the second task and the operating performance data of the second task.
  • the candidate resource scale corresponding to the second task includes the second candidate resource scale corresponding to the second task.
  • the operating performance data of the second task includes the operating performance of the second task under the sample resource allocation scheme corresponding to the second task.
  • the second candidate resource scale is the sample resource scale indicated by the sample resource allocation scheme corresponding to the second task.
  • the fluctuation coefficient model of the second task is used to predict the fluctuation coefficient of the second task corresponding to the resource allocation scheme input to the fluctuation coefficient model of the second task.
  • the fluctuation coefficient of the second task corresponding to the sample resource allocation scheme corresponding to the second task is used to indicate the difference between the operating performance of the second task under the sample resource allocation scheme corresponding to the second task and the operating performance of the second task in a case where the sample resource allocation scheme corresponding to the second task is exclusively occupied.
  • the performance inversion of the running performance data of the task under the sample resource allocation scheme is performed based on the fluctuation coefficient of the task to predict the running performance of the task when it monopolizes resources, which is conducive to realizing accurate performance analysis of each task, thereby providing a basis for the subsequent reasonable allocation of computing resources.
  • the candidate resource scale corresponding to each task is a partial resource scale among the candidate resource scales corresponding to each task, and the partial resource scale among the candidate resource scales corresponding to each task and the operating performance of each task when exclusively occupying the partial resource scale among all candidate resource scales corresponding to each task meet the first preset condition.
  • part of the resource scale can be determined from the candidate resource scale based on the first preset condition, reducing the number of combinations of the candidate resource scales, that is, reducing the search space in the subsequent target resource allocation process, which is conducive to improving the efficiency of resource allocation.
  • the part of the resource scale is determined based on the running performance of the task when the task exclusively occupies the resource scale corresponding to the task and the resource scale, which is conducive to avoiding allocating too many resources to the task in the subsequent resource allocation, thereby causing resource waste.
  • the first preset condition includes: the ratio of the operating performance of each task when monopolizing part of the resource scale in the candidate resource scale corresponding to each task and the part of the resource scale in the candidate resource scale corresponding to each task is greater than or equal to a first threshold, and the first threshold is the threshold of the task's utilization efficiency of the resource scale.
  • the scale of this part of resources can be determined based on the ratio between the running performance of the task when it monopolizes the resource scale corresponding to the task, that is, the resource efficiency of the task, and the candidate resource scale corresponding to the higher resource efficiency is used as the candidate resource scale of this part, so as to avoid allocating too many resources to the task in the subsequent resource allocation, thereby causing resource waste.
  • determining the target resource corresponding to each of the multiple tasks according to the running performance of each of the multiple tasks when the target resource scale corresponding to each task is exclusively occupied includes: determining the target candidate resource allocation scheme under multiple resource scale combinations according to the running performance of each of the multiple tasks when the target resource scale corresponding to each task is exclusively occupied by a part of the candidate resource scale corresponding to each task, wherein the multiple resource scale combinations are based on the part of the candidate resource scale corresponding to each task.
  • a combination of resource scales is determined, and a target candidate resource allocation scheme under each resource scale combination among multiple resource scale combinations indicates a target candidate resource corresponding to each of multiple tasks under each resource scale combination; a fluctuation coefficient of multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination among multiple resource scale combinations is determined, and the fluctuation coefficient of multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination is respectively used to indicate the difference between the running performance of multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination and the running performance of multiple tasks when the resource scale corresponding to each task in each resource scale combination is exclusively occupied; the running performance of multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination is predicted according to the fluctuation coefficient of multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination; and the target resource corresponding to each of multiple tasks is determined from the target candidate resource allocation schemes under the multiple resource scale combinations according to the running performance of multiple tasks executed in parallel under the target candidate resource allocation schemes under the multiple resource scale combinations.
  • the target candidate resource allocation scheme under each resource scale combination is determined based on the operating performance when multiple tasks monopolize resources, and the fluctuation coefficient of each task is calculated based on the target candidate resource allocation scheme, so as to determine the operating performance of the multiple tasks when executed in parallel, and then determine the target resource corresponding to each task in the multiple tasks.
  • the target candidate resource allocation scheme under each resource scale combination is determined based on the operating performance when multiple tasks monopolize resources, and the fluctuation coefficient of each task is calculated based on the target candidate resource allocation scheme, so as to determine the operating performance of the multiple tasks when executed in parallel, and then determine the target resource corresponding to each task in the multiple tasks.
  • the scheme of the embodiment of the present application restores the operating performance of each task when executed in parallel based on the fluctuation coefficient, which is more in line with the actual operating conditions, and is conducive to improving the prediction accuracy of the overall performance, thereby further improving the rationality of resource allocation.
  • target candidate resource allocation schemes under multiple resource scale combinations are determined according to the operating performance of each task among the multiple tasks when it exclusively occupies a portion of the resource scale among the candidate resource scales corresponding to each task, including: determining the target candidate resource allocation scheme under each resource scale combination from the candidate resource allocation schemes under each resource scale combination according to the operating performance of each task among the multiple tasks when it exclusively occupies a portion of the resource scale among the candidate resource scales corresponding to each task, the candidate resources corresponding to each task indicated by the candidate resource allocation schemes under each group of resource scale combinations are continuous, and the overall operating performance of the multiple tasks under the target candidate resource allocation scheme under each resource scale combination when they exclusively occupy the resource scale corresponding to each task in each resource scale combination is better than the overall operating performance of the multiple tasks under other candidate resource allocation schemes under each resource scale combination when they exclusively occupy the resource scale corresponding to each task in each resource scale combination.
  • the fluctuation coefficients of multiple tasks corresponding to the target candidate resource allocation schemes under each resource scale combination in multiple resource scale combinations are determined, including: constructing fluctuation coefficient models for multiple tasks based on the operating performance data of the multiple tasks, the fluctuation coefficient models for the multiple tasks are respectively used to predict the fluctuation coefficients of the multiple tasks corresponding to the resource allocation schemes input into the fluctuation coefficient models of the multiple tasks; and determining the fluctuation coefficients of the multiple tasks corresponding to the target candidate resource allocation schemes under each resource scale combination in the multiple resource scale combinations according to the fluctuation coefficient models of the multiple tasks.
  • the multiple tasks include tasks corresponding to multiple applications, and each application in the multiple applications corresponds to one task.
  • the multiple tasks include tasks corresponding to multiple functional modules in an application, and each of the multiple functional modules corresponds to one task.
  • a resource allocation device comprising an acquisition unit for acquiring operating performance data of each of a plurality of tasks, wherein the operating performance data of each of the plurality of tasks includes the operating performance of each task under a sample resource scale corresponding to each task; a processing unit for: determining, based on the operating performance data of each of the plurality of tasks, the operating performance of each of the plurality of tasks when the scale of candidate resources corresponding to each task is exclusively occupied; and determining, based on the operating performance of each of the plurality of tasks when the scale of candidate resources corresponding to each task is exclusively occupied, a target resource corresponding to each of the plurality of tasks.
  • the target resource corresponding to each task includes one or more processes corresponding to each task, and the one or more processes are respectively bound to one or more slots.
  • the multiple processes are multiple continuous processes
  • the multiple slots are multiple continuous slots
  • the multiple tasks include a first task
  • the processing unit is specifically used to: construct a first performance model of the first task based on the running performance data of the first task, wherein the first performance model of the first task is used to predict the running performance of the first task when the resource scale of the performance model of the first task is exclusively input; determine the running performance of the first task when the first candidate resource scale corresponding to the first task is exclusively occupied according to the first performance model of the first task, and the candidate resource scale corresponding to the first task includes the first candidate resource scale corresponding to the first task.
  • the processing unit is specifically configured to: the candidate resource scale corresponding to each task is a partial resource scale among all candidate resource scales corresponding to each task, and the partial resource scale among all candidate resource scales corresponding to each task is The running performance of each task in the case of monopolizing a part of the candidate resource scales corresponding to each task satisfies the first preset condition.
  • the first preset condition includes: the ratio of the operating performance of each task when monopolizing part of the resource scale in the candidate resource scale corresponding to each task and the part of the resource scale in the candidate resource scale corresponding to each task is greater than or equal to a first threshold, and the first threshold is the threshold of the task's utilization efficiency of the resource scale.
  • the processing unit is specifically used to: determine the target candidate resource allocation schemes under multiple resource scale combinations according to the operating performance of each task in the case of monopolizing a portion of the resource scale in the candidate resource scale corresponding to each task, the multiple resource scale combinations are determined based on the combination of the portion of the resource scale in the candidate resource scale corresponding to each task, and the target candidate resource allocation scheme under each resource scale combination in the multiple resource scale combinations indicates the target candidate resource corresponding to each task in the multiple tasks under each resource scale combination; determine the fluctuation coefficients of the multiple tasks corresponding to the target candidate resource allocation schemes under each resource scale combination in the multiple resource scale combinations, and the target candidate resource allocation scheme under each resource scale combination indicates the target candidate resource corresponding to each task in the multiple tasks under each resource scale combination.
  • the fluctuation coefficients of the multiple tasks corresponding to the selected resource allocation scheme are respectively used to indicate the difference between the running performance of the multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination and the running performance of the multiple tasks when the resource scale corresponding to each task in each resource scale combination is exclusively occupied; the running performance of the multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination is predicted according to the fluctuation coefficients of the multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination; the target resource corresponding to each task in the multiple tasks is determined from the target candidate resource allocation schemes under multiple resource scale combinations according to the running performance of the multiple tasks executed in parallel under the target candidate resource allocation schemes under multiple resource scale combinations.
  • the processing unit is specifically used to: determine the target candidate resource allocation scheme under each resource scale combination from the candidate resource allocation schemes under each resource scale combination according to the operating performance of each task in the case of exclusively occupying a part of the resource scale among the candidate resource scales corresponding to each task, the candidate resources corresponding to each task indicated by the candidate resource allocation schemes under each group of resource scale combinations are continuous, and the overall operating performance of the multiple tasks in the case of exclusively occupying the resource scale corresponding to each task in each resource scale combination under the target candidate resource allocation scheme under each resource scale combination is better than the overall operating performance of the multiple tasks in the case of exclusively occupying the resource scale corresponding to each task in each resource scale combination under other candidate resource allocation schemes under each resource scale combination.
  • the processing unit is specifically used to: construct fluctuation coefficient models for multiple tasks based on the operating performance data of multiple tasks, the fluctuation coefficient models for multiple tasks are respectively used to predict the fluctuation coefficients of multiple tasks corresponding to the resource allocation schemes input into the fluctuation coefficient models of multiple tasks; and determine the fluctuation coefficients of multiple tasks corresponding to the target candidate resource allocation schemes under each resource scale combination in multiple resource scale combinations according to the fluctuation coefficient models of multiple tasks.
  • the multiple tasks include tasks corresponding to multiple applications, and each application in the multiple applications corresponds to one task.
  • the multiple tasks include tasks corresponding to multiple functional modules in an application, and each of the multiple functional modules corresponds to one task.
  • a device for resource allocation comprising a processor and a memory, and optionally, an input/output interface, wherein the processor is used to control the input/output interface to send and receive information, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device executes the method described in the first aspect or any possible implementation of the first aspect.
  • the processor may be a general-purpose processor, which may be implemented by hardware or software.
  • the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor, which is implemented by reading software code stored in a memory, which may be integrated in the processor or located outside the processor and exist independently.
  • a computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory.
  • the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method in the first aspect and any one of the implementations of the first aspect.
  • a computer-readable medium comprising computer program instructions.
  • the computing device cluster executes the method in the first aspect and any one of the implementations of the first aspect.
  • a computer program product comprising instructions.
  • the computing device cluster executes the method in the above-mentioned first aspect and any one of the implementations of the first aspect.
  • these computer readable storages include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and hard drive.
  • ROM read-only memory
  • PROM programmable ROM
  • EPROM erasable PROM
  • Flash memory electrically EPROM (EEPROM)
  • hard drive electrically EPROM
  • the above-mentioned storage medium may specifically be a non-volatile storage medium.
  • FIG1 is a schematic diagram of different levels of computing resource allocation.
  • FIG. 2 is a schematic diagram of a method for allocating computing resources.
  • FIG3 is a schematic diagram of a method for allocating computing resources according to an embodiment of the present application.
  • FIG4 is a schematic block diagram of a resource allocation device according to an embodiment of the present application.
  • FIG5 is a schematic flowchart of a resource allocation method according to an embodiment of the present application.
  • FIG6 is a schematic flowchart of another resource allocation method according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a processing flow of a performance matrix according to an embodiment of the present application.
  • FIG8 is a schematic diagram of a processing flow of a box packing algorithm according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a resource allocation scheme according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of another resource allocation solution according to an embodiment of the present application.
  • FIG. 11 is a schematic block diagram of a resource allocation device according to an embodiment of the present application.
  • FIG. 12 is a schematic block diagram of a computing device according to an embodiment of the present application.
  • FIG. 13 is a schematic block diagram of a computing device cluster according to an embodiment of the present application.
  • FIG. 14 is a schematic block diagram of a connection relationship between computing devices according to an embodiment of the present application.
  • references to "one embodiment” or “some embodiments” etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application.
  • the phrases “in one embodiment”, “in some embodiments”, “in some other embodiments”, “in some other embodiments”, etc. appearing in different places in this specification do not necessarily all refer to the same embodiment, but mean “one or more but not all embodiments", unless otherwise specifically emphasized in other ways.
  • the terms “including”, “comprising”, “having” and their variations all mean “including but not limited to”, unless otherwise specifically emphasized in other ways.
  • At least one means one or more
  • plural means two or more.
  • “And/or” describes the association relationship of associated objects, indicating that three relationships may exist.
  • a and/or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural.
  • the character “/” generally indicates that the previous and next associated objects are in an “or” relationship.
  • “At least one of the following” or similar expressions refers to any combination of these items, including any combination of single or plural items.
  • At least one of a, b, or c can mean: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple.
  • Parallel applications refer to the parallel execution of different functional components within an application.
  • the component can also be Process different data to achieve parallelism within components.
  • Parallel applications are conducive to the rational use of computing resources, thereby improving the operating performance of applications.
  • an application can be abstracted into an expanded computational graph, which is clustered into several clusters, or can also be called a functional module.
  • Each functional module can be used to implement a certain part of the application's functions. Different functional modules can be executed simultaneously.
  • a module may also be referred to as a functional module, a component, a functional component or a component module.
  • the communication within the module is relatively close, while the communication between modules is relatively sparse.
  • the execution of an application can be abstracted as the allocation of tasks on computing resources and the maintenance of task execution.
  • Parallel applications can decompose an application into multiple subtasks, which can be assigned to different processors. Different processors can process the multiple subtasks simultaneously, thereby speeding up the execution of the application.
  • the performance matrix of an application can be used to represent the performance of different functional modules in the application under different resource allocation schemes.
  • the performance matrix can be used to represent the time required for different functional modules to run under different resource allocation schemes.
  • the multiple tasks may be multiple applications.
  • the solution of the embodiment of the present application can be used to allocate resources to the multiple applications.
  • one or more users can submit multiple job tasks to the computing system, and the system manages and runs these tasks in a unified manner.
  • the computing system can be called a batch job processing system or a batch processing system.
  • the solution of the embodiment of the present application can be applied to the batch processing system to allocate computing resources to the multiple job tasks.
  • multiple users submit multiple job tasks. These multiple job tasks have the characteristics of consuming a lot of computing power, long running time, and high parallelism.
  • the multiple job tasks can be deployed to limited computing nodes (such as one computing node) as much as possible.
  • the solution of the embodiment of the present application can deploy these multiple job tasks to the specified computing nodes, while optimizing the overall operating performance of the multiple job tasks, for example, the overall running time of the multiple job tasks.
  • multiple users submit multiple job tasks.
  • the resources that can be allocated to the multiple job tasks in the current system are limited.
  • the solution of the embodiment of the present application can allocate corresponding resources to the multiple job tasks from the limited resources, while optimizing the overall performance of the multiple job tasks, for example, the overall running time of the multiple job tasks.
  • the multiple tasks may be multiple subtasks of a parallel application.
  • the solution of the embodiment of the present application may be applied to a resource allocation scenario of a parallel application, and is used to allocate computing resources to multiple subtasks of a parallel application.
  • the solution of the embodiments of the present application can be applied to parallel application scenarios such as scientific computing or high performance computing (HPC).
  • HPC high performance computing
  • the solution of the embodiments of the present application is particularly suitable for large-scale parallel application scenarios, for example, large-scale scientific computing, big data processing, large-scale graph computing and other parallel application scenarios.
  • the community earth system model is one of the applications of high-performance computing and can be used to simulate long-term climate change.
  • the system is composed of modules responsible for various subsystems that are coupled to each other.
  • the subsystems include atmosphere (ATM), ocean (OCN), sea ice (ICE), land (LND), river runoff (ROF), land ice (GLC), ocean wave (WAV), coupler (CPL), etc.
  • Modules can exchange data through multipoint interface (MPI).
  • MPI multipoint interface
  • the operation performance of the coupled mode is an important indicator for measuring HPC performance.
  • Computing resource allocation is an important factor in improving the performance of tasks.
  • Figure 1 shows a schematic diagram of different levels of computing resource allocation.
  • Computing resource allocation for parallel applications usually considers four levels: application layer, data layer, system execution layer, and hardware computing layer.
  • the application layer abstracts multiple functional modules from a functional perspective.
  • the multiple functional modules correspond to multiple tasks respectively.
  • the multiple tasks can be processed in parallel, that is, task parallelism.
  • There may be interactive relationships such as communication, cooperation, and dependence between multiple functional modules.
  • the data layer divides the processing data of a single functional module into multiple groups from a data perspective.
  • the multiple groups of data can be processed in parallel, that is, data parallelism.
  • the system execution layer includes execution abstractions provided by the operating system, such as execution abstractions such as processes or threads, which are used to carry and run specific functional modules to process specific data groups. Multiple processes can be processed in parallel, that is, process parallelism.
  • the hardware computing power layer includes hardware units that are specifically responsible for program execution, that is, computing resources, such as the CPU shown in Figure 1.
  • computing resources such as the CPU shown in Figure 1.
  • One or more execution abstractions in the system execution layer are mapped to computing resources, thereby ultimately completing the allocation of computing power resources.
  • the parallelism of the system execution layer is computing power. parallel.
  • computing resource allocation strategies are usually determined based on general algorithms or by users.
  • general algorithms cannot perceive the characteristics of tasks, that is, they cannot allocate computing resources according to the characteristics of different tasks, making it difficult to achieve the best match between tasks and computing resources.
  • the method of manually determining resource allocation solutions has high requirements for users, and this method is inefficient and difficult to meet operational requirements.
  • FIG2 shows a schematic flow chart of a method for allocating computing resources.
  • the client interface obtains the user's job requirements.
  • the computing resource management system generates a waiting queue based on the user's job requirements, and then allocates computing nodes to multiple tasks in the waiting queue.
  • the system can determine the computing resource allocation strategy through a general algorithm.
  • the user can indicate the computing resource allocation strategy through commands or plug-ins.
  • the system executes the computing resource allocation strategy to implement node allocation. For example, the user can specify different computing resource allocation strategies for different applications or customize the computing resource allocation strategy according to different applications, thereby improving the adaptability of the computing resource allocation strategy to the application.
  • the system sends multiple task instances to the computing node based on the computing resource allocation strategy, and the operating system (OS) in the computing node implements the scheduling and calculation of each task instance.
  • the operating system in the computing node schedules the processor in the computing node to calculate each task instance, or in other words, the operating system in the computing node allocates a processor (such as the CPU in FIG2) to each task instance, thereby implementing the calculation of each task instance.
  • the computing resource allocation strategy determined based on the general framework cannot perceive the behavioral characteristics of the application, and it is difficult to achieve the best match between the application and the underlying computing resources. If the running performance of the application is to be further improved, it is necessary to manually specify the computing resource allocation strategy or manually formulate the computing resource allocation strategy, which is inefficient and has high requirements on users.
  • an embodiment of the present application provides a method for resource allocation, which is conducive to allocating more reasonable computing resources to multiple tasks, thereby improving the overall operating performance of multiple tasks.
  • FIG3 shows a schematic flowchart of a method for allocating computing resources according to an embodiment of the present application.
  • the client interface can obtain the user's job requirements.
  • the computing resource management system generates a waiting queue based on the user's job requirements, and then allocates resources to multiple tasks in the waiting queue.
  • the system performs data collection operations to obtain the operating performance data of multiple tasks. Based on the operating performance data of multiple tasks, the multiple tasks are analyzed for performance, and resource allocation is implemented according to the results of the performance analysis.
  • the system can perform resource allocation globally to achieve fine-grained resource allocation, for example, as shown in FIG3, to achieve CPU-level computing power allocation.
  • the system can allocate task instances to corresponding processors. Compared with FIG2, there is no need for computing power nodes to schedule processors to implement task instances, avoiding the overhead caused by secondary scheduling.
  • the granularity of resource allocation in FIG3 is only an example, and the granularity of resource allocation can also be other granularities.
  • the CPU is used as a processor in FIG3 for example only, and does not limit the solution of the embodiment of the present application. In other implementations, other processors can also be used.
  • Fig. 4 shows a schematic block diagram of a resource allocation device 400 provided in an embodiment of the present application.
  • the resource allocation device 400 can be deployed in the computing power resource management system shown in Fig. 3 .
  • the resource allocation apparatus 400 may include an information collector 410 , a data fusion unit 420 , an application analyzer 430 , and a resource allocator 440 .
  • a waiting queue may be generated based on the job tasks submitted by the user.
  • the resource allocation device 400 may allocate computing resources to multiple tasks in the waiting queue.
  • the information collector 410 is used to obtain the information of the multiple tasks and the information of available resources.
  • the information collector 410 can also be used to obtain user information.
  • the data fusion unit 420 is used to obtain the operating performance data of the multiple tasks.
  • the application analyzer 430 is used to analyze the performance characteristics of the multiple tasks based on the running performance data of the multiple tasks.
  • the application analyzer 430 may be used to predict the running performance of the multiple tasks when they monopolize resources based on the running performance data of the multiple tasks.
  • the resource allocator 440 is used to predict the overall running performance of the multiple tasks under different resource allocation schemes according to the performance characteristics of the multiple tasks, so as to determine a target resource allocation scheme.
  • the task instance can be allocated to the corresponding target resource for processing.
  • the target resource allocation scheme may be used to indicate processes corresponding to the multiple tasks.
  • the processes may be bound to hardware processing units.
  • task instances can be assigned to corresponding hardware units.
  • the resource allocation device 400 is exemplarily described below by taking the multiple tasks as multiple subtasks of a parallel application as an example.
  • a waiting queue may be generated based on the application submitted by the user.
  • the multiple tasks in the waiting queue include multiple subtasks corresponding to the application.
  • the resource allocation device 400 may allocate computing resources to the multiple tasks in the waiting queue.
  • the information collector 410 is used to obtain relevant information of the application, including information of multiple tasks and available resources of the application.
  • the information collector 410 can also be used to obtain user information.
  • the data fusion unit 420 is used to obtain the operation performance data of the multiple application instances of the application.
  • the operation performance data of the multiple application instances includes the operation performance data of the multiple tasks.
  • the application analyzer 430 is used to analyze the performance characteristics of the application based on the running performance data of the multiple application instances.
  • the application analyzer 430 may be used to predict the running performance of the multiple tasks when they monopolize resources based on the running performance data of the multiple application instances.
  • the resource allocator 440 is used to predict the overall operating performance of the application under different resource allocation schemes according to the performance characteristics of the application, so as to determine a target resource allocation scheme.
  • the task instance can be allocated to the corresponding target resource for processing.
  • Fig. 5 shows a schematic flow chart of a method for resource allocation provided in an embodiment of the present application.
  • the method 500 shown in Fig. 5 can be performed by a resource allocation device.
  • the method shown in Fig. 5 can be performed by the device 400 shown in Fig. 4.
  • method 500 includes the following steps.
  • the running performance data of each task in the plurality of tasks includes the running performance of each task under the sample resource scale corresponding to each task.
  • step 530 can be understood as determining a target resource allocation scheme according to the running performance of the multiple tasks when the multiple tasks exclusively occupy the candidate resource scales corresponding to the multiple tasks.
  • the target resource allocation scheme is used to indicate the target resource corresponding to each of the multiple tasks.
  • tasks can be analyzed based on the running performance data of multiple tasks to identify the behavioral characteristics of each task itself, for example, the running performance of each task when it monopolizes resources, and the target resources corresponding to each task in the multiple tasks can be determined based on the behavioral characteristics of the task itself, which is conducive to improving the matching degree between tasks and resources, thereby improving the overall running performance of the multiple tasks.
  • method 500 may further include step 501 (not shown in the figure).
  • step 501 may be performed by the information collector 410 in FIG. 4 .
  • the information of the plurality of tasks may be used to indicate the plurality of tasks or the number of the plurality of tasks.
  • the information of available resources may be used to indicate the available resources or the scale of available resources.
  • the information of available resources may be represented by hardware parameters.
  • the scale of available resources may be the number of available computing nodes.
  • the computing power node may be a terminal device.
  • the computing power node may be a server.
  • the size of the available resources may be the number of available processors.
  • the processor may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP).
  • CPU central processing unit
  • GPU graphics processing unit
  • DSP digital signal processor
  • the scale of available resources may be the number of available processor cores.
  • a processor is a CPU and a processor core is a CPU core.
  • the method 500 may further include: obtaining other information related to resource allocation of the multiple tasks, such as information of users who submitted the multiple tasks.
  • the multiple tasks include tasks corresponding to multiple applications.
  • the multiple tasks and the multiple applications may correspond one to one, that is, each application may correspond to one task.
  • the plurality of applications may be indicated by at least one user.
  • the multiple tasks may include a compression task submitted by user #1 and a data processing task and a compression task submitted by user #2, that is, the multiple tasks may include two compression tasks and one data processing task.
  • One or more users can submit multiple job tasks to the computing system, and the system will manage and run these tasks.
  • the computing system can be called a batch job processing system or a batch processing system.
  • the solution of the embodiment of the present application can be applied to the resource allocation scenario of the batch processing system to allocate computing resources to the multiple job tasks.
  • the multiple tasks include tasks corresponding to multiple functional modules in an application.
  • the multiple tasks may include multiple subtasks of an application.
  • Each functional module corresponds to a task.
  • the application is a parallel application.
  • the multiple tasks may include tasks corresponding to the functional modules responsible for ATM, OCN, and ICE in the CESM.
  • a user may submit an application task to a computing system, and the application is a parallel application.
  • the solution of the embodiment of the present application may be applied to a resource allocation scenario of a parallel application, and is used to allocate computing resources to multiple subtasks of a parallel application.
  • the specific description of the application of the solution of the embodiment of the present application to the resource allocation scenario of a parallel application may refer to method 600 in the following text.
  • the multiple tasks may include tasks corresponding to application #1 and tasks corresponding to multiple functional modules in application #2, application #1 may be a non-parallel application, and application #2 may be a parallel application.
  • the running performance of a task may include the running time of the task, the response delay of the task, the resource utilization of the task, or data collected by a hardware performance monitoring unit (PMU) counter.
  • PMU hardware performance monitoring unit
  • the running performance of a task under a resource scale may include at least one of the following: the running performance of the task when the resource scale is exclusively occupied; the running performance of the task when the resource scale is not exclusively occupied.
  • the situation where the task exclusively occupies the resource scale refers to the situation where the resources allocated to the task are exclusively occupied by the task.
  • the situation that the task does not exclusively occupy the resource scale refers to the situation that at least part of the resources allocated to the task are shared with other tasks, that is, the task is executed in parallel with other tasks that share at least part of the resources with the task.
  • a task monopolizes the scale of resources corresponding to the task, which can be understood as only executing the task on the resources corresponding to the task.
  • a task does not monopolize the scale of resources corresponding to the task, which can be understood as executing the task and other tasks simultaneously on the resources corresponding to the task.
  • the task and other tasks compete for the resources corresponding to the task.
  • the scale of resources corresponding to the task that is monopolized by a task is referred to as the task's exclusive resources
  • the scale of resources corresponding to the task that is not monopolized by a task is referred to as the task's non-exclusive resources or the task's non-exclusive resources.
  • the running performance data of at least one task among the multiple tasks includes the running performance of the at least one task under the condition of non-exclusive sample resource scale corresponding to the at least one task.
  • the operating performance of the task when it monopolizes resources can be determined based on the operating performance of the task when it does not monopolize resources, reducing the restrictions on the operating performance data of the multiple tasks.
  • the operating performance of multiple tasks obtained in the scheme of the embodiment of the present application can include both the operating performance when the task monopolizes resources and the operating performance when the task does not monopolize resources, which increases the amount of data of the acquired operating performance data and lays a data foundation for subsequent processing, thereby facilitating a more accurate analysis of the multiple tasks and achieving reasonable resource allocation.
  • step 510 may include: obtaining operation performance data of multiple application instances of the application.
  • the operation performance data of the multiple application instances includes the operation performance data of the multiple tasks.
  • the running performance data of each application instance may include running performance data of part or all of the multiple tasks.
  • the sample resource scale corresponding to each task may include one sample resource scale or multiple sample resource scales.
  • the running performance data of each task includes the running performance of the task under at least one sample resource scale.
  • sample resource sizes corresponding to different tasks can be the same or different.
  • the following is an exemplary description of a method for determining the sample resource scale corresponding to a task.
  • the sample resource scale corresponding to the task may be determined by the user.
  • sample resource scale corresponding to the task may be randomly generated.
  • the sample resource scale corresponding to the task may be generated based on a default resource scale corresponding to the task.
  • the default resource scale corresponding to a task may be a resource scale pre-set for the task.
  • the default resource scale corresponding to a task may be determined based on a general algorithm.
  • the sample resource scale corresponding to a task may include a default resource scale corresponding to the task and a resource scale selected from resource scales surrounding the default resource scale.
  • the resource scale surrounding the default resource scale may be a resource scale whose difference with the default resource scale is less than or equal to a scale threshold. For example, if the default resource scale is 5 processes and the scale threshold is 3, then the resource scale surrounding the default resource scale is a resource scale within the range of [2,8], from which part or all are selected and used together with the default resource scale as the sample resource scale.
  • the scale threshold may be a pre-set fixed value. Alternatively, the scale threshold may also be a value determined according to the default resource scale. This is not limited in the embodiments of the present application.
  • sample resource scale corresponding to the task can also be determined by other methods.
  • the methods for determining the corresponding sample resource scales may be the same or different.
  • resources may also be referred to as computing power, computing power resources or computing resources.
  • the resource may include a process resource.
  • One unit resource may be one process.
  • the resource scale may be the number of processes.
  • the resource allocation scheme is used to indicate the resources corresponding to the task, that is, the resources allocated to the task.
  • the resource allocation schemes are different under different resource scales.
  • the running performance data of each task includes the running performance of the task under at least one sample resource scale, which can be understood as follows.
  • the running performance data of each task includes the running performance of the task under at least one sample resource allocation scheme, and the resource scale corresponding to the task indicated by the at least one sample resource allocation scheme includes at least one resource scale.
  • the resources indicated by the sample resource allocation scheme for the task may be continuous.
  • the processes indicated by the sample resource allocation scheme for the task may be continuous.
  • the processes allocated to the task may be represented by the ID of the starting process allocated to the task and the ID of the terminating process.
  • the processes allocated to the task may be represented by the ID of the starting process allocated to the task and the number of processes allocated to the task.
  • the processes allocated to the task may also be represented by other parameters.
  • step 510 may be performed by the data fuser 420 in FIG. 4 .
  • the following is an exemplary description of a method for obtaining the running performance data of a task.
  • the operation performance data of the task is obtained from the historical operation data of the task.
  • the operation data under the sample resource scale corresponding to the task is collected from the historical operation data of the task.
  • the running performance data of the task is obtained from monitoring data of the running task instance.
  • the running performance of the task instance can be obtained from the monitoring data as the running performance data of the task.
  • the task is pre-executed at a sample resource scale corresponding to the task.
  • Resources are allocated to the task based on the sample resource scale corresponding to the task, the task is pre-executed for a period of time under the resources, and the running performance data of the task is obtained from the monitoring data.
  • the running performance data of a task can be obtained by any one of the above methods, or by a combination of the above methods. For example, for different sample resource scales corresponding to a task, the corresponding running performance can be collected by different methods.
  • the methods for obtaining the running performance data of different tasks may be the same or different.
  • step 520 may be performed by the application analyzer 430 in FIG. 4 .
  • step 520 for a task, the running performance of the task in the case of monopolizing the candidate resource scale corresponding to the task can be determined according to the running performance data of the task.
  • the sample resource scale corresponding to a task and the candidate resource scale corresponding to the task may be the same or different.
  • the candidate resource scale corresponding to each task may include one candidate resource scale or multiple candidate resource scales.
  • the corresponding candidate resource sizes may be the same or different.
  • the multiple tasks may include task #1 and task #2.
  • the candidate resource scales corresponding to task #1 include two unit resources and three unit resources.
  • the candidate resource scales corresponding to task #2 include two unit resources.
  • the operating performance of task #1 when it monopolizes three unit resources and the operating performance of task #1 when it monopolizes two unit resources can be determined based on the operating performance data of task #1, and the operating performance of task #2 when it monopolizes two unit resources can be determined based on the operating performance data of task #2.
  • Step 520 is exemplarily described below with reference to three examples (Example 1, Example 2, and Example 3).
  • multiple tasks can be a first task.
  • a first performance model of the first task is constructed according to the running performance data of the first task, wherein the first performance model of the first task is used to predict the running performance of the first task when the resource scale input to the first performance model of the first task is exclusive; and the running performance of the first task when the first candidate resource scale corresponding to the first task is exclusive is determined according to the first performance model of the first task.
  • the candidate resource scale corresponding to the first task includes the first candidate resource scale corresponding to the first task.
  • the first task may be any one of the multiple tasks.
  • a first performance model of the task can be constructed according to the running performance of the task.
  • the first performance model of the task is used to predict the running performance of the task when the resource scale corresponding to the task is exclusively occupied.
  • the input of the first performance model of the task may include the resource scale corresponding to the task, and the output of the first performance model of the task may include the running performance of the task when the resource scale corresponding to the task is exclusively occupied.
  • the first performance model of the first task is used to indicate a mapping relationship between the resource scale corresponding to the first task and the operating performance of the first task when the resource scale is exclusively occupied. Based on the mapping relationship and the first candidate resource scale, the operating performance of the first task when the first candidate resource scale corresponding to the first task is exclusively occupied can be determined.
  • step 520 may include: constructing a first performance model for each task according to the operating performance data of each task in the multiple tasks; and determining the operating performance of each task in the case of monopolizing the corresponding first candidate resource scale according to the first performance model of each task.
  • the resource scales used as inputs of the first performance model of a task can all be used as the first candidate resource scales corresponding to the task.
  • the first candidate resource scales may include one or more.
  • the corresponding first candidate resource scales may be the same or different.
  • the multiple tasks may include task #1 and task #2.
  • the candidate resource scale corresponding to task #1 includes two unit resources and three unit resources.
  • the candidate resource scale corresponding to task #2 includes two unit resources.
  • the first candidate resource scale corresponding to task #1 may be two unit resources.
  • the two unit resources may be input into the first performance model of task #1, and the first performance model of task #1 predicts the operating performance of task #1 when it monopolizes the two unit resources.
  • the operating performance of task #2 when it monopolizes the two unit resources may be determined by other means.
  • the candidate resource scale corresponding to task #2 does not include the first candidate resource corresponding to task #2.
  • the first performance model may be an AI model.
  • the first performance model may be a neural network model. Constructing the first performance model of the task according to the running performance data of the task may be understood as training with the running performance data of the task as training data to obtain the first performance model of the task.
  • the first performance model may also be other models.
  • performance modeling of multiple tasks is performed using the running performance data of the multiple tasks, which is conducive to accurate performance analysis of each task, thereby providing a basis for the subsequent reasonable allocation of computing resources.
  • method 500 further includes: constructing fluctuation coefficient models for the multiple tasks according to the operation performance data of the multiple tasks.
  • the fluctuation coefficient models for the multiple tasks are respectively used to predict fluctuation coefficients of the multiple tasks corresponding to the resource allocation schemes input into the fluctuation coefficient models for the multiple tasks.
  • the fluctuation coefficient model of the task can be used to predict the fluctuation coefficient of the task corresponding to the resource allocation scheme input to the fluctuation coefficient model.
  • the input of the fluctuation coefficient model is the resource allocation scheme
  • the output is the fluctuation coefficient of the task corresponding to the resource allocation scheme, or in other words, the fluctuation coefficient of the task under the resource allocation scheme.
  • the fluctuation coefficient model of the task is used to indicate the mapping relationship between the resource allocation scheme and the fluctuation coefficient of the task under the resource allocation scheme.
  • the fluctuation coefficient of a task corresponding to a resource allocation scheme is used to indicate the difference between the running performance of the task under the resource allocation scheme and the running performance of the task when the task exclusively occupies the resource scale corresponding to the task.
  • the resource scale corresponding to the task is indicated by the resource allocation scheme.
  • the fluctuation coefficient of a task corresponding to a resource allocation scheme may be a ratio between the running performance of the task under the resource allocation scheme and the running performance of the task when the task exclusively occupies the resource scale corresponding to the task.
  • the resource scale corresponding to the task is indicated by the resource allocation scheme.
  • the fluctuation coefficient model may be an AI model.
  • the fluctuation coefficient model may be a neural network model. Constructing the fluctuation coefficient model of each task according to the running performance data of each task in the multiple tasks may be understood as training with the running performance data of each task as training data to obtain the fluctuation coefficient model of each task.
  • the fluctuation coefficient model may also be other models.
  • the plurality of tasks include the second task.
  • the operation performance of the second task in the case of exclusively occupying the second candidate resource scale corresponding to the second task is determined according to the fluctuation coefficient model of the second task in the plurality of tasks and the operation performance data of the second task.
  • the candidate resource scale corresponding to the second task includes the second candidate resource scale corresponding to the second task.
  • the operation performance data of the second task includes the second candidate resource scale corresponding to the second task.
  • the second candidate resource scale is the sample resource scale indicated by the sample resource allocation scheme corresponding to the second task.
  • the second task may be any one of the multiple tasks.
  • a fluctuation coefficient model of the task may be constructed based on the operational performance data of the task.
  • the fluctuation coefficient model of the task is used to predict the fluctuation coefficient of the task corresponding to the resource allocation scheme input to the fluctuation coefficient model of the task.
  • the resource allocation scheme input to the fluctuation coefficient model of the task is the resource allocation scheme corresponding to the task.
  • the input of the fluctuation coefficient model of the task may include the resource allocation scheme
  • the output of the fluctuation coefficient model of the task may include the fluctuation coefficient of the task corresponding to the resource allocation scheme, that is, the fluctuation coefficient of the task under the resource allocation scheme.
  • the fluctuation coefficient model of the task is used to indicate the mapping relationship between the resource allocation scheme input into the fluctuation coefficient model and the fluctuation coefficient of the task under the resource allocation scheme. Based on the mapping relationship and the resource allocation scheme corresponding to the task, the fluctuation coefficient of the task under the resource allocation scheme corresponding to the task can be determined.
  • the sample resource allocation scheme corresponding to the second task can be input into the fluctuation model of the second task, and the fluctuation model of the second task predicts the fluctuation coefficient of the second task under the sample resource allocation scheme. Based on the fluctuation coefficient and the operating performance of the second task under the sample resource allocation scheme, the operating performance of the second task in the case of monopolizing the second candidate resource scale corresponding to the second task can be predicted.
  • the second candidate resource scale corresponding to the second task is the sample resource scale indicated by the sample resource allocation scheme.
  • the process of determining the running performance of a task under the resource allocation scheme corresponding to the task based on the running performance of the task under the resource allocation scheme corresponding to the task can be called performance inversion.
  • step 520 may include: determining the operating performance of each task when the second candidate resource scale corresponding to each task is exclusively occupied according to the fluctuation coefficient model of each task and the operating performance data of each task.
  • the operating performance data of each task includes the operating performance of each task under the sample resource allocation scheme corresponding to each task.
  • the second candidate resource scale corresponding to each task is the sample resource scale indicated by the sample resource allocation scheme corresponding to each task.
  • the candidate resource scale corresponding to each task includes the sample resource scale corresponding to each task.
  • the scale of resources indicated by the resource allocation scheme as the input of the fluctuation coefficient model of the task can be used as the second candidate resource scale corresponding to the task.
  • the second candidate resource scale can include one or more.
  • the corresponding second candidate resource scales can be the same or different.
  • the multiple tasks may include task #1 and task #2.
  • the candidate resource scale corresponding to task #1 includes two unit resources and three unit resources.
  • the candidate resource scale corresponding to task #2 includes two unit resources.
  • the scale of resources indicated by the sample resource allocation scheme corresponding to task #1 is three unit resources. That is, the second resource scale corresponding to task #1 can be three unit resources.
  • the sample resource allocation scheme corresponding to task #1 can be input into the fluctuation coefficient model of task #1, and the fluctuation coefficient model of task #1 predicts the fluctuation coefficient of task #1 under the sample resource allocation scheme. Based on the fluctuation coefficient, the performance of task #1 under the resource allocation scheme corresponding to task #1 is reversed to obtain the operating performance of task #1 when it monopolizes three unit resources.
  • the operating performance of task #2 when it monopolizes two unit resources can be determined by other means.
  • the candidate resource scale corresponding to task #2 does not include the second candidate resource scale corresponding to task #2.
  • the performance inversion of the running performance data of the task under the sample resource allocation scheme is performed based on the fluctuation coefficient of the task to predict the running performance of the task when it monopolizes resources, which is conducive to realizing accurate performance analysis of each task, thereby providing a basis for the subsequent reasonable allocation of computing resources.
  • the running performance data of the multiple tasks acquired in step 510 includes the running performance of a third task in the multiple tasks when the third task exclusively occupies the third candidate resource scale corresponding to the third task.
  • the candidate resource scale corresponding to the third task includes the third candidate resource scale corresponding to the third task.
  • the third task may be any one of the multiple tasks.
  • the scale of the resources exclusively occupied by the task may be used as the scale of the third candidate resources corresponding to the task, and accordingly, the running performance of the task when the task exclusively occupies the resources is the running performance of the task when the task exclusively occupies the scale of the third candidate resources corresponding to the task.
  • the third candidate resource scale may include one or more types. For different tasks, the corresponding third candidate resource scales may be the same or different.
  • the multiple tasks may include task #1 and task #2.
  • the candidate resource scales corresponding to task #1 include two unit resources and three unit resources.
  • the candidate resource scales corresponding to task #2 include two unit resources.
  • the running performance data of the multiple tasks acquired in step 510 include the running performance of task #2 when it occupies two unit resources.
  • the third candidate resource scale corresponding to task #2 is The running performance of task #1 when it occupies two unit resources or three unit resources can be determined by other methods.
  • the candidate resource scale corresponding to task #1 does not include the third candidate resource scale corresponding to task #3.
  • the first task, the second task and the third task may be the same or different.
  • some or all of the above three examples may be used to determine the running performance of the task when the candidate resource scale corresponding to the task is exclusively occupied.
  • the candidate resource scale corresponding to the task includes the first candidate resource scale corresponding to the task, the second candidate resource scale corresponding to the task, and the third candidate resource scale corresponding to the task, and the corresponding methods are used to determine the running performance of the task when the candidate resource scale corresponding to the task is exclusively occupied.
  • step 520 can also be implemented in other ways.
  • step 520 can also be implemented in other ways.
  • the running performance of multiple tasks when they exclusively occupy the candidate resource scales corresponding to the multiple tasks can be represented by a performance matrix of the multiple tasks.
  • the elements at different positions in the performance matrix of the multiple tasks can be used to represent the running performance of the task corresponding to the position when the task exclusively occupies the resource scale corresponding to the position.
  • step 530 may be performed by resource allocator 440 in FIG. 4 .
  • Step 530 may include: determining a target resource allocation scheme according to the running performance of the multiple tasks when the multiple tasks exclusively occupy the target candidate resource scales corresponding to the multiple tasks.
  • the candidate resource scales corresponding to the multiple tasks include the target candidate resource scales corresponding to the multiple tasks.
  • the target candidate resource scale corresponding to each task may include one or more types.
  • the corresponding target candidate resource scales may be the same or different, and the corresponding target candidate resource scales may be determined in the same or different ways.
  • the target candidate resource scale corresponding to each task may be a partial resource scale among all candidate resource scales corresponding to each task.
  • the candidate resource scale corresponding to each task may be a partial resource scale among all candidate resource scales corresponding to each task.
  • the partial resource scale is referred to as a target candidate resource scale.
  • the embodiment of the present application provides the following scheme to determine the scale of some resources from the candidate resource scales, reducing the number of combinations of resource scales, that is, reducing the search space in the subsequent target resource allocation process, which is conducive to improving the efficiency of resource allocation.
  • the operating performance of the fourth task among the multiple tasks when monopolizing the target candidate resource scale corresponding to the fourth task satisfies preset condition #1, the target candidate resource scale corresponding to the fifth task among the multiple tasks satisfies preset condition #2, or, the target candidate resource scale corresponding to the sixth task among the multiple tasks and the operating performance of the sixth task when monopolizing the target candidate resource scale corresponding to the sixth task satisfy preset condition #3 (an example of the first preset condition).
  • the fourth task, the fifth task and the sixth task may be the same or different.
  • the preset conditions can be set as needed. For different tasks, the preset conditions can be the same or different.
  • preset condition #1 may include: the running performance of the fourth task when monopolizing the target candidate resource scale corresponding to the fourth task is greater than or equal to threshold #1.
  • the fourth task may be any one of the multiple tasks, and may be the same as or different from the first task, the second task, and the third task.
  • threshold #1 may be manually set, or threshold #1 may be preset.
  • the candidate resource scale In the candidate resource scales corresponding to a task, if the running performance of the task when occupying a candidate resource scale exclusively is greater than or equal to threshold #1, the candidate resource scale can be used as the target candidate resource scale corresponding to the task.
  • the target candidate resource scales corresponding to other tasks among the multiple tasks may also be determined by referring to the target candidate resource scale corresponding to the fourth task.
  • the running performance of the multiple tasks when they exclusively occupy the target candidate resource scale corresponding to the multiple tasks is greater than or equal to threshold #1.
  • the target candidate resource scale can be determined based on the running performance of the task, and the candidate resource corresponding to the higher running performance can be Selecting the resource scale as the target candidate resource scale is conducive to allocating appropriate resources to the task in the subsequent resource allocation to ensure the running performance of the task.
  • preset condition #1 may also be other conditions related to the running performance of the task.
  • preset condition #2 may include: the scale of target candidate resources corresponding to the fifth task is greater than or equal to threshold #2.
  • the fifth task may be any one of the multiple tasks, and may be the same as or different from the first task, the second task, and the third task.
  • the candidate resource scale can be used as the target candidate resource scale corresponding to the task.
  • threshold #2 may be manually set, or threshold #2 may be preset.
  • the target candidate resource scales corresponding to other tasks among the multiple tasks may also be determined by referring to the target candidate resource scale corresponding to the fifth task.
  • the target candidate resource scales corresponding to the multiple tasks are greater than or equal to threshold #2.
  • the parallel efficiency of the task is low, affecting the overall performance of the multiple tasks.
  • avoiding using too low a candidate resource scale as the target candidate resource scale is conducive to subsequently allocating appropriate resources to the task to ensure the parallel efficiency of the task.
  • preset condition #2 may also be other conditions related to the scale of candidate resources.
  • preset condition #3 includes: the ratio between the running performance of the sixth task when monopolizing the target candidate resource scale corresponding to the sixth task and the target candidate resource scale corresponding to the sixth task is greater than or equal to threshold #3 (an example of the first threshold).
  • the sixth task may be any one of the multiple tasks, and may be the same as or different from the first task, the second task, and the third task.
  • the ratio between the running performance of the task and the resource scale when the task exclusively occupies the resource scale corresponding to the task can be called resource efficiency, that is, the efficiency of the task in utilizing the resource scale.
  • Threshold #3 can be used as an efficiency threshold, that is, the threshold of the efficiency of the task in utilizing the resource scale.
  • the candidate resource scale can be used as the target candidate resource scale corresponding to the task.
  • threshold #3 may be manually set, or threshold #3 may be preset.
  • the resource efficiency of the task will gradually decrease.
  • the resource efficiency before and after the resource efficiency is significantly reduced can be used as the efficiency threshold.
  • the target candidate resource scales corresponding to other tasks among the multiple tasks may also be determined by referring to the target candidate resource scale corresponding to the sixth task.
  • a ratio of the running performance of each task when monopolizing a portion of the candidate resource scales corresponding to each task to the portion of the candidate resource scales corresponding to each task is greater than or equal to a threshold #3.
  • the ratio between the running performance of the multiple tasks when they exclusively occupy the target candidate resource scale corresponding to the multiple tasks and the target candidate resource scale corresponding to the multiple tasks is greater than or equal to the threshold #3.
  • the running performance of a task will significantly improve as the scale of resources allocated to the task increases. However, after the scale of resources allocated to the task reaches a certain threshold, the running performance of the task will no longer increase significantly, that is, the resource efficiency of the task decreases. When the resource efficiency is low, it will lead to resource waste.
  • the target candidate resource scale is determined based on the running performance of the task when it monopolizes the resource scale corresponding to the task and the resource scale, which is conducive to avoiding allocating too many resources to the task in subsequent resource allocation, thereby causing resource waste.
  • the target candidate resource scale can be determined based on the ratio between the running performance of the task when it monopolizes the resource scale corresponding to the task, that is, the resource efficiency of the task, and the candidate resource scale corresponding to the higher resource efficiency is used as the target candidate resource scale, so as to avoid allocating too many resources to the task in subsequent resource allocation, thereby causing resource waste. Waste of resources.
  • preset condition #3 may also be other conditions related to the running performance of the task and the scale of candidate resources corresponding to the task.
  • step 530 all resource allocation schemes under each resource scale combination in the multiple resource scale combinations may be enumerated, and the overall operating performance of the multiple tasks under all resource allocation schemes may be calculated, thereby obtaining an optimal solution.
  • the optimal solution refers to a resource allocation scheme that optimizes the overall operating performance of the multiple tasks, and the optimal solution may be used as a target resource allocation scheme.
  • the multiple resource scale combinations are determined based on a combination of target candidate resource scales corresponding to the multiple tasks.
  • the embodiment of the present application provides the following solution to determine the target resource allocation solution, which is conducive to obtaining a more reasonable target resource allocation solution in a shorter time, thereby ensuring the overall running performance of the multiple tasks.
  • step 530 may be implemented through the following steps.
  • Step 531 determining target candidate resource allocation schemes under multiple resource scale combinations according to the operation performance of each task in the multiple tasks when the target candidate resource scale corresponding to each task is exclusively occupied.
  • the multiple resource scale combinations are determined based on the combination of target candidate resource scales corresponding to the multiple tasks.
  • the target candidate resource allocation scheme under each resource scale combination in the multiple resource scale combinations indicates the target candidate resource corresponding to each task in the multiple tasks under each resource scale combination.
  • Step 532 determining the fluctuation coefficients of the multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination in the multiple resource scale combinations.
  • the fluctuation coefficients of the multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination are respectively used to indicate the difference between the running performance of the multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination and the running performance of the multiple tasks when the resource scale corresponding to each task in each resource scale combination is exclusively occupied.
  • the resource scale corresponding to each task in each resource scale combination is, that is, the target candidate resource scale corresponding to each task in each resource scale combination.
  • Step 533 predict the running performance of the multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination according to the fluctuation coefficient of the multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination and the running performance of the multiple tasks when monopolizing the target candidate resource scale corresponding to each task in each resource scale combination.
  • Step 534 determining a target resource allocation scheme from the target candidate resource allocation schemes under various resource scale combinations according to the running performance of the multiple tasks executed in parallel under the target candidate resource allocation schemes under various resource scale combinations.
  • the multiple resource scale combinations include a combination of target candidate resource scales corresponding to the multiple tasks. Selecting a target candidate resource scale from all target candidate resource scales corresponding to each task constitutes a resource scale combination.
  • a resource scale combination may include a target candidate resource scale corresponding to each task.
  • step 531 target candidate resource allocation schemes under each resource scale combination may be determined respectively.
  • Step 531 may include: determining target candidate resource allocation schemes under multiple resource scale combinations from candidate resource allocation schemes under multiple resource scale combinations according to the running performance of each task in the case of exclusively occupying the target candidate resource scale corresponding to each task.
  • the overall operating performance of the multiple tasks when they exclusively occupy the target candidate resource scale corresponding to each task in each resource scale combination under the target candidate resource allocation scheme under each resource scale combination is better than the overall operating performance of the multiple tasks when they exclusively occupy the target candidate resource scale corresponding to each task in each resource scale combination under other candidate resource allocation schemes under each resource scale combination.
  • a target candidate resource allocation scheme under the resource scale combination can be selected from the candidate resource allocation schemes under the resource scale combination.
  • the target candidate resource allocation scheme under the resource scale combination can be the optimal candidate resource allocation scheme among all the candidate resource allocation schemes under the resource scale combination.
  • the optimal candidate resource allocation scheme is the candidate resource allocation scheme that makes the overall operating performance of the multiple tasks optimal when they exclusively occupy the target candidate resource scales corresponding to each task in the resource scale combination.
  • the candidate resource allocation schemes under a resource scale combination may include all resource allocation schemes that satisfy the resource scale combination.
  • the resource scales corresponding to the tasks indicated by the candidate resource allocation schemes under a resource scale combination are consistent with the resource scale combination.
  • the target candidate resource allocation scheme under the resource scale combination may be a candidate resource allocation scheme that optimizes the overall operating performance of the multiple tasks when they exclusively occupy resources among all candidate resource allocation schemes under the resource scale combination.
  • the candidate resources corresponding to each task indicated by the candidate resource allocation scheme under each group of resource scale combinations are continuous.
  • the resources corresponding to each task indicated by the candidate resource allocation scheme may be continuous.
  • the resources corresponding to each task It can be indicated by the starting position of the resource corresponding to the task and the scale of the resource corresponding to the task.
  • the target candidate resource allocation scheme under each resource scale combination can be determined by a binning algorithm.
  • a binning algorithm For a detailed description, please refer to the relevant description in method 600.
  • step 531 may be implemented using a binning algorithm.
  • method 600 For a detailed description, please refer to method 600.
  • step 532 for each resource scale combination, the fluctuation coefficients of multiple tasks corresponding to the target candidate resource allocation scheme under the resource scale combination may be determined respectively.
  • the fluctuation coefficients of some tasks corresponding to the target candidate resource allocation scheme under the resource scale combination can be calculated.
  • the partial tasks refer to the tasks that share resources with other tasks in the target candidate resource allocation scheme.
  • the resources allocated to task #1 include process #1
  • the resources allocated to task #2 also include process #1
  • the resources allocated to task #3 are not allocated to other tasks.
  • the fluctuation coefficients of the corresponding tasks #1 and #2 can be calculated, without calculating the fluctuation coefficient of task #3.
  • the fluctuation coefficient of each task is used to indicate the difference between the running performance of the task when it is executed in parallel with other tasks under the target resource allocation scheme and the running performance of the task when it exclusively occupies the target candidate resource scale corresponding to the task in the resource scale combination.
  • the fluctuation coefficients of multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination are respectively used to indicate the ratio between the running performance of the multiple tasks executed in parallel under the target candidate resource allocation scheme under the resource scale combination and the running performance of the multiple tasks when they exclusively occupy the target candidate resource scale corresponding to the multiple tasks in the resource scale combination.
  • the fluctuation coefficient of the task corresponding to the target candidate resource allocation scheme under each resource scale combination is used to indicate the ratio between the running performance of the task when it is executed in parallel with other tasks under the target candidate resource allocation scheme and the running performance of the task when it exclusively occupies the target candidate resource scale corresponding to the task in the resource scale combination.
  • step 532 includes: determining, according to a fluctuation coefficient model of the multiple tasks, fluctuation coefficients of the multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination in the multiple resource scale combinations.
  • the fluctuation coefficient model of the task can be used to predict the fluctuation coefficient of the task corresponding to the resource allocation solution input into the fluctuation coefficient model.
  • the target candidate resource allocation scheme under the resource scale combination is input into the fluctuation coefficient models of multiple tasks respectively, so as to obtain the fluctuation coefficients of multiple tasks corresponding to the target candidate resource allocation scheme.
  • step 533 may include: determining the operating performance of multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination based on the fluctuation coefficients of multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination and the operating performance of the multiple tasks when exclusively occupying the target candidate resource scale corresponding to each task in each resource scale combination; determining the overall operating performance of the multiple tasks executed in parallel under the target candidate resource allocation schemes corresponding to the multiple resource scale combinations based on the operating performance of the multiple tasks executed in parallel under the target candidate resource allocation schemes under the multiple resource scale combinations.
  • the fluctuation coefficient of a task is used to indicate the difference between the running performance of the task when it is executed in parallel with other tasks under the target resource allocation scheme and the running performance of the task when it exclusively occupies the target candidate resource scale corresponding to the task in the resource scale combination.
  • the running performance of the task when it is executed in parallel with other tasks under the target resource allocation scheme can be calculated based on the fluctuation coefficient of the task and the running performance of the task when it exclusively occupies the target resource scale corresponding to the task.
  • the overall running performance of the multiple tasks executed in parallel under the target resource allocation scheme can be determined based on the running performance of the multiple tasks executed in parallel under the target resource allocation scheme.
  • the target candidate resource allocation scheme with the best overall operating performance may be used as the target resource allocation scheme.
  • the target resource corresponding to each task includes one or more processes corresponding to each task.
  • the one or more processes are respectively bound to one or more slots.
  • Slots and processor cores can correspond one to one.
  • One or more processes are bound to one or more slots, which can also be understood as one or more processes are bound to one or more processor cores.
  • the computing power node can allocate a processor core to the task, thereby realizing the calculation of the task.
  • the process is bound to the slot, or in other words, the process is bound to the processor core, so that the process can be allocated on a global scale, that is, the allocation of processor cores on a global scale can be realized, which is conducive to obtaining the optimal resource allocation scheme.
  • the scheme of the embodiment of the present application binds the process to the processor core, realizes refined computing power allocation, and does not require computing power nodes.
  • the secondary scheduling of the kernel means that there is no need for the computing power node to schedule the processor core for the process, thus avoiding the overhead caused by the secondary scheduling.
  • the target resource corresponding to each task includes one or more processes corresponding to each task.
  • the multiple processes are respectively bound to multiple continuous slots.
  • the multiple slots being multiple consecutive slots means that the numbers of the multiple slots are consecutive.
  • the multiple processes are multiple continuous processes.
  • the multiple slots are multiple continuous slots.
  • allocating continuous slots to each task is beneficial to reducing the communication cost during the execution of each task and further improving the running performance of the task.
  • the method 500 further includes: outputting indication information of the target resource allocation solution.
  • FIG6 shows a schematic flow chart of a method for resource allocation provided in an embodiment of the present application.
  • the method 600 shown in FIG6 can be regarded as a specific implementation of the method 500 described in FIG5 .
  • the relevant description can refer to the method 500. To avoid repetition, some descriptions are appropriately omitted when describing the method 600.
  • the method 600 can be executed by a resource allocation device.
  • the device can be deployed as a module in an application, or deployed outside an application, for example, deployed in a management and control system of computing resources.
  • method 600 includes the following steps.
  • Step 610 Obtain application related information, including information about multiple functional modules and available resources in the application.
  • a user may submit a processing request for an application, and the resource allocation device may obtain relevant information of the application.
  • the information of multiple functional modules in an application may be used to indicate the multiple functional modules in the application or the number of functional modules in the application.
  • the functional modules in the application refer to the functional modules enabled during the execution of the current application.
  • the multiple functional modules in the application correspond to the multiple tasks in the method 500.
  • the tasks in the method 500 are replaced with functional modules for description.
  • the information of available resources is used to indicate the available resources or the scale of available resources.
  • the information of available resources can be represented by hardware parameters.
  • the scale of available resources may be the number of available computing nodes.
  • the size of the available resources may be the number of available processors.
  • the scale of available resources may be the number of available processor cores.
  • the application-related information may also include other contents, for example, the application-related information may also include application architecture information.
  • the application architecture information may be used to indicate the coupling mode of the functional modules in the application.
  • the application-related information may also include user information.
  • the relevant information of the CESM may include the number of functional modules enabled in the CESM, the number of available processor cores, and the simulation time of the CESM.
  • Step 610 corresponds to step 501 in method 500.
  • step 501 for a detailed description, reference may be made to the relevant description of step 501, which will not be repeated here.
  • Step 620 Obtain the operation performance data of multiple application instances of the application.
  • the operation performance data of the multiple application instances includes the operation performance data of the multiple functional modules.
  • the operation performance data of the multiple application instances may include the operation performance data of the multiple application instances under multiple sample resource scale combinations.
  • the multiple application instances may be application instances under multiple sample resource scale combinations.
  • the sample resource scale combination may include a combination of sample resource scales corresponding to the multiple function modules.
  • the resource allocation scheme is used to indicate the resources corresponding to the functional modules, that is, the resources allocated to the functional modules.
  • the resource allocation schemes under different resource scale combinations are different.
  • the operating performance data of the multiple application instances include the operating performance data of the multiple application instances under multiple sample resource scale combinations, which can also be understood as follows.
  • the operating performance data of the multiple application instances may include the performance data of the multiple application instances under multiple resource allocation schemes, and the combination of resource scales corresponding to each functional module indicated by the multiple sample resource allocation schemes includes multiple sample resource scale combinations.
  • sample resource allocation scheme may be used to indicate the process allocation status of each functional module.
  • the process allocated to each functional module can be continuous.
  • the process allocated to each functional module can be represented by the ID of the starting process allocated to the functional module and the ID of the terminating process.
  • the process allocated to each functional module can be represented by the ID of the starting process allocated to the functional module and the number of processes allocated to the functional module.
  • the process allocated to each functional module can also be represented by other parameters.
  • the sample resource scale can be represented by a resource parameter, and accordingly, the sample resource scale combination can be represented by a resource parameter combination. Multiple sample resource scales can be represented by multiple resource parameter combinations.
  • the resource parameter combination is used to indicate the parameters of the resources allocated to each functional module.
  • the functional modules started by CESM may include ATM and OCN.
  • the sample resource scale combination of ATM and OCN may be represented by a combination of parameters for indicating the resources allocated to ATM and OCN.
  • the parameter may be the number of processes allocated to the ATM NATM
  • the parameter indicating the resources allocated to the OCN may be the number of processes allocated to the OCN NOCN.
  • the sample resource size combination may be expressed as a resource parameter combination (NATM, NOCN).
  • step 620 may be implemented through the following steps.
  • a default resource scale combination of the application is obtained based on the relevant information of the application.
  • the multiple sample resource scale combinations are determined based on the default resource scale combination of the application.
  • the sample resource scale can be represented by a resource parameter, and accordingly, the sample resource scale combination can be represented by a resource parameter combination.
  • the default resource size combination can be used as a baseline and basis for performance analysis.
  • a resource scale combination is selected from the neighborhood of the default resource scale combination of the application, and the resource scale combination and the default resource scale combination together constitute the multiple sample resource scale combinations. This process can be called parameter augmentation.
  • the default resource scale combination may be specified by a user.
  • a user may indicate the default resource scales corresponding to the various functional modules, and the combination of the default resource scales corresponding to the various functional modules may be used as the default scale combination.
  • the default resource size combination may be preset.
  • the default resource scale combination may also be determined by the resource allocation device based on other related solutions.
  • the resource allocation device determines a default resource size combination based on a general algorithm.
  • the neighborhood of the default resource scale combination may include a combination of resource scales whose difference with the corresponding default resource scale in the default resource scale combination is less than or equal to z.
  • Z is the scale threshold mentioned above. That is, a parameter combination of resource scales whose difference between the default resource scales corresponding to each functional module is within z.
  • the resource scale combination may be a combination of the number of processes allocated to each functional module.
  • the default resource scale combination is a combination of the default number of processes allocated to each functional module.
  • the functional modules started by CESM may include ATM and OCN, and the resource scale combination may be a combination of the number of processes allocated to ATM and OCN.
  • the default resource scale combination may be expressed as (5,5), that is, the default number of processes allocated to ATM is 5, and the default number of processes allocated to OCN is 5.
  • the neighborhood of the default resource scale combination may include a combination of the number of processes that differs by 3 processes from the default number of processes allocated to each module.
  • the resource scale combinations selected from the neighborhood of the default resource scale combination may include (4,7) and (6,3).
  • the multiple sample resource scale combinations may include (5,5), (4,7) and (6,3).
  • the multiple sample resource size combinations can also be determined in other ways.
  • the multiple sample resource size combinations can be randomly determined.
  • the multiple sample resource size combinations can be determined by the user.
  • the operation performance data of the application instance that meets the sample resource scales in the plurality of sample resource scale combinations is obtained.
  • the multiple sample resource scale combinations determined based on step 621 may include (5,5), (4,7) and (6,3).
  • step 622 the operation performance data of the application instances that meet the sample resource scales in (5,5), (4,7) and (6,3) may be obtained.
  • an application instance to which five processes are allocated to the ATM can serve as one of the multiple application instances.
  • step 622 may be implemented in at least one of the following ways:
  • the operating performance data of the application instances that meet the combination of the multiple sample resource scales are collected.
  • the operating performance data of the application instance can be used as the operating performance data of multiple application instances in step 620.
  • the running performance data of the application instances that meet the multiple sample resource scale combinations are collected.
  • the resource scale combination used by an application instance belongs to the multiple sample resource scale combinations, its operating performance data can be obtained from the monitoring data of the application instance as the operating performance data of multiple application instances in step 620.
  • Resources are allocated to each functional module based on a sample resource scale combination, and pre-executed for a period of time under the resource.
  • the operating performance data obtained from the monitoring data can be used as the operating performance data of the application instance under the sample resource scale combination.
  • the running performance data of the application instances under some of the sample resource scale combinations in the multiple sample resource scale combinations are obtained from historical data
  • the running performance data of the application instances under the remaining sample resource scale combinations are obtained from historical data.
  • the running performance data of the instance may be obtained by pre-executing the application under a corresponding sample resource scale combination.
  • the performance data of the application instance may include at least one of the following: running time, response delay, or resource utilization, etc.
  • running time may include the running time of the application and the running time of each functional module in the application.
  • Step 620 corresponds to step 510 in method 500.
  • Step 620 corresponds to step 510 in method 500.
  • step 510 for a detailed description, please refer to the relevant description of step 510, which will not be repeated here.
  • Step 630 Perform performance analysis based on the performance data of the multiple application instances to obtain the operating performance of each functional module under the condition of exclusive resources.
  • the behavioral characteristics of an application can also be called the performance characteristics of the application. For example, given the same resources, the operating performance of different applications is different. For another example, when the given resources increase, the operating performance of the application will generally improve, but the degree of improvement in the operating performance of different applications is different.
  • the behavioral characteristics of an application can be determined by the performance data of the application instance and the functional logic of the application instance.
  • the functional logic of the application instance is the functional module enabled in the application instance.
  • step 630 performance modeling can be performed based on the performance data of the multiple application instances to obtain a performance model of the application.
  • performance modeling can be performed based on the performance data of the multiple application instances and the functional logic of the multiple application instances to obtain a performance model of the application.
  • the performance model of the application can be used to indicate the mapping relationship between the resources of a given application and the operating performance of the application under the resources. That is, in step 630, the relationship between the resources and the operating performance of the application is constructed to analyze the behavioral characteristics of the application.
  • step 630 may be implemented through the following steps.
  • Step 631 construct an application model.
  • the application model is used to indicate the relationship between the overall operating performance of the application and the operating performance of multiple functional modules of the application.
  • the application model can be used to predict the overall operating performance of the application.
  • the input of the application model can be the operating performance of each functional module under different resource allocation schemes
  • the output of the application model is the overall operating performance of the application under the corresponding resource allocation scheme predicted by the application model.
  • the comparison between the overall operating performance of the application predicted by the application model and the actual overall operating performance of the application can be used to measure the modeling accuracy of the application model.
  • the overall operating performance of an application refers to the overall operating performance of multiple functional modules of the application when they are executed in parallel.
  • the application model may satisfy the following formula:
  • T tot represents the running time of the application
  • Ti represents the running time of the functional module i.
  • C represents the set of functional modules.
  • Ri represents the ID of the starting process of the functional module i.
  • Ni represents the number of processes used by the functional module i.
  • Ni is an integer.
  • the resources allocated to the functional module i in a single thread may include processes whose IDs belong to [ ri , ri + ni -1].
  • N represents the total number of processes that the application needs to call, and j represents the ID of the process.
  • Each process may be assigned to one or more functional modules.
  • the total number of processes that the application needs to call can satisfy the following formula:
  • f(r,n) represents the communication cost function between coupling functions, which is used to determine the communication cost between each functional module under a given resource allocation scheme.
  • the function value of f(r,n) is related to the resource allocation scheme, specifically, it is strongly related to the relative position of the resources of each functional module and the coupling module.
  • r represents the vector of the starting processes of each functional module
  • n represents the vector of the number of processes used by each functional module.
  • f(r,n) is usually small.
  • ⁇ represents the resource coordination and initialization time of the system after the user submits the application.
  • H(x) is the Heaviside step function.
  • H(x) can satisfy the following formula:
  • Step 632 construct performance models of multiple functional modules of the application and determine interference conditions between the multiple functional modules.
  • the performance model of the functional module may be a first performance model of the functional module.
  • the first performance model of each functional module may be used to predict the operating performance of the functional module under the condition of exclusively occupying the resource scale input to the first performance model.
  • the performance model of the functional module may be the second performance model of the kinetic energy module.
  • the second performance model of each functional module may be This is used to predict the operating performance of the functional module when it is executed in parallel with other functional modules under the resource allocation scheme input into the second performance model.
  • the interference between the functional modules may be indicated by the fluctuation coefficient of each functional module.
  • the interference between the functional modules may be different.
  • the fluctuation coefficient of the same functional module may also be different.
  • the fluctuation coefficient of each functional module can be determined by a fluctuation coefficient model of each functional module.
  • the fluctuation coefficient model of each functional module can be used to indicate the relationship between the resource allocation scheme and the fluctuation coefficient of the functional module.
  • the performance model of the application may include an application model, performance models of each module, and a fluctuation coefficient model of each functional module.
  • ⁇ i represents the fluctuation coefficient of function module i under a given resource allocation scheme.
  • T i,init represents the initialization time.
  • T i,comp represents the computing time, and T i,comm represents the communication time.
  • the running time T i ' of function module i under the condition of exclusive resources may include three parts: T i,init , T i,comp and T i,comm . Since function module i may compete with other function modules for resources, the running time Ti of function module i needs to consider the fluctuation coefficient ⁇ i , that is, the fluctuation coefficient ⁇ i is multiplied by the running time under the condition of exclusive resources to obtain the running time T i of function module i.
  • ⁇ i,1 and ⁇ i,2 are coefficients.
  • ⁇ i,1 and ⁇ i,2 indicate that the initialization time consists of a part related to the number of processes used by functional module i and a part that is irrelevant.
  • T i,comp can satisfy the following formula:
  • Wi represents the total load of functional module i.
  • computing tasks can be divided into parallelizable parts and non-parallelizable parts.
  • Ai represents the proportion of the parallelizable parts.
  • p represents the processor performance, and it can be assumed that the processor performance is consistent here. 0 ⁇ ai ⁇ 1 . Wi >0. p>0.
  • ⁇ i represents the communication delay.
  • the information transmission within the functional module is mainly completed through global communication, and the tree reduction method is mainly used, and the time complexity is logn i . ⁇ i >0.
  • ⁇ i can be determined by the harmonic mean of the function module including the process load. ⁇ i > 1. ⁇ i can satisfy the following formula:
  • the above formula can be used as a model for the fluctuation coefficient of functional module i.
  • L j can satisfy the following formula:
  • Lj represents the load of the jth process.
  • the correlation coefficient of the model can be fitted to the above model based on the performance data of the multiple application instances.
  • the performance model of the functional module i and the correlation coefficient in the fluctuation coefficient model can be determined based on the acquired running time of each functional module.
  • Step 633 predicting the operating performance of the multiple functional modules when they monopolize resources based on the performance models of the multiple functional modules of the application.
  • the performance models of the multiple functional modules of the application are used to predict the running performance of the multiple functional modules when the multiple functional modules exclusively occupy the candidate resource scales corresponding to the multiple functional modules.
  • step 633 may include predicting the running performance of the multiple functional modules when they monopolize resources based on the first performance model of the multiple functional modules of the application.
  • step 634 may include predicting the operating performance of the multiple functional modules when they monopolize resources based on the second performance models of the multiple functional modules of the application and the interference conditions between the multiple functional modules.
  • the operating performance of the multiple functional modules when they monopolize resources can be represented by a performance matrix of the multiple functional modules.
  • the performance matrix is used to represent the operating performance of different functional modules when they monopolize different resources.
  • the dimension of the performance matrix can be
  • a row of the performance matrix corresponds to a functional module, and a column in the performance matrix corresponds to a resource scale, for example, the number of processes.
  • Each element of the performance matrix represents the operating performance of the functional module corresponding to the row where the element is located when it monopolizes the resource scale corresponding to the column where the element is located.
  • the candidate resource scale corresponding to the multiple functional modules is the resource scale in the performance matrix.
  • Predicting the operating performance of the multiple functional modules when they exclusively occupy the candidate resource scales corresponding to the multiple functional modules is to predict the elements of the performance matrix of the multiple modules.
  • the following is an exemplary description of the method for determining the elements in the performance matrix.
  • the running performance data of multiple application instances includes the running performance data of multiple application instances under multiple sample resource scale combinations.
  • the running performance data of the multiple application instances include the running performance data of the function module when it exclusively occupies the resource
  • the running performance data of the function module when it exclusively occupies the resource can be used as the element at the position indicated by the function module and the resource in the performance matrix. This method corresponds to Example 3 in method 500.
  • the running performance data of the multiple application instances include the running performance data of the function module i when it monopolizes n i processes
  • the running performance data of the function module i when it monopolizes n i processes can be used as the element at (i, n i ) in the performance matrix.
  • the function module i can be used as an example of the third task
  • the n i processes can be used as an example of the third candidate resource scale corresponding to the third task.
  • the fluctuation coefficient of the functional module can be calculated by the fluctuation coefficient model of the functional module, and the performance inverse solution is performed based on the fluctuation coefficient to predict the operational performance of the functional module when the resources are exclusive.
  • the predicted operational performance of the functional module when the resources are exclusive can be used as the element at the position indicated by the functional module and the resource in the performance matrix. This method corresponds to Example 2 in method 500.
  • the running performance data of the multiple application instances include the running performance data of the function module i when it non-exclusively occupies n i processes.
  • the sample resources corresponding to the function module i are indicated by a sample resource allocation scheme, in which at least one of the n i processes is simultaneously allocated to other function modules.
  • the fluctuation coefficient of the function module is calculated based on the sample resource allocation scheme, and the running performance data of the function module i when it non-exclusively occupies n i processes is inversely solved, and the result obtained can be used as the element at (i, n i ) in the performance matrix.
  • the function module i can be used as an example of the second task
  • the n i processes can be used as an example of the second candidate resource scale corresponding to the second task.
  • the performance models of the functional modules can be used to predict the operating performance of the functional modules when they monopolize resources. This approach corresponds to Example 1 in method 500 .
  • the performance model of each functional module is obtained by fitting the performance data of multiple application instances under multiple sample resource scale combinations, the accuracy of predicting the operating performance of each functional module under the parameters around the multiple sample resource scale combinations using the performance model of each functional module is higher. Therefore, the operating performance of each functional module under the parameters around the multiple sample resource scale combinations can be predicted based only on the performance model of each module.
  • the complete performance matrix can be obtained by solving the following minimization problem.
  • X ab represents the value at the position (a, b) observed in the performance matrix
  • ⁇ represents the set of positions
  • Z represents the original matrix
  • * represents the nuclear norm of Z
  • Z ab represents the position (a, b) in the original matrix Z
  • ⁇ represents a small quantity.
  • Step 640 Determine a target resource allocation scheme based on the operating performance of each functional module when the modules exclusively occupy resources.
  • the target resource allocation scheme is used to indicate the target resources corresponding to each functional module.
  • the target resource allocation scheme is used to indicate the target resources allocated to each functional module.
  • step 640 may be implemented through the following steps.
  • feasible resource scale combinations may be determined by enumerating all combinations of candidate resource scales in the performance matrix.
  • the enumeration space of this method is too large, and it is difficult to efficiently determine the target resource allocation solution from it in the subsequent processing process.
  • the embodiment of the present application provides a solution based on search pruning, which can quickly determine a feasible resource scale combination. Specifically, the solution reduces the search space in the subsequent determination of target resource allocation through pruning processing, improves search efficiency, and thus quickly determines the target resource allocation solution.
  • a global search is performed for each functional module (i.e., each row element) in the performance matrix to perform a pruning operation. All functional modules are traversed to complete the pruning operation.
  • a pruning operation is performed based on the resource module efficiency of the element in the performance matrix.
  • the resource module efficiency of the element is lower than the efficiency threshold, the element is pruned.
  • the functional module corresponding to the row of the element in the performance matrix after the pruning operation is the sixth task in method 500, and the resource scale corresponding to the column of the element is the target candidate resource scale corresponding to the sixth task.
  • the resource module efficiency of an element When the resource module efficiency of an element is lower than the efficiency threshold, it can be considered that the resource utilization efficiency of the module under the current resources is low, resulting in resource waste, and the element can be removed.
  • the resource module efficiency of each element is the ratio between the operating performance of the functional module and the scale of the resources.
  • a pruning operation is performed based on the resource scale corresponding to the element in the performance matrix.
  • the resource scale corresponding to the element is small, the parallel efficiency of the task is low, which affects the overall running performance of the multiple tasks, and the element can be removed.
  • the processing flow of the performance matrix is exemplarily described below with reference to FIG. 7 .
  • the performance data of application instances under various combinations of sample resource scales are obtained by sampling.
  • the fluctuation coefficient of each functional module is calculated, and the performance is inversely solved to obtain the operating performance of each functional module when the corresponding resource is exclusively occupied, which is used as the sampling point on the performance matrix.
  • the internal characteristic fitting is applied, that is, the operating performance around the sampling point is predicted by the performance model of each functional module to obtain the fitting point on the performance matrix. Predicting the operating performance around the sampling point can also be called interpolation or extension around the sampling point. As shown in FIG7, interpolation or extension is only performed around the sampling point, which is conducive to ensuring the accuracy of the predicted operating performance.
  • the missing elements in the current performance matrix are repaired by matrix completion, that is, the completion points on the performance matrix are obtained.
  • the pruning points in the performance matrix are pruned to screen out the target candidate resource scales corresponding to each functional module.
  • the pruning points can be determined based on the marginal effect, that is, based on the efficiency threshold.
  • the combination of the target candidate resource scales corresponding to each functional module can be used as a feasible resource scale combination.
  • step 642 may include: enumerating all resource allocation schemes under feasible resource scale combinations, and determining a target resource allocation scheme therefrom.
  • This method is highly complex and takes a long time to run, and it is difficult to efficiently determine the target resource allocation solution.
  • the embodiment of the present application provides an allocation method using a binning algorithm, which can achieve efficient allocation of resources.
  • the feasible resource scale combinations are traversed.
  • the packing algorithm is executed to minimize the overall height of the packing. In this way, the location of the resources corresponding to each functional module can be obtained, so as to determine the optimal resource allocation plan under the resource scale combination.
  • the running time of each functional module when non-exclusive resources is restored by the fluctuation coefficient of each functional module to determine the overall running time of the application.
  • a fast two-dimension first-fit decreasing height is performed.
  • height, 2D-FFDH) box packing algorithm is improved, forward boxing is performed once and then reverse boxing is performed, with two layers as a group, and the complexity is nlogn, where n represents the number of boxes, that is, the number of functional modules in the embodiment of the present application, to minimize the overall height of the box.
  • the application may include 5 functional modules.
  • the feasible resource scale combination is the combination of target candidate resource scales corresponding to the 5 functional modules in the pruned performance matrix.
  • Each functional module is abstracted into a matrix based on the corresponding elements in the performance matrix.
  • the width of the matrix corresponds to the resource scale allocated to the functional module in the resource scale combination, and the length of the matrix corresponds to the running time of the functional module when it exclusively occupies the resource scale.
  • functional module i is abstracted into a matrix, the width of the matrix corresponds to the number of processes n i allocated to the functional module in the resource scale combination, and the length of the matrix corresponds to the value at (i, n i ) in the performance matrix.
  • the matrices corresponding to all functional modules are packed into the resource pool through a packing algorithm to minimize the overall height.
  • the width of the resource pool can be the available resource scale.
  • Determining the target resource allocation scheme based on the 2D-FFDH bin packing algorithm may include the following steps:
  • steps S1 to S4 are respectively executed.
  • S1 arrange the functional modules in descending order of running time, and place the functional modules based on the order.
  • the five functional modules in FIG. 8 are arranged in descending order of running time, and the resulting order may be: functional module #3, functional module #1, functional module #2, functional module #4, functional module #5.
  • the remaining space is greater than or equal to the number of resources corresponding to the functional module, that is, the width of the remaining space is greater than or equal to the width of the matrix corresponding to the functional module.
  • the remaining space is less than the number of resources corresponding to the functional module, that is, the width of the remaining space is less than the width of the matrix corresponding to the functional module.
  • function module #3 is placed first in the lower layer, and the remaining space on the left side of the lower layer is calculated.
  • the function module to be placed after function module #3 is function module #1.
  • the remaining space on the right side of the lower layer is greater than the number of resources corresponding to function module #1.
  • Function module #1 is placed in the lower layer, and the remaining space on the right side of the lower layer after function module #1 is placed is calculated.
  • the function module to be placed after function module #1 is function module #2.
  • the remaining space on the right side of the lower layer is less than the number of resources corresponding to function module #2.
  • Function module #2 is placed on the rightmost side of the upper layer, and the remaining space on the left side of the upper layer after function module #2 is placed is calculated, and so on.
  • step S3 if the remaining space of the lower layer and the remaining space of the upper layer are both smaller than the resources corresponding to the functional modules to be placed, two more layers are enabled to place the functional modules. Repeat step S2 until all functional modules are placed, and obtain the target candidate resource allocation plan under the resource scale combination.
  • the fluctuation coefficients of the five functional modules are calculated.
  • the resources corresponding to each functional module may be indicated by a starting position of the resources corresponding to the functional module and a scale of the resources corresponding to the functional module.
  • the predicted running time of the application may be determined based on the application model in step 631 and the running time of each functional module when non-exclusive resources are occupied.
  • Step 640 corresponds to step 530 in method 500.
  • Step 640 corresponds to step 530 in method 500.
  • step 530 For a detailed description, please refer to the related description of step 530.
  • the tasks corresponding to each functional module can be distributed to the processor, and the processor performs the calculation.
  • the process can be bound to the processor core.
  • the optimal resource allocation solution can be searched globally, avoiding the inability to obtain the optimal resource allocation solution due to the lack of global information inside the computing power node.
  • the solution of the embodiment of the present application binds the process to the processor core, realizes refined computing power allocation, and does not require secondary scheduling of the computing power node core, that is, there is no need for the computing power node to schedule the processor core for the process, avoiding the overhead caused by secondary scheduling.
  • the method 600 is exemplarily described below using the CESM application as an example.
  • the function modules enabled by CESM may include ATM, LND, ICE, ROF, GLC, WAV, OCN and coupling component CPL.
  • the scheme of the embodiment of the present application can allocate corresponding process resources to the above function modules. The same process resource is allowed to be allocated to multiple function modules.
  • the resource allocation process of CESM may include the following steps.
  • CESM Obtain relevant information of CESM, which includes information of multiple functional modules and information of available resource scale.
  • the information of the multiple functional modules is used to indicate ATM, LND, ICE, ROF, GLC, WAV, OCN and a coupling component CPL.
  • the available resource scale information is used to indicate the available resources of the HPC.
  • the multiple sample resource allocation schemes indicate multiple combinations of sample resource scales corresponding to each functional module.
  • the fluctuation coefficient of each functional module is calculated respectively.
  • the running performance data of multiple application instances are reversed to obtain the running time of each functional module when it monopolizes resources, which is used as the sampling point in the performance matrix.
  • the target resource allocation scheme traverses the feasible resource scale combinations, for each feasible resource scale combination, determine the target candidate resource allocation scheme by using the 2D-FFDH fast packing algorithm; calculate the fluctuation coefficient of each functional module based on the target candidate resource allocation scheme; calculate the running time of each functional module when executed in parallel under the target candidate resource allocation scheme by using the fluctuation coefficient of each functional module; predict the overall running time of the application based on the running time when executed in parallel.
  • the target candidate resource allocation scheme corresponding to the optimal overall running time obtained after traversal is used as the target resource allocation scheme.
  • the target resource allocation scheme can be shown in FIG9 .
  • Table 1 shows the performance test results of CESM in the test environment. As shown in Table 1, performance tests were conducted based on the CESM dataset B1850G and the dataset BW1850. The resolution (res) of the scene is f09_g17, the number of threads per process NTHRD is 1, and the experimental simulation time is 5 days.
  • the hardware structure used in the test environment is a non-uniform memory access (NUMA) structure node, including: two CPU sockets, 52 cores, and 768GB of random access memory (RAM).
  • NUMA non-uniform memory access
  • the solution of the embodiment of the present application improves performance by more than 30% compared with the resource layout method of the related solution.
  • One or more users can submit multiple job tasks to the computing system, and the system will manage and run these tasks.
  • the computing system can be called a batch job processing system or a batch processing system.
  • the solution of the embodiment of the present application can be applied to the resource allocation scenario of the batch processing system to allocate computing resources to the multiple job tasks.
  • FIG10 shows a resource allocation scheme in a batch processing system with a mixed layout of multiple job tasks.
  • the multiple job tasks include a data processing task, a decoding task, an encryption task, a file sorting task, and two compression tasks.
  • the multiple job tasks can be used as multiple tasks in method 500.
  • the scheme of the embodiment of the present application can allocate computing power resources to the multiple job tasks, while improving the utilization rate of system resources, optimizing the overall operating performance of the multiple job tasks, for example, the overall running time of the multiple tasks.
  • Fig. 11 is a schematic block diagram of a device for resource allocation according to an embodiment of the present application.
  • the device 1100 shown in Fig. 11 can be used to execute a method for resource allocation according to an embodiment of the present application, for example, method 500 or method 600.
  • the device 1100 includes an acquisition unit 1110 and a processing unit 1120 .
  • an acquisition unit is used to acquire running performance data of each of a plurality of tasks, wherein the running performance data of each of the plurality of tasks includes the running performance of each task under a sample resource scale corresponding to each task;
  • a processing unit is used to: determine, based on the running performance data of each of the plurality of tasks, the running performance of each of the plurality of tasks when the candidate resource scale corresponding to each task is exclusively occupied; and determine the target resource corresponding to each of the plurality of tasks based on the running performance of each of the plurality of tasks when the candidate resource scale corresponding to each task is exclusively occupied.
  • the target resource corresponding to each task includes one or more processes corresponding to each task, and the one or more processes are respectively bound to one or more slots.
  • the multiple processes are multiple consecutive processes
  • the multiple slots are multiple consecutive slots.
  • multiple tasks include a first task
  • the processing unit is specifically used to: construct a first performance model of the first task based on the running performance data of the first task, wherein the first performance model of the first task is used to predict the running performance of the first task when the resource scale of the performance model of the first task is exclusively input; determine the running performance of the first task when the first candidate resource scale corresponding to the first task is exclusively occupied according to the first performance model of the first task, and the candidate resource scale corresponding to the first task includes the first candidate resource scale corresponding to the first task.
  • the candidate resource scale corresponding to each task is a partial resource scale among all candidate resource scales corresponding to each task, and the partial resource scale among the candidate resource scales corresponding to each task and the operating performance of each task when monopolizing the partial resource scale among the candidate resource scales corresponding to each task meet the first preset condition.
  • the first preset condition includes: the ratio of the operating performance of each task when monopolizing part of the resource scale in the candidate resource scale corresponding to each task and the part of the resource scale in the candidate resource scale corresponding to each task is greater than or equal to a first threshold, and the first threshold is the threshold of the task's utilization efficiency of the resource scale.
  • the processing unit is specifically used to: determine the target candidate resource allocation schemes under multiple resource scale combinations according to the operating performance of each task in the case of monopolizing a part of the resource scale in the candidate resource scale corresponding to each task, the multiple resource scale combinations are determined based on the combination of the part of the resource scale in the candidate resource scale corresponding to each task, and the target candidate resource allocation scheme under each resource scale combination in the multiple resource scale combinations indicates the target candidate resource corresponding to each task in the multiple tasks under each resource scale combination; determine the fluctuation coefficients of the multiple tasks corresponding to the target candidate resource allocation scheme under each resource scale combination in the multiple resource scale combinations, and the target candidate resource allocation scheme under each resource scale combination corresponds to The fluctuation coefficients of the multiple tasks are respectively used to indicate the difference between the running performance of the multiple tasks executed in parallel under the target candidate resource allocation scheme under each resource scale combination and the running performance of the multiple tasks when the resource scale corresponding to each task in each resource scale combination is exclusively occupied; the running performance of the multiple tasks executed in parallel under the target candidate resource allocation scheme under each
  • the processing unit is specifically used to: determine a target candidate resource allocation scheme under each resource scale combination from the candidate resource allocation schemes under each resource scale combination according to the operating performance of each task in the case of exclusively occupying a part of the resource scale among the candidate resource scales corresponding to each task, the candidate resources corresponding to each task indicated by the candidate resource allocation schemes under each group of resource scale combinations are continuous, and the overall operating performance of the multiple tasks in the case of exclusively occupying the resource scale corresponding to each task in each resource scale combination under the target candidate resource allocation scheme under each resource scale combination is better than the overall operating performance of the multiple tasks in the case of exclusively occupying the resource scale corresponding to each task in each resource scale combination under other candidate resource allocation schemes under each resource scale combination.
  • the processing unit is specifically used to: construct fluctuation coefficient models for multiple tasks based on the operating performance data of multiple tasks, the fluctuation coefficient models for multiple tasks are respectively used to predict the fluctuation coefficients of multiple tasks corresponding to the resource allocation schemes input into the fluctuation coefficient models of multiple tasks; determine the fluctuation coefficients of multiple tasks corresponding to the target candidate resource allocation schemes under each resource scale combination in multiple resource scale combinations according to the fluctuation coefficient models of multiple tasks.
  • the multiple tasks include tasks corresponding to multiple applications, and each of the multiple applications corresponds to one task.
  • the multiple tasks include tasks corresponding to multiple functional modules in an application, and each of the multiple functional modules corresponds to one task.
  • unit herein may be implemented in software and/or hardware form and is not specifically limited thereto.
  • a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the above functions.
  • the implementation of the processing unit is described below by taking the processing unit as an example.
  • the implementation of the acquisition unit and the output unit can refer to the implementation of the processing unit.
  • the processing unit may include code running on a computing instance.
  • the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container.
  • the computing instance may be one or more.
  • the processing unit may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts/virtual machines/containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Generally, a region may include multiple AZs.
  • AZ availability zone
  • VPC virtual private cloud
  • multiple hosts/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs.
  • VPC virtual private cloud
  • a VPC is set up in a region.
  • a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.
  • the processing unit may include at least one computing device, such as a server, etc.
  • the processing unit may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc.
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • the PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
  • CPLD complex programmable logical device
  • FPGA field-programmable gate array
  • GAL generic array logic
  • the multiple computing devices included in the processing unit can be distributed in the same region or in different regions.
  • the multiple computing devices included in the processing unit can be distributed in the same AZ or in different AZs.
  • the multiple computing devices included in the processing unit can be distributed in the same VPC or in multiple VPCs.
  • the multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
  • modules of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.
  • the processing unit can be used to execute any step in the resource allocation method
  • the acquisition unit can be used to execute any step in the resource allocation method
  • the output unit can be used to execute any step in the resource allocation method.
  • the steps that the acquisition unit, the processing unit, and the output unit are responsible for implementing can be specified as needed.
  • the full functions of the device 1100 are realized by respectively implementing different steps in the resource allocation method by the acquisition unit, the processing unit, and the output unit.
  • the present application also provides a computing device 1000.
  • the computing device 1000 includes: a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008.
  • the processor 1004, the memory 1006, and the communication interface 1008 communicate with each other through the bus 1002.
  • the computing device 1000 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1000.
  • the bus 1002 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc.
  • the bus may be divided into an address bus, a data bus, a control bus, etc.
  • FIG. 12 is represented by only one line, but does not mean that there is only one bus or one type of bus.
  • the bus 1004 may include a path for transmitting information between various components of the computing device 1000 (e.g., the memory 1006, the processor 1004, and the communication interface 1008).
  • Processor 1004 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).
  • processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).
  • CPU central processing unit
  • GPU graphics processing unit
  • MP microprocessor
  • DSP digital signal processor
  • the memory 1006 may include a volatile memory, such as a random access memory (RAM).
  • the processor 1004 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
  • ROM read-only memory
  • HDD hard disk drive
  • SSD solid state drive
  • the memory 1006 stores executable program codes, and the processor 1004 executes the executable program codes to respectively implement the functions of the acquisition unit and the unit module, thereby implementing the resource allocation method. That is, the memory 1006 stores instructions for executing the resource allocation method.
  • the communication interface 1003 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1000 and other devices or a communication network.
  • a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1000 and other devices or a communication network.
  • the embodiment of the present application also provides a computing device cluster.
  • the computing device cluster includes at least one computing device.
  • the computing device can be a server, such as a central server, an edge server, or a local server in a local data center.
  • the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.
  • the computing device cluster includes at least one computing device 1000.
  • the memory 1006 in one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the resource allocation method.
  • the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the resource allocation method.
  • the combination of one or more computing devices 1000 may jointly execute the instructions for executing the resource allocation method.
  • the memory 1006 in different computing devices 1000 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the resource allocation apparatus.
  • the instructions stored in the memory 1006 in different computing devices 1000 may implement the functions of one or more of the acquisition unit and the processing unit.
  • one or more computing devices in the computing device cluster can be connected via a network.
  • the network can be a wide area network or a local area network, etc.
  • FIG. 14 shows a possible implementation. As shown in FIG. 14 , two computing devices 1000A and 1000B are connected via a network. Specifically, the network is connected via a communication interface in each computing device.
  • the memory 1006 in the computing device 1000A stores instructions for executing the functions of the acquisition unit. At the same time, the memory 1006 in the computing device 1000B stores instructions for executing the functions of the processing unit.
  • the functions of the computing device 1000A shown in FIG14 may also be completed by multiple computing devices 1000.
  • the functions of the computing device 1000B may also be completed by multiple computing devices 1000.
  • the present application also provides a computer program product including instructions.
  • the computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium.
  • the computer program product is run on at least one computing device, the at least one computing device executes the method in the present application embodiment.
  • the present application embodiment also provides a computer-readable storage medium.
  • the computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media.
  • the available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).
  • the computer-readable storage medium includes instructions that instruct a computing device to execute a method in an embodiment of the present application, or instructs a computing device to execute a method in an embodiment of the present application.
  • the size of the serial numbers of the above-mentioned processes does not mean the order of execution.
  • the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
  • the disclosed systems, devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of this application is essentially or partly or partly contributed to the prior art.
  • Part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请实施例提供了一种资源分配的方法及装置，该方法包括：获取多个任务中的每个任务的运行性能数据，每个任务的运行性能数据包括每个任务在对应的样本资源规模下的运行性能；根据多个任务中的每个任务的运行性能数据确定每个任务在独占对应的候选资源规模的情况下的运行性能；根据多个任务中的每个任务在独占对应的候选资源规模的情况下的运行性能确定该多个任务中的每个任务对应的目标资源。本申请实施例的方案有利于为多个任务分配较为合理的算力资源，从而提高多个任务的整体运行性能。

Description

资源分配的方法及装置
本申请要求在2022年12月16日提交中国国家知识产权局、申请号为202211624304.6的中国专利申请的优先权，发明名称为“资源分配的方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及数据处理技术领域，并且更具体地，涉及一种资源分配的方法及装置。
背景技术
算力资源分配是影响任务的运行性能的重要因素之一。在一些方案中，算力资源分配方案是基于通用算法确定的。然而，通用算法无法感知任务的特点，即无法根据不同的任务的特点来分配算力资源，难以实现任务与算力资源的最佳匹配。在另一些方案中，算力分配方案是由用户确定的，例如，由用户指定任务所需要的算力资源，或者，由用户为任务定制算力资源分配方案。然而人为确定资源分配方案的方式对用户的要求较高，且该方式效率较低，难以满足运行需求。
如何实现算力资源分配成为一个亟待解决的问题。
发明内容
本申请实施例提供一种资源分配的方法及装置，有利于为多个任务分配较为合理的算力资源，从而提高多个任务的整体运行性能。
第一方面，提供了一种资源分配的方法，包括：获取多个任务中的每个任务的运行性能数据，多个任务中的每个任务的运行性能数据包括每个任务在每个任务对应的样本资源规模下的运行性能；根据多个任务中的每个任务的运行性能数据确定多个任务中的每个任务在独占多个任务对应的候选资源规模的情况下的运行性能；根据多个任务中的每个任务在独占每个任务对应的候选资源规模的情况下的运行性能确定多个任务中的每个任务对应的目标资源。
在本申请实施例的方案中，可以基于多个任务的运行性能数据分析任务，以识别各个任务自身的行为特点，例如，各个任务独占资源时的运行性能，并基于任务自身的行为特点确定多个任务中的每个任务对应的目标资源，有利于提高任务与资源的匹配度，从而提高该多个任务的整体运行性能。
示例性地，任务的运行性能可以包括任务的运行时间、任务的响应时延、任务的资源利用率或通过硬件性能监控单元(performance monitoring unit，PMU)计数器采集的数据等。
一个任务在一种资源规模下的运行性能可以包括以下至少一项：该任务在独占该资源规模的情况下的运行性能；该任务在非独占该资源规模的情况下的运行性能。
每个任务对应的样本资源规模可以包括一种样本资源规模，也可以包括多种样本资源规模。
不同任务对应的样本资源规模可以相同，也可以不同。
示例性地，资源可以包括进程资源。一个单位资源可以为一个进程。资源规模可以为进程的数量。
结合第一方面，在第一方面的某些实现方式中，该多个任务中的至少一个任务的运行性能数据包括该至少一个任务在非独占该至少一个任务对应的样本资源规模的情况下的运行性能。
在实际应用的场景中，通常任务不独占资源。在本申请实施例的方案中，可以根据基于任务在不独占资源时的运行性能确定任务在独占资源时的运行性能，减少了对该多个任务的运行性能数据的限制。本申请实施例的方案中获取到的多个任务的运行性能既可以包括任务独占资源的情况下的运行性能，还可以包括任务在不独占资源的情况下的运行性能，提高了获取到的运行性能数据的数据量，为后续的处理过程奠定了数据基础，从而有利于更准确地分析该多个任务，进而实现合理的资源分配。
结合第一方面，在第一方面的某些实现方式中，每个任务对应的目标资源包括每个任务对应的一个或多个进程，一个或多个进程分别与一个或多个资源槽(slot)绑定。
slot和处理器核可以是一一对应的。一个或多个进程分别与一个或多个slot绑定，也可以理解为一个 或多个进程分别与一个或多个处理器核绑定。
在本申请实施例的方案中，将进程与slot绑定，可以在全局范围内实现进程的分配，也即实现全局范围内的slot的分配，有利于得到最优的资源分配方案。同时本申请实施例的方案，将进程与slot绑定，实现了精细化的算力分配，无需算力节点内核的二次调度，即无需由算力节点为进程调度处理器核，避免了二次调度所带来的开销。
结合第一方面，在第一方面的某些实现方式中，多个进程为多个连续的进程，多个slot为多个连续的slot。
多个slot为多个连续的slot指的是该多个slot的编号连续。
在本申请实施例中，为各个任务分配连续的slot，有利于减少各个任务执行过程中的通信代价，进一步提高任务的运行性能。
结合第一方面，在第一方面的某些实现方式中，多个任务包括第一任务，根据第一任务的运行性能数据确定第一任务在独占第一任务对应的候选资源规模的情况下的运行性能，包括：根据第一任务的运行性能数据构建第一任务的第一性能模型，其中，第一任务的第一性能模型用于预测第一任务在独占输入至第一任务的性能模型的资源规模的情况下的运行性能；根据第一任务的第一性能模型确定第一任务在独占第一任务对应的第一候选资源规模的情况下的运行性能，第一任务对应的候选资源规模包括第一任务对应的第一候选资源规模。
第一任务可以为该多个任务中的任一任务。
本申请实施例中，通过任务的运行性能数据对该任务进行性能建模，有利于实现对任务的准确的性能分析，从而为后续算力资源的合理分配提供基础。
结合第一方面，在第一方面的某些实现方式中，多个任务包括第二任务，根据第二任务的运行性能数据确定第二任务在独占第二任务对应的候选资源规模的情况下的运行性能，包括：根据第二任务的波动系数模型和第二任务的运行性能数据确定第二任务在独占第二任务对应的第二候选资源规模的情况下的运行性能。第二任务对应的候选资源规模包括第二任务对应的第二候选资源规模。第二任务的运行性能数据包括第二任务在第二任务对应的样本资源分配方案下的运行性能。第二候选资源规模为第二任务对应的样本资源分配方案所指示的样本资源规模。第二任务的波动系数模型用于预测输入至第二任务的波动系数模型的资源分配方案对应的第二任务的波动系数。第二任务对应的样本资源分配方案对应的第二任务的波动系数用于指示第二任务在第二任务对应的样本资源分配方案下的运行性能和第二任务在独占第二任务对应的样本资源规模的情况下的运行性能之间的差异。
在本申请实施例的方案中，基于任务的波动系数对任务在样本资源分配方案下的运行性能数据进行性能反解，以预测任务在独占资源时的运行性能，有利于实现对各个任务的准确的性能分析，从而为后续算力资源的合理分配提供基础。
结合第一方面，在第一方面的某些实现方式中，每个任务对应的候选资源规模为每个任务对应的候选资源规模中的部分资源规模，每个任务对应的候选资源规模中的部分资源规模与每个任务在独占每个任务对应的所有候选资源规模中的部分资源规模的情况下的运行性能满足第一预设条件。
在本申请实施例的方案中，可以基于第一预设条件从候选资源规模中确定部分资源规模，减少了候选资源规模的组合的数量，即减少了后续目标资源分配过程中的搜索空间，有利于提高资源分配的效率。具体地，基于该任务在独占该任务对应的资源规模的情况下的运行性能与该资源规模确定该部分资源规模，有利于避免在后续的资源分配中为任务分配过多的资源，从而导致资源浪费。
结合第一方面，在第一方面的某些实现方式中，第一预设条件包括：每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能与每个任务对应的候选资源规模中的部分资源规模之间的比值大于或等于第一阈值，第一阈值为任务对资源规模的利用效率的阈值。
在本申请实施例的方案中，可以基于任务在独占该任务对应的资源规模的情况下的运行性能与该资源规模之间的比值，即任务的资源效率，确定该部分资源规模，将较高的资源效率对应的候选资源规模作为该部分候选资源规模，避免在后续的资源分配中为任务分配过多的资源，从而导致资源浪费。
结合第一方面，在第一方面的某些实现方式中，根据多个任务中的每个任务在独占所述每个任务对应的候选资源规模的情况下的运行性能确定多个任务中的每个任务对应的目标资源，包括：根据多个任务中的每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案，多种资源规模组合基于每个任务对应的候选资源规模中的部分 资源规模的组合确定，多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案指示每种资源规模组合下多个任务中的每个任务对应的目标候选资源；确定多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数，每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数分别用于指示多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能和多个任务在独占每种资源规模组合中的每个任务对应的资源规模的情况下的运行性能之间的差异；根据每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数预测多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能；根据多个任务在多种资源规模组合下的目标候选资源分配方案下并行执行的运行性能从多种资源规模组合下的目标候选资源分配方案中确定多个任务中的每个任务对应的目标资源。
在本申请实施例的方案中，基于多个任务独占资源时的运行性能确定各个资源规模组合下的目标候选资源分配方案，并基于目标候选资源分配方案计算各个任务的波动系数，从而确定该多个任务在并行执行时的运行性能，进而确定多个任务中的每个任务对应的目标资源。这样，可以避免对所有资源规模组合下的所有的资源分配方案进行枚举，大大减少了计算量，提高了算力资源的效率。同时，本申请实施例的方案基于波动系数还原了各个任务在并行执行时的运行性能，更符合实际运行情况，有利于提高整体性能的预测准确度，从而进一步提高资源分配的合理性。
结合第一方面，在第一方面的某些实现方式中，根据多个任务中的每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案，包括：根据多个任务中的每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能从每种资源规模组合下的候选资源分配方案中确定每种资源规模组合下的目标候选资源分配方案，每组资源规模组合下的候选资源分配方案所指示的每个任务对应的候选资源连续，多个任务在每种资源规模组合下的目标候选资源分配方案下独占每种资源规模组合中的每个任务对应的资源规模的情况下的整体运行性能优于多个任务在每种资源规模组合下的其他候选资源分配方案下独占每种资源规模组合中的每个任务对应的资源规模的情况下的整体运行性能。
结合第一方面，在第一方面的某些实现方式中，确定多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数，包括：根据多个任务的运行性能数据构建多个任务的波动系数模型，多个任务的波动系数模型分别用于预测输入至多个任务的波动系数模型的资源分配方案对应的多个任务的波动系数；根据多个任务的波动系数模型分别确定多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数。
结合第一方面，在第一方面的某些实现方式中，多个任务包括多个应用对应的任务，多个应用中的每个应用对应一个任务。
结合第一方面，在第一方面的某些实现方式中，多个任务包括一个应用中的多个功能模块对应的任务，多个功能模块中的每个功能模块对应一个任务。
第二方面，提供了一种资源分配装置，包括获取单元，用于获取多个任务中的每个任务的运行性能数据，多个任务中的每个任务的运行性能数据包括每个任务在每个任务对应的样本资源规模下的运行性能；处理单元，用于：根据多个任务中的每个任务的运行性能数据确定多个任务中的每个任务在独占每个任务对应的候选资源规模的情况下的运行性能；根据多个任务中的每个任务在独占每个任务对应的候选资源规模的情况下的运行性能确定多个任务中的每个任务对应的目标资源。
结合第二方面，在第二方面的某些实现方式中，每个任务对应的目标资源包括每个任务对应的一个或多个进程，一个或多个进程分别与一个或多个slot绑定。
结合第二方面，在第二方面的某些实现方式中，多个进程为多个连续的进程，多个slot为多个连续的slot。
结合第二方面，在第二方面的某些实现方式中，多个任务包括第一任务，处理单元具体用于：根据第一任务的运行性能数据构建第一任务的第一性能模型，其中，第一任务的第一性能模型用于预测第一任务在独占输入至第一任务的性能模型的资源规模的情况下的运行性能；根据第一任务的第一性能模型确定第一任务在独占第一任务对应的第一候选资源规模的情况下的运行性能，第一任务对应的候选资源规模包括第一任务对应的第一候选资源规模。
结合第二方面，在第二方面的某些实现方式中，处理单元具体用于：每个任务对应的候选资源规模为每个任务对应的所有候选资源规模中的部分资源规模，每个任务对应的候选资源规模中的部分资源规 模与每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能满足第一预设条件。
结合第二方面，在第二方面的某些实现方式中，第一预设条件包括：每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能与每个任务对应的候选资源规模中的部分资源规模之间的比值大于或等于第一阈值，第一阈值为任务对资源规模的利用效率的阈值。
结合第二方面，在第二方面的某些实现方式中，处理单元具体用于：根据多个任务中的每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案，多种资源规模组合基于每个任务对应的候选资源规模中的部分资源规模的组合确定，多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案指示每种资源规模组合下多个任务中的每个任务对应的目标候选资源；确定多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数，每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数分别用于指示多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能和多个任务在独占每种资源规模组合中的每个任务对应的资源规模的情况下的运行性能之间的差异；根据每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数预测多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能；根据多个任务在多种资源规模组合下的目标候选资源分配方案下并行执行的运行性能从多种资源规模组合下的目标候选资源分配方案中确定多个任务中的每个任务对应的目标资源。
结合第二方面，在第二方面的某些实现方式中，处理单元具体用于：根据多个任务中的每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能从每种资源规模组合下的候选资源分配方案中确定每种资源规模组合下的目标候选资源分配方案，每组资源规模组合下的候选资源分配方案所指示的每个任务对应的候选资源连续，多个任务在每种资源规模组合下的目标候选资源分配方案下独占每种资源规模组合中的每个任务对应的资源规模的情况下的整体运行性能优于多个任务在每种资源规模组合下的其他候选资源分配方案下独占每种资源规模组合中的每个任务对应的资源规模的情况下的整体运行性能。
结合第二方面，在第二方面的某些实现方式中，处理单元具体用于：根据多个任务的运行性能数据构建多个任务的波动系数模型，多个任务的波动系数模型分别用于预测输入至多个任务的波动系数模型的资源分配方案对应的多个任务的波动系数；根据多个任务的波动系数模型分别确定多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数。
结合第二方面，在第二方面的某些实现方式中，多个任务包括多个应用对应的任务，多个应用中的每个应用对应一个任务。
结合第二方面，在第二方面的某些实现方式中，多个任务包括一个应用中的多个功能模块对应的任务，多个功能模块中的每个功能模块对应一个任务。
应理解，在上述第一方面中对相关内容的扩展、限定、解释和说明也适用于第二方面中相同的内容。
第三方面，提供了一种资源分配的装置，包括处理器和存储器，可选地，还包括输入输出接口。其中所述处理器用于控制所述输入输出接口收发信息，所述存储器用于存储计算机程序，所述处理器用于从存储器中调用并运行该计算机程序，使得该装置执行第一方面或第一方面任意一种可能的实现方式中所述的方法。
可选地，该处理器可以是通用处理器，可以通过硬件来实现也可以通过软件来实现。当通过硬件实现时，该处理器可以是逻辑电路、集成电路等；当通过软件来实现时，该处理器可以是一个通用处理器，通过读取存储器中存储的软件代码来实现，该存储器可以集成在处理器中，可以位于该处理器之外，独立存在。
第四方面，提供了一种计算设备集群，包括至少一个计算设备，每个计算设备包括处理器和存储器。至少一个计算设备的处理器用于执行至少一个计算设备的存储器中存储的指令，以使得计算设备集群执行第一方面以及第一方面的任意一种实现方式中的方法。
第五方面，提供一种计算机可读介质，包括计算机程序指令，当计算机程序指令由计算设备集群执行时，计算设备集群执行第一方面以及第一方面的任意一种实现方式中的方法。
第六方面，提供一种包含指令的计算机程序产品，当所述指令被计算设备集群运行时，使得计算设备集群执行上述第一方面以及第一方面的任意一种实现方式中的方法。
作为示例，这些计算机可读存储包括但不限于如下的一个或者多个：只读存储器(read-only memory，ROM)、可编程ROM(programmable ROM，PROM)、可擦除的PROM(erasable PROM，EPROM)、Flash存储器、电EPROM(electrically EPROM，EEPROM)以及硬盘驱动器(hard drive)。
可选地，作为一种实现方式，上述存储介质具体可以是非易失性存储介质。
附图说明
图1为一种算力资源分配的不同层级的示意图。
图2为一种算力资源分配的方法的示意图。
图3为本申请实施例的一种算力资源分配的方法的示意图。
图4为本申请实施例的一种资源分配的装置的示意性框图。
图5为本申请实施例的一种资源分配的方法的示意性流程图。
图6为本申请实施例的另一种资源分配的方法的示意性流程图。
图7为本申请实施例的一种性能矩阵的处理流程的示意图。
图8为本申请实施例的一种装箱算法的处理流程的示意图。
图9为本申请实施例的一种资源分配方案的示意图。
图10为本申请实施例的另一种资源分配方案的示意图。
图11为本申请实施例的一种资源分配的装置的示意性框图。
图12为本申请实施例的一种计算设备的示意性框图。
图13为本申请实施例的一种计算设备集群的示意性框图。
图14为本申请实施例的一种计算设备之间的连接关系的示意性框图。
具体实施方式
下面将结合附图，对本申请实施例中的技术方案进行描述。
本申请将围绕包括多个设备、组件、模块等的系统来呈现各个方面、实施例或特征。应当理解和明白的是，各个系统可以包括另外的设备、组件、模块等，并且/或者可以并不包括结合附图讨论的所有设备、组件、模块等。此外，还可以使用这些方案的组合。
本申请实施例中，“相应的(corresponding，relevant)”和“对应的(corresponding)”有时可以混用，应当指出的是，在不强调其区别时，其所要表达的含义是一致的。
本申请实施例描述的业务场景是为了更加清楚地说明本申请实施例的技术方案，并不构成对于本申请实施例提供的技术方案的限定，本领域普通技术人员可知，随着网络架构的演变和新业务场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
在本说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
本申请中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：包括单独存在A，同时存在A和B，以及单独存在B的情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b，或c中的至少一项(个)，可以表示：a，b，c，a-b，a-c，b-c，或a-b-c，其中a，b，c可以是单个，也可以是多个。
为了更好地说明本申请实施例的方案，下面先对本申请中可能涉及的术语进行说明。
(1)并行应用；
由于并行应用能够显著提升应用性能，目前，并行应用已成为应用开发的一种主流模式。随着应用并行度的增加，应用所需要的算力资源规模迅速增加。
并行应用指一个应用内不同的功能组件并行执行。此外，对于一个组件而言，该组件也可以同时处 理不同数据，以实现组件内并行。
并行应用有利于合理利用算力资源，从而提高应用的运行性能。
具体地，一个应用可以被抽象为展开的计算图，聚类呈若干簇，或者，也可以称为功能模块。每个功能模块可以用于实现应用的某一部分的功能。不同的功能模块可以同时执行。
在本申请实施例中，模块也可以称为功能模块、组件、功能组件或组件模块。
相对来说，模块内部的通信较为紧密，模块之间的通信较为稀疏。
(2)任务调度；
一个应用的执行可以被抽象为任务在计算资源的分配以及维护任务的执行。
并行应用可以将一个应用分解为多个子任务，该多个子任务可以分配给不同的处理器。不同的处理器可以同时处理该多个子任务，从而加速应用的执行。
(3)性能矩阵；
一个应用的性能矩阵可以用于表示该应用中的不同功能模块在不同的资源分配方案条件下运行的性能。例如，性能矩阵可以用于表示不同功能模块在不同的资源分配方案条件下运行所需要消耗的时间。
本申请实施例的方案能够应用于多任务并行的场景中。
在一种可能的实现方式中，该多个任务可以为多个应用。本申请实施例的方案能够用于为该多个应用分配资源。
示例性地，一个或多个用户可以提交多个作业任务到计算系统中，由系统统一管理、运行这些任务。该计算系统可以称为批作业处理系统或批处理系统。本申请实施例的方案可以应用于批处理系统中，为该多个作业任务分配算力资源。
例如，多个用户提交了多个作业任务。该多个作业任务具有消耗算力多、运行时间长、并行度高等特点。在该情况下，为了最大化系统的资源利用率，可以尽可能将该多个作业任务部署到有限的计算节点(比如一个计算节点)上。采用本申请实施例的方案可以将该多个作业任务部署到指定的计算节点上，同时最优化该多个作业任务的整体运行性能，例如，该多个作业任务的整体运行时间。
再如，多个用户提交了多个作业任务。当前系统中能够分配给该多个作业任务的资源是有限的。采用本申请实施例的方案可以在有限的资源中为该多个作业任务分配相应的资源，同时最优化该多个作业任务的整体性能，例如，该多个作业任务的整体运行时间。
在另一种可能的实现方式中，该多个任务可以为一个并行应用的多个子任务。本申请实施例的方案可以应用于并行应用的资源分配场景中，用于为并行应用的多个子任务分配算力资源。
示例性地，本申请实施例的方案可以应用于科学计算或高性能计算(high performance computing，HPC)等并行应用的场景中。
本申请实施例的方案尤其适用于大规模的并行应用场景中，例如，大规模的科学计算、大数据处理以及大规模图计算等并行应用场景中。
例如，通用地球系统模型(community earth system model，CESM)是高性能计算的应用之一，可以用于仿真模拟长期的气候变化情况。该系统由负责各个子系统的模块相互耦合而成。例如，子系统包括大气(atmosphere，ATM)、海洋(ocean，OCN)、海冰(sea ice，ICE)、陆地(land，LND)、径流(river runoff，ROF)、陆冰(land ice，GLC)、海浪(ocean wave，WAV)、耦合器(coupler，CPL)等。模块之间可以通过多点接口(multipoint interface，MPI)交换数据。耦合模式的运行性能是衡量HPC性能的重要指标。
算力资源分配是提升任务的运行性能的重要因素。
图1示出了一种算力资源分配的不同层级的示意图。面向并行应用的算力资源分配通常考虑4个层级：应用层、数据层、系统执行层以及硬件算力层。
如图1所示，应用层从功能角度抽象出多个功能模块。该多个功能模块分别对应多个任务。该多个任务可以并行处理，即任务并行。多个功能模块之间可能存在通信、合作以及依赖等交互关系。数据层从数据角度出发将单个功能模块的处理数据分为多组。该多组数据可以并行处理，即数据并行。多组数据之间可能存在同步或共享等关系。系统执行层包括由操作系统提供的执行抽象，例如，进程或线程等执行抽象，用于承载和运行具体的功能模块来处理特定的数据分组。多个进程可以并行处理，即进程并行。硬件算力层包括具体负责程序运行的硬件单元，即计算资源，例如图1所示的CPU。系统执行层中的一个或多个执行抽象映射到计算资源上，从而最终完成算力资源的分配。系统执行层的并行即为算力 并行。
在相关方案中，算力资源分配策略通常是基于通用算法确定或者由用户确定。然而，通用算法无法感知任务的特点，即无法根据不同的任务的特点来分配算力资源，难以实现任务与算力资源的最佳匹配。人为确定资源分配方案的方式对用户的要求较高，且该方式效率较低，难以满足运行需求。
下面以并行应用的一种算力资源分配的方法为例进行说明。图2示出了一种算力资源分配的方法的示意性流程图。
如图2所示，用户提交作业后，由客户端接口获取用户的作业需求。算力资源管理系统基于用户的作业需求生成等待队列，进而为等待队列中的多个任务分配算力节点。示例性地，系统可以通过通用算法确定算力资源分配策略。或者，如图2所示，用户可以通过命令或插件等形式指示算力资源分配策略。系统执行该算力资源分配策略以实现节点分配。例如，用户可以为不同的应用指定不同的算力资源分配策略或者根据不同的应用分别定制算力资源分配策略，从而提高算力资源分配策略与应用的适配性。具体地，系统基于算力资源分配策略将多个任务实例下发至算力节点，由算力节点中的操作系统(operation system，OS)实现每个任务实例的调度与计算。算力节点中的操作系统调度算力节点中的处理器来计算每个任务实例，或者说，算力节点中的操作系统为每个任务实例分配处理器(如图2中的CPU)，从而实现每个任务实例的计算。
在上述方案中，基于通用框架确定的算力资源分配策略无法感知应用的行为特点，难以实现应用与底层算力资源的最佳匹配。若要进一步提高应用的运行性能，则需要人为指定算力资源分配策略或者人为制定算力资源分配策略，效率较低，且对用户的要求较高。
有鉴于此，本申请实施例提供了一种资源分配的方法，有利于为多个任务分配较为合理的算力资源，从而提高多个任务的整体运行性能。
图3示出了本申请实施例的一种算力资源分配的方法的示意性流程图。
如图3所示，用户提交作业后，可以由客户端接口获取用户的作业需求。算力资源管理系统基于用户的作业需求生成等待队列，进而为等待队列中的多个任务分配资源。系统执行数据采集操作，以得到多个任务的运行性能数据。基于多个任务的运行性能数据对该多个任务进行性能分析，并根据性能分析的结果实现资源分配。
示例性地，系统可以在全局范围内进行资源分配，实现细粒度的资源分配，例如，如图3所示，实现CPU级的算力分配。系统可以将任务实例分配到对应的处理器上。相较于图2而言，无需算力节点调度处理器来实现任务实例，避免了二次调度带来的开销。
应理解，图3中的资源分配的粒度仅为示例，资源分配的粒度还可以为其他粒度。图3中以CPU作为处理器仅为示例，不对本申请实施例的方案构成限定。在其他实现方式中，还可以采用其他处理器。
资源分配的具体描述可以参考后文中的方法500。
为了更好地说明本申请实施例的方案，下面先对本申请实施例的资源分配的装置进行说明。
图4示出了本申请实施例提供的一种资源分配的装置400的示意性框图。示例性地，资源分配的装置400可以部署于图3所示的算力资源管理系统中。
如图4所示，资源分配的装置400可以包括信息采集器410、数据融合器420、应用分析器430以及资源分配器440。
基于用户提交的作业任务可以生成等待队列。资源分配的装置400可以为等待队列中的多个任务分配算力资源。
信息采集器410用于获取该多个任务的信息和可用资源的信息。
进一步地，信息采集器410还可以用于获取用户信息。
数据融合器420用于获取该多个任务的运行性能数据。
应用分析器430用于基于该多个任务的运行性能数据分析该多个任务的性能特性。
具体地，应用分析器430可以用于基于该多个任务的运行性能数据预测该多个任务在独占资源时的运行性能。
资源分配器440用于根据该多个任务的性能特性预测不同的资源分配方案下该多个任务的整体运行性能，从而确定目标资源分配方案。
根据该目标资源分配方案可以将任务实例分配至对应的目标资源上进行处理。
示例性地，目标资源分配方案可以用于指示该多个任务对应的进程。进程可以与硬件处理单元绑定。 这样，可以将任务实例分配至对应的硬件单元上。
下面以该多个任务为并行应用的多个子任务为例对资源分配的装置400进行示例性说明。
基于用户提交的应用可以生成等待队列。该等待队列中的多个任务包括该应用对应的多个子任务。资源分配的装置400可以为等待队列中的多个任务分配算力资源。
信息采集器410用于获取该应用的相关信息。该应用的相关信息包括该应用的多个任务的信息和可用资源的信息。
进一步地，信息采集器410还可以用于获取用户信息。
数据融合器420用于获取该应用的多个应用实例的运行性能数据。该多个应用实例的运行性能数据包括该多个任务的运行性能数据。
应用分析器430用于基于该多个应用实例的运行性能数据分析该应用的性能特性。
具体地，应用分析器430可以用于基于该多个应用实例的运行性能数据预测该多个任务在独占资源时的运行性能。
资源分配器440用于根据该应用的性能特性预测不同的资源分配方案下该应用的整体运行性能，从而确定目标资源分配方案。
根据该目标资源分配方案可以将任务实例分配至对应的目标资源上进行处理。
应理解，图4所示的资源分配的装置仅为示例，不对本申请实施例的方案构成限定。
图5示出了本申请实施例提供的一种资源分配的方法的示意性流程图。图5所示的方法500可以由资源分配的装置执行。示例性地，图5所示的方法可以由图4所示的装置400执行。
如图5所示，方法500包括以下步骤。
510，获取多个任务中的每个任务的运行性能数据。该多个任务中的每个任务的运行性能数据包括每个任务在每个任务对应的样本资源规模下的运行性能。
520，基于该多个任务中的每个任务的运行性能数据确定每个任务在独占该多个任务对应的候选资源规模的情况下的运行性能。
530，根据该多个任务中的每个任务在独占每个任务对应的候选资源规模的情况下的运行性能确定多个任务中的每个任务对应的目标资源。
换言之，步骤530可以理解为，根据该多个任务在独占该多个任务对应的候选资源规模的情况下的运行性能确定目标资源分配方案。目标资源分配方案用于指示该多个任务中的每个任务对应的目标资源。
在本申请实施例的方案中，可以基于多个任务的运行性能数据分析任务，以识别各个任务自身的行为特点，例如，各个任务独占资源时的运行性能，并基于任务自身的行为特点确定多个任务中的每个任务对应的目标资源，有利于提高任务与资源的匹配度，从而提高该多个任务的整体运行性能。
可选地，方法500还可以包括步骤501(图中未示出)。
501，获取该多个任务的信息和可用资源的信息。
示例性地，步骤501可以由图4中的信息采集器410执行。
该多个任务的信息可以用于指示该多个任务或该多个任务的数量。
可用资源的信息可以用于指示可用资源或可用资源的规模。例如，可用资源的信息可以由硬件参数表示。
示例性地，可用资源的规模可以为可用的算力节点的数量。
例如，算力节点可以为终端设备。再如，算力节点可以为服务器。
示例性地，可用资源的规模可以为可用的处理器的数量。
例如，处理器可以包括中央处理器(central processing unit，CPU)、图形处理器(graphics processing unit，GPU)或者数字信号处理器(digital signal processor，DSP)等处理器中的任意一种或多种。
示例性地，可用资源的规模可以为可用的处理器核的数量。
例如，处理器为CPU，处理器核为CPU核。
此外，方法500还可以包括：获取其他与该多个任务的资源分配相关的信息。例如，提交该多个任务的用户的信息。
可选地，该多个任务包括多个应用对应的任务。
换言之，该多个任务和该多个应用可以是一一对应的，即每个应用可以对应一个任务。
示例性地，该多个应用可以是由至少一个用户指示的。
例如，该多个任务可以包括用户#1提交的压缩任务以及用户#2提交的数据处理任务和压缩任务，即该多个任务可以包括两个压缩任务以及一个数据处理任务。
一个或多个用户可以提交多个作业任务到计算系统中，由系统统一管理、运行这些任务。该计算系统可以称为批作业处理系统或批处理系统。本申请实施例的方案可以应用于批处理系统的资源分配场景中，为该多个作业任务分配算力资源。
可选地，该多个任务包括一个应用中的多个功能模块对应的任务。
换言之，该多个任务可以包括一个应用的多个子任务。每个功能模块对应一个任务。该应用为并行应用。
例如，该多个任务可以包括CESM中负责ATM、OCN以及ICE的功能模块对应的任务。
用户可以提交应用任务到计算系统中，该应用为并行应用。本申请实施例的方案可以应用于并行应用的资源分配场景中，用于为并行应用的多个子任务分配算力资源。本申请实施例的方案应用于并行应用的资源分配场景的具体描述可以参考后文中的方法600。
应理解，以上仅为示例，不对本申请实施例中的多个任务构成限定，只要该多个任务能够并行执行即可。例如，该多个任务可以包括应用#1对应的任务和应用#2中的多个功能模块对应的任务，应用#1可以为非并行应用，应用#2可以为并行应用。
示例性地，任务的运行性能可以包括任务的运行时间、任务的响应时延、任务的资源利用率或通过硬件性能监控单元(performance monitoring unit,PMU)计数器采集的数据等。
一个任务在一种资源规模下的运行性能可以包括以下至少一项：该任务在独占该资源规模的情况下的运行性能；该任务在非独占该资源规模的情况下的运行性能。
该任务独占该资源规模的情况指的是为该任务分配的资源由该任务独占的情况。
该任务非独占该资源规模的情况指的是为该任务分配的资源的至少部分资源与其他任务共享的情况，即该任务与该任务共享至少部分资源的其他任务并行执行的情况。
一个任务独占该任务对应的资源规模，可以理解为，在该任务对应的资源上仅执行该任务。相对应地，一个任务非独占该任务对应的资源规模，可以理解为，该任务对应的资源上执行同时执行该任务和其他任务。或者，该任务和其他任务争抢该任务对应的资源。为了便于描述，在本申请实施例中，将任务独占该任务对应的资源规模简称为该任务独占资源，任务非独占该任务对应的资源规模简称为该任务不独占资源或任务非独占资源。
可选地，该多个任务中的至少一个任务的运行性能数据包括该至少一个任务在非独占该至少一个任务对应的样本资源规模的情况下的运行性能。
在实际应用的场景中，通常任务不独占资源。在本申请实施例的方案中，可以根据基于任务在不独占资源时的运行性能确定任务在独占资源时的运行性能，减少了对该多个任务的运行性能数据的限制。本申请实施例的方案中获取到的多个任务的运行性能既可以包括任务独占资源的情况下的运行性能，还可以包括任务在不独占资源的情况下的运行性能，提高了获取到的运行性能数据的数据量，为后续的处理过程奠定了数据基础，从而有利于更准确地分析该多个任务，进而实现合理的资源分配。
进一步地，在该多个任务包括一个应用中的多个功能模块对应的任务的情况下，步骤510可以包括：获取该应用的多个应用实例的运行性能数据。该多个应用实例的运行性能数据包括该多个任务的运行性能数据。
其中，每个应用实例的运行性能数据可以包括该多个任务中的部分或全部任务的运行性能数据。
每个任务对应的样本资源规模可以包括一种样本资源规模，也可以包括多种样本资源规模。
换言之，每个任务的运行性能数据包括该任务在至少一种样本资源规模下的运行性能。
不同任务对应的样本资源规模可以相同，也可以不同。
下面对任务对应的样本资源规模的确定方式进行示例性说明。
示例性地，任务对应的样本资源规模可以是用户确定的。
可替换地，任务对应的样本资源规模可以是随机生成的。
可替换地，任务对应的样本资源规模可以基于该任务对应的默认资源规模生成的。
例如，任务对应的默认资源规模可以为预先为该任务设置的资源规模。再如，任务对应的默认资源规模可以是基于通用算法确定的。
例如，任务对应的样本资源规模可以包括该任务对应的默认资源规模和从该默认资源规模周围的资源规模中选择的资源规模。该默认资源规模周围的资源规模可以为与该默认资源规模之间的差值小于或等于规模阈值的资源规模。比如，默认资源规模为5个进程，规模阈值为3，则该默认资源规模的周围的资源规模即为[2,8]范围内的资源规模，从中选择部分或全部，与默认资源规模共同作为样本资源规模。其中，规模阈值可以是预先设置的固定值。或者，规模阈值也可以是根据默认资源规模确定的值。本申请实施例对此不做限定。
应理解，以上仅为示例，任务对应的样本资源规模还可以通过其他方式确定。
对于不同的任务，其对应的样本资源规模的确定方式可以相同，也可以不同。
在本申请实施例中，资源也可以称为算力、算力资源或计算资源。
示例性地，资源可以包括进程资源。一个单位资源可以为一个进程。资源规模可以为进程的数量。
资源分配方案用于指示任务对应的资源，即为任务分配的资源。不同的资源规模下的资源分配方案不同。每个任务的运行性能数据包括该任务在至少一种样本资源规模下的运行性能可以理解按照如下方式理解。每个任务的运行性能数据包括该任务在至少一种样本资源分配方案下的运行性能，该至少一个样本资源分配方案所指示的该任务对应的资源规模包括至少一种资源规模。
示例性地，样本资源分配方案为任务指示的资源可以是连续的。例如，样本资源分配方案为任务指示的进程可以是连续的。为任务分配的进程可以由为该任务分配的起始进程的ID和终止进程的ID表示。或者，为任务分配的进程可以由为该任务分配的起始进程的ID和为该任务分配的进程的数量表示。或者，为任务分配的进程还可以通过其他参数表示。
示例性地，步骤510可以由图4中的数据融合器420执行。
下面对任务的运行性能数据的获取方式进行示例性说明。
示例性地，从任务的历史运行数据中获取该任务的运行性能数据。
例如，从该任务的历史运行数据中采集该任务对应的样本资源规模下的运行数据。
可替换地，从正在运行的任务实例的监控数据中获取该任务的运行性能数据。
例如，当前正在运行的任务实例处于该任务对应的样本资源规模下，可以从监控数据中获取该任务实例的运行性能作为该任务的运行性能数据。
可替换地，在任务对应的样本资源规模下预执行该任务。
基于任务对应的样本资源规模为该任务分配资源，在该资源下预执行该任务一段时间，从监控数据中获取该任务的运行性能数据。
一个任务的运行性能数据可以采用上述任一种方式获取，也可以采用上述多种方式共同获取。例如，对于一个任务对应的不同的样本资源规模，可以采用不同的方式采集对应的运行性能。
不同任务的运行性能数据的获取方式可以相同，也可以不同。
应理解，以上任务的运行性能数据的获取方式仅为示例，不对本申请实施例的方案构成限定。
示例性地，步骤520可以由图4中的应用分析器430执行。
示例性地，在步骤520中，对于一个任务而言，可以根据该任务的运行性能数据确定该任务在独占该任务对应的候选资源规模的情况下的运行性能。
一个任务对应的样本资源规模和该任务对应的候选资源规模可以相同，也可以不同。
每个任务对应的候选资源规模可以包括一种候选资源规模，也可以包括多种候选资源规模。
对于不同的任务而言，其对应的候选资源规模可以相同，也可以不同。
例如，该多个任务可以包括任务#1和任务#2。任务#1对应的候选资源规模包括两个单位资源和三个单位资源。任务#2对应的候选资源规模包括两个单位资源。在步骤520中，可以根据任务#1的运行性能数据确定任务#1在独占三个单位资源的情况下的运行性能以及任务#1在独占两个单位资源的情况下的运行性能，根据任务#2的运行性能数据确定任务#2在独占两个单位资源的情况下的运行性能。
下面结合三个示例(示例1，示例2和示例3)对步骤520进行示例性说明。
示例1；
可选地，多个任务可以第一任务。根据第一任务的运行性能数据构建第一任务的第一性能模型，其中，第一任务的第一性能模型用于预测第一任务在独占输入至第一任务的第一性能模型的资源规模的情况下的运行性能；根据第一任务的第一性能模型确定第一任务在独占第一任务对应的第一候选资源规模的情况下的运行性能。第一任务对应的候选资源规模包括第一任务对应的第一候选资源规模。
第一任务可以为该多个任务中的任一任务。
对于该多个任务中的任一任务而言，可以根据该任务的运行性能构建该任务的第一性能模型。任务的第一性能模型用于预测任务在独占该任务对应的资源规模的情况下的运行性能。任务的第一性能模型的输入可以包括该任务对应的资源规模，该任务的第一性能模型的输出可以包括该任务在独占该任务对应的资源规模的情况下的运行性能。将第一候选资源规模输入至第一任务的对性能模型中，可以预测出第一任务在独占第一任务对应的第一候选资源规模的情况下的运行性能。
或者，也可以按照如下方式理解，第一任务的第一性能模型用于指示第一任务对应的资源规模与第一任务在独占该资源规模的情况下的运行性能之间的映射关系。基于该映射关系以及第一候选资源规模，可以确定第一任务在独占第一任务对应的第一候选资源规模的情况下的运行性能。
示例性地，该多个任务中的其他任务也可以参考上述方式确定运行性能。例如，步骤520可以包括：根据该多个任务中的每个任务的运行性能数据分别构建每个任务的第一性能模型；分别根据每个任务的第一性能模型确定每个任务在独占对应的第一候选资源规模的情况下的运行性能。
作为任务的第一性能模型的输入的资源规模均可以作为该任务对应的第一候选资源规模。第一候选资源规模可以包括一种，也可以包括多种。对于不同的任务而言，其对应的第一候选资源规模可以相同，也可以不同。
例如，该多个任务可以包括任务#1和任务#2。任务#1对应的候选资源规模包括两个单位资源和三个单位资源。任务#2对应的候选资源规模包括两个单位资源。任务#1对应的第一候选资源规模可以为两个单位资源。在该情况下，可以将两个单位资源输入至任务#1的第一性能模型中，由任务#1的第一性能模型预测任务#1在独占两个单位资源的情况下的运行性能。任务#2在独占两个单位资源的情况下的运行性能可以通过其他方式确定，在该情况下，任务#2对应的候选资源规模不包括任务#2对应的第一候选资源。
示例性地，第一性能模型可以为AI模型。例如，该第一性能模型可以为神经网络模型。根据任务的运行性能数据构建该任务的第一性能模型，可以理解为，以该任务的运行性能数据作为训练数据进行训练，得到该任务的第一性能模型。
可替换地，第一性能模型也可以其他模型。具体描述可以参考方法600。
本申请实施例中，通过多个任务的运行性能数据对该多个任务进行性能建模，有利于实现对各个任务的准确的性能分析，从而为后续算力资源的合理分配提供基础。
示例2；
可选地，方法500还包括：根据该多个任务的运行性能数据分别构建该多个任务的波动系数模型。该多个任务的波动系数模型分别用于预测输入至该多个任务的波动系数模型的资源分配方案对应的多个任务的波动系数。
对于一个任务而言，该任务的波动系数模型可以用于预测输入至该波动系数模型的资源分配方案对应的该任务的波动系数。该波动系数模型的输入为资源配置方案，输出为该资源配置方案对应的该任务的波动系数，或者说，该任务在该资源配置方案下的波动系数。
或者可以按照如下方式理解，任务的波动系数模型用于指示资源分配方案与该任务在该资源分配方案下的波动系数之间的映射关系。
一个资源分配方案对应的一个任务的波动系数用于指示该任务在该资源分配方案下的运行性能和该任务在独占该任务对应的资源规模的情况下的运行性能之间的差异。该任务对应的资源规模由该资源分配方案指示。
示例性地，一个资源分配方案对应的一个任务的波动系数可以为该任务在该资源分配方案下的运行性能与该任务在独占该任务对应的资源规模的情况下的运行性能之间的比值。该任务对应的资源规模由该资源分配方案指示。
示例性地，波动系数模型可以为AI模型。例如，该波动系数模型可以为神经网络模型。根据该多个任务中的每个任务的运行性能数据构建每个任务的波动系数模型，可以理解为，以每个任务的运行性能数据作为训练数据进行训练，得到每个任务的波动系数模型。
可替换地，波动系数模型也可以其他模型。具体描述可以参考方法600。
可选地，该多个任务包括第二任务。根据该多个任务中的第二任务的波动系数模型和第二任务的运行性能数据确定第二任务在独占第二任务对应的第二候选资源规模的情况下的运行性能。第二任务对应的候选资源规模包括第二任务对应的第二候选资源规模。第二任务的运行性能数据包括第二任务在第二 任务对应的样本资源分配方案下的运行性能。第二候选资源规模为第二任务对应的样本资源分配方案所指示的样本资源规模。
第二任务可以为该多个任务中的任一任务。对于该多个任务中的任一任务而言，可以根据该任务的运行性能数据构建该任务的波动系数模型。该任务的波动系数模型用于预测输入至该任务的波动系数模型的资源配置方案对应的该任务的波动系数。输入至该任务的波动系数模型的资源配置方案为该任务对应的资源配置方案。任务的波动系数模型的输入可以包括资源分配方案，该任务的波动系数模型的输出可以包括该资源分配方案对应的该任务的波动系数，即该任务在该资源分配方案下的波动系数。
或者，也可以按照如下方式理解。该任务的波动系数模型用于指示输入至该波动系数模型的资源分配方案与该任务在该资源分配方案下的波动系数之间的映射关系。基于该映射关系以及该任务对应的资源分配方案，可以确定该任务在该任务对应的资源分配方案下的波动系数。
在步骤520中，可以将第二任务对应的样本资源分配方案输入至第二任务的波动模型中，由第二任务的波动模型预测出第二任务在该样本资源分配方案下的波动系数。基于该波动系数和第二任务在该样本资源分配方案下的运行性能可以预测第二任务在独占第二任务对应的第二候选资源规模的情况下的运行性能。第二任务对应的第二候选资源规模为该样本资源分配方案所指示的样本资源规模。
基于一个任务在该任务对应的资源分配方案下的运行性能确定该任务在独占该任务对应的资源分配方案所指示的资源规模的情况下的运行性能的过程可以称为性能反解。
示例性地，该多个任务中的其他任务也可以参考上述方式确定运行性能。例如，步骤520可以包括：分别根据每个任务的波动系数模型和每个任务的运行性能数据确定每个任务在独占每个任务对应的第二候选资源规模的情况下的运行性能。每个任务的运行性能数据包括每个任务在每个任务对应的样本资源分配方案下的运行性能。每个任务对应的第二候选资源规模即为每个任务对应的样本资源分配方案所指示的样本资源规模。每个任务对应的候选资源规模包括每个任务对应的样本资源规模。
作为任务的波动系数模型的输入的资源分配方案所指示的资源的规模均可以作为该任务对应的第二候选资源规模。第二候选资源规模可以包括一种，也可以包括多种。对于不同的任务而言，其对应的第二候选资源规模可以相同，也可以不同。
例如，该多个任务可以包括任务#1和任务#2。任务#1对应的候选资源规模包括两个单位资源和三个单位资源。任务#2对应的候选资源规模包括两个单位资源。任务#1对应的样本资源分配方案指示的资源的规模为三个单位资源。即任务#1对应的第二资源规模可以为三个单位资源。在该情况下，可以将任务#1对应的样本资源分配方案输入至任务#1的波动系数模型中，由任务#1的波动系数模型预测任务#1在该样本资源分配方案下的波动系数。基于该波动系数对任务#1在任务#1对应的资源分配方案下的运行性能进行性能反解，以得到任务#1在独占三个单位资源的情况下的运行性能。任务#2在独占两个单位资源的情况下的运行性能可以通过其他方式确定，在该情况下，任务#2对应的候选资源规模不包括任务#2对应的第二候选资源规模。
在本申请实施例的方案中，基于任务的波动系数对任务在样本资源分配方案下的运行性能数据进行性能反解，以预测任务在独占资源时的运行性能，有利于实现对各个任务的准确的性能分析，从而为后续算力资源的合理分配提供基础。
示例3；
可选地，步骤510中获取的多个任务的运行性能数据包括该多个任务中的第三任务在独占第三任务对应的第三候选资源规模的情况下的运行性能。第三任务对应的候选资源规模包括第三任务对应的第三候选资源规模。
第三任务可以为该多个任务中的任一任务。对于该多个任务中的任一任务而言，若在步骤510中获取的该任务的运行性能数据包括任务在独占资源的情况下的运行性能，则该任务独占的资源规模即可作为该任务对应的第三候选资源规模，相应地，该任务在独占该资源的情况下的运行性能即为该任务在独占该任务对应的第三候选资源规模的情况下的运行性能。
第三候选资源规模可以包括一种，也可以包括多种。对于不同的任务而言，其对应的第三候选资源规模可以相同，也可以不同。
例如，该多个任务可以包括任务#1和任务#2。任务#1对应的候选资源规模包括两个单位资源和三个单位资源。任务#2对应的候选资源规模包括两个单位资源。在步骤510中获取的多个任务的运行性能数据包括任务#2在独占两个单位资源的情况下的运行性能。在该情况下，任务#2对应的第三候选资源规模 可以为两个单位资源。任务#1在独占两个单位资源和三个单位资源的情况下的运行性能可以通过其他方式确定，在该情况下，任务#1对应的候选资源规模不包括任务#3对应的第三候选资源规模。
第一任务、第二任务和第三任务可以相同，也可以不同。
换言之，对于一个任务而言，可以采用上述三种示例中部分或全部的方式确定该任务在独占该任务对应的候选资源规模的情况下的运行性能。例如，若采用上述三种示例确定该任务在独占该任务对应的候选资源规模的情况下的运行性能，则该任务对应的候选资源规模包括该任务对应的第一候选资源规模，该任务对应的第二候选资源规模，和该任务对应的第三候选资源规模，分别采用对应的方式确定该任务在独占该任务对应的候选资源规模的情况下的运行性能。
应理解，以上仅为示例，步骤520还可以通过其他方式实现。具体描述可以参考后文中的方法600，此处不展开描述。
示例性地，多个任务在独占多个任务对应的候选资源规模的情况下的运行性能可以通过该多个任务的性能矩阵表示。
该多个任务的性能矩阵中的不同位置处的元素可以用于表示该位置所对应的任务在独占该位置所对应的资源规模的情况下的运行性能。
示例性地，步骤530可以由图4中的资源分配器440执行。
步骤530可以包括：根据多个任务在独占多个任务对应的目标候选资源规模的情况下的运行性能确定目标资源分配方案。多个任务对应的候选资源规模包括多个任务对应的目标候选资源规模。
每个任务对应的目标候选资源规模可以包括一种，也可以包括多种。对于不同的任务而言，其对应的目标候选资源规模可以相同，也可以不同，其对应的目标候选资源规模的确定方式可以相同，也可以不同。
每个任务对应的目标候选资源规模可以为每个任务对应的所有候选资源规模中的部分资源规模。
换言之，每个任务对应的候选资源规模可以为每个任务对应的所有候选资源规模中的部分资源规模。为了便于描述，在本申请实施例中，将该部分资源规模称为目标候选资源规模。
若每个任务对应的候选资源规模较多，则该多个任务对应的候选资源规模的组合较多，确定目标资源分配方案时所涉及的计算量可能过大，影响资源分配的效率。本申请实施例提供了如下方案以便从候选资源规模中确定部分资源规模，减少了资源规模的组合的数量，即减少了后续目标资源分配过程中的搜索空间，有利于提高资源分配的效率。
可选地，多个任务中的第四任务在独占第四任务对应的目标候选资源规模的情况下的运行性能满足预设条件#1，该多个任务中的第五任务对应的目标候选资源规模满足预设条件#2，或者，该多个中的第六任务对应的目标候选资源规模和第六任务在独占第六任务对应的目标候选资源规模的情况下的运行性能满足预设条件#3(第一预设条件的一例)。
第四任务、第五任务和第六任务可以相同，也可以不同。
预设条件可以根据需要设置。对于不同的任务而言，预设条件可以相同，也可以不同。
下面对预设条件#1进行示例性说明。
可选地，预设条件#1可以包括：第四任务在独占第四任务对应的目标候选资源规模的情况下的运行性能大于或等于阈值#1。
第四任务可以为该多个任务中的任一任务。第四任务与第一任务、第二任务、第三任务可以相同，也可以不同。
示例性地，阈值#1可以是人为设定的，或者，阈值#1也可以是预先设置的。
在一个任务对应的候选资源规模中，若任务在独占一个候选资源规模的情况下的运行性能大于或等于阈值#1，则该候选资源规模可以作为该任务对应的目标候选资源规模。
该多个任务中的其他任务对应的目标候选资源规模也可以参考第四任务对应的目标候选资源规模确定。
可选地，该多个任务在独占该多个任务对应的目标候选资源规模的情况下的运行性能大于或等于阈值#1。
应理解，以上仅为示例。例如，在实际应用中，还可以为各个任务设置不同的阈值#1。本申请实施例对此不做限定。
在本申请实施例中，可以基于任务的运行性能确定目标候选资源规模，将较高的运行性能对应的候 选资源规模作为目标候选资源规模，有利于在后续的资源分配中为任务分配合适的资源以保证任务的运行性能。
以上仅为对预设条件#1的示例，在其他可能的场景中，预设条件#1还可以为其他与任务的运行性能相关的条件。
下面对预设条件#2进行示例性说明。
可选地，预设条件#2可以包括：第五任务对应的目标候选资源规模大于或等于阈值#2。
第五任务可以为该多个任务中的任一任务。第五任务与第一任务、第二任务、第三任务可以相同，也可以不同。
对于一个任务而言，若一个候选资源规模大于或等于阈值#2，则该候选资源规模可以作为该任务对应的目标候选资源规模。
示例性地，阈值#2可以是人为设定的，或者，阈值#2也可以是预先设置的。
该多个任务中的其他任务对应的目标候选资源规模也可以参考第五任务对应的目标候选资源规模确定。
可选地，该多个任务对应的目标候选资源规模大于或等于阈值#2。
应理解，以上仅为示例。例如，在实际应用中，还可以为各个任务设置不同的阈值#2。本申请实施例对此不做限定。
当任务对应的候选资源规模较小时，任务的并行效率较低，影响该多个任务的整体运行性能。根据本申请实施例的方案，避免将过低的候选资源规模作为目标候选资源规模，有利于后续为任务分配合适的资源以保证任务的并行效率。
以上仅为对预设条件#2的示例，在其他可能的场景中，预设条件#2还可以为其他与候选资源规模相关的条件。
下面对预设条件#3进行示例性说明。
可选地，预设条件#3包括：第六任务在独占第六任务对应的目标候选资源规模的情况下的运行性能与第六任务对应的目标候选资源规模之间的比值大于或等于阈值#3(第一阈值的一例)。
第六任务可以为该多个任务中的任一任务。第六任务与第一任务、第二任务、第三任务可以相同，也可以不同。
对于一个任务而言，该任务在独占该任务对应的资源规模的情况下的运行性能与该资源规模之间的比值可以称为资源效率，即任务对资源规模的利用效率。阈值#3可以作为效率阈值，即任务对资源规模的利用效率的阈值。在一个任务对应的候选资源规模中，若一个候选资源规模对应的资源效率大于或等于效率阈值，则该候选资源规模可以作为该任务对应的目标候选资源规模。
示例性地，阈值#3可以是人为设定的，或者，阈值#3也可以是预先设置的。
通常随着给任务分配的资源规模的增加，任务的资源效率会逐渐降低。示例性地，资源效率明显降低前后对应的资源效率可以作为效率阈值。
该多个任务中的其他任务对应的目标候选资源规模也可以参考第六任务对应的目标候选资源规模确定。
可选地，每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能与每个任务对应的候选资源规模中的部分资源规模之间的比值大于或等于阈值#3。
换言之，该多个任务在独占该多个任务对应的目标候选资源规模的情况下的运行性能与该多个任务对应的目标候选资源规模之间的比值大于或等于阈值#3。
应理解，以上仅为示例。例如，在实际应用中，还可以为各个任务设置不同的效率阈值。本申请实施例对此不做限定。
在一定范围内，任务的运行性能会随着为该任务分配的资源规模的增大而显著提高，但在为该任务分配的资源规模达到一定的阈值后，任务的运行性能不会再明显增加，即任务的资源效率降低。当资源效率较低时，会导致资源浪费。本申请实施例中，基于该任务在独占该任务对应的资源规模的情况下的运行性能与该资源规模确定目标候选资源规模，有利于避免在后续的资源分配中为任务分配过多的资源，从而导致资源浪费。例如，在本申请实施例中，可以基于任务在独占该任务对应的资源规模的情况下的运行性能与该资源规模之间的比值，即任务的资源效率，确定目标候选资源规模，将较高的资源效率对应的候选资源规模作为目标候选资源规模，避免在后续的资源分配中为任务分配过多的资源，从而导致 资源浪费。
以上仅为对预设条件#3的示例，在其他可能的场景中，预设条件#3还可以为其他与任务的运行性能以及任务对应的候选资源规模相关的条件。
示例性地，在步骤530中，可以枚举多个资源规模组合中的每个资源规模组合下的所有资源分配方案，计算所有资源分配方案下的该多个任务的整体运行性能，进而求出最优解。最优解指的是使得该多个任务的整体运行性能最优的资源分配方案，该最优解可以作为目标资源分配方案。该多个资源规模组合基于该多个任务对应的目标候选资源规模的组合确定。
在资源规模组合的数量较多时，该方式的复杂度较高，运行时间较长。本申请实施例提供了如下方案来确定目标资源分配方案，有利于在较短的时间内得到较合理的目标资源分配方案，从而保证该多个任务的整体运行性能。
可选地，步骤530可以通过如下步骤实现。
步骤531，根据该多个任务中的每个任务在独占每个任务对应的目标候选资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案。该多种资源规模组合基于该多个任务对应的目标候选资源规模的组合确定。多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案指示每种资源规模组合下多个任务中的每个任务对应的目标候选资源。
步骤532，确定该多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的该多个任务的波动系数。每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数分别用于指示该多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能与该多个任务在独占每种资源规模组合中的每个任务对应的资源规模的情况下的运行性能之间的差异。
每种资源规模组合中的每个任务对应的资源规模，即为，每种资源规模组合中的每个任务对应的目标候选资源规模。
步骤533，根据每种资源规模组合下的目标候选资源分配方案对应的该多个任务的波动系数和该多个任务在独占每种资源规模组合中的每个任务对应的目标候选资源规模的情况下的运行性能分别预测该多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能。
步骤534，根据该多个任务在多种资源规模组合下的目标候选资源分配方案下并行执行的运行性能从多种资源规模组合下的目标候选资源分配方案中确定目标资源分配方案。
示例性地，该多种资源规模组合包括该多个任务对应的目标候选资源规模的组合。从每个任务对应的所有目标候选资源规模中分别选择一个目标候选资源规模，即构成了一个资源规模组合。换言之，一个资源规模组合可以包括每个任务对应的一个目标候选资源规模。
在步骤531中可以分别确定每种资源规模组合下的目标候选资源分配方案。
步骤531可以包括：根据该多个任务中的每个任务在独占每个任务对应的目标候选资源规模的情况下的运行性能分别从多个资源规模组合下的候选资源分配方案中确定多种资源规模组合下的目标候选资源分配方案。
示例性地，该多个任务在每种资源规模组合下的目标候选资源分配方案下独占每种资源规模组合中的每个任务对应的目标候选资源规模的情况下的整体运行性能优于多个任务在每种资源规模组合下的其他候选资源分配方案下独占每种资源规模组合中的每个任务对应的目标候选资源规模的情况下的整体运行性能。
换言之，对于该多种资源规模组合中的一种资源规模组合而言，可以从该资源规模组合下的候选资源分配方案中选择该资源规模组合下的目标候选资源分配方案。该资源规模组合下的目标候选资源分配方案可以为该资源规模组合下的所有候选资源分配方案中的最优候选资源分配方案。最优候选资源分配方案即为使得该多个任务在独占该资源规模组合中的各个任务对应的目标候选资源规模的情况下的整体运行性能最优的候选资源分配方案。
一种资源规模组合下的候选资源分配方案可以包括满足该资源规模组合的所有资源分配方案。换言之，一个资源规模组合下的候选资源分配方案所指示的各个任务对应的资源规模与该资源规模组合一致。
对于每种资源规模组合而言，该资源规模组合下的目标候选资源分配方案可以为，该资源规模组合下的所有候选资源分配方案中使得该多个任务在独占资源时的整体运行性能最优的候选资源分配方案。
可选地，每组资源规模组合下的候选资源分配方案所指示的每个任务对应的候选资源连续。
候选资源分配方案所指示的各个任务对应的资源可以是连续的。在该情况下，每个任务对应的资源 可以由该任务对应的资源的起始位置以及该任务对应的资源规模指示。
例如，各个资源规模组合下的目标候选资源分配方案可以通过装箱算法确定。具体描述可以参考方法600中的相关描述。
示例性地，步骤531可以利用装箱的算法实现。具体描述可以参考方法600。
在步骤532中，对于每种资源规模组合而言，可以分别确定该资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数。
对于一种资源规模组合而言，可以仅计算该资源规模组合下的目标候选资源分配方案对应的部分任务的波动系数。该部分任务指的是在该目标候选资源分配方案中，与其他任务共享资源的任务。例如，在目标候选资源分配方案中，为任务#1分配的资源包括进程#1，为任务#2分配的资源也包括进程#1，为任务#3分配的资源未分配给其他任务，则在目标候选资源分配方案下，可以计算其对应的任务#1的波动系数和任务#2的波动系数，而无需计算任务#3的波动系数。
其中，每个任务的波动系数用于指示该任务在该目标资源分配方案下与其他任务并行执行时的运行性能和该任务在独占该资源规模组合中的该任务对应的目标候选资源规模的情况下的运行性能之间的差异。
示例性地，每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数分别用于指示该多个任务在该资源规模组合下的目标候选资源分配方案下并行执行的运行性能与该多个任务在独占该资源规模组合中的该多个任务对应的目标候选资源规模的情况下的运行性能之间的比值。
换言之，对于一个任务而言，每种资源规模组合下的目标候选资源分配方案对应的该任务的波动系数用于指示该任务在该目标候选资源分配方案下和其他任务并行执行时的运行性能与该任务在独占该资源规模组合中的该任务对应的目标候选资源规模的情况下的运行性能之间的比值。
可选地，步骤532包括：根据多个任务的波动系数模型确定该多种资源规模组合中的中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数。
如前所述，对于一个任务而言，该任务的波动系数模型可以用于预测输入至该波动系数模型的资源分配方案对应的该任务的波动系数。
对于每种资源规模组合而言，将该资源规模组合下的目标候选资源分配方案分别输入至多个任务的波动系数模型中，即可得到该目标候选资源分配方案对应的多个任务的波动系数。
具体地，步骤533可以包括：根据多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数和该多个任务在独占每种资源规模组合中的每个任务对应的目标候选资源规模的情况下的运行性能确定多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能；根据该多个任务在多种资源规模组合下的目标候选资源分配方案下并行执行的运行性能确定该多个任务在多种资源规模组合对应的目标候选资源分配方案下并行执行的整体运行性能。
如前所述，对于每种资源规模组合而言，一个任务的波动系数用于指示该任务在该目标资源分配方案下与其他任务并行执行时的运行性能和该任务在独占该资源规模组合中的该任务对应的目标候选资源规模的情况下的运行性能之间的差异。根据该任务的波动系数和该任务在独占该任务对应的目标资源规模的情况下的运行性能即可计算出该任务在该目标资源分配方案下与其他任务并行执行时的运行性能。基于该多个任务在该目标资源分配方案下并行执行的运行性能可以确定该多个任务在该目标资源分配方案下并行执行的整体运行性能。
示例性地，在步骤534中，可以将整体运行性能最优的目标候选资源分配方案作为目标资源分配方案。
可选地，该每个任务对应的目标资源包括每个任务对应的一个或多个进程。该一个或多个进程分别与一个或多个slot绑定。
slot和处理器核可以是一一对应的。一个或多个进程分别与一个或多个slot绑定，也可以理解为一个或多个进程分别与一个或多个处理器核绑定。
在相关方案中，将任务分配至算力节点后，可以由算力节点为任务分配处理器核，从而实现任务的计算。然而算力节点内没有全局信息，即算力节点内没有全部可用资源的信息，该相关方案难以得到最优的资源分配方案。在本申请实施例的方案中，将进程与slot绑定，或者说，将进程与处理器核绑定，可以在全局范围内实现进程的分配，也即实现全局范围内的处理器核的分配，有利于得到最优的资源分配方案。同时本申请实施例的方案，将进程与处理器核绑定，实现了精细化的算力分配，无需算力节点 内核的二次调度，即无需由算力节点为进程调度处理器核，避免了二次调度所带来的开销。
可选地，每个任务对应的目标资源包括每个任务对应的一个或多个进程。该多个进程分别与多个连续的slot绑定。
多个slot为多个连续的slot指的是该多个slot的编号连续。
可选地，该多个进程为多个连续的进程。该多个slot为多个连续的slot。
在本申请实施例中，为各个任务分配连续的slot，有利于减少各个任务执行过程中的通信代价，进一步提高任务的运行性能。
可选地，方法500还包括：输出该目标资源分配方案的指示信息。
图6示出了本申请实施例提供的一种资源分配的方法的示意性流程图。图6所示的方法600可以视为图5所述的方法500的一种具体实现方式。相关描述可以参考方法500，为避免重复，在描述方法600时适当省略部分描述。方法600可以由资源分配的装置执行。该装置可以作为一个模块部署于应用中，或者，部署于应用以外，例如，部署于计算资源的管理和控制系统中。
如图6所示，方法600包括以下步骤。
步骤610，获取应用的相关信息。应用的相关信息包括应用中的多个功能模块的信息和可用资源的信息。
示例性地，用户可以提交应用的处理请求。资源分配的装置可以获取该应用的相关信息。
应用中的多个功能模块的信息可以用于指示该应用中的多个功能模块或该应用中的功能模块的数量。
应用中的功能模块指的是当前应用的执行过程中启用的功能模块。应用中的多个功能模块分别对应方法500中的多个任务。为了便于理解，在方法600中，将方法500中的任务替换为功能模块来进行描述。
可用资源的信息用于指示可用资源或可用资源的规模。例如，可用资源的信息可以由硬件参数表示。
示例性地，可用资源的规模可以为可用的算力节点的数量。
示例性地，可用资源的规模可以为可用的处理器的数量。
示例性地，可用资源的规模可以为可用的处理器核的数量。
此外，应用的相关信息还可以包括其他内容，例如，应用的相关信息还可以包括应用架构的信息。应用架构的信息可以用于指示应用中的功能模块的耦合方式。再如，应用的相关信息还可以包括用户的信息。
以CESM应用为例，CESM的相关信息可以包括CESM中启用的功能模块的数量、可用的处理器核的数量以及CESM的模拟时间等。
步骤610对应于方法500中的步骤501。具体描述可以参考步骤501的相关描述，此处不再赘述。
步骤620，获取该应用的多个应用实例的运行性能数据。该多个应用实例的运行性能数据包括该多个功能模块的运行性能数据。
该多个应用实例的运行性能数据可以包括该多个应用实例在多种样本资源规模组合下的运行性能数据。或者说，该多个应用实例可以为多种样本资源规模组合下的应用实例。
样本资源规模组合可以包括该多个功能模块对应的样本资源规模的组合。
资源分配方案用于指示功能模块对应的资源，即为功能模块分配的资源。不同的资源规模组合下的资源分配方案不同。该多个应用实例的运行性能数据包括该多个应用实例在多种样本资源规模组合下的运行性能数据也可以理解按照如下方式理解。该多个应用实例的运行性能数据可以包括该多个应用实例在多种资源分配方案下的性能数据，该多种样本资源分配方案所指示的各个功能模块对应的资源规模的组合包括多种样本资源规模组合。
示例性地，样本资源分配方案可以用于指示各个功能模块的进程分配情况。
例如，为每个功能模块分配的进程可以是连续的。为每个功能模块分配的进程可以由为该功能模块分配的起始进程的ID和终止进程的ID表示。或者，为每个功能模块分配的进程可以由为该功能模块分配的起始进程的ID和为该功能模块分配的进程的数量表示。或者，为每个功能模块分配的进程还可以通过其他参数表示。
样本资源规模可以由资源参数表示，相应地，样本资源规模组合可以由资源参数组合表示。多种样本资源规模可以表示为多种资源参数组合。资源参数组合用于指示分配给各个功能模块的资源的参数。
以CESM为例，CESM启动的功能模块可以包括ATM和OCN。ATM和OCN的样本资源规模组合可以由用于指示分配给ATM和OCN的资源的参数的组合表示。例如，用于指示分配给ATM的资源的 参数可以为分配给ATM的进程的数量NATM，用于指示分配给OCN的资源的参数可以为分配给OCN的进程的数量NOCN。样本资源规模组合可以表示为资源参数组合(NATM，NOCN)。
示例性地，步骤620可以通过如下步骤实现。
621，确定多种样本资源规模组合。
示例性地，基于应用的相关信息获取该应用的默认资源规模组合。基于应用的默认资源规模组合确定该多种样本资源规模组合。样本资源规模可以由资源参数表示，相应地，样本资源规模组合可以由资源参数组合表示。
默认资源规模组合可以作为性能分析的基线与依据。
例如，从应用的默认资源规模组合的邻域中选取资源规模组合，和默认资源规模组合共同构成该多种样本资源规模组合。该过程可以称为参数扩增。
示例性地，该默认资源规模组合可以是用户指定的。例如，用户可以指示各个功能模块对应的默认资源规模，各个功能模块对应的默认资源规模的组合可以作为默认规模组合。
可替换地，该默认资源规模组合可以是预先设定的。
可替换地，该默认资源规模组合也可以是资源分配的装置基于其他相关方案确定的。
例如，资源分配的装置基于通用算法确定默认资源规模组合。
示例性地，默认资源规模组合的邻域可以包括与默认资源规模组合中的对应默认资源规模之间的差值小于或等于z的资源规模构成的组合。Z即为前文中的规模阈值。即与各个功能模块对应的默认资源规模之间的差值在z以内的资源规模构成的参数组合。z为正整数。比如，z＝3。
例如，资源规模组合可以为分配给各个功能模块的进程数量的组合。默认资源规模组合即为分配给各个功能模块的默认的进程数量的组合。以CESM为例，CESM启动的功能模块可以包括ATM和OCN，资源规模组合可以为分配给ATM和OCN的进程数量的组合。比如，默认资源规模组合可以表示为(5,5)，即默认分配给ATM的进程数量为5，默认分配给OCN的进程数量为5。该默认资源规模组合的邻域可以包括与默认分配给各个模块的进程数量相差3个进程的进程数量构成的组合。从默认资源规模组合的邻域选择的资源规模组合可以包括(4,7)和(6,3)。该多种样本资源规模组合可以包括(5,5)、(4,7)和(6,3)。
应理解，以上仅为示例，该多种样本资源规模组合还可以通过其他方式确定。例如，该多种样本资源规模组合可以是随机确定的。或者，该多种样本资源规模组合可以是用户确定的。
622，获取该多种样本资源规模组合下的多个应用实例的运行性能数据。
换言之，获取符合该多种样本资源规模组合中的样本资源规模的应用实例的运行性能数据。
以CESM为例，基于步骤621确定的多种样本资源规模组合可以包括(5,5)、(4,7)和(6,3)。在步骤622中可以获取符合(5,5)、(4,7)和(6,3)中样本资源规模的应用实例的运行性能数据。
例如，ATM被分配了5个进程的应用实例即可作为该多个应用实例中的一个应用实例。
示例性地，可以通过以下至少一种方式实现步骤622：
(1)从该应用的历史数据中获取运行性能数据。
在该应用的历史数据中，采集符合该多种样本资源规模组合的应用实例的运行性能数据。
换言之，在该应用的历史数据中，若一个应用实例所采用的资源规模组合属于该多种样本资源规模组合，则该应用实例的运行性能数据可以作为步骤620中的多个应用实例的运行性能数据。
(2)从正在执行的应用实例的监控数据中获取运行性能数据。
从正在执行的应用实例中，采集符合该多种样本资源规模组合的应用实例的运行性能数据。
换言之，对于正在执行的应用实例，若一个应用实例采用的资源规模组合属于该多种样本资源规模组合，则可以从该应用实例的监控数据中获取其运行性能数据，作为步骤620中的多个应用实例的运行性能数据。
(3)在多种样本资源规模组合下预执行该应用。从监控数据中获取该多种样本资源规模组合下的多个应用实例的运行性能数据。
基于一种样本资源规模组合为各个功能模块分配资源，在该资源下预执行一段时间，从监控数据中获取到的运行性能数据可以作为该样本资源规模组合下的应用实例的运行性能数据。
上述三种方式可以独立使用，或者，也可以结合使用。例如，该多种样本资源规模组合中的部分样本资源规模组合下的应用实例的运行性能数据是从历史数据中获取的，其余样本资源规模组合下的应用 实例的运行性能数据可以是通过在对应的样本资源规模组合下预执行该应用得到的。
应理解，上述三种方式仅为示例，不对本申请实施例的方案构成限定。
示例性地，应用实例的性能数据可以包括以下至少一项：运行时间、响应时延或资源利用率等。为了便于理解，方法600中主要以运行时间为例进行说明，不对本申请实施例的方案构成限定。运行时间可以包括应用的运行时间和应用中的各个功能模块的运行时间。
步骤620对应于方法500中的步骤510，具体描述可以参考步骤510的相关描述，此处不再赘述。
步骤630，基于该多个应用实例的性能数据进行性能分析，以得到各个功能模块在独占资源的情况下的运行性能。
不同的应用在执行过程中具有不同的行为特点。应用的行为特点也可以称为应用的性能特性。例如，给定相同的资源，不同的应用的运行性能是不同的。再如，给定的资源增加时，通常应用的运行性能会提高，但不同的应用的运行性能提高的程度是不同的。应用的行为特点可以由应用实例的性能数据和应用实例的功能逻辑确定。应用实例的功能逻辑即为应用实例中启用的功能模块。
在步骤630中可以基于该多个应用实例的性能数据进行性能建模，以得到应用的性能模型。或者说，在步骤630中可以基于该多个应用实例的性能数据和多个应用实例的功能逻辑进行性能建模，以得到应用的性能模型。应用的性能模型可以用于指示给定应用的资源与该资源下应用的运行性能之间的映射关系。即在步骤630中构建资源与应用的运行性能之间的关系，分析应用的行为特点。
这样，可以在不同的场景下为应用自适应地分配合适的算力资源，或者说，在不同的场景下为应用提供合适的资源分配方案。
示例性地，步骤630可以通过以下步骤实现。
步骤631，构建应用模型。应用模型用于指示应用的整体运行性能与应用的多个功能模块的运行性能之间的关系。
换言之，应用模型可以用于预测应用的整体运行性能。应用模型的输入可以为不同的资源分配方案下的各个功能模块的运行性能，应用模型的输出即为由应用模型预测得到的对应的资源分配方案下的应用的整体运行性能。应用模型预测得到的应用的整体运行性能和应用的实际整体运行性能之间的比较可以用于衡量应用模型的建模准确度。
应用的整体运行性能即为应用的多个功能模块在并行执行时的整体运行性能。
下面以运行性能为运行时间为例对应用模型的构建方法进行示例性说明。
可选地，应用模型可以满足如下公式：
其中，Ttot表示应用的运行时间，Ti表示功能模块i的运行时间。C表示功能模块的集合。ri表示功能模块i的起始进程的ID。ni表示功能模块i使用的进程的数量。ni为整数。单线程中功能模块i分配的资源可以包括ID属于[ri,ri+ni-1]的进程。N表示应用需要调用的总进程数，j表示进程的ID。
每个进程可能分配给一个或多个功能模块。示例性地，应用需要调用的总进程数可以满足如下公式：
f(r,n)表示耦合函数之间的通信代价函数，用于确定为给定资源分配方案下各个功能模块之间的通信代价。f(r,n)的函数值与资源分配方案方式相关，具体地，与各个功能模块和耦合模块的资源的相对位置强相关。r表示各个功能模块的起始进程构成的向量，n表示各个功能模块使用的进程数构成的向量。f(r,n)通常很小。ε表示用户提交应用后系统的资源协调与初始化时间。
H(x)为海维赛德阶跃函数(Heaviside step function)。H(x)可以满足如下公式：
应理解，以上仅为示例，应用模型还可以表示为其他的形式，本申请实施例对此不做限定。
步骤632，构建应用的多个功能模块的性能模型，确定该多个功能模块之间的干扰情况。
示例性地，功能模块的性能模块可以为功能模块的第一性能模型。每个功能模块的第一性能模型可以用于预测该功能模块在独占输入至该第一性能模型的资源规模的情况下的运行性能。
示例性地，功能模块的性能模块可以为动能模块的第二性能模型。每个功能模块的第二性能模型可 以用于预测该功能模块在输入至第二性能模型的资源分配方案下与其他功能模块并行执行时的运行性能。
示例性地，各个功能模块之间的干扰情况可以由各个功能模块的波动系数指示。不同的资源分配方案下，各个功能模块之间的干扰情况可能不同。相应地，不同的资源分配方案下，同一个功能模块的波动系数也可能不同。
各个功能模块的波动系数可以通过各个功能模块的波动系数模型确定。各个功能模块的波动系数模型可以用于指示资源分配方案与该功能模块的波动系数之间的关系。
在该情况下，应用的性能模型可以包括应用模型、各个模块的性能模型以及各个功能模块的波动系数模型。
下面以运行性能为运行时间为例对功能模块的性能模型进行示例性说明。
功能模块i的第二性能模型可以满足如下公式：
Ti＝γi(Ti,init+Ti,comp+Ti,comm)；
功能模块i的第一性能模型可以满足如下公式：
Ti′＝Ti,init+Ti,comp+Ti,comm；
其中，γi表示功能模块i在给定资源分配方案下的波动系数。Ti,init表示初始化时间。Ti,comp表示计算时间，Ti,comm表示通信时间。功能模块i在独占资源的情况下的运行时间Ti'可以包括Ti,init、Ti,comp和Ti,comm三部分。由于功能模块i可能与其他功能模块抢占资源，功能模块i的运行时间Ti需要考虑波动系数γi，即将该波动系数γi乘以在独占资源的情况下的运行时间以得到功能模块i的运行时间Ti。
Ti,init可以满足如下公式：
Ti,init＝αi,1ni+αi,2；
其中，αi,1和αi,2为系数。αi,1和αi,2表明初始化时间由与功能模块i使用的进程的数量相关的部分和无关的部分构成。αi,1＞0。αi,2＞0
Ti,comp可以满足如下公式：
其中，Wi表示功能模块i的总负载。根据阿姆达尔定律(Amdahl's Law)，计算任务可以分为可并行部分和不可并行部分。ai表示可并行部分所占的比例。p表示处理器性能，此处可以假设处理器性能一致。0≤ai≤1。Wi＞0。p＞0。
Ti,comm可以满足如下公式：
Ti,comm＝βilog ni；
其中，βi表示通信延迟。此处假设主要通过全局通信完成功能模块内部的信息传递，主要使用树规约方法，时间复杂度为logni。βi＞0。
γi可以由功能模块包含进程负载的调和平均数确定。γi＞1。γi可以满足如下公式：
上述公式可以作为功能模块i的波动系数的模型。
其中，Lj可以满足如下公式：
Lj表示第j个进程的负载。
基于该多个应用实例的性能数据可以对上述模型拟合模型的相关系数。例如，可以基于获取的各个功能模块的运行时间确定功能模块i的性能模型以及波动系数模型中的相关系数。
应理解，以上各个模型即为示例，上述模型还可以表示为其他形式，例如，神经网络模型等。
步骤633，基于应用的多个功能模块的性能模型预测该多个功能模块在独占资源时的运行性能。
换言之，基于应用的多个功能模块的性能模型预测该多个功能模块在独占该多个功能模块对应的候选资源规模的情况下的运行性能。
示例性地，步骤633可以包括基于应用的多个功能模块的第一性能模型预测该多个功能模块在独占资源时的运行性能。
示例性地，步骤634可以包括基于应用的多个功能模块的第二性能模型和多个功能模块之间的干扰情况预测该多个功能模块在独占资源时的运行性能。
示例性地，该多个功能模块在独占资源时的运行性能可以由该多个功能模块的性能矩阵表示。该性能矩阵用于表示在不同的功能模块在独占不同的资源时的运行性能。该性能矩阵的维度可以为|C|×N，其中，|C|表示该多个功能模块的数量。该性能矩阵的一行对应一个功能模块，该性能矩阵中的一列对应一个资源规模，例如，进程的数量。性能矩阵的每个元素即表示该元素所在行对应的功能模块在独占该元素所在的列对应的资源规模的情况下的运行性能。该多个功能模块对应的候选资源规模即为该性能矩阵中的资源规模。
预测该多个功能模块在独占该多个功能模块对应的候选资源规模的情况下的运行性能即为预测该多个模块的性能矩阵的元素。
下面对该性能矩阵中的元素的确定方式进行示例性说明。
如前所述，多个应用实例的运行性能数据包括多种样本资源规模组合下的多个应用实例的运行性能数据。
若该多个应用实例的运行性能数据中包括功能模块在独占资源时的运行性能数据，则该功能模块在独占该资源时的运行性能数据即可作为性能矩阵中该功能模块和该资源所指示的位置处的元素。该方式对应方法500中的示例3。
例如，该多个应用实例的运行性能数据包括功能模块i在独占ni个进程时的运行性能数据，则功能模块i在独占ni个进程时的运行性能数据可以作为性能矩阵中(i，ni)处的元素。在该情况下，该功能模块i可以作为第三任务的一例，ni个进程可以作为第三任务对应的第三候选资源规模的一例。
对于该多个应用实例的运行性能数据中的功能模块在非独占资源时的运行性能数据，可以通过上述功能模块的波动系数模型计算出该功能模块的波动系数，基于该波动系数进行性能反解，以预测该功能模块在独占资源的情况下的运行性能。预测得到的该功能模块在独占资源的情况下的运行性能即可作为性能矩阵中该功能模块和该资源所指示的位置处的元素。该方式对应方法500中的示例2。
例如，该多个应用实例的运行性能数据包括功能模块i在非独占ni个进程时的运行性能数据。功能模块i对应的样本资源由样本资源分配方案指示，在该样本资源分配方案中，该ni个进程中至少有一个进程被同时分配给其他功能模块。基于该样本资源分配方案计算该功能模块的波动系数，对功能模块i在非独占ni个进程时的运行性能数据进行性能反解，得到的结果可以作为性能矩阵中(i，ni)处的元素。在该情况下，该功能模块i可以作为第二任务的一例，ni个进程可以作为第二任务对应的第二候选资源规模的一例。
对于性能矩阵中的除了该多个样本资源规模组合对应的位置以外的其他位置处的元素，可以采用各个功能模块的性能模型预测各个功能模块在独占资源时的运行性能。该方式对应方法500中的示例1。
由于各个功能模块的性能模型是基于多个样本资源规模组合下的多个应用实例的性能数据拟合得到的，利用各个功能模块的性能模型预测各个功能模块在该多个样本资源规模组合周围的参数下的运行性能的准确度更高。因此，可以仅基于各个模块的性能模型预测各个功能模块在该多个样本资源规模组合周围的参数下的运行性能。
对于性能矩阵中其他未补全的部分，可以通过矩阵补全的方式进行修复填充。
不同的功能模块的运行时间的情况存在相似性，去除波动系数的影响后的性能矩阵本身是低秩的。
示例性地，可以通过求解以下最小化问题得到完整的性能矩阵。
minimize ||Z||*；
其中，Xab表示性能矩阵中观测到的位置(a,b)上的数值，Ω表示位置的集合。Z表示原始矩阵，||Z||*表示Z的核范数，Zab表示原始矩阵Z中位置(a,b)，δ表示一个小量。
应理解，以上确定性能矩阵中的元素的方式仅为示例，不对本申请实施例的方案构成限定。
步骤640，基于各个功能模块在独占资源的情况下的运行性能确定目标资源分配方案。
该目标资源分配方案用于指示各个功能模块对应的目标资源。或者说，该目标资源分配方案用于指示为各个功能模块分配的目标资源。
示例性地，步骤640可以通过如下步骤实现。
641，确定可行的资源规模组合。
示例性地，可行的资源规模组合可以通过枚举性能矩阵中的所有的候选资源规模的组合确定。
该方式的枚举空间过大，在后续处理过程中难以高效地从中确定目标资源分配方案。
本申请实施例提供了一种基于搜索剪枝的方案，该方案能够较快速地确定可行的资源规模组合。具体地，该方案通过剪枝处理减小了后续确定目标资源分配过程中的搜索空间，提高搜索效率，从而快速确定目标资源分配方案。
对于性能矩阵中的每个功能模块(即每一行元素)全局搜索执行剪枝操作。遍历所有功能模块，完成剪枝操作。
示例性地，基于性能矩阵中的元素的资源模块效率执行剪枝操作。当元素的资源模块效率低于效率阈值时，对该元素进行剪枝。剪枝操作后的性能矩阵中的元素所在行对应的功能模块即为方法500中的第六任务，该元素所在列对应的资源规模即为第六任务对应的目标候选资源规模。
当元素的资源模块效率低于效率阈值时可以认为该模块在当前的资源下的资源利用效率较低，导致资源浪费，可以去除该元素。
每个元素的资源模块效率，即功能模块的运行性能和资源的规模之间的比值。
进一步地，基于性能矩阵中的元素对应的资源规模执行剪枝操作。当元素对应的资源规模较小时，任务的并行效率较低，影响该多个任务的整体运行性能，可以去除该元素。
应理解，以上剪枝方式仅为示例，其他方案可以参考方法500，此处不再赘述。
下面结合图7对性能矩阵的处理流程进行示例性说明。
通过采样的方式获取多种样本资源规模组合下的应用实例的性能数据。计算各个功能模块的波动系数，进行性能反解，以得到各个功能模块在独占对应资源的情况下的运行性能，作为性能矩阵上的采样点。然后通过应用内部特性拟合，即通过各个功能模块的性能模型预测采样点周围的运行性能，以得到性能矩阵上的拟合点。预测采样点周围的运行性能，也可以称为，在采样点周围内插或外拓。如图7所示，仅在采样点周围内插或外拓，这样有利于保证预测的运行性能的准确性。对当前的性能矩阵中缺失的元素通过矩阵补全的方式完成修复，即得到性能矩阵上的补全点。然后对性能矩阵中的剪枝点进行剪枝处理，筛选出各个功能模块对应的目标候选资源规模。例如，剪枝点可以是基于边际效应确定的，即基于效率阈值确定的。各个功能模块对应的目标候选资源规模的组合可以作为可行的资源规模组合。
642，基于可行的资源规模组合确定目标资源分配方案。
示例性地，步骤642可以包括：枚举可行的资源规模组合下的所有资源分配方案，并从中确定目标资源分配方案。
该方式的复杂度较高，所需的运行时间较长，难以高效地从中确定目标资源分配方案。
本申请实施例提供了一种利用装箱的算法的分配方式，能够实现资源的高效分配。
根据剪枝后的性能矩阵遍历可行的资源规模组合，对于其中的每一种资源规模组合，执行装箱算法，以最小化装箱的总体高度。由此可以得到各个功能模块对应的资源的位置，从而确定该资源规模组合下的最优资源分配方案。通过各个功能模块的波动系数还原得到各个功能模块非独占资源时的运行时间，以确定应用的整体运行时间。
示例性地，对于每一种资源规模组合，执行快速二维初配递减高度(two-dimension first-fit decreasing  height，2D-FFDH)装箱算法改进，正向装箱一次再反向装箱，以两层为一组，复杂度为nlogn，n表示箱子的数量，即本申请实施例中的功能模块的数量，以最小化装箱的总体高度。
下面以可行的资源规模组合中的一种资源规模组合为例进行示例性说明。
例如，如图8所示，应用可以包括5个功能模块。可行的资源规模组合即为剪枝后的性能矩阵中该5个功能模块对应的目标候选资源规模的组合。
基于性能矩阵中的对应元素将每个功能模块抽象为矩阵。矩阵的宽度对应于该资源规模组合中分配给该功能模块的资源规模，矩阵的长度对应于该功能模块在独占该资源规模的情况下的运行时间。例如，将功能模块i抽象为矩阵，该矩阵的宽度对应于该资源规模组合中分配给该功能模块的进程数量ni，矩阵的长度对应于性能矩阵中(i，ni)处的值。
如图8所示，通过装箱算法将所有功能模块对应的矩阵装入资源池，以最小化总体高度。例如，资源池的宽度可以为可用的资源规模。
基于2D-FFDH装箱算法确定目标资源分配方案，可以包括如下步骤：
对于每个资源规模组合，分别执行步骤S1至步骤S4。
S1，将各个功能模块按照运行时间从高到低的顺序排列。基于该顺序放置各个功能模块。
例如，图8中的5个功能模块按照运行时间从高到低的顺序排列，得到的顺序可以为：功能模块#3，功能模块#1，功能模块#2，功能模块#4，功能模块#5。
S2，以两层为一组，从下层开始按照从左到右的顺序放置功能模块。若资源池中下层右侧的剩余空间大于或等于待放置的功能模块对应的资源的数量，则该功能模块放置在下层。计算当前下层右侧的剩余空间。若资源池中下层右侧的剩余空间小于待放置的功能模块对应的资源的数量，则该功能模块放置在上层，上层按照从右到左的顺序放置。计算当前上层左侧的剩余空间。待放置的功能模块是根据步骤S1中的各个功能模块的放置顺序确定的。剩余空间大于或等于功能模块对应的资源的数量，也即剩余空间的宽度大于或等于该功能模块对应的矩阵的宽度。剩余空间小于功能模块对应的资源的数量，也即剩余空间的宽度小于功能模块对应的矩阵的宽度。
例如，如图8所示，在下层开始先放置功能模块#3，并计算下层左侧的剩余空间。功能模块#3之后的待放置的功能模块为功能模块#1，下层右侧的剩余空间大于功能模块#1对应的资源的数量，该功能模块#1放置在下层，并计算放置功能模块#1之后下层右侧的剩余空间。功能模块#1之后的待放置的功能模块为功能模块#2。下层右侧的剩余空间小于功能模块#2对应的资源的数量，该功能模块#2放置在上层的最右侧，计算放置功能模块#2之后上层左侧的剩余空间，以此类推。
S3，若下层的剩余空间和上层的剩余空间均小于待放置的功能模块对应的资源，则再启用两层放置功能模块。重复步骤S2，直至所有功能模块完成放置，得到该资源规模组合下的目标候选资源分配方案。
S4，根据模块耦合，计算该5个功能模块的波动系数。基于各个功能模块的波动系数得到各个功能模块在非独占资源时的运行时间。基于各个功能模块在非独占资源时的运行时间确定应用的预测运行时间。
换言之，根据该资源规模组合下的目标候选资源分配方案，计算该5个功能模块的波动系数。
示例性地，在目标候选资源分配方案中，每个功能模块对应的资源可以由该功能模块对应的资源的起始位置以及该功能模块对应的资源规模指示。
示例性地，可以基于步骤631中的应用模型以及各个功能模块在非独占资源时的运行时间确定应用的预测运行时间。
S5，遍历各个资源规模组合下的应用的预测运行时间，将最小的预测运行时间对应的资源分配方案作为目标资源分配方案。
应理解，以上装箱算法仅为示例，本申请实施例对此不做限定。例如，还可以根据算法的时间要求、输出结果的精度要求等选择其他算法确定目标资源分配方案。
步骤640对应于方法500中的步骤530。具体描述可以参考步骤530的相关描述。
基于该目标资源分配方案可以将各个功能模块对应的任务分发至处理器，由处理器执行计算。
本申请实施例的方案中，进程可以与处理器核绑定。这样，可以在全局范围内搜索最优的资源分配方案，避免了算力节点内部不具备全局信息而导致无法得到最优的资源分配方案，同时本申请实施例的方案，将进程与处理器核绑定，实现了精细化的算力分配，无需算力节点内核的二次调度，即无需由算力节点为进程调度处理器核，避免了二次调度所带来的开销。
在本申请实施例的方案中，通过识别应用的行为特点，可以在全局范围内为各个功能模块计算资源，得到细粒度的资源分配方案，有利于实现应用与算力资源的最佳匹配，从而提高总体端到端的运行性能。随着应用中的并行执行的功能模块的数量的增加，本申请实施例的方案对整体运行性能的提升更加明显。
下面以CESM应用为例，对方法600进行示例性说明。
例如，CESM启用的功能模块可以包括ATM、LND、ICE、ROF、GLC、WAV、OCN以及耦合组件CPL。通过本申请实施例的方案可以为上述功能模块分配对应的进程资源。同一进程资源允许被分配给多个功能模块。
具体地，CESM的资源分配的过程可以包括如下步骤。
1)，获取CESM的相关信息。CESM的相关信息包括多个功能模块的信息和可用的资源规模的信息。
该多个功能模块的信息用于指示ATM、LND、ICE、ROF、GLC、WAV、OCN以及耦合组件CPL。
可用的资源规模的信息用于指示HPC的可用的资源。
2)，获取CESM的程序运行指令，在多种样本资源分配方案下执行CESM的短时间的预执行，以得到多个应用实例的运行性能数据。该多种样本资源分配方案指示的各个功能模块对应的样本资源规模的组合包括多种。
3)，根据该多种样本资源分配方案，分别计算各个功能模块的波动系数。
4)，基于各个功能模块的波动系数对多个应用实例的运行性能数据进行性能反解，以得到各个功能模块在独占资源时的运行时间，将其作为性能矩阵中的采样点。
5)，通过拟合插值的方式以及低秩补全的方式得到完整的性能矩阵。
6)，对该性能矩阵进行剪枝处理，以得到可行的资源规模组合。
7)，遍历可行的资源规模组合，对于每种可行的资源规模组合，通过2D-FFDH快速装箱算法确定目标候选资源分配方案；基于目标候选资源分配方案计算各个功能模块的波动系数；通过各个功能模块的波动系数计算各个功能模块在目标候选资源分配方案下并行执行时的运行时间；基于并行执行时的运行时间预测应用的整体运行时间。将遍历后得到的最优的整体运行时间对应的目标候选资源分配方案作为目标资源分配方案，例如，目标资源分配方案可以如图9所示。
8)，基于该目标资源分配方案为应用中的各个功能模块分配资源，实现该多个功能模块的并行执行。
表1示出了CESM在测试环境中的性能测试结果。如表1所示，分别基于CESM的数据集B1850G和数据集BW1850进行了性能测试。场景的分辨率(resolution，res)为f09_g17，每个进程的线程数NTHRD为1，实验模拟时间为5天。
测试环境所采用的硬件结构为非一致存储访问(non-uniform memory access，NUMA)结构节点(node)，包括：两个CPU插槽(socket),52核(52cores),768GB的随机存取存储器(random access memory，RAM)。
表1
如表1所示，本申请实施例的方案相对于相关方案的资源布局方式，性能提升30％以上。
一个或多个用户可以提交多个作业任务到计算系统中，由系统统一管理、运行这些任务。该计算系统可以称为批作业处理系统或批处理系统。本申请实施例的方案可以应用于批处理系统的资源分配场景中，为该多个作业任务分配算力资源。
图10示出了一个多个作业任务混合布局的批处理系统中的资源分配方案。如图10所示，该多个作业任务包括数据处理任务、解码任务、加密任务、文件整理任务和两个压缩任务。该多个作业任务可以作为方法500中的多个任务。采用本申请实施例的方案能够为该多个作业任务分配算力资源，在提高系统资源利用率的同时，优化该多个作业任务的整体运行性能，例如，该多个任务的整体运行时间。
下面将结合图11-图14描述本申请装置的实施例。应理解，方法实施例的描述与装置实施例的描述相互对应，具体描述可以参考前文中的相关描述，为了避免不必要的重复，下面在介绍装置的实施例时 适当省略重复的描述。
图11是本申请实施例的资源分配的装置的示意性框图。图11所示的装置1100可以用于执行本申请实施例的资源分配的方法，例如，方法500或方法600。
如图11所示，该装置1100包括获取单元1110和处理单元1120。
在一种可能的实现方式中，获取单元，用于获取多个任务中的每个的运行性能数据，多个任务中的每个任务的运行性能数据包括每个任务在每个任务对应的样本资源规模下的运行性能；处理单元，用于：根据多个任务中的每个任务的运行性能数据确定多个任务中的每个任务在独占每个任务对应的候选资源规模的情况下的运行性能；根据多个任务中的每个任务在独占每个任务对应的候选资源规模的情况下的运行性能多个任务中的每个任务对应的目标资源。
可选地，每个任务对应的目标资源包括每个任务对应的一个或多个进程，一个或多个进程分别与一个或多个slot绑定。
可选地，多个进程为多个连续的进程，多个slot为多个连续的slot。
可选地，多个任务包括第一任务，处理单元具体用于：根据第一任务的运行性能数据构建第一任务的第一性能模型，其中，第一任务的第一性能模型用于预测第一任务在独占输入至第一任务的性能模型的资源规模的情况下的运行性能；根据第一任务的第一性能模型确定第一任务在独占第一任务对应的第一候选资源规模的情况下的运行性能，第一任务对应的候选资源规模包括第一任务对应的第一候选资源规模。
可选地，每个任务对应的候选资源规模为每个任务对应的所有候选资源规模中的部分资源规模，每个任务对应的候选资源规模中的部分资源规模与每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能满足第一预设条件。
可选地，第一预设条件包括：每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能与每个任务对应的候选资源规模中的部分资源规模之间的比值大于或等于第一阈值，第一阈值为任务对资源规模的利用效率的阈值。
可选地，处理单元具体用于：根据多个任务中的每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案，多种资源规模组合基于每个任务对应的候选资源规模中的部分资源规模的组合确定，多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案指示每种资源规模组合下多个任务中的每个任务对应的目标候选资源；确定多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数，每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数分别用于指示多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能和多个任务在独占每种资源规模组合中的每个任务对应的资源规模的情况下的运行性能之间的差异；根据每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数预测多个任务在每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能；根据多个任务在多种资源规模组合下的目标候选资源分配方案下并行执行的运行性能从多种资源规模组合下的目标候选资源分配方案中确定多个任务中的每个任务对应的目标资源。
可选地，处理单元具体用于：根据多个任务中的每个任务在独占每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能从每种资源规模组合下的候选资源分配方案中确定每种资源规模组合下的目标候选资源分配方案，每组资源规模组合下的候选资源分配方案所指示的每个任务对应的候选资源连续，多个任务在每种资源规模组合下的目标候选资源分配方案下独占每种资源规模组合中的每个任务对应的资源规模的情况下的整体运行性能优于多个任务在每种资源规模组合下的其他候选资源分配方案下独占每种资源规模组合中的每个任务对应的资源规模的情况下的整体运行性能。
可选地，处理单元具体用于：根据多个任务的运行性能数据构建多个任务的波动系数模型，多个任务的波动系数模型分别用于预测输入至多个任务的波动系数模型的资源分配方案对应的多个任务的波动系数；根据多个任务的波动系数模型分别确定多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数。
可选地，多个任务包括多个应用对应的任务，多个应用中的每个应用对应一个任务。
可选地，多个任务包括一个应用中的多个功能模块对应的任务，多个功能模块中的每个功能模块对应一个任务。
这里的术语“单元”可以通过软件和/或硬件形式实现，对此不作具体限定。
例如，“单元”可以是实现上述功能的软件程序、硬件电路或二者结合。示例性的，接下来以处理单元为例，介绍处理单元的实现方式。类似的，获取单元以及输出单元的实现方式可以参考处理单元的实现方式。
处理单元作为软件功能单元的一种举例，处理单元可以包括运行在计算实例上的代码。其中，计算实例可以包括物理主机(计算设备)、虚拟机、容器中的至少一种。进一步地，上述计算实例可以是一台或者多台。例如，处理单元可以包括运行在多个主机/虚拟机/容器上的代码。需要说明的是，用于运行该代码的多个主机/虚拟机/容器可以分布在相同的区域(region)中，也可以分布在不同的region中。进一步地，用于运行该代码的多个主机/虚拟机/容器可以分布在相同的可用区(availability zone，AZ)中，也可以分布在不同的AZ中，每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中，通常一个region可以包括多个AZ。
同样，用于运行该代码的多个主机/虚拟机/容器可以分布在同一个虚拟私有云(virtual private cloud，VPC)中，也可以分布在多个VPC中。其中，通常一个VPC设置在一个region内，同一region内两个VPC之间，以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关，经通信网关实现VPC之间的互连。
处理单元作为硬件功能单元的一种举例，处理单元可以包括至少一个计算设备，如服务器等。或者，处理单元也可以是利用专用集成电路(application-specific integrated circuit，ASIC)实现、或可编程逻辑器件(programmable logic device，PLD)实现的设备等。其中，上述PLD可以是复杂程序逻辑器件(complex programmable logical device，CPLD)、现场可编程门阵列(field-programmable gate array，FPGA)、通用阵列逻辑(generic array logic，GAL)或其任意组合实现。
处理单元包括的多个计算设备可以分布在相同的region中，也可以分布在不同的region中。处理单元包括的多个计算设备可以分布在相同的AZ中，也可以分布在不同的AZ中。同样，处理单元包括的多个计算设备可以分布在同一个VPC中，也可以分布在多个VPC中。其中，所述多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA和GAL等计算设备的任意组合。
因此，在本申请的实施例中描述的各示例的模块，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
需要说明的是，在其他实施例中，处理单元可以用于执行资源分配的方法中的任意步骤，获取单元可以用于执行资源分配的方法中的任意步骤,输出单元可以用于执行资源分配的方法中的任意步骤，获取单元、处理单元和输出单元负责实现的步骤可根据需要指定，通过获取单元、处理单元和输出单元分别实现资源分配的方法中不同的步骤来实现装置1100的全部功能。
本申请还提供一种计算设备1000。如图12所示，计算设备1000包括：总线1002、处理器1004、存储器1006和通信接口1008。处理器1004、存储器1006和通信接口1008之间通过总线1002通信。计算设备1000可以是服务器或终端设备。应理解，本申请不限定计算设备1000中的处理器、存储器的个数。
总线1002可以是外设部件互连标准(peripheral component interconnect，PCI)总线或扩展工业标准结构(extended industry standard architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图12中仅用一条线表示，但并不表示仅有一根总线或一种类型的总线。总线1004可包括在计算设备1000各个部件(例如，存储器1006、处理器1004、通信接口1008)之间传送信息的通路。
处理器1004可以包括中央处理器(central processing unit，CPU)、图形处理器(graphics processing unit，GPU)、微处理器(micro processor，MP)或者数字信号处理器(digital signal processor，DSP)等处理器中的任意一种或多种。
存储器1006可以包括易失性存储器(volatile memory)，例如随机存取存储器(random access memory，RAM)。处理器1004还可以包括非易失性存储器(non-volatile memory)，例如只读存储器(read-only memory，ROM)，快闪存储器，机械硬盘(hard disk drive，HDD)或固态硬盘(solid state drive，SSD)。
存储器1006中存储有可执行的程序代码，处理器1004执行该可执行的程序代码以分别实现前述获取单元和单元模块的功能，从而实现资源分配的方法。也即，存储器1006上存有用于执行资源分配的方法的指令。
通信接口1003使用例如但不限于网络接口卡、收发器一类的收发模块，来实现计算设备1000与其他设备或通信网络之间的通信。
本申请实施例还提供了一种计算设备集群。该计算设备集群包括至少一台计算设备。该计算设备可以是服务器，例如是中心服务器、边缘服务器，或者是本地数据中心中的本地服务器。在一些实施例中，计算设备也可以是台式机、笔记本电脑或者智能手机等终端设备。
如图13所示，所述计算设备集群包括至少一个计算设备1000。计算设备集群中的一个或多个计算设备1000中的存储器1006中可以存有相同的用于执行资源分配的方法的指令。
在一些可能的实现方式中，该计算设备集群中的一个或多个计算设备1000的存储器1006中也可以分别存有用于执行资源分配的方法的部分指令。换言之，一个或多个计算设备1000的组合可以共同执行用于执行资源分配的方法的指令。
需要说明的是，计算设备集群中的不同的计算设备1000中的存储器1006可以存储不同的指令，分别用于执行资源分配的装置的部分功能。例如，不同的计算设备1000中的存储器1006存储的指令可以实现获取单元和处理单元中的一个或多个单元的功能。
在一些可能的实现方式中，计算设备集群中的一个或多个计算设备可以通过网络连接。其中，所述网络可以是广域网或局域网等等。图14示出了一种可能的实现方式。如图14所示，两个计算设备1000A和1000B之间通过网络进行连接。具体地，通过各个计算设备中的通信接口与所述网络进行连接。在这一类可能的实现方式中，计算设备1000A中的存储器1006中存有执行获取单元的功能的指令。同时，计算设备1000B中的存储器1006中存有执行处理单元的功能的指令。
应理解，图14中示出的计算设备1000A的功能也可以由多个计算设备1000完成。同样，计算设备1000B的功能也可以由多个计算设备1000完成。
本申请实施例还提供了一种包含指令的计算机程序产品。所述计算机程序产品可以是包含指令的，能够运行在计算设备上或被储存在任何可用介质中的软件或程序产品。当所述计算机程序产品在至少一个计算设备上运行时，使得至少一个计算设备执行本申请实施例中的方法。
本申请实施例还提供了一种计算机可读存储介质。所述计算机可读存储介质可以是计算设备能够存储的任何可用介质或者是包含一个或多个可用介质的数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘)等。该计算机可读存储介质包括指令，所述指令指示计算设备执行本申请实施例中的方法，或指示计算设备执行本申请实施例中的方法。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者 该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (26)

  1. 一种资源分配的方法，其特征在于，包括：
    获取多个任务中的每个任务的运行性能数据，所述多个任务中的每个任务的运行性能数据包括所述每个任务在所述每个任务对应的样本资源规模下的运行性能；
    根据所述多个任务中的每个任务的运行性能数据确定所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模的情况下的运行性能；
    根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模的情况下的运行性能确定所述多个任务中的每个任务对应的目标资源。
  2. 根据权利要求1所述的方法，其特征在于，所述每个任务对应的目标资源包括所述每个任务对应的一个或多个进程，所述一个或多个进程分别与一个或多个资源槽绑定。
  3. 根据权利要求2所述的方法，其特征在于，所述多个进程为多个连续的进程，所述多个资源槽为多个连续的资源槽。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，所述多个任务包括第一任务，根据所述第一任务的运行性能数据确定所述第一任务在独占所述第一任务对应的候选资源规模的情况下的运行性能，包括：
    根据所述第一任务的运行性能数据构建所述第一任务的第一性能模型，其中，所述第一任务的第一性能模型用于预测所述第一任务在独占输入至所述第一任务的性能模型的资源规模的情况下的运行性能；
    根据所述第一任务的第一性能模型确定所述第一任务在独占所述第一任务对应的第一候选资源规模的情况下的运行性能，所述第一任务对应的候选资源规模包括所述第一任务对应的第一候选资源规模。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，所述每个任务对应的候选资源规模为所述每个任务对应的所有候选资源规模中的部分资源规模，所述每个任务对应的候选资源规模中的部分资源规模与所述每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能满足第一预设条件。
  6. 根据权利要求5所述的方法，其特征在于，所述第一预设条件包括：所述每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能与所述每个任务对应的候选资源规模中的部分资源规模之间的比值大于或等于第一阈值，所述第一阈值为任务对资源规模的利用效率的阈值。
  7. 根据权利要求5或6所述的方法，其特征在于，所述根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模的情况下的运行性能确定所述多个任务中的每个任务对应的目标资源，包括：
    根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案，所述多种资源规模组合基于所述每个任务对应的候选资源规模中的部分资源规模的组合确定，所述多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案指示所述每种资源规模组合下所述多个任务中的每个任务对应的目标候选资源；
    确定所述多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数，所述每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数分别用于指示所述多个任务在所述每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能和所述多个任务在独占所述每种资源规模组合中的所述每个任务对应的资源规模的情况下的运行性能之间的差异；
    根据所述每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数预测所述多个任务在所述每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能；
    根据所述多个任务在所述多种资源规模组合下的目标候选资源分配方案下并行执行的运行性能从所述多种资源规模组合下的目标候选资源分配方案中确定所述多个任务中的每个任务对应的目标资源。
  8. 根据权利要求7所述的方法，其特征在于，所述根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案，包括：
    根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况 下的运行性能从所述每种资源规模组合下的候选资源分配方案中确定所述每种资源规模组合下的目标候选资源分配方案，所述每组资源规模组合下的候选资源分配方案所指示的每个任务对应的候选资源连续，所述多个任务在所述每种资源规模组合下的目标候选资源分配方案下独占所述每种资源规模组合中的所述每个任务对应的资源规模的情况下的整体运行性能优于所述多个任务在所述每种资源规模组合下的其他候选资源分配方案下独占所述每种资源规模组合中的所述每个任务对应的资源规模的情况下的整体运行性能。
  9. 根据权利要求7或8所述的方法，其特征在于，所述确定所述多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数，包括：
    根据所述多个任务的运行性能数据构建所述多个任务的波动系数模型，所述多个任务的波动系数模型分别用于预测输入至所述多个任务的波动系数模型的资源分配方案对应的多个任务的波动系数；
    根据所述多个任务的波动系数模型分别确定所述多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数。
  10. 根据权利要求1至9中任一项所述的方法，其特征在于，所述多个任务包括多个应用对应的任务，所述多个应用中的每个应用对应一个任务。
  11. 根据权利要求1至9中任一项所述的方法，其特征在于，所述多个任务包括一个应用中的多个功能模块对应的任务，所述多个功能模块中的每个功能模块对应一个任务。
  12. 一种资源分配的装置，其特征在于，包括：
    获取单元，用于获取多个任务中的每个任务的运行性能数据，所述多个任务中的每个任务的运行性能数据包括所述每个任务在所述每个任务对应的样本资源规模下的运行性能；
    处理单元，用于：
    根据所述多个任务中的每个任务的运行性能数据确定所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模的情况下的运行性能；
    根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模的情况下的运行性能确定所述多个任务中的每个任务对应的目标资源。
  13. 根据权利要求12所述的装置，其特征在于，所述每个任务对应的目标资源包括所述每个任务对应的一个或多个进程，所述一个或多个进程分别与一个或多个资源槽绑定。
  14. 根据权利要求13所述的装置，其特征在于，所述多个进程为多个连续的进程，所述多个资源槽为多个连续的资源槽。
  15. 根据权利要求12至14中任一项所述的装置，其特征在于，所述多个任务包括第一任务，所述处理单元具体用于：
    根据所述第一任务的运行性能数据构建所述第一任务的第一性能模型，其中，所述第一任务的第一性能模型用于预测所述第一任务在独占输入至所述第一任务的性能模型的资源规模的情况下的运行性能；
    根据所述第一任务的第一性能模型确定所述第一任务在独占所述第一任务对应的第一候选资源规模的情况下的运行性能，所述第一任务对应的候选资源规模包括所述第一任务对应的第一候选资源规模。
  16. 根据权利要求12至15中任一项所述的装置，其特征在于，所述每个任务对应的候选资源规模为所述每个任务对应的所有候选资源规模中的部分资源规模，所述每个任务对应的候选资源规模中的部分资源规模与所述每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能满足第一预设条件。
  17. 根据权利要求16所述的装置，其特征在于，所述第一预设条件包括：所述每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能与所述每个任务对应的候选资源规模中的部分资源规模之间的比值大于或等于第一阈值，所述第一阈值为任务对资源规模的利用效率的阈值。
  18. 根据权利要求16或17所述的装置，其特征在于，所述处理单元具体用于：
    根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能确定多种资源规模组合下的目标候选资源分配方案，所述多种资源规模组合基于所述每个任务对应的候选资源规模中的部分资源规模的组合确定，所述多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案指示所述每种资源规模组合下所述多个任务中的每个任务对应的目标候选资源；
    确定所述多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数，所述每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数分别用于指示所述多个任务在所述每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能和所述多个任务在独占所述每种资源规模组合中的所述每个任务对应的资源规模的情况下的运行性能之间的差异；
    根据所述每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数预测所述多个任务在所述每种资源规模组合下的目标候选资源分配方案下并行执行的运行性能；
    根据所述多个任务在所述多种资源规模组合下的目标候选资源分配方案下并行执行的运行性能从所述多种资源规模组合下的目标候选资源分配方案中确定所述多个任务中的每个任务对应的目标资源。
  19. 根据权利要求18所述的装置，其特征在于，所述处理单元具体用于：
    根据所述多个任务中的每个任务在独占所述每个任务对应的候选资源规模中的部分资源规模的情况下的运行性能从所述每种资源规模组合下的候选资源分配方案中确定所述每种资源规模组合下的目标候选资源分配方案，所述每组资源规模组合下的候选资源分配方案所指示的每个任务对应的候选资源连续，所述多个任务在所述每种资源规模组合下的目标候选资源分配方案下独占所述每种资源规模组合中的所述每个任务对应的资源规模的情况下的整体运行性能优于所述多个任务在所述每种资源规模组合下的其他候选资源分配方案下独占所述每种资源规模组合中的所述每个任务对应的资源规模的情况下的整体运行性能。
  20. 根据权利要求18或19所述的装置，其特征在于，所述处理单元具体用于：
    根据所述多个任务的运行性能数据构建所述多个任务的波动系数模型，所述多个任务的波动系数模型分别用于预测输入至所述多个任务的波动系数模型的资源分配方案对应的多个任务的波动系数；
    根据所述多个任务的波动系数模型分别确定所述多种资源规模组合中的每种资源规模组合下的目标候选资源分配方案对应的多个任务的波动系数。
  21. 根据权利要求12至20中任一项所述的装置，其特征在于，所述多个任务包括多个应用对应的任务，所述多个应用中的每个应用对应一个任务。
  22. 根据权利要求12至20中任一项所述的装置，其特征在于，所述多个任务包括一个应用中的多个功能模块对应的任务，所述多个功能模块中的每个功能模块对应一个任务。
  23. 一种资源分配的装置，其特征在于，包括处理器和存储器，所述处理器用于执行所述存储器中存储的指令，以使得所述装置执行如权利要求1至11中任一项所述的方法。
  24. 一种计算设备集群，其特征在于，包括至少一个计算设备，每个计算设备包括处理器和存储器；
    所述至少一个计算设备的处理器用于执行所述至少一个计算设备的存储器中存储的指令，以使得所述计算设备集群执行如权利要求1至11中任一项所述的方法。
  25. 一种计算机可读存储介质，其特征在于，包括计算机程序指令，当所述计算机程序指令由计算设备集群执行时，所述计算设备集群执行如权利要求1至11中任一项所述的方法。
  26. 一种包含指令的计算机程序产品，其特征在于，当所述指令被计算设备集群运行时，使得所述计算设备集群执行如权利要求1至11中任一项所述的方法。
PCT/CN2023/133357 2022-12-16 2023-11-22 资源分配的方法及装置 Ceased WO2024125251A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP23902450.8A EP4625168A4 (en) 2022-12-16 2023-11-22 METHOD AND APPARATUS FOR RESOURCE ALLOCATION

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211624304.6A CN118210615A (zh) 2022-12-16 2022-12-16 资源分配的方法及装置
CN202211624304.6 2022-12-16

Publications (1)

Publication Number Publication Date
WO2024125251A1 true WO2024125251A1 (zh) 2024-06-20

Family

ID=91455292

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/133357 Ceased WO2024125251A1 (zh) 2022-12-16 2023-11-22 资源分配的方法及装置

Country Status (3)

Country Link
EP (1) EP4625168A4 (zh)
CN (1) CN118210615A (zh)
WO (1) WO2024125251A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119512835A (zh) * 2024-10-31 2025-02-25 浪潮云信息技术股份公司 云平台自适应适配国产ai加速卡的装置及方法
US20250274401A1 (en) * 2025-03-24 2025-08-28 Chengdu Qinchuan Iot Technology Co., Ltd. Methods and systems for dynamic computing resource allocation based on iiot data center

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119378944A (zh) * 2024-12-30 2025-01-28 山东省计算中心（国家超级计算济南中心） 使用不稳定清洁能源的零碳数据中心资源调度方法及系统
CN120723476A (zh) * 2025-08-28 2025-09-30 苏州元脑智能科技有限公司 计算资源分配方法、电子设备、存储介质及程序产品

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010160552A (ja) * 2009-01-06 2010-07-22 Nec Corp リソース配分装置、その方法及びプログラム
CN107045456A (zh) * 2016-02-05 2017-08-15 华为技术有限公司 一种资源分配方法及资源管理器
CN112328378A (zh) * 2020-11-05 2021-02-05 南京星环智能科技有限公司 任务调度方法、计算机设备及存储介质
CN113485838A (zh) * 2021-07-26 2021-10-08 北京沃东天骏信息技术有限公司 服务器分配方法及装置、电子设备和计算机可读存储介质
US20220012089A1 (en) * 2020-07-13 2022-01-13 Accenture Global Solutions Limited System for computational resource prediction and subsequent workload provisioning
CN114371926A (zh) * 2022-03-22 2022-04-19 清华大学 一种精细化资源分配方法、装置、电子设备及介质
CN115098257A (zh) * 2022-06-23 2022-09-23 中国电信股份有限公司 一种资源调度方法、装置、设备以及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3447642B1 (en) * 2017-08-24 2022-03-23 Tata Consultancy Services Limited System and method for predicting application performance for large data size on big data cluster

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010160552A (ja) * 2009-01-06 2010-07-22 Nec Corp リソース配分装置、その方法及びプログラム
CN107045456A (zh) * 2016-02-05 2017-08-15 华为技术有限公司 一种资源分配方法及资源管理器
US20220012089A1 (en) * 2020-07-13 2022-01-13 Accenture Global Solutions Limited System for computational resource prediction and subsequent workload provisioning
CN112328378A (zh) * 2020-11-05 2021-02-05 南京星环智能科技有限公司 任务调度方法、计算机设备及存储介质
CN113485838A (zh) * 2021-07-26 2021-10-08 北京沃东天骏信息技术有限公司 服务器分配方法及装置、电子设备和计算机可读存储介质
CN114371926A (zh) * 2022-03-22 2022-04-19 清华大学 一种精细化资源分配方法、装置、电子设备及介质
CN115098257A (zh) * 2022-06-23 2022-09-23 中国电信股份有限公司 一种资源调度方法、装置、设备以及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4625168A1

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119512835A (zh) * 2024-10-31 2025-02-25 浪潮云信息技术股份公司 云平台自适应适配国产ai加速卡的装置及方法
US20250274401A1 (en) * 2025-03-24 2025-08-28 Chengdu Qinchuan Iot Technology Co., Ltd. Methods and systems for dynamic computing resource allocation based on iiot data center

Also Published As

Publication number Publication date
EP4625168A4 (en) 2026-03-18
EP4625168A1 (en) 2025-10-01
CN118210615A (zh) 2024-06-18

Similar Documents

Publication Publication Date Title
WO2024125251A1 (zh) 资源分配的方法及装置
Guo et al. A survey of FPGA-based neural network accelerator
US8676874B2 (en) Data structure for tiling and packetizing a sparse matrix
Gautam et al. A survey on job scheduling algorithms in big data processing
CN110619595A (zh) 一种基于多fpga加速器互联的图计算优化方法
US20120210081A1 (en) Optimizing Output Vector Data Generation Using A Formatted Matrix Data Structure
WO2017124713A1 (zh) 一种数据模型的确定方法及装置
US20200073677A1 (en) Hybrid computing device selection analysis
WO2024245037A1 (zh) 虚拟计算资源的调度方法及控制面组件
CN110187969A (zh) 一种基于gpu的分布式大数据并行计算方法
US11194623B2 (en) Resource scheduling method and related apparatus
CN114443236A (zh) 一种任务处理方法、装置、系统、设备及介质
Clemente-Castelló et al. Performance model of mapreduce iterative applications for hybrid cloud bursting
Fujiki et al. Near-memory data transformation for efficient sparse matrix multi-vector multiplication
JP2012530976A (ja) 仮想化超並列プログラマブルハードウェアによる正規表現の検索
Cong et al. CPU-FPGA coscheduling for big data applications
CN110187970A (zh) 一种基于Hadoop MapReduce的分布式大数据并行计算方法
WO2024255174A1 (zh) 基于异构计算的参数处理方法和装置
CN110222410B (zh) 一种基于Hadoop MapReduce的电磁环境仿真方法
Shepovalov et al. FPGA and GPU-based acceleration of ML workloads on Amazon cloud-A case study using gradient boosted decision tree library
CN119046022A (zh) 一种分布式并行方案的确定方法、装置、设备及介质
CN106155822A (zh) 一种处理能力评估方法及装置
Kim et al. A parallel migration scheme for fast virtual machine relocation on a cloud cluster
CN107493205B (zh) 一种设备集群扩容性能预测方法及装置
Dhar et al. Leveraging dynamic partial reconfiguration with scalable ILP based task scheduling

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23902450

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023902450

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023902450

Country of ref document: EP

Effective date: 20250625

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 2023902450

Country of ref document: EP