WO2023217146A1 - 一种算法模型运行处理方法、装置及边缘智能设备 - Google Patents

一种算法模型运行处理方法、装置及边缘智能设备 Download PDF

Info

Publication number
WO2023217146A1
WO2023217146A1 PCT/CN2023/093032 CN2023093032W WO2023217146A1 WO 2023217146 A1 WO2023217146 A1 WO 2023217146A1 CN 2023093032 W CN2023093032 W CN 2023093032W WO 2023217146 A1 WO2023217146 A1 WO 2023217146A1
Authority
WO
WIPO (PCT)
Prior art keywords
engine
virtual
algorithm model
algorithm
task
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/093032
Other languages
English (en)
French (fr)
Other versions
WO2023217146A9 (zh
Inventor
巩赫赫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to EP23802915.1A priority Critical patent/EP4524736A4/en
Publication of WO2023217146A1 publication Critical patent/WO2023217146A1/zh
Publication of WO2023217146A9 publication Critical patent/WO2023217146A9/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/34—Graphical or visual programming
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading

Definitions

  • This application relates to the field of intelligent analysis, and specifically relates to an algorithm model operation processing method, device and edge intelligent equipment.
  • edge smart devices are devices used for intelligent analysis of videos/pictures. Each edge smart device can only support one algorithm.
  • Model For example, edge smart devices have a physics engine that runs an algorithm model to realize the functions supported by the algorithm model. Or, in order to support multiple algorithm models, it is necessary to increase the number of physics engines in the edge intelligent device. Each physics engine can only support one algorithm model.
  • edge smart devices have multiple physics engines, and an algorithm model is run on one physics engine to realize the functions supported by the algorithm model.
  • an algorithm model is run on one physics engine to realize the functions supported by the algorithm model.
  • some simultaneous analysis scenarios that require support from multiple algorithm models cannot be implemented on edge smart devices.
  • the cost is high and it is not suitable to deploy edge intelligent devices with multiple physical engines.
  • Embodiments of the present application provide an algorithm model operation processing method, device and edge intelligent device, which can implement simultaneous analysis scenarios that require the support of multiple algorithm models on the edge intelligent device.
  • an algorithm model operation processing method including: acquiring at least one algorithm model, wherein the algorithm model is an artificial intelligence algorithm model; acquiring multiple virtual engines, wherein the virtual engines are For a physical engine that is virtualized after software encapsulation, the number of virtual engines obtained is greater than or equal to the number of algorithm models obtained; each algorithm model in the at least one algorithm model is separately bound to run in a virtual engine on, wherein a virtual engine is used to bind an algorithm model; receive tasks, and execute the tasks based on the algorithm model on the virtual engine.
  • performing the task on the virtual engine based on the algorithm model includes: obtaining a pre-configured predetermined order, wherein the predetermined order is used to indicate that multiple algorithm models cooperate to execute the task.
  • the running order of the multiple algorithm models based on the predetermined order, run the multiple algorithm models sequentially on multiple virtual engines corresponding to the multiple algorithm models to perform the task; Wherein, in the process of running the plurality of algorithm models to perform the task, the output of the algorithm model run earlier is used as the input of the algorithm model run later.
  • the method further includes: when executing tasks on the multiple virtual engines, limiting the performance of each virtual engine in the multiple virtual engines according to the performance of the physical engine, wherein the physical engine The performance of the engine is used to indicate the upper limit of the processing capability of the physical engine, and the performance of the virtual engine is used to indicate the upper limit of the processing capability of the virtual engine, where the multiple virtual engines are running The sum of the performance in does not exceed the performance of the physics engine.
  • the tasks include video analysis tasks and/or picture analysis tasks; wherein, for video analysis tasks, the performance of the physics engine is to support a maximum of N-channel video analysis, where N is a positive integer, and for picture analysis tasks, The performance of the physics engine is to analyze up to M pictures per unit time, and M is a positive integer.
  • limiting the performance of each virtual engine in the plurality of virtual engines according to the performance of the physical engine includes: in the case where the plurality of virtual engines all perform video analysis, limiting the The total number of video channels enabled by multiple virtual engines is less than or equal to N; or, when the multiple virtual engines all perform picture analysis, limit the total number of pictures analyzed by the multiple virtual engines per unit time to less than or equal to M; Or, limit the maximum number of pictures analyzed per unit time by at least one virtual engine used for picture analysis to: equivalent picture processing amount * (N-N1), where N1 is less than N, N1 is greater than or equal to 0, and N1 is The number of video channels enabled by the virtual engine of video analysis.
  • the equivalent image processing capacity is the number of images that can be analyzed when the ability to process one channel of video analysis is used for image analysis.
  • the limit of the maximum number of pictures analyzed by at least one virtual engine used for picture analysis per unit time is: equivalent picture processing amount * (N-N1), including: configuring picture scheduling tasks, wherein, the The picture scheduling task is used to send pictures to each virtual engine in at least one virtual engine used for picture analysis in sequence; the picture scheduling task is controlled to send a picture at a predetermined time interval to transfer the pictures used for picture analysis.
  • the maximum number of pictures analyzed by at least one virtual engine per unit time is limited to: the equivalent picture processing amount * (N-N1), where the predetermined time is used to indicate the time interval between sending two pictures, and the The predetermined time is: the unit time/[the equivalent image processing amount*(N-N1)].
  • the equivalent image processing amount is: M/q*N; where q is a configured control coefficient, and the value range of q is 1-M/N.
  • the method further includes: obtaining the analysis results obtained after the virtual engine performs the task based on the algorithm model;
  • the alarm condition determines whether the analysis result needs to be alarmed, and if so, the alarm is performed.
  • binding each algorithm model in the at least one algorithm model to run on a virtual engine includes: obtaining an operation signal, wherein the operation signal is used to transfer the first object representing the algorithm model from the first Drag a display area to a second display area.
  • the first display area displays at least one first object.
  • Each first object corresponds to an algorithm model.
  • the second display area displays at least one second object.
  • Each first display area displays at least one second object.
  • the second object corresponds to a virtual engine; in response to the operation signal, when the coincidence degree between the first object and the predetermined second object in the second display area exceeds a threshold, the first object is The corresponding algorithm model is bound to the virtual engine corresponding to the predetermined second object.
  • binding each algorithm model in the at least one algorithm model to run on a virtual engine includes: after each algorithm model is loaded on the physical engine, creating an algorithm model corresponding to the loaded algorithm model. the model handle; obtain the identification information of the virtual engine to be bound to the loaded algorithm model; establish the corresponding relationship between the model handle corresponding to the loaded algorithm model and the identification information, so as to bind the loaded algorithm model
  • the algorithm model is bound to the virtual engine corresponding to the identification information.
  • an algorithm model operation processing device including: a first acquisition module, used to acquire at least one algorithm model, wherein the algorithm model is an artificial intelligence algorithm model; a second acquisition module , used to obtain multiple virtual engines, where the virtual engine is an engine virtualized after software encapsulation of a physical engine, and the number of obtained virtual engines is greater than or equal to the number of obtained algorithm models; the binding module is used Each algorithm model in the at least one algorithm model is run and bound to a virtual engine respectively, wherein a virtual engine is used to bind an algorithm model; the execution module is used to receive tasks and execute them on the virtual engine. The task is performed on the engine based on the algorithm model.
  • the execution module is configured to obtain a pre-configured predetermined order, wherein the predetermined order is used to indicate the operation of the multiple algorithm models when multiple algorithm models are run together to perform the task. Sequence; based on the predetermined sequence, sequentially run the multiple algorithm models on multiple virtual engines corresponding to the multiple algorithm models to perform the task; wherein, after running the multiple algorithm models to execute the During the task, the output of the algorithm model run earlier is used as the input of the algorithm model run later; and/or,
  • a limiting module configured to limit the performance of each virtual engine in the plurality of virtual engines according to the performance of the physical engine when tasks are executed on the plurality of virtual engines, wherein: The performance of the physical engine is used to indicate the upper limit of the processing capability of the physical engine, and the performance of the virtual engine is used to indicate the upper limit of the processing capability of the virtual engine, wherein the plurality of virtual engines are The sum of the performance during operation does not exceed the performance of the physics engine; and/or,
  • the tasks include video analysis tasks and/or picture analysis tasks; wherein, for video analysis tasks, the performance of the physics engine is to support a maximum of N-channel video analysis, where N is a positive integer; for picture analysis tasks, the physics engine The performance of the engine is to analyze up to M pictures per unit time, where M is a positive integer; and/or,
  • the restriction module is configured to limit the total number of video channels enabled by the multiple virtual engines to be less than or equal to N when the multiple virtual engines all perform video analysis; or, when the multiple virtual engines all perform video analysis, In the case of picture analysis, limit the total number of pictures analyzed by the multiple virtual engines per unit time to less than or equal to M; or limit the maximum number of pictures analyzed per unit time by at least one virtual engine used for picture analysis to: etc.
  • the restriction module is used to configure a picture scheduling task, wherein the picture scheduling task is used to send pictures to each virtual engine in at least one virtual engine used for picture analysis in turn; control the picture scheduling task each time Send a picture at a predetermined time interval to limit the maximum number of pictures analyzed by at least one virtual engine for picture analysis per unit time to: the equivalent picture processing amount * (N-N1), where the predetermined Time is used to indicate the time interval between sending two pictures, and the predetermined time is: the unit time/[the equivalent picture processing amount*(N-N1)]; and/or,
  • the equivalent image processing capacity is: M/q*N, q is the configured control coefficient, and the value range of q is 1-M/N; and/or,
  • an alarm module used to obtain the analysis results obtained after the virtual engine performs the task based on the algorithm model; determine whether it is necessary to alarm the analysis results according to preconfigured alarm conditions, and issue an alarm if necessary ;and / or,
  • the binding module is also used to obtain an operation signal, wherein the operation signal is used to drag the first object representing the algorithm model from the first display area to the second display area, and the first display area displays at least A first object, each first object corresponds to an algorithm model, the second display area displays at least one second object, each second object corresponds to a virtual engine; in response to the operation signal, in the first When the degree of coincidence between the object and the predetermined second object in the second display area exceeds the threshold, the algorithm model corresponding to the first object is bound to the virtual engine corresponding to the predetermined second object; and / or,
  • the binding model is used to create a model handle corresponding to the loaded algorithm model after each algorithm model is loaded on the physics engine; and obtain the identification information of the virtual engine to which the loaded algorithm model is to be bound. ; Establish a corresponding relationship between the model handle corresponding to the loaded algorithm model and the identification information, so as to bind the loaded algorithm model to the virtual engine corresponding to the identification information.
  • an edge intelligent device including a memory and a processor, wherein the memory is used to store a program, the processor is used to run the program, and the program is used to execute the above Algorithm model operation processing method.
  • an algorithm model operation and processing system which is characterized in that the algorithm model operation and processing system includes an edge intelligent device and an application platform, and the user logs in to the application platform through a WEB page to perform all operations on the application platform.
  • the edge intelligent devices are managed.
  • the application platform is used to obtain at least one algorithm model from the AI open platform, and deliver the at least one algorithm model to the edge intelligent device, where the algorithm model is an artificial intelligence algorithm model; the application platform , used to manage the virtual engine of the edge intelligent device and process the binding relationship between the virtual engine and the algorithm model; , the virtual engine is an engine virtualized after software encapsulation of a physical engine, and the number of obtained virtual engines is greater than or equal to the number of obtained algorithm models; the binding relationship between the virtual engine and the algorithm model is: Each algorithm model is bound and run on a virtual engine, and a virtual engine is used to bind an algorithm model; the application platform is used to deliver tasks to the edge intelligent device; the edge intelligent device is used Receive the task issued by the application platform, and execute the task based on the algorithm model on the virtual engine.
  • the algorithm model is an artificial intelligence algorithm model
  • the application platform used to manage the virtual engine of the edge intelligent device and process the binding relationship between the virtual engine and the algorithm model
  • the virtual engine is an engine virtualized after software encapsul
  • the edge intelligent device is configured to, after executing the task, obtain the analysis results obtained by the virtual engine after executing the task based on the algorithm model, and return the analysis results to the application platform.
  • the application platform is configured to obtain the analysis results according to the preset
  • the configured alarm condition determines whether the analysis result needs to be alarmed, and if so, an alarm is generated.
  • the edge intelligent device includes: an intelligent engine subsystem for managing virtual engines; a model package management module for adding, updating, and deleting algorithm models; and an engine management module for binding algorithm models. , update and unbind; the task management module is used for task opening and/or task closing; the rule management module is used for configuring rules.
  • the rules are the rules used by the algorithm model. According to the rules, the algorithm model is made to perform the corresponding tasks. The calculation of the above rule matching; the IPC access management module, used to manage the accessed IPC and obtain the video stream from the IPC; the intelligent alarm module, used to upload alarms.
  • the edge intelligent device when it performs the task based on the algorithm model on the virtual engine, it is specifically used to: obtain a pre-configured predetermined order, wherein the predetermined order is used to indicate that the predetermined order is obtained through multiple algorithm models.
  • the running order of the multiple algorithm models In the case of cooperative operation to perform the task, the running order of the multiple algorithm models; based on the predetermined order, run the multiple algorithm models sequentially on multiple virtual engines corresponding to the multiple algorithm models to Execute the task; wherein, in the process of running the plurality of algorithm models to perform the task, the output of the algorithm model run earlier is used as the input of the algorithm model run later.
  • the edge intelligent device is also configured to limit the performance of each virtual engine in the multiple virtual engines according to the performance of the physical engine when executing tasks on the multiple virtual engines.
  • the performance of the physical engine is used to indicate the upper limit of the processing capability of the physical engine
  • the performance of the virtual engine is used to indicate the upper limit of the processing capability of the virtual engine, wherein the multiple The sum of the performance of the virtual engines during operation does not exceed the performance of the physical engine;
  • the tasks include video analysis tasks and/or picture analysis tasks; wherein, for the video analysis task, the performance of the physics engine is to support a maximum of N-channel video analysis, and the N is a positive integer.
  • the performance of the physics engine is to analyze up to M pictures per unit time, and M is a positive integer;
  • the edge intelligent device limits the performance of each virtual engine in the plurality of virtual engines according to the performance of the physical engine, it is specifically used: when all the plurality of virtual engines perform video analysis. , limit the total number of video channels enabled by the multiple virtual engines to be less than or equal to N; or, in the case where the multiple virtual engines all perform picture analysis, limit the total number of pictures analyzed by the multiple virtual engines per unit time. Less than or equal to M; or, limit the maximum number of images analyzed per unit time by at least one virtual engine used for image analysis to: equivalent image processing amount * (N-N1), N1 is less than N, N1 is greater than or equal to 0, N1 It is the number of video channels opened by the virtual engine for video analysis.
  • the equivalent image processing capacity is the number of images that can be analyzed when the ability to process one channel of video analysis is used for image analysis.
  • the edge intelligent device is also used to obtain an operation signal, the operation signal is used to drag the first object representing the algorithm model from the first display area to the second display area, the first display area displays at least A first object, each first object corresponds to an algorithm model, the second display area displays at least one second object, each second object corresponds to a virtual engine; in response to the operation signal, in the first When the degree of coincidence between the object and the predetermined second object in the second display area exceeds the threshold, the algorithm model corresponding to the first object is bound to the virtual engine corresponding to the predetermined second object; Or, after each algorithm model is loaded on the physics engine, create a model handle corresponding to the loaded algorithm model; obtain Obtain the identification information of the virtual engine to which the loaded algorithm model is to be bound; establish a corresponding relationship between the model handle corresponding to the loaded algorithm model and the identification information to bind the loaded algorithm model On the virtual engine corresponding to the identification information.
  • At least one algorithm model is obtained, where the algorithm model is an artificial intelligence algorithm model; multiple virtual engines are obtained, where the virtual engines are virtual outputs after software encapsulation of a physical engine. engine; bind each algorithm model in the at least one algorithm model to run on a virtual engine, wherein a virtual engine is used to bind an algorithm model; receive tasks, and run on the virtual engine based on The algorithm model performs the task.
  • simultaneous analysis scenarios that require support from multiple algorithm models can be implemented on an edge intelligent device (which can be an edge intelligent device with a physics engine), so that multiple algorithm models can be run simultaneously on the edge intelligent device to meet the needs of low cost. , the needs of multi-algorithm model fusion analysis.
  • Figure 1 is a schematic diagram of a single-engine single-algorithm architecture according to an embodiment of the present application
  • Figure 2 is a schematic diagram of a single-engine multi-algorithm architecture according to an embodiment of the present application
  • Figure 3 is a flow chart of an algorithm model operation processing method according to an embodiment of the present application.
  • Figure 4 is a schematic diagram of the overall architecture of the system according to an embodiment of the present application.
  • Figure 5 is a schematic structural diagram of an edge intelligent device according to an embodiment of the present application.
  • Figure 6 is a schematic architectural diagram of a single physics engine with multiple algorithms according to an embodiment of the present application
  • Figure 7 is a schematic diagram of software configuration according to an embodiment of the present application.
  • Figure 8 is a schematic diagram of a virtual engine bound within an edge intelligent device according to an embodiment of the present application.
  • Figure 9 is a schematic diagram of edge intelligent device task and rule delivery according to an embodiment of the present application.
  • Figure 10 is a flow chart of real-time video analysis tasks according to an embodiment of the present application.
  • Figure 11 is a flow chart of polling video analysis tasks according to an embodiment of the present application.
  • Figure 12 is a flow chart of a scheduled snapshot analysis task according to an embodiment of the present application.
  • Figure 13 is a flow chart of an offline image analysis task according to an embodiment of the present application.
  • Figure 14 is a flow chart of offline picture scheduling tasks according to an embodiment of the present application.
  • Figure 1 is a schematic diagram of a single engine and single algorithm architecture according to an embodiment of the present application.
  • the engine is not virtualized at the function implementation layer, there will be only one physical engine at the function implementation layer.
  • the UI level (such as AI configuration interface) also shows that there is an engine available for use.
  • AI model package 1 only one algorithm model (i.e. AI model package 1) can be configured.
  • the function implementation layer is the processing logic used to process user instructions. For example, the user issues instructions to the UI layer, the UI layer forwards the user instructions to the function implementation layer, and the function implementation layer executes the user instructions to implement related functions. In the embodiment, there is no restriction on the processing process of this function implementation layer.
  • the physics engine is an actual processing resource, which is a physical unit with image/video and other data processing capabilities, such as a processor and other units, such as a GPU (Graphics Processing Unit, graphics processor), etc.
  • a processor and other units, such as a GPU (Graphics Processing Unit, graphics processor), etc.
  • GPU Graphics Processing Unit, graphics processor
  • FIG. 2 is a schematic diagram of a single engine multi-algorithm architecture according to an embodiment of the present application, as shown in Figure 2 It shows that four virtual engines 1001, 1002, 1003 and 1004 are virtualized on a single physical engine in the function implementation layer (of course, other numbers of virtual engines can also be virtualized), so these four can also be seen in the AI configuration interface.
  • Virtual engine when configuring the algorithm model, you can configure different algorithm models for the four virtual engines respectively (that is, configure AI model package 1 for virtual engine 1001, configure AI model package 2 for virtual engine 1002, configure AI model for virtual engine 1003 Package 3. Configure the AI model package 4) for the virtual engine 1004.
  • the virtual engine is an engine that is virtualized after software encapsulation of the physical engine.
  • a physical engine can virtualize at least one virtual engine.
  • the virtual engine is the virtualization unit of the physical engine.
  • the virtual engine can independently load an algorithm model for intelligent analysis.
  • the virtual engine after software encapsulation can be understood as: based on the data processing capabilities of a physical engine, it is defined into multiple capabilities through software using software interfaces.
  • the physical engine can be used through any one or more software interfaces. Data processing capabilities.
  • This "software interface” can be equivalent to a "virtual engine”.
  • software interfaces can generally be understood as: various software development APIs, etc., or can be understood as the interactive interface between humans and software.
  • the user can see the logos or engine entrances of N engines (i.e. virtual engines) on the interactive interface.
  • N engines i.e. virtual engines
  • These displayed virtual engines are equivalent to the aforementioned "software interfaces”.
  • N virtual engines Able to use the data processing capabilities of a certain physics engine.
  • Figure 3 is a flow chart of an algorithm model operation processing method according to an embodiment of the present application.
  • Figure 3 shows the process of configuring the algorithm model for the virtual engine. As shown in Figure 3, the process includes the following steps:
  • Step S302 Obtain at least one algorithm model, where the algorithm model is an artificial intelligence algorithm model.
  • Step S304 Acquire multiple virtual engines, where the virtual engine is an engine virtualized after software encapsulation of a physical engine.
  • the number of acquired virtual engines is greater than or equal to the number of acquired algorithm models.
  • Step S306 Bind each algorithm model in the above at least one algorithm model to run on a virtual engine, where a virtual engine is used to bind an algorithm model.
  • Step S308 Receive the task and execute the task on the virtual engine based on the bound algorithm model.
  • edge intelligent device which can be an edge intelligent device with a physics engine
  • multiple algorithm models can be run simultaneously on the edge intelligent device to meet the needs of low-end users. Cost and multi-algorithm model fusion analysis requirements.
  • the algorithm model in the above steps can be obtained from multiple channels, for example, from the AI open platform.
  • the above steps can be run on edge smart devices, which can obtain algorithm models from the AI liberalization platform, and can also manage edge smart devices and algorithm models through industry integrated application platforms, WEB pages, or other clients. Binding and other management on the virtual engine.
  • FIG 4 is a schematic diagram of the overall system architecture according to an embodiment of the present application.
  • edge smart devices can be connected to the industry integrated application platform and log in to the industry integrated application platform (referred to as the platform) through the WEB page.
  • the industry integrated application platform can also obtain algorithm models (simply referred to as devices) from the AI open platform called a model), manage the model, and establish a model resource library.
  • the industry-integrated application platform can also manage the virtual engine of edge intelligent devices and process the binding relationship between the virtual engine and the model; in order to perform intelligent analysis tasks, the industry-integrated application platform can also deliver data to edge intelligent devices.
  • Tasks and rules, edge intelligent devices can return analysis results to the industry integrated application platform after performing tasks. That is, obtain the analysis results obtained after the virtual engine performs tasks based on the algorithm model; determine whether it is necessary to alarm the analysis results according to the preconfigured alarm conditions, and if so, issue an alarm.
  • FIG. 5 is a schematic structural diagram of an edge intelligent device according to an embodiment of the present application.
  • the edge intelligent device communicates with the platform (ie, industry integrated application platform) and network camera (IPC) through a protocol interface.
  • the intelligent engine subsystem of the edge intelligent device is used to manage the virtual engine.
  • engine management module used for models Binding, updating and unbinding of packages
  • task management module used to open and/or close tasks
  • rule management module used to configure rules (this rule is a rule used by each algorithm model, according to which the algorithm model is made to perform Corresponding calculation)
  • IPC access management module used to manage accessed IPCs and obtain video streams from IPCs
  • intelligent alarm module used to upload alarms.
  • edge intelligent devices can be accessed through the platform (i.e., industry integrated application platform) and WEB, and algorithm models can be added, updated, deleted, engine binding/unbinding models, and task rules added to the edge intelligent devices. Release operations, etc.
  • the platform adds the algorithm model to the device storage model partition, and when the algorithm model is bound to the virtual engine, it loads the algorithm model into the virtual engine for running.
  • the platform can issue tasks and rules and notify the virtual engine to perform intelligent analysis of the corresponding algorithm model.
  • the IPC access management module connects to the front-end IPC and obtains video streams or pictures in real time and sends them to the intelligent engine subsystem.
  • the intelligent engine subsystem performs intelligent analysis on the results of the virtual engine analysis and reports alarms to the platform.
  • the intelligent engine subsystem is used to manage the virtual engine.
  • an algorithm model in terms of usage configuration, an algorithm model is bound to a physical engine.
  • the physical engine is encapsulated within the software and virtualized into multiple virtual engines. This processing method is similar to the multiple logical computing cores of the CPU.
  • the edge intelligent device converts the received engine information into the internal corresponding virtual engine unit corresponding to the actual physical engine, and completes the corresponding action.
  • the advantage of this design is that it can be directly connected to the platform quickly without modifying the external interface logic.
  • binding the virtual engine and the algorithm model There are many implementation methods for binding the virtual engine and the algorithm model.
  • the following binding method is provided: after each algorithm model is loaded on the physical engine, the creation and loading of The model handle corresponding to the algorithm model; obtain the identification information (referred to as ID) of the virtual engine to be bound to the loaded algorithm model; establish the corresponding relationship between the model handle corresponding to the loaded algorithm model and the identification information, so as to load the The completed algorithm model is bound to the virtual engine corresponding to the identification information.
  • This binding method can be understood as supporting the simultaneous loading of multiple algorithm models. Every time an algorithm model is loaded on the physics engine, a model handle is created, and the model handle is associated with the virtual engine ID. In this way, the application layer performs operations such as algorithm model rule configuration, video streaming, and picture sending for a single virtual engine. Both can be sent to the corresponding algorithm model for analysis through the association between the virtual engine ID and the algorithm model, thereby realizing virtualization of a single physical engine and mixed operation of multiple algorithm models.
  • Figure 6 is a schematic architectural diagram of a single physical engine with multiple algorithms according to an embodiment of the present application.
  • the architecture mainly includes: AI functional interface (for configuring algorithm models, binding algorithm models to virtual engines, etc.) , intelligent engine subsystem, function implementation layer and virtual conversion layer.
  • the virtual conversion layer exists between the AI function interface, the intelligent engine subsystem and the function implementation layer of the edge intelligent device. Its purpose is to perform conversion between the physical engine and the virtual engine, as well as the interaction of protocols and data.
  • Intelligent engine The system virtualizes 4 virtual engines for a single physical engine (other numbers of virtual engines can also be virtualized), and can load and run different algorithm models at the same time for intelligent analysis.
  • each virtual engine After each virtual engine loads the algorithm model, it will analyze the current algorithm model Create a unique operation handle, and subsequent model rule configuration and data interaction are all performed on a single handle, so the algorithm model can be associated with the virtual engine.
  • edge smart devices Before engine virtualization, edge smart devices had only one physical engine and could only run one algorithm model at the same time.
  • Figure 6 can quickly implement multiple algorithms on a single physical engine by changing the software architecture implementation without changing the hardware architecture. Model analysis, expand product application scenarios, reduce product prices, and improve product competitiveness.
  • a binding software interface can be provided.
  • binding can be performed by dragging.
  • operation signals can be obtained, such as receiving from The user's operation signal, where the operation signal is used to drag the first object representing the algorithm model from the first display area to the second display area.
  • the first display area displays at least one first object, and each first object corresponds to An algorithm model
  • the second display area displays at least one second object, each second object corresponds to a virtual engine; in response to the operation signal, the coincidence degree between the first object and the predetermined second object in the second display area exceeds
  • the algorithm model corresponding to the first object is bound to the virtual engine corresponding to the predetermined second object.
  • Figure 7 is a schematic diagram of software configuration according to an embodiment of the present application.
  • Figure 7 shows the AI configuration page, through which the algorithm model can be configured on the virtual engine.
  • the helmet detection model in the model library can be dragged Go to the virtual engine (or virtual engine unit) 1001, that is, bind the helmet detection model to the virtual engine 1001, and drag the pig detection model to the virtual engine 1002, that is, bind the pig detection model to the virtual engine 1002. Binding.
  • the engines displayed to the user are all virtual engines.
  • a single algorithm model can be bound to a virtual engine corresponding to the physical engine by dragging and dropping.
  • a safety helmet detection model can also be bound to a virtual engine.
  • bind the pig detection model to the virtual engine 1002 bind the battery car detection model to the virtual engine 1003, and bind the fire channel occupied detection model to the virtual engine 1004, so that a single physical engine can Realize multiple algorithm models for simultaneous intelligent analysis.
  • the tasks corresponding to each virtual engine can also be configured.
  • the analysis mode real-time video analysis, polling video analysis, scheduled image analysis, offline image analysis, etc. can be selected; the analysis channel can also be selected. For example, select IPCamera01 or IPCamera02 channel.
  • Figure 7 also shows whether the running status of the algorithm model on each virtual engine is normal.
  • Figure 8 is a schematic diagram of binding a virtual engine within an edge intelligent device according to an embodiment of the present application.
  • the network management module parses the protocol content and notifies the engine management module to perform the AI model (i.e., algorithm model) is bound to the virtual engine unit (i.e., virtual engine).
  • the engine management module issues engine loading model instructions to the intelligent engine subsystem, and loads the algorithm model to the designated idle virtual engine.
  • the algorithm model can also be configured with tasks.
  • the task can be configured as real-time video analysis, and the channel of the real-time video analysis can also be added.
  • Channel 1 is configured in Figure 7 and channel 2 for real-time video analysis.
  • FIG 9 is a schematic diagram of edge intelligent device task and rule delivery according to an embodiment of the present application.
  • the platform delivers the task and the parameters required to execute the task (for example, the parameters can be carried in the rules) to the engine.
  • the protocol is parsed and distributed to the task management module and rule management module through the network management module inside the device.
  • the task management module issues different tasks to different virtual engine units (i.e. virtual engines), and the rule management module issues different tasks respectively.
  • the rules are assigned to different virtual engine units. Each virtual engine unit starts an analysis task and performs intelligent analysis of videos and pictures according to different rules.
  • the analysis results are reported to the platform through the intelligent alarm management module.
  • alarm conditions can be configured in advance: that is, if it is detected that no helmet is worn, an alarm will be issued; after analysis, if it is found that no helmet is worn, it is determined that an alarm needs to be issued.
  • Different algorithm models can be loaded on different virtual engines, and various algorithm models can also be combined to realize the analysis of multiple scenarios. That is, a pre-configured predetermined sequence can be obtained, where the predetermined sequence is used to indicate when passing through multiple scenarios.
  • algorithm model When running together to perform a task, the running sequence of multiple algorithm models running together; based on the predetermined order, run multiple algorithm models in sequence on multiple virtual engines corresponding to the multiple algorithm models running together to execute the task .
  • the output of the algorithm model run earlier is used as the input of the algorithm model run later.
  • virtual engine 1 detects an image of a specified part, it can send the image of the specified part to virtual engine 2 to continue image comparison and confirm which user this is; or if virtual engine 1 detects the user's behavior, it can send the image By continuing to compare images with Virtual Engine 2, you can confirm which user's behavior it is; expand a variety of application scenarios from simple AI detection, classification, and hybrid model analysis to meet customer needs.
  • Video analysis tasks can include real-time video analysis tasks, polling video analysis tasks, etc.
  • Image analysis tasks can include scheduled snapshot analysis tasks and offline image analysis tasks. These tasks are explained below.
  • Figure 10 is a flow chart of a real-time video analysis task according to an embodiment of the present application.
  • the real-time video analysis task issued by the parsing platform. If the virtual engine already has a task being analyzed, the process ends. If the virtual engine If there is no task being analyzed, save the task-related information (for example, task ID, associated channel), send the real-time video data of the associated multiple channels to the virtual engine, receive the results analyzed by the virtual engine and send them to the intelligent alarm management module.
  • the task-related information for example, task ID, associated channel
  • FIG 11 is a flow chart of a polling video analysis task according to an embodiment of the present application.
  • the parsing platform issues a polling video analysis task. If the virtual engine already has a task being analyzed, the process ends. If If there is no task being analyzed in the virtual engine, the task-related information (such as task ID, polling interval, associated channel) is saved, the associated 1-channel real-time video data is sent to the virtual engine, the virtual engine analysis results are received, and It is sent to the intelligent alarm management module to determine whether the current channel analysis time reaches the polling interval. If the polling interval is reached, the next new channel associated with the task is switched for real-time video analysis.
  • the task-related information such as task ID, polling interval, associated channel
  • Figure 12 is a flow chart of a scheduled image capture analysis task according to an embodiment of the present application.
  • the analysis platform issues a scheduled image capture task. If the virtual engine already has a task being analyzed, the process ends. If the virtual engine already has a task being analyzed, the process ends. If there is no task being analyzed in the engine, the task-related information (for example, task ID, polling interval, associated channel) is saved, the snapshot data of the associated channel is sent to the virtual engine, and the analysis results of the virtual engine are received and It is sent to the intelligent alarm management module to determine whether the current channel analysis time reaches the polling interval. If the polling interval is reached, the next new channel associated with the task is switched for image analysis.
  • the task-related information for example, task ID, polling interval, associated channel
  • the performance of each virtual engine in the multiple virtual engines can be evaluated based on the performance of the physical engine.
  • Limitation where the performance here is used to indicate the upper limit of the engine's processing capability, that is, the performance of the physical engine is used to indicate the upper limit of the physical engine's processing capability, and the performance of the virtual engine is used to indicate the processing of the virtual engine
  • the upper limit of the capability in which the sum of the performance of multiple virtual engines during operation does not exceed the performance of the physical engine.
  • the performance of the physics engine is to support N-channel video analysis at most, N is a positive integer; or, for image analysis tasks, the performance of the physics engine is to analyze up to M pictures per unit time, and M is a positive integer; Under this kind of request, the total number of video analysis channels supported by all virtual engines cannot exceed N channels, and the total number of pictures analyzed by all virtual engines per unit time cannot exceed M.
  • a single physics engine supports up to 4 channels of real-time video analysis performance. Since the virtual engine is virtualized based on the physical engine, the total computing resources in a single physical engine are 4-channel real-time video analysis In the case of The computing resources of the four virtual engines can be dynamically adjusted. For example, if virtual engine 1 is bound to algorithm model 1 and 1 channel of real-time video analysis is enabled, algorithm model 2 bound to virtual engine 2 can also enable 3 channels of real-time video analysis; if If virtual engine 1 turns on 4 channels, other virtual engines can no longer enable real-time video analysis, because the total computing resources of all virtual engines in a single physical engine have used 4 channels. When the total computing resources of all virtual engines in a single physical engine have used 4 channels, , when the channel real-time video analysis is turned on, the prompt "Insufficient engine resources" will be given.
  • the real-time video analysis task will occupy 1 channel of real-time video analysis performance if it is associated with one channel; because the polling video analysis task performs single-channel real-time video analysis according to the associated channel polling, only one channel will be polled at the same time.
  • Enable 1 channel real-time video analysis so the polling video task only occupies 1 channel of real-time video analysis performance no matter how many channels are associated;
  • a single physics engine supports offline image analysis performance of 16 images/second, and is the same as real-time video analysis performance.
  • Mutually exclusive that is, if 4-channel real-time video analysis is enabled, offline image analysis cannot be performed; the above performance limitations are for a single physics engine.
  • a single physics engine needs to be encapsulated and virtualized 4
  • Each virtual engine has the same task performance limits as the physical engine, and the total analysis performance of the four virtual engines corresponding to the physical engine is the same as the performance of a single physical engine.
  • the other three virtual engines can no longer enable real-time video analysis or image analysis. If virtual engine 1 enables 1-channel real-time video analysis, virtual engine 2 enables 2-channel real-time video analysis, and virtual engine 3 can also enable 1-channel real-time video analysis.
  • the 4 virtual engines enable a total of 4 channels of real-time video analysis.
  • each virtual engine in the virtual engine also include: when some virtual engines among multiple virtual engines perform video analysis and another part of the virtual engines perform image analysis, some virtual engines turn on the N1 video, and the other virtual engines (That is, at least one virtual engine used for image analysis)
  • equivalent image processing amount * (N-N1) where N1 is less than N, N1 is greater than 0, and N1 is The number of video channels enabled by the video analysis virtual engine.
  • the equivalent image processing capacity is the number of images that can be analyzed when the ability to process one channel of video analysis is used for image analysis.
  • the equivalent image processing capacity can be calculated based on M and N.
  • the equivalent image processing capacity can be: M/q*N; for example, M/q*N is rounded up, or M/q*N is rounded down; where q is the configured control coefficient, and the value range of q is 1-M/N.
  • the value of q can be configured based on experience, as long as it is 1-M/N. If q is 1, the equivalent image processing capacity can be M/N. If q is M/N, the equivalent image processing capacity can be The processing capacity can be 1, that is to say, the equivalent image processing capacity is a value between 1-M/N.
  • the speed of sending images to another part of the virtual engine can also be controlled based on the delay time, so that the speed of the other part of the virtual engine per unit time can be controlled.
  • the maximum number of analyzed pictures is limited to: equivalent picture processing * (N-N1), where the delay time is used to indicate the time interval between sending two pictures, and the delay time is unit time/[equivalent picture processing Quantity*(N-N1)].
  • a predetermined time interval ie, delay time
  • the maximum number of pictures analyzed by at least one virtual engine used for picture analysis per unit time is limited to: equivalent picture processing capacity * (N-N1), where the predetermined time is used to indicate the time interval between sending two pictures,
  • the scheduled time is: unit time/[equivalent image processing amount*(N-N1)].
  • the virtual engine is allowed to start offline image tasks.
  • the virtual engine 1 has 3 channels of real-time video analysis enabled, and the other three A virtual engine can also start offline picture tasks.
  • a single physics engine supports up to 4 channels of real-time video analysis or 16 pictures/second analysis performance, so the corresponding offline image analysis performance of 1 channel of real-time video analysis can be converted to 4 pictures/second.
  • Scenario 1 3 virtual engines perform real-time video analysis, and 1 virtual engine performs offline image analysis.
  • Virtual engines 1/2/3 respectively enable 1 channel of real-time video analysis, occupying 3 channels of real-time video analysis performance, and the corresponding physical engine has remaining capacity.
  • the performance of 1-channel real-time video analysis and picture analysis performance is 4 pictures/second, so Virtual Engine 4 can also enable offline picture analysis tasks, and the performance of this picture analysis task needs to be limited to 4 pictures/second.
  • Scenario 2 All four virtual engines perform offline image analysis, and each of the four virtual engines starts offline image analysis tasks. That is, the four offline image tasks together occupy 16 images/second analysis performance.
  • Figure 13 is a flow chart of an offline picture analysis task according to an embodiment of the present application.
  • Figure 14 is a flow chart of an offline picture scheduling task according to an embodiment of the present application.
  • the offline image task dynamically adjusts the delay time uiDelayTime when sending offline images. To control the speed of sending pictures to prevent multiple offline picture tasks from sending too many pictures, resulting in insufficient virtual engine picture analysis performance. It also prevents multiple offline picture tasks from delaying and waiting.
  • an offline picture The task has been seizing CPU resources to start sending pictures, and other offline picture tasks have been waiting.
  • the operation situation is as follows: the scheduling task notifies offline picture task 1 to send pictures, and other offline picture tasks delay and wait. After offline picture task 1 sends the picture, it starts delayed waiting. After that, the offline picture scheduling task is notified to poll and schedule the next virtual engine's offline picture task 2, and the offline picture task 2 is notified to send the picture. The offline picture task 2 preempts to send the picture, and other offline picture tasks are delayed and waited, and each task turn is followed in turn. Ask for dispatch.
  • the calculation of the offline image task delay time uiDelayTime is based on the remaining real-time video analysis performance of a single physics engine.
  • the calculation principle is as follows:
  • a)iUsedChanNum is the statistical number of real-time video analysis channels occupied by a single physics engine
  • c)iResAnalysePicNum iResChanNum*4; is the remaining picture analysis performance of a single physics engine (1 channel real-time video analysis performance corresponds to a picture analysis performance of 4 pictures/s).
  • the software architecture implementation method is changed without changing the hardware architecture, and the physical engine is encapsulated and virtualized into multiple virtual engines through a single-engine multi-algorithm solution, which can quickly perform single-physical engine multi-algorithm analysis and expand Product application scenarios, reduce product prices, and enhance product competitiveness.
  • each virtual engine loads different algorithm models and can perform integrated intelligent analysis on the same video. For example, if virtual engine 1 detects a specified part, it can send the image of the specified part to virtual engine 2 to continue image comparison, which can confirm the which user. Expand a variety of application scenarios from simple AI detection, classification, and hybrid model analysis to meet customer needs.
  • time slice polling and dynamically adjusting the picture sending speed are also used to ensure that a single physics engine can start multiple offline picture tasks for picture analysis, which meets the current engine picture analysis performance.
  • an edge intelligent device including a memory and a processor.
  • a computer program is stored in the memory, and the processor is configured to run the computer program to execute the method in the above embodiment.
  • Computer-readable media include permanent and non-permanent, removable and non-removable media that can be stored in memory by any method or technology to achieve information storage. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory.
  • PRAM phase change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • read-only memory read-only memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other memory technology
  • compact disc read-only memory CD-ROM
  • DVD digital versatile disc
  • Magnetic tape cassettes tape disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device.
  • computer-readable media does not include transient computer-readable media (transitory media), such as modulated data signals and carrier waves.
  • These computer programs may also be loaded onto a computer or other programmable data processing device such that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processes, thereby causing instructions to be executed on the computer or other programmable device
  • steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one or multiple boxes in the block diagram, and corresponding to different steps can be implemented through different modules.
  • the device is called an algorithm model operation processing device and includes: a first acquisition module, used to acquire at least one algorithm model, wherein the algorithm model is an artificial intelligence algorithm model; a second acquisition module, used to acquire multiple virtual engines , wherein the virtual engine is an engine virtualized after software encapsulation of a physical engine, and the number of obtained virtual engines is greater than or equal to the number of obtained algorithm models; the binding module is used to combine the at least one algorithm model Each algorithm model in is bound to run on a virtual engine, where a virtual engine is used to bind an algorithm model; the execution module is used to receive tasks and execute them based on the algorithm model on the virtual engine. said task.
  • the system or device is used to implement the functions of the method in the above embodiments.
  • Each module in the system or device corresponds to each step in the method. Those that have been described in the method will not be described again here. .
  • an alarm module used to obtain the analysis results obtained after the virtual engine performs the task based on the algorithm model; determine whether it is necessary to alarm the analysis results according to preconfigured alarm conditions, and if so, Provide an alert.
  • the execution module is used to obtain a preconfigured predetermined sequence, wherein the predetermined sequence is used to indicate the operation of the multiple algorithm models when multiple algorithm models are run together to perform the task. Sequence; based on the predetermined sequence, sequentially run the multiple algorithm models on multiple virtual engines corresponding to the multiple algorithm models to perform the task; wherein, in In the process of running the plurality of algorithm models to perform the task, the output of the algorithm model run earlier is used as the input of the algorithm model run later.
  • the method further includes: a restriction module configured to limit the performance of each virtual engine in the multiple virtual engines according to the performance of the physical engine when tasks are executed on the multiple virtual engines, wherein, the performance of the physical engine is used to indicate an upper limit value of the processing capability of the physical engine, and the performance of the virtual engine is used to indicate an upper limit value of the processing capability of the virtual engine, wherein the plurality of The sum of the performance of the virtual engine during operation does not exceed the performance of the physical engine.
  • a restriction module configured to limit the performance of each virtual engine in the multiple virtual engines according to the performance of the physical engine when tasks are executed on the multiple virtual engines, wherein, the performance of the physical engine is used to indicate an upper limit value of the processing capability of the physical engine, and the performance of the virtual engine is used to indicate an upper limit value of the processing capability of the virtual engine, wherein the plurality of The sum of the performance of the virtual engine during operation does not exceed the performance of the physical engine.
  • the tasks include video analysis tasks and/or picture analysis tasks; wherein, for video analysis tasks, the performance of the physics engine is to support a maximum of N-channel video analysis, where N is a positive integer, and for picture analysis tasks , the performance of the physics engine is to analyze up to M pictures per unit time, and M is a positive integer.
  • the restriction module is configured to limit the total number of video channels enabled by the multiple virtual engines to less than or equal to N when the multiple virtual engines all perform video analysis; or, when the multiple virtual engines perform video analysis, When all virtual engines perform picture analysis, limit the sum of the number of pictures analyzed by the multiple virtual engines per unit time to be less than or equal to M; or, limit the maximum number of pictures analyzed per unit time by at least one virtual engine used for picture analysis.
  • the limit is: equivalent image processing capacity * (N-N1), where N1 is less than N, N1 is greater than or equal to 0, N1 is the number of video channels enabled by the virtual engine for video analysis, and the equivalent image processing capacity is The ability of video analysis is used to analyze the number of pictures that can be analyzed when performing picture analysis.
  • the restriction module is used to configure a picture scheduling task, wherein the picture scheduling task is used to send pictures to each virtual engine in at least one virtual engine used for picture analysis in turn; control the picture
  • the scheduling task sends a picture at a predetermined time interval to limit the maximum number of pictures analyzed by at least one virtual engine for picture analysis per unit time to: the equivalent picture processing amount * (N-N1), where , the predetermined time is used to indicate the time interval between sending two pictures, and the predetermined time is: the unit time/[the equivalent picture processing amount * (N-N1)].
  • the equivalent image processing amount is: M/q*N; where q is a configured control coefficient, and the value range of q is 1-M/N.
  • the binding module is also used to obtain an operation signal, wherein the operation signal is used to drag the first object representing the algorithm model from the first display area to the second display area, and the first display area
  • the area displays at least one first object, each first object corresponds to an algorithm model, the second display area displays at least one second object, and each second object corresponds to a virtual engine; in response to the operation signal, the When the degree of overlap between the first object and the predetermined second object in the second display area exceeds a threshold, the algorithm model corresponding to the first object is bound to the virtual virtual object corresponding to the predetermined second object. on the engine.
  • the binding model is used to create a model handle corresponding to the loaded algorithm model after each algorithm model is loaded on the physics engine; and obtain the virtual object to be bound to the loaded algorithm model.
  • Identification information of the engine establishing a corresponding relationship between the model handle corresponding to the loaded algorithm model and the identification information, so as to bind the loaded algorithm model to the virtual engine corresponding to the identification information.
  • simultaneous analysis scenarios that require the support of multiple algorithm models can be implemented on an edge intelligent device (which can be an edge intelligent device with a physics engine), so that multiple algorithm models can be run simultaneously on the edge intelligent device to meet the requirements of low cost. Cost and multi-algorithm model fusion analysis requirements.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本申请公开了一种算法模型运行处理方法、装置及边缘智能设备，该方法包括：获取至少一个算法模型，其中，所述算法模型为人工智能算法模型；获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎；将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型；接收任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。通过本申请能够在边缘智能设备上实现多算法模型同时运行，满足低成本、多算法模型融合分析的需求。

Description

一种算法模型运行处理方法、装置及边缘智能设备 技术领域
本申请涉及到智能分析领域，具体涉及一种算法模型运行处理方法、装置及边缘智能设备。
背景技术
随着AI(Artificial Intelligence，人工智能)开放平台的推广，AI开放平台需要支持多个算法模型，如实现视频结构化处理的算法模型、实现图像分析的算法模型等。为了支持多个算法模型，需要部署多个边缘智能设备(也可以称作边缘智能盒子)，边缘智能设备是用于对视频/图片进行智能分析的设备，每个边缘智能设备只能支持一个算法模型。比如说，边缘智能设备具有一颗物理引擎，在一颗物理引擎上运行一个算法模型，从而实现该算法模型支持的功能。或者，为了支持多个算法模型，需要在边缘智能设备中增加物理引擎的个数，每个物理引擎只能支持一个算法模型。比如说，边缘智能设备具有多颗物理引擎，在一颗物理引擎上运行一个算法模型，从而实现该算法模型支持的功能。显然，针对具有一颗物理引擎的边缘智能设备，一些需要多算法模型支持的同时分析场景无法在边缘智能设备上实现。针对具有多颗物理引擎的边缘智能设备，其成本较高，不适合部署具有多颗物理引擎的边缘智能设备。
发明内容
本申请实施例提供了一种算法模型运行处理方法、装置及边缘智能设备，能够在边缘智能设备上实现需要多算法模型支持的同时分析场景。
根据本申请的一个方面，提供了一种算法模型运行处理方法，包括：获取至少一个算法模型，其中，所述算法模型为人工智能算法模型；获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，获取的虚拟引擎的数量大于或等于获取的算法模型的数量；将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型；接收任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。
进一步地，所述在所述虚拟引擎上基于所述算法模型执行所述任务，包括：获取预先配置的预定顺序，其中，所述预定顺序用于指示在通过多个算法模型配合运行以执行所述任务的情况下，所述多个算法模型的运行顺序；基于所述预定顺序，在所述多个算法模型对应的多个虚拟引擎上依次运行所述多个算法模型以执行所述任务；其中，在运行所述多个算法模型以执行所述任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入。
进一步地，还包括：在所述多个虚拟引擎上执行任务的情况下，根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，其中，所述物理引擎的性能用于指示所述物理引擎的处理能力的上限值，所述虚拟引擎的性能用于指示所述虚拟引擎的处理能力的上限值，其中，所述多个虚拟引擎在运行过程中的性能之和不超过所述物理引擎的性能。
进一步地，所述任务包括视频分析任务和/或图片分析任务；其中，针对视频分析任务，所述物理引擎的性能为最大支持N路视频分析，所述N为正整数，针对图片分析任务，所述物理引擎的性能为单位时间内最多分析M张图片，所述M为正整数。
进一步地，所述根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，包括：在所述多个虚拟引擎均进行视频分析的情况下，限制所述多个虚拟引擎开启的视频路数总和小于等于N；或者，在所述多个虚拟引擎均进行图片分析的情况下，限制所述多个虚拟引擎单位时间内分析图片数量的总和小于等于M；或者，将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，其中，N1小于N，N1大于等于0，N1为进行视频分析的虚拟引擎开启的视频路数，所述等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量。
进一步地，所述将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，包括：配置图片调度任务，其中，所述图片调度任务用于依次向用于进行图片分析的至少一个虚拟引擎中的每个虚拟引擎发送图片；控制所述图片调度任务每次间隔预定时间发送一张图片，以将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：所述等效图片处理量*(N-N1)，其中，所述预定时间用于指示发送两张图片之间的时间间隔，所述预定时间为：所述单位时间/[所述等效图片处理量*(N-N1)]。
进一步地，所述等效图片处理量为：M/q*N；其中，q为已配置的控制系数，q的取值范围是1-M/N。
进一步地，在所述虚拟引擎上基于所述算法模型执行所述任务之后，所述方法还包括：获取所述虚拟引擎基于所述算法模型执行所述任务之后得到的分析结果；根据预先配置的告警条件确定是否需要对所述分析结果进行告警，如果需要，则进行告警。
进一步地，将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，包括：获取操作信号，其中，所述操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，所述第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，所述第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；响应于所述操作信号，在所述第一对象与所述第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将所述第一对象对应的算法模型绑定在所述预定的第二对象对应的虚拟引擎上。
进一步地，将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，包括：在所述物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；获取所述加载完的算法模型待绑定的虚拟引擎的标识信息；建立所述加载完的算法模型对应的模型句柄与所述标识信息的对应关系，以将所述加载完的算法模型绑定在所述标识信息对应的虚拟引擎上。
根据本申请的另一个方面，还提供了一种算法模型运行处理装置，包括：第一获取模块，用于获取至少一个算法模型，其中，所述算法模型为人工智能算法模型；第二获取模块，用于获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，获取的虚拟引擎的数量大于或等于获取的算法模型的数量；绑定模块，用于将所述至少一个算法模型中的每个算法模型分别运行绑定在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型；执行模块，用于接收任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。
进一步地，所述执行模块用于获取预先配置的预定顺序，其中，所述预定顺序用于指示在通过多个算法模型配合运行以执行所述任务的情况下，所述多个算法模型的运行顺序；基于所述预定顺序，在所述多个算法模型对应的多个虚拟引擎上依次运行所述多个算法模型以执行所述任务；其中，在运行所述多个算法模型以执行所述任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入；和/或，
还包括：限制模块，用于在所述多个虚拟引擎上执行任务的情况下，根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，其中，所述物理引擎的性能用于指示所述物理引擎的处理能力的上限值，所述虚拟引擎的性能用于指示所述虚拟引擎的处理能力的上限值，其中，所述多个虚拟引擎在运行过程中的性能之和不超过所述物理引擎的性能；和/或，
所述任务包括视频分析任务和/或图片分析任务；其中，针对视频分析任务，所述物理引擎的性能为最大支持N路视频分析，所述N为正整数，针对图片分析任务，所述物理引擎的性能为单位时间内最多分析M张图片，所述M为正整数；和/或，
所述限制模块，用于在所述多个虚拟引擎均进行视频分析的情况下，限制所述多个虚拟引擎开启的视频路数总和小于等于N；或者，在所述多个虚拟引擎均进行图片分析的情况下，限制所述多个虚拟引擎单位时间内分析图片数量的总和小于等于M；或者，将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，其中，N1小于N，N1大于等于0，N1为进行视频分析的虚拟引擎开启的视频路数，所述等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量；和/或，
所述限制模块，用于配置图片调度任务，其中，所述图片调度任务用于依次向用于进行图片分析的至少一个虚拟引擎中的每个虚拟引擎发送图片；控制所述图片调度任务每次间隔预定时间发送一张图片，以将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：所述等效图片处理量*(N-N1)，其中，所述预定时间用于指示发送两张图片之间的时间间隔，所述预定时间为：所述单位时间/[所述等效图片处理量*(N-N1)]；和/或，
所述等效图片处理量为：M/q*N，q为已配置的控制系数，q的取值范围是1-M/N；和/或，
还包括：告警模块，用于获取所述虚拟引擎基于所述算法模型执行所述任务之后得到的分析结果；根据预先配置的告警条件确定是否需要对所述分析结果进行告警，如果需要则进行告警；和/或，
所述绑定模块还用于获取操作信号，其中，所述操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，所述第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，所述第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；响应于所述操作信号，在所述第一对象与所述第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将所述第一对象对应的算法模型绑定在所述预定的第二对象对应的虚拟引擎上；和/或，
所述绑定模型用于在所述物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；获取所述加载完的算法模型待绑定的虚拟引擎的标识信息；建立所述加载完的算法模型对应的模型句柄与所述标识信息的对应关系，以将所述加载完的算法模型绑定在所述标识信息对应的虚拟引擎上。
根据本申请的一个方面，还提供了一种边缘智能设备，包括存储器和处理器，其中，所述存储器用于存储程序，所述处理器用于运行所述程序，所述程序用于执行上述的算法模型运行处理方法。
根据本申请的一个方面，还提供了一种算法模型运行处理系统，其特征在于，所述算法模型运行处理系统包括边缘智能设备和应用平台，用户通过WEB页面登陆到所述应用平台来对所述边缘智能设备进行管理。其中：所述应用平台，用于从AI开放平台中获取至少一个算法模型，将所述至少一个算法模型下发给所述边缘智能设备，所述算法模型为人工智能算法模型；所述应用平台，用于对所述边缘智能设备的虚拟引擎进行管理，对虚拟引擎与算法模型之间的绑定关系进行处理；其 中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，且获取的虚拟引擎的数量大于或等于获取的算法模型的数量；虚拟引擎与算法模型之间的绑定关系是：将每个算法模型分别绑定运行在一个虚拟引擎上，一个虚拟引擎用于绑定一个算法模型；所述应用平台，用于向所述边缘智能设备下发任务；所述边缘智能设备，用于接收所述应用平台下发的所述任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。
进一步的，所述边缘智能设备，用于在执行所述任务之后，获取虚拟引擎基于算法模型执行任务之后得到的分析结果，向所述应用平台返回分析结果，所述应用平台，用于根据预先配置的告警条件确定是否需要对所述分析结果进行告警，如果需要，则进行告警。
进一步的，所述边缘智能设备包括：智能引擎子系统，用于进行虚拟引擎的管理；模型包管理模块，用于算法模型的添加、更新和删除；引擎管理模块，用于算法模型的绑定、更新和解绑；任务管理模块，用于任务开启，和/或，任务关闭；规则管理模块，用于配置规则，所述规则为算法模型使用的规则，根据所述规则使得算法模型进行与所述规则匹配的计算；IPC接入管理模块，用于管理接入的IPC以及获取来自IPC的视频流；智能告警模块，用于进行告警上传。
进一步的，所述边缘智能设备在所述虚拟引擎上基于所述算法模型执行所述任务时具体用于：获取预先配置的预定顺序，其中，所述预定顺序用于指示在通过多个算法模型配合运行以执行所述任务的情况下，所述多个算法模型的运行顺序；基于所述预定顺序，在所述多个算法模型对应的多个虚拟引擎上依次运行所述多个算法模型以执行所述任务；其中，在运行所述多个算法模型以执行所述任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入。
进一步的，所述边缘智能设备，还用于在所述多个虚拟引擎上执行任务的情况下，根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，其中，所述物理引擎的性能用于指示所述物理引擎的处理能力的上限值，所述虚拟引擎的性能用于指示所述虚拟引擎的处理能力的上限值，其中，所述多个虚拟引擎在运行过程中的性能之和不超过所述物理引擎的性能；
其中，所述任务包括视频分析任务和/或图片分析任务；其中，针对视频分析任务，所述物理引擎的性能为最大支持N路视频分析，所述N为正整数，针对图片分析任务，所述物理引擎的性能为单位时间内最多分析M张图片，所述M为正整数；
其中，所述边缘智能设备根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制时具体用于：在所述多个虚拟引擎均进行视频分析的情况下，限制所述多个虚拟引擎开启的视频路数总和小于等于N；或者，在所述多个虚拟引擎均进行图片分析的情况下，限制所述多个虚拟引擎单位时间内分析图片数量的总和小于等于M；或者，将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，N1小于N，N1大于等于0，N1为进行视频分析的虚拟引擎开启的视频路数，所述等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量。
进一步，所述边缘智能设备还用于获取操作信号，所述操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，所述第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，所述第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；响应于所述操作信号，在所述第一对象与所述第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将所述第一对象对应的算法模型绑定在所述预定的第二对象对应的虚拟引擎上；或者，在所述物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；获 取所述加载完的算法模型待绑定的虚拟引擎的标识信息；建立所述加载完的算法模型对应的模型句柄与所述标识信息的对应关系，以将所述加载完的算法模型绑定在所述标识信息对应的虚拟引擎上。
在本申请实施例中，采用了获取至少一个算法模型，其中，所述算法模型为人工智能算法模型；获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎；将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型；接收任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。通过本申请能够在边缘智能设备(可以为具有一颗物理引擎的边缘智能设备)上实现需要多算法模型支持的同时分析场景，从而能够在边缘智能设备上实现多算法模型同时运行，满足低成本、多算法模型融合分析的需求。
附图说明
图1是根据本申请实施例的单引擎单算法架构示意图；
图2是根据本申请实施例的单引擎多算法架构示意图；
图3是根据本申请实施例的一种算法模型运行处理方法的流程图；
图4是根据本申请实施例的系统整体架构的示意图；
图5是根据本申请实施例的边缘智能设备的结构示意图；
图6是根据本申请实施例的单物理引擎多算法的架构示意图；
图7是根据本申请实施例的软件配置示意图；
图8是根据本申请实施例的边缘智能设备内部绑定虚拟引擎示意图；
图9是根据本申请实施例的边缘智能设备任务和规则下发示意图；
图10是根据本申请实施例的实时视频分析任务流程图；
图11是根据本申请实施例的轮询视频分析任务流程图；
图12是根据本申请实施例的定时抓图分析任务流程图；
图13是根据本申请实施例的离线图片分析任务流程图；
图14跟根据本申请实施例的离线图片调度任务流程图。
具体实施方式
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。
需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图1是根据本申请实施例的单引擎单算法架构示意图，如图1所示，对于单物理引擎来说，如果在功能实现层没有进行引擎的虚拟，则在功能实现层只有一颗物理引擎，在UI层面(如AI配置 界面)也显示有一颗引擎可供使用，为该一颗引擎进行配置时，只能配置一个算法模型(即AI模型包1)。其中，功能实现层是用于处理用户指令的处理逻辑，比如说，用户下发指令到UI层面，UI层面将用户指令转发给功能实现层，功能实现层执行该用户指令以实现相关功能，本实施例中对此功能实现层的处理过程不做限制。其中，物理引擎是实际处理资源，是具有图像/视频等数据处理能力的物理单元，例如处理器等单元，如GPU(Graphics Processing Unit，图形处理器)等，对此不做限制，当然，该物理单元在进行数据处理时也是需要依赖必要的程序进行控制的。
在本实施例中，针对单物理引擎虚拟出了多个虚拟引擎，从而通过多个虚拟引擎运行多个算法模型，图2是根据本申请实施例的单引擎多算法架构示意图，如图2所示，在功能实现层的单颗物理引擎上虚拟出四个虚拟引擎1001、1002、1003和1004(当然也可以虚拟出其它数量的虚拟引擎)，从而在AI配置界面也可以看到这四个虚拟引擎，在配置算法模型时，可以为四个虚拟引擎分别配置不同的算法模型(即为虚拟引擎1001配置AI模型包1、为虚拟引擎1002配置AI模型包2、为虚拟引擎1003配置AI模型包3、为虚拟引擎1004配置AI模型包4)。其中，虚拟引擎为对物理引擎进行软件封装之后虚拟出的引擎，一个物理引擎可以虚拟出至少一个虚拟引擎，虚拟引擎是物理引擎的虚拟化单元，虚拟引擎可以单独加载一种算法模型进行智能分析。其中，软件封装后虚拟出引擎可以理解为：基于一颗物理引擎的数据处理能力，通过软件定义成多个能力使用软件接口，通过任何一个或多个软件接口均可以使用该一颗物理引擎的数据处理能力。该“软件接口”可相当于是一种“虚拟引擎”。其中，软件接口，一般可理解为：各种软件开发API等，或者可理解为人与软件之间的交互界面。例如在一个例子中，用户可以在交互界面上看到N个引擎(即虚拟引擎)的标识或者引擎的入口，这些被展示的虚拟引擎相当于是前述的“软件接口”，通过这N个虚拟引擎能使用某个物理引擎的数据处理能力。
图3是根据本申请实施例的一种算法模型运行处理方法的流程图，在图3中显示了为虚拟引擎配置算法模型的流程，如图3所示，该流程包括如下步骤：
步骤S302，获取至少一个算法模型，其中，所述算法模型为人工智能算法模型。
步骤S304，获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，获取的虚拟引擎的数量大于或等于获取的算法模型的数量。
步骤S306，将上述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型。
步骤S308，接收任务，并在虚拟引擎上基于绑定的算法模型执行该任务。
通过上述步骤，能够在边缘智能设备(可以为具有一颗物理引擎的边缘智能设备)上实现需要多算法模型支持的同时分析场景，从而能够在边缘智能设备上实现多算法模型同时运行，满足低成本、多算法模型融合分析的需求。
上述步骤中的算法模型可以从多个渠道获取，例如，可从AI开放平台获取。上述步骤可运行在边缘智能设备上，该智能设备可从AI放开平台获取到算法模型，还可以通过行业一体化应用平台、WEB页面端或者其它客户端来进行边缘智能设备的管理以及算法模型在虚拟引擎上的绑定等管理。
图4是根据本申请实施例的系统整体架构的示意图，如图4所示，边缘智能设备可以与行业一体化应用平台连接，并通过WEB页面登陆到行业一体化应用平台(简称为平台)，来对边缘智能设备(简称为设备)进行管理，并且行业一体化应用平台还可以从AI开放平台中获取算法模型(简 称为模型)，并对模型进行管理，建立模型资源库。行业一体化应用平台还可以对边缘智能设备的虚拟引擎进行管理，对虚拟引擎与模型之间的绑定关系进行处理；为了执行智能分析任务，行业一体化应用平台还可以向边缘智能设备下发任务和规则，边缘智能设备在执行任务之后，可以向行业一体化应用平台返回分析结果。即，获取虚拟引擎基于算法模型执行任务之后得到的分析结果；根据预先配置的告警条件确定是否需要对分析结果进行告警，如果需要，则进行告警。
在边缘智能设备侧，图5是根据本申请实施例的边缘智能设备的结构示意图，如图5所示，边缘智能设备通过协议接口同平台(即行业一体化应用平台)以及网络摄像头(IPC)等进行交互，边缘智能设备的智能引擎子系统用于进行虚拟引擎的管理。在智能引擎子系统和协议接口之间包括多个用于进行管理的模块，包括：模型包管理模块，用于模型包(即算法模型)的添加、更新和删除；引擎管理模块，用于模型包的绑定、更新和解绑；任务管理模块，用于任务开启和/或关闭等；规则管理模块，用于配置规则(该规则为每个算法模型使用的规则，根据该规则使得算法模型进行相应的计算)；IPC接入管理模块，用于管理接入的IPC以及获取来自IPC的视频流等；智能告警模块，用于进行告警上传。
基于图4和图5，通过平台(即行业一体化应用平台)、WEB可以接入边缘智能设备，对边缘智能设备进行算法模型的添加、更新、删除、引擎绑定/解绑模型、任务规则下发操作等。平台添加算法模型到设备存储模型分区，并在算法模型绑定虚拟引擎时，加载算法模型到虚拟引擎中进行运行。平台可以下发任务和规则，通知虚拟引擎进行相应的算法模型智能分析。IPC接入管理模块接入前端IPC，并实时获取视频流或者图片发送给智能引擎子系统，智能引擎子系统对虚拟引擎分析后的结果进行智能分析，并将告警上报平台。
在边缘智能设备侧，智能引擎子系统用于进行虚拟引擎的管理，对于算法模型来说，在使用配置上，是将一个算法模型绑定在一个物理引擎上，在本实施例中，通过将物理引擎进行软件内部封装、虚拟出来多个虚拟引擎，这种处理方式类似于CPU的多个逻辑运算核心。每次平台进行请求时，边缘智能设备将接收到的引擎信息，转换为内部对应的实际物理引擎对应的虚拟引擎单元，完成相应的动作。这样设计的好处是，可以直接快速对接平台，不用修改外部接口逻辑。
对于虚拟引擎和算法模型的绑定，可有多种实现方式，例如，在本实施例中提供了一种如下的绑定方式：在物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；获取加载完的算法模型待绑定的虚拟引擎的标识信息(简称为ID)；建立加载完的算法模型对应的模型句柄与该标识信息的对应关系，以将加载完的算法模型绑定在该标识信息对应的虚拟引擎上。
该绑定方式可以理解为支持同时加载多个算法模型。在物理引擎上每次加载完一个算法模型，就创建一个模型句柄，并且模型句柄和虚拟引擎ID进行关联，这样应用层针对单个虚拟引擎的算法模型规则配置、视频取流、图片发送等操作，都能通过虚拟引擎ID和算法模型的关联关系发送给对应的算法模型进行分析，从而实现单物理引擎虚拟化，进行多种算法模型混合运行。
通过上述绑定方式，在单物理引擎上模拟出多个虚拟引擎之后，通过模拟多个虚拟引擎的方式，可以加载多个算法模型，从而使得多个算法模型功能均可以在单颗物理引擎上运行。
图6是根据本申请实施例的单物理引擎多算法的架构示意图，如图6所示，该架构主要包括：AI功能界面(用于配置算法模型、进行算法模型与虚拟引擎的绑定等)、智能引擎子系统、功能实现层和虚拟转换层。虚拟转换层存在于AI功能界面、智能引擎子系统同边缘智能设备的功能实现层之间，其目的就是用来进行物理引擎和虚拟引擎之间的转换，以及协议和数据的交互。智能引擎子 系统针对单个物理引擎虚拟出4个虚拟引擎(也可以虚拟出其它数量的虚拟引擎)，可以同时加载运行不同的算法模型进行智能分析，每个虚拟引擎加载完算法模型后，会针对当前算法模型创建唯一的操作句柄，后续的模型规则配置、数据交互都是针对单个句柄进行操作，因此算法模型能够与虚拟引擎进行关联。没有进行引擎虚拟化之前，边缘智能设备就一颗物理引擎，同时只能运行一种算法模型，图6在不改变硬件架构的前提下，改变软件架构实现方式，可以快速进行单物理引擎多算法模型分析，扩展产品应用场景，降低产品价格，提升产品竞争力。
为了便于进行虚拟引擎和算法模型的绑定，可以提供绑定的软件界面，在该界面中可以通过拖动的方式来进行绑定，在该可选方式中，可以获取操作信号，如接收来自用户的操作信号，其中，操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；响应于操作信号，在第一对象与第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将第一对象对应的算法模型绑定在预定的第二对象对应的虚拟引擎上。图7是根据本申请实施例的软件配置示意图，图7示出了AI配置页面，通过页面可以在虚拟引擎上配置算法模型，在图7中，可以将模型库中的安全帽检测模型拖动到虚拟引擎(或者称为虚拟引擎单元)1001上，即将安全帽检测模型和虚拟引擎1001进行了绑定，将猪检测模型拖动到虚拟引擎1002上，即将猪检测模型和虚拟引擎1002进行了绑定。在图7中对外展示给用户的引擎都为虚拟引擎，可以通过拖拽的方式绑定单个算法模型到物理引擎对应的某个虚拟引擎上，例如，还可以将安全帽检测模型绑定到虚拟引擎1001上，将猪检测模型绑定到虚拟引擎1002上，将电瓶车检测模型绑定到虚拟引擎1003上，将消防通道被占用检测模型绑定到虚拟引擎1004上，这样单颗物理引擎就可以实现多种算法模型同时进行智能分析。
在图7中，还可对每个虚拟引擎对应的任务进行配置，例如，分析模式中可选择实时视频分析、轮询视频分析、定时图片分析、离线图片分析等；还可选择分析的通道，例如，选择IPCamera01或者IPCamera02通道。在图7还可显示每个虚拟引擎上的算法模型的运行状态是否正常。
图8是根据本申请实施例的边缘智能设备内部绑定虚拟引擎示意图，如图8所示，平台下发模型绑定引擎协议后，网络管理模块进行协议内容解析，通知引擎管理模块进行AI模型(即算法模型)和虚拟引擎单元(即虚拟引擎)绑定，引擎管理模块下发引擎加载模型指令给智能引擎子系统，加载算法模型到指定的空闲虚拟引擎。
在绑定之后还可对算法模型进行任务的配置，例如，在安全帽检测模型中，其任务可配置为实时视频分析，还可添加该实时视频分析的通道，在图7中配置了通道1和通道2进行实时视频分析。
图9是根据本申请实施例的边缘智能设备任务和规则下发示意图，如图9所示，平台下发任务和执行该任务所需要的参数(例如，该参数可以携带在规则中)给引擎，设备内部通过网络管理模块进行协议解析和分发到任务管理模块和规则管理模块，任务管理模块分别下发不同的任务到不同的虚拟引擎单元(即虚拟引擎)，规则管理模块分别下发不同的规则到不同的虚拟引擎单元，每个虚拟引擎单元开启分析任务，按照不同规则进行视频和图片智能分析，分析结果通过智能告警管理模块上报给平台。例如，在安全帽检测过程中，可以预先配置告警条件：即检测到未佩戴安全帽则进行告警；在进行分析之后，如果发现没有佩戴安全帽的情况，则确定需要进行告警。
在不同的虚拟引擎上可以加载不同的算法模型，各个算法模型之间还可以进行组合从而实现多种场景的分析，即，可以获取预先配置的预定顺序，其中，预定顺序用于指示在通过多个算法模型 配合运行以执行任务的情况下，配合运行的多个算法模型的运行顺序；基于该预定顺序，在配合运行的多个算法模型对应的多个虚拟引擎上依次运行多个算法模型以执行该任务。其中，在运行多个算法模型以执行任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入。
例如，虚拟引擎1检测到指定部位的图像，可以将指定部位的图像发送给虚拟引擎2继续进行图像比对，可以确认这是哪个用户；或者虚拟引擎1检测到用户的行为，可以将图像发送给虚拟引擎2继续进行图像比对，可以确认是哪个用户的行为；从单纯的AI检测、分类、混合模型分析扩展多种应用场景，满足客户的需求。
图7中可以配置的任务包括视频分析任务和图片分析任务，视频分析任务可以包括实时视频分析任务、轮询视频分析任务等，图片分析任务可以包括定时抓图分析任务、离线图片分析任务。下面对这几个任务分别进行说明。
图10是根据本申请实施例的实时视频分析任务流程图，如图10所示，解析平台下发的实时视频分析任务，如果虚拟引擎已经存在正在分析的任务，则该流程结束，如果虚拟引擎中未存在正在分析的任务，则保存任务相关信息(例如，任务ID、关联通道)，下发关联的多个通道的实时视频数据给虚拟引擎，接收虚拟引擎分析的结果并发送给智能告警管理模块。
图11是根据本申请实施例的轮询视频分析任务流程图，如图11所示，解析平台下发的轮询视频分析任务，如果虚拟引擎已经存在正在分析的任务，则该流程结束，如果虚拟引擎中未存在正在分析的任务，则保存任务相关信息(例如任务ID、轮询间隔、关联通道)，下发关联的1通道的实时视频数据给虚拟引擎，接收虚拟引擎分析的结果，并发送给智能告警管理模块，判断当前通道分析时间是否达到轮询间隔，如果达到轮询间隔，则切换任务关联的下一个新通道进行实时视频分析。
图12是根据本申请实施例的定时抓图分析任务流程图，如图12所示，解析平台下发的定时抓图任务，如果虚拟引擎已经存在正在分析的任务，则该流程结束，如果虚拟引擎中未存在正在分析的任务，则保存任务相关信息(例如，任务ID、轮询间隔、关联通道)，下发关联的1个通道的抓图数据给虚拟引擎，接收虚拟引擎的分析结果并发送给智能告警管理模块，判断当前通道分析时间是否达到轮询间隔，如果达到轮询间隔，则切换任务关联的下一个新通道进行图片分析。
需要说明的是，由于虚拟引擎是在物理引擎上虚拟得到的，因此，在多个虚拟引擎上执行任务的情况下，可以根据物理引擎的性能对多个虚拟引擎中的每个虚拟引擎的性能进行限制，其中，这里的性能用于指示引擎的处理能力的上限值，即，物理引擎的性能用于指示物理引擎的处理能力的上限值，虚拟引擎的性能用于指示虚拟引擎的处理能力的上限值，其中，多个虚拟引擎在运行过程中的性能之和不超过物理引擎的性能。例如，针对视频分析任务，物理引擎的性能为最大支持N路视频分析，N为正整数，或者，针对图片分析任务，物理引擎的性能为单位时间内最多分析M张图片，M为正整数；在这种请求下，所有的虚拟引擎一共支持的视频分析的路数不能超过N路，所有虚拟引擎单位时间内分析的图片总量不能超过M张。
此时，可选地，在多个虚拟引擎均进行视频分析的情况下，限制多个虚拟引擎开启的视频路数总和小于等于N；或者，在多个虚拟引擎均进行图片分析的情况下，限制多个虚拟引擎单位时间内分析图片数量的总和小于等于M。
下面以一个例子进行说明。在该例子中，单物理引擎支持最大4路实时视频分析性能。由于虚拟引擎是在物理引擎的基础上虚拟得到的，因此，在单颗物理引擎总计算资源为4路实时视频分析 的情况下，如果仅仅运行一个虚拟引擎，则该单个虚拟引擎总计算资源也是4路；如果运行4个虚拟引擎，则该物理引擎对应的4个虚拟引擎的计算资源总和也是4路，此时4个虚拟引擎的计算资源可以进行动态调整，例如，虚拟引擎1绑定算法模型1，开启1路实时视频分析后，虚拟引擎2绑定的算法模型2还可以开启3路实时视频分析；若虚拟引擎1开启4路，则其它虚拟引擎就不能再开启实时视频分析，因为单颗物理引擎所有虚拟引擎总计算资源已使用4路，当单颗物理引擎所有虚拟引擎总计算资源已使用4路，再开启通道实时视频分析时，会给出“引擎资源不足”提示。
对于平台下发给各虚拟引擎的任务，实时视频分析任务关联一路通道就占用1路实时视频分析性能；轮询视频分析任务因为是按照关联通道轮询进行单个通道实时视频分析，同时刻只会开启1路通道实时视频分析，所以轮询视频任务不论关联多少路通道都只占用1路实时视频分析性能；单物理引擎支持离线图片分析性能为16张图片/秒，并且和实时视频分析性能是互斥的，也就是开启4路实时视频分析，就不能再进行离线图片分析；以上性能限制都是对于单个物理引擎的，在新智能分析方案中，需要将单个物理引擎进行封装、虚拟出来4个虚拟引擎，单个虚拟引擎的各任务性能限制和物理引擎一样，并且物理引擎对应的4个虚拟引擎的总分析性能和单个物理引擎性能一样，例如，虚拟引擎1开启4路实时视频分析后，其它三个虚拟引擎就不能再开启实时视频分析或者图片分析，如果虚拟引擎1开启了1路实时视频分析，虚拟引擎2开启2路实时视频分析，虚拟引擎3还可以开启1路实时视频分析，4个虚拟引擎开启实时视频分析总路数4路。
还有一种可能，不同的虚拟引擎执行的任务是不一致的，例如，一部分虚拟引擎用于执行视频分析，另一部分虚拟引擎用于执行图片分析，此时，根据物理引擎的性能对多个虚拟引擎中的每个虚拟引擎的性能进行限制还包括：在多个虚拟引擎中一部分虚拟引擎进行视频分析并且另一部分虚拟引擎进行图片分析的情况下，一部分虚拟引擎开启N1路视频，将另一部分虚拟引擎(即用于进行图片分析的至少一个虚拟引擎)单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，其中，N1小于N，N1大于等0，N1为进行视频分析的虚拟引擎开启的视频路数。当N1等于0时，所有的虚拟引擎均用于图片分析。其中，等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量。可选的，等效图片处理量可以根据M和N计算得到，比如说，等效图片处理量可以为：M/q*N；比如说，对M/q*N向上取整，或者，对M/q*N向下取整；其中，q为已配置的控制系数，q的取值范围是1-M/N。比如说，q的取值可以根据经验配置，只要位于1-M/N即可，若q为1，则等效图片处理量可以为M/N，若q为M/N，则等效图片处理量可以为1，也就是说，等效图片处理量是位于1-M/N的数值。
在另一个可选方式中，为了保证虚拟引擎分析图片的能力不超过物理引擎能力的限制，还可以根据延时时间控制向另一部分虚拟引擎发送图片的速度，以将另一部分虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，其中，延时时间用于指示发送两张图片之间的时间间隔，延时时间为单位时间/[等效图片处理量*(N-N1)]。
在该可选方式中，可以通过一个图片调度任务(如离线图片调度任务，后续以离线图片调度任务为例进行说明)来控制向另一部分虚拟引擎发送图片的速度，配置离线图片调度任务，其中，离线图片调度任务用于依次向用于进行图片分析的至少一个虚拟引擎中的每个虚拟引擎发送图片；控制离线图片调度任务每次间隔预定时间(即延迟时间)发送一张图片，以将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，其中，预定时间用于指示发送两张图片之间的时间间隔，预定时间为：单位时间/[等效图片处理量*(N-N1)]。
下面结合一个例子进行说明。在该例子中，物理引擎只要有剩余实时视频分析性能和空闲的虚拟引擎，就允许虚拟引擎离线图片任务的开启，例如对于单个物理引擎，需要保证虚拟引擎1开启3路实时视频分析，其它三个虚拟引擎还可以开启离线图片任务。假设单颗物理引擎最大支持4路实时视频分析或者16张图片/秒分析性能，所以可以换算1路实时视频分析性能对应离线图片分析性能为4张图片/秒。因此，当单物理引擎的4个虚拟引擎开启有实时视频分析任务和离线图片分析任务时，需要动态调整离线图片发送给虚拟引擎的速度，以满足于物理引擎剩余图片分析性能。实际可遇到如下两种情况：
情况1：3个虚拟引擎进行实时视频分析，1个虚拟引擎进行离线图片分析，虚拟引擎1/2/3分别开启1路实时视频分析，就占用3路实时视频分析性能，对应物理引擎还剩余1路实时视频分析性能，对于图片分析性能就是4张图片/秒，所以虚拟引擎4还能开启离线图片分析任务，需要限制此图片分析任务的性能为4张图片/秒。
情况2：4个虚拟引擎均进行离线图片分析，4个虚拟引擎分别开启离线图片分析任务，也就是4个离线图片任务共同占用16张图片/秒分析性能。
针对上述两种情况，需要解决情况1中的一个离线图片任务情况下的调整虚拟引擎离线图片发送给虚拟引擎的速度，需要解决情况2中的四个离线图片任务情况下的各任务离线图片依次发送给每个虚拟引擎的速度，以及四个离线图片任务抢占资源，各虚拟引擎图片分析性能占用不均的情况。
图13是根据本申请实施例的离线图片分析任务流程图，图14是根据本申请实施例的离线图片调度任务流程图，如图13和图14所示，当开启多个离线图片任务共同占用单物理引擎图片分析性能时，需要增加一个离线图片调度任务，可以轮询调度每个虚拟引擎的离线图片任务发送图片给虚拟引擎，另外离线图片任务在发送离线图片时，动态调整延时时间uiDelayTime来控制发送图片的速度，以防止多个离线图片任务发送图片过多，导致虚拟引擎图片分析性能不足的问题，同时也防止多个离线图片任务延时等待后，开始发送图片时，一个离线图片任务一直抢占CPU资源开始发送图片，其它离线图片任务一直处于等待状态的情况。按照这种延时时间片轮询的方式后，运行情况如下：调度任务通知离线图片任务1发送图片，其它离线图片任务延时等待，离线图片任务1发送完图片后，开始延时等待，延时后，通知离线图片调度任务轮询调度下一个虚拟引擎的离线图片任务2，通知离线图片任务2发送图片,离线图片任务2抢占发送图片，其它离线图片任务延时等待，后续依次各任务轮询调度。
离线图片任务延时时间uiDelayTime的计算是根据单物理引擎剩余实时视频分析性能进行计算的，计算原理如下：
a)iUsedChanNum为统计的单物理引擎已占用的实时视频分析路数；
b)iResChanNum＝4-iUsedChanNum；为单物理引擎剩余实时视频分析路数；
c)iResAnalysePicNum＝iResChanNum*4；为单物理引擎剩余图片分析性能(1路实时视频分析性能对应图片分析性能4张/s)。
d)uiDelayTime＝1秒/iResAnalysePicNum；为单个物理引擎对应的多个离线图片任务发送图片的延时时间，单位秒，保证单个物理引擎剩余图片分析性能iResAnalysePicNum张/s。上述uiDlelayTime如果将秒作为单位则数值有可能比较小，在一个可选的实施中，可以将微秒(1秒等于1000000微秒)作为uiDelayTime的单位，此时uiDelayTime＝1000000(微秒)/iResAnalysePicNum，为单个物理 引擎对应的多个离线图片任务发送图片的延时时间，单位微秒。
通过上述实施例在不改变硬件架构的前提下，改变软件架构实现方式，通过单引擎多算法方案，将物理引擎进行封装、虚拟出来多个虚拟引擎，可以快速进行单物理引擎多算法分析，扩展产品应用场景，降低产品价格，提升产品竞争力。另外，各虚拟引擎加载不同的算法模型，可以针对同一路视频进行融合智能分析，例如，虚拟引擎1检测到指定部位，可以将指定部位图像发送给虚拟引擎2继续进行图像比对，可以确认这是哪个用户。从单纯的AI检测、分类、混合模型分析扩展多种应用场景，满足客户的需求。
在上述实施例中还通过时间片轮询以及动态调整图片发送速度的方式，保证了单物理引擎可以开启多个离线图片任务进行图片分析，满足当前引擎图片分析性能。
在本实施例中，提供一种边缘智能设备，包括存储器和处理器，存储器中存储有计算机程序，处理器被设置为运行计算机程序以执行以上实施例中的方法。
上述程序可以运行在处理器中，或者也可以存储在存储器中(或称为计算机可读介质)，计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
这些计算机程序也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤，对应与不同的步骤可以通过不同的模块来实现。
本实施例中就提供了这样的一种装置或系统。该装置被称为算法模型运行处理装置，包括：第一获取模块，用于获取至少一个算法模型，其中，所述算法模型为人工智能算法模型；第二获取模块，用于获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，获取的虚拟引擎的数量大于或等于获取的算法模型的数量；绑定模块，用于将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型；执行模块，用于接收任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。
该系统或者装置用于实现上述的实施例中的方法的功能，该系统或者装置中的每个模块与方法中的每个步骤相对应，已经在方法中进行过说明的，在此不再赘述。
例如还包括：告警模块，用于获取所述虚拟引擎基于所述算法模型执行所述任务之后得到的分析结果；根据预先配置的告警条件确定是否需要对所述分析结果进行告警，如果需要，则进行告警。
又例如，所述执行模块用于获取预先配置的预定顺序，其中，所述预定顺序用于指示在通过多个算法模型配合运行以执行所述任务的情况下，所述多个算法模型的运行顺序；基于所述预定顺序，在所述多个算法模型对应的多个虚拟引擎上依次运行所述多个算法模型以执行所述任务；其中，在 运行所述多个算法模型以执行所述任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入。
又例如，还包括：限制模块，用于在所述多个虚拟引擎上执行任务的情况下，根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，其中，所述物理引擎的性能用于指示所述物理引擎的处理能力的上限值，所述虚拟引擎的性能用于指示所述虚拟引擎的处理能力的上限值，其中，所述多个虚拟引擎在运行过程中的性能之和不超过所述物理引擎的性能。
可选地，所述任务包括视频分析任务和/或图片分析任务；其中，针对视频分析任务，所述物理引擎的性能为最大支持N路视频分析，所述N为正整数，针对图片分析任务，所述物理引擎的性能为单位时间内最多分析M张图片，所述M为正整数。
可选地，所述限制模块，用于在所述多个虚拟引擎均进行视频分析的情况下，限制所述多个虚拟引擎开启的视频路数总和小于等于N；或者，在所述多个虚拟引擎均进行图片分析的情况下，限制所述多个虚拟引擎单位时间内分析图片数量的总和小于等于M；或者，将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，其中，N1小于N，N1大于等于0，N1为进行视频分析的虚拟引擎开启的视频路数，所述等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量。
可选地，所述限制模块，用于配置图片调度任务，其中，所述图片调度任务用于依次向用于进行图片分析的至少一个虚拟引擎中的每个虚拟引擎发送图片；控制所述图片调度任务每次间隔预定时间发送一张图片，以将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：所述等效图片处理量*(N-N1)，其中，所述预定时间用于指示发送两张图片之间的时间间隔，所述预定时间为：所述单位时间/[所述等效图片处理量*(N-N1)]。
可选地，所述等效图片处理量为：M/q*N；其中，q为已配置的控制系数，q的取值范围是1-M/N。
又例如，所述绑定模块还用于获取操作信号，其中，所述操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，所述第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，所述第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；响应于所述操作信号，在所述第一对象与所述第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将所述第一对象对应的算法模型绑定在所述预定的第二对象对应的虚拟引擎上。
可选地，所述绑定模型用于在所述物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；获取所述加载完的算法模型待绑定的虚拟引擎的标识信息；建立所述加载完的算法模型对应的模型句柄与所述标识信息的对应关系，以将所述加载完的算法模型绑定在所述标识信息对应的虚拟引擎上。
通过上述实施例能够在边缘智能设备(可以为具有一颗物理引擎的边缘智能设备)上实现需要多算法模型支持的同时分析场景，从而能够在边缘智能设备上实现多算法模型同时运行，满足低成本、多算法模型融合分析的需求。
以上仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (19)

  1. 一种算法模型运行处理方法，其特征在于，包括：
    获取至少一个算法模型，其中，所述算法模型为人工智能算法模型；
    获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，获取的虚拟引擎的数量大于或等于获取的算法模型的数量；
    将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型；
    接收任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。
  2. 根据权利要求1所述的方法，其特征在于，所述在所述虚拟引擎上基于所述算法模型执行所述任务，包括：
    获取预先配置的预定顺序，所述预定顺序用于指示在通过多个算法模型配合运行以执行所述任务的情况下，所述多个算法模型的运行顺序；
    基于所述预定顺序，在所述多个算法模型对应的多个虚拟引擎上依次运行所述多个算法模型以执行所述任务；其中，在运行所述多个算法模型以执行所述任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入。
  3. 根据权利要求1所述的方法，其特征在于，还包括：
    在所述多个虚拟引擎上执行任务的情况下，根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，其中，所述物理引擎的性能用于指示所述物理引擎的处理能力的上限值，所述虚拟引擎的性能用于指示所述虚拟引擎的处理能力的上限值，其中，所述多个虚拟引擎在运行过程中的性能之和不超过所述物理引擎的性能。
  4. 根据权利要求3所述的方法，其特征在于，所述任务包括视频分析任务和/或图片分析任务；其中，针对视频分析任务，所述物理引擎的性能为最大支持N路视频分析，所述N为正整数，针对图片分析任务，所述物理引擎的性能为单位时间内最多分析M张图片，所述M为正整数。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，包括：
    在所述多个虚拟引擎均进行视频分析的情况下，限制所述多个虚拟引擎开启的视频路数总和小于等于N；或者，
    在所述多个虚拟引擎均进行图片分析的情况下，限制所述多个虚拟引擎单位时间内分析图片数量的总和小于等于M；或者，
    将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，其中，N1小于N，N1大于等于0，N1为进行视频分析的虚拟引擎开启的视频路数，所述等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量。
  6. 根据权利要求5所述的方法，其特征在于，所述将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，包括：
    配置图片调度任务，其中，所述图片调度任务用于依次向用于进行图片分析的至少一个虚拟引擎中的每个虚拟引擎发送图片；
    控制所述图片调度任务每次间隔预定时间发送一张图片，以将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：所述等效图片处理量*(N-N1)，所述预定时间用于指示发送两张图片之间的时间间隔，所述预定时间为：所述单位时间/[所述等效图片处理量*(N-N1)]。
  7. 根据权利要求5或6所述的方法，其特征在于，所述等效图片处理量为：M/q*N，q为已配 置的控制系数，q的取值范围是1-M/N。
  8. 根据权利要求1至6中任一项所述的方法，其特征在于，在所述虚拟引擎上基于所述算法模型执行所述任务之后，所述方法还包括：
    获取所述虚拟引擎基于所述算法模型执行所述任务之后得到的分析结果；
    根据预先配置的告警条件确定是否需要对所述分析结果进行告警，如果需要，则进行告警。
  9. 根据权利要求1至6中任一项所述的方法，其特征在于，将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，包括：
    获取操作信号，其中，所述操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，所述第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，所述第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；
    响应于所述操作信号，在所述第一对象与所述第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将所述第一对象对应的算法模型绑定在所述预定的第二对象对应的虚拟引擎上。
  10. 根据权利要求1至6中任一项所述的方法，其特征在于，将所述至少一个算法模型中的每个算法模型分别绑定运行在一个虚拟引擎上，包括：
    在所述物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；
    获取所述加载完的算法模型待绑定的虚拟引擎的标识信息；
    建立所述加载完的算法模型对应的模型句柄与所述标识信息的对应关系，以将所述加载完的算法模型绑定在所述标识信息对应的虚拟引擎上。
  11. 一种算法模型运行处理装置，其特征在于，包括：
    第一获取模块，用于获取至少一个算法模型，其中，所述算法模型为人工智能算法模型；
    第二获取模块，用于获取多个虚拟引擎，其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，获取的虚拟引擎的数量大于或等于获取的算法模型的数量；
    绑定模块，用于将所述至少一个算法模型中的每个算法模型分别运行绑定在一个虚拟引擎上，其中，一个虚拟引擎用于绑定一个算法模型；
    执行模块，用于接收任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。
  12. 根据权利要求11所述的装置，其特征在于，
    所述执行模块用于获取预先配置的预定顺序，所述预定顺序用于指示在通过多个算法模型配合运行以执行所述任务的情况下，所述多个算法模型的运行顺序；基于所述预定顺序，在所述多个算法模型对应的多个虚拟引擎上依次运行所述多个算法模型以执行所述任务；在运行所述多个算法模型以执行所述任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入；和/或，
    还包括：限制模块，用于在所述多个虚拟引擎上执行任务的情况下，根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，所述物理引擎的性能用于指示所述物理引擎的处理能力的上限值，所述虚拟引擎的性能用于指示所述虚拟引擎的处理能力的上限值，所述多个虚拟引擎在运行过程中的性能之和不超过所述物理引擎的性能；和/或，
    所述任务包括视频分析任务和/或图片分析任务；其中，针对视频分析任务，所述物理引擎的性能为最大支持N路视频分析，所述N为正整数，针对图片分析任务，所述物理引擎的性能为单位时间内最多分析M张图片，所述M为正整数；和/或，
    所述限制模块，用于在所述多个虚拟引擎均进行视频分析的情况下，限制所述多个虚拟引擎开启的视频路数总和小于等于N；或者，在所述多个虚拟引擎均进行图片分析的情况下，限制所述多个虚拟引擎单位时间内分析图片数量的总和小于等于M；或者，将用于进行图片分析的至少一个虚 拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，N1小于N，N1大于等于0，N1为进行视频分析的虚拟引擎开启的视频路数，所述等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量；和/或，
    所述限制模块，用于配置图片调度任务，其中，所述图片调度任务用于依次向用于进行图片分析的至少一个虚拟引擎中的每个虚拟引擎发送图片；控制所述图片调度任务每次间隔预定时间发送一张图片，以将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：所述等效图片处理量*(N-N1)，所述预定时间用于指示发送两张图片之间的时间间隔，所述预定时间为：所述单位时间/[所述等效图片处理量*(N-N1)]；和/或，
    所述等效图片处理量为：M/q*N，q为已配置的控制系数，q的取值范围是1-M/N；和/或，
    还包括：告警模块，用于获取所述虚拟引擎基于所述算法模型执行所述任务之后得到的分析结果；根据预先配置的告警条件确定是否需要对所述分析结果进行告警，如果需要，则进行告警；和/或，
    所述绑定模块还用于获取操作信号，所述操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，所述第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，所述第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；响应于所述操作信号，在所述第一对象与所述第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将所述第一对象对应的算法模型绑定在所述预定的第二对象对应的虚拟引擎上；和/或，
    所述绑定模型用于在所述物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；获取所述加载完的算法模型待绑定的虚拟引擎的标识信息；建立所述加载完的算法模型对应的模型句柄与所述标识信息的对应关系，以将所述加载完的算法模型绑定在所述标识信息对应的虚拟引擎上。
  13. 一种边缘智能设备，其特征在于，包括存储器和处理器，其中，所述存储器用于存储程序，所述处理器用于运行所述程序，所述程序用于执行权利要求1至10中任一项所述的方法。
  14. 一种算法模型运行处理系统，其特征在于，所述算法模型运行处理系统包括边缘智能设备和应用平台，用户通过WEB页面登陆到所述应用平台来对所述边缘智能设备进行管理；其中：
    所述应用平台，用于从AI开放平台中获取至少一个算法模型，将所述至少一个算法模型下发给所述边缘智能设备，所述算法模型为人工智能算法模型；
    所述应用平台，用于对所述边缘智能设备的虚拟引擎进行管理，对虚拟引擎与算法模型之间的绑定关系进行处理；其中，所述虚拟引擎为对一个物理引擎进行软件封装之后虚拟出的引擎，且获取的虚拟引擎的数量大于或等于获取的算法模型的数量；虚拟引擎与算法模型之间的绑定关系是：将每个算法模型分别绑定运行在一个虚拟引擎上，一个虚拟引擎用于绑定一个算法模型；
    所述应用平台，用于向所述边缘智能设备下发任务；
    所述边缘智能设备，用于接收所述应用平台下发的所述任务，并在所述虚拟引擎上基于所述算法模型执行所述任务。
  15. 根据权利要求14所述的系统，其特征在于，
    所述边缘智能设备，用于在执行所述任务之后，获取虚拟引擎基于算法模型执行任务之后得到的分析结果，向所述应用平台返回分析结果，
    所述应用平台，用于根据预先配置的告警条件确定是否需要对所述分析结果进行告警，如果需要，则进行告警。
  16. 根据权利要求14所述的系统，其特征在于，所述边缘智能设备包括：
    智能引擎子系统，用于进行虚拟引擎的管理；
    模型包管理模块，用于算法模型的添加、更新和删除；
    引擎管理模块，用于算法模型的绑定、更新和解绑；
    任务管理模块，用于任务开启，和/或，任务关闭；
    规则管理模块，用于配置规则，所述规则为算法模型使用的规则，根据所述规则使得算法模型进行与所述规则匹配的计算；
    IPC接入管理模块，用于管理接入的IPC以及获取来自IPC的视频流；
    智能告警模块，用于进行告警上传。
  17. 根据权利要求14所述的系统，其特征在于，所述边缘智能设备在所述虚拟引擎上基于所述算法模型执行所述任务时具体用于：
    获取预先配置的预定顺序，所述预定顺序用于指示在通过多个算法模型配合运行以执行所述任务的情况下，所述多个算法模型的运行顺序；基于所述预定顺序，在所述多个算法模型对应的多个虚拟引擎上依次运行所述多个算法模型以执行所述任务；在运行所述多个算法模型以执行所述任务的过程中，在先运行的算法模型的输出作为在后运行的算法模型的输入。
  18. 根据权利要求14所述的系统，其特征在于，
    所述边缘智能设备，还用于在所述多个虚拟引擎上执行任务的情况下，根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制，所述物理引擎的性能用于指示所述物理引擎的处理能力的上限值，所述虚拟引擎的性能用于指示所述虚拟引擎的处理能力的上限值，所述多个虚拟引擎在运行过程中的性能之和不超过所述物理引擎的性能；
    其中，所述任务包括视频分析任务和/或图片分析任务，针对视频分析任务，所述物理引擎的性能为最大支持N路视频分析，所述N为正整数，针对图片分析任务，所述物理引擎的性能为单位时间内最多分析M张图片，所述M为正整数；
    其中，所述边缘智能设备根据所述物理引擎的性能对所述多个虚拟引擎中的每个虚拟引擎的性能进行限制时具体用于：在所述多个虚拟引擎均进行视频分析的情况下，限制所述多个虚拟引擎开启的视频路数总和小于等于N；或者，在所述多个虚拟引擎均进行图片分析的情况下，限制所述多个虚拟引擎单位时间内分析图片数量的总和小于等于M；或者，将用于进行图片分析的至少一个虚拟引擎单位时间内最多分析的图片数量限制为：等效图片处理量*(N-N1)，N1小于N，N1大于等于0，N1为进行视频分析的虚拟引擎开启的视频路数，所述等效图片处理量为处理一路视频分析的能力用来进行图片分析时能够分析的图片数量。
  19. 根据权利要求14所述的系统，其特征在于，所述边缘智能设备，还用于：
    获取操作信号，所述操作信号用于将代表算法模型的第一对象从第一显示区域向第二显示区域进行拖动，所述第一显示区域显示至少一个第一对象，每个第一对象对应一个算法模型，所述第二显示区域显示至少一个第二对象，每个第二对象对应一个虚拟引擎；响应于所述操作信号，在所述第一对象与所述第二显示区域中的预定的第二对象的重合度超过阈值的情况下，将所述第一对象对应的算法模型绑定在所述预定的第二对象对应的虚拟引擎上；或者，在所述物理引擎上每加载完一个算法模型之后，创建与加载完的算法模型对应的模型句柄；获取所述加载完的算法模型待绑定的虚拟引擎的标识信息；建立所述加载完的算法模型对应的模型句柄与所述标识信息的对应关系，以将所述加载完的算法模型绑定在所述标识信息对应的虚拟引擎上。
PCT/CN2023/093032 2022-05-09 2023-05-09 一种算法模型运行处理方法、装置及边缘智能设备 Ceased WO2023217146A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP23802915.1A EP4524736A4 (en) 2022-05-09 2023-05-09 METHOD AND APPARATUS FOR PROCESSING AND EXECUTIVE ALGORITHM MODEL, AND INTELLIGENT PERIPHERAL DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210501689.0A CN117076037A (zh) 2022-05-09 2022-05-09 一种算法模型运行处理方法、装置及边缘智能设备
CN202210501689.0 2022-05-09

Publications (2)

Publication Number Publication Date
WO2023217146A1 true WO2023217146A1 (zh) 2023-11-16
WO2023217146A9 WO2023217146A9 (zh) 2024-09-12

Family

ID=88710235

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/093032 Ceased WO2023217146A1 (zh) 2022-05-09 2023-05-09 一种算法模型运行处理方法、装置及边缘智能设备

Country Status (3)

Country Link
EP (1) EP4524736A4 (zh)
CN (1) CN117076037A (zh)
WO (1) WO2023217146A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118506026A (zh) * 2024-07-16 2024-08-16 杭州海康威视数字技术股份有限公司 一种巡检设备以及巡检方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150355933A1 (en) * 2012-08-31 2015-12-10 Oregon State Board Of Higher Education On Behalf Of Portland State University System and methods for generating and managing a virtual device
CN107783818A (zh) * 2017-10-13 2018-03-09 北京百度网讯科技有限公司 深度学习任务处理方法、装置、设备及存储介质
CN107861820A (zh) * 2017-12-08 2018-03-30 郑州云海信息技术有限公司 一种虚拟机资源分配方法及系统
CN111448550A (zh) * 2017-11-22 2020-07-24 亚马逊技术有限公司 网络可访问的机器学习模型训练和托管系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200193221A1 (en) * 2018-12-17 2020-06-18 At&T Intellectual Property I, L.P. Systems, Methods, and Computer-Readable Storage Media for Designing, Creating, and Deploying Composite Machine Learning Applications in Cloud Environments
EP4141663A4 (en) * 2020-07-17 2023-05-31 Huawei Technologies Co., Ltd. DATA PROCESSING METHOD AND APPARATUS, AND INTELLIGENT VEHICLE

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150355933A1 (en) * 2012-08-31 2015-12-10 Oregon State Board Of Higher Education On Behalf Of Portland State University System and methods for generating and managing a virtual device
CN107783818A (zh) * 2017-10-13 2018-03-09 北京百度网讯科技有限公司 深度学习任务处理方法、装置、设备及存储介质
CN111448550A (zh) * 2017-11-22 2020-07-24 亚马逊技术有限公司 网络可访问的机器学习模型训练和托管系统
CN107861820A (zh) * 2017-12-08 2018-03-30 郑州云海信息技术有限公司 一种虚拟机资源分配方法及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4524736A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118506026A (zh) * 2024-07-16 2024-08-16 杭州海康威视数字技术股份有限公司 一种巡检设备以及巡检方法

Also Published As

Publication number Publication date
EP4524736A4 (en) 2025-12-03
CN117076037A (zh) 2023-11-17
EP4524736A1 (en) 2025-03-19
WO2023217146A9 (zh) 2024-09-12

Similar Documents

Publication Publication Date Title
US11765051B2 (en) Instantiation of cloud-hosted functions based on call to function graph
US20160314008A1 (en) Method for implementing gpu virtualization and related apparatus, and system
CN113301102A (zh) 资源调度方法、设备、边缘云网络、程序产品及存储介质
CN106406987A (zh) 一种集群中的任务执行方法及装置
CN103699428A (zh) 一种虚拟网卡中断亲和性绑定的方法和计算机设备
CN111338769B (zh) 一种数据处理方法、装置及计算机可读存储介质
WO2023221844A1 (zh) 资源调度方法、设备及存储介质
CN105335211A (zh) 一种基于Xen虚拟化集群的FPGA加速器调度系统及方法
CN114327846A (zh) 集群的扩容方法、装置、电子设备及计算机可读存储介质
WO2023217146A1 (zh) 一种算法模型运行处理方法、装置及边缘智能设备
CN120011093A (zh) 面向加速器的多任务处理方法以及相关装置
CN116204321A (zh) 模型获取、模型部署方法、装置、设备和介质
US9847908B2 (en) Resource management method and management server
CN108829516B (zh) 一种图形处理器资源虚拟化调度方法
CN114466227B (zh) 一种视频分析方法、装置、电子设备及存储介质
CN119336448B (zh) 业务处理方法、处理装置、设备、存储介质和程序产品
CN114691873A (zh) 自动驾驶日志数据的语义化处理方法、装置及存储介质
CN114401232B (zh) 一种数据传输方法、装置、电子设备及存储介质
CN116510312A (zh) 一种云游戏多开实现方法、装置、设备及存储介质
CN104702663B (zh) 云处理装置和方法
CN115168028A (zh) 任务处理方法及装置
CN116149804A (zh) 任务调度方法、装置和服务器
CN113032098A (zh) 一种虚拟机调度方法、装置、设备及可读存储介质
US20230195546A1 (en) Message Management Method and Apparatus, and Serverless System
CN120104248A (zh) 虚拟机的资源调度方法、装置、系统、车辆及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23802915

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023802915

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023802915

Country of ref document: EP

Effective date: 20241209

WWE Wipo information: entry into national phase

Ref document number: 11202407855Y

Country of ref document: SG