WO2020101108A1 - Plateforme de modèle d'intelligence artificielle et procédé de fonctionnement de plateforme de modèle d'intelligence artificielle - Google Patents

Plateforme de modèle d'intelligence artificielle et procédé de fonctionnement de plateforme de modèle d'intelligence artificielle Download PDF

Info

Publication number
WO2020101108A1
WO2020101108A1 PCT/KR2018/015476 KR2018015476W WO2020101108A1 WO 2020101108 A1 WO2020101108 A1 WO 2020101108A1 KR 2018015476 W KR2018015476 W KR 2018015476W WO 2020101108 A1 WO2020101108 A1 WO 2020101108A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature information
performance
normalization
model
combination
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2018/015476
Other languages
English (en)
Korean (ko)
Inventor
송중석
권태웅
최상수
최윤수
이윤수
박진학
신익수
이혁로
박학수
박진형
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Korea Institute of Science and Technology Information KISTI
Original Assignee
Korea Institute of Science and Technology Information KISTI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Korea Institute of Science and Technology Information KISTI filed Critical Korea Institute of Science and Technology Information KISTI
Publication of WO2020101108A1 publication Critical patent/WO2020101108A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/2205—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/26—Functional testing
    • G06F11/263—Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass

Definitions

  • the present invention relates to a technology for generating artificial intelligence models for security control.
  • the Science and Technology Cyber Safety Center provides real-time security control services for public research institutes based on the TMS.
  • the real-time security control service is provided as a service structure that provides analysis and response support by security control personnel based on security events detected and collected by the intrusion threat management system (TMS).
  • TMS intrusion threat management system
  • an artificial intelligence model platform capable of generating an artificial intelligence model for security control.
  • the present invention is intended to provide an artificial intelligence model platform that enables an ordinary user who is not familiar with security control technology to generate an optimal artificial intelligence model for security control.
  • the object to be reached in the present invention is to provide a method (technology) for implementing an AI model platform that enables generation of an AI model for security control.
  • An artificial intelligence model platform includes: a data collection module for collecting security events to be used as learning / test data by specific search conditions from source security data; A feature extraction module that extracts preset feature information for the collected security event; A normalization module that performs preset normalization on the extracted feature information of the security event; A data output module that extracts learning data or test data from the security event where the specific information normalization is completed according to a given condition; And a model generation module that applies an artificial intelligence algorithm to the learning data to generate an artificial intelligence model for security control.
  • test data it may further include a performance management module for testing the accuracy of the artificial intelligence model.
  • a UI module that provides a user interface (UI) for setting at least one of specific search conditions of the data collection module, feature information of the feature extraction module, normalization method of the normalization module, and conditions of the data output module. It may further include.
  • UI user interface
  • the data collection module sequentially stores the number of collections exceeding the maximum number of collections in a queue and sequentially proceeds.
  • the security event can be collected only for data prior to the occurrence point of the collection case in the source security data.
  • the feature extraction module may recommend a change to the feature information to increase the accuracy of the AI model based on the result of the accuracy test of the performance management module.
  • the normalization module may recommend changing the normalization method for the normalization to increase the accuracy of the artificial intelligence model.
  • the feature information recommendation apparatus is a model performance confirmation unit that checks model performance with respect to an AI model generated based on learning predetermined feature information among all feature information that can be set when generating an AI model. ;
  • a combination performance checking unit configured to set a plurality of feature information combinations from the entire feature information to check the performance of the artificial intelligence model generated based on learning for each of the plurality of feature information combinations;
  • a recommendation unit recommending a specific feature information combination having a higher performance than the model performance confirmed by the model performance checking unit among performances of the plurality of feature information combinations.
  • the combination of the plurality of feature information is a combination in which at least one of the remaining specific information excluding the preset feature information from the entire feature information is sequentially added to the preset feature information, and the specific feature information combination is: Among the plurality of feature information combinations, it may be the top N having higher performance than the model performance.
  • the predetermined specific information is the entire feature information
  • the combination performance checking unit is the maximum performance among the performances of the artificial intelligence model generated based on learning for each single feature information in the whole feature information.
  • Single feature information performance comparison process to check if the maximum performance is higher than the model performance, the single feature information of the maximum performance is reset to the feature information, and the feature information is preset to the feature information from the whole feature information.
  • An apparatus for recommending a normalization method includes: an attribute confirmation unit that checks an attribute of feature information used for learning when generating an artificial intelligence model; Determining unit for determining a normalization method according to the attribute of the feature information, from among all the settable normalization method; And a recommendation unit recommending the determined normalization method.
  • the determination unit determines a first normalization method according to the whole number pattern of the feature information, and the feature
  • a second normalization method for expressing as a non-zero characteristic value is determined only at a designated location for each category of feature information in a vector defined as the total number of categories of the feature information, and the feature information
  • the attribute of is a combination of a number and a category
  • the second normalization scheme and the first normalization scheme may be determined.
  • the first normalization method includes a standard score normalization method, a mean normalization normalization method, and a feature scaling normalization method according to a predefined priority
  • the determining unit includes standard deviations for all numeric patterns of feature information and Based on whether there is an upper / lower limit of the normalization scaling range, a normalization scheme having the highest priority applicable among the first normalization schemes may be determined.
  • the determining unit has the highest priority applicable among the normalization method and the normalizing method of the feature scaling for the attribute attribute type field in the feature information.
  • a high normalization method is determined, and for the field of the attribute whose number of attributes is attribute in the feature information, a normalization method having the highest priority applicable among the mean normalization normalization method and the feature scaling normalization method is determined, and the attribute attribute is the ratio attribute in the feature information.
  • the normalization method is not determined and excluded from the normalization target for the field of, or the standard score normalization method is determined, and whether the attribute exists in the feature information, the normalization scheme is not determined and excluded from the normalization target. have.
  • An artificial intelligence model platform operating method includes: a data collection step of collecting security events to be used as learning / test data according to specific search conditions from source security data; A feature extraction step of extracting predetermined feature information for the collected security event; A normalization step of performing preset normalization on the extracted feature information of the security event; A data output step of extracting training data or test data according to a given condition from the security event in which the specific information normalization is completed; And a model generation step of applying an artificial intelligence algorithm to the learning data to generate an artificial intelligence model for security control.
  • a performance management step of testing the accuracy of the artificial intelligence model using the test data may be further included.
  • a user interface for setting at least one of a specific search condition of the data collection step, feature information of the feature extraction module, normalization method of the normalization module, and condition of the data output module. It may further include.
  • the number of collection cases exceeding the maximum number of collection cases is stored in a queue and sequentially performed.
  • the security event can be collected only for data prior to the occurrence point of the collection case in the source security data.
  • it may further include the step of recommending a change to the feature information to increase the accuracy of the artificial intelligence model.
  • the normalization step it is possible to recommend changing the normalization method for the normalization to increase the accuracy of the artificial intelligence model.
  • a computer program is a model that checks model performance with respect to an artificial intelligence model generated based on learning preset feature information among all feature information that can be set when generating an artificial intelligence model in combination with hardware.
  • Performance check step A combination performance checking step of setting a combination of a plurality of feature information from the whole feature information, and confirming the performance of the artificial intelligence model generated based on learning for each of the plurality of feature information combinations; And a performance of recommending a specific feature information combination having a higher performance than the model performance confirmed by the model performance checking unit among performances of the plurality of feature information combinations.
  • the combination of the plurality of feature information is a combination in which at least one of the remaining specific information excluding the preset feature information from the entire feature information is sequentially added to the preset feature information, and the specific feature information combination is: Among the plurality of feature information combinations, it may be the top N having higher performance than the model performance.
  • the predetermined specific information is the entire feature information
  • the combination performance checking step is the maximum performance among the performances of the artificial intelligence model generated based on learning for each single feature information in the whole feature information.
  • Single feature information performance comparison process to check whether it is higher, if the maximum performance is higher than the model performance, the single feature information of the maximum performance is reset to the feature information, and the predetermined feature is set in the whole feature information in the feature information.
  • the combination setting process of setting the combination of the plurality of feature information by sequentially adding one by one of the specific information except the information, each of the feature information combinations having higher performance than the model performance of the re-set feature information among the plurality of feature information combinations Resetting as the feature information, so that the combination setting process is repeatedly performed for each re-set feature information, there is no feature information combination having a higher performance than the model performance among the multiple feature information combinations.
  • a process of delivering the previous feature information as the specific feature information combination to the recommender may be performed.
  • the predetermined specific information is the entire feature information
  • the combination performance checking step is the maximum performance among the performances of the artificial intelligence model generated based on learning for each single feature information in the whole feature information.
  • Single feature information performance comparison process to check if it is higher, if the maximum performance is not higher than the model performance
  • combination setting process to set the combination of the plurality of feature information excluding one specific information from the feature information, the plurality of Among the feature information combinations, each of the feature information combinations having a performance higher than the model performance is reset as feature information, and a reset process is performed so that the combination setting process is repeatedly performed for each re-set feature information. If there is no feature information combination having a higher performance than the model performance, a process of delivering the immediately preceding feature information as the specific feature information combination to the recommendation unit may be performed.
  • a computer program comprises: an attribute checking step in combination with hardware to check the attribute of feature information used for learning when creating an artificial intelligence model; A determining step of determining a normalization method according to the attribute of the feature information from among all the settable normalization methods; And a recommendation step of recommending the determined normalization method.
  • the determining step if the same normalization method is applied to all the feature information fields, if the attribute of the feature information is a numeric attribute, the first normalization method according to the whole number pattern of the feature information is determined, and the When the attribute of the feature information is a category attribute, a second normalization method for expressing as a non-zero characteristic value is determined only at a designated location for each category of the feature information in a vector defined as the total number of categories of the feature information, and the feature When the attribute of the information is a combination attribute of a number and a category, the second normalization method and the first normalization method may be determined.
  • the first normalization method includes a standard score normalization method, a mean normalization normalization method, and a feature scaling normalization method according to a predefined priority
  • the determining step includes standard deviation of the entire numeric pattern of feature information.
  • the attribute has the highest priority applicable to the normalization method of the feature normalization method and the feature scaling normalization method for the attribute type attribute field in the feature information.
  • Determines a normalization method having a high value determines a normalization method having the highest priority applicable among a means normalization method and a feature scaling normalization method for the field of the attribute number attribute in the feature information, and the attribute ratio in the feature information
  • the normalization method is not determined and excluded from the normalization target, or the standard score normalization method is determined, and whether the attribute is present in the feature information or not. Can be.
  • an artificial intelligence model platform capable of generating an artificial intelligence model for security control is implemented, in particular, a feature directly related to the performance of the artificial intelligence model.
  • the optimal artificial intelligence model suitable for the purpose and requirements for security control can be flexibly and variously generated and applied, the quality improvement of the security control service can be maximized, and large scale It can be expected to have the effect of supporting the construction of an AI-based infringement response system to efficiently analyze the signs of cyber attacks and anomalies.
  • FIG. 1 is a conceptual diagram showing an AI model platform according to an embodiment of the present invention.
  • FIG. 2 is a configuration diagram showing the configuration of the AI model platform according to an embodiment of the present invention.
  • FIG. 3 is a block diagram showing the configuration of a feature information recommendation device according to an embodiment of the present invention.
  • FIG. 4 is a configuration diagram showing the configuration of a normalization method recommendation apparatus according to an embodiment of the present invention.
  • FIG. 5 is a flowchart illustrating an artificial intelligence model platform operating method according to an embodiment of the present invention.
  • FIG. 6 is a flowchart illustrating a method of operating a feature information recommendation device according to an embodiment of the present invention.
  • FIG. 7 is a flowchart illustrating an operation method of a normalization method recommendation apparatus according to an embodiment of the present invention.
  • the real-time security control service provided by the Science and Technology Cyber Safety Center is based on security events detected and collected by the intrusion threat management system (TMS), and provides rule-based analysis and response support by security control personnel. It has a service structure that is made.
  • TMS intrusion threat management system
  • an artificial intelligence model platform capable of generating an artificial intelligence model for security control.
  • the present invention is intended to provide an artificial intelligence model platform that enables an ordinary user who is not familiar with security control technology to generate an optimal artificial intelligence model for security control.
  • the AI model platform of the present invention is based on various data collected and processed in a collection function and a collection function for collecting and processing various data necessary for generating an AI model for security control.
  • the artificial intelligence function that creates an intelligent model and manages the performance and history associated with it, and the management responsible for various settings and user management related to the collection / artificial intelligence function based on the user interface (UI) provided to system administrators and general users. It can be divided into functions.
  • the artificial intelligence model platform of the present invention includes a search engine that periodically collects the newly generated source security data from the big data integrated storage storage, and loads various data from the collection function into the search engine to search the data. Can be used as storage.
  • various modules belonging to the collection function may operate based on a search engine (data storage).
  • the artificial intelligence model platform 100 of the present invention includes a data collection module 110, a feature extraction module 120, a normalization module 130, a data output module 140, and a model generation module 150.
  • the AI model platform 100 of the present invention may further include a performance management module 160 and a UI module 170.
  • All or at least part of the configuration of the AI model platform 100 may be implemented in the form of a hardware module or a software module, or a combination of a hardware module and a software module.
  • the software module may be understood as, for example, instructions executed by a processor that controls operations within the AI model platform 100, and these instructions are in a form mounted in a memory in the AI model platform 100. Will have.
  • the artificial intelligence model platform 100 realizes the technology proposed in the present invention through the above-described configuration, that is, the technology capable of generating an optimal artificial intelligence model for security control. , Hereinafter, each configuration in the AI model platform 100 for realizing this will be described in more detail.
  • the UI module 170 has at least one of a specific search condition of the data collection module 110, feature information of the feature extraction module 120, normalization method of the normalization module 130, and condition of the data output module 140. Provides a UI (User Interface) for setting.
  • UI User Interface
  • the UI module 170 according to the operation of a system administrator or a general user (hereinafter referred to as a user) to create an AI model for security control in the AI model platform 100 of the present invention, data Provides a UI for setting at least one of a specific search condition of the collection module 110, feature information of the feature extraction module 120, a normalization method of the normalization module 130, and a condition of the data output module 140.
  • the UI module 170 based on the provided UI, various settings related to the collection / artificial intelligence function, specifically the specific search condition of the data collection module 110 for the artificial intelligence model to be generated later, feature extraction module
  • the feature information of 120, the normalization method of the normalization module 130, the conditions of the data output module 140, etc. are stored / managed in the user information / setting information storage.
  • the data collection module 110 collects security events to be used as learning / testing data based on specific search conditions, that is, predetermined search conditions previously set by the user, from the source security data.
  • a date (or period) to be used as learning / test data number of cases, IP, detection pattern name, detection pattern type, and the like may be set.
  • the detection pattern name means a representative name of security logs detected by the intrusion threat management system (TMS)
  • the detection pattern type means a group of detection patterns having similar detection pattern characteristics (property, type).
  • the detection pattern type can be divided into six types: worm virus damage, data corruption and leakage, waypoint abuse, homepage alteration, service rejection attack damage, and simple intrusion attempt.
  • the data collection module 110 may collect security events belonging to a set date (or period) from the original security data.
  • the data collection module 110 may collect the security events of the number (for example, 500,000) set at the specified time from the original security data.
  • the data collection module 110 may collect a security event in which the IP set from the source security data matches the source IP or destination IP.
  • a combination of date (or period), number of cases, IP, detection pattern name, and detection pattern type may be set.
  • the data collection module 110 may collect security events according to a combination of date (or period), number, IP, detection pattern name, detection pattern type, etc. set from the source security data.
  • the data collection module 110 in collecting security events from the original security data, as described above, may limit the maximum number of simultaneous executions to reduce the load on the system.
  • the total number of security event collection cases belonging to the set date (or period) is 1000,000, and the maximum number of concurrent collections It can be assumed that 500,000 cases.
  • the data collection module 110 determines that the total number of collections this time exceeds the maximum number of collections that can be performed simultaneously, and stores the collections that exceed the maximum number of collections in the queue in a queue After that, you can proceed sequentially.
  • the data collection module 110 collects / progresses the maximum number of collections of 500,000 according to the time sequence among the total number of collections of 1000,000, but queues for 500,000 collections exceeding the maximum number of collections of 500,000 After storing in (queue), it can be collected / progressed sequentially.
  • the data collection module 110 collects security events only for the data prior to the occurrence of the collection case from the source security data in the case of 500,000 collection cases that proceed after being stored in the queue.
  • the artificial intelligence model platform 100 of the present invention was previously mentioned that it includes a search engine that periodically collects the newly generated source security data from the big data integrated storage storage.
  • the data collection module 110 may collect security data from source security data in a search engine (data store).
  • the big data integrated storage storage is a storage utilized not only in the AI model platform 100 of the present invention, but also in other systems, when a large amount of data (security events) is collected from the big data integrated storage storage, the big data integrated storage storage Loads can also affect other systems.
  • the data collection module 110 does not collect security events directly from the big data integrated storage storage, but periodically only the source security data newly generated from the big data integrated storage storage. Since security events are collected based on the collected search engine, it is possible to avoid the big data integrated storage storage load problem described above.
  • the feature extraction module 120 extracts pre-set feature information for the security event collected by the data collection module 110, that is, pre-set feature information by the user.
  • the feature extraction module 120 is responsible for performing a feature information extraction process for security events collected by the data collection module 110.
  • the feature information of each security event extracted by the feature extraction module 120 will be used for machine learning (eg, deep learning) when creating an artificial intelligence model described later.
  • the user can set a single feature as feature information and set a composite feature.
  • the single feature means features that can be extracted from one security event.
  • detection time For example, detection time, source IP, source port, destination IP, destination port, protocol, security event name, security event type, number of attacks, attack direction, packet size, automatic analysis result, dynamic analysis result, organization number, jumbo Whether it is a payload, a payload using a word2vec conversion method, or the like may belong to a single feature.
  • the payload conversion method using Word2Vec is a method of converting a word into a vector, and is a method of determining a vector of a corresponding word through a relationship between adjacent words.
  • words can be distinguished on a space-by-space basis, but payload is very difficult to distinguish in semantic units and contains a lot of special characters, so pre-processing is required to apply word2vec.
  • the composite feature means a feature that can be extracted by using aggregate and statistical techniques between various security events.
  • a security event group is formed based on a period or the number of cases, and one feature (eg, a result of an operation) that can be extracted through intra-group operations (eg, aggregation, statistical technique, etc.) is a complex feature. Can belong to.
  • one feature eg, a result of an operation
  • intra-group operations eg, aggregation, statistical technique, etc.
  • a security event group as shown in Table 1 below is formed based on a period (8.22 to 9.3).
  • the feature extraction module 120 may extract pre-set feature information (single feature and / or composite feature) with respect to the security event collected by the data collection module 110.
  • the normalization module 130 performs predetermined normalization on the extracted feature information of the security event.
  • Normalization refers to the process of consistently matching the range of values of the extracted features. If field A has a range of 50 to 100 and field B has a range of 0 to 100, the meaning is different because even the same 50 is a value measured by different scales. Therefore, it is necessary to adjust the values of different fields to a common scale to have a certain meaning and this is called normalization.
  • the normalization module 130 performs normalization on the extracted feature information of the security event to adjust the values of different fields to a common scale according to a preset normalization method to have a certain meaning.
  • the preset normalization scheme means a normalization scheme preset by the user.
  • the following three normalization methods are provided to allow a user to pre-set.
  • Equation 1 means Feature scaling [a, b] normalization
  • Equation 2 means Mean normalization [-1,1] normalization
  • Equation 3 means Standard score normalization.
  • the normalization module 130 performs normalization on the extracted feature information of the security event according to the normalization method preset by the user among the three normalization methods described above.
  • the data output module 140 extracts training data or test data from a security event in which the normalization of specific information is completed, based on a given condition, that is, a preset (given) condition by the user.
  • the data output module 140 outputs the security event for which the specific information is normalized, to a screen or a file according to a user's desired value, order, format, learning / test data ratio, and file division method.
  • the output training data or test data is managed through database or file storage for each date and user so that they can be used immediately when creating an artificial intelligence model.
  • the model generation module 150 applies an artificial intelligence algorithm to learning data managed in the output / file storage in the data output module 140 to generate an artificial intelligence model for security control.
  • the model generation module 150 may apply an artificial intelligence algorithm to the learning data, and generate an artificial intelligence model for security control, for example, an artificial intelligence model of a function required by a user.
  • the model generation module 150 may generate an artificial intelligence detection model for detecting whether a security event is malicious or not according to a user request, and an artificial intelligence classification model for classifying spying / falsification of a security event. You can also create an artificial intelligence detection model for detecting whether a security event is malicious or not according to a user request, and an artificial intelligence classification model for classifying spying / falsification of a security event. You can also create an artificial intelligence detection model for detecting whether a security event is malicious or not according to a user request, and an artificial intelligence classification model for classifying spying / falsification of a security event. You can also create an artificial intelligence detection model for detecting whether a security event is malicious or not according to a user request, and an artificial intelligence classification model for classifying spying / falsification of a security event. You can also create an artificial intelligence detection model for detecting whether a security event is malicious or not according to a user request, and an artificial intelligence classification model for classifying spying / falsification of a security event. You
  • the model generation module 150 based on the learning data managed in the output / file storage in the data output module 140, to an artificial intelligence algorithm, such as a machine learning (eg, Deep Learning) algorithm previously selected by the user. Accordingly, an artificial intelligence model for security control can be generated.
  • an artificial intelligence algorithm such as a machine learning (eg, Deep Learning) algorithm previously selected by the user.
  • the model generation module 150 uses a learning function (Loss function) representing a deviation between a predicted result and an actual result through a model in a machine learning technique based on Backward Propagation calculation. Accordingly, an artificial intelligence model in which the deviation of the loss function is zero based on the learning data can be generated.
  • a learning function Loss function
  • the artificial intelligence model platform 100 of the present invention by providing a platform environment that enables to create an artificial intelligence model for security control based on the UI without any programming, the security control technology Even unfamiliar general users can create artificial intelligence models suitable for their purposes and requirements for security control.
  • the performance management module 160 utilizes test data managed in the output / file storage in the data output module 140, of the generated artificial intelligence model. Test accuracy.
  • the performance management module 160 is for managing the artificial intelligence model generated by the model generation module 150, 'who' 'when' 'some data' 'some field' 'some sampling method' 'some normalization method 'Records and manages performance information on the system (file storage), such as' what model' the artificial intelligence model was created for, and how much performance (correct answer rate) the created artificial intelligence model has.
  • the performance management module 160 can compare conditions and performance for model generation at a glance based on such performance information management, so that it is easy to grasp the correlation between conditions and performance.
  • the accuracy (performance) test of the artificial intelligence model generated in the platform environment of the present invention is provided by providing a platform environment that allows an ordinary user who is not familiar with security control technology to generate an artificial intelligence model. It may be necessary.
  • the performance management module 160 utilizes test data (security events that know the actual result of detection and detection of malicious or false positives) managed in the output / file storage in the data output module 140, Test the accuracy of the AI model created above.
  • the performance management module 160 uses the test data to test the artificial intelligence model generated above, and the accuracy of the model (performance) ) That is, it can be output as a test result.
  • the feature extraction module 120 recommends a change to the feature information (Feature) to increase the accuracy of the above-described artificial intelligence model, based on the accuracy test result of the performance management module 160 Can be.
  • the normalization module 130 may recommend changing the normalization method for normalization to increase the accuracy of the artificial intelligence model.
  • FIG. 3 is a block diagram of a feature information recommendation apparatus according to an embodiment of the present invention.
  • the feature information recommendation device 200 of the present invention includes a model performance confirmation unit 210, a combination performance confirmation unit 220, and a recommendation unit 230.
  • All or at least a part of the configuration of the feature information recommendation device 200 may be implemented in the form of a hardware module or a software module, or a combination of a hardware module and a software module.
  • the software module may be understood as, for example, an instruction executed by a processor that controls an operation within the feature information recommendation apparatus 200, and these instructions may include a form mounted in the memory in the feature information recommendation apparatus 200. Will have.
  • the feature information recommendation apparatus 200 After all, the feature information recommendation apparatus 200 according to an embodiment of the present invention, through the above-described configuration, the technology proposed in the present invention, that is, the technology for recommending feature information (Feature) change to increase the accuracy of the artificial intelligence model
  • the technology proposed in the present invention that is, the technology for recommending feature information (Feature) change to increase the accuracy of the artificial intelligence model
  • the model performance checking unit 210 checks model performance with respect to the artificial intelligence model generated based on learning the preset feature information among all the feature information that can be set when the artificial intelligence model is generated.
  • the model performance checking unit 210 checks the performance (accuracy) of the artificial intelligence model generated based on learning feature information set by the user.
  • the artificial intelligence model platform learning / generated feature information (hereinafter, user set feature information) set by the user in the AI model platform 100 of the present invention.
  • the model performance checking unit 210 checks the model performance with respect to the artificial intelligence model generated by learning the user set feature information in the artificial intelligence model platform 100 as described above.
  • model performance checking unit 210 with respect to the artificial intelligence model, test data output from the artificial intelligence model platform 100 (especially the data output module 140) of the present invention (sense / false classification and malignantness) Model performance (accuracy) can be tested / verified by utilizing the security event (which knows the actual result of detection).
  • the model performance checking unit 210 targets the artificial intelligence model generated in the artificial intelligence model platform 100 (especially the data output module 140) of the present invention, and tests the artificial intelligence model using test data,
  • the matching ratio between the predicted result value and the known actual result value can be output as the model's accuracy (performance), that is, the test result.
  • the combination performance checking unit 220 sets a plurality of feature information combinations from the whole feature information, and checks the performance of the artificial intelligence model generated based on learning for each of the multiple feature information combinations.
  • the combination performance checking unit 220 sets a variety of feature information combinations by setting various feature information combinations in addition to user-set feature information learned at the time of creation of the AI model, from all feature information that can be set when the AI model is generated. You can check the performance of the artificial intelligence model generated based on learning.
  • the recommendation unit 230 among a plurality of feature information combination-specific performances confirmed by the combination performance confirmation unit 220, model performances confirmed by the model performance confirmation unit 210, that is, performances of the artificial intelligence model generated based on the user setting this time Higher performance specific feature information combinations can be recommended.
  • the combination of the plurality of feature information set by the combination performance checking unit 220 is specified in the user set feature information learned when the artificial intelligence model is generated, except for the user set feature information in the whole feature information. It may be a combination of at least one piece of information sequentially added.
  • the combination performance checking unit 220 includes user-set feature information (a, b, c, d, e) out of all feature information (n) in user-set feature information (a, b, c, d, e, f).
  • a plurality of feature information combinations may be set by sequentially adding at least one of the specific information other than, f).
  • the combination performance checking unit 220, the user-set feature information (a, b, c, d, e, f) set by the user, the user-set feature information (a, out of all the feature information n) Among the remaining specific information except b, c, d, e, f), 1 ⁇ (nk) feature information can be sequentially added to set a plurality of feature information combinations as follows.
  • the combination performance checking unit 220 performs the performance of the artificial intelligence model generated based on learning for each of a plurality of feature information combinations as described above, 82%, 80%, ... 88%, ... 85%. Can be confirmed.
  • top N are the number that can be specified / changed by the system administrator or user.
  • the combination performance checking unit 220 includes user-set feature information (a) among all feature information (n) in the user-set feature information (a, b, c, d, e, f) set by the user.
  • a plurality of characteristic information combinations can be set as follows, by sequentially adding the remaining specific information one by one except for (b, c, d, e, f).
  • the combination performance checking unit 220 may check the performance, 82%, 80%, ... 90% of the artificial intelligence model generated based on learning for each combination of feature information as described above.
  • top N are the number that can be specified / changed by the system administrator or user.
  • the performance of a single feature information comparison process may be performed to check the performance of the artificial intelligence model and to determine whether the maximum performance (Max (m 1 )) of the performance of each single feature information is higher than the model performance (m 26 ).
  • Max (m 1 )) is high
  • the maximum performance single feature information (c) is reset to feature information, and the remaining specific information except feature information (c) from the whole feature information (n) in the feature information (c)
  • By adding one by one it is possible to perform a combination setting process of setting a plurality of combinations of feature information.
  • the combination performance checking unit 220 may check the performance for each combination of the plurality of feature information as described above.
  • the combination performance checking unit 220 resets and resets each combination of feature information having a performance higher than the model performance (m 1 ) of the feature information (c) that is reset among a plurality of feature information combinations, and resets the feature information. For each feature information, a reset process may be performed so that the combination setting process is repeatedly performed.
  • the combination performance checking unit 220 deletes feature information combinations having a performance equal to or lower than the model performance (m 1 ) of the feature information (c) among a plurality of feature information combinations, and performs model performance of the feature information (c).
  • m 1 Only the combination of feature information with higher performance is left as follows, and each of them is reset to feature information to reset the combination setting process repeatedly for each feature information reset as shown in Table 2 below. You can carry out the process.
  • the combination performance checking unit 220 repeats the above-described combination setting process and resetting process, and among a plurality of feature information combinations, there is a feature information combination having a higher performance than the artificial intelligence model generated based on the previous feature information. If not, the process of selecting the previous feature information as a specific feature information combination and passing it to the recommender 230 is performed.
  • the recommendation unit 230 has higher performance than the performance of the artificial intelligence model generated by using the preset feature information from the feature information transmitted from the combination performance checking unit 220 among performances of a plurality of feature information combinations. It can be recommended as a combination of specific feature information.
  • the combination performance checking unit 220 may check the performance for each combination of the plurality of feature information as described above.
  • the combination performance checking unit 220 resets each combination of feature information having higher performance than the model performance (m 26 ) among a plurality of feature information combinations as feature information, and the combination setting process is performed for each re-set feature information.
  • a reset process may be performed to be repeatedly performed.
  • the combination performance checking unit 220 deletes feature information combinations having a performance lower than or equal to model performance (m 26 ) among a plurality of feature information combinations, and combinations of feature information having higher performance than model performance (m 26 ). It is possible to perform a reset process in which the combination setting process is repeatedly performed for each of the characteristic information that is reset by resetting each of them as characteristic information, leaving only the following as follows.
  • the combination performance checking unit 220 repeats the above-described combination setting process and resetting process, and among a plurality of feature information combinations, there is a feature information combination having a higher performance than the artificial intelligence model generated based on the previous feature information. If not, the process of selecting the previous feature information as a specific feature information combination and passing it to the recommender 230 is performed.
  • the recommendation unit 230 has higher performance than the performance of the artificial intelligence model generated by using the preset feature information from the feature information transmitted from the combination performance checking unit 220 among performances of a plurality of feature information combinations. It can be recommended as a combination of specific feature information.
  • the artificial intelligence model platform 100 recommending an optimal feature having optimal performance (accuracy) to a user generating an artificial intelligence model for security control based on UI. / By making it applicable, even an average user who is not familiar with security control technology can create an optimal AI model for security control.
  • FIG. 4 illustrates a configuration of a normalization method recommendation apparatus according to an embodiment of the present invention.
  • the normalization method recommendation apparatus 300 of the present invention includes an attribute confirmation unit 310, a determination unit 320, and a recommendation unit 330.
  • the whole or at least part of the configuration of the normalization method recommendation device 300 may be implemented in the form of a hardware module or a software module, or a combination of a hardware module and a software module.
  • the software module may be understood as, for example, an instruction executed by a processor that controls an operation within the normalization method recommendation apparatus 300, and these instructions may include a form mounted in the memory in the normalization method recommendation apparatus 300. Will have.
  • the normalization method recommendation apparatus 300 realizes the technique proposed in the present invention, that is, the technique of recommending the normalization method change to increase the accuracy of the artificial intelligence model through the above-described configuration,
  • each configuration in the normalization method recommendation apparatus 300 for realizing this will be described in more detail.
  • the attribute checking unit 310 checks the attribute of feature information used for learning when the artificial intelligence model is generated.
  • the feature information used for learning when the AI model is generated may be feature information that is directly set by a user based on a UI among all the feature information that can be set when the AI model is generated, or a specific feature that is recommended among all feature information.
  • the feature information combination may be feature information applied / set.
  • the attribute of the characteristic information can be largely divided into a number attribute and a category attribute.
  • the attribute checking unit 310 may check whether the attribute of the feature information (direct setting or recommendation application) used for learning when the artificial intelligence model is generated is a numeric attribute, a category attribute, or a number and category combination attribute. .
  • the determination unit 320 determines a normalization method according to the attribute of the feature information checked by the attribute confirmation unit 310 among all the settable normalization methods.
  • the determination unit 320 determines whether the same normalization method is applied to all the feature information fields or the normalization method for each field in the whole feature information field. It can be distinguished first whether it is applied.
  • the determination unit 320 may classify that the same normalization method is applied to the entire feature information field.
  • the determining unit 320 determines the first normalization method according to the entire numeric pattern of the feature information when the feature information attribute is a numeric attribute, and when the feature information attribute is a category attribute, the feature information is the whole of the feature information. If a second normalization method for expressing as a non-zero characteristic value is determined only at a location designated for each category of feature information in a vector defined by the number of categories, and if the attribute of the feature information is a number and category combination attribute, the second The normalization scheme and the first normalization scheme can be determined.
  • the first normalization method includes a standard score normalization method, a mean normalization normalization method, and a feature scaling normalization method according to a predefined priority (see Equations 1, 2, and 3).
  • the determination unit 320 classifies the attribute of the feature information as a numeric attribute when only numeric data exists in the entire feature information field, and determines the first normalization method according to the whole numeric pattern of the feature information.
  • the determining unit 320 determines the standard score normalization method, the mean normalization normalization method, and the feature scaling normalization method according to the priority among the first normalization methods, but the standard deviation and normalization of the entire numeric pattern of the feature information Based on the existence of the upper / lower limit of the scaling range, the normalization method having the highest priority applicable among the first normalization methods may be determined.
  • the determination unit 320 classifies the attribute of the feature information as a category attribute, and in this case, the feature information in a vector defined as the total number of categories of the feature information.
  • a second normalization method that expresses a non-zero characteristic value only at a location designated by each category may be determined.
  • the determination unit 320 has a non-zero characteristic value (eg, 1) in a location designated for each category of feature information in a vector defined as the total number of categories of the feature information. ) To determine the second normalization method _One Hot Encoding.
  • the second normalization method _One Hot Encoding briefly, assumes that feature information has a category attribute of fruit, and that apples, pears, and persimmons (expressed as a three-dimensional vector because there are three kinds of fruits) are the total number of categories.
  • each feature information having apple, pear, and persimmon as data may be expressed as follows according to the second normalization method _One Hot Encoding.
  • the determination unit 320 classifies the attribute of the feature information as a numeric and category combination attribute, and in this case, the second normalization method and the first normalization method described above. Can decide.
  • the determination unit 320 first applies the second normalization method _One Hot Encoding described above to the data of the category attribute in the feature information, and then the whole of the feature information.
  • the second normalization method and the first normalization method may be determined in order to determine the highest priority normalization method applicable among the first normalization methods based on the existence of the upper and lower limit of the standard deviation and the normalization scaling range for the numeric pattern. .
  • the feature information is a composite feature (one feature that can be extracted using aggregation and statistical techniques between multiple security events)
  • the feature information is applied to the normalization method for each field in the entire feature information field. Can be distinguished.
  • the determination unit 320 may determine a normalization method having the highest priority that can be applied among a means normalization method and a feature scaling normalization method for a field of attribute type attribute in the feature information.
  • the determination unit 320 may determine a normalization method having the highest priority that is applicable among a means normalization method and a feature scaling normalization method for a field of a number attribute whose attribute is in the feature information.
  • the determining unit 320 may determine whether to normalize the normalization method for the attribute attribute field in the attribute information and exclude it from the normalization target, or determine the standard score normalization method.
  • the determination unit 320 may determine that the normalization method is not determined and excluded from the normalization target for the field of the attribute presence or absence (for example, presence / absence of an operation result value) in the feature information.
  • the recommendation unit 330 recommends the normalization method determined by the determination unit 320.
  • the recommendation / applying the optimal normalization method with optimal performance (accuracy) to the user generating the AI model for security control based on the UI By doing so, even an average user who is not familiar with security control technology can create an optimal artificial intelligence model for security control.
  • an artificial intelligence model platform that enables the creation of an artificial intelligence model for security control is implemented, but in particular, feature information and normalization methods directly related to the performance of the artificial intelligence model are optimally recommended /
  • an artificial intelligence model platform that allows an ordinary user who is not familiar with security control technology to generate an optimal AI model for security control.
  • the optimal artificial intelligence model suitable for the purpose and requirements for security control can be flexibly and variously generated and applied, the quality improvement of the security control service can be maximized, and large scale It can be expected to have the effect of supporting the construction of an AI-based infringement response system to efficiently analyze the signs of cyber attacks and anomalies.
  • the artificial intelligence model platform 100 of the present invention periodically collects the newly generated source security data from the big data integrated storage storage (S10).
  • the artificial intelligence model platform 100 of the present invention collects / artificial intelligence functions through a UI according to the operation of a system administrator or a general user (hereinafter referred to as a user) who wants to create an artificial intelligence model for security control. It receives various related settings and stores / manages them as setting information (S20).
  • the artificial intelligence model platform 100 of the present invention collects security events to be used as learning / test data based on a specific search condition, that is, a specific search condition previously set by the user from the original security data (S30).
  • the AI model platform 100 of the present invention extracts pre-set feature information for the security event collected in step S30, that is, pre-set feature information by the user (S40).
  • the AI model platform 100 of the present invention performs normalization preset by the user on the extracted feature information of the security event (S50).
  • the above three normalization methods are provided to allow a user to pre-set.
  • the artificial intelligence model platform 100 of the present invention since the normalization scheme set by the user may not be optimal, it may recommend the optimal normalization scheme to increase the accuracy of the artificial intelligence model (S50).
  • the AI model platform 100 of the present invention extracts training data or test data from a security event in which normalization of specific information is completed, based on a given condition, that is, a predetermined (given) condition by the user (S60).
  • the artificial intelligence model platform 100 of the present invention to output a security event that has been normalized specific information, the screen or file according to the value, order, format, learning / test data ratio, file division method, etc. do.
  • the artificial intelligence model platform 100 of the present invention applies an artificial intelligence algorithm to the learning data to generate an artificial intelligence model for security control (S70).
  • the artificial intelligence model platform 100 of the present invention may apply an artificial intelligence algorithm to learning data to generate an artificial intelligence model for security control, for example, an artificial intelligence model of a function required by a user.
  • the artificial intelligence model platform 100 of the present invention may generate an artificial intelligence detection model for detecting whether a security event is malicious or not according to a user's request, and to classify the spying / falsification of the security event. You can also create an artificial intelligence classification model.
  • the AI model platform 100 of the present invention based on the learning data managed in the output / file storage in step S60, AI algorithms, such as machine learning (eg, Deep Learning) algorithms previously selected by the user Accordingly, an artificial intelligence model for security control can be generated.
  • AI algorithms such as machine learning (eg, Deep Learning) algorithms previously selected by the user Accordingly, an artificial intelligence model for security control can be generated.
  • the artificial intelligence model platform 100 of the present invention is a learning loss function (Loss) indicating a deviation between a predicted result and an actual result through a model in a machine learning technique based on computation of backward propagation. function), it is possible to generate an artificial intelligence model in which the deviation of the loss function is zero based on the learning data.
  • Loss learning loss function
  • the artificial intelligence model platform 100 of the present invention utilizes test data (security events that know the actual result of detection and detection of malicious or false positives) managed in the output / file storage in step S60, The accuracy of the artificial intelligence model generated above is tested (S80).
  • the artificial intelligence model platform 100 of the present invention uses the test data to test the artificial intelligence model generated above, and model the matching ratio between the predicted result value and the known actual result value through the model.
  • the accuracy (performance) of ie can be output as a test result.
  • the AI model platform 100 of the present invention 'who', 'when', 'some data', 'some field', 'some sampling method', 'some normalization method' and 'some model' using the 'AI model' Performance information such as whether or not the generated artificial intelligence model has a certain performance (correct answer rate) can be recorded and managed in a system (file storage).
  • the artificial intelligence model platform 100 of the present invention based on such performance information management, can compare conditions and performance for model generation at a glance so that it is easy to grasp the correlation between conditions and performance.
  • the AI model platform 100 of the present invention may recommend a change to the feature information (Feature) to increase the accuracy of the generated AI model based on the accuracy test result of step S80 ( S90, S100).
  • the AI model platform 100 of the present invention has a combination of other feature information capable of improving the accuracy of the AI model, compared to the feature information (hereinafter, user set feature information) used for learning when the AI model is generated. If there is (S90 Yes), this is the recommended method (S100).
  • FIG. 6 is referred to as an operation method of the feature information recommendation device 200 for convenience of description. I will explain.
  • the performance (accuracy) of the artificial intelligence model generated based on the feature information learning set by the user is checked (S110).
  • the artificial intelligence model platform learning / generated feature information (hereinafter, user set feature information) set by the user in the AI model platform 100 of the present invention.
  • the operation method of the feature information recommendation apparatus 200 checks model performance with respect to the artificial intelligence model generated by learning user-set feature information in the artificial intelligence model platform 100 as described above (S110). .
  • the operation method of the feature information recommendation device 200 according to the present invention for the artificial intelligence model, the test data output from the artificial intelligence model platform (100, especially the data output module 140) of the present invention (Model performance (accuracy) can be tested / confirmed by utilizing the security event (which knows the actual result of the detection of false positives / false positives and malicious detection).
  • the operation method of the feature information recommendation device 200 is based on the artificial intelligence model generated by the artificial intelligence model platform 100 (especially the data output module 140) of the present invention, and utilizes test data.
  • the artificial intelligence model By testing the artificial intelligence model, it is possible to output the accuracy (performance) of the model, that is, the test result, as the ratio of the predicted result through the model and the known actual result.
  • the operation method of the feature information recommendation apparatus 200 sets a plurality of feature information combinations from the whole feature information, and checks the performance of the AI model generated based on learning for each combination of the feature information ( S120, S130).
  • the operation method of the feature information recommendation apparatus 200 sets a combination of various feature information in addition to user-set feature information learned at the time of creation of the AI model, from all feature information that can be set when the AI model is generated Thus, it is possible to check the performance of the artificial intelligence model generated based on learning for each combination of feature information.
  • user-set feature information e.g., a, b, c, d, e
  • the verified artificial intelligence model performance m k is 85%.
  • the operation method of the feature information recommendation apparatus 200 includes user-set feature information (a, out of all feature information (n) in user-set feature information (a, b, c, d, e, f).
  • a plurality of feature information combinations may be set by sequentially adding at least one of the specific information other than b, c, d, e, f) (S120).
  • the operation method of the feature information recommendation apparatus 200 includes the user set feature information (a, b, c, d, e, f) set by the user, among the whole feature information (n)
  • One to (nk) feature information among the remaining specific information except for the user-set feature information (a, b, c, d, e, f) can be sequentially added to set a plurality of feature information combinations as follows.
  • the operation method of the feature information recommendation apparatus 200 is the performance of the artificial intelligence model generated based on learning for each combination of multiple feature information as described above, 82%, 80%, ... 88% , ... 85% can be confirmed (S130).
  • the top N (for example, 4) having performance may be selected / recommended as a specific feature information combination (S140 Yes, S150).
  • the operating method of the feature information recommendation apparatus 200 includes user-specified feature information (a, b, c, d, e, f), and overall feature information (n) Among the user-specific feature information (a, b, c, d, e, f), the remaining specific information may be sequentially added one by one to set a plurality of feature information combinations as follows (S120).
  • the operation method of the feature information recommendation device 200 is the performance of the artificial intelligence model generated based on learning for each combination of a plurality of feature information, as described above, 82%, 80%, ... 90% It can be confirmed (S130).
  • the top N for example, three
  • the top N may be selected / recommended as a specific feature information combination (S140 Yes, S150).
  • the artificial intelligence model platform 100 recommending an optimal feature having optimal performance (accuracy) to a user generating an artificial intelligence model for security control based on UI. / By making it applicable, even an average user who is not familiar with security control technology can create an optimal AI model for security control.
  • the normalization method recommendation apparatus 300 of the present invention when generating an artificial intelligence model, the property of feature information used for learning is checked (S200).
  • the feature information used for learning when the AI model is generated may be feature information that is directly set by a user based on a UI among all the feature information that can be set when the AI model is generated, or a specific feature that is recommended among all feature information.
  • the feature information combination may be feature information applied / set.
  • the attribute of the characteristic information can be largely divided into a number attribute and a category attribute.
  • the operation method of the normalization method recommendation apparatus 300 is whether the attribute of feature information (direct setting or recommendation application) used for learning when generating an artificial intelligence model is a numeric attribute or a category attribute or a number and It can be checked whether the category is a combination attribute (S200).
  • the operation method of the normalization method recommendation apparatus 300 determines a normalization method according to the attribute of the feature information identified in step S200 among all the settable normalization methods.
  • the method of operation of the apparatus 300 for recommending a normalization method according to the present invention prior to determining the normalization method according to the attribute of the feature information, is the same normalization method applied to all of the feature information fields or this feature In the entire information field, whether a normalization method is applied for each field may be distinguished first (S210).
  • the same normalization method is applied to the entire feature information field. It can be classified as being (S210 Yes).
  • the first normalization method when the attribute of the feature information is a numeric attribute, the first normalization method according to the entire number pattern of the feature information is determined, and the attribute of the feature information is In the case of a category attribute, a second normalization method for expressing as a non-zero characteristic value is determined only at a designated location for each category of feature information in a vector defined as the total number of categories of the feature information, and the attribute of the feature information is a number and In the case of a category combination attribute, the second normalization method and the first normalization method may be determined (S220).
  • the first normalization method includes a standard score normalization method, a mean normalization normalization method, and a feature scaling normalization method according to a predefined priority (see Equations 1, 2, and 3).
  • the operation method of the normalization method recommendation apparatus 300 is to classify the attribute of the feature information as a numeric attribute when only numeric data exists in the entire feature information field, and in this case, the method according to the whole number pattern of the feature information 1 Determine the normalization method.
  • the operation method of the normalization method recommendation apparatus 300 according to the present invention is determined in the order of the standard score normalization method, the mean normalization normalization method, and the feature scaling normalization method according to the priority of the first normalization method. Based on the existence of the standard deviation and the upper / lower limit of the normalized scaling range for the entire numeric pattern, the highest normalized normalization method applicable among the first normalization methods may be determined.
  • the attribute of the feature information is classified as a category attribute, and in this case, the total number of categories of the feature information
  • a second normalization method _One Hot Encoding that expresses a non-zero characteristic value (eg, 1) only at a designated location for each category of feature information in a defined vector can be determined.
  • the attribute of the feature information is divided into a number and category combination attribute.
  • the second normalization method and the first normalization method may be determined.
  • the second normalization method of the above-mentioned second normalization method_One Hot After Encoding is applied, the second normalization method is used to determine the highest normalization method applicable among the first normalization methods based on whether there is a standard deviation and a normalization scaling range upper / lower limit for the entire numeric pattern of the feature information. And a first normalization method.
  • the feature information is a composite feature (a single feature that can be extracted by using statistical and statistical methods between multiple security events)
  • the entire feature information of this time It can be classified as being applied to the normalization method for each field in the field (S210 No).
  • the operation method of the normalization method recommendation apparatus 300 includes a normalization method having the highest priority, which is applicable among the means of the normalization method and the feature scaling normalization method for the field of attribute type attribute in the feature information. It can be determined (S230).
  • the operation method of the normalization method recommendation apparatus 300 determines the normalization method having the highest priority among the normalization method, the mean normalization normalization method, and the feature scaling normalization method for the field of the attribute whose number of attributes is the feature information. It can be (S230).
  • the attribute in the attribute information determines the normalization method for the field of the ratio attribute and decides to exclude it from the normalization target or determines the standard score normalization method. It can be (S230).
  • the normalization method is not determined and the normalization target is determined for the field of the attribute presence or absence (for example, presence / absence of an operation result value) in the feature information. It may be decided to exclude (S230).
  • the operation method of the normalization method recommendation apparatus 300 recommends the normalization method determined in step S220 or step S230 (S240).
  • the recommendation / applying the optimal normalization method with optimal performance (accuracy) to the user generating the AI model for security control based on the UI By doing so, even an average user who is not familiar with security control technology can create an optimal artificial intelligence model for security control.
  • an artificial intelligence model platform that enables the creation of an artificial intelligence model for security control is implemented, but in particular, feature information and normalization methods directly related to the performance of the artificial intelligence model are optimally recommended /
  • an artificial intelligence model platform that allows an ordinary user who is not familiar with security control technology to generate an optimal AI model for security control.
  • the optimal artificial intelligence model suitable for the purpose and requirements for security control can be flexibly and variously generated and applied, the quality improvement of the security control service can be maximized, and large scale It can be expected to have the effect of supporting the construction of an AI-based infringement response system to efficiently analyze the signs of cyber attacks and anomalies.
  • the artificial intelligence model platform operating method may be implemented in a form of program instructions that can be executed through various computer means and may be recorded in a computer readable medium.
  • the computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination.
  • the program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in computer software.
  • Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks.
  • -Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like.
  • Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler.
  • the hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne une technique, qui met en œuvre une plateforme de modèle d'intelligence artificielle capable de créer un modèle d'intelligence artificielle pour une commande de sécurité et, en particulier, peut recommander/appliquer de manière optimale des informations de caractéristiques ainsi que des procédés de normalisation directement liés aux performances du modèle d'intelligence artificielle, ce qui permet aux utilisateurs généraux, qui ne connaissent pas bien une technique de commande de sécurité, de créer un modèle d'intelligence artificielle optimal pour une commande de sécurité.
PCT/KR2018/015476 2018-11-17 2018-12-07 Plateforme de modèle d'intelligence artificielle et procédé de fonctionnement de plateforme de modèle d'intelligence artificielle Ceased WO2020101108A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020180142166A KR102271449B1 (ko) 2018-11-17 2018-11-17 인공지능 모델 플랫폼 및 인공지능 모델 플랫폼 운영 방법
KR10-2018-0142166 2018-11-17

Publications (1)

Publication Number Publication Date
WO2020101108A1 true WO2020101108A1 (fr) 2020-05-22

Family

ID=70731462

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2018/015476 Ceased WO2020101108A1 (fr) 2018-11-17 2018-12-07 Plateforme de modèle d'intelligence artificielle et procédé de fonctionnement de plateforme de modèle d'intelligence artificielle

Country Status (2)

Country Link
KR (1) KR102271449B1 (fr)
WO (1) WO2020101108A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112306829A (zh) * 2020-10-12 2021-02-02 成都安易迅科技有限公司 性能信息的确定方法及装置、存储介质、终端

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102357630B1 (ko) * 2020-07-10 2022-02-07 한국전자통신연구원 제어시스템 보안이벤트의 공격전략 분류 장치 및 방법
KR102532757B1 (ko) * 2020-09-24 2023-05-12 서강대학교산학협력단 라만 분광 신호를 기초로 수용액의 용존 가스 농도를 예측하기 위한 장치 및 이를 위한 방법
JP2023551267A (ja) * 2020-11-27 2023-12-07 コリア インスティテュート オブ サイエンス アンド テクノロジー インフォメーション セキュリティデータ処理装置、セキュリティデータ処理方法及びセキュリティデータを処理するプログラムを格納するコンピュータ読み取り可能な格納媒体
KR102470364B1 (ko) * 2020-11-27 2022-11-25 한국과학기술정보연구원 보안 이벤트 학습데이터 생성 방법 및 보안 이벤트 학습데이터 생성 장치
KR102785546B1 (ko) * 2021-09-09 2025-03-26 라인플러스 주식회사 기기에 탑재된 다수의 연합 학습 모델을 관리하는 방법, 시스템, 및 컴퓨터 프로그램
KR102433830B1 (ko) 2021-11-10 2022-08-18 한국인터넷진흥원 인공지능 기반 보안위협 이상행위 탐지 시스템 및 방법
CN116151601A (zh) * 2021-11-15 2023-05-23 中兴通讯股份有限公司 一种流业务建模方法、装置、平台、电子设备和存储介质
KR102809822B1 (ko) 2021-11-24 2025-05-21 주식회사 윈스테크넷 인공지능 기반 정오탐 식별 모델 생성 방법과 장치 및 인공지능 기반 정오탐 식별 방법과 장치
KR102620130B1 (ko) * 2021-12-08 2024-01-03 한국과학기술정보연구원 APT (Advanced Persistent Threat) 공격 탐지 방법 및 장치
KR102381776B1 (ko) * 2021-12-24 2022-04-01 주식회사 코난테크놀로지 인공지능 기능 처리 및 데이터 수집을 동시 수행하는 데이터 처리장치 및 그 방법
KR102491688B1 (ko) * 2022-02-03 2023-01-26 주식회사 데이터스튜디오 금융투자상품의 방향성 예측 모델링 방식을 결정하기 위한 전자 장치의 제어 방법
KR102771051B1 (ko) * 2022-11-25 2025-02-19 한전케이디엔주식회사 네트워크 공격 정오탐 판별 장치 및 그 방법

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6018345B2 (ja) * 1981-07-24 1985-05-09 東洋工業株式会社 ３本組無結節網の編網機と編網方法
KR101623071B1 (ko) * 2015-01-28 2016-05-31 한국인터넷진흥원 공격의심 이상징후 탐지 시스템
US20160358099A1 (en) * 2015-06-04 2016-12-08 The Boeing Company Advanced analytical infrastructure for machine learning
KR20180080111A (ko) * 2017-01-03 2018-07-11 한국전자통신연구원 자가 학습을 위한 데이터 메타 스케일링 장치 및 방법
KR20180120056A (ko) * 2017-04-26 2018-11-05 김정희 학습 데이터 전처리 방법 및 시스템

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10268820B2 (en) * 2014-06-11 2019-04-23 Nippon Telegraph And Telephone Corporation Malware determination device, malware determination system, malware determination method, and program

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6018345B2 (ja) * 1981-07-24 1985-05-09 東洋工業株式会社 ３本組無結節網の編網機と編網方法
KR101623071B1 (ko) * 2015-01-28 2016-05-31 한국인터넷진흥원 공격의심 이상징후 탐지 시스템
US20160358099A1 (en) * 2015-06-04 2016-12-08 The Boeing Company Advanced analytical infrastructure for machine learning
KR20180080111A (ko) * 2017-01-03 2018-07-11 한국전자통신연구원 자가 학습을 위한 데이터 메타 스케일링 장치 및 방법
KR20180120056A (ko) * 2017-04-26 2018-11-05 김정희 학습 데이터 전처리 방법 및 시스템

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112306829A (zh) * 2020-10-12 2021-02-02 成都安易迅科技有限公司 性能信息的确定方法及装置、存储介质、终端
CN112306829B (zh) * 2020-10-12 2023-05-09 成都安易迅科技有限公司 性能信息的确定方法及装置、存储介质、终端

Also Published As

Publication number Publication date
KR20200057903A (ko) 2020-05-27
KR102271449B1 (ko) 2021-07-01

Similar Documents

Publication Publication Date Title
WO2020101108A1 (fr) Plateforme de modèle d'intelligence artificielle et procédé de fonctionnement de plateforme de modèle d'intelligence artificielle
WO2018117619A1 (fr) Appareil d'affichage, procédé de reconnaissance de contenu et support d'enregistrement lisible par ordinateur non transitoire
WO2016017975A1 (fr) Procédé de modification d'une image consistant à photographier un élément limité, et dispositif et système pour réaliser le procédé
WO2016089009A1 (fr) Procédé et serveur cloud pour dispositif de gestion
WO2017084337A1 (fr) Procédé, appareil et système de vérification d'identité
WO2012074338A2 (fr) Procédé de traitement de langage naturel et de formule mathématique et dispositif associé
WO2023153818A1 (fr) Procédé de fourniture d'un modèle de réseau neuronal et appareil électronique pour sa mise en œuvre
WO2010021527A2 (fr) Système et procédé d'indexation d'objet dans une image
WO2016032021A1 (fr) Appareil et procédé de reconnaissance de commandes vocales
WO2019177182A1 (fr) Appareil de recherche de contenu multimédia et procédé de recherche utilisant une analyse d'informations d'attributs
WO2014021567A1 (fr) Procédé pour la fourniture d'un service de messagerie, et dispositif et système correspondants
WO2021215787A1 (fr) Système et procédé de détection de caméra à ip sans fil
WO2019035491A1 (fr) Procédé et dispositif d'authentification d'utilisateur
CN107113177A (zh) 数据连接、传送、接收、交互的方法及系统，及存储器、飞行器
WO2022075609A1 (fr) Appareil électronique de réponse à des questions utilisant plusieurs agents conversationnels et procédé de commande de celui-ci
WO2020060161A1 (fr) Système d'analyse statistique et méthode d'analyse statistique utilisant une interface conversationnelle
WO2019000466A1 (fr) Procédé et appareil de reconnaissance faciale, support de stockage et dispositif électronique
WO2024185962A1 (fr) Dispositif de gestion de la qualité de l'eau, procédé de fonctionnement et procédé de gestion de la qualité de l'eau
WO2019045320A1 (fr) Procédé et dispositif électronique permettant de prédire une structure électronique de matériau
WO2017188497A1 (fr) Procédé d'authentification d'utilisateur à intégrité et sécurité renforcées
WO2022114828A1 (fr) Dispositif de traitement de données de sécurité, procédé de traitement de données de sécurité, et support de stockage lisible par ordinateur pour stocker un programme pour traiter des données de sécurité
WO2017113587A1 (fr) Procédé et appareil de création de mot de passe wep
WO2015093790A1 (fr) Procédé et appareil de commande de commutation virtuelle
WO2021187733A1 (fr) Dispositif électronique et son procédé de commande
WO2024143637A1 (fr) Appareil de recommandation d'agencement optimal de machine virtuelle et système d'exploitation de serveur le comprenant

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18939927

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18939927

Country of ref document: EP

Kind code of ref document: A1