WO2021004161A1 - 一种异常检测方法和装置 - Google Patents

一种异常检测方法和装置 Download PDF

Info

Publication number
WO2021004161A1
WO2021004161A1 PCT/CN2020/090936 CN2020090936W WO2021004161A1 WO 2021004161 A1 WO2021004161 A1 WO 2021004161A1 CN 2020090936 W CN2020090936 W CN 2020090936W WO 2021004161 A1 WO2021004161 A1 WO 2021004161A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
abnormal
detection
result
reliability
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/090936
Other languages
English (en)
French (fr)
Inventor
杜永生
罗颖燕
潘春锦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to US17/625,078 priority Critical patent/US11777824B2/en
Priority to EP20837813.3A priority patent/EP3979416B1/en
Priority to JP2021578200A priority patent/JP2022539578A/ja
Publication of WO2021004161A1 publication Critical patent/WO2021004161A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/04—Arrangements for maintaining operational condition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2178—Validation; Performance evaluation; Active pattern learning techniques based on feedback of a supervisor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/142—Network analysis or design using statistical or mathematical methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/08—Testing, supervising or monitoring using real traffic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W76/00—Connection management
    • H04W76/20—Manipulation of established connections
    • H04W76/27—Transitions between radio resource control [RRC] states
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
    • H04W16/22—Traffic simulation tools or models

Definitions

  • the embodiments of the present application relate to, but are not limited to, the field of communication operation and maintenance, in particular to an abnormality detection method and device.
  • KPI Key Performance Index
  • the embodiment of the present application provides an anomaly detection method, including: for each anomaly detection algorithm in N types of anomaly detection algorithms, using the anomaly detection algorithm to detect the first characteristic data of the collected first data to obtain the first feature data A detection result; where N is an integer greater than or equal to 1; when the first detection result of one or more anomaly detection algorithms is abnormal, the reliability of the first characteristic data of the first data is checked Obtain the first verification result; determine the second detection result according to whether the reliability of the abnormality detection algorithm whose first detection result is abnormal is greater than or equal to the first preset threshold and the first verification result.
  • the embodiment of the present application provides an abnormality detection device, which includes a processor and a computer-readable storage medium.
  • the computer-readable storage medium stores instructions. When the instructions are executed by the processor, any one of the foregoing An anomaly detection method.
  • the embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above-mentioned abnormal detection methods are realized.
  • the embodiment of the present application provides an abnormality detection device, including: a detection module, configured to use the abnormality detection algorithm for each of the N abnormality detection algorithms to determine the first feature of the collected first data Data is detected to obtain a first detection result; where N is an integer greater than or equal to 1; when the first detection result of one or more anomaly detection algorithms is abnormal, the first characteristic data of the first data Perform reliability verification to obtain the first verification result; the determining module is used to determine the second detection according to whether the reliability of the abnormality detection algorithm whose first detection result is abnormal is greater than or equal to the first preset threshold and the first verification result result.
  • FIG. 1 is a flowchart of an abnormality detection method proposed in an embodiment of the application
  • FIG. 2 is a schematic diagram of the structural composition of an abnormality detection device proposed in another embodiment of the application.
  • an embodiment of the present application proposes an anomaly detection method, including: step 100, for each anomaly detection algorithm in N types of anomaly detection algorithms, use the anomaly detection algorithm to analyze the first data collected The first feature data is detected to obtain the first detection result; where N is an integer greater than or equal to 1.
  • the first data may be any data that needs to be detected for anomaly, such as KPI data.
  • the first data includes the time series data data1 and related data data2.
  • data1 is recorded as ⁇ (t 1 ,x 1 ),...,(t w ,x w ) ⁇ , t w is the time to be detected, and x w is the value of the success rate of RRC establishment connection corresponding to t w ,
  • data2 is denoted as [x eff , x ref , pre_res, obj_id, KPI_id]
  • x eff is the number of RRC connection establishment requests corresponding to t w , x ref It is the number of RRC connection establishment failures
  • pre_res is whether it is abnormal at t w-1
  • obj_id is the cell id
  • KPI_id is the id of the KPI to be detected.
  • all the first feature data required by the N types of anomaly detection algorithms can be extracted from the first data at one time, and each anomaly detection algorithm is used to compare the collected first feature data.
  • the first feature data of the data is detected to obtain the first detection result, the first feature data required by the anomaly detection algorithm is selected from the extracted feature data;
  • the first feature data required by the anomaly detection algorithm is extracted from the collected first data .
  • the first data need to be feature extraction, and for the first data that does not need feature extraction, the first data can be directly used as the first feature data.
  • the first data is cleaned first, and then the feature data is extracted from the cleaned first data.
  • the first data is cleaned first, and then the feature data is extracted from the cleaned first data.
  • the first data is cleaned first, and part of the first data may be cleaned, which is not limited in the embodiment of the present application.
  • data1 is cleaned, but data2 is not cleaned.
  • data1 has missing values
  • use methods such as linear interpolation and mean to fill in the missing data.
  • extract the first feature data from the cleaned data1 for example, perform feature statistics on the cleaned data1 (such as maximum x max , minimum x min , average x mean , median x median , Standard deviation x std, etc.), periodic judgment, classification feature construction (for example, one-hot encoding of t w , including hour, day of the week, etc.), calculation of the year-on-year ring ratio of x w , first-order difference, two Order difference etc.
  • feature statistics such as maximum x max , minimum x min , average x mean , median x median , Standard deviation x std, etc.
  • classification feature construction for example, one-hot encoding of t w , including hour, day of the week, etc.
  • calculation of the year-on-year ring ratio of x w for example, one-hot encoding of t w , including hour, day of the week, etc.
  • the anomaly detection algorithm may be based on a statistical learning algorithm (for example, 3-sigma, Exponentially Weighted Moving-Average (EWMA), and differential integrated moving average autoregressive model (ARIMA, Autoregressive Integrated Moving-Average). Average Model, etc.), unsupervised classification algorithms (such as Isolation Forest, Support Vector Machine (SVM) (one-class SVM), Variational Auto-Encode, etc.) , Supervised learning algorithms (such as logistic regression, extreme gradient boosting (XGBoost, eXtreme Gradient Boosting), deep neural network (DNN, Deep Neural Network), etc.).
  • a statistical learning algorithm for example, 3-sigma, Exponentially Weighted Moving-Average (EWMA), and differential integrated moving average autoregressive model (ARIMA, Autoregressive Integrated Moving-Average). Average Model, etc.
  • unsupervised classification algorithms such as Isolation Forest, Support Vector Machine (SVM) (one-class SVM), Variational Auto-Encode,
  • the embodiment of the present application uses a variety of anomaly detection algorithms to perform anomaly detection, with the purpose of detecting all possible abnormal data.
  • Step 101 When the first detection result of one or more abnormal detection algorithms is abnormal, perform reliability verification on the first characteristic data of the first data to obtain the first verification result.
  • the user can set the verification conditions for reliability verification, select the reliability evaluation method, and set related parameters through the interface, or import the verification conditions, reliability evaluation methods, and related parameters in the form of writing configuration files. parameter.
  • the specific implementation manner is not limited in the embodiment of this application.
  • performing reliability verification on the first characteristic data of the first data includes: performing at least one of the following on the first characteristic data of the first data: data validity verification, safety interval verification, and complete Network contribution test and abnormal continuity test.
  • the data validity check on the first feature data is to detect whether the number of RRC establishment connection requests x eff is greater than the second preset threshold x eff _threshold, when RRC is established When the number of connection requests is greater than the second preset threshold, the first data is considered valid, and the security interval check is continued on the first characteristic data; when the number of RRC establishment connection requests is less than or equal to the second preset threshold, the first data is considered invalid, Then the first data is normal (that is, the first check result is normal), and the check ends.
  • x eff _threshold is given by experts based on business experience and domain knowledge, which can be specifically set by the user on the page or import a configurable file.
  • Performing a safe interval check on the first characteristic data is to check whether the RRC connection establishment success rate x w is within the safe interval [x min , x max ].
  • the first data is considered normal ( That is, the first check result is normal), and the check ends; when the RRC connection establishment success rate is not within the safe interval, the whole network contribution degree check is continued on the first characteristic data.
  • the safety interval [x min , x max ] is given by experts based on business experience and domain knowledge, which can be specifically set by the user on the page or import a configurable file.
  • x ref _threshold is given by experts based on business experience and domain knowledge, which can be specifically set by the user on the page or import a configurable file.
  • the continuity test of the first feature is to check whether the test result pre_res for a period of time before t w is abnormal, and whether there is a continuous deterioration trend through the value of the second-order difference, when the test for a period of time before t w
  • the first verification result is determined to be abnormal; when at least one of the test results for a period of time before t w is normal, or there is a tendency to change for the better, it is determined
  • the first check result is normal and the check ends.
  • Step 102 Determine a second detection result according to whether the reliability of the abnormality detection algorithm whose first detection result is abnormal is greater than or equal to the first preset threshold and the first verification result.
  • the second detection result includes any one or more of the following: When the reliability of one or more of the abnormality detection algorithms in the abnormality detection algorithms whose first detection result is abnormal is greater than or equal to the first preset threshold, it is determined that the second detection result is abnormal; when When the reliability of the abnormality detection algorithm whose first detection result is abnormal is less than the first preset threshold, the first verification result is used as the second detection result.
  • the second detection result is determined to be abnormal; when the reliability of the abnormal detection algorithms XGBoost and 3-sigma are both less than the first preset threshold, the first verification result is used as the second detection result.
  • the reliability of the anomaly detection algorithm includes: similarity comparison of anomaly detection algorithms, statistical testing, and the like. For example, the coincidence rate of the anomaly detection algorithm.
  • the method before determining the second detection result according to whether the reliability of the abnormality detection algorithm whose first detection result is abnormal is greater than or equal to the first preset threshold and the first verification result, the method further includes: For each of the abnormality detection algorithms, the coincidence rate of the abnormality detection algorithms is calculated.
  • calculating the coincidence rate of the anomaly detection algorithm includes: determining the ratio of the first number to the second number as the coincidence rate; wherein, the first number is all second numbers collected within a preset time.
  • a third detection result obtained by detecting the second characteristic data of the second data using the abnormality detection algorithm and a second verification result obtained by performing a reliability check on the second characteristic data of the second data The same number of second data; the second number is the third detection result obtained by using the abnormal detection algorithm to detect the second characteristic data of the second data among all the second data collected within the preset time Is the number of abnormal second data.
  • the test results of the RRC connection success rate index for all cells are abnormal samples.
  • the detection results obtained by the anomaly detection algorithm are abnormal samples.
  • the method further includes any one or more of the following: when the second detection result is abnormal and the user’s abnormal cancellation information is received, marking the first data as normal When the second detection result is abnormal, and the user’s abnormal confirmation information is received, the first data is added to the labeled sample library as an abnormal sample; wherein, the label The samples marked as normal or abnormal in the sample library are used to train the supervised learning model of the abnormality detection algorithm, and the abnormality detection algorithm detects the first feature data of the first data based on the supervised learning model to obtain the first feature data One test result. Add the first data as labeled samples to the labeled sample library.
  • the anomaly detection algorithm based on supervised learning or semi-supervised learning apply new labeled samples Incremental learning or periodic retraining of the supervised learning model improves the reliability of the supervised learning model, thereby improving the detection accuracy of the anomaly detection algorithm.
  • the abnormal information when the second detection result is abnormal, the abnormal information is displayed.
  • the abnormal information includes the object name "cell 1" corresponding to the abnormal data (ie the first data), the occurrence time t w , and the KPI name "RRC connection establishment Success rate", KPI value x w, etc.
  • the user can choose to enter the exception confirmation information to confirm the exception, or enter the exception cancellation information to cancel the exception; or do nothing.
  • the abnormality confirmation information the first data is added as an abnormal sample to the labeled sample library; when the user inputs abnormal cancellation information, the first data is added to the sample library as a normal sample.
  • the embodiment of the present application determines the final detection result based on the detection results of various anomaly detection algorithms and the reliability verification results. Since the reliability verification is based on the reliability verification of expert business experience and domain knowledge, the detection is improved. Reliability of results.
  • an abnormality detection device which includes a processor and a computer-readable storage medium.
  • the computer-readable storage medium stores instructions. When the instructions are executed by the processor, the foregoing Any anomaly detection method.
  • Another embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above-mentioned abnormal detection methods are realized.
  • an anomaly detection device including: a detection module 201, for each anomaly detection algorithm in N types of anomaly detection algorithms, using the anomaly detection algorithm to collect The first feature data of the first data is detected to obtain the first detection result; where N is an integer greater than or equal to 1; when the first detection result of one or more abnormal detection algorithms is abnormal, the first detection result Perform reliability verification on the first characteristic data of a data to obtain a first verification result; the determining module 202 is configured to determine whether the reliability of the abnormality detection algorithm for which the first detection result is abnormal is greater than or equal to the first preset threshold and the first A verification result determines the second detection result.
  • the first data may be any data that needs to be detected for anomaly, such as KPI data.
  • the first data includes the time series data data1 and related data data2.
  • data1 is recorded as ⁇ (t 1 ,x 1 ),...,(t w ,x w ) ⁇ , t w is the time to be detected, and x w is the value of the success rate of RRC establishment connection corresponding to t w ,
  • data2 is denoted as [x eff , x ref , pre_res, obj_id, KPI_id]
  • x eff is the number of RRC connection establishment requests corresponding to t w , x ref It is the number of RRC connection establishment failures
  • pre_res is whether it is abnormal at t w-1
  • obj_id is the cell id
  • KPI_id is the id of the KPI to be detected.
  • the detection module 201 may extract all the first characteristic data required by the N types of anomaly detection algorithms from the first data at one time after collecting the first data, and then use each anomaly detection algorithm to collect When the first feature data of the first data is detected to obtain the first detection result, the first feature data required by the abnormal detection algorithm is selected from the extracted feature data;
  • the detection module 201 uses each anomaly detection algorithm to detect the first characteristic data of the collected first data to obtain the first detection result, it extracts from the collected first data the first feature required by the anomaly detection algorithm. One feature data.
  • the first data need to be feature extraction, and for the first data that does not need feature extraction, the first data can be directly used as the first feature data.
  • the detection module 201 before extracting the characteristic data of the first data, the detection module 201 first cleans the first data, and then extracts the characteristic data from the cleaned first data.
  • the first data needs to be cleaned, and part of the first data may be cleaned, which is not limited in the embodiment of the present application.
  • data1 is cleaned, but data2 is not cleaned.
  • data1 has missing values
  • use methods such as linear interpolation and mean to fill in the missing data.
  • extract the first feature data from the cleaned data1 for example, perform feature statistics on the cleaned data1 (such as maximum x max , minimum x min , average x mean , median x median , Standard deviation x std, etc.), periodic judgment, classification feature construction (for example, one-hot encoding of t w , including hour, day of the week, etc.), calculation of the year-on-year ring ratio of x w , first-order difference, two Order difference etc.
  • feature statistics such as maximum x max , minimum x min , average x mean , median x median , Standard deviation x std, etc.
  • classification feature construction for example, one-hot encoding of t w , including hour, day of the week, etc.
  • calculation of the year-on-year ring ratio of x w for example, one-hot encoding of t w , including hour, day of the week, etc.
  • the anomaly detection algorithm may be based on a statistical learning algorithm (for example, 3-sigma, Exponentially Weighted Moving-Average (EWMA), and differential integrated moving average autoregressive model (ARIMA, Autoregressive Integrated Moving-Average). Average Model, etc.), unsupervised classification algorithms (such as Isolation Forest, Support Vector Machine (SVM) (one-class SVM), Variational Auto-Encode, etc.) , Supervised learning algorithms (such as logistic regression, extreme gradient boosting (XGBoost, eXtreme Gradient Boosting), deep neural network (DNN, Deep Neural Network), etc.).
  • a statistical learning algorithm for example, 3-sigma, Exponentially Weighted Moving-Average (EWMA), and differential integrated moving average autoregressive model (ARIMA, Autoregressive Integrated Moving-Average). Average Model, etc.
  • unsupervised classification algorithms such as Isolation Forest, Support Vector Machine (SVM) (one-class SVM), Variational Auto-Encode,
  • the embodiment of the present application uses a variety of anomaly detection algorithms to perform anomaly detection, with the purpose of detecting all possible abnormal data.
  • the user can set the verification conditions for reliability verification, select the reliability evaluation method, and set related parameters through the interface, or import the verification conditions, reliability evaluation methods, and related parameters in the form of writing configuration files. parameter.
  • the specific implementation manner is not limited in the embodiment of this application.
  • the detection module 201 is specifically configured to implement the reliability check on the first characteristic data of the first data in the following manner: perform at least one of the following on the first characteristic data of the first data: data Validity test, safety interval test, network-wide contribution test, abnormal continuity test.
  • the data validity check on the first feature data is to detect whether the number of RRC establishment connection requests x eff is greater than the second preset threshold x eff _threshold, when RRC is established When the number of connection requests is greater than the second preset threshold, the first data is considered valid, and the security interval check is continued on the first characteristic data; when the number of RRC establishment connection requests is less than or equal to the second preset threshold, the first data is considered invalid, Then the first data is normal (that is, the first check result is normal), and the check ends.
  • x eff _threshold is given by experts based on business experience and domain knowledge, which can be specifically set by the user on the page or import a configurable file.
  • Performing a safe interval check on the first characteristic data is to check whether the RRC connection establishment success rate x w is within the safe interval [x min , x max ].
  • the first data is considered normal ( That is, the first check result is normal), and the check ends; when the RRC connection establishment success rate is not within the safe interval, the first characteristic data is continuously checked for the contribution of the whole network.
  • the safety interval [x min , x max ] is given by experts based on business experience and domain knowledge, which can be specifically set by the user on the page or import a configurable file.
  • x ref _threshold is given by experts based on business experience and domain knowledge, which can be specifically set by the user on the page or import a configurable file.
  • the continuity test of the first feature is to check whether the test result pre_res for a period of time before t w is abnormal, and whether there is a continuous deterioration trend through the value of the second-order difference, when the test for a period of time before t w
  • the first verification result is determined to be abnormal; when at least one of the test results for a period of time before t w is normal, or there is a tendency to change for the better, it is determined
  • the first check result is normal and the check ends.
  • the determining module 202 is specifically configured to execute any one or more of the following: when the first detection result is abnormal, the reliability of one or more abnormal detection algorithms is greater than Or equal to the first preset threshold, it is determined that the second detection result is abnormal; when the reliability of the abnormality detection algorithm for which the first detection result is abnormal is less than the first preset threshold, all The first verification result is used as the second detection result.
  • the second detection result is determined to be abnormal; when the reliability of the abnormal detection algorithms XGBoost and 3-sigma are both less than the first preset threshold, the first verification result is used as the second detection result.
  • the reliability of the anomaly detection algorithm includes: similarity comparison of anomaly detection algorithms, statistical testing, and the like. For example, the coincidence rate of the anomaly detection algorithm.
  • the determining module 202 is further configured to: for each of the abnormality detection algorithms, calculate the coincidence rate of the abnormality detection algorithms.
  • the determining module 202 is specifically configured to calculate the coincidence rate of the anomaly detection algorithm in the following manner: determine that the ratio of the first number to the second number is the coincidence rate; wherein, the first number is Among all the second data collected within the preset time, the third detection result obtained by using the abnormality detection algorithm to detect the second characteristic data of the second data and the reliability of the second characteristic data of the second data.
  • the number of second data with the same second verification result obtained by the verification; the second number is the second feature of the second data using the abnormality detection algorithm among all the second data collected within the preset time
  • the third detection result obtained by data detection is the number of abnormal second data.
  • the test results of the RRC connection success rate index for all cells are abnormal samples.
  • the detection results obtained by the anomaly detection algorithm are abnormal samples.
  • the determining module 202 is further configured to perform any one or more of the following: when the second detection result is abnormal, and the user's abnormal cancellation information is received, the first data is used as The samples marked as normal are added to the marked sample library; when the second detection result is abnormal and the user's abnormal confirmation information is received, the first data is added to the marked sample library as the samples marked as abnormal; wherein, The samples labeled as normal or abnormal in the labeled sample library are used to train the supervised learning model of the abnormality detection algorithm, and the abnormality detection algorithm performs the first feature data of the first data based on the supervised learning model. The first test result is obtained. Add the first data as labeled samples to the labeled sample library.
  • the anomaly detection algorithm based on supervised learning or semi-supervised learning apply new labeled samples Incremental learning or periodic retraining of the supervised learning model improves the reliability of the supervised learning model, thereby improving the detection accuracy of the anomaly detection algorithm.
  • the abnormal information when the second detection result is abnormal, the abnormal information is displayed.
  • the abnormal information includes the object name "cell 1" corresponding to the abnormal data (ie the first data), the occurrence time t w , and the KPI name "RRC connection establishment Success rate", KPI value x w, etc.
  • the user can choose to enter the exception confirmation information to confirm the exception, or enter the exception cancellation information to cancel the exception; or do nothing.
  • the user inputs abnormality confirmation information the first data is added to the labeled sample library as a sample labeled as abnormal; when the user inputs abnormality cancellation information, the first data is added to the sample library as a labeled normal sample.
  • the embodiment of the present application determines the final detection result based on the detection results of various anomaly detection algorithms and the reliability verification results. Since the reliability verification is based on the reliability verification of expert business experience and domain knowledge, the detection is improved. Reliability of results.
  • Such software may be distributed on a computer-readable medium
  • the computer-readable medium may include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium).
  • the term computer storage medium includes volatile and non-volatile memory implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data).
  • Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage device, or Any other medium used to store desired information and that can be accessed by a computer.
  • communication media usually contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as carrier waves or other transmission mechanisms, and may include any information delivery media .

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Security & Cryptography (AREA)
  • Environmental & Geological Engineering (AREA)
  • Computing Systems (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Analysis (AREA)
  • Algebra (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Pure & Applied Mathematics (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本申请实施例公开了一种异常检测方法和装置，所述方法包括：对于每一种异常检测算法，采用所述异常检测算法对采集的第一数据的第一特征数据进行检测；其中，N为大于或等于1的整数；当一种或一种以上异常检测算法的第一检测结果为异常时，对所述第一数据的第一特征数据进行可靠性校验得到第一校验结果；根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果。

Description

一种异常检测方法和装置
交叉引用
本申请基于申请号为201910605053.9、申请日为2019年7月5日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式并入本申请。
技术领域
本申请实施例涉及但不限于通讯运维领域，尤指一种异常检测方法和装置。
背景技术
在通讯运维领域中，检测电信运营服务的一些关键绩效指标(KPI，Key Performance Index)，比如无线接通率、丢包率、当前用户数、上下流量等等。当KPI出现异常(或劣化)时，很可能由于通讯环境发生了变化或者软硬件等发生故障造成的。运维人员及时发现问题、解决问题，能有效提升用户满意度，维护公司的声誉。然而，通讯网络中KPI数目繁多，种类成千上万，同时，即将到来的5G通讯技术，将成倍增加KPI数目，使得依靠运维人员人工实时对全网KPI数据监测维护越来越难，所以对于通讯网络KPI异常检测维护的智能化也越来越迫切。
相关的智能化异常检测系统大多基于机器学习技术，对于通讯运维领域而言，在引入机器学习初期，基本上很难积累足够的标注样本，用以训练出可靠的模型进行检测。原因是标注成本高，也就是说KPI数目繁多，同时，人工手动标注门槛较高，需要依赖大量的业务知识。然而，依赖无监督学习算法，或者用数量较少的样本来训练监督学习算法，很难保证检测结果的可靠性。
发明内容
本申请实施例提供了一种异常检测方法，包括：对于N种异常检测算法中的每一种异常检测算法，采用所述异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果；其中，N为大于或等于1的整数；当一种或一种以上异常检测算法的第一检测结果为异常时，对所述第一数据的第一特征数据进行可靠性校验得到第一校验结果；根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果。
本申请实施例提供了一种异常检测装置，包括处理器和计算机可读存储介质，所述计算机可读存储介质中存储有指令，当所述指令被所述处理器执行时，实现上述任一种异常检测方法。
本申请实施例提供了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一种异常检测方法的步骤。
本申请实施例提供了一种异常检测装置，包括：检测模块，用于对于N种异常检测算法中的每一种异常检测算法，采用所述异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果；其中，N为大于或等于1的整数；当一种或一种以上异常检测算法的第一检测结果为异常时，对所述第一数据的第一特征数据进行可靠性校验得到第一校验结果；确定模块，用于根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果。
本申请实施例的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本申请实施例而了解。本申请实施例的目的和其他优点可通过在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。
附图说明
附图用来提供对本申请实施例技术方案的进一步理解，并且构成说明书的一部分，与本申请实施例的实施例一起用于解释本申请实施例的技术方案， 并不构成对本申请实施例技术方案的限制。
图1为本申请一个实施例提出的异常检测方法的流程图；
图2为本申请另一个实施例提出的异常检测装置的结构组成示意图。
具体实施方式
下文中将结合附图对本申请实施例进行详细说明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互任意组合。
在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行。并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
参见图1，本申请一个实施例提出了一种异常检测方法，包括：步骤100、对于N种异常检测算法中的每一种异常检测算法，采用所述异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果；其中，N为大于或等于1的整数。
在本申请实施例中，第一数据可以是任意需要进行异常检测的数据，例如KPI数据等。
例如，对于小区1的无线资源控制(RRC，Radio Resource Control)建立连接成功率指标，第一数据包括时间序列数data1和相关数据data2。
其中，data1记为{(t 1,x 1),...,(t w,x w)}，t w为待检测的时刻，x w为t w对应的RRC建立连接成功率的值，采样时间粒度为T，即t w-t w-1＝T，data2记为[x eff，x ref，pre_res，obj_id，KPI_id]，x eff为t w时刻对应的RRC建立连接请求数、x ref为RRC建立连接失败次数、pre_res为t w-1时刻是否异常，obj_id为小区id，KPI_id为待检测KPI的id。
在本申请实施例中，可以在采集第一数据后，一次性从第一数据中提取N种异常检测算法所需要的所有第一特征数据，在采用每一种异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果时，从提取的特征数据中选择该种异常检测算法所需要的第一特征数据；
或者，在采用每一种异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果时，从采集的第一数据中提取该种异常检测算法所需要的第一特征数据。
当然，也不是所有的第一数据都需要进行特征提取，对于不需要进行特征提取的第一数据，直接将第一数据作为第一特征数据即可。
在一个示例性实例中，在提取第一数据的特征数据之前，首先对第一数据进行清洗，再从清洗后的第一数据中提取特征数据。当然，也不是所有的第一数据均需要进行清洗，可以对部分第一数据进行清洗，本申请实施例对此不作限定。
例如，对于上述data1和data2，对data1进行清洗，不对data2进行清洗。例如，当data1有缺失值时，采用例如线性插值、均值等方法对缺失的数据进行填充。
对data1进行清洗后，从清洗后的data1中提取第一特征数据，例如，对清洗后的data1进行特征统计(例如最大值x max、最小值x min、平均值x mean、中位数x median、标准差x std等)、周期性判断、分类特征构造(例如对t w进行独热(one-hot)编码，包括小时，星期几等)、计算x w的同比环比，一阶差分，二阶差分等。
直接将data2加入第一特征数据中，最终得到第一特征数据，记作feature data＝[f 1,f 2,...,f n]。
在本申请实施例中，异常检测算法可以是基于统计学习算法(例如3-sigma，指数加权移动平均法(EWMA，Exponentially Weighted Moving-Average)，差分整合移动平均自回归模型(ARIMA，Autoregressive Integrated Moving Average Model)等)、无监督分类算法(例如孤立森林(Isolation Forest)，单分类支持向量机(SVM，Support Vector Machine)(one-class SVM)，变分自编码(Variational Auto-Encode)等)、监督学习算法(例如逻辑回归，极端梯度提升(XGBoost，eXtreme Gradient Boosting)，深度神经网络(DNN，Deep Neural Network)等等)。本申请实施例采用多种异常检测算法进行异常检测，目的是将可能的异常数据都检测出来。例如，对于从上述data1和data2 提取的第一特征数据，采用3-sigma，EWMA，Holt-Winters，以及XGBoost算法进行检测得到的第一检测结果分别为：res1＝1、res2＝0、res3＝0、res4＝1，其中，0表示正常，1表示异常。
步骤101、当一种或一种以上异常检测算法的第一检测结果为异常时，对所述第一数据的第一特征数据进行可靠性校验得到第一校验结果。
在本申请实施例中，用户可以通过界面设置可靠性校验的校验条件、选择可靠性评估方法以及设置相关参数，也可以通过编写配置文件的形式导入校验条件、可靠性评估方法以及相关参数。具体的实现方式本申请实施例不作限定。
在一个示例性实例中，对第一数据的第一特征数据进行可靠性校验包括：对所述第一数据的第一特征数据进行以下至少之一：数据有效性检验、安全区间检验、全网贡献度检验、异常持续性检验。
上述进行可靠性校验的具体规则由专家根据业务经验、领域知识给出，可以由用户在页面进行相关设置或者导入可配置文件。
例如，对于上述从data1和data2提取的第一特征数据，对第一特征数据进行数据有效性校验也就是检测RRC建立连接请求数x eff是否大于第二预设阈值x eff_threshold，当RRC建立连接请求数大于第二预设阈值时，认为第一数据有效，继续对第一特征数据进行安全区间检验；当RRC建立连接请求数小于或等于第二预设阈值时，认为第一数据无效，那么第一数据正常(即第一校验结果为正常)，检验结束。其中，x eff_threshold由专家根据业务经验、领域知识给出，具体可以由用户在页面进行相关设置或者导入可配置文件。
对第一特征数据进行安全区间检验也就是检测RRC建立连接成功率x w是否在安全区间[x min,x max]内，当RRC建立连接成功率在安全区间内时，认为第一数据正常(即第一校验结果为正常)，检验结束；当RRC建立连接成功率不在安全区间内时，继续对第一特征数据进行全网贡献度检验。其中，安全区间[x min,x max]由专家根据业务经验、领域知识给出，具体可以由用户在页面进行相关设置或者导入可配置文件。
对第一特征数据进行全网贡献度检验也就是检测RRC建立连接失败次数 x ref是否大于第三预设阈值x ref_threshold，当RRC建立连接失败次数大于第三预设阈值时，确定第一校验结果为异常；当RRC建立连接失败次数小于或等于第三预设阈值时，继续对第一特征数据进行持续性检验。其中，x ref_threshold由专家根据业务经验、领域知识给出，具体可以由用户在页面进行相关设置或者导入可配置文件。
对第一特征进行持续性检验也就是对t w之前的一段时间的检验结果pre_res是否为异常，以及通过二阶差分的值判断是否有持续变坏的趋势，当t w之前的一段时间的检验结果均为异常，且有变坏的趋势时，确定第一校验结果为异常；当t w之前的一段时间的检验结果中的至少一个检验结果为正常，或者有变好的趋势时，确定第一校验结果为正常，检验结束。
步骤102、根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果。
在一个示例性实例中，根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果包括以下任意一种或多种：当所述第一检测结果为异常的异常检测算法中的一种或一种以上异常检测算法的可靠性大于或等于所述第一预设阈值时，确定所述第二检测结果为异常；当所述第一检测结果为异常的异常检测算法的可靠性均小于所述第一预设阈值时，将所述第一校验结果作为所述第二检测结果。
例如，上述对从data1和data2提取的第一特殊数据的检测中，当异常检测算法XGBoost的可靠性大于或等于第一预设阈值，且异常检测算法3-sigma的可靠性小于第一预设阈值时，确定第二检测结果为异常；当异常检测算法XGBoost和3-sigma的可靠性均小于第一预设阈值时，将第一校验结果作为第二检测结果。
在一个示例性实例中，异常检测算法的可靠性包括：异常检测算法的相似度比对、统计检验等。例如，所述异常检测算法的重合率。
在另一个示例性实例中，根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果之前，该方法还包括：对于每一种所述异常检测算法，计算所述异常检测算法的重 合率。
在一个示例性实例中，计算异常检测算法的重合率包括：确定第一数量和第二数量的比值为所述重合率；其中，所述第一数量为在预设时间内采集的所有第二数据中，采用所述异常检测算法对第二数据的第二特征数据进行检测得到的第三检测结果和对所述第二数据的第二特征数据进行可靠性校验得到的第二校验结果相同的第二数据的数量；所述第二数量为在预设时间内采集的所有第二数据中，采用所述异常检测算法对第二数据的第二特征数据进行检测得到的第三检测结果为异常的第二数据的数量。
例如，最近一个月，所有小区，RRC建立连接成功率指标的检测结果。即依据过去一个月，所有小区，RRC建立连接成功率指标的所有样本中，采用异常检测算法进行检测得到的检测结果为异常的样本，假设一共有m个，并且记录各个异常检测算法在每个样本上的检测结果，以及对采集的数据进行的可靠性校验得到的检验结果，在此基础上，计算异常检测算法i的与检验结果一致的样本数量c i，则重合率为c i/m。
在本申请另一个实施例中，该方法还包括以下任意一个或多个：当所述第二检测结果为异常，且接收到用户的异常取消信息时，将所述第一数据作为标注为正常的样本加入标注样本库；当所述第二检测结果为异常，且接收到用户的异常确认信息时，将所述第一数据作为标注为异常的样本加入标注样本库中；其中，所述标注样本库中的标注为正常或异常的样本用于训练所述异常检测算法的监督学习模型，所述异常检测算法基于所述监督学习模型对所述第一数据的第一特征数据进行检测得到第一检测结果。将第一数据作为标注样本加入标注样本库中，随着时间的推移，标注样本库中积累的标注样本数目越来越多，使得基于监督学习或半监督学习的异常检测算法应用新增标注样本进行增量学习或周期性地重新训练监督学习模型，提高了监督学习模型的可靠性，从而提高了异常检测算法的检测准确率。
也就是说，当第二检测结果为异常时，显示异常信息，例如，异常信息包括异常数据(即第一数据)对应的对象名称“小区1”、发生时间t w、KPI名称“RRC建立连接成功率”、KPI值x w等。用户可以选择输入异常确认信息对异常进行确认，或者输入异常取消信息对异常进行取消；或者不进行任 何操作。当用户输入异常确认信息时，将第一数据作为标注为异常的样本加入标注样本库中；当用户输入异常取消信息时，将第一数据作为标注为正常的样本加入样本库中。
本申请实施例基于多种异常检测算法的检测结果以及可靠性的校验结果确定最终检测结果，由于可靠性的校验是基于专家业务经验、领域知识所做的可靠性校验，提高了检测结果的可靠性。
本申请另一个实施例提出了一种异常检测装置，包括处理器和计算机可读存储介质，所述计算机可读存储介质中存储有指令，当所述指令被所述处理器执行时，实现上述任一种异常检测方法。
本申请另一个实施例提出了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一种异常检测方法的步骤。
参见图2，本申请另一个实施例提出了一种异常检测装置，包括：检测模块201，用于对于N种异常检测算法中的每一种异常检测算法，采用所述异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果；其中，N为大于或等于1的整数；当一种或一种以上异常检测算法的第一检测结果为异常时，对所述第一数据的第一特征数据进行可靠性校验得到第一校验结果；确定模块202，用于根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果。
在本申请实施例中，第一数据可以是任意需要进行异常检测的数据，例如KPI数据等。
例如，对于小区1的无线资源控制(RRC，Radio Resource Control)建立连接成功率指标，第一数据包括时间序列数data1和相关数据data2。
其中，data1记为{(t 1,x 1),...,(t w,x w)}，t w为待检测的时刻，x w为t w对应的RRC建立连接成功率的值，采样时间粒度为T，即t w-t w-1＝T，data2记为[x eff，x ref，pre_res，obj_id，KPI_id]，x eff为t w时刻对应的RRC建立连接请求数、x ref为RRC建立连接失败次数、pre_res为t w-1时刻是否异常，obj_id为小区id，KPI_id为待检测KPI的id。
在本申请实施例中，检测模块201可以在采集第一数据后，一次性从第一数据中提取N种异常检测算法所需要的所有第一特征数据，在采用每一种异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果时，从提取的特征数据中选择该种异常检测算法所需要的第一特征数据；
或者，检测模块201在采用每一种异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果时，从采集的第一数据中提取该种异常检测算法所需要的第一特征数据。
当然，也不是所有的第一数据都需要进行特征提取，对于不需要进行特征提取的第一数据，直接将第一数据作为第一特征数据即可。
在一个示例性实例中，检测模块201在提取第一数据的特征数据之前，首先对第一数据进行清洗，再从清洗后的第一数据中提取特征数据。当然，也不是所有的第一数据均需要进行清洗，可以对部分第一数据进行清洗，本申请实施例对此不作限定。
例如，对于上述data1和data2，对data1进行清洗，不对data2进行清洗。例如，当data1有缺失值时，采用例如线性插值、均值等方法对缺失的数据进行填充。
对data1进行清洗后，从清洗后的data1中提取第一特征数据，例如，对清洗后的data1进行特征统计(例如最大值x max、最小值x min、平均值x mean、中位数x median、标准差x std等)、周期性判断、分类特征构造(例如对t w进行独热(one-hot)编码，包括小时，星期几等)、计算x w的同比环比，一阶差分，二阶差分等。
直接将data2加入第一特征数据中，最终得到第一特征数据，记作feature data＝[f 1,f 2,...,f n]。
在本申请实施例中，异常检测算法可以是基于统计学习算法(例如3-sigma，指数加权移动平均法(EWMA，Exponentially Weighted Moving-Average)，差分整合移动平均自回归模型(ARIMA，Autoregressive Integrated Moving Average Model)等)、无监督分类算法(例如孤立森林(Isolation Forest)，单分类支持向量机(SVM，Support Vector Machine)(one-class SVM)，变 分自编码(Variational Auto-Encode)等)、监督学习算法(例如逻辑回归，极端梯度提升(XGBoost，eXtreme Gradient Boosting)，深度神经网络(DNN，Deep Neural Network)等等)。本申请实施例采用多种异常检测算法进行异常检测，目的是将可能的异常数据都检测出来。例如，对于从上述data1和data2提取的第一特征数据，采用3-sigma，EWMA，Holt-Winters，以及XGBoost算法进行检测得到的第一检测结果分别为：res1＝1、res2＝0、res3＝0、res4＝1，其中，0表示正常，1表示异常。
在本申请实施例中，用户可以通过界面设置可靠性校验的校验条件、选择可靠性评估方法以及设置相关参数，也可以通过编写配置文件的形式导入校验条件、可靠性评估方法以及相关参数。具体的实现方式本申请实施例不作限定。
在一个示例性实例中，检测模块201具体用于采用以下方式实现对第一数据的第一特征数据进行可靠性校验：对所述第一数据的第一特征数据进行以下至少之一：数据有效性检验、安全区间检验、全网贡献度检验、异常持续性检验。
上述进行可靠性校验的具体规则由专家根据业务经验、领域知识给出，可以由用户在页面进行相关设置或者导入可配置文件。
例如，对于上述从data1和data2提取的第一特征数据，对第一特征数据进行数据有效性校验也就是检测RRC建立连接请求数x eff是否大于第二预设阈值x eff_threshold，当RRC建立连接请求数大于第二预设阈值时，认为第一数据有效，继续对第一特征数据进行安全区间检验；当RRC建立连接请求数小于或等于第二预设阈值时，认为第一数据无效，那么第一数据正常(即第一校验结果为正常)，检验结束。其中，x eff_threshold由专家根据业务经验、领域知识给出，具体可以由用户在页面进行相关设置或者导入可配置文件。
对第一特征数据进行安全区间检验也就是检测RRC建立连接成功率x w是否在安全区间[x min,x max]内，当RRC建立连接成功率在安全区间内时，认为第一数据正常(即第一校验结果为正常)，检验结束；当RRC建立连接成功率不在安全区间内时，继续对第一特征数据进行全网贡献度检验。其中，安 全区间[x min,x max]由专家根据业务经验、领域知识给出，具体可以由用户在页面进行相关设置或者导入可配置文件。
对第一特征数据进行全网贡献度检验也就是检测RRC建立连接失败次数x ref是否大于第三预设阈值x ref_threshold，当RRC建立连接失败次数大于第三预设阈值时，确定第一校验结果为异常；当RRC建立连接失败次数小于或等于第三预设阈值时，继续对第一特征数据进行持续性检验。其中，x ref_threshold由专家根据业务经验、领域知识给出，具体可以由用户在页面进行相关设置或者导入可配置文件。
对第一特征进行持续性检验也就是对t w之前的一段时间的检验结果pre_res是否为异常，以及通过二阶差分的值判断是否有持续变坏的趋势，当t w之前的一段时间的检验结果均为异常，且有变坏的趋势时，确定第一校验结果为异常；当t w之前的一段时间的检验结果中的至少一个检验结果为正常，或者有变好的趋势时，确定第一校验结果为正常，检验结束。
在一个示例性实例中，确定模块202具体用于执行以下任意一种或多种：当所述第一检测结果为异常的异常检测算法中的一种或一种以上异常检测算法的可靠性大于或等于所述第一预设阈值时，确定所述第二检测结果为异常；当所述第一检测结果为异常的异常检测算法的可靠性均小于所述第一预设阈值时，将所述第一校验结果作为所述第二检测结果。
例如，上述对从data1和data2提取的第一特殊数据的检测中，当异常检测算法XGBoost的可靠性大于或等于第一预设阈值，且异常检测算法3-sigma的可靠性小于第一预设阈值时，确定第二检测结果为异常；当异常检测算法XGBoost和3-sigma的可靠性均小于第一预设阈值时，将第一校验结果作为第二检测结果。
在一个示例性实例中，异常检测算法的可靠性包括：异常检测算法的相似度比对、统计检验等。例如，所述异常检测算法的重合率。
在另一个示例性实例中，确定模块202还用于：对于每一种所述异常检测算法，计算所述异常检测算法的重合率。
在一个示例性实例中，确定模块202具体用于采用以下方式实现计算异 常检测算法的重合率：确定第一数量和第二数量的比值为所述重合率；其中，所述第一数量为在预设时间内采集的所有第二数据中，采用所述异常检测算法对第二数据的第二特征数据进行检测得到的第三检测结果和对所述第二数据的第二特征数据进行可靠性校验得到的第二校验结果相同的第二数据的数量；所述第二数量为在预设时间内采集的所有第二数据中，采用所述异常检测算法对第二数据的第二特征数据进行检测得到的第三检测结果为异常的第二数据的数量。
例如，最近一个月，所有小区，RRC建立连接成功率指标的检测结果。即依据过去一个月，所有小区，RRC建立连接成功率指标的所有样本中，采用异常检测算法进行检测得到的检测结果为异常的样本，假设一共有m个，并且记录各个异常检测算法在每个样本上的检测结果，以及对采集的数据进行的可靠性校验得到的检验结果，在此基础上，计算异常检测算法i的与检验结果一致的样本数量c i，则重合率为c i/m。
在本申请另一个实施例中，确定模块202还用于执行以下任意一个或多个：当所述第二检测结果为异常，且接收到用户的异常取消信息时，将所述第一数据作为标注为正常的样本加入标注样本库；当所述第二检测结果为异常，且接收到用户的异常确认信息时，将所述第一数据作为标注为异常的样本加入标注样本库中；其中，所述标注样本库中的标注为正常或异常的样本用于训练所述异常检测算法的监督学习模型，所述异常检测算法基于所述监督学习模型对所述第一数据的第一特征数据进行检测得到第一检测结果。将第一数据作为标注样本加入标注样本库中，随着时间的推移，标注样本库中积累的标注样本数目越来越多，使得基于监督学习或半监督学习的异常检测算法应用新增标注样本进行增量学习或周期性地重新训练监督学习模型，提高了监督学习模型的可靠性，从而提高了异常检测算法的检测准确率。
也就是说，当第二检测结果为异常时，显示异常信息，例如，异常信息包括异常数据(即第一数据)对应的对象名称“小区1”、发生时间t w、KPI名称“RRC建立连接成功率”、KPI值x w等。用户可以选择输入异常确认信息对异常进行确认，或者输入异常取消信息对异常进行取消；或者不进行任何操作。当用户输入异常确认信息时，将第一数据作为标注为异常的样本加 入标注样本库中；当用户输入异常取消信息时，将第一数据作为标注为正常的样本加入样本库中。
本申请实施例基于多种异常检测算法的检测结果以及可靠性的校验结果确定最终检测结果，由于可靠性的校验是基于专家业务经验、领域知识所做的可靠性校验，提高了检测结果的可靠性。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施例中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些组件或所有组件可以被实施为由处理器，如数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
虽然本申请实施例所揭露的实施方式如上，但所述的内容仅为便于理解本申请实施例而采用的实施方式，并非用以限定本申请实施例。任何本申请实施例所属领域内的技术人员，在不脱离本申请实施例所揭露的精神和范围的前提下，可以在实施的形式及细节上进行任何的修改与变化，但本申请实施例的专利保护范围，仍须以所附的权利要求书所界定的范围为准。

Claims (10)

  1. 一种异常检测方法，包括：
    对于N种异常检测算法中的每一种异常检测算法，采用所述异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果；其中，N为大于或等于1的整数；
    当一种或一种以上异常检测算法的第一检测结果为异常时，对所述第一数据的第一特征数据进行可靠性校验得到第一校验结果；
    根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果。
  2. 根据权利要求1所述的方法，其中，该方法还包括以下任意一个或多个：
    当所述第二检测结果为异常，且接收到用户的异常取消信息时，将所述第一数据作为标注为正常的样本加入标注样本库；
    当所述第二检测结果为异常，且接收到用户的异常确认信息时，将所述第一数据作为标注为异常的样本加入标注样本库中；
    其中，所述标注样本库中的标注为正常或异常的样本用于训练所述异常检测算法的监督学习模型，所述异常检测算法基于所述监督学习模型对所述第一数据的第一特征数据进行检测得到所述第一检测结果。
  3. 根据权利要求1或2所述的方法，其中，所述对第一数据的第一特征数据进行可靠性校验包括：对所述第一数据的第一特征数据进行以下至少之一：
    数据有效性检验、安全区间检验、全网贡献度检验、异常持续性检验。
  4. 根据权利要求1或2所述的方法，其中，所述根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果包括以下任意一种或多种：
    当所述第一检测结果为异常的异常检测算法中的一种或一种以上异常检 测算法的可靠性大于或等于所述第一预设阈值时，确定所述第二检测结果为异常；
    当所述第一检测结果为异常的异常检测算法的可靠性均小于所述第一预设阈值时，将所述第一校验结果作为所述第二检测结果。
  5. 根据权利要求1或2所述的方法，，其中，所述异常检测算法的可靠性包括：所述异常检测算法的重合率。
  6. 根据权利要求5所述的方法，其中，所述根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果之前，该方法还包括：
    对于每一种所述异常检测算法，计算所述异常检测算法的重合率。
  7. 根据权利要求6所述的方法，其中，所述计算异常检测算法的重合率包括：
    确定第一数量和第二数量的比值为所述重合率；
    其中，所述第一数量为在预设时间内采集的所有第二数据中，采用所述异常检测算法对所述第二数据的第二特征数据进行检测得到的第三检测结果和对所述第二数据的第二特征数据进行可靠性校验得到的第二校验结果相同的第二数据的数量；
    所述第二数量为在预设时间内采集的所有第二数据中，采用所述异常检测算法对所述第二数据的第二特征进行检测得到的第三检测结果为异常的第二数据的数量。
  8. 一种异常检测装置，包括处理器和计算机可读存储介质，所述计算机可读存储介质中存储有指令，其中，当所述指令被所述处理器执行时，实现如权利要求1～7任一项所述的异常检测方法。
  9. 一种计算机可读存储介质，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1～7任一项所述的异常检测方法的步骤。
  10. 一种异常检测装置，包括：
    检测模块，用于对于N种异常检测算法中的每一种异常检测算法，采用所述异常检测算法对采集的第一数据的第一特征数据进行检测得到第一检测结果；其中，N为大于或等于1的整数；当一种或一种以上异常检测算法的第一检测结果为异常时，对所述第一数据的第一特征数据进行可靠性校验得到第一校验结果；
    确定模块，用于根据第一检测结果为异常的异常检测算法的可靠性是否大于或等于第一预设阈值和第一校验结果确定第二检测结果。
PCT/CN2020/090936 2019-07-05 2020-05-19 一种异常检测方法和装置 Ceased WO2021004161A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/625,078 US11777824B2 (en) 2019-07-05 2020-05-19 Anomaly detection method and apparatus
EP20837813.3A EP3979416B1 (en) 2019-07-05 2020-05-19 Anomaly detection method and apparatus
JP2021578200A JP2022539578A (ja) 2019-07-05 2020-05-19 異常検出方法および装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910605053.9 2019-07-05
CN201910605053.9A CN112188534B (zh) 2019-07-05 2019-07-05 一种异常检测方法和装置

Publications (1)

Publication Number Publication Date
WO2021004161A1 true WO2021004161A1 (zh) 2021-01-14

Family

ID=73915346

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/090936 Ceased WO2021004161A1 (zh) 2019-07-05 2020-05-19 一种异常检测方法和装置

Country Status (5)

Country Link
US (1) US11777824B2 (zh)
EP (1) EP3979416B1 (zh)
JP (1) JP2022539578A (zh)
CN (1) CN112188534B (zh)
WO (1) WO2021004161A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113761025A (zh) * 2021-08-30 2021-12-07 上海电气国轩新能源科技有限公司 储能电站电池数据异常的检测方法、系统、设备及介质

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113535445B (zh) * 2021-01-06 2026-04-24 腾讯科技（深圳）有限公司 一种异常检测方法、设备及计算机可读存储介质
CN115242600A (zh) * 2021-04-23 2022-10-25 北京华为数字技术有限公司 一种网络异常检测方法及装置
US12386721B2 (en) * 2021-08-04 2025-08-12 Verizon Patent And Licensing Inc. Anomaly detection using user behavioral biometrics profiling method and apparatus
US11832119B2 (en) * 2021-08-31 2023-11-28 Verizon Patent And Licensing Inc. Identification of anomalous telecommunication service
CN115374851A (zh) * 2022-08-19 2022-11-22 北京市燃气集团有限责任公司 一种燃气数据异常检测方法及装置
KR102593981B1 (ko) * 2022-11-10 2023-10-25 주식회사 이노와이어리스 네트워크 로그 데이터의 결측치 처리 및 이를 통한 통신 결함 근원 분류 방법
CN117851907B (zh) * 2024-01-10 2024-06-11 山东省水利勘测设计院有限公司 一种基于物联网技术的水闸渗流监测方法
CN119829914B (zh) * 2024-01-24 2025-12-02 乌鲁木齐大数据产业发展投资有限公司 一种基于多维数据的数据处理方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1996888A (zh) * 2006-12-15 2007-07-11 华为技术有限公司 一种网络流量异常的检测方法及检测装置
CN105630885A (zh) * 2015-12-18 2016-06-01 国网福建省电力有限公司泉州供电公司 一种用电异常检测方法及系统
CN106357622A (zh) * 2016-08-29 2017-01-25 北京工业大学 基于软件定义网络的网络异常流量检测防御系统
CN107276851A (zh) * 2017-06-26 2017-10-20 中国信息安全测评中心 一种节点的异常检测方法、装置、网络节点及控制台
CN108322347A (zh) * 2018-02-09 2018-07-24 腾讯科技（深圳）有限公司 数据检测方法、装置、检测服务器及存储介质
US20180292806A1 (en) * 2017-04-06 2018-10-11 Renesas Electronics Corporation Abnormality detection system, semiconductor device manufacturing system and semiconductor device manufacturing method

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2002235516A1 (en) * 2001-01-08 2002-07-16 Vextec Corporation Method and apparatus for predicting failure in a system
US20060063521A1 (en) * 2004-09-21 2006-03-23 Benjamin Cheung Method of monitoring wireless network performance
JP5431235B2 (ja) * 2009-08-28 2014-03-05 株式会社日立製作所 設備状態監視方法およびその装置
US10592093B2 (en) * 2014-10-09 2020-03-17 Splunk Inc. Anomaly detection
US9872188B2 (en) * 2015-07-28 2018-01-16 Futurewei Technologies, Inc. Adaptive filtering based network anomaly detection
CN105471854B (zh) 2015-11-18 2019-06-28 国网智能电网研究院 一种基于多级策略的自适应边界异常检测方法
US10164991B2 (en) * 2016-03-25 2018-12-25 Cisco Technology, Inc. Hierarchical models using self organizing learning topologies
JP6805554B2 (ja) * 2016-06-02 2020-12-23 富士電機株式会社 監視装置及び監視方法
US10904276B2 (en) 2017-09-14 2021-01-26 Nokia Technologies Oy Systems and methods for anomaly detection
CN108762993A (zh) 2018-06-06 2018-11-06 山东超越数控电子股份有限公司 一种基于人工智能的虚拟机故障迁移方法与装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1996888A (zh) * 2006-12-15 2007-07-11 华为技术有限公司 一种网络流量异常的检测方法及检测装置
CN105630885A (zh) * 2015-12-18 2016-06-01 国网福建省电力有限公司泉州供电公司 一种用电异常检测方法及系统
CN106357622A (zh) * 2016-08-29 2017-01-25 北京工业大学 基于软件定义网络的网络异常流量检测防御系统
US20180292806A1 (en) * 2017-04-06 2018-10-11 Renesas Electronics Corporation Abnormality detection system, semiconductor device manufacturing system and semiconductor device manufacturing method
CN107276851A (zh) * 2017-06-26 2017-10-20 中国信息安全测评中心 一种节点的异常检测方法、装置、网络节点及控制台
CN108322347A (zh) * 2018-02-09 2018-07-24 腾讯科技（深圳）有限公司 数据检测方法、装置、检测服务器及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113761025A (zh) * 2021-08-30 2021-12-07 上海电气国轩新能源科技有限公司 储能电站电池数据异常的检测方法、系统、设备及介质

Also Published As

Publication number Publication date
CN112188534A (zh) 2021-01-05
EP3979416A1 (en) 2022-04-06
US20220278914A1 (en) 2022-09-01
EP3979416B1 (en) 2026-02-11
EP3979416A4 (en) 2022-08-03
JP2022539578A (ja) 2022-09-12
CN112188534B (zh) 2023-11-03
US11777824B2 (en) 2023-10-03

Similar Documents

Publication Publication Date Title
WO2021004161A1 (zh) 一种异常检测方法和装置
US12147893B2 (en) Anomaly detection in network topology
WO2022048168A1 (zh) 一种用于故障预测神经网络模型的训练方法与设备
US20150331063A1 (en) Dynamic Modeling and Resilience for Power Distribution
US8918345B2 (en) Network analysis system
AU2019275633B2 (en) System and method of automated fault correction in a network environment
US10158522B2 (en) Network monitor and method for event based prediction of radio network outages and their root cause
CN114169604A (zh) 性能指标的异常检测方法、异常检测装置、电子设备和存储介质
CN114138610B (zh) 一种故障处理方法及装置
CN110933115B (zh) 基于动态session的分析对象行为异常检测方法及装置
US20220046039A1 (en) Method, device, and computer program product for abnormality detection
CN104899657A (zh) 社团融合事件的预测方法
CN103634158B (zh) 一种snmp管理进程的压力测试方法和装置
CN115130616A (zh) 一种告警预测方法、装置、设备及介质
CN118041608A (zh) 一种基于攻击知识与多属性嵌入的apt网络攻击检测方法
CN117456417A (zh) 交通异常事件感知方法、系统、电子设备及存储介质
CN118555187A (zh) 一种告警关联分析方法、装置、电子设备及存储介质
CN110738414B (zh) 一种风险预测方法和装置及计算机可读存储介质
CN115509802A (zh) 网络设备的备份决策方法、装置、电子设备及存储介质
US20240144075A1 (en) Updating label probability distributions of data points
CN120803785A (zh) 业务性能故障诊断方法、装置、设备、介质和程序产品
CN121509211A (zh) 基于边缘ai驱动的实时链路异常检测与采样方法及装置
CN120508427A (zh) 智能计算中心的算力资源运行故障智能诊断方法及装置
CN121579257A (zh) 云原生环境故障预测方法、计算机设备、计算机可读存储介质
CN121396754A (zh) 一种分组传输网络故障识别定位方法、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20837813

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021578200

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2020837813

Country of ref document: EP

Effective date: 20211226

WWG Wipo information: grant in national office

Ref document number: 2020837813

Country of ref document: EP