PL3916556T3 - Sposób i system do wykrywania anomalii w potoku danych - Google Patents

Sposób i system do wykrywania anomalii w potoku danych

Info

Publication number
PL3916556T3
PL3916556T3 PL20315288.9T PL20315288T PL3916556T3 PL 3916556 T3 PL3916556 T3 PL 3916556T3 PL 20315288 T PL20315288 T PL 20315288T PL 3916556 T3 PL3916556 T3 PL 3916556T3
Authority
PL
Poland
Prior art keywords
data pipeline
detecting anomalies
anomalies
detecting
pipeline
Prior art date
Application number
PL20315288.9T
Other languages
English (en)
Inventor
Olivier Nicol
Laurent Parmentier
Pauline Wauquier
Original Assignee
Ovh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ovh filed Critical Ovh
Publication of PL3916556T3 publication Critical patent/PL3916556T3/pl

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751Error or fault detection not based on redundancy
    • G06F11/0754Error or fault detection not based on redundancy by exceeding limits
    • G06F11/076Error or fault detection not based on redundancy by exceeding limits by exceeding a count or rate limit, e.g. word- or bit count limit
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0709Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3447Performance evaluation by modeling
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3452Performance evaluation by statistical analysis
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • G06F17/18Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0766Error or fault reporting or storing
    • G06F11/0769Readable error formats, e.g. cross-platform generic formats, human understandable formats

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computational Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Probability & Statistics with Applications (AREA)
  • Operations Research (AREA)
  • Computer Security & Cryptography (AREA)
  • Algebra (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
PL20315288.9T 2020-05-29 2020-05-29 Sposób i system do wykrywania anomalii w potoku danych PL3916556T3 (pl)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP20315288.9A EP3916556B1 (en) 2020-05-29 2020-05-29 Method and system for detecting anomalies in a data pipeline

Publications (1)

Publication Number Publication Date
PL3916556T3 true PL3916556T3 (pl) 2023-07-10

Family

ID=71575319

Family Applications (1)

Application Number Title Priority Date Filing Date
PL20315288.9T PL3916556T3 (pl) 2020-05-29 2020-05-29 Sposób i system do wykrywania anomalii w potoku danych

Country Status (6)

Country Link
US (1) US11288260B2 (pl)
EP (1) EP3916556B1 (pl)
CN (1) CN113742118B (pl)
CA (1) CA3117003A1 (pl)
DK (1) DK3916556T3 (pl)
PL (1) PL3916556T3 (pl)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113626447B (zh) * 2021-10-12 2022-02-22 民航成都信息技术有限公司 一种民航数据管理平台及方法
US12530524B1 (en) * 2022-11-28 2026-01-20 Amazon Technologies, Inc. Enriching dataset metadata with business semantics for natural language answering
US12367088B2 (en) * 2023-12-08 2025-07-22 Bank Of America Corporation System and method for automated data pipeline degradation detection
CN117648232B (zh) * 2023-12-11 2024-05-24 武汉天宝莱信息技术有限公司 一种应用程序数据监控方法、装置及存储介质
CN119782030A (zh) * 2025-03-10 2025-04-08 长安通信科技有限责任公司 数据抓取架构故障诊断的方法、装置、电子设备及存储介质

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6597907B1 (en) * 2000-05-05 2003-07-22 Ericsson Inc. Detection of a deadlocked resource condition in a pool of shared resources
AT10073U9 (de) * 2008-01-14 2009-02-15 Avl List Gmbh Verfahren und vorrichtung zur analyse und bewertung von messdaten eines messsystems
US8175843B2 (en) * 2008-11-14 2012-05-08 Boehringer Ingelheim Pharma Gmbh & Co. Kg Computer-implemented methods for evaluating, summarizing and presenting data on stability of drug substances and drug products and software-modified computers for such methods
US8370079B2 (en) * 2008-11-20 2013-02-05 Pacific Biosciences Of California, Inc. Algorithms for sequence determination
CN104915846A (zh) * 2015-06-18 2015-09-16 北京京东尚科信息技术有限公司 一种电子商务时间序列数据的异常检测方法及系统
US9699205B2 (en) * 2015-08-31 2017-07-04 Splunk Inc. Network security system
US20170364818A1 (en) * 2016-06-17 2017-12-21 Business Objects Software Ltd. Automatic condition monitoring and anomaly detection for predictive maintenance
US10360214B2 (en) * 2017-10-19 2019-07-23 Pure Storage, Inc. Ensuring reproducibility in an artificial intelligence infrastructure
US11403647B2 (en) * 2018-05-10 2022-08-02 Case Western Reserve University Systems and methods for data analytics for virtual energy audits and value capture assessment of buildings
US11042145B2 (en) * 2018-06-13 2021-06-22 Hitachi, Ltd. Automatic health indicator learning using reinforcement learning for predictive maintenance
US11107113B2 (en) * 2018-09-12 2021-08-31 Brandfolder, Inc. Multitrack performance scoring for assets comprising digital media
EP3623964A1 (en) * 2018-09-14 2020-03-18 Verint Americas Inc. Framework for the automated determination of classes and anomaly detection methods for time series
CN109711440B (zh) * 2018-12-13 2022-02-08 新奥数能科技有限公司 一种数据异常检测方法和装置
DE102018133196A1 (de) * 2018-12-20 2020-06-25 Carl Zeiss Microscopy Gmbh Bildbasierte wartungsvorhersage und detektion von fehlbedienungen
US11488010B2 (en) * 2018-12-29 2022-11-01 Northeastern University Intelligent analysis system using magnetic flux leakage data in pipeline inner inspection
CN111080502B (zh) * 2019-12-17 2023-09-08 清华苏州环境创新研究院 一种区域企业数据异常行为的大数据识别方法

Also Published As

Publication number Publication date
EP3916556B1 (en) 2023-05-03
EP3916556A1 (en) 2021-12-01
CN113742118A (zh) 2021-12-03
DK3916556T3 (da) 2023-05-30
CA3117003A1 (en) 2021-11-29
US20210374123A1 (en) 2021-12-02
CN113742118B (zh) 2024-04-26
US11288260B2 (en) 2022-03-29

Similar Documents

Publication Publication Date Title
PL3916556T3 (pl) Sposób i system do wykrywania anomalii w potoku danych
GB2492667B (en) Data processing method and system for checking pipeline leakage
PL2052482T3 (pl) Sposób i urządzenie do wykrywania błędów w bloku danych
GB2612167B (en) Systems and methods for synchronizing data processing in a cellular modem
PL4196417T3 (pl) System i sposób wykrywania anomalii w torach w trójwymiarowym systemie magazynowania
EP2080119A4 (en) SYSTEM AND METHOD FOR DISCOVERING ANOMALIES IN MARKET DATA
GB202408698D0 (en) System and method for detecting ethical bias in machine learning models
EP4187472A4 (en) METHOD AND DEVICE FOR DETECTING FALSE TRANSACTION ORDERS
KR102558503B9 (ko) 데이터 무결성 검증 방법 및 장치
GB2592704B (en) A pipe testing apparatus and method
GB201804063D0 (en) System and method for locating leaks in pipelines
GB201005869D0 (en) A method, system and apparatus for detecting defects and/or blockages in conduits
GB2578497B (en) Apparatus and method for communicating data in a multi-tennant computer system
EP4016025A4 (en) SYSTEM AND METHOD FOR CONTACTLESS DETERMINATION OF MECHANICAL VOLTAGES IN A CONDUIT
GB2616750B (en) System and method for measuring sag in a fluid
GB202320123D0 (en) A system and a method for detecting and characterizing a defect in an object using guided wave inspection
GB202313693D0 (en) A computer-implemented method for performing a 3D object detection using ransac and a device thereof
GB202017272D0 (en) Inspection apparatus, system and method for use in assessing the integrity of a pipe assembly
NO346137B1 (en) Method, computer program and system for detecting changes and moving objects in a video view
GB202300760D0 (en) Method and device for processing data associated with a model
GB2600761B (en) A pipe testing apparatus and method
GB2608430B (en) A data processing apparatus and method for handling stalled data
GB202102585D0 (en) A pipeline inspection system and method
PL3885731T3 (pl) System przewodów gazu płynnego, sposób wykrywania wycieku i program komputerowy
IL283055A (en) A system and method for identifying defects in a gemstone