EP3746938A4 - SYSTEM AND METHOD FOR CLASSIFYING NEUROMORPHIC VISUAL ACTIVITY USING FOVEAL DETECTION AND CONTEXTUAL FILTERING - Google Patents

SYSTEM AND METHOD FOR CLASSIFYING NEUROMORPHIC VISUAL ACTIVITY USING FOVEAL DETECTION AND CONTEXTUAL FILTERING Download PDF

Info

Publication number: EP3746938A4
Authority: EP; European Patent Office
Prior art keywords: classifying; visual activity; contextual filtering; neuromorphic; foveal
Prior art date: 2018-01-30
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.): Pending

Application number

EP19748018.9A

Other languages

German (de)

French (fr)

Other versions

EP3746938A2 (en

Inventor

Deepak Khosla

Ryan M. UHLENBROCK

Yang Chen

Huapeng SU

Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)

HRL Laboratories LLC

Original Assignee

HRL Laboratories LLC

Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)

2018-01-30

Filing date

2019-01-14

Publication date

2021-10-06

2018-01-30 Priority claimed from US15/883,822 external-priority patent/US11055872B1/en

2018-04-06 Priority claimed from US15/947,032 external-priority patent/US10997421B2/en

2019-01-14 Application filed by HRL Laboratories LLC filed Critical HRL Laboratories LLC

2020-12-09 Publication of EP3746938A2 publication Critical patent/EP3746938A2/en

2021-10-06 Publication of EP3746938A4 publication Critical patent/EP3746938A4/en

Status Pending legal-status Critical Current

Links

Classifications

- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/09—Supervised learning
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition

Landscapes

Engineering & Computer Science (AREA)
Theoretical Computer Science (AREA)
Physics & Mathematics (AREA)
General Physics & Mathematics (AREA)
Software Systems (AREA)
Computing Systems (AREA)
Artificial Intelligence (AREA)
Mathematical Physics (AREA)
Data Mining & Analysis (AREA)
Evolutionary Computation (AREA)
General Engineering & Computer Science (AREA)
Biomedical Technology (AREA)
Molecular Biology (AREA)
General Health & Medical Sciences (AREA)
Computational Linguistics (AREA)
Biophysics (AREA)
Life Sciences & Earth Sciences (AREA)
Health & Medical Sciences (AREA)
Multimedia (AREA)
Computer Vision & Pattern Recognition (AREA)
Medical Informatics (AREA)
Image Analysis (AREA)

EP19748018.9A 2018-01-30 2019-01-14 SYSTEM AND METHOD FOR CLASSIFYING NEUROMORPHIC VISUAL ACTIVITY USING FOVEAL DETECTION AND CONTEXTUAL FILTERING Pending EP3746938A4 (en)

Applications Claiming Priority (4)

Application Number	Priority Date	Filing Date	Title
US15/883,822 US11055872B1 (en)	2017-03-30	2018-01-30	Real-time object recognition using cascaded features, deep learning and multi-target tracking
US201862642959P	2018-03-14	2018-03-14
US15/947,032 US10997421B2 (en)	2017-03-30	2018-04-06	Neuromorphic system for real-time visual activity recognition
PCT/US2019/013513 WO2019152177A2 (en)	2018-01-30	2019-01-14	System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering

Publications (2)

Publication Number	Publication Date
EP3746938A2 EP3746938A2 (en)	2020-12-09
EP3746938A4 true EP3746938A4 (en)	2021-10-06

Family

ID=67479383

Family Applications (1)

Application Number	Title	Priority Date	Filing Date
EP19748018.9A Pending EP3746938A4 (en)	2018-01-30	2019-01-14	SYSTEM AND METHOD FOR CLASSIFYING NEUROMORPHIC VISUAL ACTIVITY USING FOVEAL DETECTION AND CONTEXTUAL FILTERING

Country Status (3)

Country	Link
EP (1)	EP3746938A4 (en)
CN (1)	CN111566661B (en)
WO (1)	WO2019152177A2 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
CN111652128B (en) *	2020-06-02	2023-09-01	浙江大华技术股份有限公司	A safety monitoring method, system and storage device for high-altitude electric power work
US12327345B2 (en) *	2021-08-13	2025-06-10	Baidu Usa Llc	Vision-based safety monitoring and/or activity analysis
US12165399B2 (en) *	2021-12-29	2024-12-10	Halliburton Energy Services, Inc.	Systems and methods to determine an activity associated with an object of interest
US11776247B2 (en)	2022-01-07	2023-10-03	Tomahawk Robotics	Classification parallelization architecture
KR102829715B1 (en) *	2022-03-15	2025-07-04	연세대학교 산학협력단	Multiscale object detection device and method

Citations (1)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
WO2008098188A2 (en) *	2007-02-08	2008-08-14	Behavioral Recognition Systems, Inc.	Behavioral recognition system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US8116527B2 (en) *	2009-10-07	2012-02-14	The United States Of America As Represented By The Secretary Of The Army	Using video-based imagery for automated detection, tracking, and counting of moving objects, in particular those objects having image characteristics similar to background
US9008366B1 (en)	2012-01-23	2015-04-14	Hrl Laboratories, Llc	Bio-inspired method of ground object cueing in airborne motion imagery
WO2013152205A1 (en) *	2012-04-05	2013-10-10	Augmented Vision Inc.	Wide-field of view (fov) imaging devices with active foveation capability
US9230302B1 (en) *	2013-03-13	2016-01-05	Hrl Laboratories, Llc	Foveated compressive sensing system
US9147255B1 (en)	2013-03-14	2015-09-29	Hrl Laboratories, Llc	Rapid object detection by combining structural information from image segmentation with bio-inspired attentional mechanisms
US9959468B2 (en) *	2015-11-06	2018-05-01	The Boeing Company	Systems and methods for object tracking and classification

2019
- 2019-01-14 WO PCT/US2019/013513 patent/WO2019152177A2/en not_active Ceased
- 2019-01-14 CN CN201980006835.9A patent/CN111566661B/en active Active
- 2019-01-14 EP EP19748018.9A patent/EP3746938A4/en active Pending

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
WO2008098188A2 (en) *	2007-02-08	2008-08-14	Behavioral Recognition Systems, Inc.	Behavioral recognition system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ARUN HAMPAPUR ET AL: "Smart Video Surveillance [Exploring the concept of multiscale spatiotemporal tracking]", IEEE SIGNAL PROCESSING MAGAZINE, 1 March 2005 (2005-03-01), XP055347119, Retrieved from the Internet <URL:http://www.andrewsenior.com/papers/HampapurIEEESP05.pdf> [retrieved on 20170217], DOI: 10.1109/MSP.2005.1406476 *
PEIXOTO P ET AL: "A surveillance system combining peripheral and foveated motion tracking", PATTERN RECOGNITION, 1998. PROCEEDINGS. FOURTEENTH INTERNATIONAL CONFE RENCE ON BRISBANE, QLD., AUSTRALIA 16-20 AUG. 1998, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, US, vol. 1, 16 August 1998 (1998-08-16), pages 574 - 577, XP010297420, ISBN: 978-0-8186-8512-5, DOI: 10.1109/ICPR.1998.711208 *
See also references of WO2019152177A2 *

Also Published As

Publication number	Publication date
WO2019152177A2 (en)	2019-08-08
CN111566661A (en)	2020-08-21
WO2019152177A3 (en)	2019-10-10
CN111566661B (en)	2023-11-17
EP3746938A2 (en)	2020-12-09

Legal Events

Date	Code	Title	Description
2019-08-10	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE
2020-11-06	PUAI	Public reference made under article 153(3) epc to a published international application that has entered the european phase	Free format text: ORIGINAL CODE: 0009012
2020-11-06	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE
2020-12-09	17P	Request for examination filed	Effective date: 20200403
2020-12-09	AK	Designated contracting states	Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
2020-12-09	AX	Request for extension of the european patent	Extension state: BA ME
2021-05-05	DAV	Request for validation of the european patent (deleted)
2021-05-05	DAX	Request for extension of the european patent (deleted)
2021-08-30	REG	Reference to a national code	Ref country code: DE Ref legal event code: R079 Free format text: PREVIOUS MAIN CLASS: G06K0009000000 Ipc: G06N0003020000
2021-10-06	A4	Supplementary search report drawn up and despatched	Effective date: 20210903
2021-10-06	RIC1	Information provided on ipc code assigned before grant	Ipc: G06K 9/00 20060101ALI20210830BHEP Ipc: G06N 3/02 20060101AFI20210830BHEP
2023-07-05	P01	Opt-out of the competence of the unified patent court (upc) registered	Effective date: 20230525
2023-08-25	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: EXAMINATION IS IN PROGRESS
2023-09-27	17Q	First examination report despatched	Effective date: 20230828

Publication	Publication Date	Title
EP3746938A4 (en)	2021-10-06	SYSTEM AND METHOD FOR CLASSIFYING NEUROMORPHIC VISUAL ACTIVITY USING FOVEAL DETECTION AND CONTEXTUAL FILTERING
EP3920781A4 (en)	2022-10-12	EPILEPSY SEIZURE DETECTION METHOD AND SYSTEM
EP3299976A4 (en)	2019-01-16	METHOD AND SYSTEM FOR CLASSIFYING GENE EXPRESSION DATA
EP3901816A4 (en)	2022-09-14	GLAUCOMA IMAGE RECOGNITION METHOD AND DEVICE AND SCREENING SYSTEM
EP3633612A4 (en)	2020-06-03	IMAGE SEGMENTATION METHOD AND SYSTEM
EP3846679A4 (en)	2022-05-04	STRUCTURAL HEART DISEASE SCREENING DEVICE, METHOD AND SYSTEM
EP3547252A4 (en)	2019-12-04	SYSTEM AND METHOD FOR PROCESSING MULTI-MODAL IMAGES
EP3534328A4 (en)	2019-11-06	IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, FACIAL RECOGNITION SYSTEM, PROGRAM, AND RECORDING MEDIUM
EP4075374A4 (en)	2023-02-01	IMAGE PROCESSING METHOD AND APPARATUS, AND IMAGE PROCESSING SYSTEM
EP3745359A4 (en)	2021-11-03	IMAGE RECOGNITION SYSTEM AND METHOD
EP3933575A4 (en)	2022-05-18	IMAGE PROCESSING PROCESS AND COMPUTER SYSTEM
EP3803317A4 (en)	2022-02-16	PARTICLE CONCENTRATION ANALYSIS SYSTEM AND METHOD
EP3971762A4 (en)	2022-07-27	IMAGE PROCESSING METHOD, DEVICE AND SYSTEM
EP3568778A4 (en)	2019-11-27	SYSTEM AND METHOD FOR CONTEXTUAL INTELLIGENCE
EP3648462A4 (en)	2020-06-17	IMAGE PROCESSING METHOD, DEVICE AND SYSTEM
EP3507768A4 (en)	2020-02-26	MODEL-BASED IMAGE ANALYSIS SYSTEM AND METHOD
EP3867873A4 (en)	2022-01-05	IMAGE PROCESSING METHOD AND SYSTEM
EP3627442A4 (en)	2020-05-20	IMAGE PROCESSING METHOD AND SYSTEM
EP3961562A4 (en)	2023-01-18	IMAGE PROCESSING METHOD, DEVICE AND SYSTEM
EP3663905A4 (en)	2020-06-17	INFORMATION PROCESSING DEVICE, SPEECH RECOGNITION SYSTEM AND INFORMATION PROCESSING METHOD
EP3756160A4 (en)	2021-04-21	RAPID OBJECT DETECTION SYSTEM AND METHOD
EP3857619A4 (en)	2021-12-08	QUBIT SYSTEM AND METHOD FOR TOPOLOGICAL PROTECTION
EP3896961A4 (en)	2022-01-19	IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING SYSTEM
EP3651471A4 (en)	2020-05-13	IMAGE PROCESSING SYSTEM AND METHOD
EP3896962A4 (en)	2022-01-26	IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING SYSTEM