WO2020009715A3 - Suivi des mouvements d'un utilisateur de façon à commander un modèle de squelette dans un système informatique - Google Patents

Suivi des mouvements d'un utilisateur de façon à commander un modèle de squelette dans un système informatique Download PDF

Info

Publication number
WO2020009715A3
WO2020009715A3 PCT/US2018/052092 US2018052092W WO2020009715A3 WO 2020009715 A3 WO2020009715 A3 WO 2020009715A3 US 2018052092 W US2018052092 W US 2018052092W WO 2020009715 A3 WO2020009715 A3 WO 2020009715A3
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
artificial neural
sensor module
computer system
control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2018/052092
Other languages
English (en)
Other versions
WO2020009715A2 (fr
Inventor
Viktor Vladimirovich ERIVANTCEV
Alexander Sergeevich LOBANOV
Alexey Ivanovich Kartashov
Daniil Olegovich Goncharov
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ward John P
Finch Technologies Ltd
Original Assignee
Ward John P
Finch Technologies Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US15/973,137 external-priority patent/US11474593B2/en
Priority claimed from US15/996,389 external-priority patent/US10416755B1/en
Application filed by Ward John P, Finch Technologies Ltd filed Critical Ward John P
Publication of WO2020009715A2 publication Critical patent/WO2020009715A2/fr
Publication of WO2020009715A3 publication Critical patent/WO2020009715A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/04—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by terrestrial means
    • G01C21/08—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by terrestrial means involving use of the magnetic field of the earth
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1121—Determining geometric values, e.g. centre of rotation or angular range of movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb using a particular sensing technique
    • A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb using a particular sensing technique using image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Automation & Control Theory (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • General Life Sciences & Earth Sciences (AREA)
  • Geology (AREA)
  • Computer Hardware Design (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un système comprenant des modules de capteur et un dispositif de calcul. Chaque module de capteur comporte une unité de mesure inertielle fixée à une partie d'un utilisateur. Un réseau neuronal artificiel est utilisé pour effectuer des prédictions d'orientations de parties de l'utilisateur sur la base des mesures associées aux modules basés sur l'inertie. Le réseau neuronal artificiel peut par exemple être formé de façon à prédire des orientations mesurées à l'aide d'un système de suivi optique sur la base d'orientations mesurées au moyen d'unités de mesure inertielle et/ou de façon à prédire des mesures d'orientations de certaines parties rigides dans une chaîne cinématique sur la base de mesures d'orientations d'autres parties rigides dans la chaîne cinématique. Les modules de capteur peuvent par exemple comprendre différents sous-ensembles qui partagent un module de capteur commun. De plus, le réseau neuronal artificiel peut être utilisé pour combiner différentes prédictions effectuées séparément à partir des différents sous-ensembles et destinées au module de capteur commun.
PCT/US2018/052092 2018-05-07 2018-09-20 Suivi des mouvements d'un utilisateur de façon à commander un modèle de squelette dans un système informatique Ceased WO2020009715A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US15/973,137 US11474593B2 (en) 2018-05-07 2018-05-07 Tracking user movements to control a skeleton model in a computer system
US15/973,137 2018-05-07
US15/996,389 2018-06-01
US15/996,389 US10416755B1 (en) 2018-06-01 2018-06-01 Motion predictions of overlapping kinematic chains of a skeleton model used to control a computer system

Publications (2)

Publication Number Publication Date
WO2020009715A2 WO2020009715A2 (fr) 2020-01-09
WO2020009715A3 true WO2020009715A3 (fr) 2020-02-20

Family

ID=68480436

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/052092 Ceased WO2020009715A2 (fr) 2018-05-07 2018-09-20 Suivi des mouvements d'un utilisateur de façon à commander un modèle de squelette dans un système informatique

Country Status (2)

Country Link
CN (1) CN110456902A (fr)
WO (1) WO2020009715A2 (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111340211B (zh) * 2020-02-19 2020-11-24 腾讯科技（深圳）有限公司 一种动作控制模型的训练方法、相关装置及存储介质
US11435185B2 (en) * 2020-02-21 2022-09-06 Microsoft Technology Licensing, Llc Systems and methods for deep learning-based pedestrian dead reckoning for exteroceptive sensor-enabled devices
US11237632B2 (en) * 2020-03-03 2022-02-01 Finch Technologies Ltd. Ring device having an antenna, a touch pad, and/or a charging pad to control a computing device based on user motions
WO2021211002A1 (fr) * 2020-04-13 2021-10-21 Общество С Ограниченной Ответственностью "Психо Технолоджи" Procédé de suivi optique-inertiel d'utilisateur
JP7435357B2 (ja) * 2020-08-18 2024-02-21 トヨタ自動車株式会社 動作状態監視システム、訓練支援システム、動作状態監視システムの制御方法、及び、制御プログラム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5515477A (en) * 1991-04-22 1996-05-07 Sutherland; John Neural networks
US20120214594A1 (en) * 2011-02-18 2012-08-23 Microsoft Corporation Motion recognition
US20180020978A1 (en) * 2016-07-25 2018-01-25 Patrick Kaifosh System and method for measuring the movements of articulated rigid bodies

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101689304A (zh) * 2007-07-10 2010-03-31 皇家飞利浦电子股份有限公司 对象动作捕捉系统和方法
GB0720412D0 (en) * 2007-10-18 2007-11-28 Melexis Nv Combined mems accelerometer and gyroscope
KR101483713B1 (ko) * 2008-06-30 2015-01-16 삼성전자 주식회사 모션 캡쳐 장치 및 모션 캡쳐 방법
US8279418B2 (en) * 2010-03-17 2012-10-02 Microsoft Corporation Raster scanning for depth detection
US9599634B2 (en) * 2012-12-03 2017-03-21 Vibrado Technologies, Inc. System and method for calibrating inertial measurement units
US9676098B2 (en) * 2015-07-31 2017-06-13 Heinz Hemken Data collection from living subjects and controlling an autonomous robot using the data

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5515477A (en) * 1991-04-22 1996-05-07 Sutherland; John Neural networks
US20120214594A1 (en) * 2011-02-18 2012-08-23 Microsoft Corporation Motion recognition
US20180020978A1 (en) * 2016-07-25 2018-01-25 Patrick Kaifosh System and method for measuring the movements of articulated rigid bodies

Also Published As

Publication number Publication date
WO2020009715A2 (fr) 2020-01-09
CN110456902A (zh) 2019-11-15

Similar Documents

Publication Publication Date Title
Rezaei et al. Unobtrusive human fall detection system using mmwave radar and data driven methods
EP4327776A3 (fr) Système de brosse à dents
AU2018289552A1 (en) Linking observed human activity on video to a user account
Harms et al. Ethos: Miniature orientation sensor for wearable human motion analysis
Leong et al. Accuracy of three Android-based pedometer applications in laboratory and free-living settings
MX350537B (es) Infraestructura de iluminación y modelo de ingresos.
EP2667778A2 (fr) Procédé et système de surveillance non invasive de paramètres biologiques et biochimiques d'un individu
GB2506560A (en) Method and apparatus for measuring expended energy
Xiahou et al. A feature-level fusion-based multimodal analysis of recognition and classification of awkward working postures in construction
MX2021000103A (es) Sensor de movimiento con correccion de deriva.
CN109549635B (zh) 人体体温动态在线测量方法及可穿戴设备
CN110974242B (zh) 一种用于可穿戴式设备的步态异常程度评价方法及该设备
EP3764899A4 (fr) Dispositif de mesure de la ventilation d'une personne comprenant la consommation d'oxygène
EP4270005A3 (fr) Système de surveillance comprenant une pluralité de dispositifs portables et une unité de commande
EP3444747A4 (fr) Module de reconnaissance biologique optique, dispositif d'affichage et équipement électronique
Sun Wearable glove gesture recognition based on fiber Bragg grating sensing using genetic algorithm-back propagation neural network
Zhang et al. FBG-SenseGlove: A digital twin data glove for underwater gesture recognition using deep learning
WO2020009715A2 (fr) Suivi des mouvements d'un utilisateur de façon à commander un modèle de squelette dans un système informatique
Pan et al. A sensor glove for the interaction with a nursing-care assistive robot
US20160258737A1 (en) Smart surface-mounted hybrid sensor system, method, and appratus for counting
US20250276216A1 (en) Generative-ai enhanced kinematic data measurement system
Mark et al. Design of an IoT-based Body Mass Index (BMI) Prediction Model
Gong et al. Multi-sensor and deep learning based ankle joint motion intention prediction under diverse gait speed conditions
CN116307718A (zh) 一种配网不停电作业风险预警方法、装置及计算机设备
Arjmandi et al. DeepCovPG: Deep-Learning-based Dynamic Covariance Prediction in Pose Graphs for Ultra-Wideband-Aided UAV Positioning

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18925072

Country of ref document: EP

Kind code of ref document: A2

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18925072

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 18925072

Country of ref document: EP

Kind code of ref document: A2

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16/07/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18925072

Country of ref document: EP

Kind code of ref document: A2