WO2017143341A1 - Systèmes et procédés permettant de surveiller des objets lors d'événements sportifs - Google Patents

Systèmes et procédés permettant de surveiller des objets lors d'événements sportifs Download PDF

Info

Publication number
WO2017143341A1
WO2017143341A1 PCT/US2017/018725 US2017018725W WO2017143341A1 WO 2017143341 A1 WO2017143341 A1 WO 2017143341A1 US 2017018725 W US2017018725 W US 2017018725W WO 2017143341 A1 WO2017143341 A1 WO 2017143341A1
Authority
WO
WIPO (PCT)
Prior art keywords
drone
player
image
sensor
processing system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2017/018725
Other languages
English (en)
Inventor
John Carter
Alan W. Marty
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Pillar Vision Inc
Original Assignee
Pillar Vision Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pillar Vision Inc filed Critical Pillar Vision Inc
Priority to CN202110901640.XA priority Critical patent/CN113599788B/zh
Priority to CN201780024691.0A priority patent/CN109069903B/zh
Publication of WO2017143341A1 publication Critical patent/WO2017143341A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0062—Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U80/00—Transport or storage specially adapted for UAVs
    • B64U80/60—Transport or storage specially adapted for UAVs by wearable objects, e.g. garments or helmets
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0094—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/10—Rotorcrafts
    • B64U10/13—Flying platforms
    • B64U10/14—Flying platforms with four distinct rotor axes, e.g. quadcopters
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/20—Remote controls
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30221—Sports video; Sports image
    • G06T2207/30224—Ball; Puck
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30221—Sports video; Sports image
    • G06T2207/30228—Playing field
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Definitions

  • FIG. 13 shows an exemplary sensing system, such as is depicted by FIG. 1 1 , mounted on a pole above a basketball goal.
  • the present disclosure generally pertains to systems and methods for monitoring objects at sporting events or other types of events.
  • a system in accordance with one embodiment of the present disclosure uses a wearable drone that has at least one camera or other sensor for capturing or otherwise sensing data.
  • the wearable drone may be detached from its user, and it may hover or otherwise fly within a certain position of an object to be monitored. While flying, the drone's sensor may be used to capture information, such as performance data or images, of the object during the sporting event.
  • FIG. 1 depicts an exemplary system 10 for monitoring objects at sporting events or other types of events.
  • the system 10 comprises a movable drone 15 that is capable of flying to a desired location in order to monitor objects.
  • a movable drone 15 that is capable of flying to a desired location in order to monitor objects.
  • processor 128 such as a digital signal processor (DSP) or a central processing unit (CPU), that communicates to and drives the other elements within the system 46 via a local interface 131 , which can include at least one bus.
  • DSP digital signal processor
  • CPU central processing unit
  • the processor 128 may be programmed with instructions of the control logic 122, when such control logic 122 is implemented in software, for execution of the instructions according to techniques known in the art.
  • the flight control system 52 may comprise any number and types of airfoils (e.g., wings or rotors) for providing lift.
  • FIG. 3 shows a conventional drone 65 that can be programmed or otherwise controlled or modified to implement a drone 15 for monitoring athletes, as described herein.
  • the drone 65 of FIG. 3 has a body 66 from which four flexible arms 69 extend. At the end of each arm 69 is a propeller 73 that spins under the direction and control of a flight controller (not shown) housed within the body 66 in order to generate lift and to propel the drone 65 through air.
  • the drone 65 also has a camera 77 (e.g., a video camera or other type of camera) for capturing images as the drone 65 flies.
  • a camera 77 e.g., a video camera or other type of camera
  • the drone 15 may be configured to capture images of the
  • the drone 15 may capture a video feed and wirelessly transmit the video feed so that it can be displayed for entertainment or other purposes.
  • the video feed may be included in a television or other type of video broadcast of the game.
  • the video feed may also be used for other purposes.
  • the video feed may be stored and later displayed to the quarterback or coach for assisting with the quarterback's training.
  • processing system 46 is configured to analyze the images captured by the drone 15 and to identify the quarterback within the images.
  • the processing system 46 also identifies the football within the images based on the football's color and shape. Based on the images, the processing system 46 is configured to determine when the football is thrown by the quarterback. There are various techniques that can be used to determine when the football is thrown.
  • FIG. 7B shows an exemplary image 200 that may be displayed to the user on an output interface 50 of a mobile device, such as a smartphone.
  • the image 200 shows a green 205 having a hole 207 with a flag 21 1 positioned in the hole 207 to mark the placement of the hole 207.
  • the image 200 also shows a golf ball 215 resting on the green 205 and a virtual curve 218 representing the optimal path determined by the processing system 46 for the user's putt.
  • images captured by the sensor 44 of the drone 15 or by the sensor 44 residing at other locations may be displayed to the user wearing augmented- reality HMD 216.
  • the processing system 46 is preferably aware of the approximate location of the sensor 44 and the approximate location of the HMD 216. Using these locations, the processing system 46 can be configured to adjust the images captured by the sensor 44 so that they appear as if they have been captured by the HMD 216.
  • the processing system 46 may reside on the HMD 216 or otherwise be situated on the user such that wireless communication of the HMD's location information is unnecessary.
  • RF radio frequency
  • the drone 15 may be positioned at a certain location relative to a
  • the drone 15 may be positioned a certain distance (e.g., about 10 feet) behind the player at a certain height (e.g. , about 10 feet) above the playing surface.
  • the drone 15 may be positioned on the side of the tennis court (e.g., the drone 15 may move up and down the sidelines).
  • the drone 15 may be moved based on the captured images to keep the drone 15 at a certain position relative to the player of interest, and the processing system 46 may be configured to collect ball and player data based on the captured images, to determine ball trajectory, and to determine various parameters indicative of the player performance.
  • the video images captured by the wide-angle camera may be used to provide a virtual reality or augmented reality environment to a user. From such video images, the user may be able to view the sporting event from the perspective of the athlete that is wearing the wide-angle camera. As an example, when the wide-angle camera is worn by a quarterback in a football game, the user viewing the video feed may be able to see, from the perspective of the quarterback, defensive lineman rushing the quarterback and receivers running routes. Using multiple rings of component cameras may help to increase the vertical viewing angle for the captured images. Also, using a wide-angle camera, it is possible to see lineman approaching the quarterback from many directions, such as in front of and behind (the quarterback's blindside) the quarterback.
  • the completion indicator may also be based on other factors. As an example, the completion indicator may be lowered for a defender that is rated higher than other defenders, as indicated by their respective defender ratings. The completion indicator may also be controlled based on how the defender previously performed in similar situations. As an example, during gameplay, the processing system 46 may track and store attribute data of a defender for each pass that he defends. Such data may include the separation distance between the defender and receiver at the time a pass is thrown and/or other data, such as whether the pass was completed, whether the defender is guarding the receiver's inside shoulder or outside shoulder, and the maximum speed difference between the defender and the receiver.
  • the processing system 46 may transmit to a mobile device on the quarterback data indicative of the selected receivers to help the quarterback to select a receiver for receiving a pass during gameplay.
  • headgear a display device integrated with his helmet or eyeglasses worn by the quarterback
  • the display device may project an image into the quarterback's eyes or on eyeglasses worn by the quarterback such that a graphical element appears to be superimposed on the selected receiver or receivers thereby indicating which receivers have been determined to be optimal for receiving a pass.
  • the control logic 122 is configured to analyze the depth map 350 to identify planes. That is, the control logic 122 is configured to identify at least one set of depth pixels that are within the same plane.
  • planes can be identified by finding groups of contiguous depth pixels having similar color values.
  • the surface geometry of an object within view of the camera 351 can be analyzed based on the depth pixels in order to identify depth pixels that are within the same plane.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Transportation (AREA)
  • Astronomy & Astrophysics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Automation & Control Theory (AREA)
  • Studio Devices (AREA)
  • Image Analysis (AREA)
  • Toys (AREA)

Abstract

L'invention concerne un système permettant de surveiller des objets lors d'événements sportifs ou d'autres types d'événements, ledit système utilisant un drone portable qui comporte au moins un appareil de prise de vues ou un capteur pour capturer ou détecter des données. Lorsque le drone doit être utilisé à des fins de surveillance, telles que la surveillance d'un objet lors d'un événement sportif, le drone portable peut être détaché de son utilisateur, et il peut effectuer un vol stationnaire ou voler au sein d'une zone autour d'un certain emplacement d'un objet à surveiller. Pendant le vol, le capteur du drone peut être utilisé pour capturer des informations, telles que des données de performance ou des images, relatives à l'objet au cours de l'événement sportif.
PCT/US2017/018725 2016-02-19 2017-02-21 Systèmes et procédés permettant de surveiller des objets lors d'événements sportifs Ceased WO2017143341A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110901640.XA CN113599788B (zh) 2016-02-19 2017-02-21 用于在体育事件中监测运动员表现的系统和方法
CN201780024691.0A CN109069903B (zh) 2016-02-19 2017-02-21 用于监测体育事件中的对象的系统和方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662297528P 2016-02-19 2016-02-19
US62/297,528 2016-02-19

Publications (1)

Publication Number Publication Date
WO2017143341A1 true WO2017143341A1 (fr) 2017-08-24

Family

ID=59626334

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/018725 Ceased WO2017143341A1 (fr) 2016-02-19 2017-02-21 Systèmes et procédés permettant de surveiller des objets lors d'événements sportifs

Country Status (2)

Country Link
CN (2) CN113599788B (fr)
WO (1) WO2017143341A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112802051A (zh) * 2021-02-02 2021-05-14 新华智云科技有限公司 基于神经网络的篮球投篮曲线的拟合方法及系统
US20230241455A1 (en) * 2022-01-31 2023-08-03 Breakout Hitting Llc Training system and method of using same
AU2025204327B1 (en) * 2025-06-11 2026-03-26 Jouf University Integrated drone-based motion capture and smart insole system for real-time biomechanical analysis in outdoor environments

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110141845A (zh) * 2019-06-10 2019-08-20 湖南大狗科技有限公司 一种基于无人机的赛车轨道安全管理监测系统
WO2021031159A1 (fr) * 2019-08-21 2021-02-25 深圳市大疆创新科技有限公司 Procédé de photographie d'appariement, dispositif électronique, véhicule aérien sans pilote et support de stockage
CN111408118A (zh) * 2020-03-23 2020-07-14 李志刚 一种体育设备
CN113546386A (zh) * 2020-09-30 2021-10-26 深圳华锐互动科技有限公司 一种球类运动训练板组件
CN112933574B (zh) * 2021-01-27 2022-05-24 北京驭胜晏然体育文化有限公司 一种多联机的室内滑雪比赛控制方法、系统和可读存储介质
CN113781523B (zh) * 2021-09-13 2024-04-26 浙江大学 一种足球检测跟踪方法及装置、电子设备、存储介质
US20250229158A1 (en) * 2022-03-25 2025-07-17 Mitsubishi Electric Corporation Golf support system and computer readable medium
WO2023218627A1 (fr) * 2022-05-13 2023-11-16 三菱電機株式会社 Système d'aide au golf, procédé d'aide au golf et programme d'aide au golf

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080059065A1 (en) * 2006-09-05 2008-03-06 Honeywell International Inc. Method and system for navigation of an unmanned aerial vehicle in an urban environment
US20100097442A1 (en) * 2008-10-16 2010-04-22 Peter Lablans Controller in a Camera for Creating a Panoramic Image
US7843470B2 (en) * 2005-01-31 2010-11-30 Canon Kabushiki Kaisha System, image processing apparatus, and information processing method
US20140288682A1 (en) * 2008-12-05 2014-09-25 Nike, Inc. Athletic Performance Monitoring Systems and Methods in a Team Sports Environment
WO2015200209A1 (fr) * 2014-06-23 2015-12-30 Nixie Labs, Inc. Véhicules aériens sans pilote portatifs, véhicules aériens sans pilote à lancement commandé, et systèmes et procédés associés

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10360685B2 (en) * 2007-05-24 2019-07-23 Pillar Vision Corporation Stereoscopic image capture with performance outcome prediction in sporting environments
KR101386793B1 (ko) * 2007-09-21 2014-04-21 플레이데이타, 엘엘씨 개체 위치 및 움직임 검출 시스템 및 그 방법
US8355042B2 (en) * 2008-10-16 2013-01-15 Spatial Cam Llc Controller in a camera for creating a panoramic image
AU2010229693B2 (en) * 2009-03-27 2014-04-03 Russell Brands, Llc Monitoring of physical training events
JP2014165706A (ja) * 2013-02-26 2014-09-08 Sony Corp 信号処理装置および記憶媒体
US20150373306A1 (en) * 2014-06-20 2015-12-24 OnDeck Digital LLC Real-time video capture of field sports activities

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7843470B2 (en) * 2005-01-31 2010-11-30 Canon Kabushiki Kaisha System, image processing apparatus, and information processing method
US20080059065A1 (en) * 2006-09-05 2008-03-06 Honeywell International Inc. Method and system for navigation of an unmanned aerial vehicle in an urban environment
US20100097442A1 (en) * 2008-10-16 2010-04-22 Peter Lablans Controller in a Camera for Creating a Panoramic Image
US20140288682A1 (en) * 2008-12-05 2014-09-25 Nike, Inc. Athletic Performance Monitoring Systems and Methods in a Team Sports Environment
WO2015200209A1 (fr) * 2014-06-23 2015-12-30 Nixie Labs, Inc. Véhicules aériens sans pilote portatifs, véhicules aériens sans pilote à lancement commandé, et systèmes et procédés associés

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112802051A (zh) * 2021-02-02 2021-05-14 新华智云科技有限公司 基于神经网络的篮球投篮曲线的拟合方法及系统
CN112802051B (zh) * 2021-02-02 2022-05-17 新华智云科技有限公司 基于神经网络的篮球投篮曲线的拟合方法及系统
US20230241455A1 (en) * 2022-01-31 2023-08-03 Breakout Hitting Llc Training system and method of using same
AU2025204327B1 (en) * 2025-06-11 2026-03-26 Jouf University Integrated drone-based motion capture and smart insole system for real-time biomechanical analysis in outdoor environments

Also Published As

Publication number Publication date
CN113599788B (zh) 2023-03-28
CN109069903B (zh) 2021-08-20
CN113599788A (zh) 2021-11-05
CN109069903A (zh) 2018-12-21

Similar Documents

Publication Publication Date Title
US11450106B2 (en) Systems and methods for monitoring objects at sporting events
CN109069903B (zh) 用于监测体育事件中的对象的系统和方法
US12121794B2 (en) Electronic tracking system with heads up display
US11541294B2 (en) Golf aid including heads up display for green reading
US20260000961A1 (en) Golf aid including virtual caddy
US10661149B2 (en) Mixed-reality sports tracking and simulation
US12109478B2 (en) Electronic tracking system with heads up display
EP3756738B1 (fr) Systèmes et procédés d'affichage sur écran pour activité athlétique
US9734405B2 (en) Systems and methods for monitoring objects in athletic playing spaces

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17754037

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17754037

Country of ref document: EP

Kind code of ref document: A1