WO2022057795A1 - 动作识别方法及装置、终端设备、运动监测系统 - Google Patents

动作识别方法及装置、终端设备、运动监测系统 Download PDF

Info

Publication number
WO2022057795A1
WO2022057795A1 PCT/CN2021/118320 CN2021118320W WO2022057795A1 WO 2022057795 A1 WO2022057795 A1 WO 2022057795A1 CN 2021118320 W CN2021118320 W CN 2021118320W WO 2022057795 A1 WO2022057795 A1 WO 2022057795A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
motion
athlete
action
movement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/118320
Other languages
English (en)
French (fr)
Inventor
徐腾
陈霄汉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honor Device Co Ltd
Original Assignee
Honor Device Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honor Device Co Ltd filed Critical Honor Device Co Ltd
Priority to US17/921,194 priority Critical patent/US12374161B2/en
Priority to EP21868618.6A priority patent/EP4119204B1/en
Publication of WO2022057795A1 publication Critical patent/WO2022057795A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
    • A61B5/024—Measuring pulse rate or heart rate
    • A61B5/02416—Measuring pulse rate or heart rate using photoplethysmograph signals, e.g. generated by infrared radiation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/112—Gait analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1121—Determining geometric values, e.g. centre of rotation or angular range of movement
    • A61B5/1122—Determining geometric values, e.g. centre of rotation or angular range of movement of movement trajectories
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1123—Discriminating type of movement, e.g. walking or running
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1124—Determining motor skills
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A61B5/681—Wristwatch-type devices
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813—Specially adapted to be attached to a specific body part
    • A61B5/6829—Foot or ankle
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
    • A61B5/6895—Sport equipment
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G06V40/25—Recognition of walking or running movements, e.g. gait recognition
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G09B19/003—Repetitive work cycles; Sequence of movements
    • G09B19/0038—Sports
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2505/00—Evaluating, monitoring or diagnosing in the context of a particular type of medical care
    • A61B2505/09—Rehabilitation or training
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02—Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0204—Acoustic sensors
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02—Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0219—Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30221—Sports video; Sports image

Definitions

  • the present application relates to the field of electronic technology, and in particular, to a method and device for motion recognition, a terminal device, and a motion monitoring system.
  • the statistics of the sports parameters such as the type of shot, the force of the shot, the speed of the shot, the number of shots, etc., can reflect the comprehensive physique of the athlete. , based on these sports parameters, the comprehensive sports ability of the athlete can be evaluated and a personalized training plan can be formulated. Therefore, it is necessary to accurately determine the type of hitting action of the player when hitting the ball.
  • the present application provides an action recognition method, a terminal device and a storage medium, which can effectively identify the hitting action of an athlete in ball sports, and facilitate comprehensive analysis of the athlete's comprehensive athletic ability.
  • an embodiment of the present application provides an action recognition method
  • the method may include: acquiring motion data; the motion data includes first motion data, second motion data, and third motion data;
  • the first movement data obtains the gait characteristics of the athlete during the movement; according to the second movement data, the swing posture characteristics of the athlete during the movement are obtained; according to the third movement data, the movement of the athlete is obtained.
  • Image action features in the process; according to the gait feature, the swing posture feature and the image action feature, the hitting action of the athlete during the exercise process is identified.
  • the first movement data is collected by a first data collection device located at the first preset position of the athlete; the second movement data is collected by a second data collection device set at a preset position of the racket; the The third motion data is collected by a third data collection device set at a preset shooting position.
  • the above-mentioned first motion data is motion data that can reflect the movement of the lower limbs of the athlete, and the first preset part of the athlete can be the position of the calf or thigh of the athlete, or the ankle of the athlete, and of course it can be Other preset parts of the movement data of the movement of the lower limbs of the exerciser are collected.
  • the above-mentioned second motion data is motion data that can reflect the hitting action of the upper body of the athlete.
  • the preset position of the racket can be the handle of the racket, the racket shaft, or the racket head of the racket, which is not limited herein.
  • the above-mentioned third motion data is motion data that can reflect the image motion characteristics of the athlete.
  • it may be a moving image of an athlete.
  • the above-mentioned preset shooting position refers to a position where a moving image of a moving person can be shot, and the above-mentioned preset shooting position can be determined according to actual needs, which will not be repeated here.
  • the above-mentioned gait characteristics of the athlete during exercise include, but are not limited to, the classification of lower limb movements and the movement parameters of lower limb movements.
  • the above-mentioned swing posture features of the athlete during the exercise include, but are not limited to, the type of hand motion, the motion parameters of the hand motion, and whether the shot is effective.
  • the above-mentioned image action features include, but are not limited to, the hooked leg features (leg hooked and unhooked) of the athlete in the image.
  • the motion recognition method provided by the embodiment of the present application can perform feature extraction and motion recognition based on motion data collected by various data collection devices, and use multiple motion data to identify the user's gait feature, swing gesture feature, and image action feature. And based on gait features, swing posture features and image action features to determine the type of the athlete's batting action, it can accurately identify the athlete's batting action during the exercise process, which is beneficial to the comprehensive sports ability of the athlete. Comprehensive analysis makes it easier to formulate personalized training plans for athletes.
  • the first motion data includes foot acceleration data and foot angular velocity data
  • the gait characteristics of the athlete during exercise are acquired according to the first motion data , including: drawing the foot acceleration waveform and the foot angular velocity waveform according to the foot acceleration data and the foot angular velocity data, and extracting the characteristics of the foot acceleration waveform, such as wave peak value, wave trough value, wave peak value position, wave trough value position etc.; determine the departure point and touchdown point of each step according to the characteristics of the foot acceleration waveform; perform single-step segmentation on the foot acceleration waveform and the foot angular velocity waveform according to the departure point and touchdown point of each step; according to the segmentation
  • the rear foot acceleration waveform and the segmented foot angular velocity waveform determine the classification of the lower limb movements corresponding to each step of the athlete and the action parameters of the lower limb movements.
  • Determining the lower limb action parameters of the athlete according to the segmented foot acceleration waveform characteristics and the segmented foot angular velocity waveform characteristics may specifically include: calculating the athlete's flight time according to the time of the departure point and the touch point, and according to the departure point and the touch point.
  • the integral value of the change of the acceleration in the interval time of the location calculates the movement distance, movement speed, vertical jump height, etc. of the athlete.
  • the classification of the lower limb movements corresponding to each step and the action parameters of the lower limb movements are determined according to the foot acceleration data and the foot angular velocity data, which can provide the information of the lower limbs of the athlete during the exercise for analyzing the comprehensive exercise ability of the athlete.
  • the activity situation is helpful for a more comprehensive analysis of the overall athletic ability of the athlete.
  • the second movement data includes hand acceleration data, hand angular velocity data and sound wave data
  • the movement of the athlete during exercise is acquired according to the second movement data.
  • the swing gesture feature includes: determining the classification of the hand movements of the athlete during the movement and the action parameters of the hand movements according to the hand acceleration data and the hand angular velocity data; Identify effective hitting and empty swing for each hand movement during the exercise; make effective hitting according to the classification of the athlete's hand movements and the action parameters of the hand movements and each hand movement during the exercise. The identification of the swing and the air swing determines the swing gesture characteristics of the athlete during the movement.
  • determining whether to hit the ball effectively based on sound waves, and combining with the classification of hand movements determined based on the hand acceleration data and the hand angular velocity data can more accurately count the number of effective hits of the athlete. Helps to formulate corresponding training plans for athletes.
  • the third motion data includes a motion image of an exercising person
  • the acquiring, according to the third motion data, an image action feature of the exercising person during exercise includes: The moving image of the athlete is input into the trained convolutional neural network model for processing, and the image action feature corresponding to the moving image of the athlete is obtained.
  • the image action feature of the moving person is determined by the convolutional neural network model, which can effectively improve the efficiency of action classification.
  • the action recognition method further includes: when it is recognized that the hitting action is an effective hook and smash, acquiring the hook angle, the flight time and the jump height to be corrected; The jump height to be corrected is corrected by the flying time and the hooking angle.
  • the movement data further includes fourth movement data
  • the action recognition method further includes: determining, according to the fourth movement data, the movement of the athlete during the movement. Physiological parameters.
  • the fourth motion After determining the hitting action of the athlete during the exercise and the action parameters corresponding to each hitting action based on the first motion data, the second motion data and the third motion data, the fourth motion The heart rate determined by the data can further analyze the exercise intensity and physical fitness of the athlete, and can more comprehensively evaluate the overall exercise ability of the exerciser.
  • an embodiment of the present application provides a motion recognition device, including:
  • the first acquisition unit is used to acquire motion data;
  • the motion data includes first motion data, second motion data, and third motion data, wherein the first motion data is obtained by the first preset part of the athlete.
  • a data collection device collects;
  • the second movement data is collected by a second data collection device set at the preset position of the racket;
  • the third movement data is collected by a third data collection device set at the preset shooting position ;
  • the second obtaining unit is configured to obtain the gait characteristics of the athlete during the movement according to the first movement data; obtain the swing posture characteristics of the athlete during the movement according to the second movement data; Three motion data to obtain the image action characteristics of the athlete during the exercise;
  • the identification unit is configured to identify the hitting action of the athlete during the exercise process according to the gait feature, the swing posture feature and the image action feature.
  • the motion recognition device further includes a third acquisition unit, where the third acquisition unit is configured to acquire motion parameters of the hitting motion according to the motion data.
  • the first acquisition unit is further configured to acquire fourth motion data.
  • the motion recognition device further includes a physiological parameter determination unit, which is configured to determine the physiological parameters of the athlete during exercise according to the fourth exercise data.
  • the above-mentioned action recognition device further includes a correction unit, and the above-mentioned correction unit is configured to, when the batting action is an effective hook-and-shoot smash, perform the correction of the athlete according to the flying time and the hook angle. Corrected jump height.
  • an embodiment of the present application provides a motion monitoring system, where the motion monitoring system includes a first data acquisition device, a second data acquisition device, a third acquisition device, and the motion recognition device according to the second aspect;
  • the first data acquisition device, the second data acquisition device, and the third data acquisition device are respectively connected in communication with the motion recognition device;
  • the first data collection device is used to collect first motion data
  • the second data collection device is used to collect second motion data
  • the third data collection device is used to collect third motion data
  • the motion recognition device is used for recognizing the hitting motion of the athlete in the process of exercising according to the first motion data, the second motion data and the third motion data.
  • the motion monitoring system further includes a fourth data acquisition device; the fourth data acquisition device is connected in communication with the motion recognition device;
  • the fourth data collection device is used to collect fourth motion data
  • the motion recognition device is further configured to determine the physiological parameters of the athlete during exercise according to the fourth motion data.
  • an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program
  • the action recognition method as described in the first aspect above is implemented.
  • an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the motion recognition described in the first aspect above is implemented method.
  • an embodiment of the present application provides a computer program product that, when the computer program product runs on a terminal device, enables the terminal device to execute the action recognition method described in any one of the first aspects above.
  • a chip in a seventh aspect, includes a processor, and when the processor executes an instruction, the processor is configured to execute the action recognition method involved in any one of the designs of the second aspect above.
  • the instruction can come from memory inside the chip or from memory outside the chip.
  • the chip further includes an input and output circuit.
  • FIG. 1 is a schematic diagram of the architecture of a motion monitoring system to which a motion recognition method provided by an embodiment of the present application is applicable;
  • FIG. 2 is a schematic diagram of a usage scenario of the motion monitoring system corresponding to FIG. 1;
  • Fig. 3 is the schematic block diagram of the working process of the motion monitoring system corresponding to Fig. 1;
  • Figure 4 is a comparison diagram of the corresponding foot acceleration waveform and foot angular velocity waveform when the lower limb movements are walking and jumping respectively;
  • FIG. 5 is a schematic structural diagram of another motion monitoring system provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a usage scenario of the motion monitoring system corresponding to FIG. 5;
  • Fig. 7 is a schematic block diagram of the working process of the motion monitoring system corresponding to Fig. 5;
  • FIG. 8 is a schematic flowchart of the implementation of an action method provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a motion recognition device provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
  • the term “if” may be contextually interpreted as “when” or “once” or “in response to determining” or “in response to detecting “.
  • the phrases “if it is determined” or “if the [described condition or event] is detected” may be interpreted, depending on the context, to mean “once it is determined” or “in response to the determination” or “once the [described condition or event] is detected. ]” or “in response to detection of the [described condition or event]”.
  • references in this specification to "one embodiment” or “some embodiments” and the like mean that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application.
  • appearances of the phrases “in one embodiment,” “in some embodiments,” “in other embodiments,” “in other embodiments,” etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless specifically emphasized otherwise.
  • the terms “including”, “including”, “having” and their variants mean “including but not limited to” unless specifically emphasized otherwise.
  • the action recognition method and motion monitoring system provided by the embodiments of the present application are mainly used to identify the hitting actions of ball sports such as badminton, table tennis, tennis, etc.
  • the motion monitoring system is explained, detailed as follows:
  • badminton As a competitive sport, badminton has various movements, mainly through hand movements such as high ball, smashing, picking and rubbing. In order to evaluate the comprehensive athletic ability of badminton players and formulate a personalized training plan, it is necessary to make statistics on the sports parameters such as the shot type, shot force, shot speed, and number of shots of the badminton players during the process of badminton. .
  • the current smart wearable devices can only obtain basic parameters such as the center rate, the number of moving steps, and the moving distance of the athlete during the ball game. Based on these basic parameters, the comprehensive athletic ability of the athlete during the exercise cannot be analyzed.
  • badminton motion detection devices have also appeared on the market, such as badminton motion detection accessories set at the bottom of the badminton racket.
  • badminton motion detection accessory since the badminton motion detection accessory is set on the racket, it can only detect the hitting motion of the upper limb, and cannot detect the motion of the lower limb of the user, nor the vital sign parameters of the user. Therefore, the current badminton hitting action recognition method has the problem that it cannot comprehensively and scientifically analyze and guide the comprehensive sports ability of the athlete in combination with the user's lower limb action and vital sign parameters.
  • the embodiment of the present application provides an action recognition method and motion monitoring systems.
  • FIG. 1 shows a schematic structural diagram of a motion monitoring system to which the motion recognition method provided by the embodiment of the present application is applicable.
  • the motion monitoring system 100 may include a first data collection apparatus 1001 , a second data collection apparatus 1002 , a third data collection apparatus 1003 and a motion recognition apparatus 1004 .
  • the above-mentioned first data acquisition device 1001 , second data acquisition device 1002 , and third data acquisition device 1003 are respectively connected in communication with the motion recognition device 1004 .
  • the motion recognition device 1004 can perform feature extraction and motion recognition based on the motion data collected by each data collection device, identify the user's gait feature, swing gesture feature and image action feature by using a plurality of motion data, and based on the gait feature , swing posture features and image action features determine the type of the athlete's batting action, which can accurately identify the athlete's batting action during the exercise process, which is conducive to a comprehensive analysis of the athlete's comprehensive athletic ability, and more It is convenient to develop a personalized training plan for athletes.
  • the above-mentioned motion recognition device 1004 can establish a wireless communication connection with the first data acquisition device 1001, the second data acquisition device 1002, and the third data acquisition device 1003 through a short-range communication connection, or the motion recognition device 1004 can be connected by wired communication.
  • a wired communication connection is established with the first data acquisition device 1001 , the second data acquisition device 1002 , and the third data acquisition device 1003 respectively.
  • the embodiments of the present application do not specify the specific communication methods between the motion recognition device 1004 and the first data acquisition device 1001 , between the motion recognition device 1004 and the second data acquisition device 1002 , and between the motion recognition device 1004 and the third data acquisition device 1003 make any restrictions.
  • the short-range communication connection method may be a Bluetooth connection, a near field communication (Near Field Communication, NFC) connection, a wireless fidelity (Wireless-Fidelity, WiFi) connection, or a ZigBee (ZigBee) connection.
  • a Bluetooth connection can be preferably used; the short-range communication identifier is a unique identifier related to the short-range communication connection of the terminal equipment. Control, MAC) address or other unique identification of the Bluetooth device.
  • the above-mentioned first data collection device 1001 is used to collect the first motion data.
  • the above-mentioned first data collection device 1001 may be arranged on the first preset part of the athlete.
  • the above-mentioned first motion data is motion data that can reflect the movements of the lower limbs of the athlete.
  • the first data collection device 1001 is further configured to send the first motion data collected in real time to the motion recognition device 1004 connected in communication with it.
  • the first preset part of the athlete may be the position of the calf or thigh of the athlete, or the ankle of the athlete, and of course, it may be motion data that can collect the movements of the lower limbs of the athlete other preset parts.
  • the above-mentioned first preset part is taken as an example of the ankle of an athlete for description.
  • the above-mentioned first data acquisition device 1001 may be provided only on one foot of the athlete, or may be provided with the first data acquisition device on both feet, which is not limited herein.
  • the above-mentioned first motion data may include foot acceleration data and foot angular velocity data of the athlete during the motion.
  • the above-mentioned first data acquisition apparatus 1001 may be an electronic device having an accelerometer and a gyroscope. The above-mentioned electronic device can collect the acceleration data and the angular velocity data of the feet of the athlete during the exercise.
  • the above-mentioned first data acquisition apparatus 1001 may be a wearable device including a six-axis inertial sensor.
  • the wearable device including the six-axis inertial sensor can be worn on the ankle.
  • the above-mentioned six-axis inertial sensor can collect the acceleration data and the angular velocity data of the foot of the athlete in real time during the exercise.
  • the above-mentioned first data collection device 1001 may also be a smart shoe.
  • the smart shoe has a built-in six-axis inertial sensor. After the athlete puts on the smart shoe, the six-axis inertial sensor in the smart shoe can collect the above-mentioned foot acceleration data and foot angular velocity data of the athlete in real time during the exercise. .
  • the above-mentioned second data collection device 1002 is used to collect second motion data.
  • the above-mentioned second data acquisition device may be set at a preset position of the racket.
  • the above-mentioned second motion data is motion data that can reflect the hitting action of the upper body of the athlete.
  • the above-mentioned second data collection device 1002 is further configured to send the second motion data collected in real time to the motion recognition device 1004 connected in communication with it.
  • the preset position of the racket may be the handle of the racket, the racket shaft, or the racket head, which is not limited herein.
  • the above-mentioned preset position of the racket is the handle of the racket as an example for description.
  • the second data acquisition device 1002 may be embedded in the handle of the racket, specifically, the second data acquisition device 1002 may be embedded in the back cover of the handle of the racket.
  • the second motion data that can reflect the hitting action of the upper body is collected by the second data collecting device 1002 embedded in the handle of the racket.
  • the above-mentioned second movement data may include hand acceleration data, hand angular velocity data, and sound wave data of the athlete during the movement.
  • the above-mentioned second data acquisition device 1002 may include an acceleration sensor, a gyroscope and a microphone.
  • the above-mentioned acceleration sensor is used to collect the hand acceleration data of the athlete during the movement in real time
  • the above-mentioned gyroscope is used to collect the angular velocity data of the hand of the athlete during the movement in real time
  • the above-mentioned microphone is used to collect the movement of the person in real time. sonic data.
  • the above-mentioned second data acquisition device 1002 may also include a six-axis inertial sensor and a microphone.
  • the above-mentioned six-axis inertial sensor is used to collect real-time hand acceleration data and hand angular velocity data of the athlete during the movement
  • the above-mentioned microphone is used to collect the real-time sound wave data of the athlete during the movement.
  • the above-mentioned third data collection device 1003 is configured to collect third motion data.
  • the above-mentioned third data acquisition device 1003 may be set at a preset shooting position.
  • the above-mentioned third motion data is motion data that can reflect the image motion characteristics of the athlete.
  • the above-mentioned third data collection device 1003 is further configured to send the third motion data collected in real time to the motion recognition device 1004 connected in communication with it.
  • the above-mentioned third motion data may be a motion image of an athlete.
  • the above-mentioned preset shooting position refers to a position where a moving image of a moving person can be shot, and the above-mentioned preset shooting position can be determined according to actual needs, which will not be repeated here.
  • the above-mentioned preset shooting positions may be four corners of the stadium. That is, the moving images of the athlete during the exercise are collected through the third data collection devices arranged at the four corners of the court.
  • the above-mentioned third data collection apparatus 1003 may be a terminal device capable of collecting moving images of the athlete, such as a mobile phone with a camera function, a camera, and the like.
  • the above-mentioned third data collection device 1003 as an example of a mobile phone, the above-mentioned third motion data is image data or video data captured by the mobile phone.
  • the motion recognition apparatus 1004 may be a terminal device with data processing capabilities, and the motion recognition apparatus can process the first motion data collected by the first data collection apparatus 1001 to obtain the information about the movement of the athlete during the exercise. can also process the second movement data collected by the second data collection device 1002 to obtain the swing gesture characteristics of the athlete during the movement; The third motion data is processed to obtain image action features of the athlete during the motion.
  • the above-mentioned action recognition device 1004 can also recognize the hitting action of the athlete during the exercise based on the above-mentioned gait feature, the above-mentioned swing posture feature and the above-mentioned image action feature.
  • the above-mentioned motion recognition device 1004 may be an independent terminal device, such as a wearable device, or a mobile phone, a tablet computer, an augmented reality (AR)/virtual reality (VR) ) equipment, notebook computers, ultra-mobile personal computers (ultra-mobile personal computers, UMPCs), netbooks, personal digital assistants (personal digital assistants, PDAs) and other mobile terminals, and can also be server equipment with data processing capabilities.
  • This application implements The example does not impose any limitation on the specific type of the motion recognition device 1004 .
  • the above-mentioned motion recognition device 1004 may also be a virtual terminal with data processing capability, such as a virtual computer and a cloud server that do not have a hardware structure.
  • the motion recognition apparatus 1004 and the first data collection apparatus 1001 may also be set in the same electronic device, that is, the electronic apparatus may include the first data collection apparatus 1001 and the motion recognition apparatus 1004 .
  • the above motion recognition apparatus 1004 and the second data collection apparatus 1002 may also be set in the same electronic device, that is, the electronic apparatus may include the second data collection apparatus 1002 and the motion recognition apparatus 1004 .
  • the above-mentioned motion recognition apparatus 1004 and the third data collection apparatus 1003 may also be set in the same electronic device, that is, the electronic apparatus may include the third data collection apparatus 1003 and the motion recognition apparatus 1004 .
  • the first data collection apparatus 10001 sends the first motion data collected in real time to the motion recognition apparatus 1004 .
  • the motion recognition device 1004 may perform single-step segmentation on the first motion data, thereby determining the gait characteristics of the athlete during the exercise.
  • the above-mentioned gait characteristics of the athlete during exercise include, but are not limited to, the classification of lower limb movements and the movement parameters of lower limb movements.
  • the above-mentioned first motion data includes foot acceleration data and foot angular velocity data
  • the above-mentioned process of processing the first motion data is specifically: drawing a foot acceleration waveform and a foot angular velocity waveform according to the first motion data, and then According to the foot acceleration waveform and the foot angular velocity waveform, the off point and the touch point of each step are determined.
  • the external acceleration waveform and the foot angular velocity waveform determine the single-step gait characteristics of the athlete during the exercise.
  • Two adjacent peaks in the foot acceleration waveform can represent an action cycle (ie one step), the time point corresponding to the first peak of the two adjacent peaks is the departure point, and the time point corresponding to the second peak is the touch point.
  • the time period before the location, off-point and touch-down point is an action cycle.
  • the departure point and contact point of each step can be determined, and then each action cycle can be determined.
  • the waveform of the foot acceleration and the waveform of the angular velocity of the foot can be divided into a single step.
  • the lower limb movements corresponding to each action cycle can be walking, running, and jumping.
  • the foot acceleration waveform and foot angular velocity waveform corresponding to different lower limb actions are different. Therefore, the lower limb action corresponding to each action cycle can be determined according to the foot acceleration waveform and the foot angular velocity waveform.
  • the foot acceleration waveform and the foot angular velocity waveform are shown in (a) of Figure 4; when the lower limb action is jumping, the foot acceleration waveform and the foot angular velocity The waveform is shown in (b) of Figure 4 . It should be noted that the abscissa axis in FIG.
  • the ordinate axis represents the acceleration value and the angular velocity value, when the ordinate axis represents the acceleration
  • the unit is meters per square second (m/s2), and when the ordinate represents the value of the angular velocity, the unit is radians per second (rad/s).
  • the action parameters of the lower limb actions corresponding to each action cycle can also be determined based on the acceleration, angular velocity, departure point and touchdown point in each action cycle, such as air time, jumping Height, moving distance, moving speed, etc.
  • the above-mentioned second data collection device 1002 can send the second motion data collected in real time to the motion recognition device 1004 communicatively connected to it. After acquiring the second motion data, the motion recognition device 1004 may process the second motion data, and then determine the swing gesture feature of the athlete during the exercise.
  • the above-mentioned swing posture features of the athlete during the exercise include, but are not limited to, the type of hand motion, the motion parameters of the hand motion, and whether the shot is effective.
  • the above-mentioned second motion data may include hand acceleration data, hand angular velocity data and sound wave data.
  • the above-mentioned process of processing the second data is specifically: drawing the hand according to the hand acceleration data and the hand angular velocity data.
  • the acceleration waveform and the hand angular velocity waveform are then used to determine the classification and action parameters of the hand movement in each swing based on the hand acceleration waveform and the hand angular velocity waveform.
  • the eigenvalues of the sonic data are extracted based on the sonic data, and then the effective hitting and the air swing are identified based on the eigenvalues of the sonic data.
  • the classification of the hand movement and the action parameters of each swing are determined, and the classification of hitting and air swing based on the characteristic value of the sound wave data is used to determine each swing of the athlete.
  • the classification of the hand movements during the above-mentioned swing includes, but is not limited to, high and long shots, smashing, and rubbing.
  • the hand acceleration waveform and the hand angular velocity waveform Based on the hand acceleration waveform and the hand angular velocity waveform, it is possible to determine the classification of the hand motion during each swing and the action parameters during the swing based on the existing classification algorithm model, that is, the hand acceleration waveform and the hand motion parameter.
  • the angular velocity waveform is input into the existing classification algorithm model for processing, and the classification of the hand movements and the action parameters during each swing can be obtained, which will not be repeated here.
  • the characteristic values of the above-mentioned acoustic wave data include, but are not limited to, energy, peak value, and frequency. Because the time-domain waveform of acoustic wave data only reflects the relationship of sound pressure with time, but cannot reflect the characteristic value of acoustic wave data. Therefore, it is necessary to convert the time-domain waveform of the acoustic wave data into a frequency-domain waveform that can reflect the eigenvalues, and then extract the corresponding eigenvalues.
  • MFCCs mel-frequency cepstral coefficients
  • LPCC linear prediction cepstral coefficients
  • the above preset Gaussian mixture model is a Gaussian mixture model determined based on the historical motion data of each athlete. Each athlete has its corresponding preset Gaussian mixture model, and the athlete ID can be used to associate the athlete with the preset Gaussian mixture model. Mixed model for association.
  • the action recognition device 1004 can determine the corresponding preset Gaussian mixture model according to the athlete ID of the athlete currently training.
  • the preset Gaussian mixture model is a Gaussian mixture model in which the optimal solution of the parameters of the Gaussian mixture model is determined, and the above-mentioned parameters of the Gaussian mixture model can be expressed as ⁇ .
  • the process of solving the optimal solution of ⁇ is the training process of the Gaussian mixture model.
  • N M-dimensional Gaussian density function linear weighting functions can be used to represent the above-mentioned Gaussian mixture model, where both N and M are positive integers greater than 1.
  • the function of the above Gaussian mixture model can be expressed as:
  • X is the M-dimensional characteristic parameter of the acoustic wave data
  • ⁇ i refers to the weight of the ith Gaussian density function
  • G i (X) is the Gaussian probability density function of the feature parameter X.
  • C i refers to the covariance matrix of the ith Gaussian density function
  • ⁇ i refers to the mean vector of the ith Gaussian density function
  • C i refers to the covariance matrix of the ith Gaussian density function
  • ⁇ i refers to the mean vector of the ith Gaussian density function
  • ⁇ ( ⁇ i , ⁇ i , C i ) can estimate the optimal solution of ⁇ based on the iterative calculation of the expectation maximization algorithm.
  • the swing posture feature can be determined.
  • the classification of the athlete's hand movements during the exercise process is determined as: the first hand movement is a high ball, and the second hand movement is a smash.
  • the third hand action is rubbing the ball; based on the sound wave data, it is determined that the first hand action is an empty swing, the second hand action is an effective shot, and the third hand action is an effective shot; then it can be determined
  • the characteristics of the swing posture of the athlete during the exercise are as follows: the first hand action is an invalid high-distance ball and the hitting speed, hitting strength, hitting trajectory, etc.; the second hand action is an effective kill. The ball and the batting speed, batting strength, batting trajectory, etc. when hitting the ball; the third hand action is the effective rubbing and batting speed, batting strength, batting trajectory, etc. when hitting the ball.
  • the third data collection device 1003 can send the third motion data collected in real time to the motion recognition device 1004 communicatively connected to it. After acquiring the third motion data, the motion recognition device 1004 can analyze the image motion characteristics of the athlete during the motion based on the third motion data and the convolutional neural network model.
  • the above-mentioned third motion data is a motion image of an athlete.
  • the above-mentioned motion recognition device 1004 can first perform target detection, frame and select the target image area containing the athlete in the moving image, and then identify the athlete in the image based on the image in the target image area.
  • the hook-leg feature In this way, the amount of computation when the motion recognition device 1004 uses the convolutional neural network model to process the moving image can be effectively reduced, the occupation of computing resources can be reduced, and the processing efficiency can be improved.
  • the above-mentioned image action features include, but are not limited to, the hooked leg features (leg hooked and unhooked) of the athlete in the image.
  • the above-mentioned convolutional neural network model refers to a trained convolutional neural network model that can be used to process the third motion data to determine an image action feature.
  • the convolutional neural network model used for image processing can use the existing convolutional neural network model, and the process of training the convolutional neural network model can use a large number of known classified moving images to train the convolutional neural network model,
  • the trained convolutional neural network model can input the image action feature corresponding to the third motion data when inputting the third motion data.
  • the trained convolutional neural network model can be stored in the data storage area of the action recognition device 1004. After the action recognition device 1004 receives the third motion data collected by the third data acquisition device 1003, it will automatically call the convolutional neural network model. The third motion data is processed to obtain an image action feature corresponding to the third motion data.
  • the above-mentioned image action feature may further include the classification of the hitting action, that is, the above-mentioned convolutional neural network model can also determine the classification of the hitting action of the athlete based on the input moving image. Then, combined with the gait feature determined by the first motion data and the swing posture feature determined by the second motion data, the player's batting action and batting action parameters are comprehensively determined.
  • the motion recognition device 1004 can fuse these features (ie, the gait feature, the swing gesture feature and image action feature) to finally determine the player's batting action during the movement. At the same time, it can also determine the action parameters such as the hitting speed, hitting force, air time, jumping height, moving distance, moving speed, etc. of the athlete when hitting the ball.
  • the hitting action of the athlete may include effective high ball, effective hook jump kill, effective unhook jump kill, effective in-situ kill, effective rub, etc., invalid high ball, invalid jump Kill, invalid in-situ smash, invalid rub, etc.
  • the movements of the lower limbs of the athlete during exercise determined according to the first movement data collected by the first data collection device 1001 are walking, jumping, and jumping;
  • the categories of hand movements determined by the motion data are respectively effective rubbing, invalid smashing, and effective smashing;
  • the leg hook features determined according to the third motion data collected by the third data acquisition device 1003 are respectively unhook, unhook. Legs and hooks; by merging the above features, it can be determined that the player's hitting actions during the exercise are effective rubbing, invalid jump kills, and effective hooks and jump kills.
  • the action image corresponding to the jumping action is obtained, and then based on the action image, it is identified whether the athlete has hooked his leg. Time corrects the jump height; if the athlete does not hook the leg during the jump, no correction is required. It should be noted that the correction of the jump height can be implemented based on an existing correction model, which will not be repeated here.
  • FIG. 5 is a structural block diagram of another motion monitoring system provided by an embodiment of the present application. As shown in FIG. 5 , different from the previous embodiment of the present application, the motion monitoring system further includes a fourth data collection device 1005 . The above-mentioned fourth data acquisition device 1005 is also connected in communication with the motion recognition device 1004 .
  • the above-mentioned motion recognition device 1004 can establish a wireless communication connection with the fourth data acquisition device 1005 through a short-range communication connection, or the motion recognition device 1004 can establish a wired communication connection with the fourth data acquisition device 1005 through wired communication.
  • This embodiment of the present application does not make any limitation on the specific communication mode between the motion recognition device 1004 and the fourth data collection device 1005 .
  • the short-range communication connection method may be a Bluetooth connection, a near field communication (Near Field Communication, NFC) connection, a wireless fidelity (Wireless-Fidelity, WiFi) connection, or a ZigBee (ZigBee) connection.
  • a Bluetooth connection can be preferably used; the short-range communication identifier is a unique identifier related to the short-range communication connection of the terminal equipment. Control, MAC) address or other unique identification of the Bluetooth device.
  • the above-mentioned fourth data collection device 1005 is used for collecting fourth motion data.
  • the above-mentioned fourth data collection device may be arranged at the second preset part of the athlete.
  • the above-mentioned fourth motion data is motion data that can reflect the physiological parameters of the athlete during exercise.
  • the physiological parameters include, but are not limited to, heart rate, pulse, body temperature, and the like.
  • the above-mentioned fourth exercise data includes, but is not limited to, the heart rate data of the athlete during exercise.
  • the above-mentioned second preset part of the athlete refers to a part from which fourth movement data can be collected, such as a wrist of the athlete.
  • the above-mentioned second preset part of the athlete may also be other parts from which the fourth movement data can be collected, such as fingertips, neck and other parts, which are not limited here.
  • the above-mentioned fourth data collection apparatus 1005 may include a wearable device capable of collecting heart rate data, such as a smart watch, a smart bracelet, and the like.
  • a wearable device capable of collecting heart rate data
  • the above-mentioned wearable device can be worn on the wrist.
  • the wearable device can collect the heart rate data of the athlete in real time during the exercise.
  • the above-mentioned wearable device may include a photoplethysmograph (PPG) sensor for collecting heart rate data, and the heart rate data at the wrist of the exerciser is collected through the above-mentioned PPG sensor.
  • PPG photoplethysmograph
  • the motion recognition device 1004 can also process the fourth motion data collected by the fourth data collection device 1005 to determine the heart rate of the athlete during exercise.
  • the above-mentioned fourth motion data may be based on PPG data collected by a photoplethysmograph (PPG) sensor.
  • PPG photoplethysmograph
  • the motion recognition device 1004 can perform filtering processing on the fourth motion data to determine the heart rate of the athlete during exercise.
  • the motion recognition device can perform pulse waveform peak interval feature extraction processing on the pulse data, thereby determining the heart rate value of the athlete during exercise.
  • the motion recognition device determines the hitting motion of the athlete during the exercise and the motion parameters corresponding to each hitting motion based on the first motion data, the second motion data and the third motion data.
  • the exercise intensity and physical fitness of the exerciser can be further analyzed based on the heart rate determined by the fourth exercise data. It can more comprehensively evaluate the comprehensive sports ability of the athlete.
  • the execution body of the above-mentioned motion recognition method may be the motion recognition device in the above-mentioned embodiment. As shown in FIG. 8 , the method may include the following steps:
  • the above motion data includes first motion data, second motion data, and third motion data.
  • the first motion data is motion data that can reflect the lower limb movements of the athlete during exercise.
  • the above-mentioned second motion data is motion data that can reflect the hitting action of the upper limb of the athlete during the exercise.
  • the above-mentioned third motion data is motion data capable of expressing the characteristics of the image motion of the athlete.
  • the above-mentioned first movement data may include: foot acceleration data and foot angular velocity data of the athlete during the movement.
  • the above-mentioned second movement data may include: hand acceleration data, hand angular velocity data and sound wave data of the athlete during the movement.
  • the above-mentioned third motion data may include: a motion image of the athlete.
  • the first movement data is collected by the first data collection device located at the first preset position of the athlete
  • the second movement data is collected by the second data collection device set at the preset position of the racket
  • the third movement data is collected by the set The third data collection device at the preset shooting position performs collection.
  • the first preset part of the athlete may be the calf or thigh of the athlete, or the ankle of the athlete, or other preset parts that can collect motion data of the lower limb movements of the athlete.
  • the preset position of the racket may be the handle of the racket, the shaft of the racket, or the head of the racket, which is not limited herein.
  • the above-mentioned preset shooting position refers to a position where a moving image of a moving person can be shot, and the above-mentioned preset shooting position can be determined according to actual needs, which will not be repeated here.
  • each data collection device may store the motion data collected in real time in its local memory. Based on this, when the motion recognition device wants to acquire motion data, each data acquisition device can send the motion data stored in the local memory to the motion recognition device communicatively connected to it.
  • the above-mentioned acquiring the posture feature according to the motion data includes: acquiring the gait feature of the athlete during the exercise process according to the first motion data; acquiring the swing of the athlete during the exercise process according to the second motion data Attitude feature; obtain the image action feature of the athlete during the movement according to the third movement data.
  • the above-mentioned first motion data includes foot acceleration data and foot angular velocity data
  • the process of acquiring the gait characteristics of the athlete during exercise according to the first motion data is specifically: drawing according to the first motion data Foot acceleration waveform and foot angular velocity waveform, and then determine the departure point and contact point of each step according to the foot acceleration waveform and foot angular velocity waveform, and compare the foot acceleration waveform and foot angular velocity waveform according to the departure point and contact point of each step.
  • Single-step segmentation is performed, and the gait characteristics of the athlete during exercise are determined according to the segmented foot acceleration waveform and foot angular velocity waveform.
  • the above-mentioned gait characteristics of the athlete during exercise include, but are not limited to, the classification of lower limb movements and the movement parameters of lower limb movements.
  • the lower body movements can be classified as walking, running, and jumping.
  • the action parameters of the lower body action can be the flight time, jump height, moving distance, moving speed, and the like.
  • the second motion data may include hand acceleration data, hand angular velocity data, and sound wave data.
  • the above-mentioned process of acquiring the swing gesture feature of the athlete during the exercise process according to the second motion data is specific. For: draw the hand acceleration waveform and the hand angular velocity waveform according to the hand acceleration data and the hand angular velocity data, and then determine the classification and action parameters of the hand movement in each swing based on the hand acceleration waveform and the hand angular velocity waveform ; Extract the eigenvalues of the sonic data based on the sonic data, and then carry out the identification of effective hits and empty swings based on the eigenvalues of the sonic data; The classification of the movement and the action parameters and the classification of the effective hitting and the empty swing based on the characteristic value of the sound wave data determine the swing posture characteristics of the player each time he swings.
  • the above-mentioned swing posture features of the athlete during the exercise include, but are not limited to, the type of hand motion, the motion parameters of the hand motion, and whether the shot is effective.
  • the classification of the hand movements during the above-mentioned swing includes, but is not limited to, high and long shots, smashing, and rubbing.
  • the above-mentioned third motion data may include a motion image of an athlete.
  • the process of acquiring the image action features of the athlete during the exercise process according to the third motion data is specifically: firstly perform target detection on the moving image, frame and select the target image area containing the athlete from the moving image, and then select the target image
  • the region is input into the trained convolutional neural network model for processing, and the image action characteristics of the athlete during exercise are determined.
  • the above-mentioned image action features include, but are not limited to, the hooked leg features (leg hooked and unhooked) of the athlete in the image.
  • the above-mentioned third motion data can also be an image including a stadium heat map, etc., which can be used to draw the distribution of the motion trajectories on the stadium (the motion trajectories of the players and the motion trajectories of the ball, etc.).
  • the above-mentioned stadium heat map can be used as To assist in the analysis of the gait characteristics of the movement and the position of the points gained and lost.
  • the gait feature, the swing posture feature and the image action feature are fused to finally identify the hitting action of the athlete during the exercise.
  • the determined gait characteristics of the athlete during the exercise are walking, jumping, and jumping; the swing posture characteristics of the athlete during the exercise are effective rubbing, invalid smashing, and effective smashing; image action
  • the features are not hooked, not hooked, and hooked; by merging the above features, it can be determined that the athlete's hitting actions in the process of exercise are effective rubbing, invalid jumping, and effective hooking and jumping.
  • the action parameters of the hitting action may also be acquired according to the motion data.
  • the action parameters of the above-mentioned batting action may include batting speed, batting intensity, flight time, jump height, moving distance, moving speed, and the like.
  • the action parameters of the athlete when hitting the ball it is easier to conduct a more specific analysis of the athlete's comprehensive athletic ability.
  • the jump height of the athlete is corrected according to the flight time and the hook angle.
  • the overall athletic ability of the athlete is then analyzed based on the corrected jump height.
  • the correction process of the jump height reference may be made to the foregoing embodiments, which will not be repeated here.
  • the above motion data may further include fourth motion data.
  • Physiological parameters of the athlete during exercise can be determined according to the fourth exercise data.
  • the above-mentioned fourth movement data is collected by a fourth data collection device disposed at the second preset part of the athlete.
  • the above-mentioned fourth motion data is motion data that can reflect the physiological parameters of the athlete during exercise.
  • the physiological parameters include, but are not limited to, heart rate, pulse, body temperature, and the like.
  • the above-mentioned fourth exercise data includes, but is not limited to, the heart rate data of the athlete during exercise.
  • the above-mentioned second preset part of the athlete refers to a part from which fourth movement data can be collected, such as a wrist of the athlete.
  • the above-mentioned second preset part of the athlete may also be other parts from which the fourth movement data can be collected, such as fingertips, neck and other parts, which are not limited here.
  • the heart rate of the athlete during exercise is determined by filtering the heart rate data.
  • the above process of performing filtering processing on heart rate data to determine the heart rate reference may be made to the existing method for processing heart rate data, which will not be repeated here.
  • the motion recognition method provided by this embodiment can perform feature extraction and motion recognition based on motion data collected by various data collection devices, and identify the user's gait feature, swing pose feature, and image motion feature by using multiple motion data. And based on gait features, swing posture features and image action features to determine the type of the athlete's batting action, it can accurately identify the athlete's batting action during the exercise process, which is beneficial to the comprehensive sports ability of the athlete. Comprehensive analysis makes it easier to formulate personalized training plans for athletes.
  • FIG. 9 shows a structural block diagram of an motion recognition apparatus provided by an embodiment of the present application, and each unit included in the motion recognition apparatus is used to execute each step in the foregoing embodiment.
  • the motion recognition device 90 includes a first acquisition unit 91 , a second acquisition unit 92 and an identification unit 93 . in:
  • the first acquisition unit 91 is used to acquire motion data.
  • the above motion data includes first motion data, second motion data, and third motion data.
  • the second acquiring unit 92 is configured to acquire the gesture feature according to the motion data.
  • the second obtaining unit 92 is specifically configured to obtain the gait characteristics of the athlete during the movement according to the first movement data; obtain the swing posture characteristics of the athlete during the movement according to the second movement data; The movement data obtains the image action characteristics of the athlete during the movement process.
  • the recognition unit 93 is used for recognizing the hitting action according to the posture feature.
  • the identification unit 93 is specifically configured to identify the hitting action of the athlete during the exercise process according to the gait feature, the swing posture feature and the image action feature.
  • the above-mentioned action recognition apparatus further includes a third acquisition unit, and the above-mentioned third acquisition unit is configured to acquire the action parameters of the hitting action according to the motion data.
  • the above-mentioned first acquiring unit is further configured to acquire fourth motion data.
  • the motion recognition device further includes a physiological parameter determination unit, which is configured to determine the physiological parameters of the athlete during exercise according to the fourth exercise data.
  • the motion recognition device further includes a correction unit, which is used to correct the jump height of the athlete according to the flight time and the hook angle when the hitting action is an effective hook smash.
  • a motion recognition device provided by the embodiments of the present application can also perform feature extraction and motion recognition based on motion data collected by various data collection devices, and identify the user's gait characteristics, Swing posture features and image action features, and determine the type of athlete's hitting action based on gait features, swing posture features and image action features, and can accurately identify the athlete's hitting action during the exercise process. It is conducive to a comprehensive analysis of the comprehensive sports ability of the athlete, and it is more convenient to formulate a personalized training plan for the athlete.
  • FIG. 10 is a schematic structural diagram of a terminal device provided by another embodiment of the present application.
  • the terminal device 10 in this embodiment includes: at least one processor 101 (only one is shown in FIG. 10 ), a memory 102 , and a memory 102 stored in the memory 102 and available in the at least one processor 101
  • the computer program 103 running on the processor 101 when the processor 101 executes the computer program 103, implements the steps in any of the above-mentioned action recognition method embodiments.
  • the terminal device 10 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • the terminal device may include, but is not limited to, the processor 101 and the memory 102 .
  • FIG. 10 is only an example of the terminal device 10, and does not constitute a limitation on the terminal device 10. It may include more or less components than the one shown, or combine some components, or different components , for example, may also include input and output devices, network access devices, and the like.
  • the so-called processor 101 may be a central processing unit (Central Processing Unit, CPU), and the processor 101 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuits) , ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the memory 102 may be an internal storage unit of the terminal device 10 in some embodiments, such as a hard disk or a memory of the terminal device 10 .
  • the memory 102 may also be an external storage device of the terminal device 10 in other embodiments, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc.
  • the memory 102 may also include both an internal storage unit of the terminal device 100 and an external storage device.
  • the memory 102 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as program codes of the computer program, and the like.
  • the memory 102 may also be used to temporarily store data that has been output or will be output.
  • the above-mentioned memory 102 may further store a motion recognition algorithm library determined based on the motion recognition method provided in the embodiment of the present application.
  • Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the above-mentioned action recognition method can be implemented.
  • the embodiments of the present application provide a computer program product, when the computer program product runs on a mobile terminal, the steps in the above-mentioned action recognition method can be implemented when the mobile terminal executes.
  • the embodiments of the present application also provide an apparatus, which may specifically be a chip, a component or a module, and the apparatus may include a connected processor and a memory; wherein, the memory is used for storing computer execution instructions, and when the apparatus is running, The processor can execute the computer-executed instructions stored in the memory, so that the chip executes the interaction method in each of the foregoing method embodiments.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium.
  • the present application realizes all or part of the processes in the methods of the above embodiments, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium.
  • the computer program includes computer program code
  • the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like.
  • the computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the photographing device/terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signals, telecommunication signals, and software distribution media.
  • ROM read-only memory
  • RAM random access memory
  • electrical carrier signals telecommunication signals
  • software distribution media For example, U disk, mobile hard disk, disk or CD, etc.
  • computer readable media may not be electrical carrier signals and telecommunications signals.
  • the disclosed apparatus/network device and method may be implemented in other manners.
  • the apparatus/network device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units. Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, may be located in one place, or may be respectively on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Surgery (AREA)
  • Multimedia (AREA)
  • Physiology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Business, Economics & Management (AREA)
  • Cardiology (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Geometry (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

本申请适用于电子技术领域，提供了一种动作识别方法及装置、终端设备、运动监测系统，基于各个数据采集装置采集到的运动数据进行特征提取和动作识别，利用多个运动数据识别出用户的步态特征、挥拍姿态特征以及图像动作特征，并基于步态特征、挥拍姿态特征以及图像动作特征确定出运动者的击球动作类型，能够准确地对运动过程中运动者的击球动作进行识别，有利于对运动者的综合运动能力进行全面地分析，更便于针对运动者制定个性化的训练计划。

Description

动作识别方法及装置、终端设备、运动监测系统
本申请要求于2020年9月15日提交国家知识产权局、申请号为202010970737.1、申请名称为“动作识别方法及装置、终端设备、运动监测系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及电子技术领域，尤其涉及一种动作识别方法及装置、终端设备、运动监测系统。
背景技术
在羽毛球、乒乓球、网球等球类运动的训练过程中，统计运动者在击球时的击球类型、击球力道、击球速度、击球次数等运动参数，能够反应运动者的综合体质，基于这些运动参数能够对运动者的综合运动能力进行评估并制定个性化的训练计划。因此需要准确地确定出运动者在击球时的击球动作类型。
目前，市场已出现多种智能穿戴设备，如能量手环、计步鞋等。然而目前市场上的智能穿戴设备只能获取运动者在球类运动过程中心率、移动步数、移动距离等基本参数，而无法有效地识别出运动者在击球时的击球动作，无法准确地统计出运动者在击球时的运动参数。因此，现有的动作识别方法无法满足全面分析运动者在球类运动中的综合运动能力的需求。
发明内容
有鉴于此，本申请提供一种动作识别方法、终端设备及存储介质，能够有效地识别出运动者在球类运动中的击球动作，便于全面地分析运动者的综合运动能力。
为了实现上述目的，第一方面，本申请实施例提供一种动作识别方法，该方法可以包括：获取运动数据；所述运动数据包括第一运动数据、第二运动数据、第三运动数据；根据所述第一运动数据获取运动者在运动过程中的步态特征；根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征；根据所述第三运动数据获取运动者在运动过程中的图像动作特征；根据所述步态特征、所述挥拍姿态特征和所述图像动作特征识别出运动者在运动过程中的击球动作。
其中，所述第一运动数据由位于运动者第一预设部位的第一数据采集装置进行采集；所述第二运动数据由设置于球拍预设位置的第二数据采集装置进行采集；所述第三运动数据由设置于预设拍摄位置的第三数据采集装置进行采集。
上述第一运动数据为可以体现运动者的下肢动作的运动数据，上述运动者的第一预设部位可以是运动者的小腿或大腿位置，也可以是运动者的脚踝处，当然还可以是能够采集到运动者的下肢动作的运动数据的其他预设部位。
上述第二运动数据为可以体现运动者的上肢击球动作的运动数据。上述球拍的预设位置可以是球拍的拍柄，也可以是球拍的拍杆，还可以是球拍的拍头，在此不加以 限制。
上述第三运动数据为可以体现运动者的图像动作特征的运动数据。例如可以是运动者的运动图像。上述预设拍摄位置是指能够拍摄到运动者的运动图像的位置，上述预设拍摄位置可以根据实际需要进行确定，在此不加以赘述。
上述运动者在运动过程中的步态特征包括但不限于下肢动作的分类和下肢动作的动作参数。上述运动者在运动过程中的挥拍姿态特征包括但不限于手部动作的类别、手部动作的动作参数以及是否有效击球。上述图像动作特征包括但不限于图像中运动员的勾腿特征(勾腿和未勾腿)。
本申请实施例提供的动作识别方法，能够基于各个数据采集装置采集到的运动数据进行特征提取和动作识别，利用多个运动数据识别出用户的步态特征、挥拍姿态特征以及图像动作特征，并基于步态特征、挥拍姿态特征以及图像动作特征确定出运动者的击球动作类型，能够准确地对运动过程中运动者的击球动作进行识别，有利于对运动者的综合运动能力进行全面地分析，更便于针对运动者制定个性化的训练计划。
在第一方面的一种可能的实施方式中，所述第一运动数据包括足部加速度数据和足部角速度数据，所述根据所述第一运动数据获取运动者在运动过程中的步态特征，包括：根据所述足部加速度数据和所述足部角速度数据绘制足部加速度波形和足部角速度波形，并提取足部加速度波形特征，如波峰值，波谷值，波峰值位置、波谷值位置等；根据所述足部加速度波形特征确定每一步的离地点和触地点；根据每一步的离地点和触地点对所述足部加速度波形和所述足部角速度波形进行单步分割；根据分割后的足部加速度波形和分割后的足部角速度波形确定运动者每一步对应的下肢动作的分类和下肢动作的动作参数。
根据分割后的足部加速度波形特征和分割后的足部角速度波形特征确定运动者的下肢动作参数具体可以是：根据离地点和触地点的时刻，计算运动者的腾空时间，根据离地点和触地点的间隔时间内的加速度的变化的积分值，计算运动者的移动距离、移动速度和纵跳高度等。
上述实施方式中，根据足部加速度数据和足部角速度数据确定每一步对应的下肢动作的分类以及下肢动作的动作参数，能够为分析运动者的综合运动能力提供运动者在运动过程中的下肢的活动情况，有助于更加全面的分析运动者的综合运动能力。
在第一方面的一种可能的实施方式中，所述第二运动数据包括手部加速度数据、手部角速度数据以及声波数据，所述根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征，包括：根据所述手部加速度数据和所述手部角速度数据确定运动者在运动过程中的手部动作的分类和手部动作的动作参数；根据所述声波数据对运动者在运动过程中的每一个手部动作进行有效击球和空挥的识别；根据运动者在运动过程中的手部动作的分类和手部动作的动作参数以及每一个手部动作进行有效击球和空挥的识别确定所述运动者在运动过程中的挥拍姿态特征。
上述实施方式中，基于声波确定是否有效击球，并结合基于手部加速度数据和所述手部角速度数据确定的手部动作的分类，能够更加准确的统计出运动者的有效击球数，有助于对运动者制定相应的训练计划。
在第一方面的一种可能的实施方式中，所述第三运动数据包括运动者的运动图像， 所述根据所述第三运动数据获取运动者在运动过程中的图像动作特征，包括：将运动者的运动图像输入到完成训练的卷积神经网络模型中进行处理，得到所述运动者的运动图像对应的图像动作特征。
上述实现方式中，通过卷积神经网络模型确定运动者的图像动作特征，能够有效提高动作分类的效率。
在第一方面的一种可能的实施方式中，所述动作识别方法还包括：当识别出击球动作为有效勾腿杀球时，获取勾腿角度、腾空时间以及待修正的跳跃高度；基于所述腾空时间和所述勾腿角度对所述待修正的跳跃高度进行修正。通过对运动者的跳跃高度进行修正，能够更准确地统计运动者的运动参数。
在第一方面的一种可能的实施方式中，所述运动数据还包括第四运动数据，相应地，所述动作识别方法还包括：根据所述第四运动数据确定运动者在运动过程中的生理参数。
上述实现方式中，在基于第一运动数据、第二运动数据以及第三运动数据确定出运动者在运动过程中的击球动作以及各个击球动作对应的动作参数后，还能基于第四运动数据确定的心率进一步分析运动者的运动强度和体能，能够更加全面地对运动者的综合运动能力进行评估。
第二方面，本申请实施例提供一种动作识别装置，包括：
第一获取单元，用于获取运动数据；所述运动数据包括第一运动数据、第二运动数据、第三运动数据，其中，所述第一运动数据由位于运动者第一预设部位的第一数据采集装置进行采集；所述第二运动数据由设置于球拍预设位置的第二数据采集装置进行采集；所述第三运动数据由设置于预设拍摄位置的第三数据采集装置进行采集；
第二获取单元，用于根据所述第一运动数据获取运动者在运动过程中的步态特征；根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征；根据所述第三运动数据获取运动者在运动过程中的图像动作特征；
识别单元，用于根据所述步态特征、所述挥拍姿态特征和所述图像动作特征识别出运动者在运动过程中的击球动作。
在第二方面的一种可能的实施方式中，动作识别装置还包括第三获取单元，上述第三获取单元用于根据运动数据获取击球动作的动作参数。
在第二方面的一种可能的实施方式中，第一获取单元还用于获取第四运动数据。相应地，所述动作识别装置还包括生理参数确定单元，所述生理参数确定单元用于根据所述第四运动数据确定运动者在运动过程中的生理参数。
在第二方面的一种可能的实施方式中，上述动作识别装置还包括修正单元，上述修正单元用于当击球动作为有效勾腿杀球时，根据腾空时间和勾腿角度对运动者的跳跃高度进行修正。
第三方面，本申请实施例提供了一种运动监测系统，所述运动监测系统包括第一数据采集装置、第二数据采集装置、第三采集装置以及如第二方面所述的动作识别装置；
所述第一数据采集装置、所述第二数据采集装置、所述第三数据采集装置分别与所述动作识别装置通信连接；
所述第一数据采集装置用于采集第一运动数据；
所述第二数据采集装置用于采集第二运动数据；
所述第三数据采集装置用于采集第三运动数据；
所述动作识别装置用于根据所述第一运动数据、所述第二运动数据以及所述第三运动数据识别运动者在运动过程中的击球动作。
在第三方面的一种可能的实施方式中，所述运动监测系统还包括第四数据采集装置；所述第四数据采集装置与所述动作识别装置通信连接；
所述第四数据采集装置用于采集第四运动数据；
所述动作识别装置还用于根据所述第四运动数据确定运动者在运动过程中的生理参数。
第四方面，本申请实施例提供了一种终端设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如上述第一方面所述的动作识别方法。
第五方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现如上述第一方面所述的动作识别方法。
第六方面，本申请实施例提供了一种计算机程序产品，当计算机程序产品在终端设备上运行时，使得终端设备执行上述第一方面中任一项所述的动作识别方法。
第七方面，提供一种芯片，该芯片包括处理器，当该处理器执行指令时，处理器用于执行上述第二方面中任一种设计所涉及的动作识别方法。该指令可以来自芯片内部的存储器，也可以来自芯片外部的存储器。可选的，该芯片还包括输入输出电路。
可以理解的是，上述第二方面至第七方面的有益效果可以参见上述第一方面中的相关描述，在此不再赘述。
附图说明
图1为本申请实施例提供的一种动作识别方法所适用的运动监测系统的架构示意图；
图2为图1对应的运动监测系统的使用场景示意图；
图3为图1对应的运动监测系统的工作过程的示意框图；
图4为下肢动作分别为走和跳时对应的足部加速度波形和足部角速度波形的对比图；
图5为本申请实施例提供的另一种运动监测系统的架构示意图；
图6为图5对应的运动监测系统的使用场景示意图；
图7为图5对应的运动监测系统的工作过程的示意框图；
图8为本申请实施例提供的一种动作方法的实现流程示意图；
图9为本申请实施例提供的一种动作识别装置的结构示意图；
图10为本申请实施例提供的一种终端设备的结构示意图。
具体实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有 这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
应当理解，当在本申请说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本申请说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
另外，在本申请说明书和所附权利要求书的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
羽毛球、乒乓球、网球等球类运动是深受人们喜爱的运动。本申请实施例提供的动作识别方法和运动监测系统主要用于对羽毛球、乒乓球、网球等球类运动的击球动作进行识别，下面以羽毛球运动为例，对本申请实施例提供的动作识别方法和运动监测系统进行说明，详述如下：
羽毛球作为一项竞技运动，其动作多样化，主要通过高远球、杀球、挑球、搓球等手部动作来完成动作。为了对羽毛球运动者的综合运动能力进行评估并制定个性化的训练计划，需要对羽毛球运动者在羽毛球运动过程中的击球类型、击球力道、击球速度、击球次数等运动参数进行统计。而目前的智能穿戴设备只能获取运动者在球类运动过程中心率、移动步数、移动距离等基本参数，基于这些基本参数无法分析出运动者在运动过程中的综合运动能力。
目前市场上也出现了一些羽毛球动作检测装置，例如设置在羽毛球拍底部的羽毛球动作检测配件，羽毛球动作检测配件能够进行羽毛球击球动作和识别等。然而由于羽毛球动作检测配件设置在球拍上，因此只能进行上肢击球动作的检测，不能实现对用户的下肢动作的检测，也无法对用户的生命体征参数进行检测。因此，目前的羽毛球的击球动作识别方法存在无法结合用户下肢动作和生命体征参数对运动者的综合运动能力进行全面科学的分析和指导的问题。
为了解决上述目前的羽毛球的击球动作识别方法存在无法结合用户下肢动作和生命体征参数对运动者的综合运动能力进行全面科学的分析和指导的问题，本申请实施 例提供了一种动作识别方法和运动监测系统。
请参阅图1，图1示出了本申请实施例提供的动作识别方法所适用的运动监测系统的架构示意图。如图1所示，运动监测系统100可以包括第一数据采集装置1001、第二数据采集装置1002、第三数据采集装置1003和动作识别装置1004。上述第一数据采集装置1001、第二数据采集装置1002、第三数据采集装置1003分别与动作识别装置1004通信连接。动作识别装置1004能够基于各个数据采集装置采集到的运动数据进行特征提取和动作识别，通过利用多个运动数据识别出用户的步态特征、挥拍姿态特征以及图像动作特征，并基于步态特征、挥拍姿态特征以及图像动作特征确定出运动者的击球动作类型，能够准确地对运动过程中运动者的击球动作进行识别，有利于对运动者的综合运动能力进行全面地分析，更便于针对运动者制定个性化的训练计划。
上述动作识别装置1004可以通过近距离通信连接方式分别与第一数据采集装置1001、第二数据采集装置1002、第三数据采集装置1003建立无线通信连接，或者，动作识别装置1004可以通过有线通信方式分别与第一数据采集装置1001、第二数据采集装置1002、第三数据采集装置1003建立有线通信连接。本申请实施例不对动作识别装置1004与第一数据采集装置1001之间、动作识别装置1004与第二数据采集装置1002之间、动作识别装置1004与第三数据采集装置1003之间的具体通信方式做任何限定。
其中，近距离通信连接方式可以是蓝牙连接、近场通信(Near Field Communication，NFC)连接、无线保真(Wireless-Fidelity，WiFi)连接或紫蜂(ZigBee)连接等，为了提高运动者使用的便利性，本实施例中可优选采用蓝牙连接；近距离通信标识为终端设备近距离通信连接相关的唯一性标识，若采用蓝牙连接，则近距离通信标识对应可以为蓝牙媒体访问控制(Media Access Control，MAC)地址或蓝牙设备的其他唯一性标识。
在本申请实施例中，上述第一数据采集装置1001用于采集第一运动数据。具体的，上述第一数据采集装置1001可以设置于运动者的第一预设部位上。上述第一运动数据为可以体现运动者的下肢动作的运动数据。第一数据采集装置1001还用于将实时采集到的第一运动数据发送给与其通信连接的动作识别装置1004。
在本申请实施例中，上述运动者的第一预设部位可以是运动者的小腿或大腿位置，也可以是运动者的脚踝处，当然还可以是能够采集到运动者的下肢动作的运动数据的其他预设部位。为了便于理解，下面以上述第一预设部位为运动者的脚踝为例进行说明。需要说明的是，上述第一数据采集装置1001可以只在运动者的一个脚上，也可以在两个脚上都设置第一数据采集装置，在此不加以限制。
在本申请实施例中，上述第一运动数据可以包括运动者在运动过程中的足部加速度数据和足部角速度数据。具体的，上述第一数据采集装置1001可以是具备有加速度计和陀螺仪的电子设备。上述电子设备可以采集到运动者在运动过程中足部加速度数据和足部角速度数据。
请参阅图2，在具体应用中，上述第一数据采集装置1001可以是包括六轴惯性传感器的可穿戴设备。运动者在进行羽毛球训练时，可以将上述包括六轴惯性传感器的 可穿戴设备穿戴在脚踝处。在运动者的训练过程中，上述六轴惯性传感器就能够实时采集上述运动者在运动过程中的足部加速度数据和足部角速度数据。
当然，上述第一数据采集装置1001也可以是智能鞋。智能鞋中内置有六轴惯性传感器，运动者穿上该智能鞋后，就能够通过智能鞋中的六轴惯性传感器来实时采集上述运动者在运动过程中的足部加速度数据和足部角速度数据。
在具体应用中，上述第二数据采集装置1002用于采集第二运动数据。具体的，上述第二数据采集装置可以设置于球拍的预设位置。上述第二运动数据为可以体现运动者的上肢击球动作的运动数据。上述第二数据采集装置1002还用于将实时采集到的第二运动数据发送给与其通信连接的动作识别装置1004。
在本申请实施例中，上述球拍的预设位置可以是球拍的拍柄，也可以是球拍的拍杆，还可以是球拍的拍头，在此不加以限制。在本申请实施例中，以上述球拍的预设位置为球拍的拍柄为例进行说明。
请参阅图2，上述第二数据采集装置1002可以内嵌在球拍的拍柄中，具体的，可以是将第二数据采集装置1002内嵌在球拍的拍柄的后盖上。通过内嵌在球拍的拍柄中的第二数据采集装置1002采集能够体现上肢击球动作的第二运动数据。
在本申请实施例中，上述第二运动数据可以包括运动者在运动过程中的手部加速度数据、手部角速度数据和声波数据。具体的，上述第二数据采集装置1002可以包括加速度传感器、陀螺仪和麦克风。上述加速度传感器用于实时采集运动者在运动过程中的手部加速度数据，上述陀螺仪用于实时采集运动者在运动过程中的手部角速度数据，上述麦克风用于实时采集运动者在运动过程中的声波数据。具体的，上述第二数据采集装置1002也可以包括六轴惯性传感器和麦克风。上述六轴惯性传感器用于实时采集运动者在运动过程中的手部加速度数据和手部角速度数据，上述麦克风用于实时采集运动者在运动过程中的声波数据。
在本申请实施例中，上述第三数据采集装置1003用于采集第三运动数据。上述第三数据采集装置1003可以设置于预设拍摄位置。上述第三运动数据为可以体现运动者的图像动作特征的运动数据。上述第三数据采集装置1003还用于将其实时采集到的第三运动数据发送给与其通信连接的动作识别装置1004。
在本申请实施例中，上述第三运动数据可以是运动者的运动图像。上述预设拍摄位置是指能够拍摄到运动者的运动图像的位置，上述预设拍摄位置可以根据实际需要进行确定，在此不加以赘述。
请参阅图2，在本申请实施例中，为了全方位的捕捉运动者的运动图像，上述预设拍摄位置可以是球场的四个角落。即通过设置在球场的四个角落的第三数据采集装置采集运动者在运动过程中的运动图像。
在本申请实施例中，上述第三数据采集装置1003可以是能够采集到运动者的运动图像的终端设备，例如具备摄像功能的手机、摄像头等。以上述第三数据采集装置1003为手机为例，上述第三运动数据就是手机拍摄到的图像数据或视频数据。
在本申请实施例中，动作识别装置1004可以是具备数据处理能力的终端设备，动作识别装置能够对第一数据采集装置1001采集到的第一运动数据进行处理，进而得到运动者在运动过程中的步态特征；也能够对第二数据采集装置1002采集到的第二运动 数据进行处理，进而得到运动者在运动过程中的挥拍姿态特征；还能够对第三数据采集装置1003采集到的第三运动数据进行处理，进而得到运动者在运动过程中的图像动作特征。上述动作识别装置1004还能够基于上述步态特征、上述挥拍姿态特征以及上述图像动作特征识别出运动者在运动过程中的击球动作。
在本申请实施例中，上述动作识别装置1004可以是独立的终端设备，例如可以是可穿戴设备，也可以是手机、平板电脑、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等移动终端，还可以是具备数据处理能力的服务器设备，本申请实施例不对动作识别装置1004的具体类型作任何限制。此外，上述动作识别装置1004还可以是不具备硬件结构的虚拟计算机和云服务器等具备数据处理能力的虚拟终端。
在本申请实施例中，上述动作识别装置1004还可以与第一数据采集装置1001设置在同一个电子设备中，即该电子设备可以包括第一数据采集装置1001和动作识别装置1004。可以理解的是，上述动作识别装置1004也可以与第二数据采集装置1002设置在同一个电子设备中，即该电子设备可以包括第二数据采集装置1002和动作识别装置1004。同样地，上述动作识别装置1004也可以与第三数据采集装置1003设置在同一个电子设备中，即该电子设备可以包括第三数据采集装置1003和动作识别装置1004。
请参阅图3，在本申请实施例中，第一数据采集装置10001将其实时采集到的第一运动数据发送给动作识别装置1004。动作识别装置1004在获取到第一运动数据后，可以对第一运动数据进行单步分割，进而确定出运动者在运动过程中的步态特征。上述运动者在运动过程中的步态特征包括但不限于下肢动作的分类和下肢动作的动作参数。
在具体应用中，上述第一运动数据包括足部加速度数据和足部角速度数据，上述对第一运动数据处理的过程具体为：根据第一运动数据绘制足部加速度波形和足部角速度波形，然后根据足部加速度波形和足部角速度波形确定每一步的离地点和触地点，根据每一步的离地点和触地点对足部加速度波形和足部角速度波形进行单步分割，并根据分割后的足部加速度波形和足部角速度波形确定出运动者在运动过程中的单步步态特征。
足部加速度波形中两个相邻波峰可以表示一个动作周期(即一步)，两个相邻波峰的第一个波峰所对应的时间点就是离地点，第二个波峰所对应的时间点就是触地点，离地点与触地点之前的时间段就是一个动作周期。根据足部加速度波形就能够确定出每一步的离地点和触地点，进而确定出每一个动作周期，在基于动作周期将足部加速度的波形图和足部角速度的波形图进行单步分割。
每个动作周期对应的下肢动作可以是走、跑以及跳跃等。不同的下肢动作所对应的足部加速度波形和足部角速度波形是不同的，因此，根据足部加速度波形和足部角速度波形就能够确定出每个动作周期所对应的下肢动作。示例性的，请参阅图4，当下肢动作为走时，足部加速度波形和足部角速度波形如图4中的(a)所示；当下肢动作为跳跃时，足部加速度波形和足部角速度波形如图4中的(b)所示。需要说明的是，图4中的横坐标轴表示的是第一运动数据的采集时间，单位是微秒(μs)；纵坐标轴表 示的加速度的值以及角速度的值，当纵坐标轴表示加速度的值时，单位是米每平方秒(m/s2)，当纵坐标表示角速度的值时，单位是弧度每秒(rad/s)。
当确定了每个动作周期对应的下肢动作后，还能够基于每个动作周期内的加速度、角速度、离地点以及触地点确定出每个动作周期对应的下肢动作的动作参数，例如腾空时间、跳跃高度、移动距离、移动速度等。
请继续参考图3，在本申请实施例中，上述第二数据采集装置1002能够将其实时采集到的第二运动数据发送给与其通信连接的动作识别装置1004。动作识别装置1004在获取到第二运动数据后，可以对第二运动数据进行处理，进而确定出运动者在运动过程中的挥拍姿态特征。上述运动者在运动过程中的挥拍姿态特征包括但不限于手部动作的类别、手部动作的动作参数以及是否有效击球。
在具体应用中，上述第二运动数据可以包括手部加速度数据、手部角速度数据以及声波数据，上述对第二数据进行处理的过程具体为：根据手部加速度数据和手部角速度数据绘制手部加速度波形和手部角速度波形，然后基于手部加速度波形和手部角速度波形确定出每一次挥拍时的手部动作的分类和动作参数。基于声波数据提取出声波数据的特征值，然后基于声波数据的特征值进行有效击球和空挥的识别。最后根据基于手部加速度波形和手部角速度波形确定出每一次挥拍时的手部动作的分类和动作参数和基于声波数据的特征值进行击球和空挥的分类确定出运动者每次挥拍时的挥拍姿态特征。
上述挥拍时的手部动作的分类包括但不限于高远球、杀球、搓球等，上述挥拍的动作参数包括击球速度、击球力度、击球轨迹等。
基于手部加速度波形和手部角速度波形确定出每一次挥拍时的手部动作的分类和挥拍时的动作参数可以是基于现有的分类算法模型来实现，即将手部加速度波形和手部角速度波形输入到现有的分类算法模型中进行处理，就能够得到每一次挥拍时的手部动作的分类和挥拍时的动作参数，在此不再加以赘述。
上述声波数据的特征值包括但不限于能量、峰值以及频率。由于声波数据的时域波形只体现声压随时间变化的关系,而不能体现声波数据的特征值。因此，需要将声波数据的时域波形转换为能够体现特征值的频域波形，然后在提取对相应的特征值。目前有许多声波特征值的提取方法，如基于梅尔频率倒谱系数(mel-frequency cepstral coefficients，MFCCs)提取特征值的方法、基于线性预测倒谱系数(linear prediction cepstrum coefficient，LPCC)提取特征值的方法等。基于声波数据的特征值进行有效击球和空挥的识别过程具体为：将提取到的能量、峰值以及频率通过预设高斯混合模型进行相似度匹配计算，当计算得到的相似度大于预设阈值时，确定当前手部动作为有效击球，否则确定当前手部动作是空挥(即无效击球)。
上述预设高斯混合模型是基于每个运动者的历史运动数据确定出的高斯混合模型，每个运动者都有其对应的预设高斯混合模型，可以通过运动者ID将运动者与预设高斯混合模型进行关联。动作识别装置1004能够根据当前正在进行训练的运动者的运动者ID确定出对应的预设高斯混合模型。而预设高斯混合模型就是确定了高斯混合模型参数的最优解的高斯混合模型，上述高斯混合模型参数可表示为λ。λ的最优解的求解过程即为高斯混合模型的训练过程。
在具体应用中，为了确定高斯混合模型参数λ的最优解，可以用N个M维的高斯密度函数线性加权函数来表示上述高斯混合模型，其中，N和M均是大于1的正整数。
其中，上述高斯混合模型的函数可以表示为：
Figure PCTCN2021118320-appb-000001
其中，X是声波数据的M维特征参数，X＝(X 1,X 2,...,X M)，ω i是指第i个高斯密度函数的权重，且
Figure PCTCN2021118320-appb-000002
G i(X)为特征参数X的高斯概率密度函数。
Figure PCTCN2021118320-appb-000003
其中，C i是指第i个高斯密度函数的协方差矩阵，μ i是指第i个高斯密度函数的均值向量。
其中，C i是指第i个高斯密度函数的协方差矩阵，μ i是指第i个高斯密度函数的均值向量。
具体的，λ＝(ω i,μ i,C i)可以基于期望最大化算法的迭代计算对λ的最优解进行估计。
在确定出运动者挥拍时是有效击球还是空挥后，结合每一次挥拍时的手部动作的分类，就能够确定出挥拍姿态特征。示例性的，假设基于手部加速度数据、手部角速度数据确定出运动者在运动过程中的手部动作的分类为：第一个手部动作是高远球、第二个手部动作是杀球、第三个手部动作是搓球；基于声波数据确定出第一手部动作是空挥、第二个手部动作是有效击球、第三个手部动作是有效击球；则可以确定运动者在运动过程中的挥拍姿势特征为：第一个手部动作为无效高远球以及击球时的击球速度、击球力度、击球轨迹等；第二个手部动作为有效杀球以及击球时的击球速度、击球力度、击球轨迹等；第三个手部动作为有效搓球以及击球时的击球速度、击球力度、击球轨迹等。
请继续参考图3，在本申请实施例中，上述第三数据采集装置1003能够将其实时采集到的第三运动数据发送给与其通信连接的动作识别装置1004。动作识别装置1004在获取到第三运动数据后，能够基于第三运动数据和卷积神经网络模型分析出运动者在运动过程中的图像动作特征。
在本申请实施例中，上述第三运动数据为运动者的运动图像。上述动作识别装置1004在对第三运动数据进行处理时，可以先进行目标检测，在运动图像中框选出包含运动者的目标图像区域，然后基于该目标图像区域中的图像来识别图像中运动员的勾腿特征。这样能够有效地减少动作识别装置1004利用卷积神经网络模型对运动图像进行处理时的运算量，减少计算资源的占用，提高处理效率。
上述图像动作特征包括但不限于图像中运动员的勾腿特征(勾腿和未勾腿)。
上述卷积神经网络模型是指已经训练好的能够用于对第三运动数据进行处理以确定出图像动作特征的卷积神经网络模型。用于图像处理的卷积神经网络模型可以采用现有的卷积神经网络模型，对卷积神经网络模型进行训练的过程可以使用大量的已知分类的运动图像对卷积神经网络模型进行训练，使得训练好的卷积神经网络模型能够在输入第三运动数据时，输入第三运动数据对应的图像动作特征。关于构建和训练卷积神经网络模型的过程可以参见现有的构建和训练的方法，本申请实施例在此不再加 以赘述。
训练好的卷积神经网络模型可以存储在动作识别装置1004的数据存储区，动作识别装置1004在接收到第三数据采集装置1003采集的第三运动数据后，会自动调用该卷积神经网络模型对第三运动数据进行处理，以得到第三运动数据对应的图像动作特征。
可以理解的是，上述图像动作特征还可以包括击球动作分类，即上述卷积神经网络模型还能够基于输入的运动图像确定出运动者的击球动作的分类。然后结合第一运动数据确定的步态特征和第二运动数据确定的挥拍姿态特征综合性地确定出运动者的击球动作以及击球动作参数。
请继续参考图3，在本申请实施例中，上述动作识别装置1004在得到步态特征、挥拍姿势特征以及图像动作特征后，能够融合这几个特征(即步态特征、挥拍姿势特征以及图像动作特征)最终确定出运动过程中运动者的击球动作。同时也能够确定出运动过程中运动者在击球时的击球速度、击球力度、腾空时间、跳跃高度、移动距离、移动速度等动作参数。
在本申请实施例中，上述运动者的击球动作可以包括有效高远球、有效勾腿跳杀、有效未勾腿跳杀、有效原地杀球、有效搓球等、无效高远球、无效跳杀、无效原地杀球、无效搓球等。
融合步态特征、挥拍姿势特征以及图像动作特征最终确定出运动过程中运动者的击球动作可以是通过将步态特征、挥拍姿势特征和图像动作特征先进行结合，以此来确定在击球时运动者的击球动作；然后在运动者在跳跃时存在勾腿的情况时确定勾腿角度，再基于腾空时间和勾腿角度对运动者的跳跃高度进行修正，最终识别出运动者在运动过程中存在的击球动作以及各个击球动作对应的动作参数。
示例的，假设根据第一数据采集装置1001采集到的第一运动数据确定出的运动者在运动过程中的下肢动作分别为走、跳跃、跳跃；根据第二数据采集装置1002采集到的第二运动数据确定的手部动作的类别分别为有效搓球、无效杀球、有效杀球；根据第三数据采集装置1003采集到的第三运动数据确定的勾腿特征分别为未勾腿、未勾腿、勾腿；通过将上述特征进行融合可以确定出运动者在运动过程中的击球动作分别为有效搓球、无效跳杀、有效勾腿跳杀。
在本申请实施例中，只有当检测到跳跃动作时，获取跳跃动作对应的动作图像，然后基于该动作图像识别出运动者是否勾腿，若存在勾腿的情况，则结合勾腿角度和腾空时间对跳跃高度进行修正；若运动者在跳跃动作下未勾腿，则无需进行修正。需要说明的是，对于跳跃高度的修正，可以基于已有的修正模型来实现，在此不再加以赘述。
综上可知，这样就能够全面准确地分析出运动者在运动过程中的击球动作，并确定出各个击球动作对应的动作参数，有利于对运动者的综合运动能力进行全面地分析，更便于针对运动者制定个性化的训练计划。
请参阅图5，图5是本申请实施例提供的另一种运动监测系统的结构框图。如图5所示，区别于本申请上一实施例，该运动监测系统还包括第四数据采集装置1005。上述第四数据采集装置1005也与所述动作识别装置1004通信连接。
上述动作识别装置1004可以通过近距离通信连接方式与第四数据采集装置1005建立无线通信连接，或者，动作识别装置1004可以通过有线通信方式与第四数据采集装置1005建立有线通信连接。本申请实施例不对动作识别装置1004与第四数据采集装置1005之间的具体通信方式做任何限定。
其中，近距离通信连接方式可以是蓝牙连接、近场通信(Near Field Communication，NFC)连接、无线保真(Wireless-Fidelity，WiFi)连接或紫蜂(ZigBee)连接等，为了提高运动者使用的便利性，本实施例中可优选采用蓝牙连接；近距离通信标识为终端设备近距离通信连接相关的唯一性标识，若采用蓝牙连接，则近距离通信标识对应可以为蓝牙媒体访问控制(Media Access Control，MAC)地址或蓝牙设备的其他唯一性标识。
上述第四数据采集装置1005用于采集第四运动数据。上述第四数据采集装置可以设置于运动者的第二预设部位。上述第四运动数据为能够体现运动者在运动过程中生理参数的运动数据。所述生理参数包括但不限于心率、脉搏、体温等。在本申请实施例中，上述第四运动数据包括但不限运动者在运动过程中的心率数据。
在本申请实施例中，上述运动者的第二预设部位是指能够采集到第四运动数据的部位，例如运动者的手腕。当然，上述运动者的第二预设部位还可以是其他能够采集到第四运动数据的部位，例如指尖、脖子等部位，在此不加以限制。
请参阅图6，在具体应用中，上述第四数据采集装置1005可以是包括能够采集心率数据的可穿戴设备，例如智能手表、智能手环等。运动者在进行羽毛球训练时，可以将上述可穿戴设备穿戴在手腕处。运动者在训练过程中，上述可穿戴设备就能够实时采集上述运动者在运动过程中的心率数据。
在实际应用中，上述可穿戴设备可以包括用于采集心率数据的光电容积描记(photoplethysmograph，PPG)传感器，通过上述PPG传感器来采集运动者手腕处的心率数据。
请一并参阅图7，在本申请实施例中，上述动作识别装置1004还能够对第四数据采集装置1005采集到的第四运动数据进行处理，进而确定出运动者在运动过程中的心率。
上述第四运动数据可以是基于光电容积描记(photoplethysmograph，PPG)传感器采集到的PPG数据。具体地，动作识别装置1004能够对第四运动数据进行滤波处理进而确定出运动者在运动过程中的心率。
需要说明的是，当上述第四运动数据为脉搏数据时，动作识别装置能够对脉搏数据进行脉冲波形峰值间隔特征提取处理，进而确定出运动者在运动过程中的心率值。
在本申请实施例中，上述动作识别装置在基于第一运动数据、第二运动数据以及第三运动数据确定出运动者在运动过程中的击球动作以及各个击球动作对应的动作参数后，还能基于第四运动数据确定的心率进一步分析运动者的运动强度和体能。能够更加全面地对运动者的综合运动能力进行评估。
为了更便于理解，下面对动作识别的过程进行详细说明。
图8为本申请实施例提供的动作识别方法的流程示意图.上述动作识别方法的执行主体可以是上述实施例中的动作识别装置，如图8所示，该方法可以包括如下步骤：
S11、获取运动数据。
在此，为了识别出运动者在运动过程中的击球动作，就需要先获取该运动者运动时的运动数据。在本申请实施例中，上述运动数据包括第一运动数据、第二运动数据、第三运动数据。
其中，第一运动数据为能够体现运动者在运动过程中的下肢动作的运动数据。上述第二运动数据为能够体现运动者在运动过程中的上肢击球动作的运动数据。上述第三运动数据为能够体现运动者的图像动作特征的运动数据。
在本申请实施例中，上述第一运动数据可以包括：运动者在运动过程中的足部加速度数据和足部角速度数据。上述第二运动数据可以包括：运动者在运动过程中的手部加速度数据、手部角速度数据和声波数据。上述第三运动数据可以包括：运动者的运动图像。
其中，第一运动数据由位于运动者第一预设部位的第一数据采集装置进行采集，第二运动数据由设置于球拍预设位置的第二数据采集装置进行采集，第三运动数据由设置于预设拍摄位置的第三数据采集装置进行采集。
上述运动者的第一预设部位可以是运动者的小腿或大腿位置，也可以是运动者的脚踝处，当然还可以是能够采集到运动者的下肢动作的运动数据的其他预设部位。
上述球拍的预设位置可以是球拍的拍柄，也可以是球拍的拍杆，还可以是球拍的拍头，在此不加以限制。
上述预设拍摄位置是指能够拍摄到运动者的运动图像的位置，上述预设拍摄位置可以根据实际需要进行确定，在此不加以赘述。
在本申请实施例中，各个数据采集装置可以将实时采集到的运动数据存储在其本地存储器中。基于此，动作识别装置要获取运动数据时，各个数据采集装置就能够将其存储在本地存储器中的运动数据发送给与之通信连接的动作识别装置。
S12、根据运动数据获取姿态特征。
在本申请实施例中，上述根据运动数据获取姿态特征包括：根据第一运动数据获取运动者在运动过程中的步态特征；根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征；根据所述第三运动数据获取运动者在运动过程中的图像动作特征。
在本申请实施例中，上述第一运动数据包括足部加速度数据和足部角速度数据，根据第一运动数据获取运动者在运动过程中的步态特征的过程具体为：根据第一运动数据绘制足部加速度波形和足部角速度波形，然后根据足部加速度波形和足部角速度波形确定每一步的离地点和触地点，根据每一步的离地点和触地点对足部加速度波形和足部角速度波形进行单步分割，并根据分割后的足部加速度波形和足部角速度波形确定出运动者在运动过程中的步态特征。
上述运动者在运动过程中的步态特征包括但不限于下肢动作的分类和下肢动作的动作参数。下肢动作的分类可以是走、跑以及跳跃等。下肢动作的动作参数可以是腾空时间、跳跃高度、移动距离、移动速度等。
在本申请实施例中，上述第二运动数据可以包括手部加速度数据、手部角速度数据以及声波数据，上述根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征的过程具体为：根据手部加速度数据和手部角速度数据绘制手部加速度波形和手部 角速度波形，然后基于手部加速度波形和手部角速度波形确定出每一次挥拍时的手部动作的分类和动作参数；基于声波数据提取出声波数据的特征值，然后基于声波数据的特征值进行有效击球和空挥的识别；最后根据基于手部加速度波形和手部角速度波形确定出每一次挥拍时的手部动作的分类和动作参数和基于声波数据的特征值进行有效击球和空挥的分类确定出运动者每次挥拍时的挥拍姿态特征。
上述运动者在运动过程中的挥拍姿态特征包括但不限于手部动作的类别、手部动作的动作参数以及是否有效击球。
上述挥拍时的手部动作的分类包括但不限于高远球、杀球、搓球等，上述挥拍的动作参数包括击球速度、击球力度、击球轨迹等。
在本申请实施例中，上述第三运动数据可以包括运动者的运动图像。根据所述第三运动数据获取运动者在运动过程中的图像动作特征的过程具体为：对运动图像先进行目标检测，从运动图像中框选出包含运动者的目标图像区域，然后将目标图像区域输入到训练完成的卷积神经网络模型中进行处理，确定出运动者在运动过程中的图像动作特征。
上述图像动作特征包括但不限于图像中运动员的勾腿特征(勾腿和未勾腿)。
可理解的是，上述第三运动数据还可以是包括球场热图等能够用于绘制球场上的运动轨迹分布(运动者的运动轨迹以及球的运动轨迹等)的图像，上述球场热图可以用来辅助分析运动的步态特征以及得失分位置等。
S13、根据姿态特征识别击球动作。
在本申请实施例中，在得到步态特征、挥拍姿势特征以及图像动作特征后，融合步态特征、挥拍姿势特征以及图像动作特征最终识别出运动过程中运动者的击球动作。
融合步态特征、挥拍姿势特征以及图像动作特征最终确定出运动过程中运动者的击球动作可以是通过将步态特征、挥拍姿势特征和图像动作特征先进行结合，以此来确定在击球时运动者的击球动作；然后在运动者在跳跃时存在勾腿的情况时确定勾腿角度，再基于腾空时间和勾腿角度对运动者的跳跃高度进行修正，最终识别出运动者在运动过程中存在的击球动作以及各个击球动作对应的动作参数。
示例的，确定出的运动者在运动过程中的步态特征分别为走、跳跃、跳跃；运动者在运动过程中的挥拍姿势特征为有效搓球、无效杀球、有效杀球；图像动作特征分别为未勾腿、未勾腿、勾腿；通过将上述特征进行融合可以确定出运动者在运动过程中的击球动作分别为有效搓球、无效跳杀、有效勾腿跳杀。
在本申请另一实施例中，还可以根据运动数据获取击球动作的动作参数。
在本申请实施例中，上述击球动作的动作参数可以包括击球速度、击球力度、腾空时间、跳跃高度、移动距离、移动速度等。上述动作参数的获取过程可以参见前述实施例，在此不加以赘述。通过确定出运动员在击球时的动作参数，能够更便于对运动者的综合运动能力进行更具体的分析。
在本申请另一实施例中，当击球动作为有效勾腿杀球时，根据腾空时间和勾腿角度对运动者的跳跃高度进行修正。然后基于修正后的跳跃高度来分析运动者的综合运动能力。关于对跳跃高度的修正过程可以参见前述实施例，在此不加以赘述。通过对运动者的跳跃高度进行修正，能够更准确地统计运动者的运动参数。
在本申请另一实施例中，上述运动数据还可以包括第四运动数据。根据所述第四运动数据能够确定运动者在运动过程中的生理参数。
在具体应用中，上述第四运动数据由设置于运动者的第二预设部位的第四数据采集装置进行采集。上述第四运动数据为能够体现运动者在运动过程中生理参数的运动数据。所述生理参数包括但不限于心率、脉搏、体温等。
在本申请实施例中，上述第四运动数据包括但不限运动者在运动过程中的心率数据。
在本申请实施例中，上述运动者的第二预设部位是指能够采集到第四运动数据的部位，例如运动者的手腕。当然，上述运动者的第二预设部位还可以是其他能够采集到第四运动数据的部位，例如指尖、脖子等部位，在此不加以限制。
具体的，通过对心率数据进行滤波处理，进而确定出运动者在运动过程中的心率。上述对心率数据进行滤波处理确定心率的过程可以参见现有的对心率数据的处理方法，在此不再加以赘述。
本实施例提供的动作识别方法，能够基于各个数据采集装置采集到的运动数据进行特征提取和动作识别，通过利用多个运动数据识别出用户的步态特征、挥拍姿态特征以及图像动作特征，并基于步态特征、挥拍姿态特征以及图像动作特征确定出运动者的击球动作类型，能够准确地对运动过程中运动者的击球动作进行识别，有利于对运动者的综合运动能力进行全面地分析，更便于针对运动者制定个性化的训练计划。
对应于上述实施例所述的动作识别方法，图9示出了本申请实施例提供的一种动作识别装置的结构框图，该动作识别装置包括的各单元用于执行上述实施例中的各步骤，具体请参阅上述实施例中的相关描述，为了便于说明，仅示出了与本申请实施例相关的部分。请参阅图9，该动作识别装置90包括第一获取单元91、第二获取单元92以及识别单元93。其中：
第一获取单元91用于获取运动数据。
上述运动数据包括第一运动数据、第二运动数据、第三运动数据。
第二获取单元92用于根据运动数据获取姿态特征。
第二获取单元92具体用于根据第一运动数据获取运动者在运动过程中的步态特征；根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征；根据所述第三运动数据获取运动者在运动过程中的图像动作特征。
识别单元93用于根据姿态特征识别击球动作。
识别单元93具体用于根据步态特征、挥拍姿势特征以及图像动作特征识别出运动者在运动过程中的击球动作。
在本申请一实施例中，上述动作识别装置还包括第三获取单元，上述第三获取单元用于根据运动数据获取击球动作的动作参数。
在本申请实施例中，上述第一获取单元还用于获取第四运动数据。相应地，所述动作识别装置还包括生理参数确定单元，所述生理参数确定单元用于根据所述第四运动数据确定运动者在运动过程中的生理参数。
在本申请一实施例中，上述动作识别装置还包括修正单元，上述修正单元用于当击球动作为有效勾腿杀球时，根据腾空时间和勾腿角度对运动者的跳跃高度进行修正。
以上可以看出，本申请实施例提供的一种动作识别装置，同样能够基于各个数据采集装置采集到的运动数据进行特征提取和动作识别，通过利用多个运动数据识别出用户的步态特征、挥拍姿态特征以及图像动作特征，并基于步态特征、挥拍姿态特征以及图像动作特征确定出运动者的击球动作类型，能够准确地对运动过程中运动者的击球动作进行识别，有利于对运动者的综合运动能力进行全面地分析，更便于针对运动者制定个性化的训练计划。
请参阅图10，图10是本申请另一实施例提供的终端设备的结构示意图。如图10所示，该实施例的终端设备10包括：至少一个处理器101(图10中仅示出一个)、存储器102以及存储在所述存储器102中并可在所述至少一个处理器101上运行的计算机程序103，所述处理器101执行所述计算机程序103时实现上述任意各个动作识别方法实施例中的步骤。
所述终端设备10可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。该终端设备可包括，但不仅限于，处理器101、存储器102。本领域技术人员可以理解，图10仅仅是终端设备10的举例，并不构成对终端设备10的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如还可以包括输入输出设备、网络接入设备等。
所称处理器101可以是中央处理单元(Central Processing Unit，CPU)，该处理器101还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器102在一些实施例中可以是所述终端设备10的内部存储单元，例如终端设备10的硬盘或内存。所述存储器102在另一些实施例中也可以是所述终端设备10的外部存储设备，例如所述终端设备10上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，所述存储器102还可以既包括所述终端设备100的内部存储单元也包括外部存储设备。所述存储器102用于存储操作系统、应用程序、引导装载程序(Boot Loader)、数据以及其他程序等，例如所述计算机程序的程序代码等。所述存储器102还可以用于暂时地存储已经输出或者将要输出的数据。在本申请实施例中，上述存储器102还可以存储基于本申请实施例提供的动作识别方法确定的动作识别算法库。
需要说明的是，上述装置/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参见方法实施例部分，此处不再赘述。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一 个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时可实现上述动作识别方法中的步骤。
本申请实施例提供了一种计算机程序产品，当计算机程序产品在移动终端上运行时，使得移动终端执行时可实现上述动作识别方法中的步骤。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中的交互方法。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质至少可以包括：能够将计算机程序代码携带到拍照装置/终端设备的任何实体或装置、记录介质、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区，根据立法和专利实践，计算机可读介质不可以是电载波信号和电信信号。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/网络设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/网络设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分别到 多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (13)

  1. 一种动作识别方法，其特征在于，包括：
    获取运动数据；所述运动数据包括第一运动数据、第二运动数据、第三运动数据，其中，所述第一运动数据由位于运动者第一预设部位的第一数据采集装置进行采集；所述第二运动数据由设置于球拍预设位置的第二数据采集装置进行采集；所述第三运动数据由设置于预设拍摄位置的第三数据采集装置进行采集；
    根据所述第一运动数据获取运动者在运动过程中的步态特征；
    根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征；
    根据所述第三运动数据获取运动者在运动过程中的图像动作特征；
    根据所述步态特征、所述挥拍姿态特征和所述图像动作特征识别出运动者在运动过程中的击球动作。
  2. 如权利要求1所述的动作识别方法，其特征在于，所述第一运动数据包括足部加速度数据和足部角速度数据，所述根据所述第一运动数据获取运动者在运动过程中的步态特征，包括：
    根据所述足部加速度数据和所述足部角速度数据绘制足部加速度波形和足部角速度波形，并提取足部加速度波形特征；
    根据所述足部加速度波形特征确定每一步的离地点和触地点；
    根据每一步的离地点和触地点对所述足部加速度波形和所述足部角速度波形进行单步分割；
    根据分割后的足部加速度波形和分割后的足部角速度波形确定运动者每一步对应的下肢动作的分类和下肢动作的动作参数。
  3. 如权利要求1所述的动作识别方法，其特征在于，所述第二运动数据包括手部加速度数据、手部角速度数据以及声波数据，所述根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征，包括：
    根据所述手部加速度数据和所述手部角速度数据确定运动者在运动过程中的手部动作的分类和手部动作的动作参数；
    根据所述声波数据对运动者在运动过程中的每一个手部动作进行有效击球和空挥的识别；
    根据运动者在运动过程中的手部动作的分类和手部动作的动作参数以及每一个手部动作进行有效击球和空挥的识别确定所述运动者在运动过程中的挥拍姿态特征。
  4. 如权利要求1所述的动作识别方法，其特征在于，所述第三运动数据包括运动者的运动图像，所述根据所述第三运动数据获取运动者在运动过程中的图像动作特征，包括：
    将运动者的运动图像输入到完成训练的卷积神经网络模型中进行处理，得到所述运动者的运动图像对应的图像动作特征。
  5. 如权利要求4所述的动作识别方法，其特征在于，在将运动者的运动图像输入到完成训练的卷积神经网络模型中进行处理，得到所述运动者的运动图像对应的图像动作特征之前，还包括：
    从所述运动者的运动图像中框选出目标图像区域，所述目标图像区域为只包含运 动者的图像区域。
  6. 如权利要求1至5任一项所述的动作识别方法，其特征在于，所述动作识别方法还包括：
    当识别出击球动作为有效勾腿杀球时，获取勾腿角度、腾空时间以及待修正的跳跃高度；
    基于所述腾空时间和所述勾腿角度对所述待修正的跳跃高度进行修正。
  7. 如权利要求1至6任一项所述的动作识别方法，其特征在于，所述运动数据还包括第四运动数据，相应地，所述动作识别方法还包括：
    根据所述第四运动数据确定运动者在运动过程中的生理参数。
  8. 一种动作识别装置，其特征在于，包括：
    第一获取单元，用于获取运动数据；所述运动数据包括第一运动数据、第二运动数据、第三运动数据，其中，所述第一运动数据由位于运动者第一预设部位的第一数据采集装置进行采集；所述第二运动数据由设置于球拍预设位置的第二数据采集装置进行采集；所述第三运动数据由设置于预设拍摄位置的第三数据采集装置进行采集；
    第二获取单元，用于根据所述第一运动数据获取运动者在运动过程中的步态特征；根据所述第二运动数据获取运动者在运动过程中的挥拍姿态特征；根据所述第三运动数据获取运动者在运动过程中的图像动作特征；
    识别单元，用于根据所述步态特征、所述挥拍姿态特征和所述图像动作特征识别出运动者在运动过程中的击球动作。
  9. 一种运动监测系统，其特征在于，所述运动监测系统包括第一数据采集装置、第二数据采集装置、第三采集装置以及如权利要求8所述的动作识别装置；
    所述第一数据采集装置、所述第二数据采集装置、所述第三数据采集装置分别与所述动作识别装置通信连接；
    所述第一数据采集装置用于采集第一运动数据；
    所述第二数据采集装置用于采集第二运动数据；
    所述第三数据采集装置用于采集第三运动数据；
    所述动作识别装置用于根据所述第一运动数据、所述第二运动数据以及所述第三运动数据识别运动者在运动过程中的击球动作。
  10. 根据权利要求9所述的运动监测系统，其特征在于，所述运动监测系统还包括第四数据采集装置；所述第四数据采集装置与所述动作识别装置通信连接；
    所述第四数据采集装置用于采集第四运动数据；
    所述动作识别装置还用于根据所述第四运动数据确定运动者在运动过程中的生理参数。
  11. 一种终端设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述的动作识别方法。
  12. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的动作识别方法。
  13. 一种芯片，其特征在于，包括处理器，所述处理器和存储器耦合，所述存储器用于存储计算机程序指令，当所述处理器执行所述计算机程序指令时，使得芯片执行如权利要求1至7中任意一项所述的动作识别方法。
PCT/CN2021/118320 2020-09-15 2021-09-14 动作识别方法及装置、终端设备、运动监测系统 Ceased WO2022057795A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/921,194 US12374161B2 (en) 2020-09-15 2021-09-14 Action recognition method and apparatus, terminal device, and motion monitoring system
EP21868618.6A EP4119204B1 (en) 2020-09-15 2021-09-14 Motion recognition method and apparatus, terminal device, and sport monitoring system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010970737.1 2020-09-15
CN202010970737.1A CN114259720B (zh) 2020-09-15 2020-09-15 动作识别方法及装置、终端设备、运动监测系统

Publications (1)

Publication Number Publication Date
WO2022057795A1 true WO2022057795A1 (zh) 2022-03-24

Family

ID=80776451

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/118320 Ceased WO2022057795A1 (zh) 2020-09-15 2021-09-14 动作识别方法及装置、终端设备、运动监测系统

Country Status (4)

Country Link
US (1) US12374161B2 (zh)
EP (1) EP4119204B1 (zh)
CN (1) CN114259720B (zh)
WO (1) WO2022057795A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115862130A (zh) * 2022-11-16 2023-03-28 之江实验室 基于人体姿态及其躯干运动场的行为识别方法
CN116229576A (zh) * 2023-03-13 2023-06-06 北京数感科技有限公司 着地动作类型的确定方法、装置、终端及存储介质

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114259720B (zh) * 2020-09-15 2022-10-18 荣耀终端有限公司 动作识别方法及装置、终端设备、运动监测系统
CN116310752A (zh) * 2023-02-15 2023-06-23 维沃移动通信有限公司 运动项目识别方法、装置、电子设备和介质
CN117122881A (zh) * 2023-09-26 2023-11-28 南京工程学院 一种乒乓球智能训练教学系统
CN117333947B (zh) * 2023-10-18 2024-05-10 首都体育学院 一种羽毛球动作分析方法与系统
CN118749959A (zh) * 2024-07-19 2024-10-11 所乐思(深圳)科技有限公司 智能眼镜系统、智能眼镜以及运动数据分析方法
CN119587011A (zh) * 2024-11-19 2025-03-11 维沃移动通信有限公司 运动评价方法、装置、电子设备和可读存储介质
CN120643201B (zh) * 2025-08-15 2025-10-28 陕西润之宸实业有限公司 一种心率数据分析系统及方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2953115A1 (en) * 2014-06-06 2015-12-09 Head Technology GmbH Swing analysis system
CN105898241A (zh) * 2016-01-15 2016-08-24 上海英诗帕信息科技有限公司 一种运动视觉呈现系统及方法
CN106581974A (zh) * 2017-02-13 2017-04-26 张琛 基于球拍运动轨迹的击球姿势训练装置及方法
CN107982899A (zh) * 2017-12-13 2018-05-04 郑州大学体育学院 一种羽毛球比赛数据记录管理系统
CN207898912U (zh) * 2018-02-13 2018-09-25 李宁（中国）体育用品有限公司 羽毛球训练系统
CN108671505A (zh) * 2018-03-20 2018-10-19 杭州电子科技大学 一种羽毛球运动辅助系统及其控制方法
US10248985B2 (en) * 2015-01-16 2019-04-02 Brooks Sports, Inc. Systems and methods for analyzing lower body movement to recommend footwear

Family Cites Families (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10124242A1 (de) * 2001-05-18 2002-11-28 Juergen Loeschinger Vorrichtung und Verfahren zur Haltungskontrolle einer Person
CN101158883B (zh) * 2007-10-09 2010-07-28 深圳泰山网络技术有限公司 一种基于计算机视觉的虚拟体育系统及其实现方法
JP5761505B2 (ja) * 2011-06-09 2015-08-12 セイコーエプソン株式会社 スイング分析装置、スイング分析システム、スイング分析方法、スイング分析プログラム、および記録媒体
JP5994306B2 (ja) * 2012-03-15 2016-09-21 ソニー株式会社 情報処理装置、情報処理システムおよびプログラム
CN102799263B (zh) * 2012-06-19 2016-05-25 深圳大学 一种姿态识别方法及姿态识别控制系统
CN203017737U (zh) * 2012-11-06 2013-06-26 安徽寰智信息科技股份有限公司 一种基于动作识别技术的网球训练辅助系统
EP2973215B1 (en) * 2013-03-15 2023-05-17 NIKE Innovate C.V. Feedback signals from image data of athletic performance
JP5803962B2 (ja) * 2013-03-22 2015-11-04 ソニー株式会社 情報処理装置、センサ装置、情報処理システムおよび記録媒体
US9457228B2 (en) * 2013-07-09 2016-10-04 Aditi Sinha Sport training equipment
US9710612B2 (en) * 2014-05-05 2017-07-18 Sony Corporation Combining signal information from shoes and sports racket
EP3194036A4 (en) 2014-08-11 2017-12-27 Icuemotion LLC Codification and cueing system for human interactions in tennis and other sport and vocational activities
US9731172B2 (en) * 2014-12-09 2017-08-15 Dunlop Sports Co., Ltd Sporting apparatus with monitoring device
CA3031040C (en) * 2015-07-16 2021-02-16 Blast Motion Inc. Multi-sensor event correlation system
US20190054347A1 (en) * 2015-08-18 2019-02-21 Michael Saigh Wearable sports guidance communication system and developers tool kit
US10463909B2 (en) * 2015-12-27 2019-11-05 Seismic Holdings, Inc. System and method for using performance signatures
CN105912142B (zh) * 2016-02-05 2019-06-11 深圳市爱康伟达智能医疗科技有限公司 一种基于加速传感器的记步与行为识别方法
CN107281717A (zh) * 2016-04-05 2017-10-24 深圳市酷浪云计算有限公司 羽毛球拍及其挥拍分析系统
CN106310609B (zh) * 2016-09-29 2019-10-29 广东小天才科技有限公司 一种羽毛球运动的分析方法及装置
CN106778477B (zh) * 2016-11-21 2020-04-03 深圳市酷浪云计算有限公司 网球拍动作识别方法及装置
CN106730771B (zh) * 2017-01-05 2018-12-21 大连理工大学 一种基于单元动作划分的篮球动作数据处理方法
CN107273857B (zh) * 2017-06-19 2021-03-02 深圳市酷浪云计算有限公司 运动动作的识别方法及装置、电子设备
CN107281709B (zh) * 2017-06-27 2019-04-05 深圳市酷浪云计算有限公司 一种运动视频片段的提取方法及装置、电子设备
EP3649633A4 (en) * 2017-07-06 2021-03-10 Icuemotion LLC SYSTEMS AND PROCEDURES FOR DATA-CONTROLLED MOBILITY TRAINING
US11971951B2 (en) * 2018-06-21 2024-04-30 City University Of Hong Kong Systems and methods using a wearable sensor for sports action recognition and assessment
US11379725B2 (en) * 2018-06-29 2022-07-05 International Business Machines Corporation Projectile extrapolation and sequence synthesis from video using convolution
CN109260647A (zh) * 2018-09-10 2019-01-25 郑州大学 基于多模态信号的人体跳跃指标综合评测及训练系统
CN208927548U (zh) * 2018-09-26 2019-06-04 南京优乒体育科技有限公司 一种具有计数功能的乒乓练习板
CN109453501A (zh) * 2018-10-25 2019-03-12 惠州市德赛工业研究院有限公司 球类训练数据处理方法、设备
KR102771268B1 (ko) * 2018-12-21 2025-02-24 삼성전자주식회사 전자 장치 및 그의 정보 제공 방법
CN110327595A (zh) * 2019-05-09 2019-10-15 深圳市蝙蝠云科技有限公司 基于可穿戴传感器的动作捕捉识别与评估装置及方法
CN112169296B (zh) * 2019-07-05 2021-10-22 荣耀终端有限公司 一种运动数据监测方法和装置
CN110558991B (zh) * 2019-07-30 2022-05-20 福建省万物智联科技有限公司 一种步态分析方法
CN111369629B (zh) * 2019-12-27 2024-05-24 浙江万里学院 一种基于挥拍击球动作双目视觉感知的回球轨迹预测方法
CN111318009A (zh) * 2020-01-19 2020-06-23 张衡 基于可穿戴惯性感知的体感健康娱乐系统及其工作方法
CN111437583B (zh) * 2020-04-10 2021-06-29 哈尔滨工业大学 一种基于Kinect的羽毛球基本动作辅助训练系统
CN111514561A (zh) * 2020-04-16 2020-08-11 杨飞 一种球类项目挥拍训练系统和方法
CN114004247B (zh) * 2020-07-14 2022-11-01 荣耀终端有限公司 骑行检测方法、电子设备及计算机可读存储介质
CN114259720B (zh) * 2020-09-15 2022-10-18 荣耀终端有限公司 动作识别方法及装置、终端设备、运动监测系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2953115A1 (en) * 2014-06-06 2015-12-09 Head Technology GmbH Swing analysis system
US10248985B2 (en) * 2015-01-16 2019-04-02 Brooks Sports, Inc. Systems and methods for analyzing lower body movement to recommend footwear
CN105898241A (zh) * 2016-01-15 2016-08-24 上海英诗帕信息科技有限公司 一种运动视觉呈现系统及方法
CN106581974A (zh) * 2017-02-13 2017-04-26 张琛 基于球拍运动轨迹的击球姿势训练装置及方法
CN107982899A (zh) * 2017-12-13 2018-05-04 郑州大学体育学院 一种羽毛球比赛数据记录管理系统
CN207898912U (zh) * 2018-02-13 2018-09-25 李宁（中国）体育用品有限公司 羽毛球训练系统
CN108671505A (zh) * 2018-03-20 2018-10-19 杭州电子科技大学 一种羽毛球运动辅助系统及其控制方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4119204A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115862130A (zh) * 2022-11-16 2023-03-28 之江实验室 基于人体姿态及其躯干运动场的行为识别方法
CN115862130B (zh) * 2022-11-16 2023-10-20 之江实验室 基于人体姿态及其躯干运动场的行为识别方法
CN116229576A (zh) * 2023-03-13 2023-06-06 北京数感科技有限公司 着地动作类型的确定方法、装置、终端及存储介质

Also Published As

Publication number Publication date
CN114259720B (zh) 2022-10-18
EP4119204A4 (en) 2023-11-01
US12374161B2 (en) 2025-07-29
EP4119204B1 (en) 2025-03-05
EP4119204A1 (en) 2023-01-18
US20230206697A1 (en) 2023-06-29
CN114259720A (zh) 2022-04-01

Similar Documents

Publication Publication Date Title
WO2022057795A1 (zh) 动作识别方法及装置、终端设备、运动监测系统
Rana et al. Wearable sensors for real-time kinematics analysis in sports: A review
EP2973215B1 (en) Feedback signals from image data of athletic performance
JP6080175B2 (ja) ボール運動の動作識別方法、装置及び動作支援装置
US9523704B2 (en) Flight time
CN105797319B (zh) 一种羽毛球运动数据处理方法及装置
WO2021051579A1 (zh) 形体姿态识别方法、系统、装置及存储介质
Fan et al. Sensor fusion basketball shooting posture recognition system based on CNN
Yang et al. TennisMaster: An IMU-based online serve performance evaluation system
KR20170101996A (ko) 다수의 디바이스들로부터의 데이터를 이용한 에너지 소비량 계산
CN109753868B (zh) 运动动作的评估方法及装置、智能手环
CN109364471A (zh) 一种vr系统
KR20170102952A (ko) 다수의 디바이스들로부터의 데이터를 이용한 에너지 소비량 계산
Taghavi et al. Tennis stroke detection using inertial data of a smartwatch
CN210078765U (zh) 基于可穿戴传感器的动作捕捉识别与评估装置
CN209221474U (zh) 一种vr系统
CN106310609B (zh) 一种羽毛球运动的分析方法及装置
WO2017143814A1 (zh) 球类运动数据统计方法、装置、系统、智能篮球及腕带
US20220343649A1 (en) Machine learning for basketball rule violations and other actions
CN107095401B (zh) 智能腕带
HK40069175B (zh) 动作识别方法及装置、终端设备、运动监测系统
HK40069175A (zh) 动作识别方法及装置、终端设备、运动监测系统
Kos et al. Performance Analysis
CN107096190B (zh) 一种智能篮球
Malawski et al. Automatic analysis of techniques and body motion patterns in sport

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21868618

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021868618

Country of ref document: EP

Effective date: 20221014

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 17921194

Country of ref document: US