WO2021212227A1 - Systems and methods for digital wellness - Google Patents

Systems and methods for digital wellness Download PDF

Info

Publication number
WO2021212227A1
WO2021212227A1 PCT/CA2021/050550 CA2021050550W WO2021212227A1 WO 2021212227 A1 WO2021212227 A1 WO 2021212227A1 CA 2021050550 W CA2021050550 W CA 2021050550W WO 2021212227 A1 WO2021212227 A1 WO 2021212227A1
Authority
WO
WIPO (PCT)
Prior art keywords
digital
user
wellness
screen
eye
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CA2021/050550
Other languages
French (fr)
Inventor
Michel Boivin
Jacob FISET
Martin Gamsby
Francis GUERIN
Simon ST-HILAIRE
Cosmin V. LUDUSAN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mirametrix Inc
Original Assignee
Mirametrix Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mirametrix Inc filed Critical Mirametrix Inc
Priority to EP21792987.6A priority Critical patent/EP4139937A4/en
Priority to CA3173716A priority patent/CA3173716A1/en
Priority to CN202180029886.0A priority patent/CN115956273A/en
Publication of WO2021212227A1 publication Critical patent/WO2021212227A1/en
Priority to US17/970,806 priority patent/US20230040562A1/en
Anticipated expiration legal-status Critical
Priority to US19/228,551 priority patent/US20250295313A1/en
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1079—Measuring physical dimensions, e.g. size of the entire body or parts thereof using optical or photographic means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1103—Detecting muscular movement of the eye, e.g. eyelid movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2503/00—Evaluating a particular growth phase or type of persons or animals
    • A61B2503/20—Workers
    • A61B2503/24—Computer workstation operators
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/742—Details of notification to user or communication with user or patient; User input means using visual displays
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
    • A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
    • A61M2021/0044—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the sight sense
    • A61M2021/005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the sight sense images, e.g. video
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00—General characteristics of the apparatus
    • A61M2205/33—Controlling, regulating or measuring
    • A61M2205/3303—Using a biosensor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00—General characteristics of the apparatus
    • A61M2205/33—Controlling, regulating or measuring
    • A61M2205/3306—Optical measuring means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M2205/00—General characteristics of the apparatus
    • A61M2205/50—General characteristics of the apparatus with microprocessors or computers
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
    • G07C1/10—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people together with the recording, indicating or registering of other data, e.g. of signs of identity

Definitions

  • the following relates to systems and methods for promoting digital wellness, particularly to systems and methods for monitoring digital health and safety through user detection and tracking.
  • Digital wellness can be considered to encompass aspects of life where digital devices and applications, through their continuous use, help improve human wellbeing.
  • Some notable examples of such devices and applications include activity trackers, heart rate monitors or smart watches.
  • Digital hygiene may be improved or maintained through use of a collection of devices and applications that discourage or prevent users from exceeding recommended usage time and/or frequency of certain devices, such as work and personal computers, tablets and smartphones.
  • CVS Computer Vision Syndrome
  • the methods comprise obtaining user biometric data; converting said biometric data into actionable instances of health and safety user device operation use cases; and interacting with the user based on said actionable instances in order to improve or remedy any deviations from recommended health and safety user device operation practices.
  • a system for monitoring digital health and safety of a user operating a computing device comprising:
  • At least one imaging module configured to acquire image data of a scene including the user
  • a user detection and tracking module configured to extract user anthropometric features from the image data
  • a biometric data extraction module configured to determine, based on the anthropometric features, biometric data including at least one of a three-dimensional head pose, eye position, eye openness and three-dimensional gaze data; and [d] a digital wellness monitor module configured to analyze the biometric data to determine at least one digital wellness metric for use in one or more digital wellness methods for providing the user with feedback on their operating of the computing device.
  • the imaging module includes a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, or a combination thereof.
  • a visible light sensor an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, or a combination thereof.
  • ToF Time-of-Flight
  • the imaging module is configured to operate in at least one of continuous frame mode, burst frame mode and single frame mode.
  • the biometric data extraction module is further configured to perform one or more of data noise filtering, data fusion and data dimensionality reduction of the biometric data.
  • the digital wellness monitor module is further configured to store the associated digital wellness metrics in one or more databases.
  • system further comprises a digital wellness assistant module for implementing the one or more digital wellness methods, the digital wellness assistant module interfacing with the digital wellness monitor module in order to provide the user with the feedback, the feedback being one or more of digital wellness alerts, reports, notification messages and prompts.
  • one of the digital wellness methods is a screen time management method comprising:
  • step b if a continuously monitored predefined screen time condition is satisfied, repeating the method from step a), otherwise, repeating the method from step b).
  • the predefined screen time condition is that a set screen time limit has been reached by the user within a set time frame.
  • the digital wellness monitor module is communicatively coupled to the local and/or cloud digital wellness logger to store thereon screen time statistics on a daily, weekly, monthly and yearly basis.
  • the digital wellness metrics include screen time since the last screen break, total daily screen time, and screen time over standard or user specified time periods.
  • the digital wellness monitor module is configured to enable an administrator to set the predefined screen time condition.
  • the digital wellness module is further configured to use the set screen time limit to enforce a screen time break by locking and/or turning off the screen for a specified amount of time.
  • the digital wellness methods is an eye care management method.
  • the eye care management method comprises an eye break check to determine whether the user needs to be prompted to take an eye break from the screen and an eye break validation process to confirm that the user has taken the recommended eye break.
  • the eye break is validated when the user is registered by the digital wellness monitor module as looking away from a screen, or screens, for a predefined period of time.
  • the eye break validation follows a rule where every 20 minutes, the user is prompted to look away from the screen at an object 20 feet away for 20 seconds.
  • the digital wellness module is further configured to enable an administrator to enforce the eye care break check and eye break validation.
  • the digital wellness monitor module is further configured to monitor blink rate to estimate digital eye strain and/or eye dryness of the user.
  • the digital wellness monitor module is further configured to prompt the user, via a digital assistant, to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products. [0031] In yet another implementation, the digital wellness monitor module is further configured to estimate from the blink rate fatigue or drowsiness of the user.
  • the digital wellness monitor module is configured to take immediate preventative action if the user is determined to have predetermined levels of fatigue or drowsiness and is performing safety critical tasks.
  • the digital wellness monitor module is further configured to implement a body posture monitoring method to determine posture quality of the user.
  • the head pose is a six degrees of freedom head pose.
  • the posture quality is based on whether the user is sitting at a minimum distance from the screen and their head is not at an extreme angle.
  • the digital wellness monitor module is further configured to notify the user, via the digital wellness assistant, to correct body posture, and if the posture quality is determined to be bad, the prompt is persistent until the user moves back beyond the bad posture warning distance.
  • the prompt can be dismissed or snoozed via the digital wellness assistant.
  • the digital wellness monitoring module is further configured to enable an administrator to enforce correction of body posture.
  • the one or more digital wellness methods is screen resolution assistance
  • the digital wellness monitoring module is further configured to enable passive or active adjustment of resolution of the screen based on blink rate patterns, eye movements, gaze patterns, head movements, distance of the user from the screen, or a combination thereof.
  • the digital wellness monitoring module is further configured to recommend to the user a change of screen resolution either as a prompt or by changing the resolution to a new resolution and asking the user if they would like to keep the new resolution.
  • the digital wellness monitor module is further configured to change the screen resolution dynamically, without user input, as the user is moving towards or away from the screen. [0042] In yet another implementation, the digital wellness monitoring module is further configured to suggest an improved screen resolution based on historical digital wellness data of the user, the historical digital wellness data including one or more of blink rate behaviour, squinting, and temporal analysis of motion of the user with respect to the display.
  • one of the at least one digital wellness methods is face touching prevention
  • the digital wellness monitor module is further configured to monitor face touching by using body tracking of the user, and to and to provide the user with face touching related information.
  • the digital wellness monitor module is configured to enable the user or an administrator to modify the prompts by one or more of setting frequency or time frame thresholds, or enabling snoozing or dismissal of the prompts.
  • the digital wellness monitor module is configured to body tracking information to estimate fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
  • the digital wellness monitor module is configured to detect onychophagia, and to prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
  • a method for monitoring digital health and safety of a user operating a computing device comprising:
  • biometric data extraction module determining based on the anthropometric features, biometric data including at least one of a three-dimensional head pose, eye position, eye openness and three-dimensional gaze data;
  • the imaging module acquires the image data using a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, or a combination thereof.
  • a visible light sensor an infrared light sensor
  • both a dedicated visible light sensor and a dedicated infrared light sensor a combined visible infrared light sensor
  • a visible light sensor and a Time-of-Flight (ToF) sensor both an infrared light sensor and a TOF sensor, or a combination thereof.
  • TOF Time-of-Flight
  • the imaging module is operated in at least one of continuous frame mode, burst frame mode and single frame mode.
  • the method further comprises performing one or more of data noise filtering, data fusion and data dimensionality reduction of the biometric data before step d).
  • the method further comprises storing the associated digital wellness metrics in one or more databases.
  • the method further comprises providing, via a digital wellness assistant module interfacing with the digital wellness monitor module, the user with the feedback, the feedback being one or more of digital wellness alerts, reports, notification messages and prompts.
  • one of the digital wellness methods is a screen time management method comprising:
  • step b if a continuously monitored predefined screen time condition is satisfied, repeating the method from step a), otherwise, repeating the method from step b).
  • the predefined screen time condition is that a set screen time limit has been reached by the user within a set time frame.
  • the method further comprises storing on one or more databases screen time statistics on a daily, weekly, monthly and yearly basis.
  • the digital wellness metrics include screen time since the last screen break, total daily screen time, and screen time over standard or user specified time periods.
  • the digital wellness monitor module is used to enable an administrator to set the predefined screen time condition.
  • the digital wellness monitor module uses the set screen time limit to enforce a screen time break by locking and/or turning off the screen for a specified amount of time.
  • one of the digital wellness methods is an eye care management method.
  • the eye care management method comprises an eye break check to determine whether the user needs to be prompted to take an eye break from the screen and an eye break validation process to confirm that the user has taken the recommended eye break.
  • the eye break is validated when the user is registered by the digital wellness monitor module as looking away from a screen, or screens, for a predefined period of time.
  • the eye break validation follows a rule where every 20 minutes, the user is prompted to look at an object 20 feet away for 20 seconds.
  • the digital wellness module is further configured to enable an administrator to enforce the eye care break check and eye break validation.
  • the digital wellness monitor module is used to monitor blink rate to estimate digital eye strain and/or eye dryness of the user.
  • the digital wellness monitor module is used to prompt the user, via a digital assistant, to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products.
  • the digital wellness monitor module is used to estimate from the blink rate fatigue or drowsiness of the user.
  • the digital wellness monitor module is configured to take immediate preventative action if the user is determined to have predetermined levels of fatigue or drowsiness and is performing safety critical tasks.
  • the digital wellness monitor module is further configured to implement a body posture monitoring method to determine posture quality of the user.
  • the head pose is a six degrees of freedom head pose.
  • the posture quality is based on whether the user is sitting at a minimum distance from the screen and their head is not at an extreme angle.
  • the digital wellness monitor module is further configured to notify the user, via a digital wellness assistant, to correct body posture, and if the posture quality is determined to be bad, the prompt is persistent until the user moves back beyond the bad posture warning distance.
  • the prompt can be dismissed or snoozed via the digital wellness assistant.
  • the digital wellness monitoring module is further configured to enable an administrator to enforce correction of body posture.
  • the one or more digital wellness methods is screen resolution assistance
  • the digital wellness monitoring module is further configured to enable passive or active adjustment of resolution of the screen based on blink rate patterns, eye movements, gaze patterns, head movements, distance of the user from the screen, or a combination thereof.
  • the digital wellness monitoring module is further configured to recommend to the user a change of screen resolution either as a prompt or by changing the resolution to a new resolution and asking the user if they would like to keep the new resolution.
  • the digital wellness monitor module is further configured to change the screen resolution dynamically, without user input, as the user is moving towards or away from the screen.
  • the digital wellness monitoring module is further configured to suggest an improved screen resolution based on historical digital wellness data of the user, the historical digital wellness data including one or more of blink rate behaviour, squinting, and temporal analysis of motion of the user with respect to the display.
  • one of the at least one digital wellness methods is face touching prevention
  • the digital wellness monitor module is further configured to monitor face touching by using body tracking of the user, and to and to provide the user with face touching related information.
  • the digital wellness monitor module is configured to enable the user or an administrator to modify the prompts by one or more of setting frequency or time frame thresholds or enabling snoozing or dismissal of the prompts.
  • the digital wellness monitor module is configured to body tracking information to estimate fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
  • the digital wellness monitor module is configured to detect onychophagia, and to prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
  • a computing device comprising one or more processors and at least one storage medium, the at least one storage medium containing non- transitory computer-readable instructions for execution by the one or more processors to cause the one or more processors to perform any of the methods.
  • FIG. 1 is a block diagram of a system for monitoring the digital wellness of a user.
  • FIGS. 2a and 2b are flow charts illustrating examples of computer executable instructions performed by the system when providing a screen time counter and/or screen time manager digital wellness function.
  • FIG. 3 is an example illustration of a communication interface through which the digital wellness assistant shown in FIG. 1 can present screen time counter analytics to the user.
  • FIG. 4 is a flow chart illustrating an example set of computer executable instructions performed by the system when providing an eye break digital wellness function.
  • FIG. 5 is a block diagram of an example of the interaction steps between the user and the digital wellness assistant when an eye break digital wellness functionality is provided.
  • FIG. 6 is a flow chart illustrating an example set of computer executable instructions performed by the system when providing a bad posture monitoring digital wellness function.
  • FIG. 7 is a block diagram of an example of interaction steps between the user and the digital wellness assistant when a bad posture monitoring digital wellness functionality is provided.
  • FIG. 8 is a flow chart illustrating an example of computer executable instructions performed by the system when providing an optimal display resolution digital wellness function.
  • Enabling a user to have a passive role in mitigation strategy implementation and enforcement may reduce the user’s cognitive load and assist them in focusing on their tasks, which may in turn increase effectiveness of mitigation strategies.
  • FIG. 1 illustrated is a system for digital wellness 100 comprising an imaging module 102 configured to provide an image feed of a scene containing at least one aspect of a computer user’s body, face, eye or gaze.
  • the system 100 further comprises a user detection and tracking unit 103 for receiving the image feed from the imaging module 102.
  • the user detection and tracking unit 103 may be configured to run a collection of machine executable algorithms for detecting and tracking one or more of the user’s body, face, eye and gaze. Following the detection and tracking of the user’s body, face, eye or gaze, the user detection and tracking unit 103 may output a set of user features.
  • the set of user features may include, but is not limited to, face contour landmarks, mouth and nose landmarks, eye contour landmarks, pupil and iris landmarks.
  • the user biometric data extraction unit 104 may be configured to process the set of user features in order to extract user-specific characteristics.
  • the set of user-specific characteristics may include, but is not limited to, six degrees of freedom (6DoF) head pose information, eye position, eye openness, three-dimensional gaze data, and user identity.
  • the unit 104 may be configured to send the extracted user biometric data to a biometric data post-processing unit 105.
  • the biometric data post-processing unit 105 may be configured to carry out data processing methods including, but not limited to, data noise filtering, data fusion, data dimensionality reduction.
  • the system 100 further comprises a digital wellness monitor 106 configured to receive post-preprocessed biometric data from the post-processing unit 105.
  • the digital wellness monitor 106 can use the post-processed biometric data to compute a series of digital wellness metrics and enable implementation of associated digital wellness use cases.
  • the wellness monitor 106 may also be responsible for storing the digital wellness metrics into a local and/or cloud digital wellness logger 107.
  • the digital wellness monitor 106 is also connected to the digital wellness assistant 108 which serves as a user communication interface 109 via which the digital wellness alerts, reports and any other type of messages are presented to the user 101.
  • the imaging module 102 may include a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible- infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, one or more other appropriate sensors, or any combination thereof.
  • a visible light sensor an infrared light sensor
  • both a dedicated visible light sensor and a dedicated infrared light sensor a combined visible- infrared light sensor
  • a visible light sensor and a Time-of-Flight (ToF) sensor both an infrared light sensor and a TOF sensor, one or more other appropriate sensors, or any combination thereof.
  • TOF Time-of-Flight
  • the imaging module 102 may be configured to operate in one ore more modes including, but not limited to, continuous frame mode, burst frame mode and single frame mode.
  • the digital wellness monitor 106 can measure a user’s screen time using a screen time counter loop 200 (FIG. 2).
  • the loop 200 may be referred to hereinafter as a “screen time counter use case”.
  • the screen time counter loop 200 can begin with an initialization step 201 which, in turn, activates a user biometric data logic 202.
  • the user biometric data logic 202 may provide a user state loop for starting and pausing the screen time counter depending on whether the user is looking at the screen.
  • the screen time counter may be enabled when the user looks at the screen (203,204).
  • the screen time counter may be paused (208) if the user is detected as looking elsewhere (209).
  • the screen time manager which may be part of the digital wellness monitor 106, may produce or initiate screen time counter reset commands at step 207 for local events and reinitialize the screen time counter (201), thereby restarting the loop 200.
  • Such local events may include, for example, the user 101 reaching a predefined screen time limit.
  • the screen time manager logic 206 part of the digital wellness monitor 106, is also connected to the global digital wellness logger 107 which can store screen time statistics on a daily, weekly, monthly and yearly basis.
  • the stored screen time statistics may be retrieved at the user’s 101 or system administrator’s request for display and analysis via the digital wellness assistant 108 and the user communication interface 109.
  • the digital wellness assistant 108 via the user communication interface 109, can display to the user screen time analytics, such as the screen time since the last screen break 301, the total daily screen time, in a numerical format 302 or a graphical format 303.
  • the screen time analytics can be also expanded to standardized or user specified time periods 304 or time of day standard or user specified periods 305.
  • screen time counter functionality may include, inter alia, more advanced screen time manager 206 options such as maximum screen time limits or screen time privileges. These functions can be set by an administrator on the local machine or via a remote connection and can be customized for different users, if the device is shared.
  • the screen time limits may be used by the screen time manager 206 to enable a mandatory screen time break by locking and or turning off the display for a specified amount of time.
  • the screen time manager functionality of the digital wellness monitor 106 can require only 6DoF head pose user data to be enabled.
  • the digital wellness monitor 106 can utilize the user’s gaze to enable the screen time manager functionality.
  • the digital wellness monitor 106 can make use of the user’s identity to allow for multiple screen time manager user profiles on shared devices.
  • the screen time manager in the digital wellness monitor 106 can be set up to work with multiple displays, where the user biometric data from the unit 105, such as 6DoF head pose or gaze, can be used to monitor all active displays and account for the cumulative screen time, shared between the active displays.
  • the digital wellness monitor 106 can be configured to include an eye care manager to implement an eye care management method, alone or in combination with the screen time manager 206 methods.
  • the eye care management method includes an eye break reminder and validation process.
  • the logical process flow can be the same as the screen time counter loop 200, but with the addition of an eye break check 401 performed based on the screen time logged at step 205.
  • the eye break check, or process 401 can determine whether the user 101 needs to be prompted to take an eye break from the screen.
  • the eye break check 401 follows the “20- 20-20” rule, as outlined in [1] - “every 20 minutes, look at something 20 feet away for 20 seconds”.
  • the 20-20-20 rule is a method recommended by various ophthalmology and optometry organizations as a way to prevent digital eye strain. If the 20 minutes of continuous screen time have passed 502, the eye break check 401 will notify the user 101 via the eye break prompt 402.
  • the eye break prompt 402 may be carried out by the digital wellness assistant 108 and can interact with the user 101 via the user communication interface 109.
  • the process 402 instructs the user 101 to look away from the computer display, or displays, for a predefined period of time 503.
  • the eye break is validated 403, to ensure that the recommended eye break has taken place.
  • the validation process 403 uses user biometric data, such as, for example, gaze and head pose to confirm that the user 101 is not looking at the display during the eye break activation.
  • the eye break counter only counts down from the predefined time when the user 101 is not looking at the display. Any interruption caused by the user looking back at the display 504 may pause the eye break countdown timer, which can resume when the user again looks away from the display 505.
  • the eye break validation 403 changes the display by, for example, blurring the display’s contents, dimming the display or turning the display off for the duration of the predefined eye break.
  • the eye break check can be dismissed by the user or snoozed for a set period of time.
  • the eye break intervals and duration can also be changed from the default “20-20-20” rule.
  • the eye break settings such as the check and the duration can be set by a system administrator as mandatory digital wellness features.
  • the user biometric data post-processing 105 can include anti-spoofing checks to ensure that the digital wellness functionality is not circumvented by the user.
  • the display or displays are shared by multiple users
  • the user biometric data extraction 104 includes user identification which allows the digital wellness monitor 106 to distinguish between multiple users and function accordingly.
  • Another embodiment of the digital wellness monitor 106 includes as part of the eye care functionality, alone or in combination with other use cases, a blink rate monitor to mitigate digital eye strain and dry eyes.
  • the blink rate can obtained through user eye feature extraction (with unit 104), post-processing (with unit 105) and subsequent analysis by the digital wellness monitor 106. Based on this analysis, the digital wellness monitor 106 can prompt the user 101 via the digital assistant 108 to mitigate the reduced blink rate problem.
  • the user can be instructed to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products in order to reduce digital eye strain.
  • the blink rate can also logged in the digital wellness logger 107 for user-specific analytics and daily, weekly, monthly and yearly report retrieval.
  • the blink rate functionality can also used to estimate user fatigue and drowsiness.
  • the digital wellness monitor can be configured to quickly or immediately take preventative action if the user is performing safety critical tasks.
  • Another embodiment of the digital wellness monitor 106 can implement, either as a standalone feature or together with other use cases, a body posture monitoring method.
  • An example embodiment of a body posture monitoring method 600 is shown in FIG. 6.
  • the method 600 is enabled by using the user’s 6DoF head pose as an indicator of good or bad posture.
  • the user biometric data, obtained at step 202, is used to determine the user’s posture at step 601 by analyzing head pose and distance from the display.
  • step 602 It is determined at step 602 whether the posture is good or bad based on whether the user is sitting at a minimum distance from the display and their head is not at an extreme angle. This indicates that the user may be hunching over the keyboard, experiencing neck pain and added spinal pressure. If bad posture is detected at step 602, the digital wellness monitor will prompt the user (step 605) and log user posture behavior for later reporting (step 604) and later display (step 606).
  • the digital wellness monitor can analyze the user’s distance from the display 701 prompt the user 703 if they get closer than a predefined distance from said display.
  • the bad posture prompt 704 is persistent until the user moves back beyond the bad posture warning distance 704.
  • the posture monitoring and the bad posture warning can be dismissed or snoozed via the digital wellness assistant 108.
  • the posture monitoring setting can also be restricted by an administrator, making the defined behavior and corrective bad posture action mandatory.
  • Yet another embodiment of the digital wellness monitor 106 includes, either as a standalone feature or together with other use cases, as part of the eye care functionality an optimal screen resolution assistant.
  • the digital wellness monitor can either actively or passively change the display resolution to an optimal one based on multiple user metrics.
  • the digital wellness monitor can determine the optimal screen resolution from the available ones, based on, inter alia, blink rate patterns 801, eye movements 802, gaze patterns 803, head movements 804, the user’s distance from the display 805, or any combination thereof.
  • the imaging system 102 is found in the same spatial plane as the display.
  • the digital wellness assistant 108 can then recommend the user a change of display resolution either as a prompt 812 or by actually changing the resolution 810 and asking the user if they would like to keep the new resolution.
  • the digital wellness assistant 108 changes the display resolution dynamically 810, without user input, as the user is moving towards or away from the display.
  • the digital wellness assistant may thus ensure that the user is provided most of the time, preferably always, with an appropriate or optimal available resolution where display readability is balanced with digital eye care.
  • An added advantage of this embodiment can be identified in the area of portable devices, where the dynamic resolution control can also help with battery consumption.
  • the optimal resolution can be suggested by the digital wellness assistant 108 based on the user’s historical digital wellness data 809, such as blink rate behaviour, squinting, and/or temporal analysis of the user’s motion with respect to the display.
  • the digital wellness monitor is configured to utilize body tracking alone or in combination with head positioning to enable body posture monitoring.
  • the digital wellness monitor 106 can use body tracking to monitor face touching and prompt the user with warning about potential health hazards associated with viral and bacterial transmission via face touching.
  • the functionality can be customized by the user, by setting frequency or time frame thresholds, snoozing or dismissing it altogether.
  • the digital wellness monitor 106 can use body tracking to monitor fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
  • the digital wellness monitor 106 can be configured to include onychophagia mitigation (i.e. nail biting) and prevention functionality.
  • the digital wellness assistant 108 can prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
  • any module or component exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and/or nonremovable) such as, for example, magnetic disks, optical disks, or tape.
  • Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.
  • Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD- ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the systems and/or devices described herein, any component of or related thereto, or accessible or connectable thereto. Any application or module herein described may be implemented using computer readable/executable instructions that may be stored or otherwise held by such computer readable media.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Physics & Mathematics (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Pathology (AREA)
  • Psychology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Business, Economics & Management (AREA)
  • Social Psychology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Hospice & Palliative Care (AREA)
  • Developmental Disabilities (AREA)
  • Dentistry (AREA)
  • Child & Adolescent Psychology (AREA)
  • Business, Economics & Management (AREA)
  • Radiology & Medical Imaging (AREA)
  • Psychiatry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Acoustics & Sound (AREA)
  • Anesthesiology (AREA)
  • Hematology (AREA)
  • Educational Technology (AREA)
  • Physical Education & Sports Medicine (AREA)

Abstract

Systems and associated methods are provided for monitoring a user while operating a computing device and providing active feedback to said user regarding health and safety best practices associated with operating said computing device. The methods comprise obtaining user biometric data; converting said biometric data into actionable instances of health and safety user device operation use cases; and interacting with the user based on said actionable instances in order to improve or remedy any deviations from recommended health and safety user device operation practices.

Description

SYSTEMS AND METHODS FOR DIGITAL WELLNESS
TECHNICAL FIELD
[0001] The following relates to systems and methods for promoting digital wellness, particularly to systems and methods for monitoring digital health and safety through user detection and tracking.
BACKGROUND
[0002] At a time when electronic devices, such as work and personal computers, mobile phones and smartwatches, are always on and always connected, people are spending increasing amounts of time using such devices. Smarter, more portable devices paved the way for the novel concept of digital wellness. Digital wellness can be defined from two, somewhat opposing, perspectives.
[0003] Digital wellness can be considered to encompass aspects of life where digital devices and applications, through their continuous use, help improve human wellbeing. Some notable examples of such devices and applications include activity trackers, heart rate monitors or smart watches.
[0004] However, there is another aspect of digital wellness that is less explored in the digital wellness industry, namely digital hygiene. Digital hygiene may be improved or maintained through use of a collection of devices and applications that discourage or prevent users from exceeding recommended usage time and/or frequency of certain devices, such as work and personal computers, tablets and smartphones.
[0005] According to a recent study from the American Academy of Ophthalmology [1], the average office worker spends 1,700 hours per year in front of a computer screen. This extended screen time has led to an increase in complaints of eye strain, dry eyes, headaches and insomnia. According to the same source, eye strain and dry eyes are mainly caused by a reduced blink rate as a consequence of focusing the eyes on computer screens or other digital displays. The blink rate reduction can be anywhere between a third to a half of the average blink rate.
[0006] The American Optometric Association lists as common symptoms associated with Digital Eye Strain, also known as Computer Vision Syndrome (CVS): eyestrain, headaches, blurred vision, dry eyes, neck and shoulder pain [2] Among the main causes behind CVS are improper viewing distances and poor sitting posture. [0007] Poor screen resolution, improper viewing distance and poor sitting posture are identified by the Canadian Centre for Occupational Health and Safety as the main causes behind eye discomfort [3]
[0008] While improper digital hygiene can negatively impact physical health, it has been recognized that there may also be social and psychological consequences. According to a 2018 OfficeTeam survey [4], 49% of Canadian workers spend their lunch breaks surfing the web or browsing social media. Yet another survey from 2019, by the Angus Reid Institute [5], indicates that 46% of Canadian parents are concerned that their child is spending too much time in front of a screen.
[0009] Many existing technologies that address the problem of digital hygiene are limited to schedule-based screen time applications, software-based blue light filters or simple countdown break reminders. These technologies do not and cannot account for user presence, identity or physical characteristics.
[0010] It is desirable to develop improved systems and methods for promoting digital wellness.
SUMMARY
[0011] Provided herein are systems and associated methods for monitoring a user operating a computing device and providing active feedback to said user regarding health and safety best practices associated with operating said computing device. The methods comprise obtaining user biometric data; converting said biometric data into actionable instances of health and safety user device operation use cases; and interacting with the user based on said actionable instances in order to improve or remedy any deviations from recommended health and safety user device operation practices.
[0012] In one aspect, provided herein is a system for monitoring digital health and safety of a user operating a computing device, the system comprising:
[a] at least one imaging module configured to acquire image data of a scene including the user;
[b] a user detection and tracking module configured to extract user anthropometric features from the image data;
[c] a biometric data extraction module configured to determine, based on the anthropometric features, biometric data including at least one of a three-dimensional head pose, eye position, eye openness and three-dimensional gaze data; and [d] a digital wellness monitor module configured to analyze the biometric data to determine at least one digital wellness metric for use in one or more digital wellness methods for providing the user with feedback on their operating of the computing device.
[0013] In an implementation, the imaging module includes a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, or a combination thereof.
[0014] In another implementation, the imaging module is configured to operate in at least one of continuous frame mode, burst frame mode and single frame mode.
[0015] In yet another implementation, the biometric data extraction module is further configured to perform one or more of data noise filtering, data fusion and data dimensionality reduction of the biometric data.
[0016] In yet another implementation, the digital wellness monitor module is further configured to store the associated digital wellness metrics in one or more databases.
[0017] In yet another implementation, the system further comprises a digital wellness assistant module for implementing the one or more digital wellness methods, the digital wellness assistant module interfacing with the digital wellness monitor module in order to provide the user with the feedback, the feedback being one or more of digital wellness alerts, reports, notification messages and prompts.
[0018] In yet another implementation, one of the digital wellness methods is a screen time management method comprising:
[a] an initialization step where a screen time counter is started when the user is first detected in the scene as the user looks at a screen;
[b] pausing the screen time counter when the user looks away from the screen;
[c] restarting the screen time counter when the user looks at the screen; and
[d] if a continuously monitored predefined screen time condition is satisfied, repeating the method from step a), otherwise, repeating the method from step b).
[0019] In yet another implementation, the predefined screen time condition is that a set screen time limit has been reached by the user within a set time frame. [0020] In yet another implementation, the digital wellness monitor module is communicatively coupled to the local and/or cloud digital wellness logger to store thereon screen time statistics on a daily, weekly, monthly and yearly basis.
[0021] In yet another implementation, the digital wellness metrics include screen time since the last screen break, total daily screen time, and screen time over standard or user specified time periods.
[0022] In yet another implementation, the digital wellness monitor module is configured to enable an administrator to set the predefined screen time condition.
[0023] In yet another implementation, the digital wellness module is further configured to use the set screen time limit to enforce a screen time break by locking and/or turning off the screen for a specified amount of time.
[0024] In yet another implementation, the digital wellness methods is an eye care management method.
[0025] In yet another implementation, the eye care management method comprises an eye break check to determine whether the user needs to be prompted to take an eye break from the screen and an eye break validation process to confirm that the user has taken the recommended eye break.
[0026] In yet another implementation, the eye break is validated when the user is registered by the digital wellness monitor module as looking away from a screen, or screens, for a predefined period of time.
[0027] In yet another implementation, the eye break validation follows a rule where every 20 minutes, the user is prompted to look away from the screen at an object 20 feet away for 20 seconds.
[0028] In yet another implementation, the digital wellness module is further configured to enable an administrator to enforce the eye care break check and eye break validation.
[0029] In yet another implementation, the digital wellness monitor module is further configured to monitor blink rate to estimate digital eye strain and/or eye dryness of the user.
[0030] In yet another implementation, the digital wellness monitor module is further configured to prompt the user, via a digital assistant, to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products. [0031] In yet another implementation, the digital wellness monitor module is further configured to estimate from the blink rate fatigue or drowsiness of the user.
[0032] In yet another implementation, the digital wellness monitor module is configured to take immediate preventative action if the user is determined to have predetermined levels of fatigue or drowsiness and is performing safety critical tasks.
[0033] In yet another implementation, the digital wellness monitor module is further configured to implement a body posture monitoring method to determine posture quality of the user.
[0034] In yet another implementation, the head pose is a six degrees of freedom head pose.
[0035] In yet another implementation, the posture quality is based on whether the user is sitting at a minimum distance from the screen and their head is not at an extreme angle.
[0036] In yet another implementation, the digital wellness monitor module is further configured to notify the user, via the digital wellness assistant, to correct body posture, and if the posture quality is determined to be bad, the prompt is persistent until the user moves back beyond the bad posture warning distance.
[0037] In yet another implementation, the prompt can be dismissed or snoozed via the digital wellness assistant.
[0038] In yet another implementation, the digital wellness monitoring module is further configured to enable an administrator to enforce correction of body posture.
[0039] In yet another implementation, the one or more digital wellness methods is screen resolution assistance, and the digital wellness monitoring module is further configured to enable passive or active adjustment of resolution of the screen based on blink rate patterns, eye movements, gaze patterns, head movements, distance of the user from the screen, or a combination thereof.
[0040] In yet another implementation, the digital wellness monitoring module is further configured to recommend to the user a change of screen resolution either as a prompt or by changing the resolution to a new resolution and asking the user if they would like to keep the new resolution.
[0041] In yet another implementation, the digital wellness monitor module is further configured to change the screen resolution dynamically, without user input, as the user is moving towards or away from the screen. [0042] In yet another implementation, the digital wellness monitoring module is further configured to suggest an improved screen resolution based on historical digital wellness data of the user, the historical digital wellness data including one or more of blink rate behaviour, squinting, and temporal analysis of motion of the user with respect to the display.
[0043] In yet another implementation, one of the at least one digital wellness methods is face touching prevention, and the digital wellness monitor module is further configured to monitor face touching by using body tracking of the user, and to and to provide the user with face touching related information.
[0044] In yet another implementation, the digital wellness monitor module is configured to enable the user or an administrator to modify the prompts by one or more of setting frequency or time frame thresholds, or enabling snoozing or dismissal of the prompts.
[0045] In yet another implementation, the digital wellness monitor module is configured to body tracking information to estimate fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
[0046] In yet another implementation, the digital wellness monitor module is configured to detect onychophagia, and to prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
[0047] In another aspect, provided is a method for monitoring digital health and safety of a user operating a computing device, the method comprising:
[a] acquiring, using an imaging module, image data of a scene including the user;
[b] extracting, using a user detection and tracking module, user anthropometric features from the image data;
[c] using a biometric data extraction module, determining based on the anthropometric features, biometric data including at least one of a three-dimensional head pose, eye position, eye openness and three-dimensional gaze data; and
[d] analyzing using a digital wellness monitoring module the biometric data to determine at least one digital wellness metric for use in one or more digital wellness methods for providing the user with feedback on their operating of the computing device.
[0048] In an implementation of the method, the imaging module acquires the image data using a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, or a combination thereof.
[0049] In another implementation of the method, the imaging module is operated in at least one of continuous frame mode, burst frame mode and single frame mode.
[0050] In yet another implementation, the method further comprises performing one or more of data noise filtering, data fusion and data dimensionality reduction of the biometric data before step d).
[0051] In yet another implementation, the method further comprises storing the associated digital wellness metrics in one or more databases.
[0052] In yet another implementation, the method further comprises providing, via a digital wellness assistant module interfacing with the digital wellness monitor module, the user with the feedback, the feedback being one or more of digital wellness alerts, reports, notification messages and prompts.
[0053] In yet another implementation, one of the digital wellness methods is a screen time management method comprising:
[a] an initialization step where a screen time counter is started when the user is first detected in the scene as the user looks at a screen;
[b] pausing the screen time counter when the user looks away from the screen;
[c] restarting the screen time counter when the user looks at the screen; and
[d] if a continuously monitored predefined screen time condition is satisfied, repeating the method from step a), otherwise, repeating the method from step b).
[0054] In yet another implementation, the predefined screen time condition is that a set screen time limit has been reached by the user within a set time frame.
[0055] In yet another implementation, the method further comprises storing on one or more databases screen time statistics on a daily, weekly, monthly and yearly basis.
[0056] In yet another implementation, the digital wellness metrics include screen time since the last screen break, total daily screen time, and screen time over standard or user specified time periods.
[0057] In yet another implementation, the digital wellness monitor module is used to enable an administrator to set the predefined screen time condition. [0058] In yet another implementation, the digital wellness monitor module uses the set screen time limit to enforce a screen time break by locking and/or turning off the screen for a specified amount of time.
[0059] In yet another implementation, one of the digital wellness methods is an eye care management method.
[0060] In yet another implementation, the eye care management method comprises an eye break check to determine whether the user needs to be prompted to take an eye break from the screen and an eye break validation process to confirm that the user has taken the recommended eye break.
[0061] In yet another implementation, the eye break is validated when the user is registered by the digital wellness monitor module as looking away from a screen, or screens, for a predefined period of time.
[0062] In yet another implementation, the eye break validation follows a rule where every 20 minutes, the user is prompted to look at an object 20 feet away for 20 seconds.
[0063] The method of claim 49, wherein the digital wellness module is further configured to enable an administrator to enforce the eye care break check and eye break validation.
[0064] In yet another implementation, the digital wellness monitor module is used to monitor blink rate to estimate digital eye strain and/or eye dryness of the user.
[0065] In yet another implementation, the digital wellness monitor module is used to prompt the user, via a digital assistant, to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products.
[0066] In yet another implementation, the digital wellness monitor module is used to estimate from the blink rate fatigue or drowsiness of the user.
[0067] In yet another implementation, the digital wellness monitor module is configured to take immediate preventative action if the user is determined to have predetermined levels of fatigue or drowsiness and is performing safety critical tasks.
[0068] In yet another implementation, the digital wellness monitor module is further configured to implement a body posture monitoring method to determine posture quality of the user.
[0069] In yet another implementation, the head pose is a six degrees of freedom head pose. [0070] In yet another implementation, the posture quality is based on whether the user is sitting at a minimum distance from the screen and their head is not at an extreme angle.
[0071] In yet another implementation, the digital wellness monitor module is further configured to notify the user, via a digital wellness assistant, to correct body posture, and if the posture quality is determined to be bad, the prompt is persistent until the user moves back beyond the bad posture warning distance.
[0072] In yet another implementation, the prompt can be dismissed or snoozed via the digital wellness assistant.
[0073] In yet another implementation, the digital wellness monitoring module is further configured to enable an administrator to enforce correction of body posture.
[0074] In yet another implementation, the one or more digital wellness methods is screen resolution assistance, and the digital wellness monitoring module is further configured to enable passive or active adjustment of resolution of the screen based on blink rate patterns, eye movements, gaze patterns, head movements, distance of the user from the screen, or a combination thereof.
[0075] In yet another implementation, the digital wellness monitoring module is further configured to recommend to the user a change of screen resolution either as a prompt or by changing the resolution to a new resolution and asking the user if they would like to keep the new resolution.
[0076] In yet another implementation, the digital wellness monitor module is further configured to change the screen resolution dynamically, without user input, as the user is moving towards or away from the screen.
[0077] In yet another implementation, the digital wellness monitoring module is further configured to suggest an improved screen resolution based on historical digital wellness data of the user, the historical digital wellness data including one or more of blink rate behaviour, squinting, and temporal analysis of motion of the user with respect to the display.
[0078] In yet another implementation, one of the at least one digital wellness methods is face touching prevention, and the digital wellness monitor module is further configured to monitor face touching by using body tracking of the user, and to and to provide the user with face touching related information. [0079] In yet another implementation, the digital wellness monitor module is configured to enable the user or an administrator to modify the prompts by one or more of setting frequency or time frame thresholds or enabling snoozing or dismissal of the prompts.
[0080] In yet another implementation, the digital wellness monitor module is configured to body tracking information to estimate fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
[0081] In yet another implementation, the digital wellness monitor module is configured to detect onychophagia, and to prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
[0082] In yet another aspect, provided is a computing device comprising one or more processors and at least one storage medium, the at least one storage medium containing non- transitory computer-readable instructions for execution by the one or more processors to cause the one or more processors to perform any of the methods.
BRIEF DESCRIPTION OF THE DRAWINGS
[0083] Embodiments will now be described by way of example only with reference to the appended drawings wherein:
[0084] FIG. 1 is a block diagram of a system for monitoring the digital wellness of a user.
[0085] FIGS. 2a and 2b are flow charts illustrating examples of computer executable instructions performed by the system when providing a screen time counter and/or screen time manager digital wellness function.
[0086] FIG. 3 is an example illustration of a communication interface through which the digital wellness assistant shown in FIG. 1 can present screen time counter analytics to the user.
[0087] FIG. 4 is a flow chart illustrating an example set of computer executable instructions performed by the system when providing an eye break digital wellness function.
[0088] FIG. 5 is a block diagram of an example of the interaction steps between the user and the digital wellness assistant when an eye break digital wellness functionality is provided.
[0089] FIG. 6 is a flow chart illustrating an example set of computer executable instructions performed by the system when providing a bad posture monitoring digital wellness function. [0090] FIG. 7 is a block diagram of an example of interaction steps between the user and the digital wellness assistant when a bad posture monitoring digital wellness functionality is provided.
[0091] FIG. 8 is a flow chart illustrating an example of computer executable instructions performed by the system when providing an optimal display resolution digital wellness function.
DETAILED DESCRIPTION
[0092] Provided herein are systems and methods for user-aware digital wellness use cases.
[0093] It will be appreciated that for simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the examples described herein. However, it will be understood by those of ordinary skill in the art that the examples described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the examples described herein. Also, the description is not to be considered as limiting the scope of the examples described herein.
[0094] It will also be appreciated that the examples and corresponding diagrams used herein are for illustrative purposes only. Different configurations and terminology can be used without departing from the principles expressed herein. For instance, components and modules can be added, deleted, modified, or arranged with differing connections without departing from these principles.
[0095] It has been recognized by numerous official health and safety guidelines that people who perform intensive computer work for prolonged periods of time may have an increased risk of developing a number of specific health problems. Such guidelines recommend that employees take regular short screen time breaks and maintain a correct sitting posture and screen viewing distance. While the problems, the associated negative effects and mitigation strategies are clearly laid out in health and safety guides, the implementation rests solely on the user.
[0096] It has also been recognized that the mitigation strategies may only be as effective as the user’s ability to remember to blink more frequently when viewing a computer screen for prolonged periods of time, to sit properly or to maintain the recommended distance from the screen. Keeping in mind these strategies while performing intensive computer work tends to be difficult for the computer user.
[0097] Enabling a user to have a passive role in mitigation strategy implementation and enforcement may reduce the user’s cognitive load and assist them in focusing on their tasks, which may in turn increase effectiveness of mitigation strategies.
[0098] Provided are systems and methods for implementing digital wellness strategies, or use cases, by means of detecting and tracking at least one aspect of the computer user’s body, face, eye or gaze.
[0099] Turning to FIG. 1 , illustrated is a system for digital wellness 100 comprising an imaging module 102 configured to provide an image feed of a scene containing at least one aspect of a computer user’s body, face, eye or gaze. The system 100 further comprises a user detection and tracking unit 103 for receiving the image feed from the imaging module 102. The user detection and tracking unit 103 may be configured to run a collection of machine executable algorithms for detecting and tracking one or more of the user’s body, face, eye and gaze. Following the detection and tracking of the user’s body, face, eye or gaze, the user detection and tracking unit 103 may output a set of user features. The set of user features may include, but is not limited to, face contour landmarks, mouth and nose landmarks, eye contour landmarks, pupil and iris landmarks. The user biometric data extraction unit 104 may be configured to process the set of user features in order to extract user-specific characteristics. The set of user-specific characteristics may include, but is not limited to, six degrees of freedom (6DoF) head pose information, eye position, eye openness, three-dimensional gaze data, and user identity. The unit 104 may be configured to send the extracted user biometric data to a biometric data post-processing unit 105. The biometric data post-processing unit 105 may be configured to carry out data processing methods including, but not limited to, data noise filtering, data fusion, data dimensionality reduction. The system 100 further comprises a digital wellness monitor 106 configured to receive post-preprocessed biometric data from the post-processing unit 105. The digital wellness monitor 106 can use the post-processed biometric data to compute a series of digital wellness metrics and enable implementation of associated digital wellness use cases. The wellness monitor 106 may also be responsible for storing the digital wellness metrics into a local and/or cloud digital wellness logger 107. The digital wellness monitor 106 is also connected to the digital wellness assistant 108 which serves as a user communication interface 109 via which the digital wellness alerts, reports and any other type of messages are presented to the user 101. [0100] The imaging module 102 may include a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible- infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, one or more other appropriate sensors, or any combination thereof.
[0101] The imaging module 102 may be configured to operate in one ore more modes including, but not limited to, continuous frame mode, burst frame mode and single frame mode.
[0102] In one aspect, the digital wellness monitor 106 can measure a user’s screen time using a screen time counter loop 200 (FIG. 2). The loop 200 may be referred to hereinafter as a “screen time counter use case”. The screen time counter loop 200 can begin with an initialization step 201 which, in turn, activates a user biometric data logic 202. The user biometric data logic 202 may provide a user state loop for starting and pausing the screen time counter depending on whether the user is looking at the screen. The screen time counter may be enabled when the user looks at the screen (203,204). The screen time counter may be paused (208) if the user is detected as looking elsewhere (209). Most or all starting and pausing events may be logged at step 205 to be used by a screen time manager block at step 206. At step 206, the screen time manager, which may be part of the digital wellness monitor 106, may produce or initiate screen time counter reset commands at step 207 for local events and reinitialize the screen time counter (201), thereby restarting the loop 200. Such local events may include, for example, the user 101 reaching a predefined screen time limit.
[0103] In an example embodiment of the screen time counter use case, the screen time manager logic 206, part of the digital wellness monitor 106, is also connected to the global digital wellness logger 107 which can store screen time statistics on a daily, weekly, monthly and yearly basis. The stored screen time statistics may be retrieved at the user’s 101 or system administrator’s request for display and analysis via the digital wellness assistant 108 and the user communication interface 109.
[0104] Turning to FIG. 3, the digital wellness assistant 108, via the user communication interface 109, can display to the user screen time analytics, such as the screen time since the last screen break 301, the total daily screen time, in a numerical format 302 or a graphical format 303. The screen time analytics can be also expanded to standardized or user specified time periods 304 or time of day standard or user specified periods 305.
[0105] Other screen time counter functionality may include, inter alia, more advanced screen time manager 206 options such as maximum screen time limits or screen time privileges. These functions can be set by an administrator on the local machine or via a remote connection and can be customized for different users, if the device is shared. The screen time limits may be used by the screen time manager 206 to enable a mandatory screen time break by locking and or turning off the display for a specified amount of time.
[0106] In one embodiment, the screen time manager functionality of the digital wellness monitor 106 can require only 6DoF head pose user data to be enabled. In another embodiment, the digital wellness monitor 106 can utilize the user’s gaze to enable the screen time manager functionality. A yet another embodiment, the digital wellness monitor 106 can make use of the user’s identity to allow for multiple screen time manager user profiles on shared devices.
[0107] In yet another embodiment, the screen time manager in the digital wellness monitor 106 can be set up to work with multiple displays, where the user biometric data from the unit 105, such as 6DoF head pose or gaze, can be used to monitor all active displays and account for the cumulative screen time, shared between the active displays.
[0108] The digital wellness monitor 106 can be configured to include an eye care manager to implement an eye care management method, alone or in combination with the screen time manager 206 methods. In one instance, the eye care management method includes an eye break reminder and validation process. As shown in FIG. 4 the logical process flow can be the same as the screen time counter loop 200, but with the addition of an eye break check 401 performed based on the screen time logged at step 205. The eye break check, or process 401 can determine whether the user 101 needs to be prompted to take an eye break from the screen.
[0109] In one instance, and with reference to FIG. 5, the eye break check 401 follows the “20- 20-20” rule, as outlined in [1] - “every 20 minutes, look at something 20 feet away for 20 seconds”. The 20-20-20 rule is a method recommended by various ophthalmology and optometry organizations as a way to prevent digital eye strain. If the 20 minutes of continuous screen time have passed 502, the eye break check 401 will notify the user 101 via the eye break prompt 402. The eye break prompt 402 may be carried out by the digital wellness assistant 108 and can interact with the user 101 via the user communication interface 109. The process 402 instructs the user 101 to look away from the computer display, or displays, for a predefined period of time 503. After the predefined period of time for which the user 101 is instructed to look into the distance, away from the display, the eye break is validated 403, to ensure that the recommended eye break has taken place. The validation process 403 uses user biometric data, such as, for example, gaze and head pose to confirm that the user 101 is not looking at the display during the eye break activation. The eye break counter only counts down from the predefined time when the user 101 is not looking at the display. Any interruption caused by the user looking back at the display 504 may pause the eye break countdown timer, which can resume when the user again looks away from the display 505.
[0110] In one embodiment, the eye break validation 403 changes the display by, for example, blurring the display’s contents, dimming the display or turning the display off for the duration of the predefined eye break.
[0111] In one embodiment, the eye break check can be dismissed by the user or snoozed for a set period of time. The eye break intervals and duration can also be changed from the default “20-20-20” rule.
[0112] In yet another embodiment, the eye break settings, such as the check and the duration can be set by a system administrator as mandatory digital wellness features. When this is done, the user biometric data post-processing 105 can include anti-spoofing checks to ensure that the digital wellness functionality is not circumvented by the user.
[0113] In another embodiment, the display or displays are shared by multiple users, the user biometric data extraction 104 includes user identification which allows the digital wellness monitor 106 to distinguish between multiple users and function accordingly.
[0114] Another embodiment of the digital wellness monitor 106 includes as part of the eye care functionality, alone or in combination with other use cases, a blink rate monitor to mitigate digital eye strain and dry eyes. The blink rate can obtained through user eye feature extraction (with unit 104), post-processing (with unit 105) and subsequent analysis by the digital wellness monitor 106. Based on this analysis, the digital wellness monitor 106 can prompt the user 101 via the digital assistant 108 to mitigate the reduced blink rate problem. By way of example, the user can be instructed to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products in order to reduce digital eye strain.
[0115] In another embodiment, the blink rate can also logged in the digital wellness logger 107 for user-specific analytics and daily, weekly, monthly and yearly report retrieval.
[0116] In yet another embodiment, the blink rate functionality can also used to estimate user fatigue and drowsiness. In such embodiment, the digital wellness monitor can be configured to quickly or immediately take preventative action if the user is performing safety critical tasks. [0117] Another embodiment of the digital wellness monitor 106 can implement, either as a standalone feature or together with other use cases, a body posture monitoring method. An example embodiment of a body posture monitoring method 600 is shown in FIG. 6. The method 600 is enabled by using the user’s 6DoF head pose as an indicator of good or bad posture. The user biometric data, obtained at step 202, is used to determine the user’s posture at step 601 by analyzing head pose and distance from the display. It is determined at step 602 whether the posture is good or bad based on whether the user is sitting at a minimum distance from the display and their head is not at an extreme angle. This indicates that the user may be hunching over the keyboard, experiencing neck pain and added spinal pressure. If bad posture is detected at step 602, the digital wellness monitor will prompt the user (step 605) and log user posture behavior for later reporting (step 604) and later display (step 606).
[0118] Turning to FIG. 7, in yet another embodiment of the user posture monitoring, the digital wellness monitor can analyze the user’s distance from the display 701 prompt the user 703 if they get closer than a predefined distance from said display. In one embodiment, the bad posture prompt 704 is persistent until the user moves back beyond the bad posture warning distance 704.
[0119] The posture monitoring and the bad posture warning can be dismissed or snoozed via the digital wellness assistant 108. The posture monitoring setting can also be restricted by an administrator, making the defined behavior and corrective bad posture action mandatory.
[0120] Yet another embodiment of the digital wellness monitor 106 includes, either as a standalone feature or together with other use cases, as part of the eye care functionality an optimal screen resolution assistant. With the use of the user biometric data, the digital wellness monitor can either actively or passively change the display resolution to an optimal one based on multiple user metrics. By way of example, the digital wellness monitor can determine the optimal screen resolution from the available ones, based on, inter alia, blink rate patterns 801, eye movements 802, gaze patterns 803, head movements 804, the user’s distance from the display 805, or any combination thereof. In this embodiment, the imaging system 102 is found in the same spatial plane as the display. The digital wellness assistant 108 can then recommend the user a change of display resolution either as a prompt 812 or by actually changing the resolution 810 and asking the user if they would like to keep the new resolution.
[0121] In another embodiment, the digital wellness assistant 108 changes the display resolution dynamically 810, without user input, as the user is moving towards or away from the display. The digital wellness assistant may thus ensure that the user is provided most of the time, preferably always, with an appropriate or optimal available resolution where display readability is balanced with digital eye care. An added advantage of this embodiment can be identified in the area of portable devices, where the dynamic resolution control can also help with battery consumption.
[0122] In yet another embodiment, the optimal resolution can be suggested by the digital wellness assistant 108 based on the user’s historical digital wellness data 809, such as blink rate behaviour, squinting, and/or temporal analysis of the user’s motion with respect to the display.
[0123] In yet another embodiment, the digital wellness monitor is configured to utilize body tracking alone or in combination with head positioning to enable body posture monitoring.
[0124] In yet another embodiment, the digital wellness monitor 106 can use body tracking to monitor face touching and prompt the user with warning about potential health hazards associated with viral and bacterial transmission via face touching. The functionality can be customized by the user, by setting frequency or time frame thresholds, snoozing or dismissing it altogether.
[0125] In yet another embodiment, the digital wellness monitor 106 can use body tracking to monitor fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
[0126] In yet another embodiment, the digital wellness monitor 106 can be configured to include onychophagia mitigation (i.e. nail biting) and prevention functionality. The digital wellness assistant 108 can prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
[0127] It will be appreciated that any module or component exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and/or nonremovable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD- ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the systems and/or devices described herein, any component of or related thereto, or accessible or connectable thereto. Any application or module herein described may be implemented using computer readable/executable instructions that may be stored or otherwise held by such computer readable media.
[0128] Although the above principles have been described with reference to certain specific examples, various modifications thereof will be apparent to those skilled in the art as outlined in the appended claims.
REFERENCES;
[1] American Academy of Ophthalmology, Protect Your Eyes From Too Much Screen Time, March 11 , 2019, https://www.aao.org/newsroom/news-releases/detail/protect-vour-eves- from-too-much-screen-time.
[2] American Optometric Association, Computer Vision Syndrome, 2020, https://www.aoa.orq/patients-and-public/carinq-for-vour-vision/protectinq-vour- vision/computer-vision-syndrome.
[3] Canadian Centre for Occupational Health and Safety, Eye Discomfort in the Office, April 8, 2020, https://www.ccohs.ca/oshanswers/erqonomics/office/eve discomfort.html.
[4] OfficeTeam Surveys, Canadian Workers Prioritize Screen Time Over Face Time at Lunch, September 2018, https://www.roberthalf.ca/en/canadian-workers-prioritize-screen- time-over-face-time-at-lunch
[0129] [5] Angus Reid Institute, Digital Dopamine: Half of Canadian parents concerned their child spends too much time on their devices, September 32019, http://anqusreid.org/screen- time-kids/.

Claims

Claims:
1 . A system for monitoring digital health and safety of a user operating a computing device, the system comprising: a. at least one imaging module configured to acquire image data of a scene including the user; b. a user detection and tracking module configured to extract user anthropometric features from the image data; c. a biometric data extraction module configured to determine, based on the anthropometric features, biometric data including at least one of a three- dimensional head pose, eye position, eye openness and three-dimensional gaze data; and d. a digital wellness monitor module configured to analyze the biometric data to determine at least one digital wellness metric for use in one or more digital wellness methods for providing the user with feedback on their operating of the computing device.
2. The system of claim 1 , wherein the imaging module includes a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, or a combination thereof.
3. The system of claim 1 , wherein the imaging module is configured to operate in at least one of continuous frame mode, burst frame mode and single frame mode.
4. The system of claim 1 , wherein the biometric data extraction module is further configured to perform one or more of data noise filtering, data fusion and data dimensionality reduction of the biometric data.
5. The system of claim 1 , wherein the digital wellness monitor module is further configured to store the associated digital wellness metrics in one or more databases.
6. The system of claim 1 , further comprising a digital wellness assistant module for implementing the one or more digital wellness methods, the digital wellness assistant module interfacing with the digital wellness monitor module in order to provide the user with the feedback, the feedback being one or more of digital wellness alerts, reports, notification messages and prompts.
7. The system of claim 1 , wherein one of the digital wellness methods is a screen time management method comprising: a. an initialization step where a screen time counter is started when the user is first detected in the scene as the user looks at a screen; b. pausing the screen time counter when the user looks away from the screen; c. restarting the screen time counter when the user looks at the screen; and d. if a continuously monitored predefined screen time condition is satisfied, repeating the method from step a), otherwise, repeating the method from step b).
8. The system of claim 7, wherein the predefined screen time condition is that a set screen time limit has been reached by the user within a set time frame.
9. The system of claim 7, wherein the digital wellness monitor module is communicatively coupled to the local and/or cloud digital wellness logger to store thereon screen time statistics on a daily, weekly, monthly and yearly basis.
10. The system of claim 7, wherein the digital wellness metrics include screen time since the last screen break, total daily screen time, and screen time over standard or user specified time periods.
11 . The system of claim 7, wherein the digital wellness monitor module is configured to enable an administrator to set the predefined screen time condition.
12. The system of claim 8, wherein the digital wellness module is further configured to use the set screen time limit to enforce a screen time break by locking and/or turning off the screen for a specified amount of time.
13. The system of claim 1 , wherein one of the digital wellness methods is an eye care management method.
14. The system of claim 13, wherein the eye care management method comprises an eye break check to determine whether the user needs to be prompted to take an eye break from the screen and an eye break validation process to confirm that the user has taken the recommended eye break.
15. The system of claim 14, wherein the eye break is validated when the user is registered by the digital wellness monitor module as looking away from a screen, or screens, for a predefined period of time.
16. The system of claim 15, wherein the eye break validation follows a rule where every 20 minutes, the user is prompted to look away from the screen at an object 20 feet away for 20 seconds.
17. The system of claim 14, wherein the digital wellness module is further configured to enable an administrator to enforce the eye care break check and eye break validation.
18. The system of claim 1 , wherein the digital wellness monitor module is further configured to monitor blink rate to estimate digital eye strain and/or eye dryness of the user.
19. The system of claim 18, wherein the digital wellness monitor module is further configured to prompt the user, via a digital assistant, to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products.
20. The system of claim 18, wherein the digital wellness monitor module is further configured to estimate from the blink rate fatigue or drowsiness of the user.
21. The system of claim 19, wherein the digital wellness monitor module is configured to take immediate preventative action if the user is determined to have predetermined levels of fatigue or drowsiness and is performing safety critical tasks.
22. The system of claim 1 , wherein the digital wellness monitor module is further configured to implement a body posture monitoring method to determine posture quality of the user.
23. The system of claim 22, wherein the head pose is a six degrees of freedom head pose.
24. The system of claim 23, wherein the posture quality is based on whether the user is sitting at a minimum distance from the screen and their head is not at an extreme angle.
25. The system of claim 24, wherein the digital wellness monitor module is further configured to notify the user, via the digital wellness assistant, to correct body posture, and if the posture quality is determined to be bad, the prompt is persistent until the user moves back beyond the bad posture warning distance.
26. The system of claim 24, wherein the prompt can be dismissed or snoozed via the digital wellness assistant.
27. The system of claim 24, wherein the digital wellness monitoring module is further configured to enable an administrator to enforce correction of body posture.
28. The system of claim 1 , wherein the one or more digital wellness methods is screen resolution assistance, and the digital wellness monitoring module is further configured to enable passive or active adjustment ofresolution of the screen based on blink rate patterns , eye movements, gaze patterns, head movements, distance of the user from the screen, or a combination thereof.
29. The system of claim 28, wherein the digital wellness monitoring module is further configured to recommend to the user a change of screen resolution either as a prompt or by changing the resolution to a new resolution and asking the user if they would like to keep the new resolution.
30. The system of claim 29, wherein the digital wellness monitor module is further configured to change the screen resolution dynamically, without user input, as the user is moving towards or away from the screen.
31. The system of claim 28, wherein the digital wellness monitoring module is further configured to suggest an improved screen resolution based on historical digital wellness data of the user, the historical digital wellness data including one or more of blink rate behaviour, squinting, and temporal analysis of motion of the user with respect to the display.
32. The system of claim 1 , wherein one of the at least one digital wellness methods is face touching prevention, and the digital wellness monitor module is further configured to monitor face touching by using body tracking of the user, and to and to provide the user with face touching related information.
33. The system of claim 32, wherein digital wellness monitor module is configured to enable the user or an administrator to modify the prompts by one or more of setting frequency or time frame thresholds, or enabling snoozing or dismissal of the prompts.
34. The system of claim 1 , wherein the digital wellness monitor module is configured to body tracking information to estimate fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
35. The system of claim 1 , wherein the digital wellness monitor module is configured to detect onychophagia, and to prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
36. A method for monitoring digital health and safety of a user operating a computing device, the method comprising: a. acquiring, using an imaging module, image data of a scene including the user; b. extracting, using a user detection and tracking module, user anthropometric features from the image data; c. using a biometric data extraction module, determining based on the anthropometric features, biometric data including at least one of a three- dimensional head pose, eye position, eye openness and three-dimensional gaze data; and d. analyzing using a digital wellness monitoring module the biometric data to determine at least one digital wellness metric for use in one or more digital wellness methods for providing the user with feedback on their operating of the computing device.
37. The method of claim 36, wherein the imaging module acquires the image data using a visible light sensor, an infrared light sensor, both a dedicated visible light sensor and a dedicated infrared light sensor, a combined visible infrared light sensor, both a visible light sensor and a Time-of-Flight (ToF) sensor, both an infrared light sensor and a TOF sensor, or a combination thereof.
38. The method of claim 36, the imaging module is operated in at least one of continuous frame mode, burst frame mode and single frame mode.
39. The method of claim 36, further comprising performing one or more of data noise filtering, data fusion and data dimensionality reduction of the biometric data before step d).
40. The method of claim 36, further comprising storing the associated digital wellness metrics in one or more databases.
41. The method of claim 36, further comprising providing, via a digital wellness assistant module interfacing with the digital wellness monitor module, the user with the feedback, the feedback being one or more of digital wellness alerts, reports, notification messages and prompts.
42. The method of claim 36, wherein one of the digital wellness methods is a screen time management method comprising: a. an initialization step where a screen time counter is started when the user is first detected in the scene as the user looks at a screen; b. pausing the screen time counter when the user looks away from the screen; c. restarting the screen time counter when the user looks at the screen; and d. if a continuously monitored predefined screen time condition is satisfied, repeating the method from step a), otherwise, repeating the method from step b).
43. The method of claim 42, wherein the predefined screen time condition is that a set screen time limit has been reached by the user within a set time frame.
44. The method of claim 42, further comprising storing on one or more databases screen time statistics on a daily, weekly, monthly and yearly basis.
45. The method of claim 42, wherein the digital wellness metrics include screen time since the last screen break, total daily screen time, and screen time over standard or user specified time periods.
46. The method of claim 42, wherein the digital wellness monitor module is configured to enable an administrator to set the predefined screen time condition.
47. The method of claim 43, wherein the digital wellness module is further configured to use the set screen time limit to enforce a screen time break by locking and/or turning off the screen for a specified amount of time.
48. The method of claim 42, wherein one of the digital wellness methods is an eye care management method.
49. The method of claim 48, wherein the eye care management method comprises an eye break check to determine whether the user needs to be prompted to take an eye break from the screen and an eye break validation process to confirm that the user has taken the recommended eye break.
50. The method of claim 49, wherein the eye break is validated when the user is registered by the digital wellness monitor module as looking away from a screen, or screens, for a predefined period of time.
51. The method of claim 50, wherein the eye break validation follows a rule where every 20 minutes, the user is prompted to look at an object 20 feet away for 20 seconds.
52. The method of claim 49, wherein the digital wellness module is further configured to enable an administrator to enforce the eye care break check and eye break validation.
53. The method of claim 42, wherein the digital wellness monitor module is further configured to monitor blink rate to estimate digital eye strain and/or eye dryness of the user.
54. The method of claim 53, wherein the digital wellness monitor module is further configured to prompt the user, via a digital assistant, to take a short screen break, to actively increase their blink rate, or to use eye drops or other dry eye relief products.
55. The method of claim 54, wherein the digital wellness monitor module is further configured to estimate from the blink rate fatigue or drowsiness of the user.
56. The method of claim 55, wherein the digital wellness monitor module is configured to take immediate preventative action if the user is determined to have predetermined levels of fatigue or drowsiness and is performing safety critical tasks.
57. The method of claim 42, wherein the digital wellness monitor module is further configured to implement a body posture monitoring method to determine posture quality of the user.
58. The method of claim 57, wherein the head pose is a six degrees of freedom head pose.
59. The method of claim 58, wherein the posture quality is based on whether the user is sitting at a minimum distance from the screen and their head is not at an extreme angle.
60. The method of claim 59, wherein the digital wellness monitor modules is further configured to notify the user, via a digital wellness assistant, to correct body posture, and if the posture quality is determined to be bad, the prompt is persistent until the user moves back beyond the bad posture warning distance.
61. The method of claim 59, wherein the prompt can be dismissed or snoozed via the digital wellness assistant.
62. The method of claim 59, wherein the digital wellness monitoring module is further configured to enable an administrator to enforce correction of body posture.
63. The method of claim 42, wherein the one or more digital wellness methods is screen resolution assistance, and the digital wellness monitoring module is further configured to enable passive or active adjustment ofresolution of the screen based on blink rate patterns , eye movements, gaze patterns, head movements, distance of the user from the screen, or a combination thereof.
64. The method of claim 63, wherein the digital wellness monitoring module is further configured to recommend to the user a change of screen resolution either as a prompt or by changing the resolution to a new resolution and asking the user if they would like to keep the new resolution.
65. The method of claim 64, wherein the digital wellness monitor module is further configured to change the screen resolution dynamically, without user input, as the user is moving towards or away from the screen.
66. The method of claim 63, wherein the digital wellness monitoring module is further configured to suggest an improved screen resolution based on historical digital wellness data of the user, the historical digital wellness data including one or more of blink rate behaviour, squinting, and temporal analysis of motion of the user with respect to the display.
67. The method of claim 42, wherein one of the at least one digital wellness methods is face touching prevention, and the digital wellness monitor module is further configured to monitor face touching by using body tracking of the user, and to and to provide the user with face touching related information.
68. The method of claim 67, wherein digital wellness monitor module is configured to enable the user or an administrator to modify the prompts by one or more of setting frequency or time frame thresholds, or enabling snoozing or dismissal of the prompts.
69. The method of claim 42, wherein the digital wellness monitor module is configured to body tracking information to estimate fatigue, drowsiness and eye strain by monitoring eye rubbing or any other hand and finger contact with the eyes.
70. The method of claim 42, wherein the digital wellness monitor module is configured to detect onychophagia, and to prompt the user with a warning or a series of escalating warnings which can ultimately direct the user to take a screen break.
71. A computing device comprising one or more processors and at least one storage medium, the at least one storage medium containing non-transitory computer- readable instructions for execution by the one or more processors to cause the one or more processors to perform the method of any one of claims 42-71.
PCT/CA2021/050550 2020-04-21 2021-04-21 Systems and methods for digital wellness Ceased WO2021212227A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
EP21792987.6A EP4139937A4 (en) 2020-04-21 2021-04-21 SYSTEMS AND PROCEDURES FOR DIGITAL WELL-BEING
CA3173716A CA3173716A1 (en) 2020-04-21 2021-04-21 Systems and methods for digital wellness
CN202180029886.0A CN115956273A (en) 2020-04-21 2021-04-21 System and method for digital healthcare
US17/970,806 US20230040562A1 (en) 2020-04-21 2022-10-21 Systems and Methods for Digital Wellness
US19/228,551 US20250295313A1 (en) 2020-04-21 2025-06-04 Systems and Methods for Digital Wellness

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063013513P 2020-04-21 2020-04-21
US63/013,513 2020-04-21

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/970,806 Continuation US20230040562A1 (en) 2020-04-21 2022-10-21 Systems and Methods for Digital Wellness

Publications (1)

Publication Number Publication Date
WO2021212227A1 true WO2021212227A1 (en) 2021-10-28

Family

ID=78270871

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2021/050550 Ceased WO2021212227A1 (en) 2020-04-21 2021-04-21 Systems and methods for digital wellness

Country Status (5)

Country Link
US (2) US20230040562A1 (en)
EP (1) EP4139937A4 (en)
CN (1) CN115956273A (en)
CA (1) CA3173716A1 (en)
WO (1) WO2021212227A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023224821A1 (en) * 2022-05-16 2023-11-23 Google Llc Head-mounted device for tracking screen time

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12213032B2 (en) * 2021-01-19 2025-01-28 Gluroo Imaginations, Inc. Messaging-based logging and alerting system
US20230140043A1 (en) * 2021-10-29 2023-05-04 Motorola Mobility Llc Electronic device that provides customized ergonomic prompts
US12524282B2 (en) * 2022-06-30 2026-01-13 Microsoft Technology Licensing, Llc Notification delay and auto-dismiss functionality
CN116755567A (en) * 2023-08-21 2023-09-15 北京中科心研科技有限公司 Device interaction methods, systems, electronic devices and media based on gesture data
CN117743926A (en) * 2023-12-19 2024-03-22 北京津发科技股份有限公司 Personnel on-duty state early warning method, device, equipment, system and medium
CN120050317B (en) * 2025-04-24 2025-08-01 北京爱宾果科技有限公司 Eye protection method and system of intelligent terminal

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWM370118U (en) * 2009-07-16 2009-12-01 Univ Southern Taiwan Adjustable vision health care reminding device
US20090315924A1 (en) * 2008-06-19 2009-12-24 Chi Mei Communication Systems, Inc. Method and system for adjusting screen resolution
US20100164731A1 (en) * 2008-12-29 2010-07-01 Aiguo Xie Method and apparatus for media viewer health care
US20160045108A1 (en) * 2014-08-14 2016-02-18 Roger Li-Chung Wu Vision Protection Method and System Thereof
US20190347180A1 (en) * 2018-05-08 2019-11-14 Apple Inc. User interfaces for controlling or presenting device usage on an electronic device

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8810413B2 (en) * 2010-10-15 2014-08-19 Hewlett Packard Development Company, L.P. User fatigue
JP5475893B2 (en) * 2011-05-20 2014-04-16 パナソニック株式会社 Apparatus and method for measuring visual fatigue level
US20130110004A1 (en) * 2011-10-28 2013-05-02 Brian Paul McLane User posture detection
JP5665241B2 (en) * 2012-03-21 2015-02-04 パナソニックＩｐマネジメント株式会社 Eye fatigue determination device and operation method thereof
US9854159B2 (en) * 2012-07-20 2017-12-26 Pixart Imaging Inc. Image system with eye protection
KR102025752B1 (en) * 2012-07-30 2019-11-05 삼성전자주식회사 Electronic Device Providing Content and Method Content Provision Method according to User’s Position
EP3128498B1 (en) * 2014-04-02 2018-10-31 Nissan Motor Co., Ltd Vehicular information presentation apparatus
CN107340849A (en) * 2016-04-29 2017-11-10 和鑫光电股份有限公司 Mobile device and eye protection control method thereof
US10652614B2 (en) * 2018-03-06 2020-05-12 Shoppar, Ltd. System and method for content delivery optimization based on a combined captured facial landmarks and external datasets

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090315924A1 (en) * 2008-06-19 2009-12-24 Chi Mei Communication Systems, Inc. Method and system for adjusting screen resolution
US20100164731A1 (en) * 2008-12-29 2010-07-01 Aiguo Xie Method and apparatus for media viewer health care
TWM370118U (en) * 2009-07-16 2009-12-01 Univ Southern Taiwan Adjustable vision health care reminding device
US20160045108A1 (en) * 2014-08-14 2016-02-18 Roger Li-Chung Wu Vision Protection Method and System Thereof
US20190347180A1 (en) * 2018-05-08 2019-11-14 Apple Inc. User interfaces for controlling or presenting device usage on an electronic device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MAYSEL BORIS: "How to stop touching your face? Hands and face detection with Python | by Boris Maysel | Analytics Vidhya | Medium", ANALYTICS VIDHYA, 24 March 2020 (2020-03-24), XP055864508, Retrieved from the Internet <URL:https://medium.com/analytics-vidhya/how-to-stop-touching-your-face-hands-and-face-detection-with-python-60ecb0d28d69> [retrieved on 20211123] *
See also references of EP4139937A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023224821A1 (en) * 2022-05-16 2023-11-23 Google Llc Head-mounted device for tracking screen time
US11994675B2 (en) 2022-05-16 2024-05-28 Google Llc Head-mounted device for tracking screen time

Also Published As

Publication number Publication date
US20230040562A1 (en) 2023-02-09
EP4139937A1 (en) 2023-03-01
US20250295313A1 (en) 2025-09-25
EP4139937A4 (en) 2024-02-14
CA3173716A1 (en) 2021-10-28
CN115956273A (en) 2023-04-11

Similar Documents

Publication Publication Date Title
US20250295313A1 (en) Systems and Methods for Digital Wellness
EP2573696B1 (en) Apparatus and method for assisting user to maintain correct posture
US20190098219A1 (en) Mobile device
US9406211B2 (en) Wearable posture regulation system and method to regulate posture
CN204631869U (en) Vision protection system
EP4314998A1 (en) Stress detection
CN104077122A (en) Computer system for automatically detecting fatigue and method for automatically detecting fatigue
US20160140986A1 (en) Monitoring treatment compliance using combined performance indicators
KR20190141684A (en) SYSTEM FOR ASSESSING A HEALTH CONDITION OF A USER
US8845099B2 (en) System and method for real time monitoring and dynamic treatment of oculomotor conditions
Kowalczyk et al. Blink and wink detection as a control tool in multimodal interaction
US20160135738A1 (en) Monitoring treatment compliance using passively captured task performance patterns
US20210236023A1 (en) TECHNOLOGY ADAPTED TO ENABLE IMPROVED COLLECTION OF INVOLUNTARY EYELlD MOVEMENT PARAMETERS, INCLUDING COLLECTION OF EYELlD MOVEMENT PARAMETERS TO SUPPORT ANALYSIS OF NEUROLOGICAL FACTORS
Lee et al. A new posture monitoring system for preventing physical illness of smartphone users
JP2016161835A (en) Display device, control program, and control method
US20170147871A1 (en) Fatigue-warning system
JP7207518B2 (en) DEMENTIA DETECTION DEVICE AND DEMENTIA DETECTION METHOD
CN103955273A (en) Mobile terminal and method for realizing user posture detection through operation system
KR101769506B1 (en) Control mehthod for smart glasses using brain wave
CN103927012A (en) Mobile terminal achieving user posture detection through operating system and method
KR101727155B1 (en) Smart glasses using brain wave
WO2016124415A1 (en) Safety system, workflow management system, safety method and computer program product
WO2025078876A1 (en) A spine and eye safety smart care system and method thereof
CN105303788B (en) Method and device for reminding user of protecting eyesight
Rasyid et al. Context-Aware Adaptive Feedback for Maintaining Healthy Blink Rates During Screen Use

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21792987

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 3173716

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021792987

Country of ref document: EP

Effective date: 20221121