WO2024242582A1 - Method for tracking and correcting a user's psychoemotional states - Google Patents

Abstract

The invention relates to a method for tracking a user's psychoemotional states. Said method involves taking readings of a user's brain using a portable neuroelectronic interface in an EEG data extraction mode. The EEG data collected is transmitted via a Bluetooth channel to a computing device for subsequent processing. For the purpose of determining and subsequently detecting psychoemotional and psychophysical states, an individual calibration mechanism is used which includes the following cycle: 20 seconds eyes closed, 20 seconds eyes open, 20 seconds eyes closed, 20 seconds eyes open, 40 seconds quiet wake. During calibration for determining the psychoemotional and psychophysical states, the following values are determined: the individual alpha peak frequency (IAPF) value; the alpha peak frequency suppression value; the individual alpha range boundaries; the given user's baseline cognitive fatigue score, concentration score and alpha gravity score. The metrics together with an explanation thereof are displayed to the user for monitoring. On the basis of the states detected, a message is displayed to the user that contains recommendations for training exercises with biological feedback.

Description

METHOD FOR TRACKING THE USER'S PSYCHOEMOTIONAL STATES AND THEIR CORRECTION

AREA OF TECHNOLOGY

This technical solution relates to the field of computer technology, in particular, to a method for tracking the psycho-emotional states of the user and their correction.

LEVEL OF TECHNOLOGY

A solution selected as the closest analogue, WO2022169376A1, is known from the prior art. This invention discloses a hardware and software complex for improving the user's cognitive functions, comprising: a computing device configured to display information and containing a client application; a neurocomputer interface containing a neuroamplifier; a software interface configured to control the operating modes of the neuroamplifier, namely configured to: connect to the neuroamplifier and activate commands for obtaining electrode resistances and/or for removing and recording data from an electroencephalogram of the brain (EEG); a server contained in the software interface and including: a communication module with the neurocomputer interface; a software interface core; an EEG signal classification module; a module for sending data to the client application.

The proposed technical solution is aimed at eliminating the shortcomings of the current level of technology and differs from known solutions in that the proposed method ensures high-quality and accurate tracking of the user's psycho-emotional states. This solution uses an algorithm for cleaning the EEG signal from myographic (muscle) artifacts, which allows using this system (method) outside laboratory conditions.

ESSENCE OF THE INVENTION

The technical problem that the claimed solution is aimed at solving is the creation of a method for tracking the psycho-emotional states of the user and correcting them. Additional embodiments of the present invention are presented in the dependent claims of the invention. The technical result consists of precise and high-quality tracking of the user’s psycho-emotional states, as well as the ability to correct them.

The claimed result is achieved by implementing a method for tracking the user's psycho-emotional states and correcting them, which includes the following stages: a portable neural interface, in the mode of taking data from the electroencephalogram of the brain (EEG), polls the user's brain; the collected EEG data of the user are transmitted via a Bluetooth communication channel to a computing device for further processing; the computing device receives EEG data from the neuroamplifier, after which it filters and preprocesses them; the filtered and preprocessed data are classified, while for calculating psycho-emotional and psychophysiological states, and their further detection and correction, an individual calibration mechanism is used, which includes: a cycle: 20 seconds of closed eyes, 20 seconds of open eyes, 20 seconds of closed eyes, 20 seconds of open eyes, 40 seconds of calm wakefulness; In this case, during the calibration process for calculating psycho-emotional and psychophysiological states, the following values are calculated: the value of the individual Alpha rhythm peak (iAPF); the value of Alpha rhythm peak suppression; individual Alpha range boundaries; calculation of the baseline level of the cognitive fatigue index, concentration index, alpha gravity index, characteristic of a given user (Baseline); after calibration and calculation of the individual baseline, the metric values with decoding are displayed to the user for monitoring; based on the detected states, the user is displayed notifications with recommendations for undergoing training with biofeedback.

In a particular embodiment of the described method, filtering and pre-processing of data is carried out using a fourth-order Butterworth filter, which is used for basic filtering of incoming data; by using the fast Fourier transform (FFT), the power spectral density (PSD) of the main rhythms of the brain is calculated - Delta, Theta, Alpha, Beta; by using power and amplitude filtering, myographic artifacts are eliminated and the useful physiological EEG signal is isolated.

In another particular embodiment of the described method, during the calibration process for calculating psycho-emotional and psychophysiological states, the following values are calculated: the value of the individual Alpha rhythm peak (iAPF), defined as the greatest power in the range of Alpha rhythm measurements at a certain frequency; the value of the Alpha rhythm peak suppression, defined as the difference in power between the absolute peak in the Alpha range during the calibration stages with open and closed eyes; the individual Alpha range boundaries are defined as the first value for the high-frequency alpha range, and the last value for the low-frequency alpha range, where the difference in power with open and closed eyes will be positive.

In a particular embodiment of the described method, the value of the frequency of the individual peak of the Alpha rhythm (iAPF) is determined as the frequency value in the range of Alpha rhythm measurements from 7 to 13 Hz, at which the power is greatest.

In a particular embodiment of the described method, when calculating Baseline, the values of the rhythm power and their ratios are accumulated in a cyclic buffer with a size of 5 seconds, until a total of 30 seconds of artifact-free values are accumulated, and the root mean square averaging method is used as the averaging method.

In another particular embodiment of the described method, all measurement results are saved in a local database and after the end of the session, together with the raw EEG data, are sent to a remote web platform to track statistics and progress.

In a particular embodiment of the described method, a portable neurointerface, in the mode of taking data from the electroencephalogram of the brain brain (EEG), conducts surveys of the user's brain at a frequency of 250 Hz, for each of the electrodes with a set of 8 readings.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of the embodiment of the invention, numerous implementation details are set forth in order to provide a clear understanding of the present invention. However, it will be apparent to one skilled in the art how the present invention may be used with or without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail in order not to unnecessarily obscure the features of the present invention.

In addition, it will be clear from the above description that the invention is not limited to the embodiment shown. Numerous possible modifications, changes, variations and substitutions, preserving the essence and form of the present invention, will be obvious to those skilled in the art.

This technical solution is intended for individual use and is created to analyze the user’s psycho-emotional state, as well as to correct (alleviate) negative conditions (with regular use).

The method for tracking the psycho-emotional states of the user and their correction is implemented by means of the following technical elements: a neurocomputer interface (can be implemented in various form factors) containing a neuroamplifier; a computing device that provides reception, filtering, analysis and processing of data coming from the neuroamplifier; a cloud web platform, as well as a local database (all measurement results after the end of the session, together with raw EEG data, are sent to a remote web platform to track statistics and progress).

The main operating principle of the device for recording the EEG signal (neurocomputer interface) is the principle of measuring the potential difference between the target electrode and the reference electrode, with a certain sampling frequency, each measurement cycle (data from all electrodes) is collected into a separate reading and after 8 readings have accumulated packed into a BLE packet for transmission to the target device for further filtering, processing and classification.

The accuracy of measurements is achieved thanks to a system of filtering external interference - rejection filters for cutting off external interference at frequencies of 50/60 Hz, a main filter based on a 4th-order Butterworth filter for cleaning the EEG signal from noise, as well as through the use of paradigms associated with the individual alpha peak frequency (iAPF) for selecting more accurate mechanics for tracking psycho-emotional states in different users.

In one implementation of the device for recording the EEG signal, the target electrodes are leads T3, T4, 01, 02, with a reference electrode on the forehead. The sampling frequency is 250 Hz. To start working, the device must be turned on with the button and the Bluetooth module must be paired, according to the user manual, with the target device (computing device), then put on the head, ensuring that the reference electrode touches the skin of the forehead, and the impedance values of the target electrodes in the application interface are less than 1 MOhm (megaohm). After this, it is possible to switch to the EEG signal measurement mode.

In another implementation of the device for recording the EEG signal, the target electrodes are leads C3, C4, A1, A2, with the reference electrode in the CZ location (on the crown). The sampling frequency is 250 Hz. To start working, the device must be turned on with the button and the Bluetooth module must be paired, according to the user manual, with the target device - separately for the neuroamplifier and separately for the stereo headset. Then put it on the head, making sure that all the electrodes fit and the impedance values of the target electrodes are less than 1 MOhm. After this, it is possible to switch to the EEG signal measurement mode. In the process of working with the device, it is possible to use the stereo headset for its intended purpose - for listening to music, making audio calls, in parallel with measuring the EEG signal.

The neurointerfaces support the following modes of operation: resistance measurement - to determine the quality of electrode adhesion, sending the values of the built-in battery charge, and EEG signal acquisition. The devices do not provide any hardware filtering - processing is performed on the side of the computing device (Mind Tracker VO). A computing device is a hardware and software device that enables the user to obtain information/data and/or perform the required operating algorithm and/or interact with local or global communication networks (systems) and information systems (for example, a mobile device, laptop, etc.).

A computing device may consist of: information input/output devices (audio, graphic, text, video, etc.); sensors (position, orientation, illumination, accelerometers, proximity, etc.); communication modules (peripheral devices); the core of a personal device (computing equipment (microprocessor, read-only memory, random access memory), graphic accelerators, audio stream processing equipment).

The computing device performs the necessary data processing using three main modules:

- The core responsible for device management, signal reception, conversion, filtering and sending to the interface module, the local database module and sending to the remote web platform server;

- Interface, a module responsible for displaying and entering user information on a computing device.

- Local database, a module that stores session data and sends it to a remote web platform server for storage and analysis.

Modules communicate with each other via the C API.

After the devices are launched, a search for a neurointerface ready for connection via Bluetooth is performed upon request. If such a neurointerface is found, an indication of the quality of the electrode placement is displayed. If the quality is satisfactory and the resistance values for each electrode are less than 1000 kOhm, it becomes possible to switch the device to the EEG recording mode.

In the EEG recording mode, the neurointerface conducts surveys at a frequency of 250 Hz for each of the electrodes; when 8 readings are collected, they are packed into a BLE packet and sent via the Bluetooth communication channel to the computing device.

To eliminate floating delays when transmitting data via the BLE protocol, the kernel module implements buffering functionality, with data output for processing every 0.1 sec. After receiving EEG data from the buffer, filtering and transformation algorithms are applied to them:

- fourth order Butterworth filter;

- fast Fourier transform (FFT) to obtain the frequency and amplitude values of the signal;

- calculation of the power spectral density (PSD) of the main rhythms of the brain - Theta, Alpha, Beta, including sub-ranges.

All further measurements are carried out after additional power and amplitude filtration of myographic artifacts to isolate the useful physiological EEG signal. Filtration is carried out by a sliding window, 0.8 sec wide. Only artifact-free values are taken into calculations.

Classification of psycho-emotional states is possible after passing an individual calibration, which takes 2 minutes. During the calibration process, the user is asked to sit quietly and go through a cycle: 20 seconds of closed eyes, 20 seconds of open eyes, 20 seconds of closed eyes, 20 seconds of open eyes, 40 seconds of calm wakefulness.

During the calibration process, the following parameters are measured: Artifact-free values of the Alpha, Beta, Theta rhythm powers, the value of the individual Alpha rhythm peak (iAPF), individual Alpha range boundaries, Alpha rhythm subrange powers. Measurements are made with a sliding window of 5 seconds, with subsequent averaging - separately for the calibration stage with closed eyes, separately for the stage with open eyes.

In the last 40 seconds, the Alpha, Beta, Theta rhythms of the brain are measured, including the A1 and A2 subranges, and the baseline level characteristic of the given user is calculated (Baseline).

Baseline calculation: when calculating Baseline, the values of rhythm power and their ratios are accumulated in a cyclic buffer with a size of 5 seconds, until a total of 30 seconds of artifact-free values are accumulated, the mean square averaging method is used as the averaging method. The 75th percentile of the resulting numerical series is taken as the Baseline value.

After calculating Baseline, the following values are displayed in the user interface: the iAPF value and a description of the values.

This technical solution allows the user to monitor their condition in real time in monitoring mode. The following metrics are available to the user on the display device: - Cognitive Score (cognitive load level in percent) - values in real time or as a graph for a period of up to 60 minutes.

- Indication of psycho-emotional states: at the moment, the states are displayed as indicators: Relaxation, Involvement, Mild fatigue, Moderate fatigue, Chronic fatigue, Anxiety, Stress. The states become active as soon as the application algorithm detects them.

Based on the detected states, the application displays notifications with recommendations for undergoing biofeedback training (different for different negative states).

Examples of biofeedback trainings:

- Stress relief - training is offered to the user when conditions such as fatigue and stress are detected.

The training helps users relieve stress and experience deep relaxation, suitable for users who find it difficult to relax.

- Relaxation of the mind - training is offered to the user when such conditions as fatigue and anxiety are detected. The training allows the user to gradually learn to concentrate and not be distracted by external stimuli, suitable for users who easily relax, but relaxation is easily knocked down.

The trainings have a similar structure: they consist of three levels, with each level becoming more difficult to complete, and each level requires a total of 100 points. At the end of each training, the user is shown the result in the interface - the maximum number of points scored by the user during the training.

Algorithm for interaction of modules of the system for detecting psycho-emotional states and their correction (Mind Tracker BCI):

- The user interface acts as a client, allows for communication and control of the neuroamplifier's operating modes. Displays information to the user.

- The core receives EEG data directly from the neuroamplifier, filters and preprocesses them, for basic filtering of incoming data a fourth-order Butterworth filter and a fast Fourier transform (FFT) are used to obtain values of the spectral power density, then the filtered and preprocessed data are sent to the classification modules to calculate psycho-emotional and psychophysiological states. Finished metrics are transferred to the user for display. Sends EEG data to a local DB for storage. The core is a set of components that ensure the functioning and communication between interface modules, such as:

1. EEG signal classification module.

The module contains a receiving buffer for data coming via the Bluetooth channel to ensure a constant delay of 4 ms (for a sampling frequency of 250 Hz) when outputting data to the state classification modules. The module also contains filters for clearing the signal from interference and a module for calculating Band Powers (power spectral density values). After signal cleaning and calculating the powers, the data is sent to the state classification modules for further processing and output to the display device.

2. Communication module with neurocomputer interface.

This module contains the logic for controlling the connection to the neuroamplifier via the Bluetooth communication channel. The module is also responsible for sending commands to switch the operating modes of the neuroamplifier (impedance measurement mode, data acquisition mode)

3. Module for sending data to the user.

This module contains the logic of the internal C API for communication between the classification module, the neurocomputer interface communication module, and the client application. The module is a set of API methods that allow you to issue and receive data, receive commands from the application interface in a form understandable to the user.

4. Local database, a module that stores session data and sends it to a remote web platform server for further storage and analysis.

This module contains a local SQL Lite database for recording EEG data and metrics received from classification modules when there is no connection to the remote web platform server. The stored data includes: RAW EEG Data in HDF5 format, time-based event markup data, time-based classified state data, session metadata, including the neuroamplifier model, number of channels, absolute start and end times of the session. The module also contains the functionality for resending data when the Internet connection is lost. Taking into account the “floating” delay in receiving data via the wireless protocol, the interface uses a buffer for data collection, thereby allowing for a uniform delay in transmitting EEG data for classification and further transmission to the application interface.

An additional advantage of the system for detecting psycho-emotional states and correcting them (Mind Tracker BCI) is its flexible architecture: it can be integrated with various interfaces and various signal classifiers.

Filtering and preprocessing of data in the core (includes):

Application of the fourth order Butterworth filter, which is used for basic filtering of incoming data;

Application of the Fast Fourier Transform (FFT), which is used to calculate the power spectral density (PSD) of the main brain rhythms - Delta, Theta, Alpha, Beta;

Filtering is performed by a sliding window, 0.8 sec wide. Only artifact-free values are taken into further calculations.

During the calibration process, the following values are calculated to calculate psycho-emotional and psychophysiological states:

• the value of the individual peak of the Alpha rhythm (iAPF) is defined as the greatest power in the range of Alpha rhythm measurements at a certain frequency (in a particular embodiment of the described method, the value of the frequency of the individual peak of the Alpha rhythm (iAPF) is defined as the frequency value in the range of Alpha rhythm measurements from 7 to 13 Hz, at which the power is greatest);

• the magnitude of the suppression of the Alpha rhythm peak is defined as the difference in power between the absolute peak in the Alpha range during the calibration stages with open and closed eyes;

• individual boundaries of the Alpha range are defined as the first value (for the high-frequency alpha range) and the last value (for the low-frequency alpha range), where the difference in power with open and closed eyes will be positive;

• calculation of the baseline level of the cognitive fatigue index, concentration index, and alpha-gravity index, characteristic of a given user (Baseline). The power spectral density (PSD) of the ranges of the main rhythms of the brain, as well as their subranges, are measured using a sliding window of 5 seconds.

To calculate the main indices, the following formulas are used:

Index of cognitive fatigue (FS) = (a + 0) / p, where a is the power in the Alpha rhythm range, 0 is the power in the Theta rhythm range, and the power in the Beta rhythm range.

Alpha Gravity Index (GS) = A2/A1, where A2 is the power in the Alpha rhythm sub-range from the iAPF value to the upper limit, and A1 is the power in the Alpha rhythm sub-range from the iAPF value to the lower limit.

Concentration index = B/A, where B is the power in the Beta rhythm subrange, and A is the power in the Alpha rhythm subrange

After calibration and calculation of the individual baseline level, the metric values with decoding are displayed to the user for monitoring.

The following values are displayed: iAPF, Cognitive Score graph (scale from 0 to 100%), indication of psycho-emotional states. iAPF - values from 7 to 13 Hz

Cognitive Score - a graph of involvement in the presented task (productivity graph): for users with low (iAPF<10) it is calculated based on the fatigue index, with correction factors; for users with high alpha frequency (iAPF>10) it is calculated based on the Alpha Gravity Index (GS), with correction factors.

Indication of psycho-emotional states is implemented in the form of indicators: Relaxation, Involvement, Mild fatigue, Moderate fatigue, Chronic fatigue, Anxiety, Stress. State indicators become active as soon as the system algorithm determines them.

The algorithm recognizes each state individually, based on the values of the indices described above.

• States of relaxation and involvement are identified with high iAPF values and low FS values, relative to baseline values. • Mild, moderate and chronic fatigue states are identified by lower iAPF values and increased FS values relative to baseline values.

• States of anxiety and stress are identified by low iAPF values and high stress index values, in relation to baseline indicators.

The mechanics are based on auditory feedback, which depends on maintaining the level of its Alpha peak, determined at the calibration stage (the amount of Alpha rhythm suppression). When maintaining relaxation, the sounds of nature are clearly expressed, but when falling, extraneous sounds of artificial origin are mixed in (for example, the sounds of a highway, a construction site, etc.)

All measurement results (sessions) are sent and saved to a local Postgress SQL database and after the end of the session, together with the raw EEG data, are sent to a remote web platform, which allows you to track statistics and progress when reusing the system.

The proposed technical solution allows tracking metrics and states “at the moment”, as well as session-wise - keeping statistics by days, weeks, months.

When negative psycho-emotional states are detected in the user, recommendations are displayed on the display device to help get rid of such states (recommendations for planning the “work/rest” mode, or recommendations for completing training).

Additionally, the solution allows you to build an individual predictive model and notifies the user about the most favorable periods of work/rest, based on the accumulated data.

This solution uses an algorithm for cleaning the EEG signal from myographic (muscle) artifacts, which allows the proposed technical solution to be used outside of laboratory conditions.

Although the invention has been described with reference to the disclosed embodiments, it will be apparent to those skilled in the art that the specific experiments described in detail are provided merely for the purpose of illustrating the present invention and should not be construed as limiting the scope of the invention in any way. It will be understood that that various modifications can be made without departing from the essence of the present invention.

Claims

Formula 1. A method for tracking the psycho-emotional states of a user and their correction, comprising the following stages: a portable neural interface, in the mode of recording data from the electroencephalogram of the brain (EEG), polls the user's brain; the collected EEG data of the user are transmitted via a Bluetooth communication channel to a computing device for further processing; the computing device receives EEG data from the neural amplifier, after which it filters and preprocesses them; the filtered and preprocessed data are classified, while in order to calculate the psycho-emotional and psychophysiological states, and their subsequent detection and correction, an individual calibration mechanism is used, which includes: a cycle: 20 seconds of closed eyes, 20 seconds of open eyes, 20 seconds of closed eyes, 20 seconds of open eyes, 40 seconds of calm wakefulness; wherein during the calibration process for calculating the psycho-emotional and psychophysiological states, the following values are calculated: the value of the individual peak of the Alpha rhythm (iAPF); Alpha rhythm peak suppression value; individual Alpha range boundaries; calculation of the baseline level of the cognitive fatigue index, concentration index, alpha gravity index, characteristic of a given user (Baseline); after calibration and calculation of the individual baseline level, the metric values with decoding are displayed to the user for monitoring; based on the detected states, the user is shown notifications with recommendations for undergoing training with biofeedback. 2. The method according to claim 1, wherein the filtering and preprocessing of data is carried out by means of a fourth-order Butterworth filter, which is used for basic filtering of incoming data; by means of the fast Fourier transform (FFT), the power spectral density (PSD) of the main rhythms of the brain - Delta, Theta, Alpha, Beta - is calculated; By using power and amplitude filtering, myographic artifacts are eliminated and the useful physiological EEG signal is isolated. 3. The method according to item 1, wherein during the calibration process for calculating psycho-emotional and psychophysiological states the following values are calculated: • the value of the individual peak of the Alpha rhythm (iAPF), defined as the highest power in the range of Alpha rhythm measurements at a certain frequency; • the magnitude of the suppression of the Alpha rhythm peak is defined as the difference in power between the absolute peak in the Alpha range during the calibration stages with open and closed eyes; • individual boundaries of the Alpha range are defined as the first value for the high-frequency alpha range, and the last value for the low-frequency alpha range, where the difference in power with open and closed eyes will be positive. 4. The method according to paragraph 3, in which the value of the individual Alpha rhythm peak frequency (iAPF) is determined as the frequency value in the Alpha rhythm measurement range from 7 to 13 Hz, at which the power is greatest. 5. The method according to item 1, in which, when calculating the Baseline, the values of the rhythm power and their ratios are accumulated in a cyclic buffer with a size of 5 seconds, until a total of 30 seconds of artifact-free values are accumulated, and the mean square averaging method is used as the averaging method. 6. The method according to item 1, in which all measurement results are saved in a local database and after the end of the session, together with the raw EEG data, are sent to a remote web platform for tracking statistics and progress. 7. The method according to claim 1, in which the portable neural interface, in the mode of taking data from the electroencephalogram of the brain (EEG), conducts surveys the user's brain at a frequency of 250 Hz, for each of the electrodes with a set of 8 readings.