WO2024257097A2 - Représentations de locuteur pour la classification actif/inactif de la maladie de parkinson - Google Patents

Représentations de locuteur pour la classification actif/inactif de la maladie de parkinson Download PDF

Info

Publication number
WO2024257097A2
WO2024257097A2 PCT/IL2024/050578 IL2024050578W WO2024257097A2 WO 2024257097 A2 WO2024257097 A2 WO 2024257097A2 IL 2024050578 W IL2024050578 W IL 2024050578W WO 2024257097 A2 WO2024257097 A2 WO 2024257097A2
Authority
WO
WIPO (PCT)
Prior art keywords
user
state
parkinson
disease
embeddings
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IL2024/050578
Other languages
English (en)
Other versions
WO2024257097A3 (fr
Inventor
Matthew Gibson
Filip JURCICEK
Ievgen KARAULOV
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TECHNOLOGIES OF VOICE INTERFACE Ltd
Original Assignee
TECHNOLOGIES OF VOICE INTERFACE Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by TECHNOLOGIES OF VOICE INTERFACE Ltd filed Critical TECHNOLOGIES OF VOICE INTERFACE Ltd
Publication of WO2024257097A2 publication Critical patent/WO2024257097A2/fr
Publication of WO2024257097A3 publication Critical patent/WO2024257097A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/66—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for extracting parameters related to health condition
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the present disclosure relates to speaker embeddings classification in general, and to systems that automatically classifies the state of a person with Parkinson’s disease (PwP) as ON or OFF state, given a sample of a speech, in particular.
  • PwP Parkinson’s disease
  • a person with Parkinson’s disease may experience alterations to their speech as a symptom of the current condition. Such alterations occur in approximately 75-90% of the PwP population. These alterations are a particular sub-category of dysarthric speech characterized by reduced loudness and pitch variability, breathy or hoarse voice, imprecise articulation, and more general features such as abnormalities of speech rate and pause ratio.
  • Eevodopa is a medication used to treat the symptoms of Parkinson’s disease.
  • the displayed symptoms - including speech related symptoms - of a PwP can be successfully suppressed. This state is described as the ON state.
  • the symptoms return; this is known as the OFF state.
  • the present subject matter is directed towards a method for speech-based classification of the state of a PwP.
  • a method for classifying speaker embeddings for a user having Parkinson’s disease comprising: receiving, by a computerized system, enrolment data wherein the data comprises speech samples of the user in an ON state and in an OFF state; storing the speech samples of the user; constructing from the enrolment data a profile of the user in an auxiliary model; monitoring a state of the user; receiving, by a computerized system, a test signal of the user; extracting embeddings from a test signal; computing distances between the test signal embeddings and the ON state and the OFF auxiliary models; computing the difference between the two distances; classifying the recognition signal as an ON state or an OFF state.
  • the ON state is defined to be when the user is sufficiently medicated and wherein the OFF state is defined to be when the user is not medicated.
  • the speech samples are about 25 phrases in each one of the ON state and the OFF state.
  • the profile is an average of the speaker embeddings.
  • the profile comprises two centroid vectors.
  • one of the two centroid vectors is the ON centroid defined as an average of the speaker embeddings derived from the enrollment data in the ON state
  • another one of the two centroid vectors is the OFF centroid defined as an average of the speaker embeddings derived from the enrollment data in the OFF state.
  • the auxiliary model comprises a trainable part of 384 parameters.
  • the method further comprising setting a threshold level defining the difference between the ON state and the OFF state.
  • the monitoring of the state of the user is conducted in a passive manner.
  • the threshold can be tuned.
  • the distances are a first distance between the speaker embeddings and the ON centroid and a second distance between the speaker embeddings and the OFF centroid.
  • FIG. 1 shows a flow chart of enrollment method, in accordance with some exemplary embodiments of the disclosed subject matter
  • FIG. 2 shows a flow chart of recognition method, in accordance with some exemplary embodiments of the disclosed subject matter
  • These computer program instructions may also be stored in a computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.
  • the computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • speaker embeddings models such as ECAPA-TDNN, x-vectors, d- vectors and c-vectors speaker embedding models.
  • the current subject matter uses such speaker identification models, preferably ECAPA-TDNN speaker model for the purpose of Parkinson’s disease ON/OFF state detection.
  • the current subject matter uses a deep neural network that maps a single input audio signal to a compressed vector of 192 floating point numbers. These vectors are referred to as speaker embeddings. Two stages of processing are involved; enrollment and recognition-time, as further described below.
  • a user provides to a computerized system data comprising examples of his/her speech in the ON state and in the OFF state.
  • the ON state is when the user is sufficiently medicated and symptoms are suppressed; accordingly the speech is characterized in one specific manner and the OFF state is when the user is insufficiently medicated and symptoms are displayed; and accordingly the speech is characterized in a second specific manner.
  • the computerized process does not need a specific format or storage method, and does not require additional hardware.
  • the speech samples can be general, not part of a specific dictionary or group of words or expressions, and only a small number of samples in each state are necessary. In a preferred embodiment, only 25 phrases in each state are required.
  • the variety of phrases the user can use are, for example, everyday used words such as ‘turn on the lights’, ‘Alexa’ and similar.
  • step 104 the computerized process stores the user’s speech samples and a profile of the speaker is constructed from these enrolment signals.
  • the speaker profile is an average of the speaker embeddings derived from enrolment signals.
  • the speaker profile comprises two centroid vectors.
  • the profile of the speaker is saved as an auxiliary model.
  • the centroid vectors are the same size as the speaker embedding vector, having 192 dimensions. These centroid vectors are as follows: The ON centroid: the average of the set of speaker embeddings derived from the set of enrollment signals in the ON state and the OFF centroid: the average of the set of speaker embeddings derived from the set of enrollment signals in the OFF state.
  • the trainable part of the model is relatively small - two 192-dimensional vectors (one for the ON state, one for the OFF state), so only 384 parameters. Since it is so small, it can be robustly estimated from a very small amount of data.
  • the model is designed to extract characteristics related to the speaker from input speech (accent, gender, state-of-health, etc.). These speaker characteristics remain relatively constant and stable across utterances of different phrases and words.
  • the computerized process conducts monitoring of the user’s state.
  • the monitoring step can be passive.
  • the state of the user is assessed from the speech signal, which is an audio file containing the user's speech.
  • the speaker characteristics i.e. the speaker embeddings
  • the speaker characteristics are extracted from the speech signal by the ECAPA-TDNN model.
  • the speaker characteristics are insensitive to what is being said (i.e. the word sequence); but are sensitive to how it is said (accent, gender, stat-of-health, etc.).
  • a threshold- a level, point, or value above which something is true or will take place while below that value, it is false or will not take place - is determined to classify the user’s state as an ON state or an OFF state.
  • the threshold tuning can be done by the user or can be automatically tuned by the computerized process.
  • step 206 the computerized process receives a test signal from the user.
  • step 208 the computerized process extracts recognition signal embeddings and computes the test signal.
  • step 210 the computerized process computes a distance between the test signal and the speaker profile ON and OFF states.
  • the distance between the test embedding and the ON (dist_ON) and OFF (dist_OFF) centroids for the test user are computed using negative cosine similarity metric, as an example.
  • step 212 the computerized process computes the difference between the two distances computed in step 210 (dist_OFF - dist_ON). If the distance difference exceeds the threshold set is step 204, the utterance is classified as the ON state, otherwise it is classified as OFF state.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of program code, which comprises one or more executable instructions for implementing the specified logical function(s).
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration can be implemented by special purpose hardware based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
  • the present subject matter may be a system, a method, and/or a computer program product.
  • the computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present subject matter.
  • the computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
  • the computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • a non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only.
  • memory EPROM or Flash memory
  • SRAM static random access memory
  • CD-ROM compact disc read-only memory
  • DVD digital versatile disk
  • a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
  • Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network.
  • the network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
  • Computer readable program instructions for carrying out operations of the present subject matter may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, statesetting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, python, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
  • electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present subject matter.
  • These computer readable program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
  • the computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

L'invention concerne un procédé de classification de représentations de locuteur pour un utilisateur souffrant de la maladie de Parkinson. Le procédé comprend, après enrôlement de données avec des échantillons de parole dans un état ACTIF et dans un état INACTIF, la construction d'un profil de l'utilisateur dans un modèle auxiliaire, la surveillance d'un état de l'utilisateur et la réception d'un signal de test de l'utilisateur, l'extraction de représentations à partir d'un signal de test, et finalement, la classification du signal de reconnaissance en tant qu'état ACTIF ou état INACTIF.
PCT/IL2024/050578 2023-06-12 2024-06-11 Représentations de locuteur pour la classification actif/inactif de la maladie de parkinson Ceased WO2024257097A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363507480P 2023-06-12 2023-06-12
US63/507,480 2023-06-12

Publications (2)

Publication Number Publication Date
WO2024257097A2 true WO2024257097A2 (fr) 2024-12-19
WO2024257097A3 WO2024257097A3 (fr) 2025-01-23

Family

ID=93852763

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IL2024/050578 Ceased WO2024257097A2 (fr) 2023-06-12 2024-06-11 Représentations de locuteur pour la classification actif/inactif de la maladie de parkinson

Country Status (1)

Country Link
WO (1) WO2024257097A2 (fr)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180206775A1 (en) * 2017-01-23 2018-07-26 The Johns Hopkins University Measuring medication response using wearables for parkinson's disease
GB2567826B (en) * 2017-10-24 2023-04-26 Cambridge Cognition Ltd System and method for assessing physiological state
US11901056B2 (en) * 2019-11-30 2024-02-13 Kpn Innovations, Llc Methods and systems for informed selection of prescriptive therapies

Also Published As

Publication number Publication date
WO2024257097A3 (fr) 2025-01-23

Similar Documents

Publication Publication Date Title
CN107680582B (zh) 声学模型训练方法、语音识别方法、装置、设备及介质
Jansen et al. Towards spoken term discovery at scale with zero resources.
CN112735383A (zh) 语音信号的处理方法、装置、设备及存储介质
CN111081279A (zh) 语音情绪波动分析方法及装置
US20170263240A1 (en) Combining auditory attention cues with phoneme posterior scores for phone/vowel/syllable boundary detection
Sahoo et al. Emotion recognition from audio-visual data using rule based decision level fusion
CN115132170B (zh) 语种分类方法、装置及计算机可读存储介质
CN103489445A (zh) 一种识别音频中人声的方法及装置
CN114203159B (zh) 语音情感识别方法、终端设备及计算机可读存储介质
CN117493495A (zh) 异常语言检测方法、电子设备、存储介质
CN111540363B (zh) 关键词模型及解码网络构建方法、检测方法及相关设备
CN115083397B (zh) 歌词声学模型的训练方法、歌词识别方法、设备和产品
EP3267438B1 (fr) Authentification de locuteur utilisant des réseaux de neurones artificiels:
CN116153336A (zh) 一种基于多域信息融合的合成语音检测方法
Fuchs et al. Unsupervised word segmentation using k nearest neighbors
Birla A robust unsupervised pattern discovery and clustering of speech signals
US20220270637A1 (en) Utterance section detection device, utterance section detection method, and program
CN111933153B (zh) 一种语音分割点的确定方法和装置
CN113111855A (zh) 一种多模态情感识别方法、装置、电子设备及存储介质
WO2024257097A2 (fr) Représentations de locuteur pour la classification actif/inactif de la maladie de parkinson
Itagi et al. Lisp detection and correction based on feature extraction and random forest classifier
Paulino et al. A brazilian speech database
KR101925248B1 (ko) 음성 인증 최적화를 위해 음성 특징벡터를 활용하는 방법 및 장치
Castan et al. Segmentation-by-classification system based on factor analysis
US20220277761A1 (en) Impression estimation apparatus, learning apparatus, methods and programs for the same

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE