JPH10228295A - Hierarchial feeling recognition device - Google Patents
Hierarchial feeling recognition deviceInfo
- Publication number
- JPH10228295A JPH10228295A JP9030576A JP3057697A JPH10228295A JP H10228295 A JPH10228295 A JP H10228295A JP 9030576 A JP9030576 A JP 9030576A JP 3057697 A JP3057697 A JP 3057697A JP H10228295 A JPH10228295 A JP H10228295A
- Authority
- JP
- Japan
- Prior art keywords
- feeling
- emotion
- voice
- recognition
- weighting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000008451 emotion Effects 0.000 claims description 41
- 230000008909 emotion recognition Effects 0.000 claims description 25
- 238000000605 extraction Methods 0.000 claims description 2
- 230000014509 gene expression Effects 0.000 abstract description 14
- 230000008921 facial expression Effects 0.000 abstract description 7
- 238000000034 method Methods 0.000 abstract description 4
- 238000013528 artificial neural network Methods 0.000 abstract description 2
- 210000001061 forehead Anatomy 0.000 abstract description 2
- 230000033001 locomotion Effects 0.000 abstract description 2
- 230000001815 facial effect Effects 0.000 abstract 1
- 238000002474 experimental method Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 241000134253 Lanka Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
Description
【０００１】[0001]
【発明の属する技術分野】この発明は階層的感情認識装
置に関し、特に、人間の声からの情報と顔の表情からの
情報とを統合して階層的に感情を認識できるような階層
的感情認識装置に関する。BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hierarchical emotion recognition apparatus, and more particularly to a hierarchical emotion recognition apparatus capable of hierarchically recognizing emotion by integrating information from a human voice and information from a facial expression. Related to the device.
【０００２】[0002]
【従来の技術】遠隔地にいる人々があたかも同じ卓を囲
んで会議をするような感じで話し合ったり時間や空間を
克服して本当に有意義なコミュニケーションをするに
は、人工現実感を利用した臨場感通信が望まれている。
臨場感通信で会議を行なうときに、会議に参加している
人の感情を音声と画像で表現する必要がある。そのため
には、感情認識するためのアルゴリズムが必要とされ
る。2. Description of the Related Art In order for people in remote areas to talk as if they are having a meeting around the same table, or to overcome time and space for truly meaningful communication, a sense of presence utilizing artificial reality is required. Communication is desired.
When a meeting is performed by the presence communication, it is necessary to express the emotions of the people participating in the meeting by voice and images. For that purpose, an algorithm for emotion recognition is required.
【０００３】[0003]
【発明が解決しようとする課題】従来より、感情認識の
アルゴリズムについて知られたものもあるが、そのほと
んどが画像情報と音声情報とを別々に利用したものであ
り、感情の種類によって音声情報を優先させるかあるい
は画像情報を優先させるかなどの優先度合いを利用して
いなかった。Conventionally, there have been known algorithms for emotion recognition, but most of them use image information and audio information separately. The priorities such as whether to give priority to image information or image information have not been used.
【０００４】それゆえに、この発明の主たる目的は、感
情別に音声情報と画像情報に重み付けして感情を認識し
得る階層的感情認識装置を提供することである。[0004] Therefore, a main object of the present invention is to provide a hierarchical emotion recognition device capable of recognizing an emotion by weighting voice information and image information for each emotion.
【０００５】[0005]
【課題を解決するための手段】請求項１に係る発明は、
人間の声の情報と顔の情報とから感情を認識する階層的
感情認識装置であって、人間の声の情報から音声データ
を抽出する音声抽出手段と、人間の顔の情報から画像デ
ータを抽出する画像抽出手段と、抽出された音声データ
に基づいて感情を認識する第１の感情認識手段と、抽出
された画像データに基づいて感情を認識する第２の感情
認識手段と、第１および第２の感情認識手段によってそ
れぞれ認識された感情に重み付けして統合する重み付け
手段とを備えて構成される。The invention according to claim 1 is
A hierarchical emotion recognition device for recognizing emotions from human voice information and face information, comprising: voice extraction means for extracting voice data from human voice information; and image data extraction from human face information Image extracting means, first emotion recognizing means for recognizing emotion based on the extracted voice data, second emotion recognizing means for recognizing emotion based on the extracted image data, and first and second Weighting means for weighting and integrating the emotions respectively recognized by the two emotion recognition means.
【０００６】請求項２に係る発明では、請求項１の重み
付け手段は、第１の感情認識手段によって「悲しみ」と
「恐怖」が認識されたとき、これらの感情の声の重み付
けを大きくし、第２の感情認識手段によって「怒り」と
「幸福」と「驚き」が認識されたとき、これらの感情の
画像の重み付けを大きくする。In the invention according to claim 2, the weighting means of claim 1 increases the weight of voices of these emotions when "sadness" and "fear" are recognized by the first emotion recognition means, When "anger", "happiness", and "surprise" are recognized by the second emotion recognition means, the weight of the image of these emotions is increased.
【０００７】[0007]
【発明の実施の形態】まず、本願発明者らは、感情を認
識するとき、人間の声を優先する感情と、顔の表情を優
先する感情と、声と表情の両方に依存する感情の３種類
に分けられることを被験者を使った実験により確認し
た。まず、その実験結果について説明する。BEST MODE FOR CARRYING OUT THE INVENTION First, when recognizing emotions, the inventors of the present invention have three types of emotions: one that prioritizes human voice, one that prioritizes facial expression, and one that depends on both voice and facial expression. It was confirmed by the experiment using the test subjects that they were classified into different types. First, the experimental results will be described.
【０００８】図１は感情認識実験方法を説明するための
図である。この実施形態では、スペイン語とシンハラ語
（スリランカの国語）で次に示す６つの感情を標題とし
て与え、図１に示すタイムシーケンスでその感情を表情
と音声とで表わした人の声と顔画像を録画した。FIG. 1 is a diagram for explaining an emotion recognition experiment method. In this embodiment, the following six emotions are given as titles in Spanish and Sinhala (the national language of Sri Lanka), and the emotions are expressed by facial expressions and voices in a time sequence shown in FIG. Was recorded.
【０００９】 怒り：なぜあなたは来なかったのか 幸福：おはようございます 悲しみ：私はお金を使った 驚き：なんと不快な作品だ 嫌悪：私は彼が嫌いだ 恐怖：どうか殺さないで 被験者はスペイン語とシンハラ語を理解することができ
ない日本人の大学生である。これらの言語を理解できる
人では、言葉により感情を判別してしまい、声と顔の表
情のみで感情を判別できないからである。実験は、Ａと
Ｂの２種類で行なった。実験Ａは声と映像を別々に被験
者に見せるものであり、実験Ｂは声と映像の組合せを変
えて被験者に見せるものであり、たとえば幸福な顔の映
像と悲しみの声とを組合せたようなものである。Anger: Why did you not come Happiness: Good morning Sadness: I used the money Surprise: What an unpleasant work Disgust: I hate him Fear: Don't kill me Subject is Spanish He is a Japanese college student who cannot understand Sinhala. This is because a person who can understand these languages discriminates emotions by words, and cannot discriminate emotions only by voice and facial expressions. The experiment was performed with two types, A and B. Experiment A is to show the voice and the image separately to the subject, and Experiment B is to change the combination of the voice and the image to show the subject, such as combining a happy face image and a sad voice. Things.
【００１０】図２はスペイン語での実験Ａの認識結果を
示し、図３はシンハラ語での実験Ａの認識結果を示し、
図４はスペイン語とシンハラ語における６つの感情の認
識結果を対比して示した図である。FIG. 2 shows the recognition result of Experiment A in Spanish, FIG. 3 shows the recognition result of Experiment A in Sinhala,
FIG. 4 is a diagram comparing recognition results of six emotions in Spanish and Sinhala.
【００１１】図２と図３とを対比すれば明らかなよう
に、言語で認識結果は異なっているが、両者の傾向はよ
く似ていることがわかる。すなわち、「悲しみ」と「恐
怖」はともに映像よりも音声のみで認識される度合いが
高くなっており、その他の「怒り」，「幸福」，「驚
き」，「嫌悪」は音声のみよりも表情で認識される度合
いが高くなっていることがわかる。As is apparent from a comparison between FIG. 2 and FIG. 3, the recognition results are different depending on the language, but the tendency is very similar. That is, "sorrow" and "fear" are both recognized more by voice than video, and other "anger", "happiness", "surprise", and "disgust" are more expressive than voice only. It can be seen that the degree of recognition is higher.
【００１２】図５はスペイン語での実験Ｂの認識結果を
示し、図６はシンハラ語での実験Ｂの認識結果を示し、
図７はスペイン語とシンハラ語における６つの感情の認
識結果を対比して示した図である。FIG. 5 shows the recognition result of Experiment B in Spanish, FIG. 6 shows the recognition result of Experiment B in Sinhala,
FIG. 7 is a diagram comparing the recognition results of six emotions in Spanish and Sinhala.
【００１３】この実験Ｂにおいても、「悲しみ」と「恐
怖」はともに音声のみで認識される度合いが高くなって
おり、「怒り」，「幸福」，「驚き」は映像のみで認識
される度合いが高くなっており、「嫌悪」だけではスペ
イン語では映像が優位になっており、シンハラ語では音
声が優位になっている。[0013] In this experiment B, too, "sorrow" and "fear" are both highly recognized only by voice, and "anger", "happiness" and "surprise" are recognized only by video. The image is dominant in Spanish for "disgust" alone, and the sound is dominant in Sinhala.
【００１４】上述の実験結果から、「怒り」と「幸福」
と「驚き」が表情を優先し、「悲しみ」と「恐怖」とが
声を優先していることが確かめられた。From the above experimental results, "anger" and "happiness"
It was confirmed that "surprise" and "surprise" prioritized expression, and "sadness" and "fear" prioritized voice.
【００１５】図８はこの発明の一実施形態を示すブロッ
ク図である。図８において、カメラ１は話者の顔画像を
撮像し、マイクロホン２は話者の声を取得する。顔画像
は画像データを用いた感情認識部３に入力されて感情が
認識され、音声は音声データを用いた感情認識部４に与
えられて感情が認識される。それぞれの認識結果は重み
付け処理部５に与えられて音声と表情を統合した感情が
出力される。FIG. 8 is a block diagram showing an embodiment of the present invention. 8, a camera 1 captures a face image of a speaker, and a microphone 2 acquires a voice of the speaker. The face image is input to the emotion recognition unit 3 using the image data to recognize the emotion, and the voice is applied to the emotion recognition unit 4 using the voice data to recognize the emotion. Each recognition result is given to the weighting processing unit 5, and an emotion obtained by integrating voice and expression is output.
【００１６】ここで、画像データを用いた感情認識部３
としては、Ｊ．ＳＩＣＥ（計測と制御）特集：人間と共
存するロボットの新技術、Ｖｏｌ．３４，Ｎｏ．４，ｐ
ｐ．２４８−２５４，Ａｐｒ．１９９５で発表された技
術が用いられる。すなわち、撮像した顔画像からたとえ
ば額，目，口の動きに関連する３０の特徴を抽出してニ
ューラルネットワークに与え、６つの感情を認識する。
また、音声データを用いた感情認識部４としては、In P
roceedings of Spring Symposiurm on Believable Agen
ts, Stanford University, AAAI （American Associati
on for Artificial Intelligence）, March 1994に発表
された技術が用いられる。Here, the emotion recognition unit 3 using the image data
As J. Special Issue on SICE (Measurement and Control): New Technology for Robots that Coexist with Humans, Vol. 34, no. 4, p
p. 248-254, Apr. The technology announced in 1995 is used. That is, for example, 30 features related to forehead, eye, and mouth movements are extracted from the captured face image and given to the neural network to recognize six emotions.
The emotion recognition unit 4 using voice data includes In P
roceedings of Spring Symposiurm on Believable Agen
ts, Stanford University, AAAI (American Associati
on for Artificial Intelligence), March 1994.
【００１７】重み付け処理部５は怒り，幸福，悲しみ，
驚き，嫌悪，恐怖のそれぞれの画像データをＶAng ，Ｖ
Hap ，ＶSad ，ＶSur ，ＶDis ，ＶFea で表わし、それ
ぞの音声データをＡAng ，ＡHap ，ＡSad ，ＡSur ，Ａ
Dis ，ＡFea とすると、次の第（１）式のように入力さ
れる。 （ＶAng ，ＶHap ，ＶSad ，ＶSur ，ＶDis ，ＶFea ，ＡAng ，ＡHap ，ＡSad ，ＡSur ，ＡDis ，ＡFea ）∈｛０，１｝ （１） 重み付け処理部５の中での感情選択手法は次のとおりと
なる。The weighting processing unit 5 is angry, happy, sad,
Surprise, disgust, and fear image data are represented by V Ang , V
Hap , V Sad , V Sur , V Dis , and V Fea , and their audio data are A Ang , A Hap , A Sad , A Sur , A
If Dis and A Fea are input, they are input as in the following equation (1). (V Ang , V Hap , V Sad , V Sur , V Dis , V Fea , A Ang , A Hap , A Sad , A Sur , A Dis , A Fea ) {0,1} (1) Weighting processing section 5 The following is the method of selecting emotions in the book.
【００１８】 怒り* ＝Ｗ(1,Ang) ＶAng ＋Ｗ(2,Ang) ＡAng 幸福* ＝Ｗ(1,Hap) ＶHap ＋Ｗ(2,Hap) ＡHap 悲しみ* ＝Ｗ(1,Sad) ＶSad ＋Ｗ(2,Sad) ＡSad （２） 驚き* ＝Ｗ(1,Sur) ＶSur ＋Ｗ(2,Sur) ＡSur 嫌悪* Ｗ(1,Dis) ＶDis ＋Ｗ(2,Dis) ＡDis 恐怖* Ｗ(1,Fea) ＶFea ＋Ｗ(2,Fea) ＡFea たとえば、重み付けマトリクスＷは実験の結果により、
次のように設定できる。Anger * = W (1, Ang) V Ang + W (2, Ang) A Ang Happiness * = W (1, Hap) V Hap + W (2, Hap) A Hap Sorrow * = W (1, Sad) V Sad + W (2, Sad) A Sad (2) Surprise * = W (1, Sur) V Sur + W (2, Sur) A Sur disgust * W (1, Dis) V Dis + W (2, Dis) A Dis Fear * W (1, Fea) V Fea + W (2, Fea) A Fea For example, the weighting matrix W
It can be set as follows:
【００１９】 Ｗ(1,Ang) ＝２２．５９ Ｗ(2,Ang) ＝ ０ Ｗ(1,Hap) ＝４１．８８ Ｗ(2,Hap) ＝ ０ Ｗ(1,Sad) ＝ ０ Ｗ＝(2,Sad) ２０．６５ （３） Ｗ(1,Sur) ＝１１．６４ Ｗ(2,Sur) ＝ ０ Ｗ(1,Dis) ＝２３．３０ Ｗ(2,Dis) ＝ ０ Ｗ(1,Fea) ＝ ０ Ｗ(2,Fea) ＝６．５４ 基本的に、Ｗ(1,xxx) ，Ｗ(2,xxx) の値は次のとおり設
定すれば、従来の感情認識方法により優位な結果が得ら
れる。W (1, Ang) = 22.59 W (2, Ang) = 0 W (1, Hap) = 41.88 W (2, Hap) = 0 W (1, Sad) = 0 W = ( ( 2, Sad) 20.65 (3) W (1, Sur) = 11.64 W (2, Sur) = 0 W (1, Dis) = 23.30 W (2, Dis) = 0 W (1, Fea) = 0 W (2, Fea) = 6.54 Basically, if the values of W (1, xxx) and W (2, xxx) are set as follows, the result is superior to the conventional emotion recognition method. Is obtained.
【００２０】 ｛Ｗ(1,Ang) ，Ｗ(1,Hap) ，Ｗ(2,Sad) ，Ｗ(1,Sur) ，Ｗ(1,Dis) ， Ｗ(2,Fea) ｝＞＞＞１ ｛Ｗ(2,Ang) ，Ｗ(2,Hap) ，Ｗ(1,Sad) ，Ｗ(2,Sur) ，Ｗ(2,Dis) ， Ｗ(1,Fea) ｝＜＝０ （４） 最終的に、第（５）式により入力感情が認識される。｛W (1, Ang) , W (1, Hap) , W (2, Sad) , W (1, Sur) , W (1, Dis) , W (2, Fea) ｝ >>>> 1 ｛W (2, Ang) , W (2, Hap) , W (1, Sad) , W (2, Sur) , W (2, Dis) , W (1, Fea) ｝ <= 0 (4) Final Then, the input emotion is recognized by the equation (5).
【００２１】 Ｍａｘ｛怒り* ，幸福* ，悲しみ* ，驚き* ，嫌悪* ，恐怖* ｝ （５） 図９はこの発明の一実施形態を用いて実現した臨場感通
信での会議システムを示す概念図である。臨場感通信で
の会議システムでは、互いに離れた空間１０，２０にい
る２人の人３０，４０が会議を行なう。一方の空間上に
いる人３０の表情および音声はこの発明の一実施形態に
よる認識装置５０で認識され、他方の空間２０の表現装
置８０で表現される。同様にして、他方の空間２０にい
る人４０の表情および音声は認識装置６０で認識され、
一方の空間１０の表現装置７０で表現される。そして、
各表現装置７０，８０には、会議をしている人３０，４
０の感情が表情と音声で表現される。すなわち、たとえ
ば驚きの音声を発したときには、驚いたときの顔の表情
となるように表現される。その際、実写映像を表示する
のではなく、コンピュータグラフィックス（ＣＧ）像で
再構成した人物像を表示することにより、実物の表情よ
りも強調した表情にすることができる。それによって、
自然な感情の表現で会議を進めることができる。Max {Angle * , Happiness * , Sadness * , Surprise * , Disgust * , Fear * } (5) FIG. 9 is a concept showing a conference system in a sense of presence realized by using an embodiment of the present invention. FIG. In a conference system using presence communication, two people 30, 40 in spaces 10, 20 separated from each other hold a conference. The expression and voice of the person 30 in one space are recognized by the recognition device 50 according to the embodiment of the present invention, and are expressed by the expression device 80 in the other space 20. Similarly, the expression and voice of the person 40 in the other space 20 are recognized by the recognition device 60,
One of the spaces 10 is expressed by the expression device 70. And
Each of the expression devices 70, 80 has a person 30, 4,
Zero emotions are expressed by facial expressions and voice. That is, for example, when a surprised voice is emitted, the expression is made to be the expression of the face when surprised. At this time, instead of displaying the actual image, a person image reconstructed by a computer graphics (CG) image is displayed, so that the expression can be emphasized more than the actual expression. Thereby,
The conference can be advanced with natural expressions of emotion.
【００２２】[0022]
【発明の効果】以上のように、この発明によれば、音声
データに基づいて認識した感情と、画像データに基づい
て認識した感情にそれぞれ重み付けして統合するように
したので、認識率を高めることができ、この認識結果に
基づいて感情を表現したとき、自然な表情と音声で感情
を再現できる。As described above, according to the present invention, the emotion recognized based on the voice data and the emotion recognized based on the image data are weighted and integrated, so that the recognition rate is improved. When emotions are expressed based on the recognition result, the emotions can be reproduced with natural expressions and voices.
【図１】感情認識実験結果を説明するための図である。FIG. 1 is a diagram for explaining an emotion recognition experiment result.
【図２】スペイン語での実験Ａの認識結果を示す図であ
る。FIG. 2 is a diagram showing recognition results of Experiment A in Spanish.
【図３】シンハラ語での実験Ａの認識結果を示す図であ
る。FIG. 3 is a diagram showing a recognition result of an experiment A in Sinhala language.
【図４】スペイン語とシンハラ語における６つの感情の
認識結果を対比して示した図である。FIG. 4 is a diagram showing a comparison between recognition results of six emotions in Spanish and Sinhala.
【図５】スペイン語での実験Ｂの認識結果を示す図であ
る。FIG. 5 is a diagram showing recognition results of Experiment B in Spanish.
【図６】シンハラ語での実験Ｂの認識結果を示す図であ
る。FIG. 6 is a diagram showing a recognition result of an experiment B in Sinhala.
【図７】スペイン語とシンハラ語における６つの感情の
認識結果を対比して示した図である。FIG. 7 is a diagram comparing recognition results of six emotions in Spanish and Sinhala.
【図８】この発明の一実施形態を示すブロック図であ
る。FIG. 8 is a block diagram showing one embodiment of the present invention.
【図９】この発明の一実施形態を用いて実施形態した臨
場感通信での会議システムを示す概念図である。FIG. 9 is a conceptual diagram illustrating a conference system in a sense of presence communication implemented using one embodiment of the present invention.
１ カメラ ２ マイクロホン ３ 画像データを用いた感情認識部 ４ 音声データを用いた感情認識部 ５ 重み付け処理部 Reference Signs List 1 camera 2 microphone 3 emotion recognition unit using image data 4 emotion recognition unit using voice data 5 weighting processing unit
───────────────────────────────────────────────────── フロントページの続き (72)発明者 宮里 勉 京都府相楽郡精華町大字乾谷小字三平谷５ 番地 株式会社エイ・ティ・アール知能映 像通信研究所内 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Tsutomu Miyazato 5 Sanraya, Inaya, Koika-cho, Soraku-cho, Soraku-gun, Kyoto ATI Intelligent Imaging Communications Laboratory
Claims (2)
認識する階層的感情認識装置であって、 前記人間の声の情報から音声データを抽出する音声抽出
手段、 前記人間の顔の情報から画像データを抽出する画像抽出
手段、 前記音声抽出手段によって抽出された音声データに基づ
いて感情を認識する第１の感情認識手段、 前記画像抽出手段によって抽出された画像データに基づ
いて感情を認識する第２の感情認識手段、および前記第
１および第２の感情認識手段によってそれぞれ認識され
た感情に重み付けして統合する重み付け手段を備えた、
階層的感情認識装置。1. A hierarchical emotion recognition apparatus for recognizing an emotion from human voice information and face information, comprising: a voice extraction unit for extracting voice data from the human voice information; Image extracting means for extracting image data from information; first emotion recognizing means for recognizing an emotion based on the voice data extracted by the voice extracting means; and emotion detecting means based on the image data extracted by the image extracting means. A second emotion recognition unit for recognizing, and a weighting unit for weighting and integrating the emotions recognized by the first and second emotion recognition units, respectively.
Hierarchical emotion recognition device.
識手段によって「悲しみ」と「恐怖」が認識されたと
き、これらの感情の声の重み付けを大きくし、前記第２
の感情認識手段によって「怒り」と「幸福」と「驚き」
が認識されたとき、これらの感情の画像の重み付けを大
きくすることを特徴とする、請求項１の階層的感情認識
装置。2. The weighting means, when “sadness” and “fear” are recognized by the first emotion recognition means, increases the weight of voices of these emotions, and
"Anger", "happiness" and "surprise" by means of emotion recognition
2. The hierarchical emotion recognition apparatus according to claim 1, wherein, when is recognized, the weight of these emotion images is increased.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP9030576A JP2967058B2 (en) | 1997-02-14 | 1997-02-14 | Hierarchical emotion recognition device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP9030576A JP2967058B2 (en) | 1997-02-14 | 1997-02-14 | Hierarchical emotion recognition device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JPH10228295A true JPH10228295A (en) | 1998-08-25 |
| JP2967058B2 JP2967058B2 (en) | 1999-10-25 |
Family
ID=12307686
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP9030576A Expired - Lifetime JP2967058B2 (en) | 1997-02-14 | 1997-02-14 | Hierarchical emotion recognition device |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP2967058B2 (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001083984A (en) * | 1999-09-09 | 2001-03-30 | Alpine Electronics Inc | Interface device |
| EP1555635A1 (en) * | 2004-01-19 | 2005-07-20 | Nec Corporation | Image processing apparatus, method and program |
| ES2242484A1 (en) * | 2003-01-24 | 2005-11-01 | Pedro Monagas Asensio | ANIMAL ANALYZING DEVICE FOR MAMMALS. |
| JP2006350577A (en) * | 2005-06-14 | 2006-12-28 | Fuji Xerox Co Ltd | Operation analyzing device |
| KR100903348B1 (en) * | 2007-11-28 | 2009-06-23 | 중앙대학교 산학협력단 | Emotion recognition mothod and system based on feature fusion |
| KR100958030B1 (en) * | 2007-11-28 | 2010-05-17 | 중앙대학교 산학협력단 | Decision Fusion Based Emotion Recognition Method and System |
| US7766828B2 (en) | 2004-10-27 | 2010-08-03 | Canon Kabushiki Kaisha | Estimation apparatus and its control method |
| US8396708B2 (en) | 2009-02-18 | 2013-03-12 | Samsung Electronics Co., Ltd. | Facial expression representation apparatus |
| US8407055B2 (en) | 2005-08-05 | 2013-03-26 | Sony Corporation | Information processing apparatus and method for recognizing a user's emotion |
| US8780221B2 (en) | 2008-04-09 | 2014-07-15 | Canon Kabushiki Kaisha | Facial expression recognition apparatus, image sensing apparatus, facial expression recognition method, and computer-readable storage medium |
| US8992227B2 (en) | 2001-04-28 | 2015-03-31 | Manchester Metropolitan University | Methods and apparatus for analysing the behaviour of a subject |
| US9110501B2 (en) | 2012-04-17 | 2015-08-18 | Samsung Electronics Co., Ltd. | Method and apparatus for detecting talking segments in a video sequence using visual cues |
| JP2018072876A (en) * | 2016-10-24 | 2018-05-10 | 富士ゼロックス株式会社 | Emotion estimation system, emotion estimation model generation system |
| CN108962255A (en) * | 2018-06-29 | 2018-12-07 | 北京百度网讯科技有限公司 | Emotion identification method, apparatus, server and the storage medium of voice conversation |
| JP2020181022A (en) * | 2019-04-23 | 2020-11-05 | コニカミノルタ株式会社 | Conference support device, conference support system and conference support program |
| JP2021005122A (en) * | 2019-06-25 | 2021-01-14 | Kddi株式会社 | Program, device and method to build learning model that integrates different media data occurred in time series for the same event |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5539860A (en) | 1993-12-22 | 1996-07-23 | At&T Corp. | Speech recognition using bio-signals |
-
1997
- 1997-02-14 JP JP9030576A patent/JP2967058B2/en not_active Expired - Lifetime
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001083984A (en) * | 1999-09-09 | 2001-03-30 | Alpine Electronics Inc | Interface device |
| US8992227B2 (en) | 2001-04-28 | 2015-03-31 | Manchester Metropolitan University | Methods and apparatus for analysing the behaviour of a subject |
| ES2242484A1 (en) * | 2003-01-24 | 2005-11-01 | Pedro Monagas Asensio | ANIMAL ANALYZING DEVICE FOR MAMMALS. |
| EP1555635A1 (en) * | 2004-01-19 | 2005-07-20 | Nec Corporation | Image processing apparatus, method and program |
| US7766828B2 (en) | 2004-10-27 | 2010-08-03 | Canon Kabushiki Kaisha | Estimation apparatus and its control method |
| JP2006350577A (en) * | 2005-06-14 | 2006-12-28 | Fuji Xerox Co Ltd | Operation analyzing device |
| US8116517B2 (en) | 2005-06-14 | 2012-02-14 | Fuji Xerox Co., Ltd. | Action analysis apparatus |
| US8385598B2 (en) | 2005-06-14 | 2013-02-26 | Fuji Xerox Co., Ltd. | Action analysis apparatus |
| US8407055B2 (en) | 2005-08-05 | 2013-03-26 | Sony Corporation | Information processing apparatus and method for recognizing a user's emotion |
| KR100958030B1 (en) * | 2007-11-28 | 2010-05-17 | 중앙대학교 산학협력단 | Decision Fusion Based Emotion Recognition Method and System |
| KR100903348B1 (en) * | 2007-11-28 | 2009-06-23 | 중앙대학교 산학협력단 | Emotion recognition mothod and system based on feature fusion |
| US8780221B2 (en) | 2008-04-09 | 2014-07-15 | Canon Kabushiki Kaisha | Facial expression recognition apparatus, image sensing apparatus, facial expression recognition method, and computer-readable storage medium |
| US9147107B2 (en) | 2008-04-09 | 2015-09-29 | Canon Kabushiki Kaisha | Facial expression recognition apparatus, image sensing apparatus, facial expression recognition method, and computer-readable storage medium |
| US9258482B2 (en) | 2008-04-09 | 2016-02-09 | Canon Kabushiki Kaisha | Facial expression recognition apparatus, image sensing apparatus, facial expression recognition method, and computer-readable storage medium |
| US8396708B2 (en) | 2009-02-18 | 2013-03-12 | Samsung Electronics Co., Ltd. | Facial expression representation apparatus |
| US9110501B2 (en) | 2012-04-17 | 2015-08-18 | Samsung Electronics Co., Ltd. | Method and apparatus for detecting talking segments in a video sequence using visual cues |
| JP2018072876A (en) * | 2016-10-24 | 2018-05-10 | 富士ゼロックス株式会社 | Emotion estimation system, emotion estimation model generation system |
| CN108962255A (en) * | 2018-06-29 | 2018-12-07 | 北京百度网讯科技有限公司 | Emotion identification method, apparatus, server and the storage medium of voice conversation |
| JP2020181022A (en) * | 2019-04-23 | 2020-11-05 | コニカミノルタ株式会社 | Conference support device, conference support system and conference support program |
| JP2021005122A (en) * | 2019-06-25 | 2021-01-14 | Kddi株式会社 | Program, device and method to build learning model that integrates different media data occurred in time series for the same event |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2967058B2 (en) | 1999-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| De Silva et al. | Bimodal emotion recognition | |
| US11551393B2 (en) | Systems and methods for animation generation | |
| JP2967058B2 (en) | Hierarchical emotion recognition device | |
| Chen et al. | Multimodal human emotion/expression recognition | |
| Bruce | The role of the face in communication: Implications for videophone design | |
| Chen | Audiovisual speech processing | |
| CN110446000B (en) | A method and device for generating dialogue characters | |
| US20160134840A1 (en) | Avatar-Mediated Telepresence Systems with Enhanced Filtering | |
| Huang et al. | Bimodal emotion recognition by man and machine | |
| US20210183021A1 (en) | Method to provide a video with a computer-modified visual of a desired face of a person | |
| Shan et al. | Speech-in-noise comprehension is improved when viewing a deep-neural-network-generated talking face | |
| Weber et al. | Real-time adaptation of a robotic joke teller based on human social signals | |
| CN117153195B (en) | Method and system for generating speaker face video based on adaptive region shielding | |
| CN113395569A (en) | Video generation method and device | |
| Ritschel et al. | Multimodal joke generation and paralinguistic personalization for a socially-aware robot | |
| WO2025130325A1 (en) | Face generation method and apparatus | |
| Lee et al. | AEGIS: A real-time multimodal augmented reality computer vision based system to assist facial expression recognition for individuals with autism spectrum disorder | |
| Manglani et al. | Lip reading into text using deep learning | |
| Hill et al. | Range-and domain-specific exaggeration of facial speech | |
| Khekare et al. | A Deep Dive into Existing Lip Reading Technologies | |
| Verma et al. | Animating expressive faces across languages | |
| Gupta et al. | Video Conferencing with Sign language Detection | |
| KR20190000087A (en) | Multimedia Processing Method and System using Facial Expression Recognition | |
| Voß et al. | Addressing data scarcity in multimodal user state recognition by combining semi-supervised and supervised learning | |
| Faruquie et al. | Audio Driven Facial Animation For Audio-Visual Reality. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 19990803 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20080813 Year of fee payment: 9 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20090813 Year of fee payment: 10 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20100813 Year of fee payment: 11 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20110813 Year of fee payment: 12 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20120813 Year of fee payment: 13 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20120813 Year of fee payment: 13 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20130813 Year of fee payment: 14 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
| EXPY | Cancellation because of completion of term |