JPH11242751A - Animation control apparatus and method, and text-to-speech apparatus - Google Patents

Abstract

(57) [Summary] [Problem] To read out a sentence, it is possible to automatically generate an animation of a face having an expression corresponding to the content of the sentence to be read out. A sentence analysis unit analyzes a sentence to be read, which is input from a sentence input unit, and a semantic analysis unit and a mood information extraction unit determine a mood indicating an atmosphere of the sentence. On the other hand, the reading information generation unit 194 generates reading information of the sentence. The animation generation unit 110 generates an animation of a facial expression and a mouth movement while referring to the animation dictionary 112 based on the mood information from the mood information extraction unit 109 and the read information from the read information generation unit 104. . Then, the animation display unit 111 displays the generated animation while synchronizing with the audio output of the sentence in the audio output unit 106.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to an animation control apparatus and method. In particular, the present invention relates to an animation control device and method suitable for controlling animation of a face when a text is read out by a device, and a text-to-speech device provided with the animation device.

[0002]

2. Description of the Related Art In general, a speech synthesis system having a face animation is often used as a user interface of a guidance system for asking and answering a user. In this type of system, the shape of the lips is changed during the utterance time according to the length of the sentence to be uttered,
The pretend that you are talking is performed. In addition, there is a type in which an expression such as a smiling face or a sad face is switched depending on the type of response. However, in such a guidance system that involves the switching of facial expressions, it is necessary to decide what facial expression should be generated for each prepared response sentence, hold the information on the facial expression, and use it when uttering the response sentence. , The information of the expression corresponding to the response sentence is given as control information.

[0003]

However, when reading out a given sentence, it may be possible to change the shape of the lips over the reading period of the sentence. It cannot be removed.

[0004] In other words, in the case of a guidance system in which a question and answer is given, the contents of the response are prepared in advance by the system, so that the expression can be controlled in accordance with the content. However, when reading out a given sentence, there is no means to know the content or the function to select an expression that matches it, so the facial animation either moves the lips with no expression or speaks out randomly. While changing the voice.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to provide an animation control apparatus and a method capable of performing animation control according to the contents of a sentence to be read when the sentence is read. And

[0006]

An animation control apparatus according to the present invention for achieving the above object has, for example, the following arrangement. That is, there are provided a determination unit that analyzes a sentence to be read aloud and determines a mood indicating an atmosphere of the sentence, and a display control unit that controls an animation display based on the mood determined by the determination unit.

Preferably, the display control means includes:
Further, an animation display is controlled based on the reading information corresponding to the sentence to be read.

Further, the animation control method according to the present invention for achieving the above object has, for example, the following steps. That is, the method includes a determining step of analyzing a sentence to be read and determining a mood indicating an atmosphere of the sentence, and a display controlling step of controlling an animation display based on the mood determined in the determining step.

Further, according to the present invention, there is provided a text-to-speech apparatus provided with the above-mentioned animation control device.

[0010]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

[First Embodiment] FIG. 1 is a block diagram showing a schematic configuration of a text-to-speech apparatus according to a first embodiment. In FIG. 1, reference numeral 10 denotes a CPU, which realizes various controls by executing a control program stored in a ROM 11 or a RAM 12. 11 is a ROM,
A boot program describing various processes at the time of starting the device and various data are stored. 12 is a RAM, C
The PU 10 provides a work area for executing various controls. Reference numeral 13 denotes a display, which performs various displays under the control of the CPU 10. In the present embodiment, the face of the speaker is displayed according to the utterance of the synthesized voice. 14
Is an input device including a keyboard and a mouse. Fifteen
Is an external storage device, and various dictionaries (syntax analysis dictionary 10
2. Semantic analysis dictionary 107, animation dictionary 11
2) and the control program 15a are stored. The control program 15a is loaded from the external storage device 15 to the RAM 12 at the time of execution. 16 is a speaker,
Outputs synthesized speech and the like.

FIG. 2 is a block diagram showing a functional configuration of the text-to-speech apparatus according to the first embodiment. Referring to FIG. 1, reference numeral 101 denotes a text input unit for inputting text to be read out by an input device 14 such as a keyboard. Of course, it may be a sentence stored in the external storage device 15 or the like in advance. 102 is a syntax analysis dictionary, and 103 is a syntax analysis unit. The syntax analysis unit 103 analyzes the syntax of the text input from the text input unit 101 while referring to the syntax analysis dictionary 102, and provides the result to the semantic analysis unit 108 and the reading information generation unit 104, respectively.

Reference numeral 104 denotes a reading information generation unit which generates reading information for utterance based on the input result of the syntax analysis, and provides the information to the speech synthesis unit 105 and the animation generation unit 110. The voice synthesis unit 105 performs voice synthesis based on the provided reading information to generate voice information. 106 is an audio output unit, based on audio information,
The sound is output via the speaker 16.

Reference numeral 108 denotes a semantic analysis unit.
Based on the analysis result provided by the third sentence 03, the mood of the speaker represented by the sentence is analyzed, and the result is provided to the mood information extracting unit 109. A mood information extraction unit 109 generates mood information based on the analysis result of the semantic analysis unit 108. In this example, “hearing”, “confused”, and “convenience” are prepared as mood information, and the mood information extraction unit 109 selects one of these based on the result of the semantic analysis to generate an animation. Part 11
0 is provided.

Reference numeral 110 denotes an animation generator which generates lip animation information based on the reading information provided by the reading information generator 104, and expresses the facial expression based on the mood information provided by the mood information extracting unit 109. Generate animation information for In order to synchronize the movement of the mouth with the uttered voice, the animation generating unit 110 inputs the voice information generated by the voice synthesizing unit 105, and generates lip animation information based on the voice information. Reference numeral 111 denotes an animation display unit, which performs animation display of a face on the display 13 based on the animation information generated by the animation generation unit 110.

FIG. 3 is a diagram showing a configuration example of the animation dictionary 112 according to the present embodiment. The animation dictionary 112 includes a facial expression dictionary unit for storing facial expression animation information and a lip animation dictionary unit for storing lip animation information. The expression dictionary section stores mood information and expression data in pairs. The lip animation dictionary section stores reading information and lip animation information in pairs.

The operation of the text-to-speech apparatus according to the present embodiment having the above-described configuration will be described. FIG. 4 is a flowchart illustrating the procedure of the document reading process according to the present embodiment.

First, in step S201, the text input unit 1
A sentence to be read out from 01 is input. Step S2
In 02, it is determined whether there is an unprocessed sentence, and if so, the process proceeds to step S203.

In step S203, the syntax analysis unit 103 performs syntax analysis on the sentence input in step S201. Then, in step S204, reading information of the input sentence is generated based on the result of the syntax analysis. Further, in step S205, the semantic analysis unit 108 performs semantic analysis based on the result of the syntax analysis, and in step S206, the mood information extracting unit 109 extracts mood information based on the result of the semantic analysis.

For example, in the sentence generation example in FIG. 5, "according to the Japan Meteorological Agency, it is said that the rainy season began last week."
From the semantic analysis, the mood information of "hearing" is obtained from the underlined expressions "according to the Japan Meteorological Agency" and "yes". Similarly, for example, in the sentence generation example of FIG.
In the case of "Unfortunately, this week, the low pressure was sitting down and it was raining all the time." In the case of semantic analysis, "Unfortunately" was "underlined" by semantic analysis.
Mood information "surprise" is obtained from the expression. Similarly, in the sentence generation example in FIG. 7, “But, since the high pressure is overhanging from the continent, it will surely be a good weather on Saturday”, the semantic analysis shows that the underlined “ From the expressions "" and "", the mood information of "conviction" is obtained.

Next, in step S207, animation of a face (expression and lips) is generated from the reading information and mood information using the animation dictionary 112, and in step S208, synthesized speech is generated from the reading information.
Then, in step S209, speech synthesis is output,
In step S210, an animation is displayed according to the output of the synthesized voice. Steps S209 and S210 are synchronized so that there is no deviation between the sound and the shape of the lips.

For example, in the example of FIG. 5, facial expression data of "displacement of the eyes" is obtained from the mood information of "hearing" from the animation dictionary 112. Will move your lips. When the animation display with the output of the composition is completed as described above, the process returns to step S202 to process the unprocessed sentence. When there is no unprocessed sentence for the input sentence, the process proceeds from step S202 to step S2.
Return to 01 and prepare for the next sentence input.

In the example of FIG. 6, since the mood information is "unexpected", facial expression data of "confused" is obtained from the animation dictionary 112, and a facial expression indicating "confused" is drawn. In the example of FIG. 7, since the mood information is "convinced", the expression "smile" is obtained from the animation dictionary 112, and the expression indicating "smile" is drawn.

As described above, according to the first embodiment, the sentence to be read is semantically analyzed to extract the mood information of the speaker, and based on the mood information of the speaker, the gaze position, facial expression, etc. of the animation of the face being uttered. Is controlled.
Therefore, it is possible to provide a text-to-speech apparatus capable of appropriately switching expressions according to the content of the text to be read.

[Second Embodiment] In the first embodiment, one facial expression is selected for the mood of the semantic analysis result. However, a plurality of facial expressions are selected for one mood. By registering and changing speakers,
Different facial expressions may be selected. For example, when the mood is “convinced”, the configuration may be such that if the speaker is “female”, “smile” is selected, and if the speaker is male, “sincere” expression is selected.

In addition to the mood information, analysis of treatment expressions is also performed.
For an expression that is incongruous like this, a selection may be made from a set of expressions that are “incongruous”.

In the case of a mood of "skepticism" as well as facial expressions such as eyes and mouth, an expression method such as putting a question mark on the head of the person in the animation may be used.

If the system does not extract the mood information by analyzing the sentence to be read but outputs the speech by the system by generating the sentence from the semantic expression, the mood information may be directly extracted from the semantic expression. .

The present invention may be applied to a system constituted by a plurality of devices or to an apparatus constituted by a single device. A recording medium storing software program codes for realizing the functions of the above-described embodiments is supplied to a system or an apparatus, and a computer (or CPU or MPU) of the system or apparatus executes the program code stored in the recording medium. Needless to say, this can also be achieved by executing the reading.

In this case, the program code itself read from the recording medium implements the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

As a recording medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, C
DR, magnetic tape, nonvolatile memory card, ROM
Etc. can be used. The functions of the above-described embodiment are realized by executing the program codes read by the computer, and the OS or the like running on the computer performs actual processing based on the instructions of the program codes. It goes without saying that a part or all of the above is performed, and the processing realizes the functions of the above-described embodiments.

Further, after the program code read from the recording medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instruction of the program code, The CPU provided in the function expansion board or function expansion unit performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

[0033]

As described above, according to the present invention, when a text is read aloud, animation control can be performed according to the content of the text to be read. For this reason, it is possible to automatically generate an animation of a facial expression corresponding to the content of the sentence to be read.

[0034]

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of a text-to-speech apparatus according to a first embodiment.

FIG. 2 is a block diagram illustrating a functional configuration of the text-to-speech apparatus according to the first embodiment;

FIG. 3 is an animation dictionary 112 according to the embodiment.
FIG. 3 is a diagram showing an example of the configuration of FIG.

FIG. 4 is a flowchart illustrating a procedure of a document reading process according to the embodiment;

FIG. 5 is a diagram illustrating an example of a read-aloud sentence and animation control of a face according to the embodiment.

FIG. 6 is a diagram illustrating an example of a read-out sentence and animation control of a face according to the embodiment.

FIG. 7 is a diagram illustrating an example of a read-aloud sentence and animation control of a face according to the embodiment.

Claims (15)

[Claims] A determination unit configured to analyze a sentence to be read and determine a mood indicating an atmosphere of the sentence; and a display control unit configured to control an animation display based on the mood determined by the determination unit. An animation control device characterized by the above-mentioned. 2. The animation control device according to claim 1, wherein said display control means further controls animation display based on reading information corresponding to the sentence to be read. 3. An animation dictionary for storing mood data representing a mood of a speaker and facial expression data representing various facial expressions in association with each other, and facial expression data from the animation dictionary based on the mood determined by the determining means. 2. The animation control device according to claim 1, further comprising: an acquisition unit configured to acquire, wherein the display control unit controls the animation display of the face based on the expression data acquired by the acquisition unit. 3. 4. The animation dictionary further stores reading information and motion data representing various mouth movements in association with each other, and generates reading information by analyzing the sentence to be read out, and the generating means. And a second acquisition unit for acquiring the corresponding mouth movement data from the animation dictionary based on the reading information generated in the step (a), wherein the display control unit includes the expression data acquired by the acquisition unit and the second expression data. The animation control device according to claim 3, wherein the animation display of the face is controlled based on the motion data acquired by the acquisition means. 5. The animation control device according to claim 1, wherein the expression data includes data indicating an eye movement. 6. The animation control device according to claim 1, wherein the expression data includes a graphic to be added to the animation of the face. 7. A text-to-speech apparatus comprising: the animation control device according to claim 1; and an output unit that outputs a voice based on the reading information generated by the generation unit. . 8. The text-to-speech apparatus according to claim 7, wherein the display means controls the animation in synchronization with the audio output by the output means. 9. A method for analyzing a sentence to be read out and determining a mood indicating an atmosphere of the sentence, and a display controlling step for controlling an animation display based on the mood determined in the determining step. An animation control method characterized by the following. 10. The animation control method according to claim 9, wherein the display control step further controls an animation display based on reading information corresponding to the sentence to be read aloud. 11. An acquisition step of acquiring expression data from an animation dictionary that stores mood data representing a mood of a speaker and expression data representing various expressions in association with each other based on the mood determined in the determination step. The animation control method according to claim 9, further comprising: controlling the display of the face animation based on the expression data acquired in the acquiring step. 12. The animation dictionary further stores reading information and motion data representing various mouth movements in association with each other, and generates a reading information by analyzing the sentence to be read, and the generating step. And a second acquisition step of acquiring corresponding mouth movement data from the animation dictionary based on the reading information generated in the step (a). The display control step further comprises: the facial expression data acquired in the acquisition step; The animation control method according to claim 11, wherein the animation display of the face is controlled based on the motion data acquired in the acquiring step. 13. The animation control method according to claim 9, wherein the expression data includes data indicating an eye movement. 14. The animation control method according to claim 9, wherein the expression data includes a figure to be added to the animation of the face. 15. A computer readable memory storing a control program for controlling an animation display, the control program analyzing a sentence to be read out and determining a mood indicating an atmosphere of the sentence. A computer readable memory comprising: a code; and a code of a display control step of controlling an animation display based on the mood determined in the determining step.