JPH07104962A - Dialogue system - Google Patents

Abstract

(57) [Summary] In a dialogue system having a plurality of input means, the operation guidance of the input means is not redundant, and only the guidance that the user really needs can be output. [Purpose] The dialogue management unit 13 determines whether the voice input using the transmitter (microphone) of the handset 18 or the touch input using the touch panel 12a of the touch panel unit 12 is performed. If so, the next input is predicted to be voice input, and the input mode is set to the voice input mode regardless of the current input mode, and a guidance message prompting the user to speak is output on the display 11 or the like. If it is a touch input, the next input is predicted to be a touch input, and the input mode is set to the touch input mode regardless of the current input mode, and the touch input is made to the user. A guidance message prompting the user to output is configured to be output from the output means such as the display 11.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a dialogue system represented by a multi-modal dialogue information guide apparatus which has a plurality of dialogue means with a user and enables complicated operations to be used more easily. Relates to a dialogue management method for providing operation guidance so that the plurality of dialogue means can be used efficiently.

[0002]

2. Description of the Related Art In a conventional dialogue system represented by a multi-modal dialogue information guide device provided with a plurality of dialogue means with a user, all the operation guidance of a plurality of dialogue means is given to the user. In general, the method of guiding the input means or the method of not performing guidance other than some of the input means is used.

[0003]

However, in the above-mentioned conventional dialogue system, in the method of guiding the user to all the input means, the operation explanation becomes long and redundant, and the user's attention is slowed down. There was a problem of letting it.

On the other hand, in the system which does not guide other than some input means, there is a problem that the user may not even notice the existence of other input means.
The present invention has been made in consideration of such a point, and an object thereof is to provide guidance to a plurality of input means, basically, by providing only the input means selected by the user, thereby making it redundant. It is an object of the present invention to provide an interactive system that can output only the guidance that the user wants to obtain, not the guidance.

[0005]

SUMMARY OF THE INVENTION The present invention provides a plurality of inputs in a dialogue system having a plurality of input means of different types for a user and an output means for providing a message to the user. An input mode corresponding to the input means when one of the means is selected and an input operation is performed by the user, or when the user causes an operation to select and use one of the plurality of input means Is set, and a dialogue management means for providing a guidance message peculiar to the input mode to the user through the output means is provided.

In the above configuration, the input means that the user selects from a plurality of input means and performs the input operation, or the target input means when the operation of selecting the input means used for the input operation occurs. By detecting, the guidance message most necessary for the user to use the system is provided to the user as the guidance message that is suitable for the detected input means. In this way, by providing the user with only an accurate guidance message that is not redundant, it becomes possible to construct a human interface that is easier for the user to use.

Further, the present invention is characterized in that, in the normal state, the dialogue management means is configured to accept an input from the input means irrelevant to the input mode set at that time. .

With such a configuration, even if the user changes the input means, it is possible to cope with the change and the usability is improved. Further, according to the present invention, when the input from the input means is a command for controlling the system, a guidance message specific to the input mode at that time is provided to the user regardless of the input means. Characterize.

As described above, when a control command (information indicating) for calling a specific function such as "re-input" is input, it may have a meaning different from normal operation in the mind of the user. Paying attention to the fact that there are many, the change to the input mode corresponding to the input means used for the input and the output of the guide message suitable for the input means are not performed. This will not confuse the user.

Further, according to the present invention, when the re-input from the predetermined input means continues for a predetermined number of times, the input mode is set to the specific mode in which the input from the input means is not accepted. It is characterized in that it is set again corresponding to other input means, and a guidance message specific to this reset input mode is provided to the user.

In this way, in an abnormal state in which re-input from the predetermined input means continues for a predetermined number of times, the input from the input means is accepted regardless of the input mode, that is, regardless of the output message. If this is set, incorrect input may be made.Therefore, set a specific mode that does not accept input from the input means and set the input mode to one that corresponds to another predetermined input means. Then, by providing the user with a guidance message specific to this reset input mode, efficient input can be achieved without waste.

Further, according to the present invention, when a transition is made to an initial state of the system, or in a predetermined system state,
The specific mode is released. By doing so, it becomes possible to accept an input from the input means that has not been accepted until then, and it is possible to improve usability while achieving efficient and efficient input.

Further, according to the present invention, when the user starts to use the system, the user selects an input means to be used by the user from a plurality of input means, and thereafter, a guidance message specific to the selected input means is displayed. It is characterized by being provided to users.

As described above, when the user first selects the input means, thereafter, only the message that matches the input means selected by the user is output, so that the operation guide that the user really needs. It will be possible to give only.

Further, the present invention is characterized in that information indicating the currently selected input means is provided to the user. With such a configuration, the user can always know which of the plurality of input means is selected.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following is a description of a case where one embodiment of the present invention is applied to a multi-modal dialogue system having two input means for voice input and touch input (touch panel input).
A description will be given with reference to the drawings.

FIG. 1 is a block diagram showing the overall configuration of the multimodal dialogue system according to the embodiment. The multimodal dialog system shown in FIG.
Touch panel unit 12, dialogue management unit 13, voice recognition unit 14, user monitoring unit 15, voice synthesis unit 1
6, speaker (external speaker) 17, handset 1
8 and a photoelectric sensor 19.

The display 11 is a display output means such as a CRT display or a liquid crystal display which forms an interface with the user. Touch panel unit 12
Provides a touch input means for the display 1
The touch panel 12a is provided so as to be superimposed on the display surface of No. 1, and the touch panel control unit 12b for inputting position information touched on the touch panel 12a and the like.

The dialogue management unit 13 is for managing the dialogue with the user through the display 11 and the like. The dialogue management unit 13 controls the input from the touch panel unit 12 and the input from the voice recognition unit 14 (corresponding to the voice input from the handset 18), and the input from the touch panel unit 12 or the voice recognition unit 14 ( Of the recognition result for the voice of the user)
When an input is made, basically, a guide message suitable for the input means is displayed on the display 11, for example, and voice output from the speaker 17 and the handset 18 through the voice synthesis unit 16. In addition, when there is an input from the voice recognition unit 14, the dialogue management unit 13 determines whether or not it is a recognition result for a voice uttered by a user (a voice recognition result for ambient noise). The determination is made based on the monitoring result of the person monitoring unit 15.

The voice recognition unit 14 recognizes the voice input by the user through the transmitter (microphone) of the handset 18 and inputs it to the dialogue management section 13. User monitoring unit 15
Monitors the state of the photoelectric sensor 19 provided on the receiver side of the handset 18 to detect that the user put the receiver of the handset 18 on his / her ear.

The voice synthesis unit 16 includes a dialogue management unit 13.
Synthesize the guidance message passed from to output the voice.
The speaker 17 is a voice output unit for outputting the voice synthesized by the voice synthesis unit 16 to the outside.

The handset 18 has a transmitter (microphone) as a voice input means and a receiver as a voice output means. The photoelectric sensor 19 is provided on the receiver side of the handset 18 and is used to detect that the receiver of the handset 18 is applied to the user's ear.

FIG. 2 is a block diagram showing the internal configuration of the dialogue management unit 13 in FIG. 1 in association with its peripheral configuration. As shown in the figure, the dialogue management unit 13 includes an input determination unit 13
1, a message composition unit 132 and a message database (hereinafter referred to as message DB) 133.

The input judging unit 131 is the voice recognition unit 1.
4 or the touch panel unit 12 is used to determine which input has been made. When determining the input from the voice recognition unit 14, the input determination unit 131 also determines whether or not the input is based on the voice input from the user according to the monitoring result of the user monitoring unit 15.

The message synthesizing unit 132 is the input judging unit 1.
According to the judgment result of 31, the guidance message suitable for the input means that the user is likely to use next is displayed as the message D.
Extracted from B133, the speech synthesis unit 16 and FIG.
It is output to the display 11 inside.

The message DB 133 is for preliminarily storing various guidance messages. Next, the operation of the configuration of FIGS. 1 and 2 will be described with reference to the flowcharts of FIGS.

In the interactive system of this embodiment, the voice input mode indicating that the next user operation is likely to be a voice input and the touch input mode indicating that it is likely to be a touch input. And are prepared, and when the user starts using (initial state), for example, the voice input mode is set.

First, the input judgment unit 131 in the dialogue management unit 13
Indicates that the system is not accepting voice input (regardless of the current mode).
It is checked whether or not (step S1). Here, the system is not in the voice input prohibited state, that is, in the voice input permitted state (voice input permitted mode). In this case, the input determination unit 131 in the dialogue management unit 13 does nothing and activates the message synthesis unit 132.

Then, the message synthesizing unit 132 extracts from the message DB 133 a guidance message suitable for the voice input mode which is the input mode at that time, that is, a guidance message for prompting the user to speak (voice input guidance message). , And the voice is output from the speaker 17 and the handset of the handset 18 through the voice synthesis unit 16 (step S2).
A screen display example on the display 11 at this time is shown in FIG.
Shown in.

Here, the user generally picks up the handset 18, brings the transmitter close to his mouth, puts the receiver on his ear, and follows the voice input guidance message output to the transmitter (microphone). And input necessary information by voice. At this time, the light input to the photoelectric sensor 19 provided on the handset 18 side of the handset 18 is
Since the light is blocked by the user's ear and the output of the photoelectric sensor 19 becomes less than or equal to a predetermined threshold value, the user monitoring unit 15 detects that the user has performed an operation of performing voice input using the handset 18.

The voice input by the user from the transmitter (microphone) of the handset 18 is recognized by the voice recognition unit 14, and the recognition result is sent to the input determination unit 131 in the dialogue management unit 13.

The user can also perform touch input (touch panel input) using the touch panel 12a of the touch panel unit 12 regardless of the output voice input guidance message.

When the input judging section 131 in the voice recognition unit 14 activates the message synthesizing section 132 to output a message as described above, it waits for an input from the voice recognition unit 14 or the touch panel unit 12. Then, when an input is made from any of the input means, the input determination unit 131 inputs the input information (step S3), and whether the input is from the voice recognition unit 14 or the touch panel unit 12, that is, It is determined whether the input is voice input or touch input (step S4).

In case of voice input, the input judging section 13
1 checks whether the system is in a voice input prohibited state (step S5). If the voice input is prohibited, the input determining unit 131 prompts the message synthesizing unit 132 to prompt the user for touch input (since the mode is changed to the touch input mode as described later). A guidance message (touch input guidance message) is output from the display 11, the speaker 17, and the handset of the handset 18 (step S2). In addition, the handset 18
The message output from the handset 18 does not have to be performed when the handset 18 is not placed on the ear. Further, when the handset of the handset 18 is put on the ear, the message output from the speaker 17 may not be performed.

On the other hand, if the voice input is not prohibited, the input judging section 131 checks whether or not the handset 18 is placed on the ear based on the monitoring result of the user monitoring section 15 ( Step S6).

If the handset of the handset 18 is put on the ear, the input judging section 131 determines that the input voice recognized by the voice recognition unit 14 is the voice uttered by the user, and It is predicted that input from the person will also be performed by voice input. in this case,
The input determination unit 131 sets the voice input mode regardless of the current input mode (here, the voice input mode is reset as it is) (step S7), and further displays the input content (recognition result of the voice recognition unit 14). After performing a series of system operations (step S8) such as output, voice output, and predetermined processing for input contents, step S
Return to 2.

Further, if the handset of the handset 18 is put on the ear. The input determination unit 131 determines that the input voice recognized by the voice recognition unit 14 is ambient noise and invalidates the voice input (step S9),
Without changing the current input mode (here, the voice input mode) (step S10), the process returns to step S2.

On the other hand, regardless of the current input mode (here, the voice input mode), as a result of the user performing input using the touch panel 12a of the touch panel unit 12,
When the touch input is determined in step S4, the input determination unit 131 determines that a system control command (special command) for calling a specific function, such as “re-input” indicating that the input is redone (cancel). It is checked whether or not it has been input (step S11). This special command is input, for example, when the user touches a special button prepared on the display screen with a finger. The re-input selection button 62 in the display screen shown in FIG. 6 is for inputting a "re-input" command.

If the touch input is not the special command input, the input determination unit 131 predicts that the next input from the user will also be made by the touch input. In this case, the input determination unit 131 sets the touch input mode irrespective of the current input mode (here, the voice input mode is switched to the touch input mode) (step S12), and the display output of the touch input content is further performed. other,
After performing a series of system operations (step S21) such as predetermined processing for the input content, step S1
Return to. In this step S1, since it is determined that the voice input is not prohibited, the process proceeds to step S2, and a guidance message suitable for the current touch input mode, that is, a touch input guidance message for prompting the user to make a touch input is a message. The data is extracted from the DB 133, displayed on the display 11, and output from the speaker 17 (and the handset of the handset 18) through the voice synthesis unit 16 (step S2).

On the other hand, if a special command is input by touch input, the input determination unit 131 determines that the re-input selection button 62 in FIG.
It is checked whether or not the "re-entry" command has been input (step S13). If it is a special command input other than "re-entry", such a special command input operation is normal to the user's consciousness. Since it often has a different meaning from the operation, it is assumed that the input corresponding to the current input mode (not necessarily touch input) is likely to be performed next. The voice input mode) is not changed (step S14), the system operation corresponding to the input command is performed (step S15), and the process returns to step 2. Further, if the "re-input" command is touch-input, the input determination unit 131 checks whether or not the current input mode is the voice input mode (step S1).
6), if it is not the voice input mode, step S as it is
Proceed to 14. On the other hand, if the input mode is the voice input mode, the input determination unit 131 determines that the “re-input” (input cancellation) is caused by an erroneous recognition (of the voice recognition unit 14) for the voice input, and The number of times the "input" command is continuously input is compared with a predetermined limit number of times (step S17).

Here, if the number of continuous inputs of the "re-input" command is within the limited number of times, the input determination unit 131 proceeds directly to step S14. On the contrary, if the number of continuous input of the “re-input” command exceeds the limit number of times, the input determination unit 131 cannot perform an efficient dialogue with the user in the voice input from the user. When it is determined that it is difficult, the system is set to the voice input prohibition state (voice input prohibition mode) in which the voice input from the user is not accepted (step S18). Then, the input determination unit 131 switches the input mode from the current voice input mode to the touch input mode (step S19), and further, the “re-input” command and the system operation corresponding to the screen state at that time (for example, the previous screen). (Operation to return to the state) (step S2
0), and returns to step S2. In this step S2, the guidance message suitable for the touch input mode switched in step S19 is output.

In the present embodiment, when voice input is performed by the user in the state where the voice input is prohibited and the touch input mode is switched as described above, the fact is detected in step S5. The voice input is not accepted, and the process directly returns to step S2 and a guidance message for prompting the user to input a touch is output.

On the other hand, when the touch input is performed in the state where the voice input is set to the prohibition state and switched to the touch input mode, the touch input is performed in the voice input mode described above. This is the same as in the case of the above, and the processing after step S11 is executed.

Here, when a command other than the special command is input by touch input, the input determination unit 131 sets the touch input mode as described above (here, the current touch input mode is reset as it is) (step S12), and system operation according to the input content (step S2)
After performing 1), the process returns to step S1 as in the initial state (instead of step S2). The input determination unit 131 checks in step S1 whether or not it is in the voice input prohibited state, and if it is in the voice input prohibited state as in this example, cancels the voice input prohibited state (voice input prohibited mode). Then, it switches to the voice input permission state (voice input permission mode), and proceeds to step S2. As a result, voice input can be accepted regardless of the current input mode (here, the touch input mode).

As described above, in this embodiment, when a command other than the special command is input by touch input, in addition to the above-mentioned start of use by the user (initial state), step S1 is performed.
If the voice input is prohibited at this time, the prohibition is automatically canceled, and the voice input can be used thereafter. This is because the input of anything other than a special command by touch input in the voice input prohibited state means "re-input" that caused the voice input prohibited state.
This is because it can be considered that the input of the information that has caused the above is correctly performed by touch input.

The condition for canceling the voice input prohibited state is an example, and the voice input prohibited state may be canceled when a special command other than "re-input" is input. Further, the voice input prohibition state may be canceled only when the screen state has reached a predetermined screen state.

In the above embodiment, the input means used by the user is detected to predict the input means to be used next, but the present invention is not limited to this. For example, in the initial state, an initial screen is displayed as shown in FIG. 5, and either one of the voice recognition selection button 51 and the touch panel selection button 52 provided on the screen (on the touch panel 12a corresponding to position)
When the user selectively touches, whether to use voice input (input by voice recognition according to) or touch input is selected in advance, and thereafter, the selected input method (input means) It is also possible to output an operation guidance message conforming to.

Further, in the above-mentioned embodiment, the user can judge only what the input means corresponding to the current input mode, that is, the currently selected input means, by the guide message. A mark or the like indicating the input means may be displayed in a certain area on the display screen. FIG. 6 shows an example of such a screen. In a partial area of the screen (here, a partial area of the guidance message),
The input selection icon 61 is displayed. The user can easily visually recognize the input means currently selected by the pattern of the icon 61.

Further, in the above-mentioned embodiment, when the number of continuous input of the "re-input" command by touch input exceeds the limit number, even if the voice input that is the cause is repeated a corresponding number of times, that is, the voice input is repeated. Even if the input operation (re-input) by the input is repeated a corresponding number of times, it is determined that the input was not normally performed, and it is set to the voice input prohibited state because it is useless for the further voice input dialogue. However, it is not limited to this. For example, an environment monitoring unit that monitors ambient noise (separate from the user monitoring unit 15)
When the environment monitoring unit detects that ambient noise exceeds a certain fixed level (first fixed level) for a predetermined period (first predetermined period) or more, the input voice from the user Is not correctly recognized by the voice recognition unit 14, and the dialogue management unit 13 (in the input determination unit 13
In 1), the system may be set to the voice input prohibited state and the input mode may be set to the touch input mode.

By doing so, it is possible to prevent in advance the problem that the recognition rate for the input voice is lowered and the input efficiency is lowered due to the influence of the ambient noise. The condition for canceling the voice input prohibition state set according to the state of the ambient noise is that the ambient noise does not satisfy the condition for setting the voice input prohibition state, or the ambient noise is for a predetermined period (second predetermined period). It may be used that it is below a certain level (second certain level).

[0051]

As described above in detail, according to the present invention,
Since the user is required to operate the system, the most necessary operation guide is determined by guessing the input means to be operated next by the user, so that only accurate non-redundant guide messages are given to the user. It is possible to construct a human interface that is easier for users to use.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a multimodal dialogue system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing the internal configuration of a dialogue management unit 13 in FIG. 1 in association with its peripheral configuration.

FIG. 3 is a diagram showing a part of a flowchart for explaining the operation of the embodiment.

FIG. 4 is a view showing the rest of the flowchart for explaining the operation of the embodiment.

FIG. 5 is a diagram showing an example of a display screen having a selection button for allowing a user to perform an input means selection operation in advance.

FIG. 6 is a diagram showing an example of a display screen having an icon for displaying the currently selected input means.

[Explanation of symbols]

11 ... Display, 12 ... Touch panel unit, 1
2a ... touch panel, 12b ... touch panel control unit, 1
3 ... Dialog management unit, 14 ... Voice recognition unit, 15 ... User monitoring unit, 16 ... Voice synthesis unit, 17 ... Speaker,
18 ... Handset, 19 ... Photoelectric sensor, 131 ... Input determination section, 132 ... Message synthesis section, 133 ... Message DB, 51 ... Voice recognition selection button, 52 ... Touch panel selection button, 61 ... Input selection icon, 62 ... Re Input selection button.

Claims (8)

[Claims] 1. A dialog system having a plurality of input means of different types for a user and an output means for providing a message to the user, wherein one of the plurality of input means is selected and the user is selected. A dialogue management means for setting an input mode corresponding to the selected input means and providing a guide message specific to the input mode to the user through the output means when the input operation is performed by A dialogue system characterized by. 2. A dialogue system having a plurality of input means of different types for the user and an output means for providing a message to the user, wherein the user monitors the user of the dialogue system. When it is detected by the monitoring means and the user monitoring means that the user selects and uses one of the plurality of input means, an input mode corresponding to the input means is set. And a dialogue management means for providing a guidance message peculiar to the input mode to the user through the output means. 3. The dialogue management means controls acceptance of inputs from the plurality of input means, and in the normal state, also accepts inputs from the input means irrelevant to the input mode set at that time. The dialogue system according to claim 1 or 2, characterized in that. 4. The dialogue management means, when the input from the input means is a command for controlling the system,
4. The interactive system according to claim 3, wherein a guidance message peculiar to the input mode at that time is provided to the user through the output means regardless of the input means. 5. The dialogue management means sets a specific mode in which input from the input means is not accepted when re-input from the predetermined input means continues for a predetermined number of times, and the input mode is set. 5. The guide message specific to the preset input mode is provided again to the user through the output means, and the guide message specific to the preset input mode is provided to the user. Dialogue system. 6. The dialog management means cancels the specific mode when the system enters an initial state of the system or in a predetermined system state, and the re-input is input from the input means for a predetermined number of consecutive times. 6. The dialogue system according to claim 5, wherein the dialogue system is also accepted. 7. An interactive system having a plurality of input means of different types for a user and an output means for providing a message to the user, wherein the plurality of the plurality of input means are provided when the user starts using the system. A dialogue management means is provided for selecting an input means to be used by the user from the input means, and thereafter providing a guidance message specific to the selected input means to the user through the output means. Dialogue system to do. 8. The dialogue according to claim 1, wherein the dialogue management means provides the user with information indicating the currently selected input means through the output means. system.