JPH07129594A - Automatic interpreter system - Google Patents

Abstract

(57) [Abstract] [Purpose] To provide an automatic interpretation system capable of coping with various situations occurring during use such as a voice recognition error and a translation error. [Structure] A voice input unit 1 for speech input, an instruction input unit 2 for inputting instruction information, a voice output unit 3, and a display unit 4
And a plurality of input / output means to which attributes indicating different kinds of languages are attached, and utterances from a voice input unit included in one input / output means are voice-recognized and coded based on the attributes. Voice recognition means 10 and bidirectional automatic translation means 12 for translating the code into a code corresponding to a language of another attribute.
And a voice generation means 11 for converting the code into a voice, and a dialogue for confirming the result with the operator of the one input / output means, that is, the speaker when the recognition result or the translation result of the utterance cannot be specified. In addition to the above, the dialogue means 13 is provided for making a dialogue regarding the dialogue with an operator of another input / output means, that is, a listener while the dialogue is being performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic interpreting system that enables speakers of different languages to interact with each other in their own languages.

[0002]

2. Description of the Related Art Conventionally, as a system in which speakers of different languages interact, like a portable translator, it has a simple bilingual dictionary of two or more languages, a keyboard, and a liquid crystal display unit of about one line. There was only one that was based on displaying multiple words entered from the keyboard in the same order as they were in the dictionary. Although it was named a translator, it did not perform grammatical analysis or semantic analysis of the language, so correct translations were not output and it was just a dictionary lookup tool.

With this device, for example, when he or she inputs to go to a restaurant, only words such as HE RESTAURANT GO appear. Such a device, as will be readily seen, is completely impractical for complex sentences. Also, even in simple sentences, function words such as particles cannot describe their meaning in the dictionary,
Sometimes the meaning is completely opposite. For example,
If you enter the meaning of "I like Taro Hanako." As follows, you will get Taro Hanako like Taro Hanako like, and you will not know which one you like, and in some cases you will misunderstand that Taro likes Hanako. There is a possibility that it will be done.

On the other hand, as a full-fledged automatic translation system, a machine translation system that performs grammatical analysis and semantic analysis of a source language and has a process of generating a target language has been put to practical use in the field of document translation.

However, these are exclusively used for document translation and do not serve as a device for dialogue. Typically, it is not configured to translate two or more languages simultaneously, and even if it has a bidirectional translation function,
It was necessary to complete the translation in one translation direction and then read the translation program in the reverse direction.

[0006] Further, in the conventional translator, the original sentence is input by the keyboard, and it cannot be input by voice. On the other hand, although a voice interpretation system is also being researched, at present, speech recognition is performed on the uttered voice, and the result is simply passed through a machine translation unit, and the translated result is converted into a voice by a voice generation unit. However, there is a problem in that no consideration is given to practical problems such as how to deal with voice recognition errors and translation errors.

[0007]

As described above, in the prior art,
A complete interpreter that allows bilingual speakers to talk freely by voice has not been realized. The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an automatic interpreter system capable of coping with various situations that occur during use such as voice recognition error and translation error.

[0008]

In the automatic interpreting system according to the present invention, a voice input section for inputting utterances, an instruction input section for inputting instruction information, a given voice signal is converted into voice. A plurality of input / output means having an audio output section for outputting and a display section for displaying given information, and provided with attributes indicating different kinds of languages, and one input / output of the plurality of input / output means Voice recognition means for voice-recognizing an utterance input from the voice input unit included in the means based on the attribute given to the one input / output means, and generating a corresponding code or code string; Bidirectional automatic translation means for translating a code string into a code or a code string corresponding to a language of the type indicated by the attribute assigned to the other input / output means of the one input / output means, and the code or A voice generation means for converting a code string into a voice signal to be given to the voice output part, and a case where the voice recognition means cannot specify a recognition result for the input utterance or the bidirectional automatic translation means When the translation result for the code or code string generated by the voice recognition means cannot be specified, a confirmation process for confirming the recognition or translation result is included in the one input / output means. Section and the instruction input section and at least one of the voice output section and the display section, and the display section and the voice included in the other input / output unit while the confirmation processing is performed. It is characterized in that at least one of the input sections is provided with a dialogue means for outputting information regarding the confirmation processing.

Further, preferably, the utterance input from the voice input unit is given to the voice recognition unit, and the utterance is output as it is from a voice output unit included in the other input / output unit. good.

Further, the voice recognition means uses information for specifying at least one character type in the utterance given from the instruction input unit when the utterance is input from the voice input unit, The speech may be recognized by voice.

Further, it is preferable that the speaker uses at least one of the instruction input unit and the voice input unit to notify the automatic interpreter system that his / her utterance is completed. Further, the speaker may input the utterance by using both the instruction input unit and the voice input unit.

[0012]

As a result, according to the present invention (Claim 1), the type of language to be processed is predetermined for each of the two or more input / output means, and the voice input of any one of the input / output means is performed. When the utterance is input from the section, the voice recognition means recognizes the utterance as a language of the type indicated by the attribute, the bidirectional automatic translation means translates the recognition result into another language, and the voice generation means recognizes this utterance. The translation result is converted into a voice signal, and the voice output unit corresponding to the translated language converts the voice signal into voice and outputs the voice. Whether the speech recognition of the speaker is failed as a result of speech recognition, or the speech recognition is processed as successful and fails at the translation stage, recognition or translation is performed through the dialogue unit. While not interacting with the speaker who is the operator of the one input / output means to correct the unsuccessful portion, the dialog unit interacts with the speaker for the correction while the other input / output means Silent time is given to listeners who are one or more listeners who are operators by notifying them that they will wait for a while, letting listeners know the situation one by one, and becoming a dialogue partner who answers inquiries. Prevent from causing.

In addition to the use of voice, the dialogue means can use characters and symbols using the display unit and the instruction input unit, so that correction information based only on voice will fail to recognize voice again. It is possible to prevent re-correction due to.

According to the present invention (Claim 2), the utterance input from the voice input unit included in the one input / output unit is directly output from the voice output unit included in the other input / output unit. Let Therefore, the listener who is the operator of the other input / output unit can monitor the utterance situation of the dialogue partner who is the operator of the one input / output unit, the real voice and the background situation of the other party.

That is, if the voice-interpreted response is a synthesized voice by the voice generation unit, information about the utterance partner cannot be obtained, and in the worst case, the voice of a female speaker may be synthesized with a male voice. However, by the above, information such as the other's gender, age, focus of conversation by intonation, feelings etc. can be obtained by the other person's real voice, and whether the other person is alone, talking with someone etc. You can get information about the situation, such as the background sound of the other party's place, the situation where the other person is placed like a normal telephone.

On the other hand, since the conversation between the conversation partner and the automatic interpretation system can be monitored during the operation of the dialogue unit, when the waiting time of the listener becomes long, the conversation between the speaker and the response unit can be monitored. Even if you don't understand the meaning, it can be useful for situational awareness.

Further, according to the present invention (Claim 3), at the time of utterance input, the speaker gives information specifying at least one character type in the utterance to the system from the instruction input unit, and the voice The recognition means uses this information to recognize the speech by voice.

Therefore, for example, the utterance is voice-recognized by using the linguistic information which is difficult to convey only by the voice such as distinction between uppercase or lowercase in English, common noun or proper noun in Japanese, etc. The recognition performance is improved.

[0019]

Embodiments will be described below with reference to the drawings. FIG. 1 is a schematic configuration diagram showing an automatic interpretation system according to an embodiment of the present invention. This automatic interpreter system is for interpreting between two speakers A and B who speak in different languages, and can be applied to a system such as an automatic translation telephone or a simultaneous interpreter.

As shown in the figure, the automatic interpreter system includes a display unit 1, an input unit 2, a microphone 3, and a first input / output unit for a speaker A for a speaker A, a display unit 5, an input unit 6, and a microphone. Speaker B consisting of 7 and speaker 8
A second input / output unit, a control unit 9, a voice processing unit 20, and an automatic interpreting unit 30. Further, the voice processing unit 20 has a voice recognition unit 10 and a voice generation unit 11, and the automatic interpretation unit 1
Reference numeral 5 has a bidirectional automatic translation unit 12 and a dialogue unit 13 including a natural language understanding unit 14 and a natural language generation unit 15.

The type of language to be used is determined in advance for each of the first input / output unit and the second input / output unit. Here, it is assumed that an attribute indicating the type of language is given to each input / output unit.

The display units 1 and 5 are for displaying visible information such as characters and symbols in order to convey information to the user of the system, and are composed of a liquid crystal panel or the like. The input units 2 and 6 are for inputting instruction information relating to utterance or confirmation processing to be described later other than voice such as characters and symbols, and are composed of, for example, a keyboard mouse and a touch panel.

The microphones 3 and 7 are for inputting the voice utterance of the speaker. The speakers 4 and 8 utter the output from the voice generation unit 11. The control unit 9 controls the operation of the entire system, and manages and controls the translation direction in two types of languages, the direction of various information flows, the address of information, and the like.

The voice recognition unit 10 includes microphones 3 and 7.
The voice input from is recognized based on the attribute. The voice generation unit 11 converts the output from the automatic translation unit 12 and the output from the dialogue unit 13 into voice.

The bidirectional automatic translation unit 12 is a voice recognition unit 10.
Translates the utterance of the speaker transmitted from the dialogue partner language. The dialogue unit 13 uses the natural language understanding unit 14 and the natural language generation unit 15 when the confirmation or the voice input is required again for the speaker such as when there is ambiguity in the recognition result or the translation result. Interacts with the speaker while the system interacts with the listener during this interaction. Input and output for this dialogue are also carried through the same route as the bidirectional automatic translation unit 12.

The natural language understanding unit 14 determines that the utterance of the speaker is
In the case of the input for disambiguation, the sentence / natural meaning interpretation is performed on the sentence input in natural language, and the instruction of the speaker is interpreted.

The natural language generating unit 15 is a speaker when the voice recognition result recognized by the voice recognizing unit 10 is ambiguous or when the bidirectional automatic translation unit 12 is ambiguous in the translation result. Create a sentence that informs you that there is ambiguity.

For example, if there is ambiguity in the voice recognition result, "There is ambiguity in the voice recognition. Please answer by the number which of the following two is correct. 1. I am sorry but 2. I am. I'm sorry. " The created sentence is output to the display unit on the speaker side and also sent to the voice generation unit 11 to be voiced and notified to the utterance.

When the speaker knows that there is ambiguity in the message output by voice from the speaker or the message displayed on the display unit, the speaker inputs the sentence to be resolved in natural language. For example, enter "No. 2 is the correct answer". This input is interpreted in natural language 14, and "I am gakusei desu" corresponding to the second sentence is selected.

FIG. 2 is a flow chart showing the operation of the automatic interpretation system. The operation of the automatic interpretation system will be described below with reference to FIG. Here, it is assumed that the speaker A is the speaker and the speaker B is the listener. Also, it is assumed that speaker A uses Japanese and speaker B uses English.

A) When the recognition and the interpretation are successful once, the processing flow is as follows. The utterance of the speaker A is first input through the microphone 3 (step 1).

The input voice utterance (for example, "I am a kid") is sent by the control unit 9 to the voice recognition unit 10, where it is coded based on the attribute (step 2).

The coded utterance (that is, a code or a code string) is sent to the automatic translation unit 12 via the control unit 9, and a sentence (for example, "I am a "student.") or the corresponding code or code string (step 4).

The translated result is sent again to the voice generation unit 11 through the control unit 9 and converted into voice (step 7). The control unit 9 controls the other party B
Is sent to the speaker 8 and output as voice (step 8).

B) Next, when it is determined that the proper recognition result by the voice recognition unit 10 is not obtained,
For example, a case where the recognition result cannot be specified because it cannot be recognized or because the recognition result is ambiguous will be described.

The utterance of the speaker A is input through the microphone 3 (step 1). The input voice utterance is sent to the voice recognition unit 10 by the control unit 9 and coded (step 2).

If it is determined that the proper recognition result is not obtained (step 3), the control unit 9 receives the voice recognition result and the supplementary information of the recognition result from the voice recognition unit 10, and A corresponding confirmation process is performed (step 6).

For example, as shown in FIG. 3, it is assumed that ambiguity has occurred as a recognition result for the utterance "I am a student". In Figure 3, "wa" in "I"
Either "wa" or "ka" cannot be recognized, and both are output.

The control unit 9 operates from the voice recognition unit 10 to 2 in FIG.
In addition to the two speech recognition results, the supplementary information of the recognition results, for example, "there is ambiguity", sends this information to the dialogue unit 13 and utters the natural language generation unit 15 which of "wa" and "ka" is correct. The person A is made to make an inquiry sentence, and the voice is generated by the voice generator 11, and the speaker 4
In addition to making an inquiry by voice, the same inquiry is displayed on the display unit 1 as the character code if necessary. In response to this inquiry, the speaker A utters “I am gakusei” again after paying attention to the pronunciation of “wa”, and tries the recognition process again (steps 1 to 3).

Then, the above processing loop is repeated until the utterance is correctly recognized. Further, in the present embodiment, in the case where the voice recognition result displayed on the display unit 1 is correct in response to the inquiry as in the above example, for example, the second voice recognition result of FIG. It is possible to check by inputting "I am sorry" by voice or by selecting from a keyboard etc. by the method as described above, and it is possible to omit the recognition process again.

On the other hand, while the confirmation process is being performed,
For the listener B who will be on standby, the dialogue unit 13 also performs standby processing (step 6). For example, a message such as "Please wait for a while because the arrival of the utterance from speaker A will be delayed" or "We are confirming the utterance from speaker A, please wait for a while." Avoiding this will prevent listener B from being irritated or anxious.
This guidance can be performed through the voice generation unit 11 using one or both of the speaker 8 and the display unit 5. These states are shown in FIG.

In addition to the above, it is also possible to inform the listener of the situation one by one and answer the inquiry of the listener so that the listener functions as a dialogue partner. Next, the utterance correctly coded as described above is sent to the automatic translation unit 12 through the control unit 9, and here the speaker B who is the other party.
Is translated into a sentence in the language used by (step 4).

The translated result is sent again to the voice generation unit 11 through the control unit 9 and converted into voice (step 7). The control unit 9 controls the other party B
Is sent to the speaker 8 and output as voice (step 8).

C) Next, when the speech recognition unit 10 cannot detect a recognition error and it is determined that the automatic translation unit 12 cannot obtain a proper translation result, for example, because the translation cannot be performed or the translation result is not obtained. The case where the translation result could not be specified due to ambiguity is explained.

The utterance of the speaker A is input through the microphone 3 (step 1). The input voice utterance is sent to the voice recognition unit 10 by the control unit 9 and coded therein (step 2).

The coded utterance is sent to the automatic translation unit 12 via the control unit 9 and is translated into a sentence in the language used by the speaker B who is the other party (step 4). When it is determined that a proper translation result has not been obtained (step 5), the control unit 9 causes the automatic translation unit 12
The auxiliary information of the translation result is received together with the automatic translation result, and the confirmation process is carried out accordingly (step 6).

For example, FIG. 5 shows a state in which the voice recognition unit 10 cannot detect a recognition error. If the voice recognition unit 10 determines that the voice recognition result “Katahagakuseisei” is correct, the recognition result is sent to the control unit 9.
The automatic translation unit 12 that receives this result from the control unit 9
"Katashi" cannot be found in the dictionary and is treated as an unknown word. The result is an incomplete translation. Various cases are conceivable, such as a case where no translation is possible, or a case where partial translation is not possible. This depends on the nature of the error in speech recognition and the design concept of the automatic translation unit.

Here, as an example, it is assumed that the result as shown in FIG. 5 is returned from the automatic translation unit 12 to the control unit 9.
The control unit 9 sends this result to the dialogue unit 13,
An appropriate inquiry sentence for correction is generated, and a dialogue for confirmation is made with the speaker A in the same manner as in the case of the voice recognition error described above.

In the same manner as in the case of the voice recognition error described above, a dialogue for waiting with the listener B is performed at the same time as the above dialogue. The result of correct translation in this manner is sent again to the voice generation unit 11 through the control unit 9 and converted into voice (step 7).

The voiced utterance is sent by the control unit 9 to the speaker 8 of the other party B and output as voice (step 8). As described above, in the automatic interpretation system according to the present embodiment, when there is any problem in the voice recognition result or the translation result, the conversation with the speaker for confirmation is made, and the confirmation is made to the waiting listener. Since a dialogue means for carrying out a dialogue is provided to convey information about the information, it becomes possible to deal with various problems actually caused by automatic translation by voice input, and it is possible to dramatically improve the practicality of the system. it can.

Here, although only voice is used as the utterance in the above, by using input from a keyboard or the like in addition to voice, it is difficult to input only voice input such as symbols, or what is used for voice input. It is possible to easily enter things that failed.

Further, the automatic interpreting system of this embodiment is 2
The human talkers A and B can be used while facing each other at the same place, or can be used for a dialogue with a remote party by connecting the control unit 9 to a public communication line. That is, when talking over a public communication line, the output from the speaker is synthetic voice, so information such as the age, sex, emotion, intonation of the other party, or whether the other party is alone, talking with someone However, there is a possibility that information about the situation of the other party such as the background sound of the other party's place may be missing, and the situation of the other party may not be known. However, the control unit 9 controls the microphone of the speaker A. It is possible to obtain the information that may be missing by arranging the real voice from 3 as it is to the listener B. Here, FIG. 6 shows a simplified path and a real voice path when the utterance of the speaker is translated, and FIG. 7 shows relative timings when the utterance of the speaker arrives at the listener. In each figure, t1 is the start time of speech. This real voice is immediately sent from the control unit 9 to the other party through the public communication line. This delay is about 1 second when passing through a communication satellite. That is, t2 = t1 + 1 sec. Further, t3 is composed of a time required for the speaker to speak, a time required for voice recognition, a time required for automatic translation, and a time required for inquiry / dialogue when an error occurs. Although it is difficult to specify this amount of time, it is assumed that there is no dialogue time for correcting an error, and if the utterance is short for about 10 words, it is about 30 seconds at maximum according to actual measurement. That is, t3 ≦
It is t1 + 30 sec.

Further, in the case of remote communication, there is a problem that the timing of conversation cannot be properly taken due to a time delay. With international calls, especially in the case of satellite communications, the timing of this utterance is still commonly experienced today. In a system that performs automatic interpretation, this time delay may be enormous due to voice recognition processing, machine translation processing, and the like. In this embodiment, by using the dialogue means, it is possible to give a message to the listener during the time delay.

In the case of voice input, it is difficult to automatically recognize the end of speech. Therefore, if there is no voice input for a certain period of time, it can be considered that the processing has been completed and the translation can be started. Alternatively, like a transceiver, a constant voice signal such as "Please" can be given to the control unit 9. It is more preferable to inform the system of the end of the utterance by using an input means other than voice such as a keyboard, which is more preferable.

Also, in voice, it is difficult to distinguish between uppercase and lowercase letters, or distinguish between common nouns and proper nouns. In the case of English, "Japan" must be translated as "lacquer ware" and "Japan" as "Japan", but it is extremely difficult to make this distinction by voice. Also in Japanese,
It is difficult to distinguish “Kondo” and “confused”. It is preferable to configure such that such information is also input from the keyboard. In addition, when other languages are mixed in the sentence (for example,
"How about" go men na sa
i "") and the like.

Here, if the voice generation unit 5 is configured to reflect the attributes of the speaker, such as age, sex, intonation (irony, emotions, etc.), accent, etc., in the generated voice, the listener can translate the voice. This is effective because you can get information about the person you are talking to from.

In the bidirectional automatic translation unit 12, if the dialect, for example, the vocabulary of the vocabulary, the place of origin, the educational background, etc. are reflected and translated, the listener can obtain the information about the other party from the content of the translated sentence. It is effective because it will be possible.

For example, as Japanese corresponding to English "I", various ones such as "I", "I", and "I" can be mentioned. First, the bidirectional automatic translation unit 12
In the initial stage of the conversation, "I" is used, and the vocabulary used in the process of the conversation is analyzed and sequentially replaced with an appropriate vocabulary.

Next, an automatic interpretation system according to another embodiment of the present invention will be described. FIG. 8 is a schematic configuration diagram showing the automatic interpretation system of the present embodiment. This automatic interpreting system is an example in which the automatic interpreting system in FIG. 1 is expanded to provide interpretation between N speakers who speak in different languages. It should be noted that the N speakers may include speakers who speak the same language.

As shown in the figure, the automatic interpreter system has N sets of input / output units consisting of a display unit, an input unit, a microphone and a speaker. For example, when the speaker A is a speaker, B to N are listeners.

In this case, the voice processing section 20 is provided with a function capable of processing a maximum of N kinds of languages, and the automatic interpreting section 30 has a maximum of N × as a translation from one language to the other language.
The system of FIG. 1 may be modified by adding (N-1) types of translation functions.

The operation of the automatic interpreter system for N persons is self-explanatory from the description of the system shown in FIG. 1, and a detailed description thereof will be omitted. Further, the present invention is not limited to the above-mentioned embodiments, and within the scope of the invention,
Various modifications can be implemented.

[0063]

In the automatic interpreter system according to the present invention, when a problem occurs in voice recognition or translation, a dialogue for confirmation is made with the speaker, and at the same time, the listening listener who is on standby is also involved in the confirmation. Since the dialog means for carrying out a dialog for transmitting information is provided, it becomes possible to deal with various problems actually caused by automatic translation by voice input, and it is possible to dramatically improve the practicality of the system.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram showing an automatic interpretation system according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the embodiment.

FIG. 3 is a diagram showing an output example of a recognition error in the same embodiment.

FIG. 4 is a diagram showing an output example of an inquiry from a system to a speaker in the embodiment.

FIG. 5 is a diagram showing an output example of an error in the automatic translation unit in the embodiment.

FIG. 6 is a diagram for explaining a voice utterance path in the embodiment.

FIG. 7 is a diagram showing relative timings at the time of utterance and arrival at the listener of the utterance in the embodiment.

FIG. 8 is a schematic configuration diagram showing an automatic interpretation system according to another embodiment of the present invention.

[Explanation of symbols]

1, 5 ... Display unit, 2, 6 ... Input unit, 3, 7 ... Microphone, 4, 8 ... Speaker, 9 ... Control unit, 10 ... Voice recognition unit, 11 ... Voice generation unit, 12 ... Bidirectional automatic translation unit , 13
... dialogue part, 14 ... natural language understanding part, 15 ... natural language generating part, 20 ... voice processing part, 30 ... automatic interpreting part

Claims (3)

[Claims] 1. A voice input unit for inputting a speech, an instruction input unit for inputting instruction information, a voice output unit for converting a given voice signal into voice and outputting the voice, and displaying the given information. A plurality of input / output units having a display unit, to which attributes indicating different types of languages are added, and utterances input from the voice input unit included in one of the plurality of input / output units A voice recognition means for performing voice recognition based on the attribute given to the one input / output means to generate a corresponding code or code string, and the code or code string for the other input / output means. Bidirectional automatic translation means for respectively translating into a code or code string corresponding to the type of language indicated by the attribute given to the input / output means, and a voice signal for giving the code or code string to the voice output unit. And a voice generation unit for converting the code or code string generated by the voice recognition unit when the voice recognition unit cannot specify a recognition result for the input utterance or the two-way automatic translation unit. When the result cannot be specified, a confirmation process for confirming the result of the recognition or the translation is included in at least one of the voice input unit and the instruction input unit included in the one input / output unit, and the voice output unit. And using at least one of the display units, and while the confirmation process is being performed, information regarding the confirmation process is output to at least one of the display unit and the voice input unit included in the other input / output unit. An automatic interpreting system, which is provided with a dialogue means. 2. The utterance input from the voice input unit is given to the voice recognition unit, and the utterance is output as it is from a voice output unit included in the other input / output unit. The automatic interpretation system described in 1. 3. The voice recognition means uses information for specifying at least one character type in the utterance given from the pointing unit when the utterance is input from the voice input unit, 2. The voice recognition of the utterance.
Automatic interpretation system described in.