JPH05290081A - Device for generating dictionary and device for analysing language - Google Patents

Abstract

(57) [Summary] [Purpose] Set the part of speech according to the field and the degree of use of the part of speech. [Structure] The input unit 1 inputs the degree of use of a part of speech for each word in the target field, and the source dictionary 2 stores the part of speech generally used for each word and its degree of use. The usage table 3 is the input section 1
The usage level input by is stored. The dictionary creation unit 4 creates dictionary information, and the output unit 5 outputs the dictionary information created by the dictionary creation unit 4. The dictionary creation unit 4
For words in the source dictionary, the part of speech having a usage less than the minimum usage of the part of speech that appears in the usage table is deleted.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a dictionary creation device and a language analysis device.

[0002]

2. Description of the Related Art A natural language analysis system or the like uses a dictionary that stores information about words. This dictionary often contains information about parts of speech. Also, a plurality of parts of speech may correspond to one word. Such a word is called a multi-part-of-speech word. In the analysis system, it is necessary to resolve the ambiguity of the part-of-speech in the multi-part-of-speech word and select the correct part-of-speech (part-of-speech disambiguation). A frequently used piece of information for eliminating this part of speech polysemy is the degree of part-of-speech usage. That is, the degree of use of a part of speech indicates how easily a word is used as the part of speech. Generally, the degree of use of a word depends on the text field to which the word belongs. Therefore, there is a problem that the part-of-speech disambiguation cannot be correctly resolved unless the part-of-speech and the degree of use of the part-of-speech are set according to the field.

In order to solve these problems, for example,
A "dictionary search system" is proposed in Japanese Patent Laid-Open No. 63-37474. The document in this publication accumulates information about the target field and the type of sentence for each text to be processed,
The stored information is used when searching the dictionary. The "document processing device" proposed in Japanese Patent Application Laid-Open No. 1-251160 includes a basic dictionary for registering basic words, and a plurality of field-specific dictionaries for registering the basic words and terms in each field by field. , Has a current dictionary for combining and registering terms registered in the field-specific dictionary. However, these methods have a problem in that when a dictionary is prepared for each field, various dictionary adjustments and managements are duplicated with respect to their common parts, and the cost of the work increases.

The "machine translation dictionary access method" proposed in Japanese Patent Laid-Open No. 63-236168, when accessing a multi-stage dictionary, sequentially accesses the plurality of dictionaries in descending order of priority. When the same data exists in a plurality of dictionaries, the one with the higher priority is taken in, and the different data is taken in from each dictionary. However, there is a problem that the order of priority can be performed only in dictionary units, and detailed priority for each word cannot be specified.

[0005]

[Object] The present invention has been made in view of the above situation, and a dictionary creation device and a dictionary created by the dictionary creation device configured to set a part of speech corresponding to a field and a degree of use of the part of speech. It was made for the purpose of providing a language analysis device using.

[0006]

In order to achieve the above object, the present invention provides (1)
A source dictionary that stores a part-of-speech generally used for each word and its usage, an input unit that inputs the usage of the part-of-speech for each word in the target field, and the usage that is input by the input unit are stored A usage table, a dictionary creating unit that creates dictionary information, and an output unit that outputs the dictionary information created by the dictionary creating unit, and the dictionary creating unit, regarding the words in the source dictionary, Deleting a part-of-speech having a degree of use smaller than the minimum degree of use of the part-of-speech appearing in the degree-of-use table, or (2) a source dictionary storing a part-of-speech generally used for each word and its degree of use; An input unit for inputting the degree of use of a part of speech for each word in the target field, a use degree table for storing the degree of use input by the input unit, a dictionary creating unit for creating dictionary information, and a dictionary creating unit for creating the dictionary information. An output unit that outputs the created dictionary information, and the dictionary creation unit For words in the source dictionary, delete parts of speech that are below a certain degree of usage and do not appear in the usage table, or (3) store commonly used parts of speech for each word and their usage A source dictionary, an input unit for inputting the degree of use of the part of speech for each word in the target field, a use table for storing the use degree input by the input unit, and a dictionary creating unit for creating dictionary information. And an output unit that outputs the dictionary information created by the dictionary creation unit, the dictionary creation unit having a new value for a word in the source dictionary based on the usage of the source dictionary and the usage of the usage table. Setting at least the degree of use, or (4) at least an input unit for inputting a sentence, an output unit for outputting a result, and a part-of-speech selecting unit for selecting a part-of-speech sequence of the input sentence In part, the above (1),
It is characterized by using a dictionary created by the dictionary creating device described in (2) or (3). Hereinafter, description will be given based on examples of the present invention.

FIG. 1 is a block diagram for explaining an embodiment of a dictionary creating apparatus according to the present invention, in which 1 is an input unit and 2 is
Is a source dictionary, 3 is a usage table, 4 is a dictionary creating unit, and 5 is an output unit. The input unit 1 inputs the degree of use of the part of speech for each word in the target field, and the source dictionary 2 stores the part of speech generally used for each word and the degree of use thereof. The usage table 3 stores the usage levels input by the input unit 1. The dictionary creation unit 4 creates dictionary information, and the output unit 5 outputs the dictionary information created by the dictionary creation unit 4.

The dictionary creating section 4 performs the following three operations. For words in the source dictionary, delete the part of speech that has a usage less than the minimum usage of the part of speech that appears in the usage table. For words in the source dictionary, delete parts of speech that are below a certain degree of usage and do not appear in the usage table. To set a new usage for a word in the source dictionary from the usage of the source dictionary and the usage of the usage table.

The part of speech is a classification name of a word. For example, there are nouns and verbs. On the other hand, the degree of use of a part of speech is an ordinal scale indicating the ease of use of the part of speech, and the larger the degree of use, the easier the use. Here, as an example of the degree of use, the use rate, that is, the number M of times a word is used and the number N of times a word is used as a part of speech X are defined by N / M.
Consider a probability value of 1. FIG. 2 is a diagram showing an example of the stored contents of the source dictionary. Information having the same structure is created by the dictionary creating unit and output from the output unit. Figure 3
FIG. 6 is a diagram showing an example of stored contents of a usage table. Information having a similar structure is input from the input unit and stored as shown in this figure.

FIG. 4 is a flow chart showing the operation of the embodiment (embodiment 1) of claim 1. Hereinafter, each step will be described in order. step1 : First, take out a word from the source dictionary. step2 : Next, the buffer is cleared. step3 : Extract the word form, part of speech, and usage X. step4 : Search the combination of word form and part of speech in the usage table. step5 : Determine whether there is a combination of word form and part of speech. If there is a combination of word form and part of speech, go to the next step6. If it does not exist, go to step 7 described later. Step 6 : If there is a combination of the word form and the part of speech in Step 5 , the usage X is added to the buffer.

[0011] step7: the last word form, part of speech, it is determined whether the use of X. If NO, go to step 3 above. step8 : If YES at step 7, the minimum value Min in the buffer is taken out. step9 : Extract the word form, part of speech, and usage X. step10 : If Min> use level X, output a set of word form, part of speech, and use level X. step11 : It is judged whether it is the last word form, part of speech, or usage X. If NO, go to step 9. step12 : If YES in step 11, it is determined whether or not this is the last source. If NO, go to step 1 above. If YES, the process ends.

FIG. 5 is a flow chart showing the operation of the second embodiment (second embodiment). Hereinafter, each step will be described in order. step1 : First, the usage constant Min is set. step2 : Next, take out a word from the original word book. step3 : Extract the word form, part of speech, and usage X. step4 : Search the combination of word form and part of speech in the usage table. step5 : It is judged whether or not there is a combination of word form and part of speech. If it does not exist, go to step 7 described later. Step 6 : If there is a combination of the word form and the part of speech in the step 5, the word form, the part of speech, and the degree of use are output, and the process proceeds to step 9 described later. Step7 : In the above Step5, if there is no pair of word form and part of speech, Min <X is judged. step8 : If Min <X in step 7, the word form, part of speech, and degree of use are output. step9 : It is judged whether it is the last word form, part of speech, or usage X. If NO, the process returns to step 3. step10 : If YES in step 9, it is determined whether or not this is the last word. If NO, the process returns to step 2, and if YES, the process ends.

FIG. 6 is a flowchart showing the operation of the third embodiment (third embodiment). Hereinafter, each step will be described in order. step1 : First, take out a word from the source dictionary. step2 : Next, the word form, the part of speech, and the usage X are extracted. step3 : Using the usage table, search for the usage Y using the combination of the word form and the part of speech as a key. step4 : It is judged whether the usage degree Y exists. If it exists, go to step 6 described later. step5 : If the usage Y does not exist in step 4, a constant Z is set as the usage Y. step6 : A new usage W is obtained from the usage X and the usage Y. There are various possible methods for obtaining the usage W, and for example, the following weighting method using weights A and B is available. W = A * X + B * Y step7 : Word form, part of speech, and usage W are output. step8 : Judge whether it is the last word form, part of speech, or usage X. If NO, the process returns to step 2. step9 : If YES in step 8, it is determined whether or not this is the last word. If NO, then
Return to step 1, and if YES, end.

FIG. 7 is a block diagram for explaining an embodiment of the language analyzing apparatus according to the present invention, in which 11 is an input unit,
Reference numeral 12 is an output unit, 13 is a dictionary, and 14 is a part-of-speech selection unit.
The input unit 11 inputs a sentence, and the output unit 12 outputs a language analysis result. The dictionary 13 is a dictionary created by the dictionary creating device. The part-of-speech selecting unit 14 selects a part-of-speech sequence of an input sentence, and performs language analysis using the dictionary created by the dictionary creating device described in the first to third embodiments. That is, an original sentence as shown in FIG.
The dictionary is searched for each word in the source text buffer, and then in FIG.
A part-of-speech candidate table as shown in is created and stored. Of the parts-of-speech candidates in the part-of-speech candidate table, only one with the highest degree of use is selected, and the output unit 12 outputs the result of selection.

[0015]

As is apparent from the above description, the present invention has the following effects. Since the degree of use of the part-of-speech can be easily adjusted according to the field, the system using the created dictionary can correct the part-of-speech disambiguation correctly for the field.

[Brief description of drawings]

FIG. 1 is a configuration diagram for explaining an embodiment of a dictionary creation device according to the present invention.

FIG. 2 is a diagram showing an example of stored contents of a source dictionary according to the present invention.

FIG. 3 is a diagram showing an example of stored contents of a usage table according to the present invention.

FIG. 4 is a flowchart for explaining the operation of the first exemplary embodiment of the present invention.

FIG. 5 is a flowchart for explaining the operation of the second embodiment of the present invention.

FIG. 6 is a flowchart for explaining the operation of the third embodiment of the present invention.

FIG. 7 is a configuration diagram for explaining an embodiment of a language analysis device according to the present invention.

FIG. 8 is a diagram showing an example of an original sentence according to the present invention.

FIG. 9 is a diagram showing a part-of-speech candidate table according to the present invention.

FIG. 10 is a diagram showing a part of speech selection result according to the present invention.

[Explanation of symbols]

1 ... Input unit, 2 ... Source dictionary, 3 ... Usage table, 4 ... Dictionary creating unit, 5 ... Output unit.

Claims (4)

[Claims] 1. A source dictionary storing a part of speech generally used for each word and its usage, an input unit for inputting the usage of the part of speech for each word of a target field, and the input unit A usage table that stores the usage, a dictionary creation unit that creates dictionary information, and an output unit that outputs the dictionary information created by the dictionary creation unit, wherein the dictionary creation unit is the source dictionary. A dictionary creating device for deleting a part of speech having a usage degree smaller than the minimum usage degree of the part of speech that appears in the usage degree table with respect to the words in the dictionary. 2. A source dictionary storing a part-of-speech generally used for each word and its usage, an input unit for inputting the usage of the part-of-speech for each word in a target field, and the input unit. A usage table that stores the usage, a dictionary creation unit that creates dictionary information, and an output unit that outputs the dictionary information created by the dictionary creation unit, wherein the dictionary creation unit is the source dictionary. A dictionary creating device for deleting a part-of-speech that is less than a certain level of usage and does not appear in the usage table for words in the. 3. A source dictionary storing a part-of-speech generally used for each word and its usage, an input unit for inputting the usage of the part-of-speech for each word in a target field, and the input unit. A usage table that stores the usage, a dictionary creation unit that creates dictionary information, and an output unit that outputs the dictionary information created by the dictionary creation unit, wherein the dictionary creation unit is the source dictionary. A dictionary creating device for setting a new usage degree for the words in the above from the usage degree of the source dictionary and the usage degree of the usage table. 4. An input unit for inputting a sentence, an output unit for outputting a result, and a part-of-speech selecting unit for selecting a part-of-speech sequence of the input sentence.
A language analysis device using a dictionary created by the dictionary creation device according to item 2 or 3.