JPH0336662A - Natural language processing method - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] Natural language processing devices such as proofreading word processors, kana-kanji conversion processing devices used for Japanese input in DPS, natural language analysis devices for speech recognition, machine translation, proofreading support, character recognition, etc. Regarding this, it is applied, for example, to a kana-kanji conversion device in speech synthesis, etc.

As for the conventional kana-kanji conversion processing method, the 2-bunsetsu longest teaching method is well-known, in which the first clause and the following clauses are analyzed and the first clause is determined based on the reading length of the two clauses. Although,
The drawback was that homophones could not be determined correctly based on morphological relationships such as the reading length of two clauses. Regarding this, several methods have been announced to solve this problem by writing meaning/co-occurrence information in a dictionary, but all of them either prioritize outputting candidates by matching with a dictionary, or output candidates that match with a dictionary. It is a matter of giving a certain score to
When performing analysis using different meanings and co-occurrence information in a wide variety of dimensions (for example, candidate 1 has the highest score when judged based on case relationships, but candidate 2 has the highest score when judged based on co-occurrence information, etc.) ), the major problem was how much weight to give to each piece of information, and it was difficult to add information of different dimensions.

In addition, there is an earlier application, Japanese Patent Application No. 63-288338, which uses a method that determines the first clause by giving plausibility to 1.5 clauses, and this method cannot handle information between multiple clauses. There was a drawback that there was no

The present invention was made in order to analyze the above-mentioned drawbacks, and it is necessary to accurately represent the plausibility as an independent part, accurately represent the plausibility as a clause,
By using the information of the preceding and following clauses to eliminate contradictions between multiple clauses, by easily giving weight to multiple pieces of information and making it easier to add new information, and by realizing a method for synthesizing multiple pieces of information, The purpose was to provide a natural language processing method that performs high-performance natural language analysis.

In order to achieve the above object, the present invention provides (1) a kana-kanji conversion process that converts a kana character string into a kanji-kana-mixed character string using a word dictionary that holds word information; In the process of determining a clause from a character string, from the meaning and co-occurrence relationships of prefixes, independent words, and suffixes in the independent part of the clause,
To give plausibility to candidates that include it, and furthermore, (
2) From the case relationship between the independent part within a clause and the independent part and attached part of its preceding clause, give plausibility to candidates that include it; Based on the relationship of meaning and co-occurrence with the independent part, give plausibility to the candidates that include it, and (4) use the likelihood of the case relationship between the independent part and the attached part of three consecutive candidates for analysis. Furthermore, (5) the likelihood of the meaning/co-occurrence relationship of the independent parts of three consecutive candidates is used for analysis; and (6) the likelihood of each of the above information including the frequency of words is Furthermore, (7) combining a wide variety of information with different dimensions between multiple clauses, finding the likelihood and accuracy, and based on this,
It is characterized by determining the phrase. Hereinafter, the present invention will be explained based on examples.

FIG. 1 is a block diagram for explaining an embodiment of the natural language processing method according to the present invention. In the figure, 1 is an input section, 2 is an analysis processing section, 3 is a candidate extraction section, and 4 is a dictionary search section. , 5 is a dictionary,
6 is a candidate evaluation section, 7 is a composition calculation section, and 8 is an output section.

Note that the immediately preceding phrase that has already been determined before analysis is called a determined phrase. Combine the clause and the following independent part to 1.
It is called 5 clauses.

The candidate extraction unit 3 performs a process of extracting a group of candidates in units of 1.5 bunsetsu. The candidate evaluation unit 6 evaluates and provides the extracted candidates with respect to their plausibility as 1.5 clauses. Synthesis operation section 7
calculates the first clause of the most likely candidate by performing a composition operation on the likelihood of the definite clause, the likelihood of the first 1.5 clauses, and the likelihood of the 1.5 clauses following the first clause. decide.

FIG. 2 shows a specific example of the candidate extraction section.

All candidate groups are accumulated (candidate 1, candidate 2, . . . , candidate n) in units of 1.5 clauses from the analysis start position.

Next, select the candidates following the first clause of each candidate in the same manner as 1.
Accumulate in units of 5 clauses (Candidate 1-1 for Candidate 1. Candidate 1-2, ... Candidate 1-m, Candidate 2-1 for Candidate 2, Candidate 2-2, etc.) ···)do.

FIG. 3 is a diagram showing the likelihood of prefix classification and suffix classification, and shows the likelihood of movement. The likelihood information for the classification is used to give a likelihood to each independent part.This information is added to the likelihood information of the 1.5 clauses that include each independent part.For example, "Undo Kai de...・・・・・・
When 1.5 clauses are analyzed for the string ``'',
' 0.7 for 1.5 clause candidates including "athletic day" ↓, 0.5 for 5 clause candidates including ``exercise'' ↓, 0.6 for 1.5 clause candidates including ``exercise'' Given the plausibility, the reading length of 1.5 clause candidates.

The likelihood of the 1.5 phrase candidate is determined by combining it with the likelihood obtained from the frequency, connection probability, etc.

Figures 4 (a) and (b) are diagrams showing the plausibility of the independent and attached subclauses of the preceding clause, where (a) is the plausibility for ``said'' (say), and (b) is the plausibility of the preceding clause. went"(
show the plausibility of This is for explaining the configuration of claim 2. That is, using the information on the likelihood for the independent and attached subclasses of the preceding clause, 1
．． Give plausibility to 5 clauses. As an example"·····
·said. When ↓ and 5 clauses are analyzed for the example character ''.

"Said.

"went.

Candidates such as In this case, if the previous clause is ``My friend (human being) is'', then 0.8 is given to ``I went.'', and 0.7 is given to 1. .5 Combining the likelihood obtained from the reading length, frequency, degree of connection probability, etc. of the bunsetsu candidates, 1.
Determine the plausibility of the 5 clause candidates.

FIG. 5 is a diagram showing the likelihood using the meaning/co-occurrence part information with the independent subclass of the preceding clause, and is for explaining the structure of claim 3. That is, using the meaning/co-occurrence part information with the independent part class of the previous clause, information on the likelihood of 1.5 clauses including each independent part is added. For example, for the character ``......hot.'', 1.
When analyzing 5 clauses, “Junii.

You can get candidates such as ``Passatsu''. At this time, if the previous clause is ``book,'' give a likelihood of 0.7 for ``trJ daai.'' and 0.4 for ``remarkable.'' and select ↓, 5 clause candidates. Combined with the likelihood obtained from reading length, frequency, degree of connection probability, etc., 1.
Determine the plausibility of the 5 clause candidates.

FIG. 6 is a diagram showing the likelihood of a combination of phrase candidates that match the information of the preceding and following phrases, and is for explaining the configurations of claims 4 and 5.

A likelihood is given to a combination of phrase candidates that match the information of the preceding and following phrases. By analyzing the three preceding and following clauses, it was found that the meanings and co-occurrence of "Tokyo", "1. This can represent a case where the connection between the three clauses of `` is very strong.

FIG. 7 shows a flowchart of the candidate evaluation section and the composition calculation section regarding the embodiment. In the figure, the likelihood of each candidate obtained in claims ↓ to 5 is shown as evidence-based ~5.

Figure 8 shows the first 1.5 clauses and the 1.5 clauses following that clause.
It is a flowchart which shows an example about a phrase. The following will be explained in 5 steps.

Let's spit it out; analyze the likelihood of each candidate in 1.5 clause units from each piece of evidence.

The correct answer candidate is the top one among the accumulated candidates.
If it is among the O candidates, only the top 10 candidates are accumulated. However, if the 1.5 clause length of the maximum likelihood candidate is the same as the separation length of the reading character string (the length up to the first punctuation mark/symbol that appears), the following processing will not be performed, so at this point lik likelihood lottery is determined as the first clause.

For each accumulated candidate, the next clause is analyzed in units of 1.5 clauses, which is the same as in step 2, and the top 10 candidates are accumulated.

! Atsushi-±;” The first 1.
Treat the certainty of each candidate for 5 clauses and the following 1.5 clauses as independent evidence, and use evidence 1 to 5.
is subjected to the synthesis operation 1→' process using the D-3 operation.

7i Based on the result of the above-described likelihood calculation, the first clause of the candidate with the highest likelihood is determined as the first clause.

Among these multiple pieces of evidence, evidence 1 exists for a certain candidate, but other pieces of evidence do not exist. Furthermore, the evidence is not equal for each candidate, such as only evidence 2 exists for a certain candidate, and it is extremely difficult to correctly judge the likelihood of a candidate using linear synthesis. In this regard, the inventors have already proposed a method (Japanese Patent Application No. 63-288338 mentioned above) that applies Dempster Schaefer's establishment theory to a method of synthesizing a plurality of pieces of evidence. When using this method, the effectiveness can be further increased by determining the importance of the evidence to be synthesized and weighting it before synthesizing it.

As is clear from the above description, the present invention has the following effects.

(1) Regarding claim 1, plausibility is given based on the morphological, semantic, and co-occurrence relationships of the prefixes, independent words, and adjuncts of the constituent elements of the independent part, so the plausibility of the independent part is determined accurately. can be expressed as

(2) Regarding claims 2 and 3, plausibility is given based on the case relationship with the preceding clause, meaning/co-occurrence relationship, so 1
．． The plausibility of the 5 clauses can be accurately expressed.

(3) Regarding claims 4 and 5, since the information on the preceding and following clauses is used, contradictions between a plurality of clauses can be eliminated.

(4) Regarding claim 6, since information plausibility is given to multiple pieces of information, the importance of information can be easily manipulated, and new information can be added easily. .

(5) Regarding claim 7, since a plurality of pieces of information can be easily combined, each piece of information can be provided independently, and high-performance natural language analysis can be performed.

[Brief explanation of drawings]

FIG. 1 is a block diagram for explaining an embodiment of the natural language processing method according to the present invention, FIG. 2 is a diagram showing a specific example of the candidate extraction section, and FIG. 3 is a diagram showing prefix classification and suffix classification. Figures 4 (a) and (b) are diagrams showing the likelihood of classification, and Figure 4 (a) and (b) are diagrams showing the likelihood of the independent and attached parts of the preceding clause. A diagram showing the likelihood using meaning/co-occurrence part information. Figure 6 is a diagram showing the likelihood for a combination of clause candidates that match the information of the first and previous clauses. Figure 7 is a diagram showing the candidate evaluation unit and the composition calculation unit. FIG. 8 is a diagram showing a flowchart when the process is performed for 1.5 clauses. DESCRIPTION OF SYMBOLS 1... Input unit, 2... Analysis processing unit, 3... Candidate extraction unit, 4... Dictionary search unit, 5... Dictionary, 6... Candidate evaluation unit, 7... Synthesis operation Part, 8... Output part.

Claims (1)

[Scope of Claims] 1. In a kana-kanji conversion process that converts a kana character string into a character string containing kanji and kana using a word dictionary that holds word information, and in a process that determines a clause from a character string,
A natural language processing method that is characterized by assigning plausibility to candidates containing prefixes, independent words, and suffixes based on their meanings and co-occurrence relationships within the independent part of a clause. 2. The natural language processing method according to claim 1, wherein a likelihood is given to a candidate containing the independent part in the clause based on the case relationship between the independent part in the clause and the independent part and attached part of the preceding clause. 3. The natural language processing method according to claim 1, characterized in that based on the relationship of meaning and co-occurrence between the independent part within a clause and the independent part of the preceding clause, likelihood is given to candidates containing the independent part. 4. The natural language processing method according to claim 1, wherein the likelihood of a case relationship between the independent part and the attached part of three consecutive candidates is used for analysis. 5. The natural language processing method according to claim 1, wherein the likelihood of the meaning/co-occurrence relationship of the independent parts of three consecutive candidates is used for analysis. 6. A natural language processing method according to at least one of claims 1 to 5, characterized in that each piece of information including the frequency of words is given a likelihood and used for analysis. 7. At least one of claims 1 to 6, characterized in that a wide variety of information with different dimensions is synthesized between a plurality of phrases to obtain likelihood and accuracy, and based on this, the first phrase is determined. The natural language processing method described in Section 1.