JPH0792800B2 - Connection cost table creation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial field of application) The present invention is a method of creating a connection cost table used in Japanese morpheme division processing, which is an essential component of Japanese-English machine translation systems, Japanese text-to-speech synthesis systems, and the like. It is about.

(Prior Art) Conventionally, various morpheme segmentation methods for determining word boundaries have been proposed in order to analyze Japanese text that is characterized in that word boundaries have no breaks such as spaces. These include, for example, "Information Processing" Vol. 27, No. 8, 95.
As described on page 1, Japanese morphological segmentation techniques such as longest-match method, two-bunsetsu longest-match method, minimum-bunsetsu number method, and minimum-cost method on extended bunsetsu model are known.

(Problems to be Solved by the Invention) However, in the conventional morpheme division method, a kind of optimization method that minimizes the number of clauses and a cost unique to each word are defined, and the sum of the costs is calculated. Although the optimization method of minimizing it is used, the ease of connection between adjacent word candidates is expressed in a unified manner, for example, as an amount that can be calculated, such as addition, and never used. Therefore, it is difficult to select a proper delimiter for Japanese when multiple delimiters are possible. As a method to improve this problem, a morpheme division method that defines the cost of connection between connection categories and outputs the division that minimizes the accumulation of the cost can be considered (Japanese Patent Application No. 63-201603). book).

However, in order to define the multivalued cost, which indicates the ease of connection between word candidates, between all rightward and leftward connection categories, a large-scale multivalued matrix must be created. It is very difficult to create and maintain a consistent list.

An object of the present invention is to provide a connection cost table creation method capable of easily creating such a multivalued cost table.

(Means for Solving the Problem) The present invention describes whether or not a connection between connection categories is possible.
A connection table of values and a group definition table that describes the definitions of the groups to which one or more connection categories belong,
If the connection between the connection categories belonging to the group is possible in the order described in the inter-group cost table and the storage means having the inter-group cost table describing the costs between the groups in order, the connection between the connection categories And a control means for generating a connection cost table by setting the cost to the same value as the cost between the groups to which they belong.

(Function) Even if the connection category is divided into about 200 in one direction, for example, there are multiple left (or right) facing categories that are equally easy to connect to a certain right (or left) facing category. It can be seen that they form a cohesive unit with their ease of connection in common. For example, for the right-handed connection category of "Sayuki Godan verb stem",
<Sa line 5 dan conjugation verb inflectional form>, <Sa line 5 verb conjugation verb inflection form>, <Sa line 5 dan conjugation verb inflection / termination form>, <Sa line 5 verb conjugation verb inflection form>・ Left-facing connection categories such as adnominal form, <sa line 5 verb conjugation verb ending / hypothesis form>, <sa line 5 verb conjugation verb ending / imperative form> are grouped because the connection cost is also very small. Are organized as

Therefore, in a set of right (or left) connection categories, connection categories that are considered to have similar connection properties to the left (or right) are collected to define a right (or left) connection category group. If the connection cost C between the groups is determined, the connection cost c (r, l) between the rightward connection category r and the leftward connection category 1 can be calculated as described below.

c (r, l) = ∞ if b (r, l) = 0. (1) c (r, l) = γ if b (r, l) = 1 and (2) C (R, L) = γ and r ∈ R and 1 ∈ L. where b (r, l): (r, l) element of the binary connection matrix. It is 1 if the rightward connection category r and the leftward connection category 1 are connectable, and 0 otherwise.

C (R, L): Connection cost between the rightward connection category group R and the leftward connection category group L.

∞: Indicates that connection is impossible.

A table that defines costs between connection category groups is called an inter-group cost table.

The calculation of the multivalued connection cost by the equations (1) and (2) is performed in the order described in the inter-group cost table.

Therefore, if you need to define a connection cost for a more specific group that is a subset of a group, write the cost definition for that more specific group after the cost definition for the containing group. You can leave it.

(Embodiment) Next, the present invention will be described in detail according to an embodiment with reference to FIG.

FIG. 1 is an embodiment for realizing the principle of the present invention. The binary connection table 103 is a two-dimensional matrix, and the value of its r row and 1 column element is 1 if the rightward connection category represented by the number r and the leftward connection category represented by the number are connectable. Otherwise 0. Hereafter, the r-row 1-column element of the binary connection table 103 is set to b
Call it (r, l).

The connection cost table generation buffer 104 is a buffer capable of storing a two-dimensional matrix having the same size as the binary connection table 103. This (r, l) element is called c (r, l).

FIG. 2 shows the format of the connection category group table 101. The definition of the connection category group is described in this table. L _i (i = 1, 2, ...) Is a leftward category group name, and l _i1 , l _i2 , ... Is a leftward connection category that belongs to it. R _i (i = 1,2 ...
) Is a right-facing category group, r _i1 , r _i2 ,
.. is a rightward connection category that belongs to it.

FIG. 3 shows an inter-group cost table. The connection cost between groups is defined in this table. Since the control unit 105 writes to the connection cost table generation buffer 104 in the order of description in the table, the definition of the inter-group connection cost is as follows:
The special ones are described later. L _i and R _i represent leftward and rightward connection category groups, and Ci represents the connection cost between them.

The control unit 105 operates as follows.

First, the binary connection table 103 is copied to the connection cost table generation buffer 104.

Each element of the connection cost table generation buffer 104,
If the value is 1, it is rewritten to the default value of the cost registered in the default cost holding unit, and if the value is 0, it is rewritten to ∞.

Next, the inter-group cost table 102 is scanned from the beginning, and the following processing is performed on each row of the inter-group cost table 102.

For every ordered pair (r _ix , l _iy ) of right- and left-facing connection categories belonging to Ri, Li, if b (r _ix , l _iy )
If = 1, then c (r _ix , l _iy ) = C _i .

As a result of the above processing, the content of the connection cost table generation buffer 104 becomes the connection cost table.

A method of performing morpheme division using the connection cost table generated by the procedure described above will be described with reference to FIG.

The control unit 401 operates as follows.

The dictionary 404 is searched for all morphemes starting from each character position of the input text held in the input text holding unit 402.

Extract all possible divisions of the input sentence from the above search results.

Connection cost table 40 for each extracted partition
Use 3 to find the sum of connection costs.

As a result, the division that minimizes the sum of connection costs is output.

(Effect of the invention) As described above, according to the present invention, heuristics regarding the ease of adjacency between word categories are described for input text that may be divided into a plurality of morphemes. , It becomes possible to easily create a connection cost table that enables more appropriate morpheme division.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention. FIG. 2 is a diagram for explaining the format of the connection category group table, and FIG. 3 is a diagram for explaining the inter-group cost table. FIG. 4 is a diagram for explaining an example of a morpheme division method using the connection cost table of the present invention. In the figure, 101 ... Connection category group table, 102 ... Group cost table, 103 ... Binary connection table, 104
...... Connection cost table generation buffer, 105 …… Control unit, 106 …… Default cost holding unit, 401 …… Control unit,
402: Input text storage unit, 403: Connection cost table, 404: Represent dictionary.

Claims (1)

[Claims] 1. A binary connection table that describes whether or not a connection between connection categories is possible, a group definition table that describes the definition of a group to which one or more connection categories belong, and a group that describes costs between groups in order. If the connection between each connection category belonging to the group is possible in the order described in the inter-group cost table and the storage means having the inter-cost table, the connection cost between the connection categories is the cost between the groups to which they belong. And a control means for generating a connection cost table by setting the same value as the connection cost table creation device.