JPH04195672A - Phrase segmentation device - Google Patents

Abstract

PURPOSE:To speed up a phrase segmenting process by regarding a specific HIRAGANA (cursive form of Japanese syllabary) as one phrase and correcting a phrase break insertion position when the specific HIRAGANA character string is present in a sentence inserted at the break between phrases. CONSTITUTION:When a unit of segmentation is extracted from a document stored in a document storage means 1, a character kind decision means 4 decides the kinds of characters constituting the sentence and the decision results are replaced with either of two kinds of code and stored in a decision result storage means 6 in order. Then a phrase segmenting means 7 puts a phrase break in the readout sentence when detecting a predetermined code change point. Then a phrase segmentation correcting means 9 compares an input character string with prescribed HIRAGANA characters in a specific character dictionary means 8 by tracing back the sentence from the break between the phrases, recognize a matching HIRAGANA character as one phrase, and corrects and outputs the phrase segmentation position. Consequently, language processing can be performed after the phrase position is detected, so the processing time can greatly be shortened.

Description

DETAILED DESCRIPTION OF THE INVENTION (a) Field of Industrial Application This invention relates to a phrase segmentation device, and more specifically, to devices that perform language processing such as word processors, translation devices, proofreading devices, devices that use databases, etc. The present invention relates to a suitable phrase cutting device.

(b) Conventional technology In order to extract phrases from a converted Japanese document in which phrase separation information has not been inserted, for example, for translation or proofreading, there are conventional techniques such as independent word dictionaries, attached word dictionaries, and affix dictionaries. A table representing the dictionary order of , etc., and the connection relationship of those elements,
Clauses are extracted while referring to tables such as grammar tables.

Also, a method is being considered that uses character type information, for example, to use a change in character type as a break between phrases.

(c) Problems to be solved by the invention In the conventional phrase extraction device that uses dictionaries and tables, (1) a large amount of storage device is required to store the dictionaries and tables; Become. (2)
In order to perform a dictionary search or a table search, the time period for the bunsetsu extraction process becomes long. (3) The control program for segmentation becomes m*. In addition, (4) when cutting phrases using character type information, since the phrases are mechanically cut out according to the character type, it is possible to make mistakes in cutting phrases due to fluctuations in the sending gana. (5) Various problems were encountered, such as words in kana notation sometimes causing incorrect segmentation of phrases.

The present invention has been made in consideration of the above-mentioned circumstances, and provides a phrase cutting device capable of solving the above-mentioned problems.

(d) Means for Solving the Problems FIG. 1 is a block diagram showing the basic configuration of the present invention. In the figure, the present invention includes: a text storage means 1 for storing sentences; an instruction means 2 for instructing to read a desired sentence from the text storage means 2; and a command means 2 for reading the instructed sentence from the text storage means 1. The reading means 3, the character type discriminating means 4 which discriminates the character type such as kanji, hiragana, katakana, etc. for each character of the read sentence, and the character type discriminating means 4 which discriminates the period, and the discrimination result by the character type discriminating means 4 are transferred to the 2N condyle. A conversion means 5 which replaces and outputs the code with one of the codes, a discrimination result accumulation means 6 which sequentially accumulates the output from the conversion means 5, and a predetermined code transition point for the code string accumulated in the discrimination result accumulation means 6. A phrase extraction means 7 that inserts a phrase break into the read sentence when detected, and a specific character dictionary that stores a large number of specific hiragana character strings that include adjuncts, pronouns, openings, conjunctions, etc., and that accompany phrases. Means 8 searches for whether or not the specific hiragana character string exists in the sentence inserted at the end of the phrase, and if the specific hiragana character string is present, the hiragana character string is used as one phrase in the sentence. A bunsetsu cutting correction means 9 inserts a bunsetsu break into the bunsetsu break and corrects the bunsetsu break insertion position, a storage means lO stores sentences output from the bunsetsu cutout correction means 9, and a storage means l
This phrase extraction device comprises an output means 2 for visually outputting sentences stored in O.

In this invention, each hiragana character string stored in the specific character dictionary means 8 is preferably stored in the same order as the hiragana character extraction order in the search.

The phrase extraction device of the present invention can be applied to word processors, translation devices, proofreading devices, devices that use databases, etc. Furthermore, since phrase extraction processing is required even in devices that output sentences as audio, the phrase extraction device of the present invention can be applied to devices that utilize databases. Can be applied.

(E) Effect According to the present invention, when a unit of extraction processing, for example, one sentence, is extracted from the sentences stored in the sentence storage means 1,
The character type of each character constituting the sentence is determined by the character type determination means 4, and the determination result is converted to 2N1 by the conversion means 5.
Any of the symbols [ is replaced with, for example, L or H, and the results are sequentially stored in the determination result storage means 6. Next, the phrase extraction means 7 selects a predetermined sign transition point,
For example, when a change from L to H is detected, a phrase break is inserted into the read sentence. Next, the bunsetsu cutting correction means 9 compares the input character string with the hiragana characters specified in the specific character dictionary means 8 starting from the break of the bunsetsu, recognizes the matching hiragana characters as one bunsetsu, and extracts the bunsetsu. It acts to correct the position and output the obtained result to the output means 11 so that the corrected result can be confirmed.

(F) EXAMPLES The present invention will be described in detail below based on examples shown in the figures. Note that this invention is not limited by this.

FIG. 2 is a block diagram showing an embodiment in which the present invention is applied to a word processor. In the figure, 20 is a word processor main body. 21 is a writing storage device! It can be composed of an external storage device such as a floppy disk, a hard disk, an internal storage device such as RAM, or another storage device such as a database, and can store Japanese documents consisting of sentences mixed with kana and kanji. has been done. Reference numeral 22 denotes a keyboard as an instruction means, which is equipped with a character input key 1 for editing sentences, proofreading, etc., and various instruction keys, etc., for inputting sentences and for reading out desired sentences from the sentence storage device 21. Enter the instructions for Reference numeral 23 denotes a reading device that cooperates with the CPU 24, and reads out 12 sentences from the text storage device 2 [in response to instructions from the keyboard 22]. Reference numeral 25 denotes a character type discriminating device that cooperates with the CPU 24, and for each sentence read from the sentence storage device 21, it determines the character type of each character, such as kanji, hiragana, katakana, etc., and also determines punctuation marks. 2
Reference numeral 6 denotes a result storage device as a discrimination result storage means and storage means, which is composed of a RAM, and sequentially accumulates character type discrimination results, and also stores sentences cut by a phrase cutting device to be described later.

27 is a character type digitization device as a conversion means that cooperates with the CPU 24, and a character type discrimination device! For kanji and katakana as a result of character type discrimination according to 25, the first code, specifically "H", is assigned, and for hiragana and period marks, the second code, specifically "S" is assigned. ” is added, thereby converting the character type discrimination result into two types of codes ゛H” or °L
” and the code “H” which is the character type discrimination result.

The symbol string “L” is sent to the result storage device 26 via the CPU 24.
is accumulated in

Bunseki cutouts! 28 cooperates with the CPU 24 and inserts a phrase break into the read sentence when a predetermined code transition point is detected for the code strings "H" and "L" stored in the result storage device 26.

Reference numeral 29 denotes a specific character dictionary table composed of a ROM, which stores a large number of specific hiragana character strings including adnominals, pronouns, adverbs, conjunctions, etc., and accompanied by clauses. gil! in this specific character dictionary table 29! The stored hiragana character strings are stored in the same order as the hiragana character extraction order in the search.

Reference numeral 30 denotes a phrase cutting/correction device that cooperates with the CPU 24, and searches whether or not the specific hiragana character string exists in the sentence in which the phrase break is inserted, and if the specific hiragana character string exists, A clause break is inserted into the sentence using a hiragana character string as one clause, and the insertion position of the clause break is corrected.

Reference numeral 31 denotes a display device as an output means connected to the CPU 24 via the output control unit 32, and is composed of a dot matrix type display device such as a CRT or LCD, so that the breaks in phrases can be confirmed. , display the sentence with the break inserted.

With this configuration, I created the materials to be submitted to the Patent Office using this word processor and filed them electronically. The bunsetsu segmentation process will be explained using "" and according to the flowchart shown in FIG. 12.

FIG. 3 shows a state in which, for example, L sentences are cut out from the text storage device 2L in which text files and other database records are stored in accordance with the unit of processing and are stored in the result storage device 26. In this way, you can store sentences according to processing units, such as one sentence, one paragraph, or one chapter! When a desired sentence is extracted from 21 (step 40), the code of the character type composing the sentence is determined (step 41).
).

Specifically, each character in the read sentence is assigned a unique character code such as a JIS code, so by comparing that character code with each condition in the code discrimination table shown in Figure 4, , determine the character type. In the conditions, cc is the character to be distinguished, and al and bl are
A2 and b2 represent the beginning and end of a real character code, a2 and b2 represent the beginning and end of a hirakato code, a3 and b3 represent the beginning and end of a katakana code, and fL4 represents a period.

The determined character type code is stored in the result storage device [26 (step 42), and the character type digitization device! Digitization is performed by 27 (step 43).

FIG. 5 shows a code digitization table that is referenced to digitize the determined output code. In other words, if the character type is determined to be a kanji, it is converted to the code ``H'', if it is determined to be hiragana, it is converted to the code ``L'', and similarly, katakana is converted to the code ``H'', and phrases are converted to the code ``L''. Replaced.

A collection of sentences digitized by comparison with the code digitization table is stored in the result storage device 26 in the form of a code string shown in FIG.

Next, the digitized result is compared with the cut-out discrimination table shown in Fig. 7 to determine bunsetsu breaks (step 44). Insert a keyword (clause) break at the transition point. (2) Discrimination is made by inserting a keyword break next to the period.

Next, a cutting symbol, for example "/", is inserted into the break based on the discrimination of the break by comparison with the cutting discrimination table, and the sentence into which the cutting symbol has been inserted is stored in the result storage device 26. FIG. 8 shows the phrase extraction results obtained by the above processing.

When segmenting a phrase using only the information on the change in character type, as shown in Figure 8, if a word is written in hiragana, the phrase containing the word cannot be extracted. . In the example sentence just now, the pronoun ``kono'' is absorbed into the clause ``, and ``material wo kono'' becomes one clause. A feature of the present invention is that it corrects such bunsetsu cut errors caused by hiragana notation.

In order to achieve this, a specific character dictionary table 29 is used that stores word strings in typical hirakana notation. A set of word strings written in hiragana can be created as a well-known fact from ordinary language life. For example, in terms of part of speech classification, there are many words written in hiragana in adnominals, pronouns, adverbs, conjunctions, etc.

FIG. 9 shows an example of a hiragana notation table for explaining the specific character dictionary table 29. The figure shows typical words for the purpose of explanation, sorted in the order of normal single-layer character strings. FIG. 10 shows the contents of the specific character dictionary table 29, which is obtained by sorting the above-mentioned hiragana notation table in reverse order.

Continuing to step 44, by referring to the specific character dictionary table 29 and @, the part of the hiragana character string is searched toward the beginning of the character string from the character string that has been segmented as shown in Figure 8 (step 45).

As mentioned above, the order of words stored in the specific character dictionary table 29 is the same as * for character string searches, and if a single character fails to match, matching with that word can be released. Therefore, it is possible to shorten the time required for matching. Furthermore, since it is possible to confirm that the verification has failed during the process, the buffer capacity can be reduced. This process is performed between hirakana character strings where the character type changes from hiragana to ike character type.

In this embodiment, it is successfully verified whether the part of ``wokono'' is ``noko'' (step 46). If the matching is successful, a clause break is added so that the clause starts at the beginning of the word written in hiragana, and the segment cutout position is corrected (step 47). As a result, the phrase extraction position of the sentence shown in FIG. 8 is corrected as shown in FIG. 11. The modified results are then stored in the results storage device 26 (step 48).

Next, if the end condition is no, that is, if there is a sentence that should be segmented next, the next sentence is stored in the sentence storage! 21
The process of reading data from the memory data and the storage location are controlled so that they do not overlap (step 49).

If the answer is yes in step 49, that is, if there are no more targets to be cut out, the necessary information is stored in the result storage device 26, and the process ends (step 50).

(g) Effects of the invention According to this invention, (1) when cutting out a phrase,
Since no dictionary is used, the configuration of the device can be simplified. As a result, the present invention can be applied not only to word processors and office computers, but also to other small devices, specifically electronic notebooks and calculators with program functions. (2) Phrase extraction processing and keyword search processing can be performed at high speed. (3) The control program for segmenting phrases can be simplified. (4
) When cutting out a clause, the disadvantage is that the position of the clause cannot be determined because Japanese is written in solid letters, and it requires a great deal of processing and time to determine where the clause begins and ends. There is.

According to this invention, language processing can be performed after determining the bunsetsu position, so processing time can be significantly shortened. (5) Character type discrimination results are converted into 2 parts using a digital circuit.
Since the phrases are directly replaced and processed, the processing speed is increased, the circuit is simplified, and the phrase segmentation device can be realized at low cost. (6) It is possible to reduce the Ii5 difference in phrase extraction position due to one hiragana word. (7
) It is possible to quickly compare and match the hiragana notation character string of the specific character dictionary and the input character string, and during the comparison, the capacity of the Puffer memory required for the dictionary can be realized with a small capacity.

[Brief explanation of the drawing]

Figure 1 is a block diagram that clearly shows the basic configuration of this invention, Figure 2 is a block diagram that shows the configuration of a word processor that is an embodiment of this invention, and Figure 3 is an example of one sentence stored in the sentence storage device. FIG. 4 is an explanatory diagram showing the contents of the character type discrimination table. FIG. 5 is an explanatory diagram showing the contents of the code digitization table. FIG. 6 is an explanatory diagram showing the character type discrimination results. Figure 8 is an explanatory diagram showing the contents of the bunsetsu extraction discrimination table, Figure 8 is an explanatory diagram showing the extraction results, Measures 9 and 10 are explanatory diagrams showing the contents of the specific character dictionary table,
FIG. 1I is an explanatory diagram showing the cutting position correction result, and FIG. 2 is a σ-chart (showing the processing operation of the embodiment). ! ...Text storage means, 2. ...Instruction means, 3.
- Reading means, 4 - Character type discrimination means, 5
...Conversion means, 6..Discrimination result accumulation means, 7..Phrase extraction means, 8..Specific character dictionary means, 9..Paragraph extraction correction means, 10.. Storage means, 11...-Output means. Agent Patent attorney Shintabe Nogawa So two 4, -pu,?・3 Figure 1. ,--7" to Ni-1O □-Out 1X
Door 4 ``・'' Figure 5 Figure 6 is 7 Figure 7 /Fig. 9:=IQtn No. 11 Σ Submit/materials to the NtIF Agency/217)/Create/at the entrance of the workhouse/Electronic 5 swelling t2,/Fig. 12

Claims (1)

[Scope of Claims] 1. A text storage means for storing sentences; an instruction means for instructing to read a desired sentence from the text storage means; a reading means for reading the instructed sentence from the text storage means; A character type discrimination means that discriminates the character type of each character such as kanji, hiragana, katakana, etc. for a given sentence, and also discriminates the period, and a conversion that outputs the result of the discrimination by the character type discrimination means by replacing it with one of two types of codes. a discrimination result accumulation means for sequentially accumulating outputs from the conversion means; and inserting a clause break into a read sentence when a predetermined code transition point is detected in the code string accumulated in the discrimination result accumulation means. a specific character dictionary means that stores a large number of specific hiragana character strings that include adjuncts, pronouns, adverbs, conjunctions, etc. and are accompanied by clauses; Bunsetsu extraction that searches for the existence of a character string, and if a specific hiragana character string exists, inserts a clause break into the sentence with that hiragana character string as one clause, and corrects the bunsetsu break insertion position. A phrase extraction device comprising a correction means, a storage means for storing a sentence output from the phrase extraction correction means, and an output means for visually outputting the sentence stored in the storage means. 2. The phrase extraction device according to claim 1, wherein each hiragana character string stored in the specific character dictionary means is stored in the same order as the hiragana character extraction order in the search.