JPH1185773A - Information search method and apparatus, information search system, recording medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide an information retrieval apparatus for applying a neural network to develop a synonym of a retrieval keyword. SOLUTION: A plurality of keywords extracted from all DBs 15A to 15N by morphological analysis, individual keywords D
The information search device 1 is configured to include a keyword list 13 listing importance levels in B and a predetermined output pattern, a keyword learning unit 12, and a database management unit 14. If the input search keyword is not included in the keyword list 13, the keyword learning unit 12 obtains an output pattern using the self-organizing network, and selects a keyword of an output pattern similar to the output pattern from the keyword list 13. The database management unit 14 performs an information search process on the target DB using the selected keyword.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieval method for performing an efficient retrieval process by learning a keyword in digitized information by using, for example, a self-organizing network to which a neural network is applied.

[0002]

2. Description of the Related Art When constructing a search system for digitized information, a plurality of databases are constructed in advance by classifying each type of digitized information. Processing is generally performed. (1-1) Search processing is executed for all databases, and the search results are integrated and presented to the user. (1-2) A plurality of keywords (search terms, hereinafter the same) related to the contents in a specific database are extracted and listed in advance, and whether or not a keyword to be searched is included in this keyword list According to this, the database to be searched is narrowed down, and the search processing is executed on the narrowed down database. (1-3) In order to be able to acquire digitized information related to a specific keyword among the keywords not included in the keyword list, a synonym expansion dictionary such as a thesaurus dictionary is also used for executing the search process. use.

[0003]

However, the conventional search system has the following problems. (2-1) The method of integrating all search results with all databases as search targets increases the processing load at the time of search and integration as the number of databases and the amount of data increase, and is not realistic. I can't say. (2-2) The technique of narrowing down the database using the keyword list lacks flexibility for keywords not included in the keyword list. That is,
A search result cannot be obtained for such a keyword, and the search relevance rate decreases. (2-3) In a search process using a synonym expansion dictionary, it is necessary to manually create, set, and maintain the synonym expansion dictionary, so that maintenance is complicated.

[0004] It is an object of the present invention to provide an improved information retrieval method which can solve such a conventional problem. Another object of the present invention is to provide an information search device and an information search system suitable for implementing the above information search method, and a recording medium storing a program for implementing the above information search method on a general-purpose computer device. It is in.

[0005]

According to a first aspect of the present invention, there is provided an information retrieval method comprising the steps of executing at least the following steps in a computer. (1) A keyword is extracted from all digitized information to be searched, and for each extracted keyword, the importance based on the frequency of appearance in a plurality of information storage areas and the pattern characteristics of character codes constituting the keyword are determined. Calculating and listing. (2) extracting a pattern feature of a character code constituting the designated search keyword; This pattern feature is extracted, for example, by learning an arrangement pattern of character codes in a predetermined neural network model so that the information entropy is maximized. (3) A step of specifying, from the list, a keyword having a pattern feature whose similarity with the extracted pattern feature is equal to or more than a predetermined value and an importance. (4) executing a search process for the information storage area with the specified keywords in the order of importance as search keywords.

According to another aspect of the present invention, there is provided an information search apparatus for executing a search process based on a search keyword for a plurality of information storage areas storing digitized information. Keyword extraction means for extracting a keyword having an appearance frequency exceeding a predetermined number from each of the storage areas, importance calculation means for calculating importance for each information storage area based on the appearance frequency of the extracted keyword, and comprising the extracted keywords And a keyword list creating means for creating a keyword list in which the calculated importance and the extracted pattern features are associated with each keyword. The information processing device is configured to determine the search keyword and an information storage area to be searched based on the search keyword A.

According to another information retrieval apparatus of the present invention, for a plurality of keywords extracted from digitized information, the importance based on the frequency of appearance in each information storage area and the pattern characteristics of character codes constituting each keyword are determined. A list of keywords, a feature extracting means for extracting a pattern feature of a character code constituting the designated search keyword, and comparing the extracted pattern feature with a pattern feature in the keyword list to obtain one or more keywords. And a search means for executing an information search process on the information storage area having the higher importance using the specified keyword as the search keyword.

The feature extracting means includes a neural network model for learning the configuration of the internal neuron so that the information entropy of the output pattern in the output layer is maximized corresponding to the character code group input to the input layer. Be composed.

According to another aspect of the present invention, there is provided an information retrieval system comprising: a first device having a database in which digitized information is stored; a first device which transmits a search request including a search keyword to the first device; A second device that acquires a search result corresponding to the search request is connected and configured to be capable of two-way communication. The first device and the second device are configured to extract a keyword having an appearance frequency exceeding a predetermined number from each of the plurality of information storage regions, and to determine importance of each information storage region based on the appearance frequency of the extracted keywords. Importance calculating means for calculating; a feature extracting means for extracting a pattern feature of an extracted keyword or a character code constituting a search keyword included in the search request;
A keyword list creating means for creating a keyword list by associating the calculated importance and the extracted pattern features for each of the extracted keywords; and extracting and extracting pattern features of character codes constituting a specified search keyword. A keyword specifying unit that specifies one or a plurality of keywords by comparing the obtained pattern feature with the pattern features in the keyword list, and using the specified keyword as the search keyword for the information storage area having the higher importance. And a search unit that executes information search processing, and performs the information search in cooperation.

A recording medium according to the present invention for solving the above-mentioned other problems is a recording medium in which a program for causing a computer device to execute the following processing is recorded in a form readable by the computer device. (1) a process of extracting a keyword from all digitized information stored in a plurality of information storage areas; (2) for each extracted keyword, the importance based on the frequency of appearance in each information storage area and the keyword A process of calculating and listing pattern features of constituent character codes, (3) a process of extracting a pattern feature of a character code constituting a specified search keyword, and (4) a degree of similarity with the extracted pattern feature. (5) a process of executing a search process on the information storage area with the identified keywords as search keywords in order of importance, and a process of specifying a keyword having a pattern feature whose value is equal to or more than a predetermined value and a priority.

[0011]

Embodiments of the present invention will be described below in detail with reference to the drawings. (First Embodiment) FIG. 1 is a functional block diagram showing an embodiment of an information retrieval apparatus to which the present invention is applied. This information search device 1 includes N databases (DB) 1 built in a general-purpose computer device or in an external storage device.
.. 15N (hereinafter, suffix will be omitted and expressed as DB15 in the case of a unified description)
An input processing unit 11 formed by reading and executing a predetermined program by the computer device;
It comprises a keyword learning unit 12, a keyword list 13, a database management unit 14, and a search result output unit 16.

The above program is usually stored in an internal storage device or an external storage device of a computer, and is read and executed as needed.
A recording medium that can be separated from the computer device, for example, a CD
-It may be stored in a computer readable form in a ROM, FD, or the like, and may be installed in the internal storage device or the external storage device at the time of use and provided for execution at any time.

The DB 15 stores a plurality of pieces of digitized information to be searched after being classified based on a predetermined classification standard. In this example, N DBs (DB-A, DB-B, DB-
C,... DB-N) 15 are connected in parallel. The classification criterion in this case is based on, for example, the information source name of the digitized information, the year, the type of digitized information, and the like.

The input processing section 11 is for inputting a search request input from a user via input means (not shown). The search request includes one or more search keywords, and the search keywords are input to the keyword learning unit 12.

The keyword learning section 12 extracts the pattern characteristics of the character codes constituting the keyword,
The pattern feature of the search keyword input from the input processing unit 11 is compared with the pattern feature in the keyword list 13. The keyword list 13 holds the keywords in all the digitized information stored in the DB 15 and the importance based on the appearance frequency of each keyword.

The database management unit 14 specifies a DB to be a search process from the DB 15 based on the comparison result input from the keyword learning unit 12, and executes an information search process on the specified DB. . Search results are
It is input to the search result output unit 16. Search result output unit 16
Is for outputting the result of the search processing in the database management unit 14 to an output device (not shown) such as a display device or a printing device.

The keyword list 13 of this embodiment is created as follows. (1) First, a well-known morphological analysis is performed on the digitized information stored in the DB 15 to extract keywords from the digitized information. The number of extractions is a predetermined number in order from the keyword having a high appearance frequency in each DB. This appearance frequency can be specified by, for example, a count value of the number of times the keyword appears in the digitization information of each DB.

(2) Assuming that a set of keywords in each DB 15 is Ka, Kb,... Kn, the set of these keywords has N elements. Therefore, a set U ｛= KU1, KU2, KU in which all elements are put together from these keyword sets Ka, Kb,.
Ask for 3, ... 求 め る. The number of elements in this case is “DN × N”, where DN is the number of elements in each DB. This set U preferably has no overlap between elements (the number of elements in the case of overlap is “≦ DN × N”).

(3) After keyword extraction, the importance based on the appearance frequency of each keyword is defined. It is assumed that those with low appearance frequency have high importance, and those with frequent appearance have low importance. In this example, if the appearance frequency of the keyword is, for example, DB-A, the keyword “x” in that DB
Is defined as the importance Wb (x) of the keyword “x” in the DB if the importance is Wa (x) and DB-B. The same applies to other DBs. At this time, each D
When a bias value such as a priority based on a classification criterion for each DB is set for the keyword extracted from B15, the importance W
a (x), Wb (x) ... are defined.

(4) Further, the keyword “x” (= KU)
1, KU2, KU3,...) Are input as “S (x)”, and a keyword list 13 composed of a table as shown in FIG. 2 is constructed. This output pattern is a vector pattern which is an output of the self-organizing network in the keyword learning unit 12 described later.
The keyword list 13 is created and held in advance by the keyword learning unit 12 for all keywords (keyword creating means).

Next, the contents of the keyword learning unit 12 will be described in more detail with reference to FIGS. First, a configuration example of the keyword learning unit 12 will be described with reference to FIG.
In the present embodiment, the keyword learning unit 12 is configured to include a self-organizing network to which a neural network model is applied. A self-organizing network is a neural network model that changes its internal structure so that it operates properly by external input.
This is conceptually shown in FIG.

The symbols shown in a circle in FIG. 3 are neurons (elements), and are also called units or nodes. In the following description, a neuron is described as a unit. The unit group Ui is positioned as an input layer of the self-organizing network. The maximum number of units in the input layer Ui depends on the character pattern to be input.

This self-organizing network is also called a "hidden unit layer", and is designed so that the information entropy of the output pattern in the output layer Uo is maximized corresponding to the character code group input to the input layer Ui. According to the conversion algorithm, a unit group is formed therein, and learning is performed so as to minimize the number of units. Maximizing the information entropy of the output pattern specifically means minimizing the quantified “degree of uncertainty” for the input character pattern. In other words, obtaining "reliable" output information for the input information. Therefore, in the learning condition in this case, the output pattern of the output layer Uo for the character code group of the input layer Ui is uniquely determined, and the output pattern is set so as not to overlap with the other character code groups. It is. The output pattern in the output layer Uo is output as a vector value composed of a binary pattern of “1” or “0”.

Although various types of organizing algorithms have been proposed for the self-organizing network, in the present embodiment, as an example, a simulated annealing (Simulated Annealing) in a known Boltzmann Machine neural model is used. ) Use those based on the law. This organizing algorithm has the following contents.

First, in the above set U, if the total number of elements including the duplication of keywords is N, the output pattern for the keyword KU ₁ is “S (KU ₁ )” if the output function is “S”. ] Is represented. Here, the sequence “S ′ = (S ′ ₁ ,
S ′ ₂ , S ′ ₃ , S ′ ₄ ,... S ′ _i ) ”. In this case, assuming that the total number of elements of the sequence S ′ is N ′, the occurrence probability of the output pattern S (KU _i ) for the keyword KU _i is:
Each is obtained by “1 / N”.

Further, assuming that the element S ′ _k of the sequence S ′ is an overlap of the output patterns “S (KU _k1 )” and “S (KU _{k 2} )”, the occurrence probability P becomes Can be expressed as

[0027]

P (S ' _k ) = P (KU _k1 ) + P (KU _k2 ) =
1 / N + 1 / N

Here, for example, the following energy function E is defined.

[0029]

E = ΣP (S ′ _i ) log ₂ P (S ′ _i ) (1)

Here, Σ represents the sum of (i = 1 to i = N ′. In general, the energy function E is a neural model in which the energy determined by the state of the unit, the connection coefficient, etc. Is a function (or sometimes an error function) determined so as to minimize the desired output required, and a parameter or the like is changed by a predetermined steepest descent method so as to realize this minimum value. From this, an optimal output pattern can be obtained by configuring a group of units in the self-organizing network such that the value of the energy function E takes the minimum value. Since it is a valley function and it is difficult to converge to many minimum values as it is and it is difficult to converge to the minimum value, in the simulated annealing method, the energy function E The so as to converge to a minimum value, in which stochastic behavior (fluctuation) is introduced. The following is an energy function E 'where the energy function E is set so as to facilitate to converge to the minimum value.

[0031]

E ′ = E + ε / (Emax−E) (2)

In this case, the coefficient ε is selected to be a value sufficiently smaller than the energy function E, and for example, a predetermined probability exp (− △ E / T) or the like is used. In addition,
This “T” is a parameter called temperature in the simulated annealing method. By using the energy function E ′ of the equation (2) complementarily to the energy function E of the equation (1), the calculated value converges to the minimum value, and the calculated value converges on the target character pattern. The output pattern is uniquely determined.

Next, the processing procedure of the keyword learning unit 12 having the above configuration will be described with reference to FIG. Here, the character pattern “WEB” is input to the input processing unit 11, and is converted into a digitized character code for each of the characters “W”, “E”, and “B” and input to the self-organizing network. It is assumed that it has been performed (steps S101, S102).

The keyword learning unit 12 converts each character code into a predetermined input pattern in the unit group of the input layer Ui and inputs the pattern to the self-organizing network (step S10).
3, S104). Within the self-organizing network, a unit group is configured based on the above-mentioned organizing algorithm (step S105), and an output pattern is output to the unit group of the output layer Uo (step S106). The output pattern in this case is a vector pattern including a binary pattern of “1” or “0”.

The keyword learning unit 12 further calculates the above-mentioned energy function E (step S107), and determines whether the calculated value is equal to or smaller than a preset threshold.
If the calculated value exceeds the threshold (step S108:
No), the process returns to step S105, and the unit group in the self-organizing network is reconfigured and the process is repeated. on the other hand,
If the calculated value is equal to or smaller than the threshold (Step S108: Ye
s), the output pattern is determined as an output vector value corresponding to the input character string (step S10).
9). In this example, the calculated value of the energy function E is compared with a preset threshold value. However, this is merely an example. Configurations are also possible.

Next, the operation of the information search device 1 of the present embodiment will be described with reference to FIG. The keyword learning unit 1
The self-organizing network in 2 has been trained,
In addition, the description will be made on the assumption that the keyword list 13 has already been generated.

A user inputs a search keyword such as a JIS code or the like into an input stage of the information search apparatus 1, that is, an input processing unit 11.
Is input (step S201), the input processing unit 11
Compares the search keyword with a keyword group in the keyword list 13 (step S202). When there is a keyword that matches the search keyword in the keyword list 13 (step S202: Yes), the search keyword is input to the database management unit 14. In this case, there may be a plurality of search keywords, and the number can be set arbitrarily. On the other hand, when there is no keyword that matches the search keyword in the keyword list 13 (step S202: No), the input processing unit 11 transfers the search keyword to the keyword learning unit 12 (step S203).

The keyword learning unit 12 determines an output pattern of “1” or “0” from the search keyword using the self-organizing network (step S20).
4) Further, a keyword (hereinafter, approximate keyword) whose characteristic amount is closest to the characteristic amount of the output pattern is selected from the keyword list 13 and input to the database management unit 14 (step S205). The selection of the approximate keyword in this case is not necessarily limited to a single keyword.
A plurality of approximate keywords to be selected may be set in advance. In this way, the process of selecting a keyword that approximates the output pattern is equivalent to the synonym expansion process.

The database management unit 14 selects a DB having a higher importance for the input search keyword or the selected approximate keyword. In particular,
In the keyword list 13, the importance of each database corresponding to the search keyword or the like is compared, and the DB 15 is selected in order from the one with the highest importance (step S206). In the DB selection in this case, the D to be selected
The number of B is set in advance. The database management unit 14 sequentially executes the information search process on the selected DBs and presents the search results to the user through the search result output unit 16 (Steps S207 and S208).

As described above, the information search device 1 of the present embodiment
Then, the target DB is specified and narrowed down using the improved keyword list 13, and the information search process is performed on the narrowed DB.
The load at the time of the search is reduced as compared with the case where the information search process is performed on B15, and the processing time is shortened.

Even if a search keyword that is not included in the keyword list 13 is input, a keyword whose meaning is determined to be approximate can be extracted and handled by the self-organizing network of the keyword learning unit 12.
The search relevance rate is significantly improved over the conventional method. Moreover, since there is no need to manually create a synonym expansion dictionary, maintenance is also simplified.

In the present embodiment, the self-organizing network in the keyword learning unit 12 is adapted to learning at the time of generating the keyword list 13 and detection of an approximate keyword related to a search keyword. For example, the present invention can be similarly applied to other applications such as word processing.

(Second Embodiment) In addition to a stand-alone computer device, the present invention relates to a plurality of computer devices connected in a bidirectional manner via a public network L, for example, an information retrieval device. The present invention can also be implemented as an information search system in which a search server and a client that is an information acquisition device are deployed. In this case, the information search server may be positioned as a search engine for a plurality of large-scale DBs on a network environment. For example, the information search server may be located in a computer device or in an external storage device. And a communication control unit for communicating with the client via the public network L, and an input processing unit 11 and a keyword learning unit 1 similar to the information search device 1.
2, keyword list 13, database management unit 14,
A search result output unit 15 is provided.

This information search server is the information search device 1
The point different from the above is that a communication control unit for performing communication control is provided, and the input processing unit 11 of the information search device 1 receives a search request from a client on the public network L via the communication control unit. Configuration. On the other hand, similarly, the search result from the search result output unit 15 can be replaced by transmitting the search result to the client via the communication control unit, and the same effect as that of the information search device 1 can be obtained. It becomes possible.

[0045]

As is clear from the above description, according to the present invention, a learning function for a keyword is constructed by applying a neural network, and a keyword list is generated. A search process can be performed for a search word, and there is an effect that flexibility relating to the search process is ensured. Further, since the generation of the keyword list 13 and the synonym expansion processing are automatically constructed, manual maintenance is not required, so that the utility and reliability of the system are significantly improved.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of an information search device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an example of the contents of a keyword list 13;

FIG. 3 is a conceptual diagram of processing in a keyword learning unit 12.

FIG. 4 is a processing procedure diagram showing a learning algorithm in a keyword learning unit 12;

FIG. 5 is a processing procedure diagram in the information search device 1.

[Explanation of symbols]

Reference Signs List 1 information retrieval device 11 input processing unit 12 keyword learning unit 13 keyword list 14 database management unit 15 database 15A, 15B, 15C,... 15N D to be searched
B16 search result output section

Claims (8)

[Claims] 1. A keyword is extracted from all digitized information to be searched, the importance of each extracted keyword based on the frequency of appearance in a plurality of information storage areas, and a pattern feature of a character code constituting the keyword. Calculating a list and extracting pattern features of a character code constituting the specified search keyword; and a keyword having a pattern feature whose similarity with the extracted pattern feature is equal to or greater than a predetermined value. And specifying the importance from the list, and executing a search process on the information storage area with the specified keywords as search keywords in order of importance. 2. The pattern feature according to claim 1, wherein the pattern feature is extracted by learning a character code array pattern in a predetermined neural network model so that the information entropy at the time of output is maximized. Information retrieval method. 3. An apparatus for executing a search process based on a search keyword for a plurality of information storage areas storing digitized information, wherein a keyword whose appearance frequency exceeds a predetermined number from each of the plurality of information storage areas. Keyword extraction means for extracting the keyword, importance calculation means for calculating the importance of each information storage area based on the frequency of appearance of the extracted keywords, and feature extraction means for extracting the pattern features of the character codes constituting the extracted keywords. And a keyword list creating means for creating a keyword list in which the calculated importance and the extracted pattern features are associated with each keyword. The search keyword and an information storage area to be searched are based on the created keyword list. An information retrieval apparatus characterized in that the information retrieval apparatus is configured to determine: 4. An apparatus for performing a search process based on a search keyword for a plurality of information storage areas storing digitized information, wherein each of the plurality of keywords extracted from the digitized information has a respective information storage area. A keyword list that lists the importance based on the frequency of occurrence in the keyword and the pattern features of the character codes that constitute the individual keywords; a feature extraction unit that extracts the pattern features of the character codes that make up the specified search keyword; Keyword identifying means for identifying one or more keywords and importance by comparing the identified pattern features with the pattern features in the keyword list, and an information storage area in which the importance is higher as the identified keyword as the search keyword. A search means for performing an information search process on the information search, comprising: Location. 5. A neural network model for learning a configuration of an internal neuron so as to maximize the information entropy of an output pattern in an output layer corresponding to a character code group input to an input layer. 5. The information retrieval apparatus according to claim 3, wherein the information retrieval apparatus comprises: 6. The neural network model determines that an output pattern composed of a predetermined binary pattern of 0 or 1 corresponding to a target keyword is uniquely determined in correspondence with the target keyword. The information retrieval device according to claim 5, wherein 7. A first device having a database in which digitized information is stored, and a second device for sending a search request including a search keyword to the first device and obtaining a search result corresponding to the search request. Are connected so that bidirectional communication is possible, and the first device and the second device further include keyword extraction means for extracting a keyword whose appearance frequency exceeds a predetermined number from each of the plurality of information storage areas. Importance calculating means for calculating importance for each information storage area based on the frequency of appearance of keywords; feature extracting means for extracting pattern characteristics of extracted keywords or character codes constituting search keywords included in the search request; A keyword for creating a keyword list by associating the calculated importance and the extracted pattern features with each of the extracted keywords; And Dorisuto creating means collates extracts a pattern feature of a character code constituting the designated search keyword, the extracted pattern feature and the pattern features in the keyword list 1
Or a keyword specifying unit that specifies a plurality of keywords, and a search unit that performs an information search process on the information storage area having the higher importance using the specified keyword as the search keyword. An information retrieval system characterized by the following. 8. A process for extracting a keyword from all digitized information stored in a plurality of information storage areas, and for each extracted keyword, the importance based on the appearance frequency in each information storage area and the keyword are configured. Processing to calculate and list the pattern features of the character codes to be extracted, processing to extract the pattern features of the character codes constituting the specified search keyword, and patterns in which the degree of similarity with the extracted pattern features is equal to or greater than a predetermined value. The computer device executes a program that causes the computer device to execute a process of specifying a keyword having a characteristic and a degree of importance from the list, and a process of executing a search process on the information storage area with the specified keywords as search keywords in order of importance. A recording medium recorded in a readable form.