JP2000305945A - Information retrieval method and apparatus, and storage medium - Google Patents

Abstract

(57) [Summary] [Problem] To provide an information search method and apparatus capable of knowing whether desired information is present in a search result without checking all information. SOLUTION: An information storage unit 10 is used in a coincidence calculation unit 102a.
The information stored in 3a is compared with the search condition to obtain a degree of coincidence, the similarity calculating unit 102b obtains the degree of similarity of each piece of information of the search result, and the grouping unit 10 is used by using the degree of similarity.
In step 2c, the individual information of the search result is grouped, and the display position of the individual information is determined and displayed in the search result display unit 102c so that the information of the search result belonging to each group is displayed close to each other.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for retrieving information in a computer or the like and a computer-readable storage medium storing a control program for controlling the information retrieving apparatus.

[0002]

2. Description of the Related Art Conventionally, a search condition is given, a storage section storing information to be searched is searched, a degree of matching between each piece of information and the search condition is obtained, and based on the obtained degree of matching,
As a method of displaying information as a search result, Japanese Patent Application Laid-Open
As disclosed in Japanese Patent No. 339375, information of a search result is displayed in a two-dimensional or three-dimensional space, and a search result having a higher degree of matching is displayed closer to the origin of the space, thereby increasing the degree of matching of the search result. A technique of displaying search results in a visually easy-to-understand manner is employed.

[0003]

However, in the conventional example described above, since the display position of the search result is determined only from the viewpoint of the degree of coincidence, is there any desired information in the search result? In order to know whether or not the search result has been obtained, it is necessary to examine individual information of all search results, and there is a problem that the efficiency of evaluating the search results is low.

[0004] The present invention has been made in view of the above-mentioned problems of the prior art, and a first object of the present invention is to check individual information of all search results without examining individual information. An object of the present invention is to provide an information search method and an information search device capable of knowing whether desired information is present in a search result and improving the efficiency of evaluating the search result.

It is a second object of the present invention to provide a storage medium storing a control program capable of smoothly controlling the above-described information retrieval apparatus of the present invention.

[0006]

According to a first aspect of the present invention, there is provided an information retrieval method, comprising the steps of: inputting a retrieval condition and storing an individual information to be retrieved; In an information search method for searching and displaying information satisfying the search condition as a search result, a degree of satisfying the search condition by comparing individual information stored in the information storage unit with the search condition. A matching degree calculating step of obtaining a matching degree, a similarity calculating step of comparing individual information of search results with each other to obtain a similarity degree of individual information, and a search using the similarity obtained in the similarity calculating step. A grouping step of grouping the individual information of the result; and a search result display step of determining and displaying the display position of the individual information so that the information of the search results belonging to each group is displayed close to each other. This The features.

According to a second aspect of the present invention, there is provided an information search method according to the first aspect, wherein information having the highest degree of coincidence in each group is represented by representative information. In the search result display step, a position at which information having a degree of perfect match is displayed is determined as a perfect match display position, and a display distance from the perfect match display position and a degree of match of the representative information are determined. Correspondingly, after determining the display position of the representative information, the display positions of the individual information belonging to the group are determined.

According to a third aspect of the present invention, there is provided an information search method according to the second aspect, wherein the search result display step includes displaying information other than the representative information. The position is determined from the degree of coincidence of individual information and the degree of similarity with the representative information.

According to a fourth aspect of the present invention, there is provided an information search method according to the third aspect, wherein the search result display step is performed by centering on the perfect match display position. The representative information is displayed so as to be radially displayed.

According to a fifth aspect of the present invention, there is provided an information retrieval method as set forth in the first aspect, wherein the grouping step comprises the steps of: A binary relation is created with information having the highest similarity higher than or equal to a specific value and a higher degree of coincidence than this information as group higher-level information, and a tree formed by combining the binary relations is defined as one group, The root of the tree is used as the representative information.

According to another aspect of the present invention, there is provided an information retrieval apparatus, comprising: inputting a search condition; and searching an information storage unit storing individual information to be searched. In an information search device that displays information satisfying the search condition as a search result, individual information stored in the information storage unit is compared with the search condition to determine a matching degree that is a degree satisfying the search condition. Matching degree calculating means, similarity calculating means for comparing the individual information of the search result with each other to obtain the similarity of the individual information, and individualizing the search result using the similarity obtained by the similarity calculating means. Grouping means for grouping information; and search result display means for determining and displaying the display position of individual information so that information of search results belonging to each group is displayed close to each other. That.

According to a seventh aspect of the present invention, there is provided an information retrieval apparatus according to the sixth aspect, wherein information having the highest degree of coincidence in each group is represented by representative information. The search result display means determines a position for displaying information having a perfect match degree as a perfect match display position, and determines a display distance from the perfect match display position and the degree of match of the representative information. Correspondingly, after determining the display position of the representative information, the display positions of the individual information belonging to the group are determined.

In order to achieve the first object, an information retrieval apparatus according to claim 8 is the information retrieval apparatus according to claim 7, wherein the retrieval result display means displays information other than the representative information. The position is determined from the degree of coincidence of individual information and the degree of similarity with the representative information.

According to a ninth aspect of the present invention, in order to achieve the first object, the information retrieval apparatus according to the eighth aspect, wherein the retrieval result display means is arranged so as to center on the perfect match display position. The representative information is displayed so as to be radially displayed.

According to a tenth aspect of the present invention, in order to achieve the first object, in the information search apparatus according to the sixth aspect, the grouping means is configured to perform the above-described information processing on each information. A binary relation is created with information having the highest similarity higher than or equal to a specific value and a higher degree of coincidence than this information as group higher-level information, and a tree formed by combining the binary relations is defined as one group, The root of the tree is used as the representative information.

In order to achieve the second object, a storage medium according to the present invention is characterized in that a search condition is input, and an information storage section storing individual information to be searched is searched for. In a storage medium storing a control program (information search program) for controlling an information search device that displays information satisfying a search condition as a search result, the control program includes, for each of the information stored in the information storage unit, A search condition is compared to determine a degree of matching that is a degree that satisfies the search condition. Individual information items of the search result are compared with each other to obtain a similarity value of each information item. Control group in which information of search results belonging to each group is grouped and the display position of each information is determined and displayed so that the information is displayed close to each other. It characterized by having a Lumpur.

In order to achieve the second object, the storage medium according to claim 12 is the storage medium according to claim 11, wherein the control program has information having the highest matching degree in each group. As the representative information, a position for displaying information having a perfect match degree is determined as a perfect match display position, and a display distance from the perfect match display position is associated with the degree of match of the representative information, After the display position of the representative information is determined, a control module of a step of controlling so as to determine the display position of the individual information belonging to the group is provided.

In order to achieve the second object, a storage medium according to claim 13 is the storage medium according to claim 12, wherein the control program determines a display position of information other than the representative information. And a control module for performing control so as to determine from the degree of coincidence of the information and the degree of similarity with the representative information.

According to a fourth aspect of the present invention, there is provided a storage medium according to the second aspect, wherein the control program is radially displayed around the perfect match display position. A control module for controlling the display so that the representative information is displayed.

Further, in order to achieve the second object, a storage medium according to claim 15 is the storage medium according to claim 11, wherein the control program is provided for each information. A binomial relation is created by using information having the highest similarity higher than a specific value and having a higher degree of coincidence than the information as a group higher-level information, and a tree formed by combining the binomial relations is defined as one group. A control module for controlling the route to be the representative information is provided.

[0021]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) First, the first embodiment of the present invention will be described.
The embodiment will be described with reference to FIGS.

FIG. 1 is a conceptual diagram of an information search apparatus according to a first embodiment of the present invention. In the figure, an information search apparatus includes an input unit 101, a control unit 102, a storage unit 103, and a display unit. 104.

The input unit 101 is composed of a keyboard provided with character keys, function keys, and the like, and inputs search conditions to the control unit 102. The control unit 102 is composed of a microprocessor, and includes a coincidence calculation unit 102a, a similarity calculation unit 102b, a search result display unit 102c, and a grouping unit 1.
02d. Then, the individual information of the search target stored in the information storage unit 103a of the storage unit 103 is compared with the search condition, and the matching degree calculating unit 102a calculates the matching degree between the individual information and the query, and the similarity calculating unit At 102b, the similarity of individual information of the search result is calculated, and at the grouping unit 102d, the information of the search result is grouped based on the similarity of individual information, and the search result display unit 102
The information grouped in c is output to the display unit 104 and displayed so that the information is displayed close to each other.

FIG. 2 is a system configuration diagram of the information search device according to the first embodiment of the present invention.
201 is a CPU (Central Processing Unit), 202 is a ROM
(Read only memory), 203 is RAM (random access memory), 204 is CD-ROM (compact disk read only memory) drive, 205 is CD-ROM (compact disk read only memory), and 206 is HD (hard disk) Drive, 2
07 is a keyboard, 208 is a display, 209 is a mouse, 210 is a printer, and 211 is a control bus.

The ROM 202 stores a control procedure program 401 as shown in FIG. Also, ROM
203, an information search program (control program) 603, an image database 604, coincidence area data 605, coincidence image data 606, radius increment data 607, arrangement angle data 608, similar image buffer 609, similar image as shown in FIG. Information 610, distribution information 611, group information 612, screen information 613, and the like are stored.

FIG. 3 shows that the program and the like are stored in a storage medium C.
FIG. 4 is a diagram showing that the data is supplied from the D-ROM 205 to the computer system. In FIG.
A ROM drive, 205 is a CD-ROM, and 300 is a computer system.

The CD-ROM 205 has an information retrieval program (control program) 501 as shown in FIG. 5 and image data base 502, matching area data 503, matching image data 504, and radius increment data 5 which are related data.
05 and arrangement angle data 506 are stored.

FIG. 6 shows an information retrieval program 501 stored in the CD-ROM 205 and related data such as an image database 502, coincidence area data 503, coincidence image data 504, radius increment data 505, and arrangement angle data 506. FIG. 3 is a diagram showing a memory map in a state where the memory map is loaded into a RAM 203 and becomes executable. In an executable state, the information search program 603, the image database 604, the matching area data 605, the matching image data 606, the radius increment data 607, and the arrangement angle data 6
08, the similar image buffer 609 and the similar image information 61 are used as memories used by the information search program 603.
Areas of 0, distribution information 611, group information 612, and screen information 613 are secured and initialized. In FIG. 6, reference numeral 600 denotes a memory map, and 601 denotes a basic I / O.
An (input / output) program 602 is an OS (operating system).

FIGS. 7, 8, and 9 are diagrams for explaining an example of the configuration of a screen displayed on the display 208 by the information search program 603 and an operation for performing an image search using the information search program 603. FIG.

In FIG. 7, reference numeral 701 denotes a query box (text box) for inputting a query, showing a state in which “red apple” is input as a query. A search button 702 is used to instruct a search.
By clicking the search button 702 using the mouse 209 (see FIG. 2), the image database 604 (see FIG. 6) is searched by the query stored in the query box 701, and the search result is displayed in the search result display area 70.
3 is displayed. This search result display area 703 is
This is a square area of 00 dots and 600 dots horizontally. The squares 704 and 705 are the searched images. The size of each image 704, 705 is 60 dots vertically,
It is displayed with 80 horizontal dots. 7 to 9 do not show actual images for convenience of explanation.

In FIG. 7, a numeral such as “ID: 234” in the square indicates that the image ID of the image 704 is “234”. A matching area 706 indicated by a hatched circle is for displaying an image that completely satisfies the search condition. The image displayed in the matching area 706 is an image that completely satisfies the search condition. Reference numerals 707 and 708 indicated by broken-line circles are scales for indicating the degree of satisfying the search condition. In the present embodiment, the degree of satisfying the search condition is represented by the similarity. When the search condition is completely satisfied, the similarity is “1.0”. When the search condition is not satisfied at all, the similarity is “0”. A scale 707 is a scale indicating the similarity “0.9”, and a scale 708 is a scale indicating the similarity “0.8”.

The matching area 706 indicates that the similarity is "1.0". In the present embodiment, the scales 707 and 708 are displayed at intervals of the similarity “0.1”. As can be seen, the similarity decreases as the distance from the center of the search result display area 703 increases. That is,
An image having a higher degree of satisfying the search condition is displayed closer to the center of the search result display area 703, and an image having a lower degree of satisfying the search condition is displayed farther from the center of the search result display area 703. The image IDs are “567” and “7
The images of “65” and “851” are displayed overlapping little by little. This indicates that these images are similar to each other and form one group.
The image 705 and the image 711 are connected by a straight line 710. Hereinafter, this straight line 710 is referred to as a group connection line.

Thus, even if they do not overlap, the images connected by a straight line are similar, indicating that one group is formed. In each group, a straight line 709 always extends from one image toward the center of the search result display area 703. Hereinafter, this straight line 709 is described as a representative line. Also, this representative line 709
An image having is described as a representative image. For example, the image 711
Is a representative image and has a representative line 709. The representative image indicates that the similarity is the highest in the group.

In each group, the center distance between the representative image and the other images indicates the degree of similarity with the representative image. When the similarity with the representative image is high, the group connection line 710 is not displayed as in the case of the images with the image IDs “567”, “765”, and “851”, and the images are displayed in an overlapping manner. Here, as shown in the image 704, even one image is regarded as forming a group,
A representative line 709 is displayed.

A previous screen button 712 is an operation button for displaying an image having a higher similarity than the image displayed in the search result display area 703. FIG. 7 shows the first search result display area 703 when a search is performed for the query “red apple”, so that there is no image having a higher similarity than the image displayed here,
The triangle of the previous screen button 712 is displayed as a white background.
A next screen button 713 is an operation button for displaying an image having a lower similarity than the image displayed in the search result display area 703. In FIG. 7, since the next screen exists, the triangle of the next screen button 713 is displayed as a black background.

FIG. 8 is a diagram showing the display state of the display 203 (see FIG. 2) after clicking the next screen button 713 using the mouse 209 (see FIG. 2) in the state of FIG. In the figure, 801 802 803
Indicates the similarities “0.8”, “0.7”, “0.
6 ". Scale 802 and scale 803
Is smaller than the distance between the scales 801 and 802 because there is no image displayed in the area between the two scales 802 and 803. Further, the interval between the scales 804 inside the scale 801 and the size of the matching area 706 are smaller than those in FIG. 7 because no image is displayed therein.

In FIG. 8, the previous screen button 712 is
Since there is an image with a high similarity, the triangle is displayed on a black background, but the next screen button 713 has no image with a lower similarity, and the triangle is displayed on a white background.

FIG. 9 shows an image 711 in the state of FIG.
FIG. 11 is a diagram showing a display state of the display 203 (see FIG. 2) after clicking on the mouse with the mouse 209 (see FIG. 2). When an image displayed in search result display area 703 is clicked using mouse 209 (see FIG. 2), an image similar to the clicked image is searched, and the search result is displayed in search result display area 703. You. In FIG. 9, the query box 701 is cleared because there is no query corresponding to the search result. Instead, the matching area 901 contains the image 711 from which the search was performed.
Is displayed. At this time, the image 711 is displayed in a double frame in order to distinguish it from the case where a search is performed by a query. Further, the intervals between the scales 902, 903, 904, and 905 are smaller than those in the state of FIG.

FIG. 10 is a diagram showing the configuration of the image database 604 (see FIG. 6) read into the RAM (see FIG. 2). The image database 604 stores a plurality of items, each of which corresponds to the image ID and the name and description of the image file corresponding to the image ID, as one item. For example, in FIG. 10, as one item, the image ID “0”, the image file “00000000.jpg”, and the description “a red
"apple" is stored correspondingly.

The image ID is a number for identifying an image stored in the image database 604, and a number is assigned to the image in ascending order from “0” in the order stored in the image database 604. is there. Therefore, the maximum value of the image ID stored in the image database 604 matches the value obtained by subtracting “1” from the number of images stored in the image database 604.

The image file is stored in the image database 604.
Indicating the storage location of the file of the image data stored in. Therefore, by obtaining image data from the image file, the image is displayed on the display 208 (FIG. 2).
Reference).

The explanation is a sentence describing the corresponding image, and the information search program 603 (see FIG. 6) uses this explanation to search the image database 604 for a desired image. Each item is sorted in ascending order by image ID.

FIG. 11 is a diagram showing the structure of the matching area data 605 (see FIG. 6) read into the RAM 203 (see FIG. 2). The matching area data 605 includes the matching area 7
06 (see FIG. 7 and FIG. 8).
The data for changing in accordance with the number of images to be displayed are stored. The matching area data 605 is a one-dimensional array as shown in FIG. 11, and the number of images displayed in the matching area 706 corresponds to a subscript of the array, and the number of dots indicating the radius of the matching area 706 as an element of the array Is stored. The correspondence between the number of images displayed in the matching area 706 and the size of the radius of the matching area 706 is as follows.
Are determined in advance so that the images displayed in the images can be displayed without overlapping each other. For example, from FIG.
If the number of images displayed in the area 6 is zero, the matching area 70
It can be seen that the radius of 6 is 30 dots, one is 60 dots, two is 90 dots, and three is 120 dots.

FIG. 12 is a diagram showing the structure of the coincident image data 606 (see FIG. 6) read into the RAM 203 (see FIG. 2). The matching image data 606 includes the matching area 7
The coordinates of the center of the image displaying the image of similarity “1.0” displayed in the area 06 are stored in association with the number of images displayed in the matching area 706. That is, the matching area 706
One coordinate when displaying one image, two coordinates when displaying two images, three coordinates when displaying three images, and so on. Match region 706
, The same number of coordinates are stored in correspondence with the number of images to be displayed.

Each coordinate is stored with the origin at the center of the matching area 706, the x-axis in the horizontal direction on the right side of the screen, and the y-axis in the vertical direction above. The coordinates of each image are determined in advance so that the coordinates displayed in the matching area 706 are displayed without overlapping each other. For example, FIG.
2, the position of coordinates (0, 0) when displaying one image in the matching area 706, and the position of coordinates (0, 35) and (0, -35) when displaying two images. To display three images, coordinates (0, 35), (45, -3)
5) It can be seen that information to be displayed is stored at positions (−45, −35).

FIG. 13 is a diagram showing the configuration of the radius increment data 607 (see FIG. 6) read into the RAM 203. The radius increment data 607 is used to determine the interval between the scales 707 and 708. Scale interval is 2
It is determined by the number of images displayed between two scales. Accordingly, the radius increment data 607 stores a plurality of items corresponding to the number of images displayed between the scales and the intervals between the scales, and the items are sorted in ascending order according to the number. For the scale interval, the increment of the radius from the scale which is larger by 0.1 than the target scale is stored as the number of dots. For example, from FIG.
If the number of images is 1 or more and 4 or less, a scale with a larger radius of 30 dots is displayed.
It is understood that a scale with a larger radius of 60 dots is displayed, and when the number of images is 5 or more and 10 or less, a scale with a larger radius of 90 dots may be displayed.

FIG. 14 is a diagram showing the configuration of the arrangement angle data 608 (see FIG. 6) stored in the RAM 203.
The arrangement angle data 608 is used to determine the position of the representative image displayed in the search result display area 703. The arrangement angle data 608 defines the angle at which the representative image is displayed when the angle in the horizontal right direction from the center of the search result display area 703 is 0 degree. Arrangement angle data 608
Is a one-dimensional array in which the display angles of the representative images are stored in accordance with the order in which the representative images are displayed from the top of the array. For example, from FIG. 14, the representative image displayed first is displayed in the direction of 90 degrees, the second is displayed in the direction of 270 degrees, the third is displayed in the direction of 180 degrees, and the fourth is displayed in the direction of 0 degrees. You can see that.

FIG. 15 is a diagram showing the configuration of the similar image buffer 609 (see FIG. 6) stored in the RAM 203. The similar image buffer 609 has an image database 6
04 (see FIG. 6) is stored. The image ID and the similarity are stored in association with one image as a result of the search. Each searched image is stored in descending order according to the similarity. As described above, the degree of similarity is “1.
Taking a value from "0" to "0.0", a larger value means a higher similarity, indicating that "1.0" is a perfect match.

From FIG. 15, image IDs "0", "456",
The three images “789” are searched for the similarity “1.0”, and the image with the image ID “123” is searched for the similarity “0.952”
You can see that it was searched by.

FIG. 16 is a diagram showing the structure of the similar image information 610 (see FIG. 6) stored in the RAM 203. For all the images stored in the image database 604 (see FIG. 6), the similarity with other images is calculated, and the obtained similarity is stored as a similar image list in association with each image. Each similar image list, as shown in FIG.
The similar image information 610 is stored in ascending order from the top of the similar image information 610 in order from the similar image list for the image ID “0”. The structure of each similar image list is similar to the similar image buffer 6 shown in FIG.
The structure is exactly the same as that of the structure 09.

FIG. 17 is a diagram showing the structure of the distribution information 611 (see FIG. 6) stored in the RAM 203. The distribution information 611 is for storing the distribution of the number of images existing between each scale when displaying the search result.

In FIG. 17, (T101) and (T1)
02) is distribution information when a search is performed using “red apple” as a query, distribution information when an image similar to image ID “999” is searched, and image ID “999”.
9 shows distribution information when an image similar to is searched.
From (T101), there are three images having the similarity “1.0”, five images having the similarity smaller than “1.0” and equal to or more than “0.9”, and the similarity is “0.9”. Smaller "0.
It can be seen that there are eight images of “8” or more.

FIG. 18 is a diagram showing the structure of the group information 612 (see FIG. 6) stored in the RAM 203. This group information 612 is used to group images into groups. As the images stored in the group information 612, images other than the representative image are stored.
The image ID of the representative image of the group to which the image belongs and the similarity between the image and the representative image are stored in correspondence.
For example, from FIG. 18, the image with the image ID “345” belongs to the group represented by the image ID “876”, and the image I
It can be seen that the similarity between the image of D “345” and the representative image of image ID “876” is “0.928”. Each image stored in the group information 612 is a representative image ID.
Are stored in ascending order.

FIG. 19 is a diagram showing the structure of the screen information 613 (see FIG. 6) stored in the RAM 203. The screen information 613 displays the search result display area 703 for the search result.
For storing the information of each screen to be displayed in the
The number of screens required to display all search results, the range of images to be displayed on each screen, the layout of scales on each screen, and the like can be obtained.

In FIG. 19, (T201) and (T2)
02) indicates screen information when a search is performed using “red apple” as a query and screen information when an image similar to the image ID “999” is searched. In the screen information, the subscripts of the array of the distribution information 611 (see FIG. 6) for indicating the image to start displaying and the scale width of the scale are stored corresponding to each screen. The scale width of the scale starts from the radius of the matching area 706 (see FIGS. 7 and 8), that is, the scale of the similarity “1.0”, and the width between adjacent scales, that is, the difference in radius is the number of dots. Are stored in order. "-1" for scales not displayed next to the screen
Is stored. For example, from (T201), the first screen may start displaying an image corresponding to the suffix “0” of the array of the distribution information 611 (see FIG. 6), that is, an image having a similarity of “1.0”. I understand. Further, the size of the radius of the matching area 706 on the first screen is 120 dots, and the similarity is “0.
The scale of "9" has a radius 90 dots larger than the matching area 706, the scale of the similarity "0.8" has a radius 90 dots larger than the scale of the similarity "0.9",
It is understood that the scale after the similarity “0.7” is not displayed. Further, it can be seen that the next screen starts display from an image corresponding to the subscript “3” of the array of the distribution information 611, that is, an image smaller than the similarity “0.8”. In the first screen, the size of the radius of the matching area 706 is 30 dots, and the scale of the similarity “0.9” is larger than the size of the matching area 706 by 30 dots, and the scale of the similarity “0.8”. Makes the radius 30 dots larger than the scale of the similarity “0.9”, the scale of the similarity “0.7” makes the radius 90 dots larger than the scale of the similarity “0.8”,
The scale of the similarity “0.6” has a radius 90 dots larger than the scale of the similarity “0.7”, and the similarity “0.5”.
Has a radius of 9 more than the scale of the similarity “0.6”.
0 dots larger and the scale after the similarity “0.4”
You can see that it is not displayed.

Next, the operation of the information search program 501 (see FIG. 5) will be described with reference to the flowchart of FIG.

First, an image database 502 of the information search program 501 and its related data stored in the CD-ROM 205 (see FIG. 2), a matching area data 503,
Radius increment data 505 and arrangement angle data 506 are CD-
The data is loaded from the ROM drive 204 to the RAM 203, and areas for the similar image buffer 609, similar image information 610, distribution information 611, group information 612, and screen information 613 are secured in the RAM 203 (step S200).
1).

Next, the control is shifted to the information search program 501 (see FIG. 5),
Performs initialization necessary for performing a search (step S
2002). That is, the image database 604 is searched using all the individual images stored in the image database 604, and the search result is referred to as the similar image information 6.
10 is stored. The details of this processing will be described later with reference to FIG.

Next, in step S2003, the processing branches according to the event that occurs. That is, keyboard 2
If an event of pressing a character key using 07 (see FIG. 2) occurs, the flow branches to step S2004. Further, an OK button 702 shown in FIG.
If an event of clicking using (see) occurs, the flow branches to step S2005. If an event of clicking the image 704 or 705 shown in FIG. 7 using the mouse 209 occurs, the process proceeds to step S200
Branch to 6. If the event that the next screen button 712 shown in FIG. 7 is clicked using the mouse 209 occurs, the process branches to step S2007. Further, if an event occurs that the previous screen button 713 shown in FIG. 7 is clicked using the mouse 209, the process proceeds to step S2008.
Branch to

Step S2004 is processing when a query is input. The character pressed in step S2003 is fetched, and the query box 70 shown in FIG.
1 is displayed. After this processing is completed, the process proceeds to step S20.
Return to 03.

Step S2005 is a process performed when an instruction is given to perform a search using the query displayed in the query box 701. Step S200
In 5, the process branches depending on whether or not a query has been input to the query box 701. That is, the query box 70
If a character exists in 1, it is possible to search for an image using a query. Therefore, the process proceeds to step S2009 to perform a search using the query, and then returns to step S2003. If no character exists in the query box 701, an image cannot be searched using a query. Therefore, the process proceeds to step S2010, where a search is performed using a randomly selected image, and then the next step S2011 is performed.
Proceed to.

Step S2006 is a process in a case where an instruction to search using a specific image is issued.
Here, the image ID corresponding to the clicked image is obtained and stored in the comparison source ID. The comparison source ID is RAM 203
(See FIG. 2). After this processing ends, the flow proceeds to step S2011.

Step S2007 is a process in the case where an instruction to display the next screen of the search result is issued. Details of this processing will be described later with reference to FIG. After this process ends, the process returns to the step S2003.

Step S2008 is a process in the case where an instruction to display the screen before the search result is given. Details of this processing will be described later with reference to FIG. After this processing ends, the flow returns to step S2003.

In step S2009, the image database 604 is searched using the character string stored in the query box 701, and the search result is displayed on the display 20.
8 (see FIG. 2). Details of this processing will be described later with reference to FIG. After this processing ends, the flow returns to step S2003.

In step S2010, an image is randomly selected from the image database 604, and the image ID for the selected image is stored in the comparison source ID.
The comparison source ID is secured in the RAM 203. After this processing ends, the flow proceeds to step S2011.

Step S2011 is performed in step S2.
006 or a process of displaying an image similar to the image selected in step S2010 on the display 208. The similar image is already stored in the similar image buffer 609 in step S2006 or step S2010.
, The process of searching for particularly similar images is not performed. Details of this processing will be described later with reference to FIG. After this processing is completed, step S20 is performed.
Return to 03.

FIG. 21 is a flowchart showing the operation in step S200 of FIG.
3 is a flowchart showing a flow of a processing operation of search initialization in FIG. In this processing, various necessary initializations are performed.
Further, a similar image is searched for all the individual images stored in the image database 604, and the search result is stored in the similar image information 610. Hereinafter, the details of this process will be described.

First, various initialization processes are performed in step S2101. That is, the similar image buffer 609, the similar image information 610, and the distribution information 61 secured in the RAM 203.
1. Initialize information such as group information 612 and screen information 613. Further, other initialization is performed as necessary. After finishing this process, the flow advances to the next step S2102.

In step S2102, an initial value “0” is stored in the comparison source ID indicating the comparison source image. Comparison source ID
Are secured in the RAM 203. After this processing ends, the flow proceeds to the next step S2103.

In step S2103, a description of the comparison source image is stored in the query buffer in order to search for an image similar to the comparison source image. A description corresponding to the comparison source ID is obtained from the image database 604 and copied to the query buffer. The query buffer is secured in the RAM 203. After this processing ends, the next step S210
Proceed to 4.

In step S2104, an initial value "0" is stored in the comparison destination ID indicating the image of the comparison destination. Comparison destination ID
Are secured in the RAM 203. After this processing ends, the flow proceeds to the next step S2105.

In step S2105, the similarity between the image indicated by the comparison source ID and the image indicated by the comparison destination ID is calculated. The degree of similarity is calculated from the description for the image corresponding to the comparison source ID and the description for the image corresponding to the comparison destination ID. The description for the image corresponding to the comparison source ID is stored in the query buffer. The description for the image corresponding to the comparison destination ID can be obtained from the image database 604. Since the method of obtaining the similarity between two sentences is generally performed in the field of information retrieval, a detailed description thereof will not be given here, but the maximum value of the similarity is set to “1.0” and the minimum Normalization is performed so that the value becomes “0.0”. When the similarity is obtained, the process proceeds to the next step S2106.

In step S2106, it is determined whether or not the image indicated by the comparison destination ID can be stored in the similar image buffer 609, and the process branches depending on the determination result. If the similar image buffer 609 is not full, it is determined that the image can be stored unconditionally. If the similar image buffer 609 is full, the similarity value of the last item in the similar image buffer 609 is compared with the similarity value obtained in step S2105, and the process proceeds to step S2.
If the value of the similarity obtained in 105 is larger, it is determined that the data can be stored, and if it is not larger, it cannot be stored. If it is determined that it can be stored, step S21
07, if it is determined that storage is not possible, step S2
Go to 108 respectively.

In step S2107, the comparison destination ID is stored in the similar image buffer 609 in association with the similarity calculated in step S2105. Each item in the similar image buffer 609 is stored so as to be sorted in descending order by similarity. Similar image buffer 609
Is full, the last item in the similar image buffer 609 is discarded. After this processing is completed, the next step S2
Proceed to 108.

In step S 2108, the value of the comparison destination ID is counted up by “1”, and the comparison destination image is changed to the next image in the image database 604. After this processing ends, the flow proceeds to the next step S2109.

Step S2109 is processing for branching depending on whether or not all the images in the image database 604 have been compared with the image of the comparison source. The last image I in which the value of the comparison destination ID is stored in the image database 604
If the value is larger than the value of D, it means that all the images have been compared, and the process proceeds to the next step S2110. If the value of the comparison destination ID is not larger than the value of the last image ID stored in the image database 604, the process returns to step S2105.

In step S 2110, the value of the comparison source ID is counted up by “1”, and the comparison source image is changed to the next image in the image database 604. After this processing ends, the flow proceeds to the next step S2111.

Step S2111 is processing to branch depending on whether or not images similar to all the images in the image database 604 have been obtained. The last image I in which the value of the comparison source ID is stored in the image database 604
If the value is larger than the value of D, it means that the processing has been performed on all the images, and thus the processing operation ends. If the value of the comparison source ID is not larger than the value of the last image ID stored in the image database 604, the process proceeds to step S2103.

FIG. 22 is a flowchart showing the operation in step S200 of FIG.
9 is a flowchart illustrating the flow of a query search processing operation of FIG. In this processing, the image database 604 is searched using the query stored in the query box 701, and the search result is displayed on the display 208. Hereinafter, the details of this process will be described.

At step S2201, the query box 70
Copy the character string stored in 1 into the query buffer. The query buffer is secured in the RAM 203. After this processing ends, the flow proceeds to the next step S2202.

In step S2202, the initial value “0” is stored in the comparison destination ID indicating the image of the comparison destination. Comparison destination ID
Are secured in the RAM 203. After this processing ends, the flow proceeds to the next step S2203.

In step S2203, the similarity between the sentence stored in the query buffer and the description of the image corresponding to the comparison destination ID is calculated. The description for the image corresponding to the comparison destination ID can be obtained from the image database 604. The technique for finding the similarity between two sentences is
Since it is generally performed in the field of information search, a detailed description is not given here, but the maximum similarity is set to “1.0” and the minimum similarity is set to “0.0”. Is normalized. After the similarity is obtained, the next step S220
Proceed to 4.

Step S2204 is a process of determining whether or not the image indicated by the comparison destination ID can be stored in the similar image buffer 609, and branching based on the determination result. If the similar image buffer 609 is not full, it is determined that the image can be stored unconditionally. If the similar image buffer 609 is full, the similarity value of the last item in the similar image buffer 609 is compared with the similarity value obtained in step S2203, and the process proceeds to step S2203.
If the value of the similarity obtained in 203 is larger, it is determined that the data can be stored. Also, it is determined that the data cannot be stored unless it is large. If it is determined that the image can be stored, the process proceeds to step S2205. If it is determined that the image cannot be stored, the process proceeds to step S2206.

In step S2205, the comparison ID is stored in the similar image buffer 609 in association with the similarity calculated in step S2203. Each item in the similar image buffer 609 is stored in descending order according to similarity. When the similar image buffer 609 is full, the last item in the similar image buffer 609 is discarded. After this processing ends, the next step S220
Proceed to 6.

In step S2206, the value of the comparison destination ID is counted up by “1”, and the comparison destination image is changed to the next image in the image database 604. After this processing ends, the flow proceeds to the next step S2207.

Step S2207 is processing to branch depending on whether all the images in the image database 604 have been compared with the query. If the value of the comparison destination ID is larger than the value of the last image ID stored in the image database 604, it means that all the images have been compared, and the process proceeds to the next step S2208. Also, the comparison destination ID
If the value of is not larger than the value of the last image ID stored in the image database 604, the process returns to the step S2203.

In step S 2208, a screen configuration for displaying the search result is determined from the search results stored in the similar image buffer 609, and information on the determined screen configuration is output to the screen information 613. This is the process to do.
Details of this processing will be described later with reference to FIG. After this processing ends, the flow proceeds to step S2209.

Step S2209 is processing for substituting "0" for the screen number in order to display the first screen of the search result. The screen number is secured in the RAM 203. After this processing ends, the flow proceeds to step S2210.

Step S2210 is a process of displaying the search result on the display 208 based on the screen configuration stored in the screen information 613 and the search result stored in the similar image buffer 609. The details of this processing are shown in FIG.
Will be described later. After this processing ends, this processing operation ends.

FIG. 23 is a flowchart showing step S201 in FIG.
3 is a flowchart showing a flow of a similar search processing operation in FIG. In this processing, an image similar to the image stored in the comparison source ID is obtained from the similar image information 610,
A similar image is displayed on the display 208. Hereinafter, the details of this process will be described.

In step S2301, the character string stored in the query buffer is cleared, and the query box 7
01 is cleared and displayed on the display 208. After this processing ends, the flow proceeds to the next step S2302.

In step S2302, similar image information 6
A similar image list for the image ID stored in the comparison source ID is obtained from 10, and the contents of the similar image list are copied to the similar image buffer 609. After this processing ends, the flow proceeds to step S2303.

In step S 2303, a screen configuration for displaying the search result is determined from the search results stored in the similar image buffer 609, and information on the determined screen configuration is output to the screen information 613. This is the process to do. Details of this processing will be described later with reference to FIG. After this processing ends, the flow proceeds to step S2304.

Step S2304 is a process of substituting "0" for the screen number in order to display the first screen of the search result. The screen number is secured in the RAM 23. After this processing ends, the flow proceeds to step S2305.

In step S2305, the search result is displayed on the display 2 based on the screen configuration stored in the screen information 613 and the search result stored in the similar image buffer 609.
08. Details of this processing will be described later with reference to FIG. After this processing ends, this processing operation ends.

FIG. 24 is a flowchart showing step S200 in FIG.
7 is a flowchart showing the flow of the processing operation of the next screen display in 7. In this processing, the next screen of the search result displayed on the display 208 is displayed.
Hereinafter, the details of this process will be described.

In step S2401, the process branches depending on whether the next screen exists. The item corresponding to the value obtained by adding “1” to the value stored in the screen number is the screen information 61.
If it exists in step S3, it is determined that the next screen exists, and the flow advances to the next step S2402. The item corresponding to the value obtained by adding “1” to the value stored in the screen number is the screen information 6.
If it does not exist in step 13, it is determined that the next screen does not exist, and this processing operation ends. Whether an item corresponding to a value obtained by adding “1” to the value stored in the screen number exists in the screen information 613 is determined by determining whether the display start value of the corresponding item is “−”.
"1". If the display start value of the corresponding item is “−1”, it is determined that the next screen does not exist, and the display start value of the corresponding item is “−”.
If it is not "1", it is determined that the next screen exists.

Step S2402 is processing for adding "1" to the value of the screen number in order to display the next screen currently displayed. After this processing is completed, the next step S
Proceed to 2403.

Step S2403 is a process for displaying the screen indicated by the screen number on the display 208 based on the screen configuration stored in the screen information 613 and the search result stored in the similar image buffer 609. Details of this processing will be described later with reference to FIG. After this processing ends, this processing operation ends.

FIG. 25 is a flowchart showing step S200 in FIG.
9 is a flowchart showing the flow of a previous screen display processing operation in FIG. In this processing, the screen before the search result displayed on the display 208 is displayed.
Hereinafter, the details of this process will be described.

In step S2501, the process branches depending on whether a previous screen exists. If the value of the screen number is not “0”, it is determined that the previous screen exists, and the process proceeds to the next step S2502. If the value of the screen number is "0", it is determined that the previous screen does not exist, and this processing operation is terminated.

In step S2502, the value of the screen number is set to "1" in order to display the previous screen currently displayed.
This is the process of counting down only. After this processing ends, the flow proceeds to the next step S2503.

Step S2503 is a process for displaying the screen indicated by the screen number on the display 208 based on the screen configuration stored in the screen information 613 and the search result stored in the similar image buffer 609. Details of this processing will be described later with reference to FIG. After this processing ends, this processing operation ends.

FIG. 26 is a flowchart showing step S220 in FIG.
24 is a flowchart showing the flow of the processing operation for determining the screen configuration in step S2303 of FIG. 8 and FIG. 23. In this processing, the configuration of the screen on which the search result is displayed is determined based on the information stored in the similar image buffer 609, and the information on the configuration of the screen is output to the screen information 613. Hereinafter, the details of this process will be described.

In step S2601, the distribution status for the similarity of the images stored in the similar image buffer 609 is checked, and the distribution status is stored in the distribution information 611.
First, the number of image IDs having a similarity of “1.0” is obtained from the similar image buffer 609 and stored in the distribution information 611.
Next, the number of image IDs having a similarity smaller than “1.0” and equal to or larger than “0.9” is obtained from the similar image buffer 609 and stored in the distribution information 611. Hereinafter, similarly, "0.1"
At every step, the number of image IDs stored in the similar image buffer 609 is obtained and stored in the distribution information 611. After this processing ends, the flow proceeds to the next step S2602.

Step S2602 is a process of grouping the images stored in the similar image buffer 609 according to their similarity and storing the grouped result in the group information 612. The details of this processing are shown in FIG.
7 will be described later. After this processing ends, the flow proceeds to the next step S2603.

In step S2603, the scale to be displayed on each screen is determined, and the determined result is displayed on screen information 6
13 is stored. Details of this processing will be described later with reference to FIG. After this processing ends, this processing operation ends.

FIG. 27 is a flowchart showing step S260 in FIG.
3 is a flowchart showing a flow of a grouping processing operation in FIG. Hereinafter, the details of this process will be described.

In step S2701, "0" is substituted for the buffer item indicating each item in the similar image buffer 609 so that the buffer item indicates the first item in the similar image buffer 609. Buffer item is RAM
203. After this processing ends, the flow proceeds to the next step S2702.

In step S2702, a similar image list corresponding to the image ID in the similar image buffer 609 indicated by the buffer item is obtained from the similar image information 610. As shown in FIG. 16, the similar image information 610 has an array of similar image lists, and the subscript of the array is an image ID, so that a corresponding similar image list can be easily extracted. After this processing is completed, the next step S
Proceed to 2703.

In step S2703, step S270
In step 2702, “0” is substituted for the list item indicating each item in the extracted similar image list, so that the list item indicates the first item in the similar image list.
List items are secured in the RAM 203. After this processing ends, the flow proceeds to the next step S2704.

In step S2704, the process branches depending on whether or not the similarity of the item in the similar image list indicated by the list item is equal to or greater than a specified value. If the value is equal to or more than the specified value, the process advances to step S2705. If the value is equal to or less than the specified value, the process advances to step S2709. The specified value is set to, for example, “0.8”.

In step S2705, the flow branches depending on whether or not the image ID indicated by the list item is present in an item in the similar image buffer 609 at the head of the item indicated by the buffer item. That is, if there is an image ID that matches the image ID of the item in the similar image list indicated by the list item among the items in the similar image buffer 609 corresponding to a value smaller than the value of the buffer item, the process proceeds to step S2706. , If it does not exist, step S2707
Proceed to.

In step S2706, the image ID corresponding to the buffer item and the image ID corresponding to the list item
Is stored in the group information 612. Image ID corresponding to the buffer item
In the storage area of the image ID of the group information 612, the image ID corresponding to the list item in the storage area of the representative image ID of the group information 612, and the similarity corresponding to the list item in the storage area of the similarity of the group information 612. , Respectively. After this processing ends, the flow proceeds to step S2709.

In step S2707, the value of the list item is counted up by "1", and
The next item in the similar image list extracted in step 2 is pointed to. After this processing is completed, the next step S
Proceed to 2708.

In step S2708, the process branches depending on whether the item indicated by the list item is valid. If it is not valid, the process has been performed on all valid items in the similar image list extracted in step S2702, and the process advances to step S2709. If it is valid, it is still necessary to process it,
Return to 2704.

In step S2709, the value of the buffer item is counted up by "1" and the similar image buffer 6
Point to the next item in 09. After this processing ends, the flow proceeds to the next step S2710.

In step S2710, the process branches depending on whether the item pointed to by the buffer item is valid. If it is not valid, the process has been performed on all valid items in the similar image buffer 609, and the process advances to step S2711. If it is valid, the process needs to be performed, and the process returns to step S2702.

In step S2711, group information 6
12 are sorted in ascending order according to the value of the representative image ID. After this processing ends, this processing operation ends.

FIG. 28 is a flowchart showing step 2603 of FIG.
6 is a flowchart showing a flow of a processing operation for determining a display scale in FIG. Hereinafter, the details of this process will be described.

In step S2801, initialization necessary for determining a display scale is performed. That is, “0” is stored in the screen number, and “0” is stored in the start area. The screen number is the number of the screen on which the search result is displayed, and the start area is the number of the area where image display is started in each screen. Here, the area indicates a space between adjacent scales. The region number is the distribution information 61
It also corresponds to the subscript of the array of 1. The screen number and the start area are secured in the RAM 203. In addition, the value of each element such as the start of display of the screen information 613 and the scale width is initialized to “−1”. After this processing is completed, a next step S2802 is performed.
Proceed to.

In step S2802, "0" is substituted for the area number and the scale radius. The area number and the scale radius are secured in the RAM 203. The region number corresponds to a subscript of the array of the distribution information 611, the region number for the matching region 706 is “0”, and the similarities “1.0” and “0.
The number for the area between “9” and “9” is “1”, and similarly, the numbers are assigned in descending order of similarity. The scale radius is a value expressing the radius of the scale displayed in the search result display area 703 by the number of dots. After this processing ends, the flow proceeds to the next step S2803.

In step S2803, branching is performed based on the result of comparison between the area number and the start number. If the value of the area number is smaller than the value of the start number, no image is displayed in the area corresponding to the area number.
If the value of the area number is larger than the value of the start number, an image is displayed in the area corresponding to the area number, and thus step S2806 is performed.
Proceed to.

In step S2804, the increment of the radius of the area indicated by the area number is stored in the screen information 613. The radius increment is obtained using the matching area data 605 when the area number is “0”, and using the radius increment data 607 when the area number is other than “0”.
Here, since the value of the region number is smaller than the value of the start region, no image is displayed in this region, that is, since the number of images to be displayed is "0", the radius increment is calculated from FIGS. It can be seen that the area number is 30 dots both when the area number is “0” and when it is not “0”. However, when a character string does not exist in the query box 701, an image similar to the image ID stored in the comparison source ID is stored in the similar image buffer 609, and when the start area is other than “0”, Unless the image indicated by the comparison source ID is displayed, the image of the comparison source cannot be identified. Therefore, only in this case, one image indicated by the comparison source ID is displayed in the matching area 706. Therefore,
In this case, when the area number is “0”, it is understood that the increment of the radius is 60 dots from the matching area data 605 shown in FIG. When the radius increment is determined, the radius increment is stored in the scale width corresponding to the screen number and the area number of the screen information 613. For example, query box 7
When “red apple” is stored in 01 and the value of the screen number is “1”, in this processing, (T
As shown in 201), “30” is stored as the scale width for the area corresponding to the similarity up to “1.0” or “0.8”. In this processing,
The determined radius increment is added to the scale radius. After this processing ends, the flow proceeds to the next step S2805.

In step S2805, the value of the area number is incremented by "1" in order to process the area next to the area indicated by the area number. After this processing ends, the flow returns to the step S2803.

In step S2806, the radius increment of the area indicated by the area number is determined. The radius increment can be obtained from the matching area data 605, the radius increment data 607, the distribution information 611, and the area number.
Using the distribution information 611, the number of images to be displayed in the area indicated by the area number is obtained. When the area number is "0", the matching area data 605 is used. When the area number is other than "0", the radius increment is used. Using the data 607, an increment of the radius corresponding to the number of images to be displayed in the area is obtained. For example, when “red apple” is stored in the query box 701, the distribution information 611 is (T1
01), the area number is “0”,
For “1”, “2”, “3”, “4”, and “5”, “3”, “5”, “8”, “6”,
It can be seen that “0” and “5” images are displayed, and by using the matching area data 605 shown in FIG. 11 and the radius increment data 607 shown in FIG. Radius increments are “120”, “9”
It can be seen that they are "0", "90", "90", "30", and "90". If no character string exists in the query box 701 and “999” is stored in the comparison source ID, the distribution information 611 is as shown in (T102) of FIG. 0 "," 1 "," 2 ",
For “3”, “4”, and “5”,
It can be seen that “1”, “3”, “4”, “2”, “3”, and “0” images are displayed, and the matching area data 605 shown in FIG. By using the radius increment data 607, the radius increment of each area is “60”, “60”, “60”, “6”.
It can be seen that they are "0", "60", and "60". After this processing ends, the flow proceeds to step S2807.

In step S2807, the process branches depending on whether or not the area indicated by the area number can be displayed in the search result display area 703. If the value obtained by adding the increment of the radius obtained in step S2806 to the value of the scale radius is larger than half the size of 600 dots of the search result display area 703, the area indicated by the area number is displayed in the search result display area 703. Since it cannot be performed, the process proceeds to step S2810. Also, the value obtained by adding the increment of the radius obtained in step S2806 to the value of the scale radius is
If the size of the search result display area 703 is less than half the size of 600 dots, the area indicated by the area number can be displayed in the search result display area 703.
Proceed to. For example, the distribution information 611 is (T10
In the case of 1), when the screen number is “0” and the area number is “2”, the value of the scale radius is “300”, and the process proceeds to step S2808. If the area number is “3” at “0”, the value of the scale radius is “390”, and the process proceeds to step S2810.

[0130] In step S2808, step S2808 is executed.
The increment of the area radius with respect to the area number obtained in 2806 is stored in the screen information 613. For example, distribution information 6
When 11 is as shown in (T101) in FIG.
As shown in (T201) of FIG. 19, the screen information 613 includes “120” as a scale width for a region where the similarity with respect to the screen number “0” is from “1.0” to “0.8”. , “90”, and “90” are the graduation widths for the regions whose similarity to the screen number “1” is from “0.7” to “0.5”, respectively.
“0” and “90” are stored. When the distribution information 611 is as shown in (T102) of FIG. 17, the screen information 613 contains the similarity with respect to the screen number “0” as shown in (T202) of FIG. 1.0 "
The scale widths for the regions from to “0.6” are “60”, “60”, “60”, “60”, and “6”, respectively.
"0" and "60" are stored. In this processing,
The determined radius increment is added to the scale radius. Upon completion of this process, the process advances to step S2809.

In step S2809, the value of the area number is incremented by "1" in order to process the area next to the area indicated by the area number. When this process ends, the process returns to the step S2806.

In step S2810, the process branches depending on whether all the images stored in the similar image buffer 609 have been processed. If the values of elements after the area indicated by the area number in the distribution information 611 are all “0”,
Since the processing has been performed for all the images, this processing operation ends. If the values of elements after the area indicated by the area number in the distribution information 611 are not all “0”, step S is executed to display an image that has not been processed on the next screen.
Proceed to 2811.

In step S 2811, first, the value of the start area is assigned to the start area corresponding to the screen number of the screen information 613. Then, the value of the screen number is counted up by “1” in order to process the next screen. Also, the value of the area number is substituted for the start area. When this process ends, the process returns to the step S2802.

FIG. 29 is a flowchart showing the operation of step S2210 in FIG.
Step S2305 in FIG. 23 and Step S24 in FIG.
26 is a flowchart illustrating the flow of a search result display processing operation in step S03 of FIG. In this processing, the screen configuration information stored in the screen information 613 and the similar image buffer 609 and the image database 6
A screen indicated by a screen number is displayed on the display 208 based on the information of the image stored in the image data 04. Hereinafter, the details of this process will be described.

In step S2901, the scale is displayed in the search result display area 703 using the scale width in the screen information 613 for the designated screen number. The radii of the scales are added in order from the scale width corresponding to the similarity “1.0” until the value of the scale width becomes “−1”, and the search result display area 703 is added to the radius of each scale. A circle centered on the center of is displayed. Similarity "1.0"
The inside of the scale is painted in gray. When this process ends, the flow advances to the next step S2902.

At step S2902, matching area 706
, An image having a similarity of “1.0” is displayed. An image ID for the item of similarity “1.0” is obtained from the head of the similar image buffer 609, an image file corresponding to the image ID is obtained from the image database 604, and the image file is displayed in the matching area 706. The coordinates for displaying the image file in the matching area 706 are the same as those of the matching image data 60.
6, the coordinates corresponding to the number of images to be displayed in the matching area 706 are obtained. If no character string exists in the query box 701, as shown in FIG.
Is displayed in a double frame. in this case,
The image always indicated by the comparison source ID is stored in the similar image buffer 609 as the similarity “1.0”.
When this process ends, the flow advances to the next step S2903.

In step S2903, the first image I other than the similarity “1.0” is selected from the similar image buffer 609.
This is a process of acquiring D as an image ID to be displayed. When this process ends, the flow advances to the next step S2904.

In step S2904, the process branches depending on whether the target image is a representative image. The image ID of the target image is the image ID of the group information 612.
If it is stored, it is determined that the image is not the representative image, and the process advances to step S2905. If it is not stored as an image ID, it is determined that the image is a representative image, and the process proceeds to step S290.
Proceed to 7.

In step S2905, the image ID to be displayed is changed to the next image ID in the similar image buffer 609. When this process ends, a step S2 proceeds.
Proceed to 906.

In step S2906, the flow branches depending on whether or not to end the image display. Similar image buffer 60
If all the images in the image 9 have been processed, or if the scale width of the area where the target image is displayed is “−1”, it is determined that the display of the image is to be ended, Proceed to S2911. If it is determined that the display of the image is not to be ended, it is determined that the display of the image is to be continued, and the process returns to the step S2904.

In step S2907, the display position of the image to be displayed, that is, the representative image is determined.
First, the direction of the coordinates of the center of the image to be displayed from the center of the search result display area 703 is determined using the arrangement angle data 608. By using the arrangement angle data 608, the direction from the center can be uniquely determined depending on the number of the representative image to be displayed. Next, a distance from the center of the search result display area 703 of the coordinates of the center of the image to be displayed to the coordinates of the center of the image to be displayed is determined using the screen information 613. The distance from the center of the search result display area 703 to the area where the image is displayed is determined by adding the corresponding scale width of the screen information 613 from the area of “1.0” based on the designated screen number and the similarity of the image to be displayed. Can be obtained. Since the scale width of the area can be obtained from the screen information 613, the distance from the center in the area can be obtained from the scale width and the similarity. For example, when “1” is specified as the screen number in the state of (T201) in FIG. 19 in the screen information 613, the similarity “0.5
When the image of “6” is displayed, this image is “0.5”.
The search result display area 7
The distance from the center of No. 03 to the area of “0.5” is “1.
By adding the scale width from “0” to the scale width from “0.6”, it can be seen that 30 + 30 + 30 + 90 + 30 = 210 dots. The distance from the center in the area of “0.5” is 90 × (0.6−0.56) / 90 because the scale width of the area of “0.5” is 90 dots.
It can be seen that 0.1 = 36 dots. Therefore, in this case, from the center of the search result display area 703, the similarity “0.
The distance to the center of the image of “56” is 210 + 36 = 24
It turns out that it is 6 dots. In this manner, the angle and the distance from the center of the search result display area 703 can be obtained, so that the position where the image is displayed can be determined. Upon completion of this process, the process advances to step S2908.

Step S2908 is a process for displaying a representative line from the center of the search result display area 703 to the display position obtained in step S2907. However,
A representative line is not displayed in the matching area 706. When this process ends, the process advances to step S2909.

Step S2909 is a process for displaying all the images of the group to which the target representative image belongs except for the representative image. The image of the group to which the representative image belongs can be obtained using the group information 612.
The image I in which the value of the representative image ID in the group information 612 is equal to the value of the image ID of the representative image to be displayed
All images corresponding to D are to be displayed. Further, all image IDs whose display target image IDs are representative image IDs in the group information 612 are also displayed.

The distance from the center of the image ID search result display area 703 to each image to be displayed is determined by the similar image buffer 6.
09, based on the similarity of each image in step S2907.
Can be obtained in the same manner as in the case of Further, the distance from the display position of the representative image to the display position of each image can be determined by calculating the similarity between two images from the similar image information 610. For example, from the representative image ID and the image ID of the image to be displayed, a corresponding similarity is obtained from the similar image information 610, and a value obtained by subtracting the similarity from “1.0” is 5
It can be determined by multiplying by 00. In addition, each image is displayed so as to alternate with the counterclockwise side and the clockwise side of the representative line with the search result display area 703 as the center in the order of similarity with the representative image. As described above, the distance from the center of the search result display area 703 and the center of the representative image can be obtained, and it can be determined which of the lines connecting the two points is displayed. The display position can be determined. When the display position of each image is determined, a line connecting the center of each image and the center of the representative image is displayed, and each image is displayed in ascending order of similarity with the representative image.
By displaying the images in this order, when the images overlap, the image having a higher similarity to the representative image is less likely to be blocked by other images. When this process ends, the process advances to step S2910.

In the step S2910, the step S
A representative image is displayed at the position obtained in step 2907. Thus, the representative image in the group is displayed last, so the representative image with the highest similarity in the group is
It is possible to prevent the image from being obstructed by another image. When this process ends, the process proceeds to step S290.
Go to 5.

In the step S2911, the previous screen button 7
12 and a next screen button 713 are displayed. If the specified screen number is "0", there is no previous screen, so
The triangle of the previous screen button 712 is displayed on a white background, and if the designated screen number is not "0", there is a previous screen, so the triangle of the previous screen button 712 is displayed on a black background. If the value of the start area of the screen information 613 for the screen number obtained by adding “1” to the designated screen number is “−1”, there is no next screen, so the triangle of the next screen button 713 is displayed on a white background. If the value of the start area of the screen information 613 for the screen number obtained by adding “1” to the designated screen number is not “−1”, the next screen exists because the next screen button 713 has a black triangle. I do. When this process ends, this process ends.

(Other Embodiments) In the above-described first embodiment, similar image information 610 is stored in step S2 of FIG.
002, the similar image information 610 is created in advance by performing the same processing as step S2002 in FIG. 20, and the CD-ROM is used as related data.
205, and may be loaded into the RAM 203 in step S2001 in FIG.

Further, in the first embodiment described above, an example in which the image database 502 is stored in the CD-ROM 205 has been described, but an image database stored in the hard disk 205 may be used.

In the first embodiment described above, the information retrieval program and the related data are directly loaded from the CD-ROM 205 as the external storage device to the RAM 203 and executed. ROM20
5, the information search program and related data are temporarily stored (installed) in the hard disk 205, and when the information search program is operated, the hard disk 2
05 may be loaded into the RAM 203.

The storage medium for storing the information retrieval program may be an FD (floppy (registered trademark) disk), an IC memory card, or the like, in addition to the CD-ROM 205.

The information retrieval program is stored in the ROM 202
, And can be configured to be a part of the memory map, and can be directly executed by the CPU 201.

Further, in the first embodiment described above, an example is described in which English is used as a language used as a query and an explanatory sentence. However, the present invention is not applicable only to English, It can be applied to any language including.

In the above-described first embodiment, in step S2011 of FIG. 20, in order to obtain an image similar to the designated image, the similarity is calculated using the explanatory text attached to the image. However, the present invention does not depend on a similarity calculation method. For example, the similarity may be calculated using information of the image itself, such as the brightness, color, and composition of the image.

In the first embodiment, an example in which an image is searched as an information search target has been described. However, the search target handled by the present invention is not limited to an image. For example, it is effective for various information retrieval such as a document, a video, and an audio. Also, as an example of a search result displayed in the search result display area, when searching for an image, the image itself is displayed. However, when searching for a document, the title or keyword of the document is displayed instead of the image. Is also good. In the case of a video, information representing the video such as a title or a representative scene may be displayed, or a video of each result may be displayed.

In the first embodiment, as shown in FIG. 7, the matching area is a circle area centered on the center of the search result display area, and each scale is displayed as a concentric circle with the matching area. Although each image of the search result is displayed in such a manner that the similarity is associated with the distance from the matching area and displayed so as to be arranged radially from the center of the search result display area, but the present invention is not limited to this. Absent. For example, as shown in FIG.
The matching area is a rectangular area on the left side of the search result display area shown in 3001, each scale is a parallel line with the left side as shown by the lines shown in 3002 and 3003, and the distance of the perpendicular straight line from each image to the matching area is Alternatively, the images may be displayed in correspondence with the similarity of each image. In this case, the arrangement angle data 60
As in FIG. 8, instead of specifying the arrangement position of each image by an angle with respect to the center of the matching area, instead of specifying the position by the distance from the upper side of the search result display area, the first embodiment described above is performed. The image of the search result can be displayed at a desired position in exactly the same procedure as shown in. Even when the search result is displayed in this manner, as shown in FIG. 30, in the same manner as in the first embodiment,
It is possible to group and display the images of the search results.

Further, in the above-described first embodiment, an example is shown in which the radius increment data 607 is stored with the radius increment corresponding to the number of images displayed between adjacent scales. The radius increment may be stored in correspondence with the number of representative images of the group displayed between adjacent scales. In this case, the distribution information 611 stores the number of representative images of the group existing between the scales.
It is necessary to change the order of the distribution information creation processing in step S2601 and the order of the grouping processing in step S2602, and perform the distribution information creation processing after performing the grouping processing. This makes it possible to display more search result information in a limited area.

[0157]

As described above in detail, according to the information retrieval method and apparatus of the present invention, the similarity of each piece of information of the retrieval result is obtained, the information of the retrieval result is grouped according to the similarity, and the grouping is performed. Since the displayed information is displayed so as to be close to each other, without checking all the information of the group,
Since it is possible to determine whether or not the group is the desired information, it is not necessary to check all the information, and it is possible to improve the efficiency of evaluating the search result.

Further, according to the storage medium of the present invention, there is an effect that the above-described information retrieval device can be controlled smoothly.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram of an information search device according to a first embodiment of the present invention.

FIG. 2 is a system configuration diagram of the information search device according to the first embodiment of the present invention.

FIG. 3 is a diagram showing that a program or the like of the information search device according to the first embodiment of the present invention is supplied from a storage medium to a computer system.

FIG. 4 is a configuration diagram of a ROM of the information search device according to the first embodiment of the present invention.

FIG. 5 is a configuration diagram of a storage medium of the information search device according to the first embodiment of the present invention.

FIG. 6 is a diagram illustrating a configuration of a memory map on a RAM when a processing program of the information search device according to the first embodiment of the present invention is executed.

FIG. 7 is a diagram illustrating an example of a screen configuration of a first search result when a query search is performed by the information search device according to the first embodiment of the present invention.

FIG. 8 is a diagram showing a configuration example of a screen next to a search result of a query search of the information search device according to the first embodiment of the present invention.

FIG. 9 is a diagram illustrating a configuration example of a first search result screen when a similar search is performed by the information search device according to the first embodiment of the present invention;

FIG. 10 is a diagram illustrating a configuration example of an image database on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 11 is a diagram showing a configuration example of matching area data on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 12 is a diagram showing a configuration example of matched image data on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 13 is a diagram showing a configuration example of radius increment data on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 14 is a diagram showing a configuration example of arrangement angle data on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 15 is a diagram illustrating a configuration example of a similar image buffer on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 16 is a diagram showing a configuration example of similar image information on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 17 is a diagram showing a configuration example of distribution information on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 18 is a diagram illustrating a configuration example of group information on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 19 is a diagram showing a configuration example of screen information on a memory map of the information search device according to the first embodiment of the present invention.

FIG. 20 is a flowchart showing a flow of an information search processing operation of the information search device according to the first embodiment of the present invention.

FIG. 21 is a flowchart showing a flow of a search initialization processing operation of the information search device according to the first embodiment of the present invention.

FIG. 22 is a flowchart illustrating a flow of a query search processing operation of the information search device according to the first embodiment of the present invention.

FIG. 23 is a flowchart showing a flow of a similarity search processing operation of the information search device according to the first embodiment of the present invention.

FIG. 24 is a flowchart showing a flow of a next screen display processing operation of the information search device according to the first embodiment of the present invention.

FIG. 25 is a flowchart showing a flow of a previous screen display processing operation of the information search device according to the first embodiment of the present invention.

FIG. 26 is a flowchart illustrating a flow of a screen configuration determination processing operation of the information search device according to the first embodiment of the present invention.

FIG. 27 is a flowchart illustrating a flow of a grouping processing operation of the information search device according to the first embodiment of the present invention.

FIG. 28 is a flowchart showing a flow of a display scale determination processing operation of the information search device according to the first embodiment of the present invention.

FIG. 29 is a flowchart showing a flow of a search result display processing operation of the information search device according to the first embodiment of the present invention.

FIG. 30 is a diagram showing a configuration example of a search result display area of the information search device according to another embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 101 input unit 102 control unit 102a coincidence calculation unit 102b similarity calculation unit 102c search result display unit 102d grouping unit 103 storage unit 103a information storage unit 104 display unit 201 CPU (central processing unit) 202 ROM (read only memory) 203 RAM (random access memory) 204 CD-ROM drive 205 CD-ROM (compact disk read only memory) 206 HD (hard disk) 207 keyboard 208 display 209 mouse 210 printer 211 control bus 300 computer system 400 ROM (read only memory) 401 Control procedure program 500 Storage medium 501 Information search program 502 Image database 503 Matching area data 504 Matching image data 505 Radius increment data 506 Arrangement angle data 600 Memory map 601 Basic I / O program 602 OS (operating system) 603 Information retrieval program 604 Image database 605 Matching area data 606 Matching image data 607 Radius increment data 608 Arrangement angle data 609 Similar image buffer 610 Similar image information 611 Distribution information 612 Group information 613 Screen information 701 Query box (text box) 702 OK button 703 Search result display area 704 Search result image 705 Search result image 706 Matching area 707 Scale 708 Scale 709 Representative line 710 Group Connecting line 711 Image of search result 712 Previous screen button 713 Next screen button 801 Scale 802 Scale 803 Scale 3001致領 range 3002 scale 3003 scale

Claims (15)

[Claims] 1. An information search method for inputting search conditions, searching an information storage unit storing individual information to be searched, and displaying information satisfying the search conditions as a search result. A matching degree calculating step of comparing individual information stored in a section with the search condition to obtain a match degree that is a degree satisfying the search condition; and comparing individual information of the search result with each other to obtain individual information. A similarity calculating step of calculating the similarity of the group, a grouping step of grouping the individual information of the search results using the similarity obtained in the similarity calculating step, and information of the search results belonging to each group are mutually different. A search result display step of determining and displaying the display position of each piece of information so as to be displayed nearby. 2. The information having the highest matching degree in each group is set as representative information, and the search result displaying step determines a position at which information having a perfect matching degree is displayed as a perfect matching display position. The display distance of the representative information is determined by associating the display distance from the perfect match display position with the degree of coincidence of the representative information, and then the display positions of the individual information belonging to the group are determined. 2. The information retrieval method according to claim 1, wherein: 3. The search result display step according to claim 2, wherein a display position of information other than the representative information is determined from a degree of coincidence of individual information and a degree of similarity with the representative information. Information retrieval method. 4. The information search method according to claim 3, wherein the search result display step displays the representative information so as to be radially displayed around the perfect match display position. 5. The grouping step includes, for each piece of information, information having a highest similarity to a specific value or more of the information and having a higher degree of matching than the information as group higher-level information. 2. The information retrieval method according to claim 1, wherein a relation is created, a tree formed by combining the binary relations is set as one group, and a root of the tree is set as the representative information. 6. An information search apparatus according to claim 1, wherein a search condition is input, an information storage unit storing individual information to be searched is searched, and information satisfying said search condition is displayed as a search result. Matching information calculating means for comparing the individual information stored in the section with the search condition to obtain a match degree that satisfies the search condition; and comparing the individual information of the search result with each other to obtain the individual information. A similarity calculating means for calculating the similarity of the group, a grouping means for grouping the individual information of the search result using the similarity obtained by the similarity calculating means, and information of the search results belonging to each group are mutually different. A search result display unit for determining and displaying a display position of each piece of information so as to be displayed nearby. 7. The information having the highest matching degree in each group is set as representative information, and the search result display means determines a position at which information having a perfect matching degree is displayed as a perfect matching display position. The display distance of the representative information is determined by associating the display distance from the perfect match display position with the degree of coincidence of the representative information, and then the display positions of the individual information belonging to the group are determined. The information retrieval device according to claim 6, wherein 8. The search result display unit according to claim 7, wherein a display position of information other than the representative information is determined from a degree of coincidence of individual information and a degree of similarity to the representative information. Information retrieval device. 9. The information search apparatus according to claim 8, wherein the search result display means displays the representative information so as to be radially displayed around the perfect match display position. 10. The grouping means according to claim 2, wherein, for each piece of information, information having a highest similarity to a specific value or more of said information and having a higher degree of matching than said information are regarded as group higher-level information. 7. The information search apparatus according to claim 6, wherein a tree created by combining the binary relations is created as one group, and a root of the tree is used as the representative information. 11. A control program for inputting a search condition, searching an information storage unit storing individual information to be searched, and displaying an information satisfying the search condition as a search result. In the storage medium storing the (information search program), the control program compares individual information stored in the information storage unit with the search condition to determine a degree of coincidence that is a degree satisfying the search condition. , Comparing the individual information of the search results with each other to determine the similarity of the individual information, grouping the individual information of the search results using the similarity, and displaying the information of the search results belonging to each group near each other A control module for determining a display position of each piece of information and controlling the display so as to display the information. 12. The control program determines information having the highest matching degree in each group as representative information, determines a position at which information having a perfect matching degree is displayed as a perfect matching display position, After determining the display position of the representative information by associating the display distance from the display position with the degree of coincidence of the representative information, performing control to determine the display positions of the individual information belonging to the group. The storage medium according to claim 11, further comprising: a control module. 13. The control program according to claim 1, further comprising a control module for controlling a display position of information other than the representative information so as to be determined from a degree of coincidence of each piece of information and a degree of similarity to the representative information. Claim 1 characterized by the following:
2. The storage medium according to 2. 14. The control program according to claim 13, wherein the control program includes a control module for controlling to display the representative information so as to be radially displayed around the perfect match display position. Storage media. 15. The control program, for each piece of information, information having the highest similarity to a specific value or more of the information and having a higher degree of coincidence than the information as group higher-level information. 12. A storage module according to claim 11, further comprising: a control module for creating a tree formed by combining the binary relations into one group, and controlling a root of the tree as the representative information. Medium.