JPH08249352A - Image retrieval method and device - Google Patents

Abstract

(57) [Summary] [Purpose] Even when there are many uncertain search conditions, the desired image can be searched efficiently, and the desired image can be searched efficiently while effectively browsing the vicinity of a certain image. An image search method and device capable of performing the above are provided. [Arrangement] Any two of a plurality of types of feature quantities are selected and set as search feature quantities corresponding to each of the XY axes,
Those values are arbitrarily adjusted within a predetermined adjustment range, and a fixed number of similar images are extracted from the image storage means based on the adjusted two feature amount values. It is collectively displayed on the screen of the display device in correspondence with the XY coordinate space.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieving apparatus for retrieving an image matching a user's taste from a large number of images prepared in advance while looking at a display screen. The present invention relates to an image search device capable of efficiently searching for a desired image even when there are a plurality of images.

[0002]

2. Description of the Related Art For example, when designing clothes, curtains, wallpaper, embroidery, etc., a new design is created by selecting a design concept close to the desired design concept from existing design samples and adding appropriate modifications to it. Creating is often done among designers. In order to improve the efficiency of such design work, it is necessary to find a design close to the desired design concept from the design sample as quickly as possible. However, as the number of symbol samples becomes huge, there is a limit to relying on such a symbol sample searching operation manually.

Therefore, conventionally, such a pattern sample retrieval work
An image that uses computer technology to mechanize the work.
Various image retrieval devices have been proposed. Such image retrieval device
In that case, each of the above-mentioned pattern samples
Converted to image data via a canner, CD-RO
It is stored in a mass storage medium such as M. At the same time, each image data
For data, density value conversion, noise elimination, blur restoration
Various image processing such as original and outline detection and enhancement, and extraction of connected parts
After the processing is added, various characteristics are added based on the image processing result.
Boundary line pixel count, hole count, curve
Degree, number of colors, color distribution, contrast, border pixel distribution, etc.
The various image feature quantities that have been described (hereinafter referred to as “sensitivity”
"Physical feature" for comparison with "physical feature")
Is required. Now, suppose k images are y₁, Y₂,
y₃, ... y_k, The n types of physical features are x₁,
x₂, X₃, ... x_nIn other words, each image is an n-dimensional vector.
Le (x₁₁, X _{twenty one}, X₃₁, ... x_n1), (X₁₂, X_{twenty two},
x₃₂, ... x_n2),… (X_1k, X_2k, X_3k, ... x_nk)age
Is represented. Therefore, it matches the desired design concept
Image y _pIs a vector using the above physical features
(X_1p, X_2p, X_3p, ... x_np), And this and the above
Each image y₁, Y₂, Y₃, ... y_kCorresponding to
Find the distance to Le and find the one with the closest distance
Select to mechanically search for images similar to your desired image
You can do it.

Techniques for improving this type of image retrieval device
As a document, JP-A-5-6437 (the title of the invention is
Title: "Image feature extraction device, image feature matching device and image
Search device ”) or Information CG / CAD Study Group 199
August 4th, Vol. 94, No. 72, p. 43-48
(Paper title: "Kansei characteristics and image characteristics of design images")
It has been known. In these documents, image search
It is possible to use human emotional expressions
It That is, the objects used in the image search device described above.
The physical feature amount is the result of image processing of the given image data.
By applying a predetermined feature extraction operation to the fruits,
While having the advantage of being mechanically and uniquely extracted, its table
Currently, for example, the number of border pixels, the number of holes, curvature, color
Number, color distribution, contrast, border pixel distribution, color distribution, etc.
To express any design concept like
It lacks appropriateness. So this improved image search
The equipment is warm / cold, masculine / feminine, safe
Constant / unstable, symmetric / asymmetric, simplicity / complexity, etc.
Humans (designers) make impressions of arbitrary images like
Focusing on the emotional characteristics used to identify,
Statistical correlation between the physical features and the physical features described above.
Sensitivity features using this correlation
It is possible to search images by quantity. If you explain it conceptually,
The emotional feature amount of an arbitrary image is set to w₁, W₂, W₃,…
w_m, Its physical feature x₁, X₂, X₃, ... x_nWhen
By definition, each affective feature quantity w₁, W₂, W₃, ... w_m
Are n-dimensional vectors (a₁₁・ X₁, A₁₂・ X
₂, A₁₃・ X₃, ... a_1n・ X_n), (A _{twenty one}・ X₁, A_{twenty two}
・ X₂, A_{twenty three}・ X₃, ... a_2n・ X_n), (A₃₁・ X₁,
a₃₂・ X ₂, A₃₃・ X₃, ... a_3n・ X_n), ... (a_m1・
x₁, A_m2・ X₂, A_m3・ X₃, ... a_mn・ X_n) And represented
Be done. Therefore, it is possible to convert physical features into emotional features.
Weighting value group (a₁₁, A₁₂, A₁₃, ... a_1n),
(A_{twenty one}, A_{twenty two}, A_{twenty three}, ... a_2n), (A₃₁, A₃₂, A₃₃,
... a_3n), ... (a_m1, A_m2, A_m3, ... a_mn) Designer
In advance using statistical methods through questionnaire surveys
If you ask for it, from any image data, its affective features
Can be mechanically extracted. Therefore, the image search
In the process, the sensitivity of the input image and the sense of the registered image
By calculating the distance from the sexual feature, the physical feature
Compared with the case of using,
Image search is performed.

[0005]

By the way, in this type of image retrieval apparatus, when an image that matches a desired design concept is retrieved, retrieval conditions are given from various viewpoints. Examples of such search conditions include the time when the design is created, the cost required for the creation, the physical or emotional characteristic scale value of the image described above, and the like. In addition, these search conditions are often changed while changing the search conditions while actually looking at the actually searched image. However, the conventional device is not always satisfactory in use when there are many uncertain search conditions.

Further, as a method for retrieving an image that matches a desired design concept, there is a method for exemplifying an image close to the desired image and retrieving an image similar to the image. However, in such an example image input method, an image that exactly matches the image of the search operator cannot be obtained in one search in many cases. The present invention has been made in consideration of the actual usage of such an image retrieval apparatus, and its object is to obtain a desired image even when there are many uncertain retrieval conditions. An object of the present invention is to provide an image search method and device capable of efficient search.

Another object of the present invention is to provide an image retrieval method and apparatus capable of efficiently retrieving a desired image while effectively browsing the vicinity of a certain image.

[0008]

According to a first aspect of the present invention, there is provided an image storing step of storing a plurality of images with a plurality of types of characteristic values attached to each of the plurality of images, and storing the plurality of types of characteristic amounts. A search feature quantity selection setting step of selectively setting any two of the two as search feature quantities, and arbitrarily adjusting the values of the two feature quantities selected and set as the search feature quantity within a predetermined adjustment range. Feature amount adjustment step to
A similar image extracting step of extracting a fixed number of similar images from the image storage means based on the two adjusted feature values, and a batch of the extracted fixed number of similar images on the screen of the display device. And an image display step for displaying the image.

In the invention according to claim 2 of this application, in the invention according to claim 1, the search feature quantity selection / setting step comprises a plurality of feature quantities displayed in a menu on the screen of the display device. It is characterized in that, through an operation of selecting any two of the items, any two of the plurality of types of feature quantities are selected and set as the search feature quantity.

In the invention according to claim 3 of this application, in the invention according to claim 1, the feature amount adjusting step is within a predetermined range in a horizontal direction and a vertical direction on the screen of the display device. Through the operation of individually moving a pair of movably displayed cursors, it is possible to arbitrarily adjust the values of the two feature amounts selected and set as the search feature amount within a predetermined adjustment range. Characterize.

According to a fourth aspect of the present application, in the invention according to the first aspect, the image displaying step is performed by the extracted constant image in a rectangular window displayed on a screen of a display device. The number of similar images is displayed densely without overlapping with each other, and the feature amount adjusting step is performed along any one of the upper and lower horizontal edges and the right or left vertical edge of the rectangular window. Through the operation of individually moving a pair of cursors that are movably displayed within a predetermined range, the values of the two feature quantities selected and set as the search feature quantity can be arbitrarily adjusted within a predetermined adjustment range. It is characterized by

According to a fifth aspect of this application, in the invention according to the first aspect, the similar image extracting step is defined by two feature quantities selected and set as the search feature quantity. On the two-dimensional coordinates, the Euclidean distance between the search reference image specified by the pair of adjusted feature amounts and each image stored in the image storing step is calculated, and the Euclidean distance is calculated from the calculated distances. It is characterized in that a fixed number of images selected in order from the one closest to are extracted as the target similar images.

According to a sixth aspect of the present application, an image storing step of storing a plurality of images with a plurality of types of characteristic amount values respectively stored therein, and n of the plurality of types of characteristic amounts are stored.
A selection setting step for selectively setting individual feature values,
It is similar to the input reference exemplary image from the reference exemplary image input step for inputting one reference exemplary image to be a reference for similar image search and the images stored in the image storing step. An item-based similar image extracting step of searching for and extracting a similar image in both positive and negative directions along each of the selectively set n feature amounts, and the input reference exemplary image is located at the center, and the selection is performed. Display similar images for each item around the reference exemplary image so that the similar images bidirectionally extracted for each set item are positioned on both sides of the straight line passing through the reference exemplary image position. Image display step
It is characterized by including.

According to a seventh aspect of the present application, in the invention according to the sixth aspect, the number of feature amounts selectively set in the selection setting step is four, and the item-wise similarity is obtained. The number of similar images searched by the image extracting means is one in each of the positive and negative directions along each feature amount, and in the image display step, eight similar images are arranged point-symmetrically around the reference exemplary image. It is characterized by displaying all at once.

In the invention according to claim 8 of this application, in the invention according to claim 6, the image display step additionally displays an item name corresponding to each straight line passing through the reference exemplary image position. It is characterized by doing. In the invention according to claim 9 of this application, in the invention according to claim 6, a similar image for inputting one of the similar images arranged around the reference exemplary image as a new reference exemplary image. It is characterized by comprising a re-input step.

According to a tenth aspect of the present invention, an image storage means for storing a plurality of images by assigning a plurality of types of characteristic amounts to each of the plurality of images and an arbitrary two of the plurality of types of characteristic amounts are stored. Search feature quantity selection setting means for selectively setting one as a search feature quantity, and a feature that can arbitrarily adjust the values of the two feature quantities selectively set as the search feature quantity within a predetermined adjustment range. An amount adjusting means, a similar image extracting means for extracting a fixed number of similar images from the image storage means based on the two adjusted feature values, and a display device for displaying the extracted fixed number of similar images. And an image display unit that collectively displays the image on the screen.

According to an eleventh aspect of this application, in the invention according to the tenth aspect, the search feature amount selection setting means is a plurality of feature amounts displayed on a menu of the display device. It is characterized in that, through an operation of selecting any two of the items, any two of the feature amounts of the plurality of types are selectively set as the search feature amount.

According to a twelfth aspect of the present invention, in the invention according to the tenth aspect, the characteristic amount value adjusting means is within a predetermined range in a horizontal direction and a vertical direction on the screen of the display device. The values of the two feature amounts selected and set as the search feature amount can be arbitrarily adjusted within a predetermined adjustment range through an operation of individually moving a pair of cursors that are movably displayed. Is characterized by.

According to a thirteenth aspect of the present invention, in the invention according to the tenth aspect, the image display means includes the extracted constant image in a rectangular window displayed on a screen of a display device. The number of similar images is densely arranged and displayed collectively without overlapping each other, and the feature value adjusting means is arranged along one of the upper and lower horizontal edges and the left or right vertical edge of the rectangular window. Through the operation of individually moving a pair of cursors that are movably displayed within a predetermined range, the values of the two feature quantities selected and set as the search feature quantity can be arbitrarily adjusted within a predetermined adjustment range. It is characterized by being done.

According to a fourteenth aspect of the present invention, in the invention according to the tenth aspect, the similar image extracting means is defined by two feature amounts selected and set as the search feature amount. On the two-dimensional coordinates, the Euclidean distance between the search reference image specified by the pair of adjusted feature value values and each image stored in the image storage means is calculated, and the Euclidean distance is calculated from the calculated distances. It is characterized in that a fixed number of images selected in order from the one closest to are extracted as the target similar images.

According to a fifteenth aspect of the present invention, an image storage means for storing a plurality of images by assigning a plurality of types of characteristic amount values to each of the plurality of images, and n of the plurality of types of characteristic amounts are stored. Selection setting means for selecting and setting the feature amount, reference reference image input means for inputting one reference illustration image to serve as a reference for similar image search, and an image stored in the image storage means. Item-like similar image extracting means for searching and extracting a similar image similar to the input reference exemplary image in the positive and negative directions from each of the selected n set feature amounts, and the input. The reference standard image is positioned in the center, and the similar images bidirectionally extracted for each of the selected and set items are positioned on both sides of a straight line passing through the position of the reference exemplary image. Image table displaying similar images for each item around the example image Characterized by comprising a means.

According to a sixteenth aspect of the present invention, in the invention according to the fifteenth aspect, the number of feature quantities selectively set by the selection setting means is four, and The number of similar images retrieved by the similar image extraction means is one in each of the positive and negative directions along each feature amount, and the image display means arranges eight similar images around the reference exemplary image in point symmetry. It is characterized by displaying all at once.

In the invention according to claim 17 of this application, in the invention according to claim 15, the image display means additionally displays an item name corresponding to each straight line passing through the reference exemplary image position. It is characterized by doing. The invention according to claim 18 of this application is the invention according to claim 15, wherein one of the similar images arranged around the reference exemplary image is input as a new reference exemplary image. It is characterized by comprising re-input means.

[0024]

According to the invention described in claim 1 of this application, a plurality of images are stored with a plurality of types of characteristic amounts attached to each, and any two of the plurality of types of characteristic amounts are used for retrieval. The values of the two feature amounts selected and set as the feature amount and selectively set as the search feature amount are arbitrarily adjusted within a predetermined adjustment range, and the image is calculated based on the adjusted two feature amount values. A fixed number of similar images are extracted from the storage means, and the extracted fixed number of similar images are collectively displayed on the screen of the display device.

According to the invention of claim 2 of this application, in the invention of claim 1, any two of a plurality of feature quantity items displayed on the screen of the display device are displayed as menus. Through the operation of selecting, any two of the plurality of types of feature quantities are selected and set as the search feature quantity.

According to the invention described in claim 3 of this application, in the invention described in claim 1, the image is displayed on the screen of the display device so as to be movable within a predetermined range in the horizontal direction and the vertical direction, respectively. Through the operation of individually moving the pair of cursors, the values of the two feature amounts selectively set as the search feature amount or within a predetermined adjustment range are arbitrarily adjusted.

According to the invention described in claim 4 of this application, in the invention described in claim 1, a fixed number of similar images extracted in a rectangular window displayed on the screen of the display device. Are displayed side by side in a dense arrangement without overlapping each other, and a pair of cursors that are displayed so as to be movable within a predetermined range along any one of the upper and lower horizontal edges of the rectangular window and the left or right vertical edge of the rectangular window are displayed. Through the operation of individually moving, the values of the two feature amounts selected and set as the search feature amount are arbitrarily adjusted within a predetermined adjustment range.

According to the invention of claim 5 of this application, in the invention of claim 1, on a two-dimensional coordinate defined by two feature quantities selected and set as the search feature quantity. , The Euclidean distance between each of the images stored in the image storage step and the search reference image specified by the pair of adjusted feature amounts is calculated, and the Euclidean distance is calculated from the closest distance among the calculated distances. A fixed number of images selected in sequence are extracted as the target similar images.

According to the invention of claim 6 of this application, a plurality of images are stored with a plurality of types of feature amount values respectively stored therein, and n feature amounts are selected from the plurality of types of feature amounts. Is selected and set, one reference exemplary image to be used as a reference for similar image search is input, and from the stored images, a similar image similar to the input reference exemplary image is selected and set. A similar image that is extracted and searched in both positive and negative directions along each of the n feature amounts, and the input reference exemplary image is located in the center, and that is bidirectionally extracted for each of the selected and set items. Are displayed on both sides of a straight line passing through the reference exemplary image position, and similar images for each item are displayed around the reference exemplary image.

According to the invention described in claim 7 of this application, the number of feature quantities to be selectively set is four, and the number of similar images to be searched is the positive and negative directions along each feature quantity. As a result, eight similar images are arranged in a point symmetry around the reference exemplary image and are collectively displayed.

According to the invention of claim 8 of this application, the item name corresponding to each straight line passing through the reference exemplary image position is additionally displayed.

According to the invention of claim 9 of this application, one of the similar images arranged around the reference exemplary image is input as a new reference exemplary image.

According to the invention of claim 10 of the present application, a plurality of images are stored with a plurality of types of characteristic amounts attached to each, and any two of the plurality of types of characteristic amounts are for retrieval. The values of the two feature amounts selected and set as the feature amount are arbitrarily adjusted within a predetermined adjustment range, and based on the values of the two adjusted feature amounts, a fixed number of images are stored in the image storage unit. Of similar images are extracted, and the extracted fixed number of similar images are collectively displayed on the screen of the display device.

According to the invention of claim 11 of the present application, the plurality of types of features are selected through an operation of selecting any two from the plurality of feature amount items displayed on the screen of the display device. Any two of the feature quantities are selected and set as the search feature quantity.

According to the twelfth aspect of the present application, an operation of individually moving a pair of cursors displayed on the screen of the display device so as to be movable horizontally and vertically within a predetermined range. Through the above, the values of the two feature quantities selected and set as the search feature quantity are arbitrarily adjusted within a predetermined adjustment range.

According to the thirteenth aspect of the present application, the extracted fixed number of similar images are densely arranged and collectively displayed in a rectangular window displayed on the screen of the display device without overlapping each other. And a pair of cursors that are movably displayed within a predetermined range along one of the upper and lower horizontal edges and one of the left and right vertical edges of the rectangular window are individually moved to perform the search. The values of the two feature amounts selected and set as the feature amount are arbitrarily adjusted within a predetermined adjustment range.

According to the fourteenth aspect of the present application, on the two-dimensional coordinates defined by the two feature amounts selected and set as the search feature amount, the adjusted pair of feature amounts are The Euclidean distance between each of the search reference images specified by the above and each image stored in the image storage means is obtained, and a fixed number of images selected in order from the closest one of the obtained distances is the object. It is extracted as a similar image.

According to the fifteenth aspect of the present application, a plurality of images are stored with a plurality of types of feature amount values respectively stored therein, and n feature amounts are selected from the plurality of types of feature amounts. Is selected and set, one reference exemplary image to be used as a reference for similar image search is input, and from the stored images, a similar image similar to the input reference exemplary image is selected and set. Positive and negative bidirectional searches are performed along each of the n feature amounts, the input reference exemplary image is located at the center, and the similar images extracted bidirectionally for each of the selected and set items are displayed. The similar images for each item are displayed around the reference exemplary image so that they are located on both sides of the straight line passing through the reference exemplary image position.

According to the sixteenth aspect of the present application, the number of the feature quantities selected and set is four, and the number of similar images to be searched is positive or negative depending on each feature quantity. With respect to the direction, one image is provided, so that eight similar images are arranged in a point symmetry around the reference exemplary image and are collectively displayed.

According to the seventeenth aspect of the present application, the item name corresponding to each straight line passing through the reference exemplary image position is additionally displayed.

In the eighteenth aspect of the present application, one of the similar images arranged around the reference exemplary image is input as a new reference exemplary image.

[0042]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. First, an embodiment of the invention described in claims 1 to 5 or claims 10 to 14 of this application will be described with reference to FIGS. 1 to 6 and 14.

The hardware configuration of the image retrieval apparatus of this embodiment is shown in FIG. As shown in the figure, the hardware of this image retrieval apparatus includes a central processing unit 1, a display unit 2,
The personal computer system includes an operation unit 3, a printer 4, an image storage unit 5, and an image scanner 6.

As is well known, the central processing unit 1 is mainly composed of an MPU, a RAM, a ROM, etc., and the above-mentioned system elements 2 to 2 are connected via a system bus 7.
8 is to be totally controlled.

The display unit 2 is composed of a CRT display, an LCD, etc., and is extracted as a search result on the display screen of the display unit 2 as will be described later in detail with reference to FIG. Also, a certain number (nine in this example) of similar images are displayed in color.

The operation unit 3 is composed of a keyboard, a mouse, etc., and various instruction inputs to this apparatus are performed using this operation unit 3.

The printer 4 is composed of a high-precision printer such as a laser color printer, and is used for obtaining a hard copy of the searched image.

The image storage unit 5 is composed of a large-capacity storage device such as a write-once type optical disc and a rewritable type optical disc. The image storage unit 5 will be described in detail later with reference to FIGS. 2 and 14. As described above, a large number of images to be searched are stored with the creation date, the cost, the physical feature scale value thereof, and / or the affective feature scale value added thereto.

The image scanner 6 is for converting an arbitrary pattern sample 10 displayed in color into image data, and this image scanner 6 is an image as will be described later in detail with reference to FIG. It is used for a registration process, or for inputting the pattern sample 10 as an exemplary image.

Those skilled in the art will readily understand that various operating systems, drawing application software and the like are required to realize the above personal computer system.

Next, FIGS. 3 and 4 are control flowcharts showing the software configuration of the device of the present invention, and the operation of the device of the present invention will be described with reference to these flowcharts and the explanatory diagrams of FIGS. 5 and 6. Explain systematically.

First, the image registration process will be described. As shown in FIG. 14, it is assumed that there are now 1000 images or pattern samples to be registered. In such a case, the present invention first secures 50 of them as sample analysis images, and with respect to this sample analysis image,
The emotional feature amount and the physical feature amount are extracted. The extraction of the kansei features is performed by conducting a questionnaire survey on an appropriate number of designers (for example, 100 people) or general consumers. As a result, the sensitivity features of each image forming the sample analysis image are, for example, 0.8 for sharpness / softness, 0.7 for simple decorativeness, 0.6 for dynamic / staticity, and modern / modern. The classic degree is quantified such as 0.3, and the abstract / realistic degree is 0.9. In this example, the value of each feature is 0.0
It is represented by each numerical value obtained by dividing .about.1.0 into 10 steps, but this is merely an example, and any unit system can be adopted as necessary. On the other hand, the extraction of physical features is mechanically performed by computer processing. That is, each image forming the 50 sample analysis images is first subjected to various image processing such as density value conversion, noise removal, blur restoration, contour detection enhancement, and connection portion extraction. Then, various feature extraction operations are performed based on the image processing result, and finally, for example, linearity is 0.9, contrast is 0.8, complexity is 0.1, etc. Physical feature scale values are quantified.

In this way, if the sensible feature amount and the physical feature amount are extracted from each image constituting the sample analysis image, then the mapping rule for defining the correlation between both the feature amounts is set. , Created by statistical methods. As a result, for example, sharpness / softness = 0.9
The mapping rule f is obtained such as the linearity-0.2 / complexity.

After that, the mapping rule f obtained above is obtained.
By using, the value of the affective feature amount is mechanically obtained from the 1000 images to be registered, and the physical feature amount, the affective feature amount, and other arbitrarily added feature amount ( The creation date, the cost, etc.) are respectively associated with the corresponding images to form an image database, and finally stored in the image storage unit 5. An example of the image database created in this way is shown in FIG. As shown in the figure, in this example, the image No. The creation date, cost, brightness, simplicity, warmth, brightness, linearity, etc. are adopted as the feature quantities. It should be noted that this is merely an example simplified for the sake of explanation, and the number of colors, complexity, number of stitches (in the case of embroidery pattern), curvature, symmetry, and softness are shown as needed. Any feature amount can be adopted, such as.

Next, the image retrieval processing which is the main part of the present invention will be described. First, details of the feature amount selection setting process will be described with reference to the flowchart of FIG. In the figure, when the processing is started, for example, C which constitutes the display unit 2
On the screen of the RT display device, a vertical axis feature amount item is displayed as a menu (step 301), and then a state of waiting for a selection setting operation is set (step 302 NO). In this state, when the selection setting operation is performed using the mouse, the keyboard or the like which constitutes the operation unit 3 (step 302
(YES), the selected and set feature amount is assigned to the vertical axis and stored (step 303). After that, similarly, a horizontal axis feature amount item is displayed on the screen as a menu (step 304), and then a state of waiting for a selection setting operation is set (step 305 NO). In this state,
When a selection setting operation is performed using a mouse, a keyboard or the like that constitutes the operation unit 3 (YES in step 305), the selected and set feature amount is assigned to the horizontal axis and stored (step 306). Thereafter, until the predetermined setting completion operation is performed (NO in step 307), the above-described processing (steps 301 to 306) is repeated to selectively set or change the arbitrary feature amount as the vertical axis and the horizontal axis. You can

FIG. 6 shows an example of the menu display screen used for the feature amount selection setting operation. Although FIG. 6 shows a screen display example of the search result, the menu display of the feature amount items is the same during the feature amount selecting and setting operation. As shown in the figure, the screen 9 of the CRT display device is displayed.
A vertically long rectangular window 10 is provided on the right side of
In this window 10, "creation date", "cost", "color number", "complexity", "number of stitches", "curve degree",
The feature amount name 11 is displayed such as "symmetry", "softness", and "brightness", and the set axis name 12 is displayed at the top of the window 10 as "vertical axis setting". Has been done. Further, a vertically movable cursor 13 is displayed on the right side edge of the window 10. By moving the cursor 13 up and down with a mouse or the like, the window 10 is moved.
The feature amount name 11 in the above can be appropriately scrolled. Then, for example, by moving a predetermined cursor to any of the feature amount names 11 with a mouse and performing a click operation in that state, an arbitrary feature amount can be selectively set on the vertical axis or the horizontal axis.

Next, details of the image search processing will be described with reference to the flowchart of FIG. In the figure, when the process is started, as shown in the screen display example of FIG. 6, nine images G1 to G9 are displayed on the substantially left half of the screen 9 of the display device.
A square window 14 is displayed which can be displayed vertically and horizontally without overlapping each other. A vertical axis cursor 15 that can move in a predetermined range along the vertical direction is provided on the right edge of the window 14, and a horizontal axis cursor 16 that can move within a predetermined range along the horizontal direction at the lower edge. Is displayed. Since FIG. 6 shows a screen display example of the search result, the vertical axis cursor 15 is located slightly above the vertical center, and the horizontal axis cursor 16 is located slightly right from the horizontal center. However, at the start of the search, each cursor 1
5 and 16 are initial positions within the movement range (for example, the central position)
Is displayed (step 401).

After that, from the coordinate position determined by the initial position of each of the cursors 15 and 16, nine similar images similar thereto are retrieved and extracted by a similar image extraction operation described later with reference to FIG. 5 (step 402). The nine similar images G1 to G9 thus extracted are arranged vertically and horizontally in the window 14 and displayed (step 403). Note that, as described above, FIG. 6 shows a screen display example of the search result, and is different from the nine images displayed at the time of disclosure of the search.

An example of the similar image extraction calculation will be described with reference to FIG. In this example, the vertical axis represents “cost” as a feature amount.
However, on the horizontal axis, "creation date" is assigned as the feature amount in the above-described selection setting process (steps 301 to 307). The large number of images 17 stored in the image storage unit 5 are represented as points on the XY coordinate system with the vertical axis representing “cost” and the horizontal axis representing “creation date”, and the vertical axis cursor 15 and horizontal axis. The position of the cursor 16 is represented by a dotted line parallel to each axis on the XY coordinates. In the first similar image extraction calculation, as shown in FIG. 5A, the coordinate position determined by the intersection of the vertical axis cursor position and the horizontal axis cursor position is recognized as the desired image position. And the distance between each registered image position is calculated,
Nine images (indicated by nine arrows radially extending from the desired image position in the figure) selected in order from the shortest of the obtained distances are similar to or close to the desired image. It is extracted as 9 similar images. In the second similar image extraction calculation, as shown in FIG. 5B, similarly, the coordinate position determined by the intersection of the vertical axis cursor position and the horizontal axis cursor position is recognized as the desired image position, After that, a search area 18 having a predetermined shape (square shape in the figure) that can be expanded and contracted around the desired image position is set. The size of the search area 18 is adjusted so that exactly 9 images are included therein, and in this state, the 9 images are extracted as 9 images similar to or close to the desired image. . In this way, the vertical axis cursor 15 and the horizontal axis cursor 16 are calculated by the first or second similar image extraction calculation.
That is, nine images that are similar to or close to the desired image designated by are searched and extracted.

Returning to the flow chart of FIG. 4, when starting the search, nine images are displayed in the window 14 in correspondence with the reference positions of the vertical axis cursor 15 and the horizontal axis cursor 16, and then the apparatus moves the cursor. The state of waiting for the moving operation is set (step 404 NO). In this state, the search operator displays the nine images G1 to G9 displayed on the screen.
For example, when requesting an image of “higher cost” and “older creation date”, the vertical axis cursor 15 is moved to the reference position ( As shown in FIG. 6, the cursor is moved to a position slightly higher than the center position in the vertical direction, and the cursor on the horizontal axis is moved to a position slightly right of the reference position (center position in the horizontal direction) as shown in FIG. , 406).
Then, nine images are newly searched / extracted from the image storage unit 5 (step 407) by the above-described image extraction processing shown in FIG. 5 corresponding to the new cursor position, and the extracted nine images are extracted. The images G1 to G9 are displayed vertically and horizontally in the window 14 on the screen 9 in the same manner as described above (step 408). After that, unless a predetermined search completion operation is performed from the mouse, keyboard, etc. (step 409N
O, 410 YES), and the above-described image extraction display processing by moving the cursor is repeated (steps 409 YES, 40).
5 to 408), utilizing this, the search operator can efficiently search for a desired image or pattern sample. In particular, as shown in FIG. 6, in the screen layout in this embodiment, the horizontal axis cursor 16 and the vertical axis cursor 15 are X in the XY coordinate space to be searched.
The search operator is the search target shown in FIG. 5 because the windows 14 for displaying nine images are arranged corresponding to the coordinate space area, respectively. You can proceed with the search work while actually imagining the coordinate space.

In the above embodiment, the case where the "cost" and the "creation date" are both changed and the search is performed, but it is of course possible to fix either one and change only the other. For example, it is possible to use any method such as searching for “a lower cost image” while keeping the “creation date” fixed, and even if there are many uncertain search conditions, the desired image can be displayed. You can search efficiently.

Next, claims 6 to 9 of this application
Alternatively, an embodiment of the invention described in claims 14 to 18 will be described with reference to FIGS. 7 to 13. The hardware configuration of the image search apparatus of this embodiment is the same as that shown in FIG.

FIG. 7 shows an example of the image database used in the image retrieval apparatus of this embodiment. As shown in the figure, each image data includes an image No. Is attached, and as the feature amount, the sensory feature amount such as soft / hard degree, static / dynamic degree, simple / decorative degree, modern / classical degree, abstract / realistic degree, etc. is adopted. ing.

Details of the search key selection setting process will be described with reference to the flowchart of FIG. 8 and the screen display example of FIG. When the process is started in the flowchart of FIG. 8, first, the axis number n is initially set to "1" (step 801), and subsequently the feature amount item is displayed on the screen as a menu (step 802). FIG. 12 shows a screen display example for setting the search key selection. As shown in the figure, a vertically long rectangular window 20 is arranged in the approximate center of the screen 19 of the display device, and within this window 20, sharp / soft, simple / decorative, static / dynamic, The feature amount item 22 is displayed in a menu like modern / classical. A cursor 21 for menu scrolling, which can move vertically, is displayed on the right side edge of the window 20. In this state, the mouse 23 is used to operate the cursor 21 up and down to display the required feature amount item 22 in the window 20, and in that state, the desired feature amount item 22 is selected and set by clicking the mouse 23. Then, (YES in step 803), the selected and set feature amount item is assigned to the L1 axis which is a search axis described later (step 804). Then, the axis number n is incremented by +1 (step 805), and the above-described selection setting operation is repeated, whereby the feature amount items that are sequentially selected and set are assigned to the L2 axis, the L3 axis, and the L4 axis, respectively ( Steps 804 and 806 YES).

Here, L1 axis, L2 axis, L3 axis and L4
Axis and nine windows W0, W11, W12, W21,
The positional relationship with W22, W31, W32, W41 and W42 will be described with reference to the search result screen display example of FIG.
As shown in the figure, on the screen 19 of the display device, as a virtual axis, a vertically extending axis L1 and a horizontally extending axis L are displayed.
2. Four axes are set: an axis L3 extending 45 degrees diagonally to the upper right and an axis L4 extending 45 degrees diagonally downward to the right. In addition, on the screen 19, 9 rows and 3 columns are arranged.
Square windows W0, W11, W12, W2
1, W22, W31, W32, W41 and W42 are displayed. The window W0 is located at the center of intersection of the axes L1 to L4, and the windows W11 and W12 are aligned with the axis L1.
Located on both sides of the window W0 across the top,
Windows W21 and W22 are located on both sides of window W0 on axis L2, windows W31 and W32 are located on both sides of window W0 on axis L3, and windows W41 and W42 are further located on axis L4.
The windows W0 are arranged on both sides of the window W0. As a result, these four pairs of windows (W11 and W12, W21 and W22, W31 and W3
2 and W41 and W42) are arranged in point symmetry about the window W0. In this example, the feature value item “soft / hard degree” is assigned to the axis L1 by the above-described search key selection setting process (see FIG. 8), and similarly, the axis L2, the axis L3, and the like. "Static / dynamic degree", "modern / classical degree", and "simple / decorative degree" are assigned to the axis L4, respectively. The characteristic item names are additionally displayed on the respective axes L1 to L4 as shown in the figure.

Next, on the premise of the above screen configuration, the image search processing in the image search apparatus of this embodiment will be described with reference to the flowchart of FIG. When the process starts
Although not shown here, a plurality of exemplary images are displayed as a menu on the screen 19 of the display device (step 901), and then, a state of waiting for an operation of selecting and setting one of the exemplary images is set (step 902). NO). In this state, if any of the exemplary images is selected and set (YES in step 902), the selected and set exemplary image is input as a reference exemplary image that serves as a reference for a search to be described later (step 903). Thereafter, the similar image extraction processing for each characteristic item is performed based on the input reference exemplary image (step 904).

FIG. 10 shows the details of the similar image extraction processing for each characteristic item.
It is shown in the flowchart. When the process is started in the figure, after the axis number n is initially set to "1", the reference example image is displayed on the feature amount ("soft / hard degree" in this example) assigned to the L1 axis. The similar images located at both the positive and negative adjacent positions are extracted one by one (step 1002). After that, the axis number n is incremented by +1 (step 1003), and similarly, each feature amount (in this example, “static / dynamic degree”, “modern / classical degree”, “simple / decorative degree”). ))
When a pair of similar images adjacent to the reference exemplary image are sequentially extracted (step 1002) and extraction of all four axes L1 to L4 is completed (step 1004), the process ends, and subsequently, FIG. Returning to the flow chart of No., the process proceeds to the image display process (step 905).

The details of the image display process are shown in the flowchart of FIG. When the process is started in the figure, the previously input reference exemplary image is arranged in the central window W0 (step 1101), and subsequently, the axis number n is initially set to "1", and then the L1 axis is set. The two similar images extracted in association are the windows W11 adjacent to the central window W0,
It is placed on W12 (step 1103). Then, increment the axis number n by +1 (step 1
104) in the same manner, the two similar images extracted in association with the respective axes L2 to L4 are the pair of windows (W21 and W22, W31 and W32, W31 and W32, which are adjacent to the central window W0.
W41 and W42) (Step 11)
03) When the image arrangement / display for all four axes L1 to L4 is completed (YES in step 1105), the process ends.

When the above image display processing is completed, FIG.
As shown in FIG. 5, the reference exemplary image is displayed in the central window W0, and the surrounding windows (W11 and W12, W21 and W22, W31 and W32,
In W41 and W42), only eight adjacent similar images are collectively displayed in a point-symmetrical arrangement relationship for each feature amount. Therefore, the search operator can understand at a glance what similar images are present around the reference exemplary image. In this state, for example, when an image more modern than the image displayed in the window W31 is desired, the search operator operates the mouse, the keyboard, or the like that constitutes the operation unit 3 to operate the window W31.
A selection operation is performed to the effect that the image displayed in 1 should be a new reference exemplary image. Then, returning to the flowchart of FIG. 9,
It is determined that there is a selection setting operation (step 906Y
ES), after that, the image that has been displayed in the window W31 until then is re-input as a new reference exemplary image (step 907), and based on the re-input exemplary image, the characteristic item-based similar image extraction processing described above is performed. (Step 904) and the image display process (step 905) are executed again. As a result, the image previously displayed in the window 31 is now displayed in the central window W0, while the surrounding windows (W11, W12, W2) surrounding it.
1 and W22, W31 and W32, W41 and W42)
Only eight adjacent similar images of the new reference similar image are collectively displayed in a point-symmetrical arrangement relationship for each feature amount. At this time, the window W31 displays a more modern image than the image that has been displayed so far.
Therefore, by repeating the above-described example image re-input operation in an arbitrary axis direction, a desired image can be efficiently searched while effectively browsing the vicinity of a certain image.

In the above embodiment, the central window W
0 has a square shape, and four pairs of windows (W11 and W12, W21 and W22, W31 and W3) that surround it.
2, W41 and W42) have the same square shape,
Although the point symmetrical arrangement of vertical 3 rows × horizontal 3 columns is used as a whole, the shape, size and arrangement of these windows can be designed arbitrarily. For example, if the central window is circular and the surrounding windows are substantially fan-shaped by circularly dividing the ring surrounding them, similar images can be searched for more axes corresponding to the number of radial divisions. Further, in this example, the peripheral window surrounding the central window W0 is set as a single layer, and only two adjacent similar images of the reference exemplary image are extracted in the similar image search. However, in the fan-shaped arrangement described above, the peripheral window surrounding the central window is extracted. Double,
If three layers are used, exemplary images similar to the primary exemplary image, secondary similarities, and tertiary similarities to the reference exemplary image can be simultaneously displayed in each axial direction. Further, in the above embodiment, four pairs of surrounding windows (W11 and W12, W21 and W22,
If one of W31 and W32, W41 and W42) is re-input as a new exemplary image, the image displayed in the window moves to the central window W0 as a new reference exemplary image and returns to the position of the original window. Was configured to display the first similar image to the new reference similar image, but one of four pairs of surrounding windows (W11 and W12, W21 and W22, W31 and W32, W41 and W42) was used as a mouse. Each time a click operation is performed, the images of the second similarity, the third similarity, the fourth similarity ... Of the image displayed in the central window W0 at that time may be sequentially displayed.

[0071]

According to the invention described in claim 1 or claim 10 of this application, even when there are many uncertain retrieval conditions, two conditions are appropriately selected and set from those conditions. As a result, the desired image can be searched efficiently.

According to the invention described in claim 2 or claim 11 of this application, any two can be easily selected from the plurality of feature scale items.

According to the invention described in claim 3 or claim 12 of this application, it is possible to perform a search operation while easily visualizing a two-dimensional space defined by the two feature scales that are selectively set. .

According to the invention of claim 4 or claim 13 of this application, it is possible to more easily visualize the two-dimensional space defined by the two feature scales that have been selected and set, and perform the search operation. You can

According to the invention described in claim 5 or claim 14 of this application, a certain number of similar images having a certain similar relationship with the search reference image can be reliably extracted and displayed.

According to the invention of claim 6 or claim 15 of this application, a desired image can be efficiently searched while effectively browsing the vicinity of a certain image.
According to the invention of claim 7 or claim 16 of this application, while maintaining the outline of each display image in a rectangular shape, eight primary similar images existing in the periphery thereof are displayed with good visibility. You can

According to the invention described in claim 8 or claim 17 of this application, it is possible to easily understand the meaning of each search axis direction.

According to the invention described in claim 9 or claim 18 of this application, while effectively browsing the vicinity of a certain image, a similar image in a specific search axis direction is narrowed down as necessary, and desired. Images can be searched efficiently.

[Brief description of drawings]

FIG. 1 is a block diagram showing a hardware configuration of a first embodiment of the present invention.

FIG. 2 is a configuration diagram of an image database used in the first embodiment of the present invention.

FIG. 3 is a flowchart showing details of a feature scale selection setting process according to the first embodiment of this invention.

FIG. 4 is a flowchart showing details of image search processing according to the first embodiment of this invention.

FIG. 5 is an explanatory diagram of similar image extraction processing according to the first embodiment of this invention.

FIG. 6 is a diagram showing a screen display example of search results in the first embodiment of the present invention.

FIG. 7 is a configuration diagram of an image database used in the second embodiment of the present invention.

FIG. 8 is a flowchart showing details of search key selection setting processing in the second embodiment of the present invention.

FIG. 9 is a flowchart showing the overall configuration of image search processing in the second embodiment of the present invention.

FIG. 10 is a flowchart showing details of similar image extraction processing for each characteristic item in the second embodiment of the present invention.

FIG. 11 is a flowchart showing details of image display processing in the second embodiment of the present invention.

FIG. 12 is a diagram showing an example of a screen display for search key selection setting in the second embodiment of the present invention.

FIG. 13 is a diagram showing a screen display example of search results in the second embodiment of the present invention.

FIG. 14 is a diagram for explaining the mutual relationship among the kansei feature value, the physical feature value, the mapping rule, and the like.

[Explanation of symbols]

 1 Central Processing Unit 2 Display Section 3 Operation Section 4 Printer 5 Image Storage Section 6 Image Scanner 7 System Bus 8 Design Sample 9 Screen 10 Window (for Displaying Feature Quantity Item) 11 Feature Quantity Name 12 Setting Axis Name 13 Cursor (Feature Quantity Menu) Scroll 14) Window (for batch display of images) 15 Vertical axis cursor 16 Horizontal axis cursor 17 Image in XY coordinate space 18 Search area 19 Screen 20 Window (for displaying feature amount item) 21 Cursor (for scrolling feature amount menu) 22 Characteristic amount item 23 Mouse G1 to G9 image L1 to L4 Search axis W0 Central window W11, W12 Adjacent window in axis L1 direction W21, W22 Adjacent window in axis L2 direction W31, W32 Adjacent window in axis L3 direction W41, W42 Axis L4 Adjacent window in direction

Claims (18)

[Claims] 1. An image storage step of storing a plurality of images with a plurality of types of feature values assigned thereto, and a search for selectively setting any two of the plurality of types of feature amounts as search feature amounts. Feature amount selection setting step, a feature amount adjusting step of arbitrarily adjusting the values of the two feature amounts selected and set as the search feature amount within a predetermined adjustment range, and a feature amount adjusting step of the adjusted two feature amounts. A similar image extraction step of extracting a fixed number of similar images from the image storage means based on the value; and an image display step of collectively displaying the extracted fixed number of similar images on the screen of the display device. An image retrieval method comprising: 2. The search feature quantity selection / setting step comprises:
Through the operation of selecting any two from the plurality of feature quantity items displayed on the screen of the display device, any two of the plurality of kinds of feature quantities are selected and set as the search feature quantity. The image search method according to claim 1, wherein: 3. The feature adjustment step is performed through the operation of individually moving a pair of cursors that are movably displayed in a horizontal direction and a vertical direction on a screen of the display device, respectively. The values of the two feature amounts selected and set as the feature amounts can be arbitrarily adjusted within a predetermined adjustment range.
Image retrieval method described in. 4. The image displaying step is a step of displaying a certain number of the extracted similar images in a rectangular window displayed on a screen of a display device in a closely arranged manner without overlapping each other and collectively. The feature amount adjusting step is performed by individually moving a pair of cursors that are movably displayed within a predetermined range along a horizontal edge on either the upper or lower side or a vertical edge on the left or right of the rectangular window. , 2 selected and set as the search feature amount
2. The image retrieval method according to claim 1, wherein the values of the one feature amount can be arbitrarily adjusted within a predetermined adjustment range. 5. The similar image extracting step is a search specified by the pair of adjusted feature quantities on a two-dimensional coordinate defined by two feature quantities selected and set as the search feature quantity. Euclidean distances between the reference image and each of the images stored in the image storing step are respectively obtained, and a fixed number of images selected in order of closeness to the obtained distances are extracted as target similar images. The image search method according to claim 1, wherein: 6. An image storage step of storing a plurality of images with a plurality of types of feature amount values respectively stored therein, and a selection for selectively setting n feature amounts from the plurality of types of feature amounts. A setting step; a reference example image input step for inputting one reference example image to serve as a reference for similar image search; and the input reference example from the images stored in the image storing step. Item-like similar image extraction step of searching for and extracting a similar image similar to the image in both positive and negative directions along each of the selected n set feature amounts, and the input reference exemplary image is located at the center. However, the similar images extracted bidirectionally for each of the selected and set items are positioned on both sides of the line on the straight line passing through the reference exemplary image position, and each of the items is displayed around the reference exemplary image. Image display step to display similar images An image search method characterized by comprising the. 7. The number of feature amounts selectively set in the selection setting step is four, and the number of similar images searched in the item-based similar image extracting step is positive or negative along each feature amount. 7. The image search method according to claim 6, wherein there is one image for each direction, and in the image display step, eight similar images are arranged in point symmetry around the reference exemplary image and are collectively displayed. 8. The image search method according to claim 6, wherein the image display step additionally displays an item name corresponding to each straight line passing through the reference exemplary image position. 9. The method according to claim 6, further comprising a similar image re-inputting step for inputting one of the similar images arranged around the reference exemplary image as a new reference exemplary image.
Image retrieval method described in. 10. An image storage unit that stores a plurality of images with a plurality of types of feature amounts assigned to each image, and any two of the plurality of types of feature amounts are selectively set as a search feature amount. Search feature amount selection and setting means, a feature amount adjusting device that can arbitrarily adjust the values of the two feature amounts that have been selected and set as the search feature amount within a predetermined adjustment range, and the adjusted 2 Similar image extracting means for extracting a fixed number of similar images from the image storage means based on the value of one feature amount, and image display for collectively displaying the extracted fixed number of similar images on the screen of the display device. An image retrieval apparatus comprising: 11. The search feature quantity selection setting means is configured to select any two of the feature quantity items displayed on the screen of the display device as a menu, and perform the operation to select the plurality of kinds of feature quantities. 11. The image search apparatus according to claim 10, wherein any two of the above are selectively set as search feature amounts. 12. The search is performed by the feature amount value adjusting means through an operation of individually moving a pair of cursors that are movably displayed in a horizontal direction and a vertical direction on a screen of the display device respectively within a predetermined range. The values of the two feature amounts selected and set as the feature amount for use can be arbitrarily adjusted within a predetermined adjustment range.
The image search device described in 0. 13. The image display means displays a fixed number of the extracted similar images in a rectangular window displayed on a screen of a display device in a closely arranged manner without overlapping each other, and The feature amount value adjusting means individually moves a pair of cursors movably displayed within a predetermined range along one of the upper and lower horizontal edges and the left or right vertical edge of the rectangular window. The image search apparatus according to claim 10, wherein the values of the two feature quantities selected and set as the search feature quantity are arbitrarily adjustable within a predetermined adjustment range. 14. The similar image extracting means is specified by the pair of adjusted feature amount values on a two-dimensional coordinate defined by the two feature amounts selected and set as the search feature amount. Euclidean distances between the search reference image and each image stored in the image storage means are respectively obtained, and a certain number of images selected in order from the closest distance among the obtained distances are extracted as target similar images. The image search device according to claim 10, wherein 15. An image storage unit for storing a plurality of images with a plurality of types of characteristic amount values respectively stored therein, and a selection for selectively setting n characteristic amounts from the plurality of types of characteristic amounts. Setting means, reference exemplary image input means for inputting one reference exemplary image to be a reference for similar image search, and the input reference exemplary image from the images stored in the image storage means. Item-like similar image extraction means for searching and extracting a similar image similar to the above in both positive and negative directions along each of the n feature amounts that have been selected and set, and the input reference exemplary image is located at the center. , The similar images extracted bidirectionally for each of the selected and set items are positioned on both sides of the reference exemplary image on a straight line passing through the reference exemplary image position, and the similarities for each item are set around the reference exemplary image. Image display means for displaying an image, and Image retrieval device to. 16. The number of feature quantities selected and set by the selection setting means is four, and the number of similar images searched by the feature quantity similar image extracting means is in accordance with each feature quantity. 16. The image retrieval apparatus according to claim 15, wherein there is one image in each of the positive and negative directions, and the image display means collectively displays eight similar images arranged in point symmetry around the reference exemplary image. 17. The image search device according to claim 15, wherein the image display means additionally displays an item name corresponding to each straight line passing through the reference exemplary image position. 18. A similar image re-inputting device for inputting one of similar images arranged symmetrically around the reference exemplary image as a new reference exemplary image. The image search device described in 1.