JPH09282324A - Video retrieval system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To give a diversity of indexes capable of video customization and retrieve video according to them. SOLUTION: Indexing is performed by an indexing means 1 which gives color information, edge information, etc., and a concept indexing means 2 which gives conceptual information on the basis of information that an area knowledge storage means 4 has and retrieval conditions defined by a retrieval defining means 6 which defines retrieval conditions with the edge information and a concept retrieval defining means 7 which defines retrieval conditions of conceptual level are inspected by a matching means 9 and a concept matching means 10 to provide a partial scene of desired video. An indexing customizing means 3 and a retrieval definition customizing means 8 make it possible to change an indexing system and a retrieval system without altering the said means only by addition to the area knowledge storage means 4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video search system, and more particularly to a video search system having a video search customization mechanism.

[0002]

2. Description of the Related Art In recent years, a technique for semi-automatically extracting content information such as the position and size of a subject appearing in an image based on edge information of each frame of image data has been developed. A video search method for searching for a desired scene in a video has been proposed.

Video search systems using these video contents are broadly classified into video images without using domain knowledge about the video images (for example, in the case of a soccer video image, information specific to the soccer field such as the ground, goal, ball, player, and touch line). There are a general-purpose video search system that performs indexing and video search of a video, and a video search system that provides area-specific complex video search by embedding area knowledge and performing area-specific detailed indexing.

[0006] The former general-purpose tool takes out an area which is considered to be a subject occupying a large area in a frame as a subject information candidate by using edge information, color distribution information, etc. for each frame of an image, and further extends between frames. The motion vector of the area is extracted semi-automatically and indexed.

At the time of search, the search condition is expressed by these primitive indexes, and a frame having a region similar to the search condition and a scene having a motion similar to the motion vector of the search condition can be searched. .
By using a primitive expression, it is possible to search by exemplifying a rough sketch or the like for an object or movement (for example, a search condition as shown in FIG. 1A) that is difficult to express with a keyword or the like.

The latter special tool interprets a subject appearing in a frame by using the area knowledge, and conceptual information of the area (for example, in the case of a soccer image, a ball, a goal, a player, etc.).
Index by. Further, based on the interpreted subjects, their movements, and the positional relationship between them, a specific scene is indexed as concept-level event information of a region (for example, in the case of a soccer image, dribbling, shooting, etc.).

In the search, the search can be performed using the conceptual level indexing information of the area having a higher degree of abstraction than the former tool. For example, as shown in FIG. 1B, it is possible to perform a word-based search of the concept of a region that cannot be expressed compactly with the primitive index provided by the former tool.

[0008]

In the conventional general-purpose video search system, since the indexing information is a primitive, the vocabulary that can be used for the search is a primitive such as color information, edge information, coordinate information, etc. Limited
Therefore, for example, in an example of a soccer video, it is difficult to define a search condition that can be easily defined by using information on the word-level abstraction degree of a region such as "a scene in which a goal appears".

In the conventional dedicated video search system, it is possible to define the search condition by the expression of a high level of abstraction using the domain knowledge, but on the other hand, there is a delicate motion that cannot be expressed well by the expression of a high level of abstraction. The problem is that we cannot provide the means to define. Further, since domain knowledge is embedded in the system, it is difficult to add a new interpretation to the default interpretation method for each user or to customize the original interpretation method itself.

It is an object of the present invention to search a scene that cannot be precisely defined by an abstract concept and to define it by using the primitive information of the image, and to add appropriate area knowledge. Therefore, it is an object of the present invention to provide a video search system capable of search definition with the concept of the area, for scene retrieval that can be expressed compactly with the concept of the area.

[0011]

The video search system of the present invention uses the indexing means for providing an indexing function based on the primitive information of the video and the area-specific knowledge stored in the area knowledge storage means. It has a concept indexing means for performing indexing by the area concept defined by combining.
An indexing customization unit is provided as a unit for customizing the concept indexing function, and an index storage unit is provided as a unit for storing the index generated by the indexing unit and the concept indexing unit.

Further, the video search system of the present invention includes a search definition means for providing a function of defining a search condition using primitive information, and a search condition based on a concept having a high degree of abstraction using primitive information and area knowledge. And a search definition customizing means having a function of customizing the concept search defining means and the concept search defining means.

The video search system of the present invention searches the index matching the search condition by matching the index with the search definition for each video in the video storage means holding the video data, and acquires the scene designated by the index. And a concept matching means for performing similar matching between the concept index and the concept search definition.

[0014]

The indexing means extracts color information, edge information, etc. from each frame of the video data stored in the video storage means, obtains subject area information reflected in each frame, and generates an index. The area storage unit stores the correspondence between the geometric combination of the primitive indexes generated by the indexing means and the area-specific concept corresponding to the combination. The concept indexing unit interprets the primitive index obtained by the indexing unit using the correspondence relationship held by the area storage unit to generate the concept index. The index storage means stores the index and the concept index in association with the video data.

The search is started when the searcher defines the search condition using the search definition means or the concept search definition means. The search definition defines search conditions using the primitive information provided by the indexing means. The concept search definition defines the search condition by combining the concept provided by the domain knowledge using the domain knowledge or the information corresponding to the index provided by the indexing means.

After that, the matching means or the concept matching means checks whether or not the index stored in the index storage means or the concept index matches the defined search condition, and the index that matches the search condition. Ask for. Further, the scene corresponding to the video data corresponding to the index is obtained and returned to the searcher as a search result.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION A video search system according to an embodiment of the present invention will be described below with reference to the drawings.

FIG. 2 is a system configuration diagram of the video search system of the present embodiment.

As shown in FIG. 2, the video search system according to the present embodiment has an indexing unit 1, a concept indexing unit 2, an indexing customizing unit 3, an area knowledge storage unit 4, an index storage unit 5, and a search definition unit. 6, the concept search definition means 7, the search definition customization means 8, the matching means 9, the concept matching means 10, and the video storage means 11.

The indexing means 1 is the video storage means 1
The color information, the edge information, and the like are extracted from each frame of the video data held by 1, and the subject area information reflected in each frame is obtained and an index is generated.

FIG. 3 is a diagram showing a storage structure of indexes and conceptual indexes generated by the indexing means 1 of this embodiment.

Referring to FIG. 3, the term 50 is used to index the video data. Video data is composed of frames that hold two-dimensional image information of a video at a specific time. A subject appearing over a plurality of frames is represented by a hotspot, which is a series of area information for each frame.

The indexing means 1 generates a frame index, which is area information for each frame shown by 51 in FIG. The frame index is a region 510 surrounded by edges in a specific frame and its attribute 5.
It is composed of 11 pieces of primitive information.

Further, the indexing means 1 represents the same subject by automatically extracting the identity between different area information existing in a plurality of frames or semi-automatically by the input from the index creator. Region series 520
Is generated as the hot spot index 52.

The concept indexing means 2 interprets the primitive index obtained by the indexing means 1 using the correspondence relationship held by the area storage unit (area knowledge storage means 4) to generate a concept index. The concept index represents a set 530 of specific regions of a frame, a frame concept index 53 represented as a set of concept names indicating an abstract concept represented by the set, and a temporal change over frames of a specific concept. It consists of hotspot concept indexes 54 and 540 that indicate concepts, and scene concept indexes 55 and 550 that represent the concept of the theme of the scene represented by the collection of hotspot concept indexes 54 and 540.

The indexing customizing means 3 provides a mechanism for customizing the domain knowledge to be used, a function for adding the domain knowledge, and customizes the definition of the concept index. By using this means, it is possible to give different customized definitions for the same concept.

The area knowledge storage means 4 holds area knowledge such as definition information of an abstract concept peculiar to the area of the subject appearing in the image.

Referring to FIG. 4, the domain knowledge is composed of a spatial concept 40 representing an abstract concept in one frame and a spatio-temporal concept 41 including a temporal concept across frames.

The spatial concept indicated by 401 is represented by a set of a concept name, which is an abstract representation of the concept, and a concept definition, which is the definition of the subject constituting the concept. Concept definition is area information,
An area pattern 402 is formed by a combination of video primitive information such as color information, and another area pattern 403 is defined by a combination of video primitive information and a spatial positional relationship between them.

The spatiotemporal concept 402 is composed of a concept name that is an abstract representation of the concept and a construct 410 that is a definition of the temporal and spatial characteristics of the subject constituting the concept. Construction concept 41
0 is composed of a series of spatial concepts that represent the spatiotemporal properties of a particular subject and the temporal relationship between the spatial concepts.

The index storage means 5 includes indexes 51, 52 and concept indexes 53, 54, 5.
5 is stored in association with the video data holding the frame or hotspot which is the indexing target.

The search definition means 6 is the indexing means 1
Provides a mechanism for defining search conditions using primitive information as shown by 51 and 52 in FIG.

The concept search defining means 7 has a higher abstraction level than the level provided by the concept indexing means 2 and provided by the area knowledge storage means 4 of a specific area as shown by 53, 54 and 55 in FIG. It provides a mechanism to define search conditions using the information in.

The search definition customizing means 8 is a means for selecting or changing the domain knowledge used in the search.

The matching means 9 is one of the indexes stored in the index storage means 5 and is the search definition means 6
An index that matches the search condition defined by is obtained. Further, when the index is the frame index, the video data of the frame corresponding to the frame number 51 of the frame index is returned to the searcher as the scene of the search result. When the index is a hotspot index, a scene including all frames corresponding to the frame number series 520 of the hotspot index is returned to the searcher as a search result.

The concept matching means 10 selects a concept index that matches the search condition defined by the concept search defining means 7 from the hotspot concept index and the scene concept index, which are the concept indexes stored in the index storage means 5. A search is performed and the frame corresponding to the frame concept index or the scene corresponding to the hotspot concept index or the scene concept index is returned to the searcher as a search result scene.

The concept search defining means 7 makes it possible to express the search condition compactly with the concept of the area, and the search defining means 6 allows the search condition difficult to be expressed by an abstract word to be a frame index. It is possible to express in detail with primitive video-related information such as a hotspot index.

The video storage means 11 is a storage means for holding video data.

Next, the operation of the video search system of this embodiment will be described.

The video search system of the present embodiment comprises an indexing operation for indexing video data,
An indexing operation is used to define a search condition for a video by using an index stored in advance, and an operation for searching for a scene which is a portion of a desired video.

Referring to FIG. 5, in preparation for the indexing process for explaining the indexing operation, the indexing customizing means creates an object (hereinafter referred to as Creator) for creating an index.
Initialization is performed (A1). An object is, for example, a program having a set of data and an operation related to the data. Creator is the setting information 301 held by the indexing customizing means.
(See FIG. 10) is held as data, and a procedure for generating each index based on the setting information 302 shown in FIG. 10 is held.

The actual customization is performed by customizing the domain knowledge, and the indexing customizing means provides means for appropriately using the customized information.
The domain knowledge is customized by the space concept 40 of the domain knowledge.
This is realized by inheriting the object (class) that defines (see FIG. 4) and defining the definition of difference as a subclass. For example, in the case of the hotspot concept index, if the left defining the concept on the left among the positional relation operators defined by default is redefined and further it is desired to change the similarity of the area pattern, FIG.
It is realized by inheriting the class definition of the original hotspot concept index and redefining the changed part as shown in.

This Creator is set in the indexing means 1 and the concept indexing means 2 (A
2) The video data to be indexed is read frame by frame (A3), and the indexing work is continued until the final frame is processed. Image primitive information such as color information, edge information, and texture information is extracted for each read frame, and a frame index 51 (see FIG. 3) is generated based on these pieces of information.

A frame concept index is generated in step (A7) based on the frame index information generated in step (A6). In step (A7), the area knowledge set in the indexing customizing means 3 is obtained (realized by acquiring the area knowledge set in the initialization of Creator), and obtained in step (A6). The frame index 51 (see FIG. 3) selected by the concept indexing means 2 selects a frame index that matches the spatial concept 40 of the domain knowledge (see FIG. 4), and the frame concept index 53 (of the type set in Creator) ( (See Figure 3)
Generate As shown in steps (A704) and (A710) of FIG. 6, when the specific area knowledge is not set, the frame concept index is not set. By the same condition check in steps (A9) and (A10), the processing of FIG. 5 realizes both the case of generating only the primitive index and the case of generating both the primitive index and the concept level index. To do.

When the generation of the frame index and the conceptual frame index for all the frames is completed, all the frame indexes are shown to the index creator in step (A8), and the same subject specified by the index creator (user) is expressed. The hotspot index 52 (see FIG. 3) is generated according to the input of the frame index set.

In step (A9), step (A7)
Based on the frame concept index obtained in step (A8), all frame concept indexes are shown to the index creator in the same manner, and the frame concept index set representing the same subject designated by the index creator (user) is input. Accordingly, the hotspot concept index 54 (see FIG. 3) is generated.

In step (A10), region knowledge is acquired based on the hotspot concept index obtained in step (A9) by the same processing as (A702) to (A705) in FIG. 6 (A1002) (see FIG. 9). ), After matching the space-time spatiotemporal concept 41 (see FIG. 4), a hotspot concept index set that matches the concept is indexed as a scene index (A1004) (see FIG. 9).

Next, the search operation will be described with reference to FIG. In the search processing, the search definition customizing means 8 generates a search creator using the search environment information set by the user who executes the search (B1), and uses it to generate the matching means 9 or the concept matching means 10.
It is started by setting (B2). When the user generates the concept search Creator in the search definition means 6, the concept matching means 10 is set to the normal search Creator.
When the tor is generated, the matching means 9 is activated.

Similar to the case of the Creator, the search Creator includes search definition setting information 801 (see FIG. 14) and search definition generation information 802 (see FIG. 14). There are two types of search creators, a normal search creator that creates a search definition and a concept search creator that creates a concept search definition. In the case of a retrieval concept procedure, the domain knowledge that was defined and used during the concept indexing can be selected and used, or the interpretation of these domain knowledge can be redefined (for example, the interpretation of position predicates can be redefined. ), It is also possible to search by an abstract concept with a definition different from that of the index definer. To take advantage of these customized domain knowledge, search Create
Define domain knowledge with or. If not defined here, the domain knowledge used in the concept indexing is used in the search process.

The search definition process (B3) is realized by the procedure shown in FIG. The matching unit 9 or the concept matching unit 10 requests the set search creator to generate a search definition (B301). Search Cr
The eator creates the search definition means 6 or the concept search definition means 7 set in itself (B302), and instructs the search definition creation (B303). Each search definition has its own search condition input means defined in the search definition, and this is used to acquire the search condition input from the user (B30
4), generate a search definition including an index or a concept index and variables of these constituent elements (B30).
5).

The matching means 9 or the concept matching means 10 subsequently reads all the indexes in order, performs a search definition and matching based on the similarity of the indexes (B6), and performs matching with all the indexes. After that, the matched portion in the video is displayed as a search result (B7).

The index creator can optionally create a default index and a customized index by pre-storing the index used when creating the Creator in the setting condition (see FIG. 10). Also, steps A704 and A71 in FIG.
As indicated by 0, as an initial value, it is possible to generate only an index at a primitive level or both an index at a concept level by distinguishing between a designation not generating a concept index and a designation generating it. It also has the feature that makes it possible.

Further, the difference between the default indexing work and the customized indexing work is shown in A1 of FIG. 5 and A702-A705, A7 of FIG.
08-A711 and other indexing customizing means 3
All of the above are implemented in the, and at the time of generating a primitive level index, at the time of generating a customized primitive level index, and at the time of generating a concept index based on the default domain knowledge, the concept index based on the customized domain knowledge. The characteristic is that the same indexing means (see FIG. 5) can be used also when generating. Due to this feature, the indexing means (see FIG. 5) can be used as a reusable software component for creating any index.

Therefore, the index creator only needs to set the portion of the indexing customization means 3 called from the component, for example, the domain knowledge defined by the creator himself, and the customized index. It is possible to utilize the means for making. Unlike the conventional parts, which can use only preset knowledge, the parts can be customized and can be used for a wider range of purposes.

In the past, since the entire indexing means 1 of various kinds with various customizations was provided as a component, the indexing component developer can upgrade the version such as an algorithm for extracting the image primitive information. However, it was very inefficient because it was necessary to upgrade all the products. On the contrary,
In the case of the indexing means 1 of the present embodiment, the indexing component developer, when he wants to upgrade,
It suffices to create one version of new software component having the same application interface as the component and the indexing customizing means 3 up to now.

Also regarding the search, since the search creator manages the default search condition input, the search matching work and the difference between these customized works, the matching means 9 and the concept matching means 10 are the indexing means 1. Similarly, it can be reused as a software component, and the same effect as described above can be obtained.

Here, an example of the indexing process will be described. For example, referring to FIG. 5 and FIG. 10, before starting the indexing process after step A3, redefine definitions such as the meanings of the operators of the default indexes as shown in 30 and 302 of FIG. You can For example, 301 indicates that the index creator uses a hotspot concept index that is uniquely redefined. For example, an object-oriented language such as C ₊₊ can be easily realized by defining a class having the setting information as an attribute and having the generation information as a method as a subclass of the Creator class and compiling it.

Further, as shown in FIG. 11, redefinition of an index such as a hotspot concept index is realized by inheriting the original index and overriding the part to be redefined. In the case of FIG. 11, the concept of left defined by the magnitude relationship of the x-coordinate of the center of gravity can be redefined as the magnitude relationship of the x-coordinates of the leftmost and rightmost sides of the circumscribing rectangle of both concepts. , The rest of the definitions can use the original definitions as they are.

The frame index 5 in FIG.
Area information such as 10 and attribute information regarding the area FIG. 3
The extraction of 511 is solved by using the existing image processing technology [4].

Regarding the frame concept index and the scene concept index, there is a possibility that the area knowledge to be used and the operator to be used in the concept have been customized. In order to make the difference invisible to the indexing means, it is solved by using polymorphism provided by an object-oriented language such as C ₊₊ . Polymorphism is a function in which multiple different classes provide functions with the same function name and the same arguments, and the different behaviors defined by each class can be obtained at runtime without distinction by the caller.

For example, FIG. 15 is an example of the frame concept index generation. Step A7001
The frame concept index can be obtained by executing ~ A7005, but here, regardless of the type of the frame concept index, the type specified by the Creator object is generated, and the same polymorphism is applied to any index. You can call the match function. Therefore, it is possible to generate different indexes by executing matching processing of different indexes without changing the code of the indexing means.

To realize the domain knowledge, for example, regarding the concept of a soccer shoot, refer to FIG. 3 and FIG.
51 is an approximate representation of a player in a yellow uniform, and the hotspot concept index, which is a series of regions in each frame that matches this, is the concept of a player. The frame index in FIG. 16 is an approximation of the goal concept, and the hotspot concept index, which is a series of regions of each frame that matches this, is the concept of a goal. For example, "exists and goals" that combine these with the concept of a ball
exists (ball) and exists (player)
The concept of "and contents" (goal, ball), that is, the concept that the player, the goal, and the ball are reflected and the ball is in the goal is a spatiotemporal concept that defines a view with a shot.

The search definition provides a method of redefining the definition of this concept. The method of redefinition is similar to the method shown in the frame concept index, and the domain knowledge of the operator definition redefined in the search definition and the operator whose concept definition was just redefined in the frame concept index is the same. Override and take effect.

In the search, if the search definition means 6 is used, the search input can be made by a rough sketch as shown in FIG. For example, this rough sketch is defined as an index, such as the frame index of 51 in FIG. In the search, the hotspot index 52 having the movement indicated by the arrow is matched.

On the other hand, when the concept-level search definition is performed, the search input as shown in FIG. As a result, a scene concept index having a dribble as a concept name is searched.

Next, a second embodiment of the present invention will be described.

This embodiment will be described with reference to FIG. In the first embodiment described above, a method of reading the frame of the video data frame by frame in step (A3) is adopted.

For this frame reading, for example, there is a method in which only the main frame is cut out and indexing is performed only for that part while preprocessing for cutting out the cut.

Now, as an embodiment, a cut-out process is added to step (A3) in FIG. A cut is extracted as a frame in which color information and edge information on the screen largely change before and after that. Steps (A
It will be the return result of 3). As a result, the time required for indexing an image over a long time can be significantly reduced.

Next, other modified embodiments of the present invention will be described. In the first and second embodiments, a concept name is used as information from the outside such as a frame concept index, a hotspot concept index, and a scene concept index. I only gave them. For this reason, for example, even if it is known that the player is reflected, annotation information such as the player's name, age, and career is not given. If necessary, annotation information such as these keywords can be added to the concept name portion.

In addition, when inputting search conditions using the search definition means, for example, not only rough sketches but
It is also possible to give a live-action movie similar to the image to be searched as a sample.

[0072]

The first effect is that the image indexing / searching is performed by the image primitive information such as the color information and the edge information, and the concept having a high degree of abstraction is obtained by adding the information to the interpretation by the domain knowledge. It means that it is possible to do it with specific information. The reason is that the customization function is provided separately from the indexing means and the matching means, and either one can be set by the customization function before the indexing and search processing.

The second effect is that even when either primitive information or conceptual information is used, the common indexing means and search means can be used, so that an application that uses these means from the outside is completely eliminated. Both can be used without changing the indexing, and the difference in processing that occurs between the two when performing the search is absorbed by the customizing means, providing a system configuration that does not need to change the indexing means or the searching means at all. Because it is.

The third effect is that the development efficiency of the index means and the search means is improved. The reason for this is that the customizing part is separated from the indexing means that conventionally provided various types of indexing means and search means with various customizations as products, and the same indexing means for all products. This is because the method of providing the search means is adopted. For this reason, conventionally, it was necessary to upgrade all the indexing means and the search means when upgrading the version, whereas in this method, only one indexing means and the search means are upgraded. The version upgrade will be reflected in all products.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a search condition definition method.

FIG. 2 is a system configuration diagram showing a video search system according to an embodiment of the present invention.

FIG. 3 is a storage structure diagram of an index / concept index.

FIG. 4 is a storage structure diagram of domain knowledge.

FIG. 5 is a processing flow chart of indexing.

FIG. 6 is a processing flowchart of frame concept index generation.

FIG. 7 is a processing flowchart of hot spot index generation.

FIG. 8 is a processing flow diagram of hotspot concept index generation.

FIG. 9 is a processing flowchart of scene concept index generation.

FIG. 10: Crea of indexing customization means
It is a structural diagram of tor.

FIG. 11 is a diagram showing an example of customization of domain knowledge.

FIG. 12 is a flowchart of search processing.

FIG. 13 is a flowchart of a search definition creation process.

FIG. 14: Search Create of search definition customizing means
It is a figure of a structure of or.

FIG. 15 is a diagram showing an example of frame concept index generation.

FIG. 16 is a diagram showing an example of a frame index.

Claims (8)

[Claims] 1. A means for assigning indexes such as edge information and color information in each frame of an image to input image data, and information such as rough sketch given as a search condition using the assigned index. And a means for searching for a related video by using the video search system. 2. A means for assigning an index of a concept level unique to an area to the input image data by using area information together with edge information in each frame of the image, and a search using the added index. And a means for searching related videos based on a search sentence given as a condition. 3. The video search system according to claim 1 or 2, further comprising a customization unit capable of redefining a concept previously defined as a concept index or redefining interpretation of domain knowledge. Video search system. 4. The video retrieval system according to claim 1, further comprising an indexing means for providing an indexing function based on video primitive information, an area knowledge storage means, and an area included in the area knowledge storage means. A video retrieval system comprising: a concept indexing means for indexing by a region concept defined by combining primitive information using unique knowledge. 5. The video search system according to claim 4, further comprising indexing customizing means as means for customizing the concept indexing function,
A video search system comprising an index storage means as a part for storing an index generated by the indexing means and the conceptual indexing means. 6. The video search system according to claim 4, further comprising a search definition means for providing a function of defining a search condition using primitive information, and a high degree of abstraction using primitive information and area knowledge. A video search system comprising: a concept search defining means for providing a function of defining a search condition by concept; and a search definition customizing means having a function of customizing the concept search defining means. 7. The video search system according to claim 4, further, by matching an index and a search definition for each video in a video storage unit that holds video data, an index that matches the search condition is searched, and the index is Matching means that provides the function to acquire the specified scene,
A video search system comprising: a concept matching means for performing similar matching between a concept index and a concept search definition. 8. An indexing unit for extracting color information, edge information, etc. from each frame of the video data held in the video storage unit, obtaining index of subject area information reflected in each frame, and an indexing unit. By using the correspondence relationship that the area storage unit stores the correspondence between the geometric combination of the primitive indexes and the area-specific concept corresponding to the combination, and the area storage unit has the primitive indexes obtained by the indexing means. Concept indexing means for interpreting and generating a concept index, index storage means for storing the index and the concept index in association with the video data,
The searcher defines a search condition by using the primitive information provided by the indexing means, or the area knowledge provides the concept provided by the area knowledge or the information corresponding to the index provided by the indexing means using the area knowledge. And a concept search definition means for defining a search condition, and a matching means or a concept for obtaining an index that matches the search condition by checking whether the index or concept index stored in the index storage means matches the defined search condition A video search system comprising a matching means, wherein a scene corresponding to video data corresponding to an index is obtained and returned to a searcher as a search result.