JP2000250944A - Information providing method, information providing device, information receiving device, and information description method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To extract only a portion of interest from a huge amount of multimedia information and selectively provide individual information for each user. Kind Code: A1 Abstract: By performing moving image analysis, sound / speech analysis, and text analysis on multimedia information in a database, an attribute is obtained which indicates what kind of information the information is. Is given. The search unit 105 extracts a part of the multimedia information that is of interest to the user based on the attribute and a user profile registered in advance. The associating unit 106 associates a representative image (still image) of the plurality of partial images with the multimedia image, and displays a list of the representative images and the attributes.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information providing method, an information providing apparatus, and an information receiving apparatus for providing multimedia information including video, music, text, and the like to an unspecified number of users via the Internet or the like. The present invention relates to an information providing method, an information providing apparatus, and an information receiving apparatus for selecting only information of interest to a user from a large number of multimedia information and providing individual information to each user. In addition, the present invention relates to a method of describing a user profile representing user's preference.

[0002]

2. Description of the Related Art In recent years, with the development of information infrastructure, many multimedia information has been distributed to homes by CATV (cable television broadcasting), digital satellite broadcasting, the Internet, and the like.
Here, a wide variety of programs are provided, and the number of service channels is on the order of hundreds or thousands. For this reason, it is becoming difficult for the user to appropriately select information from hundreds or thousands of channels or more than tens of thousands of programs in the channels.

In order to solve this problem, there has been proposed a broadcasting device which automatically records a program of interest to a user by using information of an electronic program guide sent from a broadcasting station (for example, Japanese Patent Application Laid-Open No. H11-163, 1988). Japanese Unexamined Patent Publication No. Hei 7-135621 "Video equipment"
There is).

In this proposal, a program that is considered to be of interest to a user is selected from information in an electronic program guide based on keywords registered in advance, and program filtering is automatically performed for each user.

[0005] In addition, there is a demand for a program provider to know what program each viewer is watching in order to create an attractive program providing program. In the past, monitors were recruited in advance, and had them fill out a questionnaire form which programs they watched, and then totaled them. However, since this questionnaire was a completed form, only rough information on whether a certain program was viewed or not was obtained.

[0006]

In a conventional system for automatically selecting an enormous number of programs provided by a program provider according to personal preference, the selection is performed on a program basis, and the selection is made roughly. There is a problem that you can only select. For example, considering programs such as news and variety, in these programs, one program is composed of units of "topics" and "corners", and for a user, one program is included in one program. There are many cases where only the department is interested. However, since automatic recording is performed on a program basis, all programs are selected and recorded from the beginning to the end of a single program, and if the user does not see the entire program, information that is of particular interest is included in any part of the program. I do not know if it is. Therefore, even if the recorded program is filtered and selected with great care, it is necessary to watch all the recorded one program from the beginning to the end, and there is much waste in terms of the amount of recording media used and the user's time. There was a problem.

[0007] When such filtering is performed, a CM included in a program may be cut off. If the broadcast program is not a paid program but a free program that uses advertising fees as a source of revenue, watching the CM is an important factor in acquiring a sponsor for providing the program. It is a big problem not to be.

Further, the conventional viewing survey is performed on a program-by-program basis, and is insufficient for grasping the user's preferences and the like.

It is an object of the present invention to provide the following information providing method, information providing device, information receiving device, and information description method.

A first object is to provide an information providing method, an information providing apparatus, and an information receiving apparatus capable of accurately selecting and providing only a part of the multimedia information, not a program unit, but a part that the user is really interested in. Is to get.

A second object is to provide an information providing method, an information providing apparatus, and an information receiving apparatus capable of accurately selecting and providing only a part of the multimedia information which is of interest to a user rather than a program unit. It is an object of the present invention to obtain an information providing method, an information providing apparatus, and an information receiving apparatus that are surely provided with a commercial message that a program provider wants a viewer to view.

A third object is to provide an information providing method, an information providing apparatus, and an information receiving apparatus capable of accurately selecting and providing only a part of the multimedia information, not a program unit, but a part that the user is really interested in. An information providing method capable of recording a user's viewing history and updating a user profile representing the user's preference accordingly;
An object is to obtain an information providing device and an information receiving device.

A fourth object of the present invention is to provide a description method for providing a user profile relating to the personal information of the user described above.

[0014]

In order to solve the above-mentioned problems and achieve the object, the present invention uses the following means.

(1) Multimedia information is compiled into a database by attaching an attribute to each information unit, and partial information of the multimedia information matching the user's preference is retrieved from the database by matching the user profile with the attribute. And providing the retrieved partial information to the user.

(2) First for storing multimedia information
The multimedia information is analyzed using a database of at least one of an analysis method of moving image analysis, sound / speech analysis, and text analysis, and the multimedia information is divided according to the analysis result, and obtained for each division. And a second database for storing attributes to be input or input from the outside, and searching for a second attribute according to a user profile to extract a predetermined attribute, and to search for multimedia information in the first database. A search unit that selects partial information of the multimedia information according to the extracted attribute.

(3) Information having a database for storing multimedia information as an attribute of at least one analysis result of moving image analysis, sound / speech analysis, and text analysis for each section, or an attribute input from the outside. In the information receiving apparatus connected to the providing server, the database is searched according to the user profile to extract a predetermined attribute, and multimedia information corresponding to the extracted attribute among the multimedia information in the database. An information receiving apparatus comprising a search unit for selecting partial information of the above.

(4) A database for storing multimedia information as an attribute as a result of at least one of moving image analysis, sound / speech analysis, and text analysis for each section, or an attribute input from the outside, A search unit for searching a database according to the profile, extracting a predetermined attribute, and selecting partial information of the multimedia information according to the extracted attribute among the multimedia information in the database; An information receiving apparatus connected to the server, comprising: an associating unit that obtains a representative image of the partial information of the multimedia information selected by the search unit and forms a display image including the representative image and the extracted attribute. Information receiving device.

(5) An information description method for describing a user profile used for searching for partial information included in multimedia information, wherein the user profile includes at least one information group relating to personal information of a user. An information description method, wherein the information group includes a plurality of pieces of information classified into at least one item, and the information is described according to a priority of a user in the item to be classified.

According to the present invention, with respect to multimedia information, an attribute indicating the content of a video is acquired by at least one of moving picture analysis, sound / speech analysis, and text analysis. Since information is finely divided based on attributes and can be managed in units of division, a corresponding partial video can be selected based on a user profile registered in advance.

Also, since the commercial message is similarly selected according to the user profile, the commercial message that the program provider wants the viewer to view is surely provided.

Further, according to the present invention, it is possible to know which program has been viewed and how many times at the same time as viewing the program, so that the user's preference can be accurately grasped.

Further, according to the present invention, it is possible to provide a user profile indicating the user's preference necessary for appropriately selecting only a part of the multimedia information that the user is really interested in.

[0024]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an information providing method, an information providing apparatus, an information receiving apparatus, and an information description method according to the present invention will be described below with reference to the drawings.

First Embodiment FIG. 1 is a diagram showing a basic configuration of an information providing apparatus according to a first embodiment of the present invention. Multimedia information database 101 composed of randomly accessible media, commercial message (CM) database 108, program attribute database 103, CM attribute database 10
9. A user profile database 104 is provided. Note that the classification of these databases is for convenience of explanation, and one database may be physically used.

The multimedia information database 101 stores a large number of multimedia program information to be provided. C
The M database 108 stores a large number of CM information provided together with the free programs. CM attributes indicating the contents of the CM information are stored in the CM attribute database 109 for each piece of CM information.

[0027] Multimedia information is a large number of programs provided by broadcasters and Internet information providers. If these are analog data, they are converted to digital data in advance and managed and stored in the multimedia information database 101. The format of digital data is M
The data may be compressed data in the PEG-2 format or data in the DV compression format. In the multimedia information, a “title name” is assigned to each program, a “frame number” is assigned to each frame in the program, and if a title name and a frame number are given, media that can be accessed from any position, for example, Stored on a hard disk. However, the media is not limited to the hard disk,
A new random accessible medium such as a DVD-RAM (ROM) may be used. The multimedia information data does not need to maintain the original analog image size and quality, and may use an MPEG-1 or MPEG-4 compression format that can save image capacity depending on the application.

The output of the multimedia information database 101 is supplied to the attribute extraction unit 102. Attribute extraction unit 10
2 performs predetermined analysis processing on all information held in the multimedia information database 101, divides information according to the analysis result, and assigns a program attribute indicating the content for each division unit (for example, frame unit). Is to be granted. The program attributes are managed in the program attribute database 103 for each division unit.

It is assumed that the CM attribute (CM program unit) is known in advance, and when the CM information is stored in the CM database 108, the CM attribute is simultaneously stored in the CM attribute database 109. However, it is assumed that the program attributes are newly obtained after the program information is stored in the multimedia information database 101 and then read from the database 101 and obtained as a result of the analysis processing. Of course, the program attributes may also be separately obtained by the subjective opinion of the operator, and the operator may input the program attributes into the program attribute database 103 using the keyboard 110. When the automatic analysis of the program attributes and the determination by the operator are used together, more accurate attributes can be given to the program information (indexing).

The attribute extracting unit 102 performs video analysis, sound / voice analysis, and the like on the multimedia information.

With regard to video analysis, video images are structured based on information such as cuts and camera movements (pan and zoom) in which video scenes change instantaneously using well-known moving image analysis. A method for determining the attribute of a video can be used.

The location where the scene changes instantaneously can be detected by comparing the similarity between the frame images of the video. The similarity can be calculated by calculating a color frequency histogram for each image and comparing the images. If the similarity is low, it is a point where the scene changes instantaneously.

In addition, the motion parameters of the camera determine the optical flow indicating where each pixel has moved from the two images, and dominant under the assumption that the optical flow coming from the background occupies the majority. The camera movement is calculated from the optical flow.

When the camera is panning, most of the optical flows appear in parallel, and when the camera is zoomed, the optical flows are directed to one point.
Details thereof are described in, for example, "Document (1): Hirono Ueno, Takafumi Miyatake, Satoshi Yoshizawa," Proposal of Interactive Video Editing System Applying Recognition Technology ", Transactions of IEICE (D-II), Vol.
J75-D-II, No. 2, pp. 216-225 "and" Reference (2): Masahiro Shibata, "Video Content Description Model and Its Application to Video Structuring," IEICE Transactions (D-I
I), VOL. J78-D-II, No. 2, pp. 754
−764 ”.

In sound / speech analysis, music has few silent parts and contains frequency components not found in human voice.
It can be separated from human voice, and human voice has the opposite characteristic,
Then, voices can be separated using the difference in pitch between men and women.

For details of the male / female voice discrimination method, see "Reference (3): Keiichi Minami, Akito Akutsu, Hiroshi Hamada, Yoshinobu Tonomura," Video Indexing Using Sound Information and Its Applications, "Transactions of the Institute of Electronics and Communication Engineers. (D-II), VOL.J81-D-II, N
o. 3, pp. 529-537 ", which shall be referred to here and will not be discussed in depth here.

In this way, the video can be divided from the video information and the audio information, and an attribute can be assigned to each division.

For example, sound data is analyzed to separate a portion where music is flowing from a male / female voice portion, and a video scene related to the sound data is related to a portion where music is flowing. That is, attributes are assigned to scenes related to a voice portion, a scene related to a male voice portion, and a scene related to a female voice portion.

When character data related to the video is included along with the video, the data is subjected to text analysis processing and used for attribute assignment. In other words, U.S. images contain character data called closed captions,
If such a situation is available, it is possible to perform a text analysis process using a conventional natural language processing technology and to assign an attribute according to the content.

That is, from the character data attached to the image, an attribute based on the analysis result of the character data content for the video can be given for each division unit.

The user profile database 104 is a file in which information on user's own preferences and fields of interest (user profiles) is registered for each user, and is managed for each user. The user profile is created by inquiring the user or taking a questionnaire in advance. Note that the user profile can be additionally modified by the user using the keyboard 110 as appropriate. The user profile is composed of text information, as shown in FIG. 2, and includes a keyword indicating the user's preference or field of interest. FIG. 2A shows preference information on the type of program,
(B) is preference information on program contents, (c) is preference information on program production, (d) is a personal profile,
(E) is a keyword / keyphrase indicating a preference.
(A) is program category and genre information, such as suspense, drama, documentary, sports,
Variety, news, etc. (B) is the type of scene in one program, for example, if it is a movie, it is information of a favorite scene such as an action scene, a love scene, a climax scene, and the like. For news, politics,
Economics, sports, society, etc. (C) is information of people who produce programs, for example, movie directors, actors, actresses, composers, lyricists, arrangers, etc. of the music used. It also includes information on the region where it was produced. (D) is personal information, such as age, gender, occupation, hometown, and birthday. (E) is a keyword or a key phrase indicating the user's preference, and is a variety of keywords and a key phrase such as a favorite food, a favorite matter, a favorite sport, a hobby, a favorite maxim, and the like. In the notation of FIG.
The number in parentheses is the number of preference information for each item. If a single item contains multiple pieces of information,
For example, the items are arranged in descending order of user priority. The priority of the information may be such that the number of pieces of information temporarily matched at the time of keyword matching described later is counted, and the order of the information is appropriately changed in descending order of the number.

[0042] The search unit 105 stores the program attribute database 1
03, search the CM attribute database 109 and select an attribute that matches the user profile in the user profile database 104.
Partial information of the user's interest is known. The search method may be to find a matching attribute based on the keyword of the user profile. In this keyword matching, if a thesaurus (thesaurus; a classification dictionary, a synonym dictionary, an index for information retrieval) is also used, an attribute matching a keyword similar to the user profile can be detected. The thesaurus also includes a dictionary that takes into account language shifts due to countries and regions, and gender and generational shifts (that is, to eliminate these shifts).

Thus, related videos can be identified / retrieved in detail in units of scenes, units relating to audio, units relating to character data, etc., and it is possible to selectively extract partial images in the category of the user's interest. Become.

The search unit 105 supplies the search result to the association unit 106.

The associating unit 106 performs a process for reproducing information. That is, the multimedia information database 1
01, the information in the CM database 108 is associated with the search result, and the partial information is reconfigured according to the user's preference.

The display unit 107 displays the image reconstructed by the association unit 106, and also includes a speaker for reproducing music information. By viewing such partial information, the user can view the video of the part of his or her interest without any unnecessary part.

The above is a brief description of the basic configuration of the present system.

Hereinafter, a method for realizing each process will be described in detail.

The details of the process of the attribute extracting unit 105 will be described with reference to FIG. FIG. 3 is a processing flow illustrating the processing of the attribute extracting unit 102 in detail.

The attribute extraction unit 102 analyzes all the multimedia information stored in the multimedia information database 101, and analyzes the information in units of frames, not in units of programs, to obtain attributes.

The multimedia information includes sound, text, etc. in addition to images. Therefore, the analysis of multimedia information is divided into three parts: text analysis, moving image analysis, and sound / speech analysis, but the processing order may be performed in any order.

In the text analysis, closed caption information in the video is extracted (steps S1 and S2).
Morphological analysis is performed (step S3), and keyword analysis is performed based on the morphological analysis (step S4). This is performed for all video programs in the multimedia information database 101.

In the moving image analysis, a cut of the moving image in the video is detected (steps S1 and S5), and then the motion parameters of the camera are extracted (step S6), and the video is divided based on this (step S7). , And so on. This is performed for all video programs in the multimedia information database 101.

In the sound / speech analysis, the sound in the video is identified (steps S1 and S8), the speech is recognized (step S9), and the keyword is extracted based on the recognition (step S10). The procedure is as follows. This is performed for all video programs in the multimedia information database 101.

Thus, the text analysis, the moving image analysis, and the sound / voice analysis are completed, and the respective analysis results are obtained.

Various attribute information relating to the multimedia information can be obtained by the video analysis processing according to the above-described procedure. However, the information is further processed by the integration processing (step S11) for integrating the attribute information individually output. High level processing is performed.

Note that text analysis, moving image analysis, sound
As individual techniques required for voice analysis, as described above, existing known analysis techniques may be used.

For example, in text analysis, a closed caption included in a video is extracted, and the role of a word is decomposed in morphological analysis. Among them, important keywords that describe scenes such as proper nouns are extracted. The keyword uses not only proper nouns but also information such as a high frequency of appearance.

In the moving image analysis, a video is divided by extracting information on a scene in which a scene changes suddenly or motion information of a camera (see Document (1)). Also, in sound / speech analysis,
For example, it is sufficient to separate a music or a voice by acoustic identification and divide it into male and female by voice recognition (see Reference (3)), and to extract keywords using voice recognition.

The role of the integration process is to associate the attribute information output in each process and store it as a database, and to generate new attribute information by matching the attribute information with each other.

For example, a process for associating individual processes is as follows.

Assuming that processing is performed in units of video division,
It is assumed that an important proper noun keyword exists in the keyword. The keyword is the caption (explanatory text, explanatory text)
, It is not known exactly from which video frame to which video frame.

Therefore, the location where the keyword exists is identified by using voice recognition, and the keyword is given as an attribute to the partial video at the location where the voice is continuous.

As a result, the analysis result is generated as a table as shown in FIG. 4, for example. That is, in FIG. 4, the title of the program is “news”, and “politics” representing characters and situations as keywords as attributes,
There are “economy” and “weather forecast”, and the appearance time (frame) of the screen related to each keyword is “0: 0”
0-0: 15 "," 0: 15-0: 16 "," 0:23 "
−0: 25 ”is a table picked up. In other words, a table is obtained by dividing a video by time (frame) for each program title and adding important keywords (attributes) appearing during that time. I have.

Next, the processing of the search unit 105 will be described in detail with reference to FIG. FIG. 5 is a diagram showing a processing flow showing the processing of the search unit 105 in detail. Here, a search related to program information will be described, but a search related to a CM is similar. The search unit 105 stores the program attribute database 103
With reference to the information of the user profile database 104 and the attribute of the user's interest, the corresponding partial video can be selected.

Each keyword is selected one by one from the user profile database 104, and related words are picked up using the thesaurus dictionary (step S21, step S21).
22).

When a related word is found, matching of the found related word with a keyword indicated in an attribute stored in the program attribute database 103 is performed. Information on whether or not the frame is a partial video is recorded (step S2).
3, Step S24, Step S25). Keyword matching is also subject to matching if they are exactly the same.

The above is the details of the processing of the search unit 105.

FIG. 6 shows an example of partial video information collected by keyword matching and registered in step S25. Here, the user profile database 104
Shows that one of the keywords is “shopping”. Searching for shopping information using the thesaurus data, keywords such as “department store” and “bakery” are selected and matched with the keywords in the program attribute database 103 to obtain the corresponding keywords as shown in FIG. Partial video information is obtained.

Although the above description has been made only on the selection of multimedia information, the same can be applied to the CM.

Next, the details of the associating unit 106 will be described with reference to the flowchart shown in FIG. The associating unit 106 obtains information as shown in FIG. 6 from the searching unit 105, extracts partial information from the multimedia information using these information, and configures a display screen (heading screen) for providing information. Here, a case where a thumbnail image as shown in FIG. 8 is displayed will be described.

First, all keywords (in the example of FIG. 8,
It is checked whether or not processing has been completed for shopping, public facilities, transportation / banking, and health / hospital (keywords). If not, processing is continued (step S3).
1). Then, a partial video selected for one keyword is acquired from the multimedia information database 101 (step S32). When acquiring a partial video from the multimedia information database 101, if random access can be performed at a sufficiently high speed, time code information (frame information) may be used as it is. If not, a partial image is obtained by copying or reducing the screen size, or by changing the compression ratio and compression method.

One or a plurality of still image frames are acquired as representative images from the acquired partial images (step S33), and are used as materials for forming a screen. The attribute of the representative image is associated with the representative image, the representative image is associated with the partial video (steps S34 and S35), and the information is described using HTML (step S3).
6).

If the partial image selected by each keyword has been processed, the processing of the next keyword is performed; otherwise, the above processing is repeated (step S37).

It is determined whether or not processing has been completed for all keywords (step S31). If processing has been completed for all keywords, the content described in HTML is output or sent to the display unit (step S38). If not, the process continues.

FIG. 8 shows an example of a screen created by the associating unit 106 in this manner. Here, a case where the keywords of the user profile are “shopping”, “public facilities”, “traffic / bank”, and “health / hospital” is shown. For this reason, partial videos of programs related to words such as “department store” and “bakery” related to “shopping” are acquired, and a representative image, which is one frame of the partial video, is pasted in a horizontal line like an index. It is rare.
The scattered CMs are advertisements of program sponsors. The CM can be selected based on the matching between the program attribute and the user profile, similarly to the program information. Therefore, it is possible to select the CM most relevant to the selected multimedia information. Further, on the screen of FIG. 8, a link is attached to a partial video from each representative image, and the partial video is displayed by a click button.

To create such a screen, HT
What is necessary is just to write the required description in ML. Where HTML
Is an abbreviation of HyperText Markup Language, and the HTML refers to a page description language used as a general format of information provided by the Internet World Wide Web (abbreviated WWW or W3) service. ing. HTML is SGML (Standard Generalized)
It is based on Markup Language, and by inserting markup called tag (TAG) into a document, it is possible to specify the logical structure of the document and the link between the documents.

Here, WWW is a client / server information service on the Internet, but a network user can access information using a dedicated Web browser. The provided information is an HTML document called a homepage, a Web page, or a WWW page, which is connected by a hyperlink, and the information can be displayed by following the link.

Since the document handled by the WWW can include multimedia information, and can execute a special process by executing a program on the server side, this function can be provided by a unique information search service. Can be used for

As an example of displaying a CM together with a selected program, an example using an HTML document has been described. Next, a case where a selected program or a part of a program is displayed as a video is displayed. The display method will be described.

The example of FIG. 9 is an example in which a CM banner advertisement is played along with a program video. A banner advertisement is placed on the bottom of the program video while streaming the video. Also in this case, as described above, it is possible to fly to WWW pages according to the user's instruction.

As shown in FIG. 10, there is also a method of displaying a CM image as a child screen of a program image screen. Even with this method, linked WWW pages are available.

As described above, the CM related most to the program scene or the C related to the user's preference from the keyword etc. related to the selected program scene and the user's preference.
Since M can be selected, it is possible to play an effective advertisement. It should be noted that the CM may be inserted according to the request of the sponsor regardless of the user's preference.

As described above, according to the present embodiment,
Apply at least one analysis method of moving image analysis, sound / speech analysis, and text analysis to the database that stores multimedia information and the multimedia information provided from the database, and based on the analysis result, the multimedia information And analyze results are managed in units of division. Then, the analysis result is searched according to the user profile, the partial information of the multimedia information is selected according to the user's preference, and the selected partial image is provided to the user.

Thus, an information providing method, an information providing apparatus, an information receiving apparatus, and an information description method capable of accurately selecting and providing only a part that the user is really interested in instead of a program unit from a large number of multimedia information. Is provided. As a result, it is possible to eliminate the disadvantage of the conventional example that the entire program is recorded or viewed even though only a part of the program is desired to be viewed.

Further, by making the CM into a database in the same way as the program information and selecting it in the same way as the program information, the program provider can surely view the CM that the viewer wants the viewer to view. Moreover, since the CM is adapted to the user's preferences and interests, the effect of the advertisement can be enhanced. As a result, when only a part of the video is cut out and recorded or viewed, the disadvantage of the conventional example in which the CM desired by the program provider to be viewed is cut can be solved.

Hereinafter, another embodiment of the present invention will be described.
In the description of the other embodiments, the same parts as those of the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

A series of processing in the first embodiment, the multimedia information stored in the database is analyzed, and the attribute of the program information as the analysis result is also stored in the database and managed, and the appropriate multimedia information is stored in accordance with the user profile. It is conceivable that the processing amount is too large to overload the broadcast receiving device owned by the user for reading out the partial information from the database, associating the partial information, and providing the information in a reconfigured manner. In order to solve this, a server / client system is used, and some processing is performed by the server.

FIG. 11 shows a server / client system according to a first modification of the first embodiment. here,
Multimedia information database 101, CM database 108, attribute extraction unit 102, program attribute database 103, CM attribute database 109, search unit 105,
And an associating unit 106 provided on the server side, and a display unit 107
And a user profile database 104 provided on the client side.

FIG. 12 shows a server / client system according to a second modification of the first embodiment. here,
Multimedia information database 101, CM database 108, attribute extraction unit 102, program attribute database 103, CM attribute database 109, and search unit 10
5 is provided on the server side, the associating unit 106, the display unit 10
7, and a user profile database 104 is provided on the client side.

FIG. 13 shows a server / client system according to a third modification of the first embodiment. here,
A multimedia information database 101, a CM database 108, an attribute extraction unit 102, a program attribute database 103, and a CM attribute database 109 are provided on the server side, and a search unit 105, an association unit 106, and a display unit 10
7, and a user profile database 104 is provided on the client side.

As described above, when the system of the first embodiment is constituted by the server / client system, FIG.
As shown in FIG. 12, there is a method in which only a part for storing and sending a user profile to a server and a part for receiving and displaying a search result from a server are placed in a client, and as shown in FIG. Is also shared by the client, and as shown in FIG.
A method in which the part to be searched is also shared by the client is conceivable. How much the client will share depends on how much processing power the client has. When all the components other than the database and the attribute extraction unit 102 are left to the client as in the configuration of FIG. 13, it is necessary to download the processing result to the client. Therefore, this configuration depends not only on the processing capability of the client but also on the information storage capability and the line capability for downloading. However, since there is an effect that processing can be distributed, the client has high performance, and it can be said that this is an effective configuration method when the line is a CATV, an optical fiber, or an intranet.

Second Embodiment The present invention can manage the attributes of video in units of frames, not in units of programs. If this is applied, it is possible to conduct viewing investigation in units of frames. Can be solved, and a viewing history useful for analyzing the user's preferences and interests can be obtained.

FIG. 14 is a diagram showing a system configuration of the second embodiment that can record viewing history information in frame units.
This system is different from the system of the first embodiment shown in FIG. 1 in that the viewing information control unit 120, the history information recording unit 12
1, a playback control unit 122 and a correspondence unit 123 are added. Viewing information control unit 120
Acquires from the playback control unit 122 viewing information when the user views a certain information program (media), and transmits the information to the associating unit 123. The associating unit 123 stores the program attribute corresponding to the viewing information in the program attribute database 103.
From the history information recording unit 12
Feed to 1. The viewing information is information of the viewed multimedia information, and information of which place has been viewed and how many times has been viewed.

The history information in which the viewing information and the program attribute are associated with each other may be uploaded to the program provider at the same time as the recording in the recording unit 12, or may be performed for a certain period or in a certain amount. At the time when the data is recorded in the program 121, the data may be uploaded to an external database (a database held by the program provider). The correspondence between the viewing information and the video may be simply history information of information about which time section of which video was watched and how often, or how to use video scene delimiter information and attribute information. Statistical information on how often a particular scene was viewed may be aggregated and used as history information.

In the method of configuring the server / client system according to the present embodiment, it is desirable that the history information recording unit 121 has the same site as the viewing information control unit 120, but various modifications are possible as described below.

FIG. 15 shows a server / client system according to a first modification of the second embodiment. here,
Multimedia information database 101, CM database 108, attribute extraction unit 102, program attribute database 103, CM attribute database 109, search unit 105,
The association unit 106 and the association unit 123 are provided on the server side, and the display unit 107, the user profile database 104, the viewing information control unit 120, the history information recording unit 12
1, and the reproduction control unit 122 is provided on the client side.

FIG. 16 shows a server / client system according to a second modification of the second embodiment. here,
Multimedia information database 101, CM database 108, attribute extraction unit 102, program attribute database 103, CM attribute database 109, and search unit 10
5 is provided on the server side, the display unit 107, the user profile database 104, the viewing information control unit 120, the history information recording unit 121, the reproduction control unit 122, and the association unit 10
6 and the associating unit 123 are provided on the client side.

FIG. 17 shows a server / client system according to a third modification of the second embodiment. here,
A multimedia information database 101, a CM database 108, an attribute extraction unit 102, a program attribute database 103, and a CM attribute database 109 are provided on the server side, and a display unit 107, a user profile database 104, a viewing information control unit 120, a history information recording unit 12
1. The reproduction control unit 122, the association unit 106, the association unit 123, and the search unit 105 are provided on the client side.

FIG. 18 shows a processing flow of the associating unit 123. When the viewing information control unit 120 is supplied with the viewing information of the viewer from the reproduction control unit 122, the viewing information control unit 120 holds the viewing start time and the viewing end time. In this case, even when the channel changes, it is determined that the end and the start have occurred. When the viewing is finished, it is detected (step S41), and the viewing start time and the corresponding end time are acquired (step S42). The information may be sent as it is to the multimedia information database 101 or the history information recording unit 121 and recorded (step S43). At that time,
The recording data is, as shown in FIG.
D and the start time and end time. As another method, the associating unit 1 determines which scene of the video corresponds to the viewing.
In step S44, the data is processed into the ID of the video, the ID of the scene, and the information of the viewing frequency, and then the multimedia information database 101 or the history information recording unit 121, as shown in FIG. To
It may be recorded. The counting method is not limited to this. For example, the frequency may be calculated for each genre or keyword regardless of the video ID and the program ID. The frequency may be calculated once when viewed once, or weighted according to time for viewing.

As a result, the viewing situation can be grasped on a frame-by-frame basis, and the tastes and interests of the user can be accurately known.

Therefore, information in the user profile database 104 may be updated based on the history information recorded in the history information recording unit 121. FIG. 20 shows the flow of this processing. First, the history information recording unit 121 extracts a scene with a high viewing frequency from the history information of a certain user (step S50), and extracts an attribute (keyword) corresponding to the scene (step S51). It is determined whether there is a corresponding keyword (item) in the profile data of the user in the user profile database 104 (step S52). If there is, the priority of the keyword in the user profile data is raised (step S53). If not, the item is added to the user profile data (step S5).
4). As described above, the keywords are arranged in descending order of priority. For this reason, raising the priority has an advantage that, for example, when there are a large number of keywords and all of the search results cannot be displayed when the search results are displayed on the screen, the display is performed in the order of the priorities of the keywords, thereby facilitating the user's convenience.

As described above, according to the second embodiment, at the same time as viewing a program, which location of which program was viewed and how many times is recorded as history information. Since the user profile is rewritten according to the history information, a user profile accurately reflecting the user's preferences and interests can be obtained, and information of interest to the user can be selectively provided to the user. . History information can be collected on a per-scene basis of a program, not on a program-by-program basis.
It is possible to analyze in detail the relationship between the user's preference and the scene and content of the program. Also, by automatically uploading the history information from the user side to the program provider side, it is possible to automatically perform cumbersome tallying.

The present invention is not limited to the above-described embodiment, but can be implemented with various modifications.

[0105]

As described above, according to the present invention,
Without recording or playing back a whole program,
It becomes possible to record or reproduce only the video of the part that the user who views the program really needs. In addition, if the partial images (images in the unit of section) are associated with each other and reconstructed, a display that is easy to understand can be obtained. In addition, since commercial messages are also selectively provided according to the user's preference, if only a part of the video is selected for recording or viewing, the commercial message that the program provider wants to view is cut. The conventional drawback of being done can also be solved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of a first embodiment of an information providing apparatus according to the present invention.

FIG. 2 is a diagram showing a data configuration of a user profile.

FIG. 3 is a flowchart showing the operation of an attribute extraction unit.

FIG. 4 is a diagram showing an example of extracted program attributes.

FIG. 5 is a flowchart illustrating an operation of a search unit.

FIG. 6 is a diagram showing an example of a search result.

FIG. 7 is a flowchart illustrating the operation of an association unit.

FIG. 8 is a diagram illustrating an example of a display screen created by an association unit.

FIG. 9 is a diagram showing an example of a CM display.

FIG. 10 is a diagram showing another example of CM display.

FIG. 11 is a first modified example of the first embodiment applied to a server / client system.

FIG. 12 is a second modified example of the first embodiment applied to a server / client system.

FIG. 13 is a third modified example of the first embodiment applied to a server / client system.

FIG. 14 is a block diagram showing a basic configuration of a second embodiment of the information providing apparatus according to the present invention.

FIG. 15 is a first modified example of the second embodiment applied to a server / client system.

FIG. 16 is a second modified example of the second embodiment applied to a server / client system.

FIG. 17 is a third modified example of the second embodiment applied to a server / client system.

FIG. 18 is a flowchart showing the operation of the viewing history recording unit.

FIG. 19 is a diagram showing an example of a viewing history.

FIG. 20 is a flowchart showing a user profile update operation.

[Explanation of symbols]

 101 multimedia information database 102 attribute extraction unit 103 program attribute database 104 user profile database 105 search unit 106 association unit 107 display unit 108 CM database 109 CM attribute database 110, 111 keyboard

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06F 15/40 310F 15/403 340A (72) Inventor Kazuo Sumita 1 Komukai Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture Address Toshiba R & D Center (72) Inventor Hideki Hirakawa 1 Tokoba Toshiba-cho, Komukai Toshiba-cho, Kawasaki-shi, Kanagawa

Claims (21)

[Claims] 1. A multimedia database is provided with an attribute for each information division unit, and a database is created. By matching a user profile with an attribute, partial information of the multimedia information matching a user's preference is searched from a database. And an information providing method for providing the searched partial information to the user. 2. The information providing method according to claim 1, wherein the commercial information is stored in a database together with its attributes, and the commercial information according to the user profile is provided when the partial information of the multimedia information is provided. 3. The multimedia information is analyzed using a first database storing multimedia information, and at least one analysis technique of moving image analysis, sound / speech analysis, and text analysis, and according to the analysis result. The multimedia information is classified, and a second database storing attributes required for each classification or an attribute input from the outside, and a second database are searched according to a user profile, and a predetermined attribute is searched. A retrieval unit that extracts and selects partial information of the multimedia information according to the extracted attribute among the multimedia information in the first database. 4. An apparatus according to claim 1, further comprising an associating unit for obtaining a representative image of the partial information of the multimedia information selected by said search unit, and forming a display image including said representative image and said extracted attributes. The information providing device according to claim 3, wherein: 5. The information providing apparatus according to claim 3, wherein the user profile includes information on user's preference. 6. The method according to claim 6, wherein the attribute input from the outside is the second attribute.
4. The information providing apparatus according to claim 3, further comprising a keyboard for inputting the information to the database. 7. A third method for storing the user profile.
4. The information providing apparatus according to claim 3, further comprising a database. 8. A system further comprising: a fourth database for storing commercial message information; and a fifth database for storing attributes of the commercial message information, wherein the search unit is configured to execute the fifth database in accordance with the user profile.
4. The database of claim 3, wherein the attribute of the commercial message information is extracted, and the commercial message information corresponding to the extracted attribute is selected from the commercial message information in the fourth database. Information providing device as described. 9. The information of the user profile includes information representing any one of a creator, a title, a performer, and a genre of the multimedia information.
Information providing device as described. 10. The information providing apparatus according to claim 5, wherein the search unit extracts information of the user profile and an attribute that matches a thesaurus word of the information. 11. The information providing apparatus according to claim 3, further comprising a history recording unit that records a viewing history of the user. 12. The information providing apparatus according to claim 11, wherein the viewing history indicates which attribute information the user has watched from what time to when. 13. The system according to claim 1, further comprising a third database storing the user profile, further comprising a rewriting unit for rewriting a user profile in the third database according to the viewing history. 11. The information providing device according to item 11. 14. The information providing apparatus according to claim 3, further comprising a display unit that displays the partial information selected by said search unit. 15. The information providing apparatus according to claim 14, wherein the display unit displays the commercial message read by the search unit as a banner advertisement separately from the partial information of the multimedia information. 16. The information providing apparatus according to claim 14, wherein the display unit inserts the commercial message read by the search unit into a part of the partial information of the multimedia information and displays the information. 17. An information providing apparatus having a database for storing multimedia information as an attribute that is at least one of a result of analysis of moving image analysis, sound / speech analysis, and text analysis for each segment, or an attribute input from the outside. In an information receiving device connected to a server, the database is searched according to a user profile, a predetermined attribute is extracted, and multimedia information according to the extracted attribute among multimedia information in the database is extracted. An information receiving device comprising a search unit for selecting partial information. 18. A database for storing multimedia information as an attribute of at least one of a moving image analysis, an audio / speech analysis, and a text analysis, or an attribute input from the outside, for each division unit, and a user profile. A search unit for searching a database according to the extracted attribute, extracting a predetermined attribute, and selecting partial information of the multimedia information according to the extracted attribute among the multimedia information in the database. An information receiving apparatus connected to the information processing apparatus, comprising: an associating unit configured to obtain a representative image of partial information of multimedia information selected by the search unit, and to form a display image including the representative image and the extracted attribute. Receiver. 19. An information description method for describing a user profile used for searching for partial information included in multimedia information, wherein the user profile includes at least one information group regarding personal information of a user, The information description method, wherein the information group includes a plurality of pieces of information classified into at least one item, and the information is described according to a priority of a user in the classified item. 20. The information description method according to claim 19, wherein the information group relates to a personal profile of a user. 21. The information description method according to claim 19, wherein the information group relates to user's preference.