Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/23—Reproducing arrangements
  • H04N1/2307—Circuits or arrangements for the control thereof, e.g. using a programmed control device, according to a measured quantity
  • H04N1/2338—Circuits or arrangements for the control thereof, e.g. using a programmed control device, according to a measured quantity according to user specified instructions, e.g. user selection of reproduction mode
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/00912—Arrangements for controlling a still picture apparatus or components thereof not otherwise provided for
  • H04N1/00957—Compiling jobs, e.g. for batch processing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N1/32101—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N1/32128—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N1/32358—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter
  • H04N1/32363—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter at the transmitter or at the receiver
  • H04N1/32368—Functions of a still picture terminal memory associated with transmission
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3225—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3261—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3278—Transmission

Definitions

• the present invention is in the field of imaging.
• the present invention relates more particularly to an automatic layout process for a composite multimedia message, intended to be displayed on a terminal screen, and intended to be printed using a printer capable of communicating with the terminal.
• the displayed composite multimedia message can comprise entities of image, text and sound (audio). These entities are usually formed from the contents of several initial multimedia messages.
• the automatic layout is performed from an analysis of the various initial multimedia messages. Such an analysis especially includes characteristics of time, contents, or even context of the images and texts forming the initial messages.
• European Patent Application EP-1 ,085,464 describes means, and particularly a method to include additional elements of text or additional figurative elements in a digital image capable of being displayed on a terminal, in order to produce a resulting image including these elements.
• the disclosed means enable the entire multimedia message comprising the resulting image, enriched with the additional elements, to be easy and fun to read and to be interpreted once displayed on a screen. For example, the covering of certain zones or parts of the image with text, should not remove certain entities of the image contents, and thus compromise the overall interpretation or understanding of the displayed message.
• the method described in European Patent Application EP- 1,085,464 enables analysis of the digital image, to recognize and identify an "optimal" zone in said digital image.
• This optimal zone also called candidate region, represents the part of the image where an additional element (text, figurative element) is placed.
• the candidate regions of the image intended to receive these additional elements are regions where the variations of color or gray levels are low, either regions not containing entities or main subjects of interest forming the contents of said image.
• the method also enables the additional element's placing or color to be modified, so as to optimize the overall presentation of the resulting image for an observer.
• European Patent Application EP- 1,117,230 describes a method and a system to present, on a terminal display screen, information coming from multimedia components, such as still or animated (video) images, text, or sound (audio).
• a presentation model (SMIL: synchronized multimedia integration language) is prograrnmed to display a multimedia message according to a configuration preset by compilation files, and corresponding to a programmed format, so as to synchronize the presentation of the various components forming a page of the multimedia message between a terminal transmitting the message and a terminal receiving said message.
• This programmed format defines zones where an image and a text are displayed respectively, the group forming the multimedia message. If the receiving terminal does not have characteristics suited to present certain components of the message, these are blanked.
• the means of the prior art do not enable a display of several multimedia messages to be obtained, for example on a single page.
• the process according to the present invention comprises the following steps: a) manually display, from the terminal keyboard, a group of at least one initial multimedia message on the terminal screen; b) manually select, from fhe terminal keyboard, at least one initial multimedia message from the group of at least one displayed initial multimedia message; c) manually validate the selection made in step b); d) automatically analyze and record sequential data of the at least one selected initial multimedia message; e) automatically analyze and record semantic data of the at least one selected initial multimedia message; f) automatically analyze and record relational data between the at least one selected initial multimedia message; g) automatically determine, according to recorded data (sequential, semantic and relational), at least one transformed multimedia message; h) automatically layout on at least one page having a first format, a composite multimedia message formed from the at least one transformed multimedia message; i)
• the layout is performed on one or more pages, fhe number of said pages being less than or equal to the number of initial multimedia messages.
• This layout is performed especially from an automatic analysis of characteristics specific to the initial multimedia messages, each comprising an image, text or sound entity. This analysis then enables an optimized layout to be determined on one page, or several pages, of a composite multimedia message.
• the composite multimedia message is determined according to the key characteristics of the contents of each of fhe initial multimedia messages. This enables not only a display with a single view or a reduced number of views to be obtained, but also the printing cost to be reduced, if printing the resulting composite message is selected. A single page or even just a few pages are produced and contain all the important image and text characteristics of each initial multimedia message.
• the composite multimedia message is formed from a selection, made by a user from the terminal keyboard, of at least one initial multimedia message, and then from an automatic analysis of the contents of the initial multimedia message.
• the number of pages forming the composite multimedia message is less than or equal to fhe number of initial multimedia messages.
• the present invention also enables, after the automatic display of the composite multimedia message, the display of the proposed composite multimedia message to be invalidated manually from the terminal keyboard.
• the invention method re-enables the automatic layout of the composite multimedia message on at least one page having a second format, and then enables the automatic display of this composite multimedia message on the terminal screen. And so on, iteratively, so long as fhe composite multimedia message displayed in a given format by the invention method does not suit the user.
• the process of the present invention also enables a printing request of the composite multimedia message, validated to a printer that comrnunicates with the terminal, to be transmitted from fhe terminal keyboard.
• Figure 2 represents a diagram of the operations executed by the invention process.
• Figure 3 represents an example of the display of initial multimedia messages on a terminal screen.
• Figure 4 represents an example of the display, according to the invention process, of a composite multimedia message on a terminal screen.
• the present invention relates to a process enabling, from one terminal 12, the automatic layout, on a single page, or even several pages, of one, or more usually, several multimedia messages.
• Advantageously fhe number of pages is between one and five.
• the multimedia message is for example, advantageously, an MMS message (Multimedia Message Service) comprising a digital image, the text associated with said image and a sound or audio message (music or melody).
• MMS message Multimedia Message Service
• These multimedia messages can thus be exchanged among terrninal users functioning in a network.
• Each of said users being able for example to modify the passing message by adding text to it for example.
• FIG. 1 represents diagrammatically a preferred embodiment for implementing fhe invention.
• a terminal 12 communicates by a link 17, 18 with a server 14.
• the server 14 communicates, by a link 20, with an image database 15.
• the image database 15 stores or saves for example previously recorded high- resolution digital images.
• the server 14 cornmunicates, by a link 19, with a printer 16.
• the printer 16 is for example a printer using specific paper (support) enabling it to produce high quality images; this type of printer, for example inkjet, thermal, laser or photographic, is commonly used in digital photographic processing laboratories (minilabs).
• Other image supports, used in online printing services, can be used: T-shirts, identity photographs, etc.
• the terminal 12 is preferably a mobile terrninal: for example a cellphone equipped with a keyboard 19A and a display screen 18 A, a cellphone equipped with a camera module (phone cam) enabling digital shots to be recorded, or a digital camera equipped with a GSM/GPRS or 3G type cell modem (wireless cam) enabling digital photographs to be downloaded from a remote online album.
• a gateway 13 enables communications between this terminal 12, the server 14 and the image database 15 to be atm inistered.
• the gateway 13, for example of WAP type enables communication, represented by the link 17, between the tenriinal 12 and fhe network, for example GSM.
• the present invention can be implemented in an Internet environment. That is, in a preferred embodiment using a mobile terminal 12 for example of "phone cam" type, this terminal 12 is capable of communicating with an Internet type network.
• the message transits by the gateway 13, where it is transformed into a message according to the HTTP protocol (Hypertext Transfer Protocol), used in the Internet.
• HTTP protocol Hypertext Transfer Protocol
• the shots are taken directly from the terminal 12, and saved or stored in a memory of said terminal 12.
• the images have for example a VGA type resolution (Video Graphics Array) of 640 by 480 pixels.
• a user of fhe terminal 12 manually displays, from the terminal keyboard 12, one, or usually several, initial multimedia messages.
• An initial multimedia message comprises a digital image, and either at least one text message and/or one sound message.
• This sound message comprises audio data, for example known music or a melody.
• the digital image contained in the multimedia message can be still or animated (video clip).
• the audio data are saved in sound files, for example on the server 14. According to the size and capacity of the display screen of the terminal 12, one or more messages are displayed on one page of said screen.
• the method of displaying the initial multimedia messages can be reduced to the display of successive lines, giving simply: “message 1", “message 2", etc.
• the corresponding multimedia message By clicking on “message 1", the corresponding multimedia message then appears on the screen in the form for example of an image (image zone 25, 26, 27, 28) under which one or more texts appear (text zone 29, 30, 31, 32).
• the text zone 29, 30, 31, 32 can also appear above or beside the image zone.
• the text zone 29, 30, 31, 32 explicitly gives one or more texts inherent to the image; this is case when the texts are relatively short, that is are only composed of a few words or expressions.
• the text zone 29, 30, 31, 32 gives lines of the type “text 1", “text 2", etc., and the user explicitly makes the corresponding text appear by clicking "text 1 ", "text 2", etc.
• initial multimedia messages 21, 22, 23, 24 appear, one under the other, as a page 20A displayed on the screen 18 A.
• the page of screen 20 has for example four messages.
• Each of these initial multimedia messages 21, 22, 23, 24 comprises an image zone 25, 26, 27, 28, a text zone 29, 30, 31, 32, and a sound zone 32A, 33; the sound zone 32A, 33 indicates that a sound message is associated with said initial multimedia message 22, 24.
• Each of the images 25, 26, 27, 28 is displayed for example with low resolution enabling the contents of said image to be viewed.
• the text zone can show, as in the first embodiment presented above, either a shortened text (e.g. "text 1 ", "text 2", etc.), or the explicit display of the text associated with the image of fhe initial multimedia message.
• an icon or distinguishing symbol 32 A, 33 is displayed; this distinguishing symbol is for example a musical note. If the user wishes to select more than four messages, they use the keyboard 19A to scroll the other initial multimedia messages.
• the user manually selects, using the keyboard 19A of terminal 12, one or more of these displayed messages from the display of these initial multimedia messages 21, 22, 23, 24.
• the selection is made using the keyboard 19A of the te ⁇ ninal 12, for example by pointing to the selected messages 21, 23, 24, by clicking on a special zone 34, 35, 36 specific to each of said messages.
• the selected messages 21, 23, 24 make up for example a story, said messages having been transmitted successively in time by different transmitters.
• they validate the selection 1 thus made, by using for example a menu specific to the invention method, and which is displayed on the screen 18A of the terminal 12.
• the invention process then enables the automatic analysis of the initial multimedia messages selected for layout.
• the resulting data of this analysis are automatically saved.
• the analysis is performed by the algorithm used to implement the invention process.
• This algorithm comprises a series of instructions represented diagrammatically in Figure 2.
• the application or program specific to the invention process is installed on the terminal 12; in this embodiment for example a terminal of the type NOKIATM 7650 with code developed in Java or C++ is used.
• the program can also be installed on the server 14.
• a first step of the analysis consists in automatically performing a sequential analysis 2 of each selected and validated initial multimedia message.
• This sequential analysis 2 enables the characterization, in time of the data corresponding to the moment when the initial multimedia message is created, the moment when the message is sent, the moment when the message is received, the identification of the transmitter of the message, fhe identification of fhe receiver of said message.
• the analyzed sequential data are saved either to a memory of the terminal 12, or to a memory of the server 14.
• a second step of fhe analysis consists in automatically performing a semantic analysis 4 of the initial multimedia message.
• This semantic analysis is an analysis of the entire contents of the initial multimedia message: Image, text, sound or voice inherent to the message contents.
• the algorithm of the invention method enables for example only part of an initially rather long text to be kept for later layout, while keeping the meaning of the text, using keywords of said text.
• the analyzed semantic data are saved either to a memory of the terminal 12, or to a memory of the server 14.
• the step of automatic analysis and recording of semantic data is performed before the step of automatic analysis and recording of sequential data.
• a third step of the analysis consists in automatically performing a relational analysis 3 between the selected initial multimedia messages.
• the algorithm of fhe invention process enables links or correspondences to be established between images, texts and sounds fo ⁇ riing each of the various selected initial multimedia messages 21, 23, 24 respectively.
• the relational analysis is performed using all the previously saved sequential and semantic data.
• This relational analysis enables relational data to be established that correspond to links or correspondences that exist between fhe respective contents of each of the selected initial multimedia messages, and are linked to the respective meaning of each of said messages; this enables the automatic deterrnination, by using the sequential and semantic data, of the relational data that weigh the existing relationship level between each of the selected initial messages.
• the relational data are saved either to a memory of the terminal 12, or to a memory of the server 14.
• the algorithm of the invention process uses all the saved sequential, semantic and relational data to automatically determine at least one transformed multimedia message 38, 39, 40, 41, 42, 43.
• the algorithm of the invention process enables resizing, zooming, only keeping one region or zone of interest (e.g. a face) of the respective images of each of said initial messages, and/or only keeping one part of the text or text parts forming said initial messages.
• the multimedia message comprises an image that is a video clip
• the invention process enables the determination for example of a key image of said video clip; this key image is then processed by the invention process to be included in a composite multimedia message.
• the invention process thus enables the deterrnination of a multimedia message that is a transformation of the initial multimedia message, because only the key parts of the initial messages are kept.
• This transformation 6 has a predictive objective, aiming especially to optimize later layout.
• the transformed multimedia message 38, 39, 40, 41, 42, 43 can be identical, in overall contents (image, text and sound), to the selected initial multimedia message 21 , 23, 24.
• the number of transformed multimedia messages is equal to the number of initial multimedia messages. All said transformed messages constitute the composite multimedia message.
• the invention process then automatically lays out 7, on at least one page 37 having a given format, the composite multimedia message, formed from each transformed multimedia message.
• Figure 4 illustrates for example a layout of six selected multimedia messages 38, 39, 40, 41, 42, 43. Then the composite multimedia message is displayed automatically 8 on the terrninal screen 12, according to a format given by the invention process.
• Figure 4 illustrates an example in which the invention process lays out six multimedia messages on a single page 37. This means that either only six multimedia messages were previously selected and validated, and then laid out on this page 37, or a larger number of messages, for example ten, twelve or eighteen were selected and validated.
• the invention process, according to the automatic analysis of these messages, and programmed formats, lays out and displays for example six messages on a first page 37, and then four messages on a second page 37. If twelve messages are selected and validated, the invention process, according to the automatic analysis of these messages, and programmed formats, lays out and displays for example six messages on a first page 37, and then six messages on a second page 37. If eighteen messages are selected and validated, the invention process, according to the automatic analysis of these messages, and programmed formats, lays out and displays for example six messages on a first page 37, six messages on a second page 37, and six messages on a third page 37.
• the display of each of said pages 37 on the screen 18, can be performed manually, by using the keyboard 19A of fhe terminal 12.
• the display of several pages 37 is performed automatically according to a set display time, for example ten or fifteen seconds display per page.
• the display time is advantageously set for each page, or the display time is set globally for a group of pages, each of said pages being displayed automatically for a time virtually equal to a sub-multiple of the global set time.
• the display time can be modified from the keyboard 19A.
• the format characterized by its width 44 and its height 45, is selected automatically by the invention method, according to initially programmed format data.
• the size ratio characterizes for example the ratio between the height and width of the format.
• the composite multimedia message is displayed as an image (image format), that is as a bitmap file (pixels).
• image format image format
• the displayed image represents for example one or more people's faces, and the texts that correspond to verbal exchanges among said people are displayed in specific zones 46, 47, 48, 49; these zones 46, 47, 48, 49 are represented for example, in the image, as balloons. If the displayed composite multimedia message 37 suits the user of the terminal 12 having requested the layout, said user manually validates 10 the proposed display, from the keyboard 19 of the terminal 12.
• the invention process also enables the user, from for example a menu displayed on the screen of fhe terminal 12, to perform a printing request 11 of the validated composite multimedia message.
• a printer 16 produces the composite multimedia message, for example on a sheet of photographic paper, in a format whose size ratio is identical to the size of the automatic layout.
• the invention process thus enables a number of printing sheets less than or equal to the number of initial multimedia messages to be produced, all said messages telling for example a story. Savings of printing paper are thus advantageously made.
• the invention process also enables the transformed multimedia message to determined by using analysis rules 5 that depend on a context linked to the multimedia message.
• analysis rules 5 are included in the algorithm, and enable instructions to be executed that add to the semantic analysis of the message, and that influence the presentation of the transformed multimedia message, and thus the layout, according to the nature of said message: professional context (property, insurance, police, etc.), holidays, family, night-environment, etc.
• the analysis rules are worked out according to the most likely events occurring in each context. These most likely events enable for example the characterization of the image entities to be kept in order to determine the transformed multimedia message, or for example the keywords to be kept in order to form the texts of the transformed multimedia message.
• the object of the algorithm of the invention process is to optimize the layout of the group of selected initial multimedia messages, in order to form a composite multimedia message, according to the available programmed formats.
• the invention process enables another layout 9 to be performed automatically, by using a second format, different than the first format corresponding to the invalidated layout.
• the user has the choice of validating 10 this second layout, or continuing to invalidate it.
• the invention process enables a third layout to be performed automatically, according to a third format different than the first and second formats. Invalidation can be so repeated, iteratively, so long as different programmed formats are available that have not been used to perform an automatic layout.
• the number of programmed formats in the invention process is "n", "n” being an integer
• fhe invention process enables the layouts automatically determined by the invention process to be invalidated "n” times.
• the invention process proposes the first invalidated format to the user again, then the second, etc. In this case, the user only has the option of validating the format, from the "n", that suits them best.
• the number of programmed formats is between one and ten. However, there is no upper limit of "n" to implement the invention process.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• General Engineering & Computer Science (AREA)
• Information Transfer Between Computers (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=30471003

Family Applications (1)

Country Status (9)

Cited By (1)

Families Citing this family (16)

Citations (6)

Family Cites Families (10)

• 2002
  • 2002-08-08 FR FR0210072A patent/FR2843515B1/fr not_active Expired - Fee Related
• 2003
  • 2003-07-25 JP JP2004530056A patent/JP4227103B2/ja not_active Expired - Fee Related
  • 2003-07-25 EP EP03792231A patent/EP1535461B1/de not_active Expired - Lifetime
  • 2003-07-25 US US10/523,658 patent/US20060052118A1/en not_active Abandoned
  • 2003-07-25 AU AU2003266249A patent/AU2003266249A1/en not_active Abandoned
  • 2003-07-25 DE DE60332636T patent/DE60332636D1/de not_active Expired - Lifetime
  • 2003-07-25 WO PCT/EP2003/008241 patent/WO2004019603A1/en not_active Ceased
  • 2003-07-25 CN CNB038189240A patent/CN100380910C/zh not_active Expired - Fee Related
  • 2003-07-25 KR KR1020057002222A patent/KR100978969B1/ko not_active Expired - Fee Related

Patent Citations (6)

Also Published As

Similar Documents

Legal Events