Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/14—Systems for two-way working
  • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
  • H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
  • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
  • H04N23/635—Region indicators; Field of view indicators
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
  • H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
  • H04N23/682—Vibration or motion blur correction
  • H04N23/684—Vibration or motion blur correction performed by controlling the image sensor readout, e.g. by controlling the integration time
  • H04N23/6842—Vibration or motion blur correction performed by controlling the image sensor readout, e.g. by controlling the integration time by controlling the scanning position, e.g. windowing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/14—Systems for two-way working
  • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
  • H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
  • H04N2007/145—Handheld terminals

Definitions

• the present invention relates to high data rate mobile phones capable of capturing and viewing images for video telephony communications. These phones are also called videophone terminals.
• the invention relates to the adjustment of the image capture means.
• UMTS Universal Mobile Telecommunications System
• Videotelephony allows two people to communicate remotely while seeing each other.
• the terminal of each person has a display screen and a camera.
• the portable terminals have a small screen with a low resolution, and it is important to have a good framing closeup if you want to see the features of his interlocutor.
• the interlocutor B having no control of the framing, it is the interlocutor A who must take care to control the shooting of his own camera.
• This control of shooting can be realized using a control sticker in a corner of its screen.
• Each interlocutor must then ensure that his image remains in the center of the control sticker.
• Such a framing system is not very practical for many reasons. It reduces the useful area of the display screen of the terminal which is already small.
• Each interlocutor must pay close attention to his own framing.
• the framing movements are not natural movements because the vignette shows a filmed image with a reversal between right and left.
• EP-A-1 304 853 discloses a portable apparatus such as a mobile phone provided with a camera and motion sensors. The camera is used to take multiple images of an object, and these images are then combined using synchronized motion information provided by the motion sensors to realign the images to be combined. This concept does not involve two remote videophone terminals.
• Each portable terminal is equipped with a camera that can have a resolution greater than the resolution of the transmitted image.
• the framing of the transmitted image is done using framing information from a remote interlocutor.
• Motion sensors are placed in each portable terminal to retrieve motion information from said terminal. The motion information is then transformed into framing commands to be sent to the other terminal.
• the invention proposes a portable videophone terminal comprising communication means, motion sensors and shaping means.
• the communication means make it possible to communicate with another videophone terminal via a radio communication network.
• the motion sensors produce information representative of movements of the terminal.
• the formatting means make it possible to transform the movement information into outgoing framing data for the other terminal.
• the means of communication are arranged to insert the outgoing framing data in data to be transmitted on the radio network to the other terminal.
• the shaping means include filtering and control developing means for comparing the motion information with a minimum motion threshold and a maximum motion threshold.
• the outgoing framing data is produced in response to detecting a movement between the minimum movement threshold and the maximum movement threshold.
• the invention proposes a portable videophone terminal comprising communication means, a camera, a framing means and a video encoding means.
• the communication means make it possible to communicate with another videophone terminal via a radio communication network.
• the camera captures images of a first size.
• the framing means extracts a portion of an image from an image captured by the camera.
• the framing means selects the image portion based on remote framing information from the other terminal. Said image portion having a second size smaller than the first size.
• the video encoding means transforms a stream of image portions from the frame means into outgoing video data.
• the communication means are arranged to extract the remote registration information from data received from the radio network from the other terminal.
• an intuitive movement of the terminal according to the first aspect allows a user A to crop the image filmed by the terminal according to the second aspect of its correspondent B.
• the two aspects are implemented on the same terminal.
• the invention relates to a method of reframing an image taken by a camera of a first portable videophone terminal with the aid of a second portable videophone terminal equipped with a screen and video sensors. movement during a videophone call.
• the method comprises a step of generating framing data, in response to a movement of the second portable terminal, from - A - motion information from the motion sensors, and a step of sending said registration data to the first portable terminal.
• the first portable terminal In response to framing information received by the first terminal and corresponding to the framing data sent by the second terminal, the first portable terminal extracts image portions corresponding to said framing information from images captured by its camera and produced a video sequence representative of a succession of image portions.
• the invention relates to a signal carrying a stream of videophone frames between a first portable terminal and a second portable terminal.
• At least one frame sent by the first terminal includes audio data, video data, and framing data.
• the registration data indicates the position and / or the displacement of an image portion taken by a camera of the second terminal. Said image portion corresponds to an image to be sent from the second terminal to the first terminal.
• FIG. 1 is a block diagram showing two portable terminals in communication according to FIG.
• FIG. 2 shows a transfer characteristic between a movement information and a framing control
• FIG. 4 shows an operating flow diagram for the terminal performing a capture.
• FIG. image according to the invention.
• FIG. 1 represents two portable terminals 100 and 200 communicating via a radiotelephone or radiocommunication network 300.
• the radiotelephone network 300 is a third generation radiocommunication network, for example in accordance with the UMTS standard.
• Third-generation radiotelephone network requires comprising a high-speed radio network for exchanging audio, video or other data between a radio terminal and the network.
• the present description is concerned with the management of the framing during a videophone communication between two terminals. Only the means implemented for the framing are detailed. The other constituent elements of the terminals and the network are well known to those skilled in the art.
• the two terminals 100 and 200 are identical. Similar references 1xx and 2xx are used to describe similar elements, the number of hundreds to differentiate the terminal. Thus, what is described with reference to the elements of the terminal 100 is applicable to the terminal 200 and vice versa.
• the first terminal 100 of the mobile phone type, comprises:
• a camera 105 connected to the registration means 103 for extracting an image area from an image captured by the camera 103, the framing means 105 selecting the image portion in accordance with the framing information c from another remote terminal,
• audio and video encoding means 106 connected to the microphone 101 and the framing means 105 for transforming the sound captured by the microphone 101 into outgoing audio data and a stream of image portions coming from the framing means 105 into video data the outgoing video data being for example a video sequence compressed according to an image compression algorithm,
• an audio and video decoding means 107 connected to the loudspeaker 102 and to the display screen 104 for transforming incoming audio data into a driving signal of the loudspeaker 102, and incoming video data into an image signal at reproduce on the display screen 104, motion sensors 108 to 110 for producing movement information of the terminal,
• a filtering means 111 connected to the motion sensors 108 to 110 for filtering the motion information
• a control generating means 112 connected to the filtering means
• a multiplexing and communication means 113 connected to the encoding means 106, the control generation means 112, the decoding decoder 107 and the framing means 105 for, on the one hand, grouping audio, video and framing data Dc to be emitted in data packets to the network, and for, on the other hand, receiving and separating received data packets into audio data, video data and frame information l c , and - an antenna 114 connected to the multiplexing means and communication 113 to exchange with the network 300 radio signals representative of the data packets transmitted and received by the terminal 100.
• a portable terminal mainly comprises a central processor, a signal processing processor and possibly an image processing processor. These three processors are used in a microprogrammed way to process all data and signals in digital form.
• the means 105-107 and 111-112 functionally described in FIG. 1 can be realized in practice by programming these processors.
• Analog / digital and digital / analog converters provide the link between the processors and the various elements 101-104 and 108-110 to which they are connected.
• the multiplexing and communication means 113 is also realized using the processors of the terminal but it also includes a radio interface connected to the antenna 114.
• the audio and video decoding means 107 receives the audio and video data from the terminal 200 and transforms them into control signals of the loudspeaker 102 and the screen 104 in order to restore to a user A of the terminal 100 the sound captured by the microphone 202 and the image captured by the camera 203 of the terminal 200 of its correspondent B.
• the screen 104 is for example of LCD type (of the English: Liquid Crystal Display) or OLED type (English: Organic Light-Emitting Diode Display) small, for example a diagonal less than two inches.
• the resolution of this screen 104 is for example less than 200 pixels per side.
• the user A of the terminal 100 can thus have a satisfactory image enabling him to capture the expressions of a close-up face. However, such a resolution does not make it possible to distinguish these same details in a broader plane.
• the camera 103 of the terminal 100 comprises a wide-angle lens and an image sensor, for example of the CCD type (Charge Coupled Device).
• the camera 103 is used for video telephony but also for other applications including digital photography.
• the photos taken by the portable terminal 100 can be sent through the network 300 to a computer.
• the camera 103 generally has a resolution greater than the resolution of the screen 104.
• the resolution of the camera 103 is at least 640 ⁇ 480 pixels in order to have a minimum quality of visualization on a screen. computer screen.
• the framing means 105 serves to extract, from each image taken by the camera 103, an image portion whose shape and resolution correspond to the screen 204 of the terminal 200. form and resolution are transmitted when initializing the communication.
• the framing means also comprises means for performing oversampling and / or subsampling of the image making it possible to perform an electronic zoom function according to a known technique.
• the electronic zoom makes it possible to transform a portion of a chosen image of any size in order to adapt it to the size of the screen 204 of the terminal 200 of the correspondent B.
• the framing means 105 includes an image stabilizer capable of compensating for low amplitude displacements of the image which correspond to to possible tremors of a user.
• the image stabilizer of a known type, is for example capable of detecting any global and uniform movement between two images coming from the camera and of carrying out a corresponding translation in number of pixels in the direction opposite to the displacement. Stabilization of the image is preferably done before the extraction of the image portion.
• the framing means 105 further moves the image portion as a function of framing information Ic from the multiplexing means 113.
• the framing is performed remotely by the correspondent who is best placed to adjust the image displayed.
• the invention provides an intuitive control of remote framing.
• the most intuitive gesture is to move his screen 104, held by hand, to follow the movement of the user of the terminal 200
• a detail is particularly eye-catching and you want to see it more closely, a user naturally brings his screen closer.
• the proposed intuitive control is to move the terminal in a natural direction to move the image portion seen on the screen.
• FIG. 2a shows an image 400 captured by the camera 203 of the terminal 200 and an image portion 401 transmitted and seen on the screen 104 of the terminal 100.
• the user of the terminal 100 wishes to move the registration of the image portion 401 to obtain an image portion 402 differently.
• the user moves his terminal
• FIG. 2b shows an image 400 captured by the camera 203 of the terminal 200 and an image portion 403 transmitted and seen on the screen 104 of the terminal 100.
• the user of the terminal 100 wishes to have a wider plan view corresponding to the image portion 404.
• the user then moves his terminal 100 eh away from the screen 104 of a movement away ⁇ z which causes a widening of the frame.
• Subsampling is then performed to adapt the image portion to the size of the screen 104, which corresponds to a zoom out.
• the registration data D 0 are produced by the control generation means 112 as a function of motion information from the motion sensors 108 to 110 after filtering in the filtering means 111.
• the motion sensors 108 to 110 consist, for example, of accelerometers, gyroscopes and / or electronic magnetometers capable of providing information relating to displacement and acceleration in translation and in rotation along three perpendicular axes.
• two of the three axes are respectively parallel to the sides of the screen 104 and the third axis is perpendicular to the screen 104.
• the translation and rotation movements are combined with each other to obtain motion information ⁇ x, ⁇ y and ⁇ z representative of a relative displacement of the screen 104 for a predetermined duration along one of the three axes.
• the predetermined duration corresponds, for example, to a sampling time of the movement information ⁇ x, ⁇ y and ⁇ z.
• the filtering means 111 then performs a filtering of the motion information ⁇ x, ⁇ y and ⁇ z.
• the motion information ⁇ x, ⁇ y and ⁇ z are representative of an amplitude and a speed of displacement. To avoid taking into account movements related to tremors (low amplitude movements) or to fatigue (slow movements) of the user, only higher movement information, in absolute value, should be taken into account. minimum threshold S m .
• the filter has the function of transfer following for the motion information ⁇ x, ⁇ y and ⁇ z:
• Au / Au if S 111 ⁇
• the command generation means 112 transforms the filtered movement information ⁇ xf, ⁇ yf and ⁇ zf into the frame data Dc.
• the framing data Dc can be of different shapes. For example, two forms of data are described below.
• a first form of the registration data is a form corresponding to registration orders.
• the orders will consist of three data representative of the modification of the framing.
• the filtered information ⁇ xf and ⁇ yf are quantized to transform the motion into a number (positive or negative) of displacement pixels of the frame defining the image portion.
• the filtered information ⁇ zf is quantized to indicate the number of pixels increasing or decreasing the frame defining the image portion.
• the position and the dimension of the frame of the image portion are then managed by the framing means 205 of the other terminal 200.
• FIG. 3 An example of a transfer characteristic between a movement information and a corresponding motion data is shown in FIG. 3.
• the abscissa axis corresponds to the possible values of a motion information, for example ⁇ x.
• Two quantization zones 413 and 414 correspond to the movement control of the frame defining the image portion. As for example, it is possible to match a value of 1 displacement pixel when the motion information corresponds to the minimum threshold S m and a value of 20 moving pixels when the motion information corresponds to the maximum threshold S M. In a simple embodiment, a linear extrapolation is performed between the two thresholds S m and SM; but note that a different curve could be used.
• a second form of the registration data may consist of a position and frame size of a selected image portion.
• the modifications of the frame are carried out by the control generating means 112.
• the framing means 205 only performs the selection of the image portion optionally accompanied by a zoom effect.
• the multiplexing and communication means 113 constitute frames of data to be sent combining a set of data destined for the terminal 200.
• the frame comprises audio data and video data relating to the videophone communication but also the frame data Dc.
• the frame is then packaged with service data to identify the frame and its destination.
• the data packet thus formed is transformed into a burst which is inserted in a radio signal consisting of a stream of frames to the network 300.
• the network 300 then needle the different frames to send them to the terminal 200 in the form of a packet. a radio signal.
• the framing means 105 has an image stabilizer which compensates for the movements of the camera when they are of low amplitude. This compensation makes it possible to avoid that the movements made to crop the image of the interlocutor in turn causes a need for mutual reframing. If the maximum threshold SM corresponds to a displacement that can be compensated by the image stabilizer, the movement made to modify the registration of the other terminal 200 is automatically compensated. If the image stabilizer does not have the capacity to compensate for the movement related to a cropping displacement, it is possible to connect the scaling means 105 to the filtering means 111 so that the actual movement of the terminal 100 is taken into account to move its framing window.
• the cropping performed in the terminal 100 is of the same nature but different amplitude and sign compared to the reframing of the other terminal 200. With the means described above, it is possible to crop the images filmed by the camera 103 of the terminal 100 from the terminal 200 and vice versa. The reframing is done intuitively in response to a movement of the terminal 200 which leads to the development of framing data through the motion sensors 208 to 210, the filtering means 211 and the control developing means 212. Thus the video sequence generated by the terminal 100 is controlled by the user B of the terminal 200 during a videophone communication.
• the image filmed is reframed using the framing means 105 of the terminal 100 if the terminal 100 is almost immobile. Detection of the quasi-immobility of the terminal 100 can be done using the image stabilizer which detects a homogeneous motion of low amplitude of the image. It is also possible to detect the movement of the terminal 100 using the motion sensors 108 to 110. Thus, if the movements are, for example below the maximum threshold SM I can be considered that the terminal is quasi-immobile.
• the framing means 105 comprises shape recognition means able to recognize the shape of a face. The form recognition is done for example with the help of a known technique for identifying that a face is present in the image.
• the terminal 100 can establish that the user is in dialogue if a face is detected.
• the framing means then takes into account the framing information if a face is detected. This allows remote framing to be controlled even if the background is moving or the terminal is moving.
• the two improvements are combined.
• the flow chart of FIG. 4 illustrates the implementation of these two improvements in the framing means 105.
• the framing means 105 receives the characteristics of the screen 204 of the 200 correspondent terminal to be able to determine the format of the image portion to be sent.
• the framing means 105 operates in free image (step 421). During this step 421, the framing means divides an image portion, for example centered, independently of any received framing information Ic.
• a test 422 is performed to determine whether conditions make it possible to switch to remote control mode of the image.
• the test 422 consists, for example, in verifying whether the image can be considered as immobile or quasi-immobile or whether a face is present in the filmed image. If one of the two conditions is achieved, proceed to step 423. If neither condition is performed, a test 424 is performed.
• the test 424 checks if the videophone communication is finished. If the communication is not completed, it returns to step 421 and the framing means operates in free image.
• Step 423 corresponds to the operation of the framing means 105 taking into account the framing information Ic.
• the image portion is moved, enlarged or narrowed according to the framing data received.
• the framing means performs, if necessary, an oversampling or a subsampling of the framed image portion to match the resolution of the screen.
• a test 425 is performed to determine whether conditions allow you to switch to remote control mode of the image.
• the test 425 consists for example in checking whether the image can be considered as immobile or quasi-immobile or if a face is present in the filmed image. If one of the two conditions is achieved, return to step 423. If neither condition is performed, a test 426 is performed.
• the test 426 checks whether the video telephony communication is complete. If the communication is not completed, the image portion is reinitialized and returns to step 421 and the framing means operates as a free image.
• the described invention can be broken down into different variants.
• the terminals 100 and 200 are identical and both capable of transmitting framing information and of receiving and taking into account framing information.
• a terminal could develop and output framing data without receiving framing information.
• a terminal can take into account framing information without itself elaborating framing data.
• the terminals implementing the invention can therefore be limited to the means necessary for the implementation of the invention without reproducing all the means included in the examples described.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Telephonic Communication Services (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
• Mobile Radio Communication Systems (AREA)
• Telephone Function (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=34953229

Family Applications (1)

Country Status (4)

Families Citing this family (17)

Family Cites Families (10)

• 2005
  • 2005-01-07 FR FR0500180A patent/FR2880762A1/fr active Pending
  • 2005-12-20 US US11/794,695 patent/US8264522B2/en not_active Expired - Fee Related
  • 2005-12-20 EP EP05850555A patent/EP1834475A2/de not_active Withdrawn
  • 2005-12-20 WO PCT/FR2005/003206 patent/WO2006075063A1/fr not_active Ceased

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events