JPH0472986A - Television telephone set image pickup method - Google Patents

Abstract

PURPOSE:To met eyes of the user with that of a talking opposite party under the condition of excellent observation of the opposite party by applying rotation processing to a picture data corresponding to a data subject to 3-dimension expansion processing based on a desired parameter and applying correction processing to a conversion picture data in existence at a dead angle at image pickup and not picked up among picture data subject to rotation processing based on the conversion picture data properly picked up. CONSTITUTION:At first a camera installed in the vicinity of a monitor screen is used to pick up the user and a video signal obtained by pickup is converted into a picture data DG and a luminance data DL by a data conversion means 40. Then an arithmetic processing means 41 is used to apply 3-dimension expansion processing 41a to the said luminance data DL and the picture data DG corresponding to the data subject to the 3-dimension expansion processing 41a is subject to rotation processing 41b based on a desired parameter and a picture data existing in a dead angle at image pickup and not picked up among the picture data subject to rotation processing 41b is subject to correction processing 41c based on the picture data picked up properly. A transformation picture data DGT is sent after the correction processing 41c.

Description

[Detailed Description of the Invention] Summary This invention relates to a videophone imaging method that allows you to make a call while viewing a moving image of the other party, and allows you to make a call without increasing the size of the entire telephone and under conditions where the other party can be clearly seen on the screen. The purpose is to provide a videophone imaging method that allows you to make eye contact with the other party, and uses a videophone equipped with a monitor screen and camera.
In a videophone imaging method that allows you to talk while viewing a moving image of the other party, the user is imaged by a camera installed near the monitor screen, and the imaged video signal obtained by the camera is converted into data. The image data is converted into image data and brightness data by the means, the brightness data is subjected to three-dimensional expansion processing by the arithmetic processing means, and image data corresponding to the data subjected to the three-dimensional expansion processing is converted based on desired parameters. Among the image data subjected to this rotation processing, the image data that was not captured due to a blind spot at the time of imaging is corrected based on the properly captured image data, and after the correction processing, the converted image is Send data.

INDUSTRIAL APPLICATION FIELD OF THE INVENTION The present invention relates to a video telephone imaging method that allows a user to make a call while viewing a moving image of the other party.

In recent years, ISD N (Integrated 5er
With the construction of the VICES Digital Network, multiplexed and high-speed data communication becomes possible, and video and
Videophones that can transmit and receive audio simultaneously are being developed. In this videophone, the key point is to be able to have a natural conversation in which the interlocutor maintains eye contact, so it is necessary to maintain eye contact with the other party displayed on the monitor screen.

2. Description of the Related Art Conventionally, in videophones, a camera 2 is installed at a position close to a monitor screen 1, as shown in FIG. I was taking pictures. Further, as another configuration example, as shown in FIG. 11, there is also one in which the camera 2 is attached to the upper part of the monitor screen 1. By bringing the camera 2 close to the monitor screen 1 to take an image as described above, it is possible to reduce the deviation of the line of sight.

Further, in FIG. 12, a half mirror 4 is installed in front of the monitor screen 1 of the monitor 1a, and an image is taken from the position of the screen 1. If the image is captured from the position of the screen 1 in this way, it is possible to align the line of sight with the other person displayed on the screen 1.

Problems to be Solved by the Invention Incidentally, in the examples shown in FIGS. 10 and 11 described above, although the deviation of the line of sight is reduced, the lines of sight cannot be aligned. In other words, there is a problem in not being able to make eye contact with the other person. Therefore, the dialogue with the other party displayed on the screen becomes uncomfortable because the line of sight is shifted.

In the example shown in FIG. 12, although it is possible to make eye contact, the use of the half mirror 4 increases the overall size, and the other party has to look at the screen 1 through the half mirror 4. There were problems in that it became difficult to see and the amount of light to the camera 2 was insufficient.

The present invention has been made in view of these points,
To provide a video telephone imaging method that allows the user to make eye contact with the other party without increasing the size of the entire telephone and under conditions where the other party can be seen clearly on the screen.

Means to solve problems FIG. 1 is a diagram showing the principle of the present invention.

According to the present invention, in a videophone imaging method that allows a videophone that is equipped with a monitor screen and a camera to make a call while viewing a moving image of the other party, firstly, a camera installed near the monitor screen allows the user to is imaged, and the video signal obtained by this imaging is converted into image data DG and luminance data DL by the data conversion means 40.

Then, the arithmetic processing means 41 calculates the brightness data DL.
A three-dimensional expansion process 41a is performed, and this three-dimensional expansion process 41a
Rotation processing 41b is performed on the image data DG corresponding to the data subjected to the rotation processing 41b based on desired parameters.
Among the image data subjected to step 1b, the image data that was not imaged due to a blind spot at the time of image capturing is subjected to correction processing 41C based on the properly imaged image data.

After this correction processing 4iC, the converted image data DGT is transmitted.

Effect: According to the above-mentioned means, even if a user who makes a call while looking at a monitor screen is photographed from an oblique angle by a camera, the oblique image is converted into a frontal image through arithmetic processing, and the other party's It can be sent to a videophone.

Therefore, the front image of the other party can be displayed on the monitor screen of the video telephone set of the other party, and the front image of the other party is also displayed on the monitor screen of this side, so that a natural conversation can be carried out with eye contact.

Embodiment Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 2 is a flowchart for explaining a videophone imaging method according to an embodiment of the present invention.

Furthermore, the external configuration of the video telephone of this embodiment is the same as that of the conventional example shown in FIG.

First, the misalignment between the line of sight of a user viewing the screen and the optical axis of the camera that images the user will be explained with reference to FIG. 6.

The numbers shown in FIG. 6 are the distance between the user 5 and the screen 1 of the videophone 6, 8 is the line of sight of the user 5, 9 is the optical axis of the camera 2, and θ is the deviation between the line of sight 8 and the optical axis 9. The angle Ll is the horizontal distance between the center of the monitor screen 1 and the camera 2. - Since the user 5 calls while fishing on a boat from a distance of about 13 cm from the videophone 6, the distance is set to 60 cm in this example. Further, even when the distance is set constant in this way, the deviation angle θ changes depending on the horizontal distance L1. This deviation angle θ and horizontal distance L1
Assuming that the distance is 60 cm, as shown in FIG.
is approximately 20 degrees, which is a linear relationship. Also, the horizontal distance L
1 depends on the size of the monitor screen 1. For example, as shown in FIG.
can be installed at a position with a horizontal distance of about Ll=lQcm, so the deviation angle θ at this time is approximately 10 degrees.

In other words, if this deviation angle θ is corrected to "0 degrees", the line of sight 8 of the user 5 and the optical axis 9 of the camera 2 can be aligned, that is, the line of sight with the other person on the screen 1 can be aligned. I can do it. In order to make the line of sight 8 and the optical axis 9 coincide in this way, it is sufficient to make the image 11 projected by the camera 2 coincide with the image 12 of the actual user 5, as shown in FIG. This can be done by rotating the image 11 by the shift angle θ through image processing and making it match the image 12. In addition, as mentioned above, the rotation angle θ may be about 10 degrees for the mainstream screens of current videophones, but since there are cases where the screen is larger than this, for example, take a margin of 10 degrees.
Assuming a screen of about 0 inches, the horizontal distance L1 is 20 cm.
Therefore, the deviation angle θ is about 20 degrees. In other words, if image processing can be performed to rotate the image 11 captured by the camera 2 by about 20 degrees at most, the image will be captured from the front, allowing the user to make eye contact with the other party.

Next, a video telephone imaging method using such image processing will be explained with reference to FIGS. 2 to 5.

In step S1 shown in the flowchart of FIG.
Import image data. As shown in FIG.
Alog/Digital) converter 20 converts the digital signal DS into a digital signal DS, and outputs it to the luminance information extraction section 21. The brightness information extraction section 21 sequentially extracts image (still image) data DC and brightness data DL from the digital signal DS, and outputs them to the filter processing section 22. Filter processing section 22
are image data DG and brightness data D consisting of a large amount of information.
L is compressed and formed into image data DGI and brightness data DL1 in units of one frame. This shaped image data DG1 is DMA (Ildirect Me
video RAM (Ra
ndom AccessMemory) 25 and stored therein. Similarly, the brightness data DLI output from the filter processing unit 22 is stored in the attribute RAM 26.
is memorized.

Since the attribute view RAM 26 has attributes on the video RAM 25, the individual information of the brightness data DLL stored in the attribute view RAM 26 is stored in the video RAM 2.
This corresponds to each piece of information of the image data DG1 stored in 5.

Next, in step S2 shown in FIG. 2, three-dimensional expansion is performed using the contrast ratio (luminance vector). This involves loading the brightness data DL1, that is, the brightness vector, stored in the attribute RAM 26 shown in FIG. expansion)
It is to do.

Next, in step S3 shown in FIG. 2, rotation processing of the image data DGI is performed. This means that the image data DGI stored in the video RAM 25 shown in FIG. 4 is rotated by the shift angle θ by the calculation by the calculation processing section 27 based on the data expanded three-dimensionally in step S2. Further, the data subjected to the rotation process is stored in the converted data video RAM 28 as converted data DT.

However, as described above, the deviation angle θ is determined in advance from the horizontal distance L1 between the center of the monitor screen l and the camera 2, and is determined as a parameter for calculating the rotation process.

Next, in step S4 shown in FIG. 2, the conversion data DT is corrected. For example, as shown in FIG. 6, if the camera 2 takes an image of the user 5, the entire right side of the user 5 will be imaged, but there will be a blind spot on the left side that will not be imaged. The data of the blind spot is calculated by the calculation processing unit 27 shown in FIG.
This is to predict and correct the data on the right side of the conversion data DT. The converted data corrected in this manner is referred to as converted image data DTG (see FIG. 5).

Next, in step S5 shown in FIG. 2, the converted image data DTG is transmitted. This is as shown in Figure 5.
First, the converted image data DTG stored in the converted data video RAM 28 is sent to the data frame configuration section 29 via the bus 24.

The converted image data DTG is converted by the data frame configuration unit 29 into a signal that can be synchronized with the video phone of the other party.
A P/S (Parallel/5erial) converter 30 converts the signal into a serial signal, and a multiplexer 31 inserts an audio data DB into the serial signal.
It is output to the line connection section 32. Then, the line connection section 32
The message is sent to the other party's video phone via the 15DN.

As explained above, according to the videophone imaging method of the present invention, a video signal obtained by imaging a videophone user with a camera is converted into image data and brightness data, and then the brightness data is expanded into three dimensions. The image data corresponding to this three-dimensional expanded data is rotated by a preset shift angle, and among the image data that has been subjected to this rotation processing, image data that was not imaged due to a blind spot at the time of imaging is properly imaged. Correction is performed based on the converted image data, and after this correction, the converted image data is transmitted to the other party's video phone.

Therefore, even if a user who makes a call while looking at a monitor screen captures an image from an oblique angle with a camera, the oblique image can be converted to a frontal image and sent to the other party's video phone, making it possible to The front image can be projected onto the monitor screen of the video phone on the side. Furthermore, since this method does not use the half mirror 4 as in the conventional example shown in FIG. It is no longer difficult to see, and furthermore, the use of the half mirror 4 does not increase the overall size.

DETAILED DESCRIPTION OF THE INVENTION As described in detail, the present invention has the effect of making it possible to maintain line of sight with the other party on the phone without increasing the size of the entire telephone and under conditions where the other party can be clearly seen on the screen.

Therefore, it is possible to have a natural conversation with eye contact.

[Brief explanation of drawings]

FIG. 1 is a diagram of the principle of the present invention, FIG. 2 is a flowchart for explaining a video telephone imaging method according to an embodiment of the present invention, and FIG. 3 is a diagram for explaining the image data acquisition step shown in FIG. 2. Figure 4 is a three-dimensional expansion using the luminance vector shown in Figure 2.
A diagram for explaining each step of image data rotation processing and conversion data correction processing, FIG. 5 is a diagram for explaining the converted image data transmission step shown in FIG. 2, and FIG. A diagram to explain the positional relationship with the telephone and the deviation angle. Figure 7 is a diagram showing the relationship between the horizontal distance between the center of the monitor screen and the optical axis of the camera and the deviation angle. Figure 8 is a 5-inch Figure 9 shows the horizontal distance on the monitor screen. Figure 9 shows the image of the user taken by the camera and the front image of the user facing the monitor screen. Figure 10 shows the camera installed next to the monitor screen. Figure 11 is a schematic front view showing the exterior configuration of a type of videophone with a camera installed above the monitor screen; Figure 12 is a front view of the monitor screen. 1 is a schematic internal configuration diagram of a video telephone of the type in which a half mirror is installed. 40... Data conversion means, 41... Arithmetic processing means, 41a... Three-dimensional expansion processing, 41b... Rotation processing, 41C... Correction processing, AS... Video signal, DG... Image data, DL...luminance data, DGT...converted image data.

Claims (1)

[Scope of Claims] In a videophone imaging method that allows a user to make a call while viewing a moving image of the other party using a videophone equipped with a monitor screen and a camera, the camera installed near the monitor screen allows the user to The video signal (AS) obtained by the imaging is converted into image data (DG) and luminance data (DL) by the data conversion means (40), and the luminance data is converted by the arithmetic processing means (41). (DL)
is subjected to three-dimensional expansion processing (41a), and this three-dimensional expansion processing (
Image data (DG
) is subjected to rotation processing (41b) based on desired parameters, and among the image data subjected to this rotation processing (41b), the converted image data that was not captured due to a blind spot at the time of image capture is transformed into a properly captured image. A video telephone imaging method characterized by: performing a correction process (41c) based on image data, and transmitting converted image data (DGT) after the correction process (41c).