JPH01202085A - Television telephone system - Google Patents

Abstract

PURPOSE:To allow a camera to correspond to a certain short moving distance without controlling the camera when a person is slightly moved by providing the title equipment with a gate circuit for extracting a part of a picture in accordance with a selection signal outputted from a selection circuit. CONSTITUTION:An output from an image pickup device 101 is stored in a frame memory 102 and then a small picture is selected based on a picture dividing selection signal outputted from a picture position size selecting circuit 103. The selected and gated division picture is coded by a coding circuit 106 and synthesized with a small picture selecting signal including the picture size and position information by a synthesizing circuit 107 and the synthesized result is outputted. On the other hand, the background of a picture is detected from a signal outputted from the frame memory 102 by a background detecting circuit 105 and the signal is also sent to the coding circuit 106. On the receiving side, the synthesized small picture selecting signal from the coded image signal by a separating circuit 108 and the coded image signal is encoded by an encoding circuit 109. Consequently, the camera can easily follow the movement of a person.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial Application Field) The present invention is a videophone device related to screen control of a videophone.

(Prior art) Conventionally, since the primary purpose of videophones is to display and transmit the face and body, the camera automatically tracks the camera when the face protrudes from the screen to display the face to some extent. I had to either switch to the self-portrait monitor or adjust the camera position myself.

Furthermore, in the latter case, the first step is to confirm that there is a misalignment.
As shown in Figure 1, the self-portrait monitor had to be displayed on a portion of the screen at all times, making it difficult to see the screen. Furthermore, when the screen is moved in this way, not only does the background detection circuit have to regenerate the background memory created by the background detection circuit, but also because the background must be created on a screen where a person is always present. If no movement occurs, the hidden background will not be recognized as the background. There were problems such as the complexity of the background recognition algorithm.

(Problems to be Solved by the Invention) As described above, conventional video phones have had the following problems because the self-portrait monitor has been controlled manually or by an automatic tracking device of the camera.

O Manual method 0 It is difficult to see the screen ■ The amount of information decreases / Automatic tracking method ■ The device becomes complicated. (Increases size) - Common to both methods - Complication of background detection circuit - Re-creation of background by movement Therefore, the present invention solves these problems and allows the device to easily follow the movement of a person. It is an object of the present invention to provide a video telephone device that can be downsized.

[Structure of the invention]

(Means for Solving the Problems) In order to solve the above problems, the present invention has the following configuration.

In other words, it consists of an imaging device that images a wider area than the screen that is actually transmitted, a selection circuit that selects the screen size and position, and a gate circuit that extracts a part of the screen in response to a selection signal output from the selection circuit. It basically consists of a background detection circuit that detects the background over the entire screen and an interpolation filter circuit whose filter characteristics change according to the selection signal, and also determines whether a person has protruded from the entire screen and performs warning control. By adding a screen control circuit, it is possible to check the screen position without looking at the self-monitor.

(Function) By electrically selecting the screen to be selected based on one's own movements and giving a warning for larger movements, it is possible to transmit a normal image to the other party without checking the self-monitor. . Furthermore, since the screen to be transmitted is determined, the background of other screens can be easily detected in advance, and transmission efficiency can be improved.

(Example) An example of the present invention will be described in detail using the drawings. We will explain the case where the entire screen captured on the camera's image sensor is N x M (hereinafter referred to as a large screen) and the divided screen is N/3xM/2 (hereinafter referred to as a small screen). .

First, a portion of the output from the imaging device (camera) 101 is stored in the frame memory 102 for encoding processing, and then is divided into one frame based on the screen division selection signal from the screen position and size selection circuit 103. Small screen is selected. Regarding this screen division, for example, it is set to six divisions as shown in FIG. 2, but it is also possible to set the small screens so that they overlap υ. In addition, various methods can be considered for selecting the divided screen, such as manual operation or selecting the screen with the largest sum of absolute values of the inter-frame differences of each divided screen as shown in FIG.

The selected and gated split screen is then processed by the encoding circuit 10.
The signal is encoded in G, and combined with a small screen selection signal including screen size and position information by a synthesis circuit 107 and output. On the other hand, the background of the screen is detected from the signal in the frame memory 102 by a background detection circuit 105, and this signal is also sent to the encoding circuit 106.

On the receiver side, after the combined small screen selection signal and the encoded image signal are separated by a separation circuit 108, the encoded image signal is decoded by a decoding circuit 109. Further, in response to background detection on the transmitting side, only background information is stored in the background memory 110. In addition, the size information of the separated small screen selection signal is used to determine the coefficients of the interpolation filter 117 for outputting the decoded image signal to the monitor, and the position information Used for registration.

The interpolation filter 117 can be easily realized by, for example, a configuration in which, as shown in FIG. 4, a selection signal selects a filter coefficient stored in a coefficient selection circuit such as a ROM, thereby adaptively changing the shape of the filter. can.

On the other hand, the background detection circuit 105 detects the background not on the selected small screen but on the large screen (the size υ, N×N in this example). As a background detection method, since there is a high possibility that a person exists in the selected split screen, it detects the inside and outside of the person based on the difference between frames (or there is a moving object), and detects the outside of the person. The background is detected by setting a certain threshold and the number of consecutive frames below that threshold, and the background is detected by the difference in the distance between the person and the background (using ultrasound, etc.) You can use methods etc. Also, for screens that are not selected, there is a high possibility that the screen consists only of the background, so either register the screen as it is as the background, or register it as the background if the difference between frames is less than the threshold. You can create a background using this simple method. By detecting the background on the entire large screen like this, it becomes complicated to detect the background in images where a person is present, but in reality, the background effect becomes more noticeable when the person moves significantly. Considering that the background of the unselected image can be easily detected,
The effectiveness of the present invention is obvious for large movements such as when a person enters the divided image to be selected next.

Next, a second embodiment is shown in FIG.

In the first embodiment, the movement of the person was handled by switching the split screen, but in this embodiment, even larger movements can be followed by moving the TV camera. explain.

First, it is assumed that the small screen (■) in FIG. 2 is selected as shown in FIG. 6 due to the movement of a person. At this time, as shown in Figure 6, the screen of ■ is further divided into smaller screens (hereinafter referred to as tiny screens), and the circuit configuration shown in Figure 7 is shown. The camera is controlled so that it is centered on the small screen, and the controlled amount is transmitted as position information. Furthermore, if a person exists across adjacent small screens, a method of searching in a slightly larger range than the small screen shown in (2) or a method of setting the small screens so that they overlap may be considered. Regarding camera control, a method of extracting and controlling a person may be considered, but this will be described in the fourth embodiment. By controlling the camera in the manner described above, the point of interest is always on the center split screen, so by securing a large background memory, it is possible to change the position of interest on the camera screen each time a new background appears on the peripheral split screen. The image of interest can be registered based on the amount of movement of the image, and the image of interest can be efficiently transmitted to the background memory created in advance.

Next, a third embodiment will be described.

A third embodiment is shown in FIG. First, a screen position determination circuit creates positions that may be off the screen depending on the screen position selected by the selection circuit. The divided screen selected by the selection signal is converted into a digital value by a binary conversion circuit,
A contour tracking circuit tracks the outside of the contour. If the screen position created in advance or the edge of the screen is reached during tracking, return to the starting point of tracking, perform tracking in reverse, and then measure the area surrounded by the edge of the screen and the outline. If the outline goes around,
In addition to measuring the area, if it is larger than the area of the tracked outline (assuming the person is the largest and visible), it is determined that the person is protruding from the screen and the monitor is changed to a self-monitor. Alternatively, the screen is controlled by blinking the decoded image to give a warning to the sender.

Next, a fourth embodiment is shown in FIG. This shows a case where the screen to be selected is not fixed or equally divided, but moves according to the movement of the person.

First, the output from the camera is stored in a frame memory, and then a person is detected to determine the screen position. Edges of the inter-frame difference signal are detected by a method such as detecting edges that have changed by more than a certain threshold value from the inter-frame difference signal of the image read from the frame memory, and the outline of the person is extracted. The extracted contour is tracked to find the entire area, the center of gravity is detected, and the screen position and size selection circuit is used to select the small screen so that the center of gravity is at the center of the small screen at a preset size. determine the position of A part of the large screen is selected as a small screen according to a position selection signal (for example, a screen address) of the screen selected in this way, and is transmitted after being encoded. If there is a part that newly appears as a background in the small screen selected in this way, since the background has been determined in advance as in the first embodiment, a signal identifying the background is Transmit. However, in many cases, the background is meaningless (or there is no need to transmit it), so a method of transmitting only the background identification signal and not transmitting the background itself may be considered.

The image signal encoded by the above method is decoded by an operation opposite to the encoding process. In particular, for newly appearing backgrounds, the newly appearing image portion is detected using the position selection signal on the screen, and the decoder side also registers what has been transmitted as the background in the background memory. . The background memory has a capacity not only for small screens but also for large screens or more, so it can efficiently transmit data even for large movements.

In all of the above embodiments, it is also possible to use a plurality of image sensors in order to reduce distortion and power consumption due to wide angle. In this case, the size of the imaging elements other than the center is increased by reducing the number of pixels and slowing down the readout time, but the performance is improved in terms of distortion and power consumption.

〔Effect of the invention〕

As described above, according to the present invention, if the person moves to a certain extent, it can be handled by electrically moving the small screen without controlling the camera. By using , images can be transmitted efficiently. Furthermore, since subsampling for reducing the amount of transmission is performed by extracting a small screen, there is also the effect that a filter for removing folding is not required.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing a first embodiment of the present invention, and FIG. 2 is a diagram showing a first embodiment of the present invention.
3 is a diagram explaining in detail the screen position and size selection circuit of the first embodiment, FIG. 4 is a block diagram of the interpolation filter circuit, and FIG. 5 is a diagram explaining the screen position and size selection circuit of the first embodiment. 6 is a diagram showing the second embodiment, FIG. 7 is a block diagram for detecting screen position information necessary for camera control, and FIG. 8 is a block diagram for explaining the second embodiment. 3 embodiment as agent Patent attorney Nori Chika Kan Hirama Matsuyama Jyo 2nd 1st
Figure 2 Figure 33
4 Figure 5 Figure 7 Figure 8

Claims (1)

[Claims] (1) In a TV phone whose basic elements are an image capturing device, an image receiving device, a telephone set, and a code/decoding circuit, the screen size is a selection circuit that selects a position; a gate circuit that extracts a part of the screen according to a selection signal output from the selection circuit; and an encoding circuit that encodes the extracted image;
A video telephone device characterized by comprising a decoding circuit that decodes encoded images.