JPH06339056A - Video camera driving device - Google Patents

Abstract

PURPOSE:To provide a video camera driving device which is capable of automatically tracking an object without requiring a thermally-sensitive sensor such as an infrared ray camera, etc. CONSTITUTION:When a part of video (for instance, a part of the face of a speaker) displayed on a speaker display monitor 5 is desigated as a noted area, the motion vector of this area is calculated as an area specific vector and area specific vector information S14 showing this vector is supplied to a control angle arithmetic part 15. Thus, the area specific vector information S14 is converted into angle vector information S15 showing the angles in the horizontal direction and vertical direction of a camera fixing base. Next, a camera fixing base driving part 3 changes the angles in the horizontal direction and vertical direction of the camera fixing base in accordance with each angle vector information S15. As a result, the angle of a video camera 4 is controlled by following up the motion of the speaker and the video of the speaker is displayed at the center of the monitor 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to, for example, in a video conference system, a video that drives a video camera in accordance with the movement of a speaker so that the speaker is always displayed at a predetermined position on a monitor. The present invention relates to a camera drive device.

[0002]

2. Description of the Related Art In recent years, various types of so-called video conference systems have been developed in which a speaker is photographed by a video camera and the image is transmitted to the other party of the conference via a communication network. In this video conference system, it is desired that the speaker is always displayed at a predetermined position on the monitor.

In order to meet this demand, a device for driving the video camera in accordance with the movement of the speaker is required. As the video camera driving device, a device described in Japanese Patent Application Laid-Open No. 4-103285 has been conventionally developed. This device integrates an infrared camera for shooting the same object and a video camera, and controls the horizontal and vertical angles of both cameras based on the heat of the speaker captured by the infrared camera. In detail, it has the following functions.

First, the heat generated by the speaker is captured by an infrared camera, the level of the heat is converted into a square-shaped character having a predetermined number of blocks, and this is compared with a standard character. This standard character is set assuming that the video of the speaker is in the center of the television monitor, for example.
Then, the angle of the infrared camera is adjusted so that the character based on the output signal of the infrared camera approaches the standard character.

[0005]

According to such a structure, the angle of the video camera integrated with the infrared camera is also corrected in accordance with the movement of the speaker so that the image of the speaker is always displayed. Displayed in the center of the TV monitor.

However, in such a configuration, since the movement of the object has to be detected by using the infrared camera, there is a problem that the device becomes large-scale and expensive.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a video camera driving device which does not require an infrared camera and can be made compact and inexpensive.

[0008]

In order to solve the above problems, the present invention provides an area designating means for designating a predetermined area of a subject displayed on a monitor and a movement of the area designated by the area designating means. A motion vector calculating means for calculating a vector, and a camera driving means for driving the video camera so that the subject is displayed at a predetermined position on the monitor based on the motion vector calculated by the motion vector calculating means are provided. It was done like this.

[0009]

In the above structure, first, the area designating means designates a predetermined area of the subject displayed on the monitor. When this designation is made, the motion vector of the designated area is sequentially calculated by the motion vector calculation means. Then, the video camera is driven based on the calculated output. As a result, the angle of the video camera is corrected according to the movement of the subject, and the subject is always displayed at a predetermined position on the monitor, for example, in the center.

[0010]

【Example】

A: Configuration of Embodiments Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention. It should be noted that this embodiment is an embodiment in which the present invention is applied to a video conference system.

In the figure, a camera movement request switch 1
Is placed at a predetermined position on the table of each attendee of the conference, and when operated by the attendee, the camera move request information S1 indicating the move request of the video camera 4 is sent to the camera move request control angle calculation unit. Send to 2.

When the camera movement request control angle calculation unit 2 receives the camera movement request signal S1, the camera movement request angle vector information indicating the angle of the video camera 4 for photographing the vicinity of the camera movement request switch 1. S2 is output to the camera fixed base drive unit 3.

The camera fixing base driving section 3 drives a camera fixing base (not shown), and the video camera 4 is attached to the camera fixing base. The camera fixing base drive unit 3 drives the camera fixing base and controls the angle of the video camera 4 based on the camera movement request angle vector information S2 or angle vector information S15 described later. The video camera 4 photographs the subject and outputs the image signal to the speaker display monitor 5 and the encoding unit 16.

The encoding unit 16 performs various processes on the image signal supplied from the video camera 4, and the network NT
It is to be transferred to the speaker on the other side of the conference via the, and has the constituent elements surrounded by a broken line in the figure. First, A
The D converter 6 converts the image signal supplied from the video camera 4 into a digital signal, and supplies the resulting image data to the current frame buffer 7. The current frame buffer 7 stores image data for one frame at present (hereinafter referred to as current screen data S7), and supplies the stored current screen data S7 to the motion vector calculation unit 9 and the DCT / quantization unit 10. To do. The motion vector calculation unit 9 calculates M × M based on the previous screen reproduction data S8 output by the previous frame buffer 8 storing the previous reproduction image data and the current screen data S7 output by the current frame buffer 7. The motion vector information S9 of the macroblock composed of pixels is calculated. The DCT / quantization unit 10
Predetermined processing (details will be described later) based on the motion vector information S9, the current screen data S7, and the previous screen reproduction data S8.
And outputs the output signal S10 to the entropy coding unit 1
1 and the inverse quantization / IDCT unit 12 are supplied. The inverse quantization / IDCT unit 12 performs a process reverse to that of the DCT / quantization unit 10, and the result of the process is reproduced image data for one screen before. The image data output by the inverse quantization / IDT unit 12 is stored in the previous frame buffer 8.
The entropy coding unit 11 performs Huffman coding for information compression using the statistical property of the signal, and the output signal is output as coded image data S11.

The attention area designating section 13 designates the area to be noticed by the speaker displayed on the speaker display monitor 5 in accordance with the operation of the operator, and designates the attention area indicating the designated attention area. The information S13 is output to the area-specific motion vector calculation unit 14. The area peculiar motion vector calculation unit 14 calculates the attention area specifying information S13 and the motion vector information S
Area-specific motion vector information S14 is created on the basis of No. 9 and supplied to the control angle calculation unit 15.

Next, the decoding section 17 demodulates the encoded data supplied from the other party of the conference via the network NT, and outputs the demodulated data obtained as a result. The demodulated data is converted into an analog signal by the DA converter 18 and supplied to the partner image display monitor 19. The partner image display monitor 19 displays the image of the partner of the conference based on the demodulated analog signal supplied from the DA converter 18.

B: Operation of the Embodiment Next, the operation of this embodiment having the above-mentioned configuration will be described.

First, when a participant presses the camera movement request switch 1 provided on the table, the camera movement request information S1 is transmitted to the camera movement request control angle calculation unit 2. The camera movement request control angle calculation unit 2 stores the location assigned to each camera movement request switch 1 in advance, and in order to move the shooting location of the video camera to that location, the horizontal angle and vertical angle of the camera fixing base are stored. The camera movement request angle vector information S2 including the angle information is output. The camera mount base drive unit 3 changes the horizontal angle and the vertical angle of the camera mount base according to the camera movement request angle vector information S2. As a result, the video camera 4 captures an image of an object to be captured, for example, a speaker who has pressed the camera movement request switch 1. The image captured by the video camera 4 is displayed on the spot by the speaker display monitor 5 and, after being compressed by the encoding unit 16, transferred to the other party via the network NT. At the other end, the transferred encoded data S11 is decoded and reproduced.

Next, with respect to the image displayed on the speaker display monitor 5, the attention area designating section 13 designates the attention area. This designation is performed by a pointing device such as a mouse provided in the attention area designation unit 13, and designates a part of the face of the speaker, for example, as shown in FIG. In the figure, R is a region of interest, and P is
This is a pointer for designating the attention area R.

The attention area designation information S13 output by the attention area designation unit 13 is created as information uniquely indicating the attention area R. For example, when the attention area R is designated by a rectangle, the information is the horizontal and vertical coordinates of one vertex and the lengths of the sides in the horizontal and vertical directions.

(Operation of Encoding Unit 16) Next, the encoding unit 1
The processing contents of No. 6 will be described. The encoding processing method of the encoding unit 16 in this embodiment is CCITT Recommendation H.264. It is the same as the encoding method defined in H.261.

First, the image data output from the video camera 4 is converted into a digital signal in the AD converter 6, and then stored in the current frame buffer 7. The motion vector calculation unit 9 uses the current screen data S7 and the previous screen reproduction data S to perform motion compensation prediction, which is an image compression technique.
8 is used to detect a motion vector for each macroblock consisting of M × M pixels (M is a positive integer).

The detection of this motion vector is performed as follows. First, as shown in FIG. 3, for each macroblock MB1 of the current frame screen, a search area SA surrounding the macroblock MB2 at the same position as the macroblock MB1 in the previous frame screen is set. Next, from this search area SA, the same M × M as the macro block
A macro block MB3 having a pixel size is cut out.
Next, the data of this block MB3 and the macroblock M
The similarity of the data of B1 is calculated. The above processing is performed for all macroblocks MB3 that can be cut out from the search area SA, and the macroblock MB3 having the highest similarity is detected. Finally, the position of the macroblock MB3 and the macroblock M having the highest degree of similarity are
Find the difference from the position of B2. This difference is macroblock M
It is the motion vector v of B1.

Here, the direction of the motion vector v is from the macro block MB2 to the maximum similar macro block MB3 in the search area SA, which is opposite to the actual motion of the subject. The component of the motion vector v is
“1” is defined to represent the distance for one pixel.

As described above, the motion vector calculation unit 9
From, the motion vector v of each macroblock MB1 of the current frame screen is output as motion vector information S9. The DCT / quantization unit 10 then, based on the motion vector information S9, each macroblock M of the current frame screen.
The M × M pixel area (macroblock MB3) of the previous frame reproduction screen corresponding to B1 is set to each macroblock MB1.
Of the image data in this area and the macroblock MB1 of the current frame screen after moving in the direction opposite to the motion vector v of
And the density difference value between and. Then, the DCT / quantization unit 1
0 is DCT (discrete cosine transform) / quantized and outputs the difference value according to the magnitude of the difference, or
DC in the macroblock MB1 of the current frame screen
T ・ Quantize and output. The discrete cosine transform is
It is a kind of orthogonal transformation, and quantization is to obtain a quotient divided by an integer called a quantization step, both of which are commonly used as a basic technique of code compression. The macro block output as the above-mentioned density difference value is called an inter macro block, and the macro block output as the current screen itself is called an intra macro block.

Next, the entropy coding unit 11 compresses the signal S10 by utilizing the statistical property and outputs the output as coded image data S11.

Further, the inverse quantization / IDCT unit 12
Processing such as inverse quantization and inverse DCT is performed on the output signal S10 of the T / quantization unit 10. In this case inte
After processing according to the difference between the r macroblock and the intra macroblock, the output is transferred to the previous frame buffer 8.

(Operation of Region Eigenvector Operation Unit 14) Next, the operation of the region eigenvector operation unit 14 will be described. C
C. H. of CITT. In the coding system defined by H.261, as described above, the calculation of the motion vector v for each macroblock MB1 is essential. Therefore, in the region-specific motion vector calculation unit 14, each macroblock MB1 calculated by the motion vector calculation unit 9 in the encoding unit 16 is calculated.
Is used to calculate the motion vector V peculiar to the attention area R.

That is, in the area eigenvector calculation unit 14, the attention area designation information S output from the attention area designation unit 13 is output.
13 and the motion vector information S9 of each macroblock MB1 output from the motion vector calculation unit 9, the area peculiar motion vector V is calculated from the following equation, and this is output as the attention area peculiar motion vector information S14.

[0031]

[Equation 1] Here, the numerator on the right side of Expression 1 is the motion vector vi of each macroblock (hereinafter, referred to as a macroblock belonging to the attention area) MB1 (i) (i = 1 to N) including pixels included in the attention area R. And its macroblock MB1 (i)
And the number of pixels included in the attention area R
Is obtained for all the macro blocks MB1 (1) to MB1 (N) belonging to the region of interest R, and the sum is obtained. The denominator on the right side of Expression 1 is included in each macroblock MB1 (i) belonging to the attention area R,
In addition, the number mi of pixels included in the attention area R is set to be the macroblocks MB1 (1) to MB1 belonging to the attention area R.
This is the sum of (N).

Therefore, the fraction on the right side of Expression 1 represents the average motion vector v of the macro blocks MB1 (1) to MB (N) belonging to the region of interest R. Also, the number 1
The negative sign on the right side inverts the direction of the motion vector, and as a result, the region-specific motion vector V obtained by Equation 1 becomes a vector indicating the actual motion of the attention region R. That is, the imaging target in the attention area R has moved by the area eigenvector V.

The area eigenvector information S14 indicating the area eigenvector V is supplied to the control angle calculator 15 and converted into angle vector information S15 indicating the horizontal angle and the vertical angle of the camera fixing base. The angle vector information S15 is obtained as follows.

First, let Φ be the rotation angle when an object shown in the center of the monitor moves by one pixel on the monitor when the camera base is horizontally swiveled. Then, if the horizontal component of the region eigenvector V is m, the horizontal angle component of the angle vector information S15 is approximated by mΦ. The vertical angle component is calculated similarly.

Next, the camera mount base drive unit 3 changes the horizontal and vertical angles of the camera mount base by the horizontal angle component and vertical angle component of the angle vector information S15, respectively. As a result, the shooting angle of the video camera 4 is corrected in accordance with the movement of the speaker, and the image reflected in the attention area R is always displayed at the center of the monitor (the partner image display monitor 19 and the speaker display monitor 5). Positioned.

When the camera move request switch 1 is newly pressed and the camera move request angle vector information S2 is output in response to this, the camera mount base drive unit 3 causes the camera move request angle vector information S2. Is prioritized, and the camera fixing base is driven accordingly. Therefore, even if the control according to the angle vector information S15 is performed, when the camera movement request switch 1 is pressed, the camera fixing base is driven so that the camera movement request switch 1 reaches the pre-stored shooting position. It After that, the angle vector information S15 is ignored until the attention area designation unit 13 sets the attention area R again, and when the attention area R is set, the control by the angle vector information S15 is performed again.

According to this embodiment described in detail above, since the motion of the object is detected by using the motion vector, compared with the structure in which the motion of the object is detected by using the infrared camera, The size and cost of the device can be reduced.

Further, since the area peculiar motion vector V of the attention area R is calculated by utilizing the motion vector v of the macroblock MB1 required at the time of image compression, the area peculiar motion vector calculation is performed. The configuration of the unit 14 can be simplified.

Further, the motion vectors vi to v of the macro blocks MB1 (1) to MB1 (N) belonging to the attention area R are included.
Since the area eigenmotion vector V is calculated by performing a predetermined averaging process on N, the area eigenmotion vector V with high accuracy can be obtained.

C: Modified Example In the above-mentioned embodiment, the average motion vector v of the attention area R is obtained, and the vector opposite to this motion vector v is set as the area peculiar motion vector V. Instead of this, , For example, the macroblock MB1 at the center of the attention area R
(I) The motion vector vi is obtained, or all macroblocks MB1 (1) to MB1 belonging to the attention area R are obtained.
Of the motion vectors v1 to vN of (N), the largest number may be obtained.

In the above-described embodiment, the angle vector information S15 is calculated for each screen, but it may be calculated in a plurality of screen cycles. For example, the sum of the angle vectors for 30 screens is used as the angle vector information S15,
You may make it output once to 30 screens.

In the above-described embodiment, the motion vector v of the macroblock MB required at the time of image compression.
Since the motion vector V of the attention area is calculated using
Although there is an advantage that the configuration of the video conference system can be shared,
When the present invention is applied to a system or the like that does not inherently have image compression, a part that performs similar processing may be created separately.

Although the angle of the video camera 4 is controlled in the above-described embodiment, control for moving the video camera 4 back and forth, left and right, and up and down may be added.

[0044]

As described above, according to the present invention, since the video camera is driven based on the motion vector of the area designated on the monitor, a special sensor such as an infrared camera is not used. In addition, the subject such as the speaker can be automatically tracked. Therefore, it is extremely suitable for use in a video conference system and the like, and the device can be made compact and inexpensive.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a schematic diagram showing setting of a region of interest in the embodiment.

FIG. 3 is a conceptual diagram showing a process of calculating a motion vector in the embodiment.

[Explanation of symbols]

1 ... Camera move request switch, 2 ... Camera move request control angle calculation unit, 3 ... Camera fixed base drive unit, 4 ... Video camera, 5 ... Speaker display monitor, 6 ... AD conversion unit, 7 ... Current frame buffer , 8 ... Previous frame buffer, 9 ... Motion vector calculation unit, 10 ... DCT / quantization unit, 11 ... Entropy coding unit, 12 ... Inverse quantization / IDCT unit, 13
... attention area designation unit, 14 ... area eigenvector operation unit, 1
5 ... Control angle calculation unit, 16 ... Encoding unit, 17 ... Decoding unit,
18 ... DA converter, 19 ... partner image display monitor.

Claims (2)

[Claims] 1. A video camera driving device for driving a video camera in accordance with the movement of a subject so that the subject is displayed at a predetermined position on the monitor, wherein a predetermined area of the subject displayed on the monitor is displayed. Based on the motion vector calculated by the motion vector calculation means, the motion vector calculation means calculates the motion vector of the area specified by the area designating means, and the subject on the monitor. A video camera drive device, comprising: a camera drive means for driving the video camera so that the video camera is displayed at a predetermined position. 2. The motion vector calculation means subdivides the screen into predetermined blocks and calculates a motion vector for each block, and the first motion vector calculation means calculates the motion vector. Of the motion vectors of a plurality of blocks existing in the area designated by the area designating means,
2. The video camera drive device according to claim 1, further comprising a second motion vector calculation means for calculating a motion vector of the designated area by performing a predetermined averaging process. .