JPH05304631A - Image pickup device - Google Patents

Abstract

(57) [Summary] [Purpose] In an imaging device that detects the point of gaze on the photographer's screen and obtains information for controlling the device, a malfunction occurs regardless of the position of the point of gaze. The objective is to always perform stable shooting with a subject tracking operation. An EVF for displaying an optical image on an image pickup surface on an EVF screen based on an image pickup signal output from an image pickup device, and an EVF.
A gazing point detection device for detecting the gazing point of the operator on the F monitor screen, and a gazing point tracking circuit for tracking the movement of the gazing point based on the position information of the gazing point detected by the gazing point detection device, A feature tracking circuit that tracks the movement of the subject based on the temporal change of the feature of the subject without using the gazing point and a control circuit that selectively operates these tracking circuits according to the position of the gazing point AF area, AS
Set the area etc.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus which detects a gazing point of an operator and controls the photographing operation using the detected information.

[0002]

2. Description of the Related Art Conventionally, in the field of cameras, automation and multi-functionalization have been remarkable, and various functions such as an automatic focus adjustment device and automatic exposure control have come to be standardly equipped.
The operability is remarkably improved.

These functions are based on, for example, information obtained from the detection area set in the image pickup screen, for example, high frequency components in the image pickup signal corresponding to the detection area in automatic focusing (hereinafter referred to as AF). Focus detection is performed so that the main subject can be reliably kept in focus.

Further, in recent years, for the purpose of further optimizing the control, in order to reflect the intention of the operator in the setting of the detection area, the gazing point, that is, the line of sight on the operator's finder screen is detected and the operator gazes. It has been proposed to introduce a line-of-sight detection device in which a detection area for detecting information associated with the operation of the above various functions is installed at a certain position. As a result, the operator does not have a malfunction such as being out of focus depending on shooting conditions or focusing on something other than the main subject, and the operator is stable even against a subject that changes from moment to moment. You can continue to focus on your will.

Disclosed as such a viewpoint detection device is USP4075657, USP410914.
5, USP 4574314, JP-A-61-272552.
There is a bulletin, etc.

However, the gaze point of the operator is not always constant, and it is expected that a considerable variation will occur. Therefore, if the detection area is simply set to the detected gaze point, the gaze point of the operator will be reduced. There is a problem that the setting position of the detection area becomes inadequate when it shifts to something other than the main subject or when it goes out of the screen of the finder, and the reliability of the information obtained from the detection area and the control accuracy are greatly reduced.

Further, when the operator tries to input each function during photographing, the operation must be performed while removing the viewfinder, and in order to operate while confirming switches of various functions. The operator has to look away from the viewfinder once, which may cause the shooting screen to be disturbed or the subject to be lost. In recent years, such a problem is caused by a camera and a camera-integrated VT.
It becomes even larger with the trend toward multi-functionality of R.

As a countermeasure against this problem, various function menus are displayed on the finder screen of the viewfinder for the monitor, the gazing point is matched with the reported various function menus, and the gazing point detecting device detects the gazing point. A VTR with a built-in camera that performs a function corresponding to the function menu that matches the gazing point is considered, which allows the operator to easily input various functions without keeping an eye on the viewfinder. Becomes

Further, with respect to functions for supplementing photographing operations such as automatic focus control (AF), automatic exposure control (AE), automatic white balance (AWB), and automatic blur correction (AS),
Considering the position of the main subject that changes moment by moment as the position where the operator is gazing, the gaze position is accurately detected by the gaze detection device, and the detection area used for the various controls is made to follow the change of the gaze point. Therefore, it is conceivable that a moving subject is tracked and, for example, the focus is continuously adjusted.

By doing so, it is expected that more accurate various functions will be realized that do not violate the operator's will.

[0011]

However, in various control operations such as tracking AF, AE, AWB, and AS in which the detection area is moved to the gazing point by using the line-of-sight detection device, the operator does not If you are gazing, no problem will occur, but if the gazing point is located outside the field of view or in an area where it is unlikely that you can actually set the detection area, set the detection area at that position. This will result in setting based on incorrect information, and when shooting a moving subject in particular, by separating the point of interest from the subject even for a moment, There is a risk of causing a large distance, causing a decrease in control as described above, and causing a problem that causes a malfunction.

[0012]

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and includes photoelectric conversion means for converting an optical image formed on an image pickup surface into an electric signal, and the photoelectric conversion means. A monitor that displays an optical image on the image pickup surface on a monitor screen based on the output electrical signal; and a gazing point detection unit that detects the gazing point of the operator on the monitor screen.
First tracking control means for tracking the movement of the gazing point based on the position information of the gazing point detected by the gazing point detecting means, and the movement of the optical image without using the gazing point A second tracking control means and a control means for selectively operating the first tracking control mode and the second tracking control mode are provided.

Further, according to the present invention, a photoelectric conversion means for converting the optical image formed on the image pickup surface into an electric signal, and an optical image on the image pickup surface based on the electric signal output from the photoelectric conversion means. A monitor displayed on the monitor screen, a gazing point detecting means for detecting a gazing point of the operator on the monitor screen, and a gazing point based on position information of the gazing point detected by the gazing point detecting means. First tracking control means for tracking the movement, second tracking control means for tracking the movement of the optical image without using the gazing point, and the gazing point within a predetermined area on the screen of the monitor. Determination means for determining whether or not there is, and the first means based on the output of the determination means
The tracking control mode and the second tracking control mode are selectively operated.

According to the invention, the photoelectric conversion means for converting the optical image formed on the image pickup surface into an electric signal, the gazing point detecting means for detecting the gazing point of the operator, and the gazing point are excluded. Detection area setting means for setting a shake detection area for extracting information about a predetermined movement in the area on the imaging surface, and shake correction for performing shake correction based on the shake information detected from the shake detection area And means.

According to the invention, the photoelectric conversion means for converting the optical image formed on the image pickup surface into an electric signal, the monitor means for displaying the image on the image pickup surface on the monitor screen, and the monitor screen A gazing point detecting means for detecting the gazing point of the operator, a first tracking control means for setting a distance measuring frame based on position information of the gazing point detected by the gazing point detecting means, and the gazing point Second tracking control means for detecting the movement of the optical image to follow the distance measuring frame without using the camera, and blurring to a region outside the distance measuring frame set in the first or second tracking control mode. A shake detection unit that sets a detection area and a control unit that selectively operates the first tracking control mode and the second tracking control mode are provided.

[0016]

As a result, in the image pickup apparatus in which the gazing point on the screen of the photographer is detected and the information for controlling the apparatus is obtained, regardless of the position of the gazing point,
It is possible to always perform stable control without causing a malfunction.

Further, the moving object is stably and surely tracked in the detection area, and is stable even when the position of the gazing point of the operator leaves the object and is located outside the screen or outside the actually controllable area. It is possible to perform various subject tracking operations, and it is possible to capture an optimum moving image.

Further, the object and the background can be accurately discriminated, and appropriate shake correction can be performed regardless of the presence or absence of movement of the object and the shooting condition, and during the object tracking operation. Also, shake correction can be performed in parallel.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which a photographing apparatus according to the present invention is applied to a camera-integrated VTR (video tape recorder) will be described in detail below with reference to the drawings.

FIG. 1 is a camera-integrated VT to which the present invention is applied.
It is a block diagram showing an example of R.

In the figure, 101 is an electronic viewfinder (EVF), and 102 is a finder screen. ED is a line-of-sight detection device that detects the line of sight of the photographer, that is, the point of gaze on the electronic viewfinder screen, and includes a photoelectric element array 6, an infrared light emitting diode 5a (5b), an eyepiece 1, a light receiving lens 4, and a dichroic mirror 2. And the signal processing circuit 9. In addition, 201 represents the eyeball of the photographer.

The infrared light emitting diodes 5a and 5b irradiate the photographer's eyeball with infrared light, and the reflected light is received on the photoelectric element array 6, whereby the photographer's line of sight is directed to the electronic viewfinder 101. This is to detect where the user is looking on the finder screen 102, that is, the gazing point.

Here, regarding the above-mentioned line-of-sight detection device ED,
This will be described in more detail with reference to FIGS.

First, the optical system ED is composed of a dichroic mirror 2, an eyepiece lens 1 and a light receiving lens 4, and the dichroic mirror 2 transmits visible light and reflects infrared light. Electronic viewfinder 1
The image on the fan screen 102 of No. 01 is passed to the eyes of the photographer, and the reflected light from the infrared light emitting diode to the eyeball is guided to the photoelectric element 6. Then, the light from the fine screen 102 is incident on the eyeball 201 through the dichroic mirror 2 and the eyepiece lens 1.
E indicates the eye point. Here is the screen 1
The axis of light incident on the eyepoint E of the eyeball 201 from 02 is defined as the X axis (see FIG. 2).

The infrared light emitting diodes 5a and 5b are disposed symmetrically with respect to the X axis near the upper end of the eyepiece 1 on the eyeball 201 side, and infrared light is placed at the center of the eyeball 201 located near the eyepoint E. It is supposed to be incident. Eyeball 2
The infrared light from 01 passes through the eyepiece lens 1, is guided to the light receiving lens 4 by the dichroic mirror 2 which transmits visible light and reflects infrared light, and enters the photoelectric element array 6. An example of the eyeball reflection image on the surface of the photoelectric element array 6 is shown in FIG. The light is guided by the dichroic mirror 2 to the light receiving lens 4 and is parallel to the axis of the light incident on the photoelectric element array 6, and X
The axis orthogonal to the axis is the Y axis, and the axis orthogonal to the plane including the X axis and the Y axis is the Z axis (see FIG. 2). In addition, the photoelectric element array 6
Has a plurality of photoelectric elements arranged on a straight line parallel to the Z axis.

The signal processing circuit 109 is an eyeball optical axis detection circuit,
It is composed of an eyeball discrimination circuit, a visual axis correction circuit, a gazing point detection circuit, and the like. The eyeball optical axis detection circuit determines the rotation angle of the eyeball optical axis. The eye discriminating circuit is the finder screen 1
It is to determine whether the eyeball gazing at 02 is on the left or right. The visual axis correction circuit corrects the visual axis based on the rotation angle of the optical axis of the eyeball and the eyeball discrimination information. The gazing point detection circuit calculates a gazing point based on an optical constant. Each of these circuits is realized by software by configuring the signal processing circuit 109 with a microcomputer.

Light fluxes from the infrared light emitting diodes 5a and 5b are respectively formed with a cornea reflection image e and a cornea reflection image d in directions parallel to the Z axis (see FIG. 4). The Z coordinate of the midpoint of the corneal reflection image e and the corneal reflection image d coincides with the Z coordinate of the center of curvature o of the cornea 21. When the optical axis of the eyeball of the observer does not rotate about the Y axis, that is, when the optical axis of the eyeball and the X axis match (the center o of the corneal curvature and the center C ′ of the pupil are on the X axis). The corneal reflection image e (d) of () is formed with a shift in the + Y direction from the X axis (see FIG. 5).

FIG. 3 is a flow chart showing the visual axis detection procedure by the signal processing circuit 109.

The rotation angle of the eyeball optical axis is detected by the eyeball optical axis detection circuit, and the image signal from the photoelectric element array 6 is indicated by -Y in FIG.
Sequentially reading from the direction, the row Yp 'of the photoelectric element column 6 on which the corneal reflection images e', d'are formed is detected (step S
1), photoelectric element array 6 on which corneal reflection images e ', d'are formed
The generation positions Zd 'and Ze' in the column direction are detected (step S2).

An example of the output signal obtained from the row Yp 'of the photoelectric element column 6 is shown in FIG. Then, the interval between corneal reflection images | Z
The image forming magnification β of the optical system is obtained from d′−Ze ′ | (step S3). The image forming magnification β of the reflected image from the eyeball is on the photoelectric element array 6 because the interval between the corneal reflected images e and d changes in proportion to the distance between the infrared light emitting diodes 5a and 5b and the eyeball of the observer. It can be obtained by detecting the positions e ′ and d ′ of the re-imaged corneal reflection image.

Then, the boundary Z between the iris 23 and the pupil 24 on the row Yp 'of the photoelectric element row 6 on which the corneal reflection images e and d are re-formed.
2b ', Z2a' are detected (step S4), and row Yp 'is detected.
The upper pupil system | Z2b'-Z2a '| is calculated (step S5).

Normally, the row Yp 'of the photoelectric element row 6 on which the corneal reflection image is formed is, as shown in FIG. 6, from the row YO' of the photoelectric element row 6 where the pupil center C'is present in the -Y direction in FIG. Is off. Another row Yl ′ of the photoelectric element column from which the image signal is to be read out is calculated from the imaging magnification β and the pupil diameter (step S6). Row Yl 'is sufficiently distant from row Yp'.

Then, the iris 23 on the row Yl 'of the photoelectric element column
And the boundaries Zlb 'and Zla' of the pupil 24 are detected (step S7), and the boundary points (Zla ', Yl') and the boundary points (Zl '
b ′, Yl ′), the boundary point (Z2a, Yp ′), and the boundary point (Z2b ′, Yp ′), at least three points are used to determine the center position C ′ (Zc ′, Yc ′) of the pupil.

Subsequently, the position of the corneal reflection image (Zd ', Y
p ′), (Ze ′, Yp ′) and the following equations (1) and (2), the rotation angles θ _z and θ _y of the eyeball optical axis are obtained (step S
8).

[0035]

[Outer 1]

Where δY 'is the infrared light emitting diode 5a,
Since 5b is arranged in the direction orthogonal to the light receiving lens 4 in the column direction of the photoelectric element array 6, the re-imaging positions e ′ and d ′ of the corneal reflection image are located on the photoelectric element array 6 on the cornea 21. This is a correction value for correcting the deviation in the Y-axis direction with respect to the Y coordinate of the center of curvature of.

Then, the eye discriminating circuit discriminates whether the eye of the observer looking into the finder is the right eye or the left eye, for example, from the distribution of the calculated rotation angles of the eyeball optical axis (step S9). , The visual axis is corrected by the correction circuit based on the eyeball discrimination information and the rotation angle of the optical axis of the eyeball (step S1).
0), the gazing point is calculated based on the optical constants of the optical system 100 (step S11).

Next, the configuration of the entire camera portion will be described. 1
20 is a focusing lens group, and 122 is a zooming lens group. An image pickup device 118 such as a CCD converts an optical signal incident through an optical system including the focusing lens group 120 and the zooming lens group 120 into an electric signal. A lens driving circuit 131 drives the focusing lens group 120.

Reference numeral 116 denotes a camera signal processing circuit, which performs predetermined signal processing on the image pickup signal output from the image pickup element 118 and converts it into a planned video signal.
Reference numeral 114 denotes a band pass filter (BPF) for extracting a predetermined high frequency component which changes from the video signal from the camera signal processing circuit 116 according to the focus state and whose level becomes higher as it approaches the in-focus state. ..

Reference numeral 110 denotes an AF control circuit, which is an image pickup element 118.
From the portion corresponding to the distance measurement frame as the control target area set in the image pickup plane of the camera signal processing circuit 116, the BPF
The high frequency component of the image signal obtained via 114 is extracted, the lens drive circuit 131 is driven so that this high frequency component is maximized, and the focusing lens group 120 is moved and controlled. Further, the AF control circuit 110 controls the movement of the gazing point subject to set the position of the ranging frame to the gazing point based on the gazing point information detected by the signal processing circuit 109 of the viewpoint detection circuit, and the feature point of the subject. Is equipped with a feature point tracking function for automatically detecting and detecting each of the features. Each function can be selectively operated, and the gazing point is within the funda screen 102 or the screen 10.
It is determined whether or not it exists within a predetermined range set in 2, and each tracking function is automatically selected according to the determination result, and the algorithm of the setting operation of the ranging frame is changed.
The specific control operation will be described later.

Reference numeral 112 denotes a frame display circuit for displaying the distance measuring frame in the finder screen. When the gazing point is in the finder screen 102, the distance measuring frame is displayed with the gazing point at the center, When the viewpoint is not within the finder screen 102, the distance measuring frame is displayed at the position of the subject set by another subject tracking operation as described later. 1
An EVF display circuit 12 displays the video signal output from the camera signal processing circuit on the EVF and controls the display of the finder screen 102 to display various control information.

A video signal processing circuit 141 converts the video signal from the camera signal processing circuit 116 into a form suitable for recording on a recording medium such as a magnetic tape. A timing recording device 140 is a video tape recorder (VTR) for recording the video signal from the video signal processing circuit 141 on a magnetic tape.

Next, the internal structure of the AF control circuit 110 will be described with reference to the block diagram of FIG. A of the present invention
When it is determined that the operator is looking at the main subject, the F control circuit 110 performs a subject tracking operation by line-of-sight detection that sets a distance measurement frame at the gazing point, and it is determined that the operator is not looking at the main subject. In this case, an automatic subject tracking operation for tracking the subject by detecting a temporal change in the feature of the subject is performed, and the main subject can be tracked in any case. Incidentally, as an automatic tracking method for detecting the movement of the subject by detecting the temporal change of the feature of the subject and tracking the movement, for example, USP 4872058,
The method disclosed in USP 5031049 or the like can be used.

In FIG. 8, the gate circuit 401 is a BP
Only the signal corresponding to the inside of the distance measuring frame in the video signal output from F114 is passed. By varying the passing range, the size and position of the distance measuring frame can be freely set. Then, the arithmetic circuit 406 inputs the gazing point information, calculates the opening / closing timing of the gate circuit when setting the distance measuring frame at the gazing point coordinates, and supplies it to the gate circuit 401 via the switch 408 to control it. To do. Thereby, the distance measuring frame can be positioned at the gazing point coordinates.

On the other hand, the output of the BPF 114 is supplied to the peak hold circuit 403, in which the peak value of the high frequency component within one field period of the video signal is detected, and the peak position detection circuit 404 causes the peak value to be detected within the screen. The peak detection position at is detected. Here, the peak value is a portion where the high frequency component level is the highest in the main subject, that is, a portion where the contrast is extremely high, and can be represented as a feature of the subject. The detected position of the peak value can be considered to indicate the position of the main subject. Based on the position coordinates of the main subject output from the peak position detection circuit 404, the calculation circuit 405 calculates the opening / closing timing of the gate circuit when setting the distance measurement frame at the coordinates, and sends it to the gate circuit 401 via the switch 408. Supply and control this. Thereby, the distance measuring frame can be positioned at the main subject position.

The determination circuit 407 inputs the gazing point information, and determines whether the gazing point is within the window, that is, inside or outside the effective screen area (white area A) on the window shown in FIG. 10 (hatched area B). ) Is determined and the switch 408 is switched according to the determination result. The effective screen area may be the entire screen, or a predetermined range within the screen, and may be set according to the design.

When it is determined that the gazing point coordinates are within the finder screen, it is considered that the photographer is gazing at the main subject, and the arithmetic circuit 406 is selected in order to perform subject tracking based on the gazing point. If the gazing point coordinates are outside the viewfinder screen, it is determined that the photographer is not gazing at the main subject, and the arithmetic circuit 405 is selected.

FIG. 9 is a flow chart showing the operation flow of the AF control circuit 110 of FIG.

First, the gazing point information obtained from the signal processing circuit 109 for detecting the gazing point, that is, a signal indicating which position on the surface of the fiber the gazinger is gazing at is currently determined by the photographer within the fiducial coordinate values (IP _X , IP _Y). )) (Step S101).

Next, the received gazing point coordinate value (IP
_X , IP _Y ) determines whether the fiber is outside the surface 102 (step S102). The determination as to whether the fiber is in-plane or out-of-plane is made based on whether it is inside or outside a predetermined area set in the fiber finder screen as shown in FIG. 10, as described above. In FIG. 10, the shaded area is the area outside the window, and is used, for example, as the area for detecting the line of sight for selecting the line of sight input screen. Therefore, in order to determine whether the above-mentioned gazing point coordinate value is within the image display screen of the finder, in this routine (step S102), the gazing point coordinate value and the coordinate value for determining the finder screen coordinate area are set. Are being compared.

When it is determined that the gazing point of the photographer is outside the image display screen of the funder, for example, the menu display screen area in the shaded area, the conventional subject tracking without using the gazing point position information, that is, the above-mentioned Further, for example, the feature point subject tracking operation is performed based on the peak position information of the high frequency component. At this time, the tracking range is also set together with the tracking coordinates (step S103).

On the other hand, when it is determined that the gazing point of the photographer is within the finder screen, the object tracking operation in which the gazing point position information is added to the object tracking based on the peak position information is performed. For example, when the peak position and the gazing point position are separated by a certain distance or more, tracking control is performed such that the gazing point position is prioritized. At this time, the tracking range is set together with the tracking coordinates (step S104).

Step S103 or Step S104
The distance measurement frame display circuit 112 is used to update the display of the distance measurement frame from the subject tracking coordinates and the tracking range obtained in step S105 (step S105).

Next, high frequency components are detected in the updated distance measuring area, and AF control is performed (step S106).

That is, the photographer does not always look at the main subject, and the subject is moving even when the photographer is taking his or her eyes away. If you don't know where you are and look at it with AF, even if you fix the range-finding frame at a specific position such as the center of the screen,
It is conceivable that the subject is not captured reliably.

Therefore, in this embodiment, when it is determined that the operator is looking at the main subject, the subject tracking operation is performed by line-of-sight detection to set the distance measuring frame at the gazing point, and the operator does not see the main subject. If it is determined that the main subject is to be tracked in any case, the automatic subject tracking operation of tracking the subject by detecting the temporal change of the feature of the subject is performed.

Next, a second embodiment of the present invention will be described. According to the above-described first embodiment, the case has been described as an example in which the gazing point of the photographer is detected within the finder screen and the distance measuring frame is set at the gazing point coordinates.

In the second embodiment, a case will be described in which an AS frame for image stabilization is set at the coordinates of the gazing point of the photographer.

First, an anti-vibration system (AS: Auto Stabilizi
ng System), the camera shake generally causes the image to deteriorate and causes all kinds of malfunctions regardless of industrial measuring instruments or consumer crises. In particular, when shooting from a moving vehicle while walking or moving, or shooting in a nephew of vibration, a screen shake is likely to occur, and a shake correction method of a seed map has been conventionally proposed.

For example, in the image stabilization camera described in Japanese Patent Laid-Open No. 61-248681, an incident image is converted into an electric signal by an image pickup system including an optical system and a photoelectric conversion element, and a video signal subjected to a predetermined signal processing is converted into a video signal. In addition to being supplied to the monitor device, the video signal is supplied to the image blur detection circuit, and the magnitude and direction of the image blur are detected by taking the correlation between the two screens at regular intervals. And based on the detection result, by controlling the drive of the optical system in the direction to cancel the image blur,
Even if the device vibrates, a stable image can be obtained.
Note that “anti-vibration” refers to the function of correcting camera shake.

As described above, in the case of detecting the blur amount from the image, if the method of detecting the blur amount with uniform sensitivity with respect to the entire screen is adopted, whether the subject is deformed or moved or the camera shake of the photographer causes It is not possible to distinguish whether the camera is tilted or vibrated, and it is possible to recognize an object contrary to the will of the photographer, and to perform incorrect blur correction.

In order to solve this problem, it is necessary to recognize the position, shape and movement of each subject and detect the amount of image blur within a predetermined area. Such an image blur detection device is a technical report of the Television Society of Japan Vol. 11, No.
3, p34-48, PPOE, '87 -12 (May, 1987) "About image shake correction device". In this document, the whole screen is divided into 140 block regions, blur detection in each region is manually turned on / off arbitrarily, and the representative point matching method motion vector is obtained only in the ON region to perform blur detection. There is. Then, as a method of determining the area for detecting the image blur, a method of utilizing the brightness difference between the background and the subject or a difference between the images at a certain time interval (frame difference)
Has been used to detect motion regions.

However, according to the AS system described above, it is difficult to reliably distinguish the subject and the background, especially when mounted on a camera-integrated VTR or the like, and which area of the image to be shake-corrected is optimal for detecting the shake. It is difficult to determine whether or not it is, and blurring is performed in a wrong detection area, resulting in an unnatural image.

Therefore, in this embodiment, as described above, it is assumed that the gazing point of the operator is on the main subject during photographing, and the line of sight of the operator is detected to detect the gazing point on the screen. A predetermined range centered on is considered as a main subject, and an area other than the range of the main subject is determined to be the background to be a blur detection area.

By doing so, it is possible to constantly detect and correct the blur for the background, and prevent the motion of the object from being erroneously determined as camera blur and malfunctioning.

Further, in the present embodiment, when the operator's gazing point is located in a region where there is no subject outside the field, the main subject is clearly not gazed.
Safety measures such as fixing the setting of the AS area to a predetermined position, for example, a peripheral position avoiding the central portion of the screen are taken.

The second embodiment will be described below with reference to the block diagram shown in FIG. In the figure, the same components as those in the first embodiment of FIG. 1 are designated by the same reference numerals and the description thereof will be omitted. A one-field delay circuit 215 delays the image signal output from the camera signal processing circuit 116 by one field period. 214 is A / D
The converter converts the image signal digital signals output from the camera signal processing circuit and the one-field delay circuit 215 at intervals of one field period. 21
Reference numeral 2 is a memory for storing each image signal output from the A / D converter 214. 210 is an AS that controls the image stabilization operation
The control circuit compares the image signals of the two screens stored in the memory 212 and shifted by one field period with the image signals corresponding to the shake detection area (AS area) for detecting the shake, and When detecting the vibration as a motion vector and reading an image signal for one field from the memory 212 based on this motion vector information,
The blur is corrected by shifting the reading range in the direction and magnitude of canceling the motion vector.
Further, the AS control circuit 210 inputs the gazing point coordinate information supplied from the signal processing circuit 109 of the line-of-sight detection circuit ED, and the gazing point coordinates are within the finder area (see FIG. 10 in this embodiment).
(The specified area inside the finder is the entire area of the finder screen, and the area other than the finder angle of view is outside the finder area.)
If so, the AS area is set in a peripheral area (background) other than the predetermined range including the gazing point coordinates, and if it is outside the fiducial area, the AS area is fixed in the peripheral area excluding the center of the screen. ..

Reference numeral 218 denotes a D / A converter, which is a memory 21.
The image signal read out from No. 2 is converted into an analog signal and output to the video signal processing circuit 141.

Here, the process of setting the AS area in the image pickup screen by the AS control circuit will be described with reference to FIGS.

FIG. 12 is a flow chart showing the procedure of the AS area setting process. In step S201, the line-of-sight detection circuit E is used.
The gazing point coordinate information (IPx, IPy) supplied from the signal processing circuit 109 of D, that is, a signal indicating which position on the picture screen the photographer is currently gazing is input by the coordinate value within the picture screen.

In step S202, step S201
The process of determining the area in which the main subject exists is performed from the range of variation of the gazing point coordinate values input in. As shown in FIG. 13, in the present embodiment, the maximum and minimum values of the gazing point coordinates in the X and Y directions within a fixed time period are stored, and that area is determined to be the subject existing area. That is, in FIG. 13, the width between the maximum value IPxmax and the minimum value IPxmin of the coordinate in the X direction of the screen is defined as the width in the X direction, and the width between the maximum value IPymax and the minimum value IPymin of the coordinate in the Y direction of the screen is Y.
The area OA having the width in the direction is determined as the main subject existing area.

In step S203, step S201
An area that does not overlap the main subject existing area obtained in step 3 is selected and set as the AS area, that is, the shake detection area. For example, as shown in FIG. 14, assuming that five areas (1) to (3) are prepared in advance as AS areas in the screen, it is determined in step S202 that the main subject existing area is determined as OA1 at the center of the screen.
If an area other than the AS area including the OA1 is considered to be the background and set as the AS area, and if the main subject existing area is determined to be OA2 at the center of the screen, the area other than the AS area including the OA2 is determined. Considering the background as a background and setting it in the AS area, it is a problem.

In step S204, the image signals separated by the I field in the AS region set in step S203 are compared to obtain a motion vector representing the size and direction of the blur, and a correction for canceling the AS vector is performed. Processing (controlling the read position of the memory) is performed.

Therefore, in the AS control, the main subject and the background, which have been difficult in the conventional art, can be discriminated from each other by detecting the gazing point, that is, the line of sight based on the intention of the operator. No AS control can be performed.

Further, in the above-mentioned embodiment, when setting the existence area of the main object, the x-direction of the gazing point coordinate within a fixed time, Y
Although the maximum value and the minimum value of the direction are stored and the area is determined to be the subject existing area (FIG. 13), the present invention is not limited to this. For example, as shown in FIG. It is also possible to set the moving average values of the x and Y directions as the center of the main subject area and set the size of the area in advance according to the focal length and the like.

Next, the AS control circuit 21 of the apparatus shown in FIG.
The entire process of 0 will be described with reference to the flow chart shown in FIG.

At step S301, the gazing point coordinate value (IP
x, IPy), that is, the coordinate value of the position gazed by the photographer is input, and in step S302, the gazing point coordinate value (IP
It is determined whether (x, IPy) is outside the field screen.
Here, the criterion for determining whether the screen is inside or outside the screen is determined based on the determination as to whether the screen is inside or outside the predetermined range as shown in FIG. 10 (the shaded area and the non-shaded area in the screen). It may be the whole or a predetermined range.

If the result of determination in step S302 is that the gazing point coordinate values (IPx, IPy) are outside the viewfinder screen, step S303 follows and the gazing point is detected and A
The subject tracking operation for determining the S detection area (set to a position other than the gazing point) is stopped, and the AS detection area is set to the peripheral area excluding the central portion of the screen in step S304, for example, the central area is excluded when viewed in FIG. The area is set as the AS detection area for blur detection, and the process proceeds to step S305 and the AS is detected.
The image reading position from the memory is shifted in the direction in which the motion vector detected from the detection area and representing the blurring is canceled, and the blurring is corrected.

If the gazing point coordinates are within the finder screen in step S302, the process proceeds to step S306 and the gazing point coordinates, that is, the area in which the main subject exists, according to the procedure described above with reference to FIGS. After setting
In step S307, the area excluding the main subject area is AS
The area is set as the detection area, and the process proceeds to step S305 to perform the AS control operation.

As a result, it is possible to accurately discriminate the main subject area that the photographer is gazing from from the background, and to perform AS operation with respect to the background all the time. Even if it goes out of the screen, stable AS control can be maintained without causing a malfunction.

In each of the above embodiments, AF and A
However, in addition to these, the setting of the photometric area for detecting the brightness signal for performing the automatic exposure control (AE) and the setting of the color information detecting area for the automatic white balance (AWB) will be described. Can also be applied to.

As described above, according to this embodiment, the gazing point is detected and, for example, the distance measuring frame is set around the gazing point coordinates, and the area other than the distance measuring frame is the AS area, that is, the shake. By setting it in the correction area, it is possible to distinguish between the main subject and the background, which was difficult in the past.
It is possible to perform more accurate AS control as compared with the related art, and it is possible to perform shake detection in the background even for a moving subject.

Next, a third embodiment of the present invention will be described. In the present embodiment, the AF operation is performed by the distance measuring frame set according to the first embodiment described above, and the blur detection is performed by setting the area outside the distance measuring frame as the AS detection area. The automatic focus control operation and the image stabilization control operation are performed in parallel.

FIG. 17 is a block diagram showing the configuration of the present embodiment, which is substantially the same as the configuration of FIG. 11 but the AF control circuit 1 of FIG.
10 are combined, and those having the same configurations as the components in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted. The important thing here is the AF control circuit 1
The information about the distance measuring frame is supplied from 10 to the AS control circuit, and the AS control circuit 210 sets the area outside the distance measuring frame as the AS detection area for blur detection.

FIG. 18 is a flow chart showing the procedure of the AS area setting process in the AS control circuit.
1, the coordinate values of the gazing point (IPx, IPy) from the signal processing circuit 109 of the visual axis detection circuit ED, that is, the coordinate values of the position gazed by the photographer are input, and step S402 is performed.
Then, it is determined whether or not the gazing point coordinate values (IPx, IPy) are out of the field screen. Here, the criteria for determining whether the window is inside or outside the screen is determined based on the determination as to whether the area is inside or outside the predetermined range (the shaded area and the non-shaded area in the screen) as shown in FIG.

If the result of determination in step S402 is that the gazing point coordinate values (IPx, IPy) are outside the viewfinder screen, step S403 follows and the AF control circuit 110.
Input the information of the range-finding frame set by the peak tracking in the above, set the area other than the range-finding frame in the screen as the AS detection area by considering it as the background, and execute the AS control in step S404, that is, shake detection and blurring Perform correction processing. That is, the blurring is corrected by shifting the cut-out range of the image from the memory in the direction in which the motion vector representing the blur detected from the AS detection area is canceled.

If it is determined in step S402 that the gazing point coordinates are within the viewfinder screen, step S402 is performed.
After shifting to 405 and setting the gazing point coordinates, that is, the area in which the main subject exists, the area excluding the main subject area is set as the AS detection area in step S406, and the flow advances to step S405 to execute the AS control operation. Done.

As a result, when the photographer is gazing at the main subject, the distance measuring frame is set at the gazing point coordinates and A
While performing the F operation, the area other than the gazing point is set as the AS area as described above, and the blur detection and the blur correction can be performed based on the background of the moving subject. Also, when the photographer is not gazing at the main subject, the distance measuring frame is set by the subject feature tracking operation based on the feature of the subject such as the peak detection position of the high frequency component,
Similarly, the outer region can be set as the AS region for blur correction. That is, regardless of whether or not the subject moves, the state of the gazing point of the photographer, and other shooting environments, the subject can always be tracked accurately and various functions such as AF and AS can be optimally controlled in parallel. Stable and optimal shooting is possible.

Further, particularly in the AS control, it is possible to accurately discriminate the main subject region which the photographer is gazing from from the background, and to always perform the AS operation with respect to the background, and Even if is out of the window, stable AS control can be maintained without causing a malfunction.

Also in this embodiment, as described with reference to FIG. 13, the X and Y directions of the gazing point coordinates within a fixed time period.
It is the same in that the maximum value and the minimum value in the direction are stored, the area is determined as the subject existing area, and the area that does not overlap with the AS main subject existing area is selected as the AS area, that is, the shake detection area. Like, ~ 5 of
One of the areas may be selected, or the entire area outside the distance measurement frame may be the AS detection area.

In each of the above embodiments, AF and A
Although S has been described as an example, in addition to these, it is also possible to set a photometric area for detecting a luminance signal for performing automatic exposure control (AE) and a color information detecting area for automatic white balance (AWB). Can be applied.

The device for recording the photographed image is not limited to the one using a magnetic tape such as a camera-integrated VTR, but a floppy disc, an optical disc,
Needless to say, the present invention can be applied to a photographing and recording device that specifies a recording medium such as a magneto-optical disk, and the type of recorded image can be either a moving image or a still image.

[0093]

As described above, according to the present invention, the position of the main subject is accurately determined by detecting the gazing point coordinates and the control target area such as AF and AS is set. , It becomes possible to shoot with more respect for the will of the photographer, and the photographer does not gaze at the main subject,
If the gaze point coordinates are detected at a position that would not be possible if a subject such as the outside of the screen is being gazed at, the feature of tracking a feature point of another subject, for example, from the gaze tracking operation By switching to the point tracking operation, regardless of the type of subject, the presence or absence of movement, and the shooting environment, it is possible to always perform reliable and highly accurate control, and to perform optimum shooting.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of an image pickup apparatus using a visual line detection device according to the present invention.

2 is a line-of-sight detection device 100 in the block diagram of FIG.
It is a figure which shows the structure of.

FIG. 3 is a diagram showing an example of a position of a corneal reflection image on a plane including an X axis and a Y axis.

FIG. 4 is a diagram showing an example of a position of a corneal reflection image on a plane including an X axis and a Y axis.

FIG. 5 is a flow chart showing an example of a visual line detection procedure by a signal processing circuit 109 of the visual line detection device.

FIG. 6 is a diagram showing an example of a reflected image from an eyeball 201.

FIG. 7 is a diagram showing an example of an output signal obtained from a row Yp ′ of a photoelectric element column.

8 is a block diagram showing an internal configuration of an AF control circuit in the block diagram of FIG.

9 is a flow chart showing a control operation of the AF control circuit in FIGS.

FIG. 10 is a diagram showing an example of a surface of a finder for explaining the gazing point coordinate determination operation on the finder screen.

FIG. 11 is a block diagram showing a second embodiment of the present invention equipped with an AS device using a line-of-sight detection device.

FIG. 12 is a flow chart for explaining AS control in the second embodiment of FIG.

FIG. 13 is a diagram illustrating a process of determining an area where a main subject exists.

FIG. 14 is a diagram for explaining a setting operation of a shake detection area for AS control.

FIG. 15 is a diagram for explaining another example of the processing for determining the area in which the main subject exists.

16 is a flow chart for explaining the operation of the second embodiment shown in FIG.

FIG. 17 is a block diagram showing a third embodiment of the present invention in which an AF and AS device using a line-of-sight detection device is mounted.

FIG. 18 is a flow chart for explaining the operation of the third embodiment shown in FIG.

Claims (5)

[Claims] 1. A photoelectric conversion unit that converts an optical image formed on an image pickup surface into an electric signal, and an optical image on the image pickup surface is displayed on a monitor screen based on an electric signal output from the photoelectric conversion unit. Monitor, and a gazing point detecting means for detecting the gazing point of the operator with respect to the monitor screen. The first point of tracking the movement of the gazing point based on the position information of the gazing point detected by the gazing point detecting means. Tracking control means, and second for tracking the movement of the optical image without using the gazing point
The tracking control means, and the control means for selectively operating the first tracking control mode and the second tracking control mode. 2. A photoelectric conversion means for converting an optical image formed on the imaging surface into an electric signal, and an optical image on the imaging surface is displayed on a monitor screen based on the electric signal output from the photoelectric conversion means. A monitor, a gazing point detecting unit that detects the gazing point of the operator with respect to the monitor screen, and a movement tracking unit that tracks the movement of the gazing point based on position information of the gazing point detected by the gazing point detecting unit. A second tracking control means, and a second tracking means for tracking the movement of the optical image without using the gazing point.
Tracking control means, determination means for determining whether or not the gazing point is within a predetermined area on the screen of the monitor, and the first tracking control mode and the first tracking control mode based on the output of the determination means. 2. An image pickup apparatus comprising: a control unit that selectively operates the tracking control mode of No. 2. 3. A photoelectric conversion means for converting an optical image formed on an imaging surface into an electric signal, a gazing point detecting means for detecting a gazing point of the operator, and an imaging area on the imaging surface excluding the gazing point. A detection area setting means for setting a shake detection area for extracting information on a predetermined movement in the area; and a shake correction means for performing the shake correction based on the shake information detected from the shake detection area, An imaging device characterized by the above. 4. A photoelectric conversion means for converting an optical image formed on an imaging surface into an electric signal, a gazing point detecting means for detecting a gazing point of the operator, and a gazing point detecting means for detecting the gazing point. Determination means for determining whether or not the gazing point exists in a predetermined area on the screen of the monitor, and a detection area for extracting predetermined information in an area on the imaging surface excluding the gazing point. The detection area setting means to be set and the detection area setting means on the basis of the output of the judging means, and when the judging means judges that the gazing point is present in the predetermined area, The detection area is set at a position within the image pickup surface excluding the viewpoint, and when it is determined that the gazing point is outside the predetermined area, the detection area is set at a predetermined position within the image pickup surface. Imaging means comprising control means and Location. 5. A photoelectric conversion means for converting an optical image formed on an image pickup surface into an electric signal, a monitor means for displaying an image on the image pickup surface on a monitor screen, and an operator's gazing point on the monitor screen. A gazing point detecting means for detecting the gazing point, a first tracking control means for setting a ranging frame based on positional information of the gazing point detected by the gazing point detecting means, and the gazing point without using the gazing point. Second tracking control means for detecting the movement of the optical image to follow the focus detection frame, and a blur detection area set in an area outside the focus detection frame set in the first or second tracking control mode. An image pickup apparatus comprising: a shake detection unit; and a control unit that selectively operates the first tracking control mode and the second tracking control mode.