JPH0694977A - Eye gaze detector - Google Patents

Abstract

(57) [Abstract] [Purpose] It is possible to accurately obtain the line of sight of the eye even with a light source at a finite distance. [Arrangement] An illumination unit 101 for illuminating an eyeball of an observer,
In a line-of-sight detection apparatus including a line-of-sight calculation unit 102 that calculates a line-of-sight of the eyeball from the reflected image of the eyeball illuminated by the illumination unit 101, the distance between the illumination unit 101 and the corneal curvature center position of the eyeball is used. The line-of-sight calculation means 102 calculates the line of sight.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight detection device for detecting a so-called line of sight, that is, a line of sight of an observer on a visual field screen provided in an optical device.

[0002]

2. Description of the Related Art Conventionally, as an apparatus for detecting the line of sight of an observer, there has been one disclosed in, for example, Japanese Patent Laid-Open No. 3-109029. The device disclosed in this publication illuminates the eyeball of an observer with a light source, reads the boundary between the pupil and the iris of the eyeball illuminated by the light source to obtain the center position of the pupil, and the cornea with the illumination light from the light source. The position of the reflection image was obtained, and the line of sight was obtained from the relative relationship between the center position of the pupil and the position of the corneal reflection image. The operation of the line-of-sight detection device according to the above publication will be described below with reference to FIG.

FIG. 36 is a sectional view of a human eyeball cut horizontally. In this figure, 1 is an eyeball, and this eyeball 1 has a substantially spherical sclera 2 filled with a vitreous body 3,
A crystalline lens 4, an iris 5 and a cornea 6 are formed in a front portion (left portion in the drawing) of the sclera 2 to be generally configured. The iris 5 is a kind of diaphragm, and the opening is called a pupil 7. The curvature of the cornea 6 is different from the curvature of the sclera 2 (vitreous body 3), and as shown in the figure, a distance ρ between the corneal curvature center C and the eyeball rotation center O ′ that can be assumed to be almost constant is generated. Also, the pupil center D
It can be assumed that the distance A between the eyeball rotation center O'and the eyeball rotation center O'is also substantially constant.

As shown in FIG. 36, the x-axis is set in the horizontal direction (vertical direction in the figure), and the position of the x-axis of the eyeball rotation center O'when the eyeball 1 is directly facing the center of the field-of-view screen (not shown) is the origin. O
And At this time, the position of the pupil center D on the x axis (also indicated by D) and the position of the corneal reflection image P on the x axis (also indicated by P) are defined by the eyeball rotation angle θ as shown in the figure. ,

## EQU1 ## D = L + A × sin θ (1) P = L + ρ × sin θ (2) The corneal reflection image P referred to here is the Purkinje first
This is called an image, and is a virtual image of light rays reflected on the surface of the cornea 6 when the cornea 6 is considered as a convex lens. From equation (1)
By subtracting the equation (2), the parallel movement component L of the eyeball rotation center can be canceled,

## EQU00002 ## DP = (A-.rho.). Times.sin .theta. (3) is obtained. At this time, assuming that the eyeball rotation angle θ is small, assuming that sin θ≈θ holds, the eyeball rotation angle θ
When we calculate

## EQU3 ## θ = (DP) / (A-ρ) (4)

[0005]

However, in the conventional line-of-sight detection device disclosed in the above publication, (1)
〜Equation (3) is only applicable to the corneal reflection image that is originally formed by parallel rays, but even for a light source for eye-gaze detection at a finite distance, the eye-gaze detection is performed using these equations (1)-(3). Since the calculation is performed, there is a problem that the obtained line of sight may deviate from the line of sight of the observer.
This will be described with reference to FIG. FIG. 37 (a) shows the case where the parallel light beam PL is incident from the front of the eyeball 1, and the corneal reflection image P is a distance r / 2 from the corneal curvature center C (r is the cornea 6).
(Curvature radius of) and the eyeball rotation center O ′, the corneal curvature center C, and the corneal reflection image P are on a straight line. When the light source S is located at a finite distance as shown in FIG. 37 (b), the light beam from the light source S at the finite distance is divergent light DL, and therefore the corneal reflection image P'is compared with the case of parallel light rays. It occurs at a position close to the light source S by Δr. Furthermore, as shown in FIG. 37 (c), when the eyeball translates downward by a distance L in the figure, a corneal reflection image P'is produced at a position rotated by α around the corneal curvature center C, and when viewed from the front, ΔL It is only shifted upward in the figure. On the other hand, if the parallel rays are incident from the front of the eyeball 1, a corneal reflection image is generated at the position P in the figure even if the eyeball 1 moves in parallel as shown in FIG. Therefore, ΔL is a shift due to placing the light source S at a finite distance.

An object of the present invention is to provide a line-of-sight detection device which can accurately obtain the line-of-sight of an eye even with a light source located at a finite distance.

[0007]

When the invention of claim 1 is described in association with FIG. 1 which is a claim correspondence diagram, the invention of claim 1 is an illumination means 101 for illuminating an eyeball of an observer and the illumination means 10.
The present invention is applied to a visual line detection device including a visual line calculation unit 102 that calculates the visual line of the eyeball from the reflection image of the eyeball illuminated by 1. The above-mentioned object is achieved by the line-of-sight calculation unit 102 calculating the line of sight using the distance between the illumination unit 101 and the corneal curvature center position of the eyeball. At this time, the line of sight may be calculated by using the distance between the illumination unit 101 and the corneal curvature center position when the line of sight is directed in a specific direction, as a representative value. Alternatively, the visual field screen is divided into a plurality of areas, and a line-of-sight determining unit 103 for determining which of the divided regions the line of sight is facing is provided, and the predetermined illumination unit 101 is provided for each of the divided regions. The line of sight may be calculated using a distance from the corneal curvature center position as a representative value. According to the invention of claim 4, a line-of-sight detection is provided that includes an illumination unit 101 that illuminates an eyeball of an observer, and a line-of-sight calculation unit 102 that calculates a line-of-sight of the eyeball from a reflected image of the eyeball illuminated by the illumination unit 101. Applies to equipment. And, the above-mentioned purpose is the illumination means 101.
This is achieved by the line-of-sight calculating unit 102 calculating the line of sight by using the distance between the illuminating unit 101 and the center of the pupil instead of the distance between the center of curvature of the cornea and the center of corneal curvature. Further, the distance calculating means 1 for calculating the distance between the illuminating means 101 and the corneal curvature center position based on the visual line calculated by the visual line calculating means 102.
04, and the line-of-sight calculation means 10 based on this calculation result.
The line-of-sight may be calculated again according to step 2, and the control unit 105 for repeatedly operating the distance calculating unit 104 and the line-of-sight calculating unit 102 may be provided until the calculation result converges. When a plurality of the illuminating means 101 are provided and the visual line calculating means 102 independently calculates the visual line for each of the illuminating means 101, based on the visual line for each of the illuminating means 101 calculated by the visual line calculating means 102. A line-of-sight determining means 106 for determining the line of sight may be provided.

[0008]

In the conventional line-of-sight detection device described above, the line-of-sight detection is performed assuming that the illumination means is at infinity.
Actual lighting means emits divergent light from a finite distance.
Therefore, by calculating the line of sight using the distance between the illumination unit 101 and the corneal curvature center position of the eyeball, the line of sight of the observer can be correctly obtained even when the illumination unit 101 is at a finite distance.

[0009]

【Example】

(1) Description Common to Embodiments First, the configuration common to each embodiment and the principle of the present invention will be described. In the following description, the eyeball of the observer and the inside thereof will be described using the reference numerals used in FIG. FIG. 2 is a configuration diagram of the visual line detection device according to the present invention. In FIG. 2, 11 is an observation surface provided in an optical device such as a camera, and
It is gazing at a specific point (gazing point) in 1. Reference numeral 12 is an observer's eye, 13 is a light source for driving a visual axis detection, and 14 is a light source for a visual axis detection.
The observer's eyes 12 are arranged in the vicinity of the position so as not to obstruct the visual field for observing the observation surface 11. Reference numeral 15 is a dichroic mirror that transmits a visible region and reflects an infrared region, 16 is an imaging lens, 17 is a photoelectric conversion element, and the optical axis L of the photoelectric conversion element 17 and the imaging lens 16
A coincides with the axis CA passing through the center of the observation surface 11 and the origin O in FIG. 36 (corresponding to the eyeball rotation center O ′ when the eyeball 1 is located at a position directly facing the center of the observation surface), and the dichroic The optical axis LA is bent by the mirror 15 so as not to obstruct the visual field for observing the observation surface 11. Reference numeral 18 is a signal processing means for processing the output of the photoelectric conversion element 17, and 19 is an arithmetic unit.
Calculates the line of sight from the control of the line-of-sight detection light source drive unit 13, the drive control of the photoelectric conversion element 17, and the output of the signal processing unit 18.

3 to 5 show another example of the arrangement of the light source 14 for detecting the line of sight and the photoelectric conversion element 17. In the example shown in FIG. 3, the imaging lens 16 and the photoelectric conversion element 17 are arranged at positions that do not coincide with the axis CA passing through the center of the observation surface 11. In the example shown in FIG. 4, the line-of-sight detection light source 14, the imaging lens 16, and the photoelectric conversion element 17 are arranged with their positions interchanged, and the line-of-sight detection light source 14 is on the axis CA passing through the center of the observation surface 11. Placed and dichroic mirror 15
Is designed so as not to obstruct the view of the observer. In the example shown in FIG. 5, the line-of-sight detection light source 14 and the imaging lens 1
6, both photoelectric conversion elements 17 are arranged on the axis CA passing through the center of the observation surface 11, and the dichroic mirror 1
Reference numerals 5 and 20 do not interfere with the visual field of the observer.

6A to 6F, 7A to 7B, and 8A to 8E, the observer's eye 12 is located at a position directly facing the center of the observation surface 11. At this time, the observer's eye 12 has the observation surface 11
It is a figure which shows the arrangement state of the gaze detection light source 14 when observing. In the illustrated example, the line-of-sight detection light source 14 has an IRED.
14 _{1 to} 14 ₃ are used. The frame 21 in the figure is assumed to be a frame that determines the range of the observation surface 11 when the observer eye 12 is in a position directly facing the center of the observation surface 11 as shown in FIG. Has a field of view equal to. Hereinafter, this frame is referred to as a “field frame”. This field frame 21 is a frame that determines the frame to be observed by the observer eye 12, and therefore the observer eye 1
It may be set at any position from immediately before 2 to the observation surface 11.

In FIGS. 6A to 6F and FIGS. 7A to 7B, the IREDs 14 _{1 to} 14 ₃ are arranged in the visual field frame 21 so that the IREDs 14 _{1 to} 14 ₃ do not interfere with the visual fields observed by the observer. It is placed on the outer circumference of. In the example shown in FIGS. 6A and 6B, IRED is used.
One 14 ₁ is arranged, (A) is arranged at the upper center of the field frame 21, and (B) is arranged at the left center of the field frame 21. In the example shown in FIGS. 6C to 6E, IRED 14 ₁ and 14 ₂
2 are arranged, (C) is at the upper center and left center of the field frame 21, (D) is at the upper part of the field frame 21, and (E) is a vertically symmetrical position across the center of the field frame 21. It is located in.
In the example shown in FIG. 6 (F) and FIGS. 7 (A)-(B), IRED 14 _1
3 to 14 ₃ are arranged, and FIG.
In the upper part of FIG. 7, (A) is arranged in the upper part of the field frame 21, and two in the lower part, and in FIG.
One at the center on the left side and one at the center on the right side.

Further, in the example shown in FIG. 8 (A) ~ (E) , using a dichroic mirror or the like, are arranged IRED14 ₁ ~14 ₃ inside the field frame 21, (A) is IRED1 of one
4 ₁ is placed in the center of the field frame 21, (B) is one IR
ED14 ₁ has another IR in the upper center of the field frame 21.
The ED 14 ₂ is arranged inside the field frame 21, and (C), two IREDs 14 ₁ and 14 ₂ are both arranged inside the field frame 21. In the example shown in FIGS. 8D and 8E, _three IREDs 14 _{1 to} 14 ₃ are arranged, (D) is arranged both on the outer periphery and the inner side of the field frame 21, and (E) is all the field frame 21.
Is located inside. In addition, IRED 14 _{1 to} 14 ₃
The arrangement is not limited to the illustrated example, and one or a plurality of IREDs may be arranged around the eyes of the observer.

FIG. 11 is a diagram showing a coordinate system introduced to explain the principle of the present invention. In FIG. 11, an eyeball rotation center position O ′ when the observer's eyeball is in a position directly facing the center of the observation surface is an origin O, and the origin O and the observation surface 11
Y axis in the axial direction passing through the center of X and X in one direction of the observation surface 11.
The axis is set to three-dimensional coordinates having the Z axis in a direction perpendicular to both the X axis and the Y axis, and this is hereinafter referred to as "real space coordinates".

FIG. 10 is a diagram showing the optical principle of the present invention. The line-of-sight detection light source 14 is near the observer eye 12,
The light source 14 forms a virtual image of the corneal reflection image on the observer eye 12 at point P. This virtual image is re-imaged at point P ′ on the photoelectric conversion element via the imaging lens 16. At this time, a coordinate system having one direction of the light receiving surface of the photoelectric conversion element 17 as the X ′ axis and the Z ′ axis in the direction perpendicular to the X ′ axis is set, and this is hereinafter referred to as “observation plane coordinate”. The point P is P: (Px, P
y, Pz), and the point P ′ is P ′: on the observation plane coordinate.
It is represented by (Px ', Pz'). At this time, the relationship between points P and P ′ is defined by the functions f _x and f _z considering the angle of the light receiving surface of the photoelectric conversion element with respect to the real space coordinate system, that is, the angle of the observation plane coordinate system with respect to the real space coordinate system. If used,

(4) Px = f _x (Px ′, Pz ′) (5) Pz = f _z (Px ′, Pz ′) (6) can be converted into the xz plane of the real space coordinate system. . Therefore, the corneal reflection image observed on the photoelectric conversion element 17 and expressed in the observation plane coordinate system, and the coordinates of the image formation point on the photoelectric conversion element 17 at the pupil center position are xz in the real space coordinate system.
It can be arbitrarily converted to a plane. Therefore, the coordinates of the image formation point formed on the photoelectric conversion element 17 can be calculated in the xz plane coordinates of the real space coordinate system using the conversion equations (5) and (6). What has been described above is that even if the positional relationship between the point P, the imaging lens 16 and the photoelectric conversion element 17 changes, the conversion formula
Only the specific shapes of (5) and (6) are changed, and the points P and P'can be converted using the conversion formulas (5) and (6).

Next, the line-of-sight detection light source 14 used in the present invention is a point light source at a finite distance and illuminates the eyeball as divergent light. The principle will be described using the above real space coordinate system. As shown in FIG. 11, the light source 1 in the real space coordinate system
The coordinate values of the Purkinje first image P: (X _p1 , Y _p1 , Z _p1 ) and the pupil center D: (Xd, Yd, Zd) according to No. 4 are the light source S:
(Sx, Sy, Sz), corneal curvature center C: (Cx, Cy,
Cz) = (L + ρcosφsin θ, ρcosφcos θ, k + ρsin
It is expressed as follows using the coordinate value of φ). Where f
Is the focal length of the corneal surface reflection, ρ is the distance between the center of rotation of the eyeball and the center of curvature of the cornea, L is the translation of the center of rotation of the eyeball in the X-axis direction, θ is the rotation angle of the eyeball in the X-axis direction, and k is the Z-axis direction. The translation of the eyeball rotation center, φ is the eyeball rotation angle in the Z-axis direction.
First, the coordinate value of the Purkinje first image position P is

## EQU00005 ## X _p1 = f / (σ−f) × Sx + (σ−2f) / (σ−f) × (L + ρ cosφsin θ) (7) Y _p1 = f / (σ−f) × Sy + (σ−2f) ) / (Σ−f) × (ρcosφcosθ) (8) Z _p1 = f / (σ−f) × Sz + (σ−2f) / (σ−f) × (k + ρsinφ) (9) However, σ = √ ( ^{(Sx-Cx) 2 + (} Sy-Cy) 2 + (Sz-Cz) 2) become (10). Here, σ represents the distance between the light source S and the corneal curvature center C.

With respect to the coordinate value of the pupil center D, the distance from the eyeball rotation center O'to the pupil center D is A,

## EQU6 ## Xd = L + A cos φ sin θ (11) Zd = k + A sin φ (12) If the equations (7) to (9), (11), and (12) are combined, θ, L, φ, and k can be obtained and the line of sight can be obtained. In the present invention, however, σ takes a specific value. Assuming that
Let us determine L, φ, k. Also, the eyeball rotation angle θ,
Considering that φ is sufficiently small, sin θ ≈ θ, cos θ ≈ 1, si
nφ≈φ and cosφ≈1 are used. At this time (7) ~
Equations (9), (11), and (12) are

X _p1 = f / (σ−f) × Sx + (σ−2f) / (σ−f) × (L + ρθ) (7 ′) Y _p1 = f / (σ−f) × Sy + (σ− 2f) / (σ−f) × ρ (8 ′) Z _p1 = f / (σ−f) × Sz + (σ−2f) / (σ−f) × (k + ρφ) (9 ′) Xd = L + Aθ (11 ′) Zd = k + Aφ (12 ′). Solving this,

[Equation 8] θ = 1 / (A−ρ) × {f / (σ−2f) × Sx + Xd− (σ−f) / (σ−2f) × X _p1 } (13) L = −A / (A −ρ) × f / (σ−2f) × Sx−ρ / (A−ρ) × Xd + A / (A −ρ) × (σ−f) / (σ−2f) × X _p1 (14) φ = 1 / (A−ρ) × {f / (σ−2f) × Sz + Zd− (σ−f) / (σ−2f) × Z _p1 } (15) k = −A / (A−ρ) × f / ( σ−2f) × Sz−ρ / (A−ρ) × Zd + A / (A−ρ) × (σ−f) / (σ−2f) × Z _p1 (16). However, σ = √ ((Sx− (L + ρθ)) ² + (Sy−ρ) ² + (Sz− (k + ρφ)) ² ) (10 ′). Here, the line of sight θ, L, and
Since φ and k are included, it becomes necessary to substitute these values with some value. An example will be described below.

(2) When there is one light source First, the line-of-sight detection when _one IRED 14 ₁ is used as the line-of-sight detection light source 14 shown in FIGS. 6A, 6B and 8A. An example of the apparatus will be described.

First Embodiment In the first embodiment, the line of sight θ, L,
This is a method in which φ and k are each represented by one value. In the present embodiment, when the observer observes the observation surface 11, the number of the eyeballs 1 is fixed at θ = L = φ = k = 0, which is considered to be the most natural position. At this time, the distance σ between the light source S and the corneal curvature center C is

## EQU7 ## σ = √ (Sx ² + (Sy−ρ) ² + Sz ² ) (17) Using the value of σ fixed in this way, (1
The lines of sight θ, L, φ, k are calculated by calculating equations 3) to (16).

FIG. 12 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in step S101, the IRED1 that is the light source 14 for line-of-sight detection.
The observer's eye 12 is illuminated by 4 ₁ and this IRED 14 ₁
The boundary between the pupil 7 and the iris 5 of the eyeball 1 illuminated by
Two points on the photoelectric conversion element _{17 x 'dL, z' dL} , x 'dR,
z 'is read as a coordinate value of _dR. The boundary detection method is well known (for example, Japanese Patent Application Laid-Open No. 3-109029), and details thereof will be omitted here. In step S102, the coordinate values x _'dL, z' of the two points on the photoelectric conversion element 17 that was read in step _{S101 dL, x 'dR, z} '
_dR is converted into real space coordinates, and the coordinate values x _dL , z _dL , x _dR , and z _dR of the boundary between the pupil 7 and the iris 7 on the real space coordinates are obtained. In step S103, the pupil 7 and the iris 5 in the real space are
The midpoint of the boundary between and is the pupil center position D, and this pupil center D
The coordinate values Xd and Zd of are calculated.

Next, in step S104, IRED1
4 Purkinje first caused by the illumination of the divergent light from ₁
The position of the image (corneal reflection image) P is detected as the coordinate values X _p1 ′ and Z _p1 ′ on the photoelectric conversion element 17. This Purkinje's first image detection method is also well known, and the details are omitted here.
In step S105, the coordinate values X _'p1, Z' _p1 of the first Purkinje image P on the photoelectric conversion element 17 is converted into real space coordinates, the coordinate values X _p1 of the first Purkinje image P in the real space coordinates,
_Find Z _p1 . Further, in step S106, σ obtained by substituting the coordinate values X _p1 and Z _{p1 of} the Purkinje first image P, the coordinate values Xd and Zd of the pupil center D, and the coordinate value of the light source S into the equation (17). Substituting in equations (13) to (16), the line of sight θ,
Find L, φ, k.

FIG. 13 shows the calculation result of the eyeball rotation angle θ in the x-axis direction thus obtained. The horizontal axis of the graph represents the actual eyeball rotation angle in the x-axis direction, and the vertical axis represents the eyeball rotation angle in the x-axis direction calculated by the line-of-sight detection device of this embodiment (both units are “radians”). Note that the eyeball rotation center O'is x
The light source S (IRED14
₁ ) is placed at the position shown in FIG. 6A, and its coordinate values are (0, 2
8, 8) (the unit is "millimeter"). In FIG. 13, a curved line S ₀ passing through the origin is an actual line of sight, and a straight line that coincides with the straight line S ₀ is obtained when correctly calculated. On the other hand, the curve S ₁ that does not pass through the origin is an example of the line of sight calculated using this embodiment, and does not pass through the origin except when L = φ = k = 0, and the distance σ between the light source S and the corneal curvature center C is σ. There is an error δ due to fixing
Is considered to be in the negligible range,
Θ, L, φ, k obtained in step S106 is the line of sight to be obtained. In the above description, θ = L = φ = k = 0
However, it may be fixed to any other suitable value, such as θ,
L, φ, and k may have different values.

Second Embodiment In the second embodiment, there are several kinds of values θ, L, φ, k in the equation (10 ') depending on the position of the Purkinje's first image P or the center D of the pupil, or both positions. The light source S
The distance σ between the center of curvature C and the corneal curvature center C is fixed to several values. In this embodiment, the x-axis is divided into three areas, and σ is fixed to three kinds of values. Specifically, the x-coordinate value Xd of the pupil center D is less than χ1, χ1 or more χ
When the value is 2 or less and exceeds χ2, the values of the parallel movement amount L of the eyeball rotation center in the x-axis direction are fixed to λ1, λ2, and λ3, and the other values are fixed to θ = φ = k = 0. Σ at this time
Is the translation amount L of the eyeball rotation center O ′ in the x-axis direction, which is represented by L ′ for convenience.

[Expression 10] σ = √ ((Sx−L ′) ² + (Sy−ρ) ² + Sz ² ) (18)

FIG. 14 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in steps S201 to S205, operations similar to those in steps S101 to 105 of the above-described first embodiment are performed. In step S206, it is determined whether the x-coordinate value Xd of the pupil center D is smaller than a predetermined value χ1. If the determination is affirmative, the process jumps to step S207, and the translational movement of the eyeball rotation center O ′ in the x-axis direction is performed. The quantity L ′ is set to λ1 and if the determination is negative, the process proceeds to step S208. In step S208, it is determined whether or not the x-coordinate value Xd of the pupil center D is larger than a predetermined value χ2. If the determination is affirmative, the process jumps to step S209, and the eyeball rotation center O ′ in the x-axis direction is translated. The amount L ′ is set to λ3, and if the determination is negative, the process jumps to step S210 to set the parallel movement amount L ′ of the eyeball rotation center O ′ in the x-axis direction to λ2. Further, in step S211, the coordinate values X _p1 and Z _{p1 of} the Purkinje first image P, the coordinate values Xd and Zd of the pupil center D, the coordinate value of the light source S, and the translation amount L ′ of the eyeball rotation center O ′ in the x-axis direction. Substituting σ into equation (18) into equations (13) to (16), the line of sight θ,
Find L, φ, k.

FIG. 15 shows the calculation result of the eyeball rotation angle θ in the x-axis direction thus obtained. The horizontal axis of the graph represents the actual eyeball rotation angle in the x-axis direction, and the vertical axis represents the eyeball rotation angle in the x-axis direction calculated by the line-of-sight detection device of this embodiment (both units are “radians”). Note that the eyeball rotation center O'is x
-It is assumed that the light source position S moves in the z plane, and the light source position S is shown in FIG.
, And the coordinate value is (0, 28, 8) (the unit is "millimeter"). Also, the values of χ1 to χ3 and λ1 to λ3 are χ1 = -2.5, χ2 = 2.5, λ1 = -5, λ2 = 0, using the values converted on the real space coordinates.
λ3 = 5 (unit is “millimeter”). That is, as shown in Table 1, the value of Xd in the equation (18) is divided into three types of "less than -2.5", "more than -2.5 and less than 2.5", and "more than 2.5", and the value of L'at this time. "-5",
It is fixed to “0” and “5”, and the other is fixed to θ = φ = k = 0.

[Table 1] In FIG. 15, a straight line S ₀ passing through the origin is an actual line of sight, and if it is correctly calculated, a straight line that matches the straight line S ₀ is obtained. On the other hand, the curve S ₂ that does not pass through the origin
Is an example of the line of sight calculated using the present embodiment, and L = −5,
φ = k = 0, L = 0, φ = k = 0, and L = 5, φ =
There is an error δ due to fixing the distance σ between the light source S and the corneal curvature center C without passing through the origin except when k = 0.
According to the present embodiment, as shown in Table 1, the amount of movement L ′ of the eyeball rotation center O ′ in the x-axis direction is appropriately changed according to the position of the pupil center D (x coordinate value Xd). Even in this case, the error δ can be reduced as compared with the first embodiment in which the value of L is fixed to one value.

In the present embodiment, the position of the pupil center D was converted into the real space coordinates and compared with χ1 and χ2. However, the values on the photoelectric conversion element 17 are directly used to obtain appropriate χ1 and χ2. You can also compare. Further, in the present embodiment, the x-axis direction is divided into three and σ is fixed to three values, but the z-axis direction can be similarly divided, and the number of divisions can be divided into two or more. It can be divided.

-Third Embodiment- In the third embodiment, the corneal curvature center position C in the equation (10) is used.
This is a method of substituting the pupil center positions Xd and Zd for x and Cz. That is, when both the eyeball rotation angles θ and φ are small, the corneal curvature center position Cx = L + ρθ, Cz
= K + ρφ value and pupil center position Xd = L + Aθ, Zd =
Assuming that the value of k + Aφ is not significantly different, (10 ')
In the formula, L + ρθ≈L + Aθ, k + ρφ≈k + A
φ At this time, the distance σ between the light source S and the corneal curvature center C is

Σ = √ ((Sx−Xd) ² + (Sy−ρ) ² + (Sz−Zd) ² ) (19) The value of σ obtained in this way is (13)
The line-of-sight is obtained by using the equation (16).

FIG. 16 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in steps S301 to S305, operations similar to those in steps S101 to 105 of the above-described first embodiment are performed. In step S306, the coordinate values X _p1 , Z of the first Purkinje image P are determined.
_p1 , the coordinate values Xd and Zd of the pupil center D, and the corneal curvature C
Substituting the coordinate values of Eq. (19) with the coordinate values of the pupil center D into Eqs. (13) to (16), the line of sight θ, L,
Find φ and k.

FIG. 17 shows the result of obtaining the eyeball rotation angle θ in the x-axis direction in this manner. The horizontal axis of the graph represents the actual eyeball rotation angle in the x-axis direction, and the vertical axis represents the eyeball rotation angle in the x-axis direction calculated by the line-of-sight detection device of this embodiment (the unit is "radian"). The center of rotation O ′ of the eyeball is assumed to move in the xz plane, and the light source S (IRED14 ₁ ) is shown in FIG.
It is arranged at the position (A), and its coordinate value is (0, 28, 8) (the unit is "millimeter"). In FIG. 17, a straight line S ₀ passing through the origin is an actual line of sight, and if it is correctly calculated, a straight line that matches the straight line S ₀ is obtained. On the other hand, the curve S ₃ that does not pass through the origin is an example of the line of sight calculated using this embodiment. The error δ due to the fixed distance σ between the light source S and the corneal curvature center C is small when the eyeball rotation angle θ in the x-axis direction is small, even if the values of L, φ, and k change. As the eyeball rotation angle θ increases, the error δ due to the changes in the values of L, φ, and k increases, but the magnitude of the error δ at this time is considered to be within a negligible range, so step S306. Θ, L, φ, k obtained in step 3 is the line of sight to be obtained.

-Fourth Embodiment-The fourth embodiment uses the above-mentioned first to third embodiments as they are to reduce the error δ caused by fixing the distance σ between the light source S and the corneal curvature center C. The method is used. That is, the line-of-sight θ, L, φ, obtained in the first to third embodiments,
Assuming that k is a tentative line of sight with a large error δ, let these be θ ′, L ′, φ ′, k ′, and θ in the equation (10 ′)
Expression in which L, φ, k are replaced by θ ', L', φ ', k'

Σ = √ ((Sx− (L ′ + ρθ ′)) ² + (Sy−ρ) ² + (Sz− (k ′ + ρφ ′)) ² ) (20) θ ′, L ′, φ The error δ can be reduced by substituting ′ and k ′ again to obtain a new σ and calculating the equations (13) to (16) again. This calculation may be repeated until the lines of sight θ, L, φ, k converge.

FIG. 18 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in step S401, the method of any one of the above-described first to third examples, or the distance σ between the light source S and the corneal curvature center C is fixed by some method and the line of sight θ, L. , Φ, k
The line-of-sight θ, L, φ, k is obtained by the method of obtaining In step S402, it is assumed that the line of sight θ, L, φ, k calculated in step S401 is a temporary line of sight with a large error δ due to the fixed σ, and θ, L, φ, k are respectively set to temporary values θ. Let ', L', φ ', k'. In step S403, the coordinate values of θ ′, L ′, φ ′, k ′ and the light source S replaced in step S402 are set to (20)
Substituting into the equation, a new σ is obtained, and this σ is (13) to (16)
Substituting into the equation, the line of sight θ, L, φ, k is obtained again. In step S404, the provisional eyeball rotation angle θ ′ in the x-axis direction obtained in step S401 is compared with the size of the eyeball rotation angle θ in the x-axis direction newly obtained in step S403, and the difference between them | θ′− It is determined whether θ | is less than or equal to a predetermined threshold value ε. As a result, if the determination is positive, step S40.
When the determination is negative, the process returns to step S402 and the above procedure is repeated.
In this way, the line-of-sight detection operation is continued until the value of | θ′−θ | converges to the threshold value ε or less.

The line-of-sight θ, L, φ, k obtained in the first embodiment
, Iteratively computes until the values are well focused, x
FIG. 19 shows the calculation result of the eyeball rotation angle θ in the axial direction. The horizontal axis of the graph represents the actual eyeball rotation angle in the x-axis direction, and the vertical axis represents the eyeball rotation angle in the x-axis direction calculated by the line-of-sight detection device of this embodiment (the unit is "radian"). It is assumed that the eyeball rotation center O ′ moves in the xz plane, the light source S is arranged at the position shown in FIG. 6A, and its coordinate value is (0,
28, 8) (the unit is "millimeter"). FIG. 19
In the range of the accuracy that can be illustrated, the line converges to the straight line S ₀ when the line of sight is accurately obtained. Therefore, the values θ, L, φ when the determination is affirmative in step S404,
k is the desired line of sight.

In this embodiment, the value | θ'-θ | is used to determine whether or not the line of sight has converged in step S404, but the values | θ'-θ |, | L'-L |, | φ′−φ |, |
Any value of k′−k | may be used, and a plurality or all of them may be used for the determination. Further, according to the present embodiment, the error δ due to fixing the distance σ between the light source S and the corneal curvature center C by performing repeated calculation.
Therefore, it is possible to reduce the line of sight θ, L, which is obtained by repeating the calculation a predetermined number of times without determining whether or not the values of the line of sight θ, L, φ, k have converged in step S404. , Φ, k. Furthermore, the method of repeating the calculation until it converges and the method of repeating the calculation a predetermined number of times are used together, and if the calculation does not converge even if the calculation is performed a predetermined number of times, the iterative calculation is aborted and the calculation is performed at that point. The line-of-sight θ, L, φ, k to obtain the line-of-sight can be used. Further, the method of obtaining the tentative line of sight is not limited to the above-described first to third embodiments,
As long as the distance σ between the light source S and the corneal curvature center C is fixed by some method to obtain the line of sight θ, L, φ, k, a method not introduced in this embodiment may be used.

(3) When there are a plurality of light sources Next, FIGS. 6C to 6F, 7A, 7B and 8B.
As shown by (E) to (E)
A case where a plurality of D14 _{1 to} 14 ₃ are arranged will be described. In the present invention, the IRE which is the light source 14 for detecting the line of sight
Providing a plurality of D14 _{1 to} 14 ₃ has the following advantages. IRED 14, which is the nth line-of-sight detection light source 14.
using the position of the position and the pupil center D of the first Purkinje image P _n by _n, each IRED using the aforementioned sight line detection method when the IRED14 ₁ is provided only one
14 ₁ to 14 _n obtains the position of the first image P ₁ to P _n Purkinje by each sight theta ₁ through? _N now, L ₁ ~L _n, phi
_{1 ~φ n,} determine the k ₁ ~k _n. And each IRED14 ₁
˜14 _n of the Purkinje's first images P _{1 to} P _n obtained by using the positions of the lines of sight θ _{1 to} θ _n , L _{1 to} L _n , φ _1
If the values of φ _n and k _{1 to} k _n are compared, the n-th light source S _n
(That is, IRED14 _n ) and the distance σ _n between the corneal curvature center C are fixed, the magnitude of the error δ _n can be determined, or the value of the line of sight θ, L, φ, k can be obtained by performing arithmetic processing. Therefore, the error δ can be made smaller and the calculation can be made faster than in the case where only one IRED 14 ₁ is provided. IRED14 ₁ ~14 ₃ Another advantage of providing a plurality of, by the IRED14 ₁ ~14 ₃ there are a plurality, is blocked illumination light of one IRED is by eyelashes and eyelids like forming the first Purkinje image If you can't image,
This is a point where the line of sight can be detected by using the Purkinje first image by another IRED. Below, a line-of-sight detection device using _two IREDs 14 _{1 to} 14 ₂ as the line-of-sight detection light source 14 will be described.

-Fifth Embodiment- A fifth embodiment is the IR which is the first light source 14 for sight line detection.
ED14 ₁ and I, which is the second light source 14 for detecting the line of sight
This is a method of independently determining the line of sight by the RED 14 ₂ and performing a calculation using the obtained values of both to determine the line of sight. That, IRED14 _1, 14 ₂ of sight theta ₁ from the position and the position of the pupil center D of the first Purkinje image P1, P ₂ caused by each in accordance with the first to fourth embodiments described _{above, L 1, φ 1, k} 1 And the line of sight θ ₂ , L ₂ ,
φ ₂ and k ₂ are calculated independently, and these values θ ₁ , L ₁ , φ ₁ and k _{1 are obtained.}
And the line of sight θ, L, φ, k is determined by taking the average value of θ ₂ , L ₂ , φ ₂ , k ₂ . FIG. 20 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in step S501, the above-described first to fourth
According to the embodiment, the first eye-gaze detecting light source 14 IR
The line-of-sight θ ₁ , L ₁ , φ ₁ , k ₁ is obtained by using the ED 14 ₁ .
Similarly, in step S502, the IRED that is the second light source 14 for detecting the line of sight according to the above-described first to fourth embodiments.
The line of sight θ ₂ , L ₂ , φ ₂ , k ₂ is obtained using 14 ₂ . In step S503, the line-of-sight θ ₁ , L ₁ , by the IRED 14 ₁ obtained in steps S501 and S502, respectively.
φ ₁ , k ₁ and line of sight θ ₂ , L ₂ , φ ₂ , k by IRED14 ₂
By taking the average value with ₂ , the line of sight θ, L, φ, k
Determine the value of.

In this embodiment, the lines of sight θ ₁ , L ₁ ,
The line-of-sight θ, L, φ, k was obtained from the average value of φ ₁ , k ₁ and θ ₂ , L ₂ , φ ₂ , k ₂ , and the line-of-sight θ ₁ , L ₁ , φ ₁ , k ₁ by IRED14 ₁ Line of sight θ ₂ , L by IRED14 _2
2 , φ ₂ , k ₂ and their difference | θ ₁ −θ ₂ |, | L ₁ −L ₂ |, |
It is also possible to determine the values of the line-of-sight θ, L, φ, k by performing weighting calculation with reference to φ ₁ −φ ₂ |, | k ₁ −k ₂ |, or both. Further, IRED14 ₁ sight theta _{1 by, L 1, φ 1, k} 1 and, IRED14 ₂ sight theta ₂ by, L _2, method of obtaining the phi _2, k ₂ is limited to the first to fourth embodiments described above The distance σ between the light source S and the corneal curvature center C
A method which is not introduced in the present embodiment may be used as long as it is a method of obtaining the line-of-sight θ, L, φ, k by fixing

-Sixth Embodiment- The sixth embodiment is the IR which is the first light source 14 for detecting the line of sight.
ED14 ₁ and I, which is the second light source 14 for detecting the line of sight
This is a method in which the line-of-sight obtained by the RED 14 ₂ is obtained independently, and the line-of-sight is obtained by repeatedly performing calculations until the obtained values converge to the same value. That is, I
From the positions of the Purkinje first images P ₁ and P _{2 and} the position of the pupil center D generated by the REDs 14 ₁ and 14 ₂ respectively, the line of sight θ ₁ , L ₁ , φ ₁ , k ₁ according to the above-described first to fourth embodiments.
And the line-of-sight θ ₂ , L ₂ , φ ₂ , k ₂ are independently obtained, and these values θ ₁ , L ₁ , φ ₁ , k ₁ and θ ₂ , L ₂ , φ ₂ , k ₂ are taken to be almost the same value. If not, in other words, the difference between them | θ ₁ −θ
_{When 2} | has a certain size or more, the repeated calculation is performed as in the fourth embodiment. Specifically, IRED14 ₁
Line-of-sight θ _{1 by, L 1, φ 1, k} 1 , but this IRED14 ₁
Θ ₁ ′, L ₁ ′, which is a tentative line of sight with a large error δ ₁ due to the fixed distance σ ₁ between the corneal curvature center C and
φ ₁ ′, k ₁ ′, and the line of sight θ ₂ , L by IRED14 _2
2 , φ ₂ , k ₂ are θ ₂ ′, L ₂ ′, φ ₂ ′, assuming that the distance δ ₂ between the IRED 14 ₂ and the corneal curvature center C is fixed and the error δ ₂ is large. k ₂ ′ and | θ ₁ by iteratively calculating each line of sight independently
The value of the line of sight θ, L, φ, k is determined by calculating until −θ ₂ | converges or by calculating a predetermined number of times. In this case, by some reason, IRED14 ₁ line-of-sight θ _{1 by, L 1, φ 1, k} 1 and IRED14 ₂ by line-of-sight θ _2, L
_{When 2} , φ ₂ and k ₂ converge to different values, the average value of the two is taken to obtain the desired line of sight θ, L, φ, k.

FIG. 21 is a flow chart showing the visual axis detection calculation by the visual axis detecting device of this embodiment. First, in steps S601 and S602, the same operations as steps S501 and S502 of the fifth embodiment described above are performed. In step S603, the value of the counter count is reset, and in step S604, steps S602 and S602.
Compare the magnitudes of the eyeball rotation angles θ ₁ and θ _{2 in} the x-axis direction by IRED14 ₁ and IRED14 ₂ obtained by
It is determined whether or not θ ₁ −θ ₂ | is less than or equal to a predetermined threshold value ε ₁₂ . As a result, if the determination is positive, step S61.
1, the line of sight θ ₁ , L ₁ , φ ₁ , k by the IRED 14 _1
The line of sight θ ₂ , L ₂ , φ ₂ , k ₂ obtained by ₁ or IRED 14 ₂ is set as the line of sight θ, L, φ, k, and if the determination is negative, the process proceeds to step S 605.

[0039] In step S605, IRED14 ₁ sight by _{θ 1, L 1, φ 1} , k 1 is the first light source S ₁ and the center C of curvature of the cornea and the distance error caused by fixing the sigma ₁ [delta] ₁ big temporary Θ ₁ ′, L ₁ ′, φ ₁ ′,
k ₁ 'Distant, in step S606, IRED14 ₂ by line-of-sight theta _2, L _2, phi _2, k ₂ is the error [delta] ₂ due to the fixed distance sigma ₂ between the second light source S ₂ and the cornea curvature center C Θ ₂ ′, L ₂ ′, φ ₂ ′, where
Let k ₂ ′. At step S607, the temporary sight values _{θ 1 ', L 1',} φ 1 ', k 1', IRED14 1 by again first image P ₁ and the pupil center D Purkinje (13) - (1
By substituting equations (6) and (20) for calculation,
IRED14 ₁ line-of-sight θ ₁ by, L _1, to again calculate the φ _1, k _1. Similarly in step S608, the provisional line-of-sight value _{θ 2 ', L 2',} φ 2 ', k 2', IRED14 2 by again first image P ₂ and pupil center D Purkinje (13) - (1
By substituting equations (6) and (20) for calculation,
IRED14 ₂ by line-of-sight θ _2, L _2, to calculate φ _2, k ₂ again.

At step S609, the counter counter is counted.
It is determined whether or not the value of t has reached a predetermined upper limit value m, and if the determination is affirmative, the process proceeds to step S610, and if the determination is negative, the counter counter is counted in step S612.
The value of t is incremented and the process returns to step S604.
In this way, the line-of-sight detection operation is continued until the value of | θ ₁ −θ ₂ | converges to the threshold value ε ₁₂ or less, or the line-of-sight calculation operation is repeated m times.

The number of times m may be set for the purpose of limiting the number of times of repeated calculation.
When θ ₁ , L ₁ , φ ₁ , k ₁ by ED 14 ₁ and θ ₂ , L ₂ , φ ₂ , k ₂ by IRED 14 ₂ converge to different values, it is necessary to determine for the purpose of exiting from the loop of steps S604 to S609. is there. Therefore, in step S610, I
Θ ₁ , L ₁ , φ ₁ , k ₁ by RED14 ₁ and IRED14 ₂
The line-of-sight θ, L, φ, k is obtained by taking the average value of θ ₂ , L ₂ , φ ₂ , k _{2 according} to In the above description, the visual lines θ ₁ , L ₁ , φ ₁ , k ₁ obtained by using the IRED 14 ₁ which is the _first visual line detection light source 14 and the IRED 14 ₂ which is the second visual line detection light source 14 are used. The eyeball rotation angle difference | θ ₁ −θ ₂ | was used to judge whether or not the visual lines θ ₂ , L ₂ , φ ₂ , and k ₂ obtained by using value
Any value of | θ ₁ −θ ₂ |, | L ₁ −L ₂ |, | φ ₁ −φ ₂ |, | k ₁ −k ₂ | may be used, and more than one or all of them may be used for determination. You may. Further, in this embodiment, IRE
D14 ₁ line-of-sight θ ₁ by, L _1, and φ _1, k _1, IRED1
4 ₂ by line-of-sight theta _2, L _2, when converged to phi _2, k ₂ and are different values, the line-of-sight obtaining the average value of the two theta, L, phi, was a k, both values theta _1, L ₁ , Φ ₁ , k ₁ and θ ₂ , L ₂ , φ
₂ , k ₂ and the difference between them | θ ₁ −θ ₂ |, | L ₁ −L ₂ |, | φ ₁ −φ
It is also possible to perform weighting calculation with reference to ₂ |, | k ₁ −k ₂ |, or both, and obtain the desired line of sight θ, L, φ, k. Further, according to the present embodiment, the errors δ ₁ and δ ₂ due to fixing the distances σ ₁ and σ ₂ between the light source (IRED 14 ₁ and 14 ₂ ) and the corneal curvature center C by performing the iterative calculation, Since each can be made smaller, |
A predetermined number of times without judging whether or not the values of θ ₁ −θ ₂ |, | L ₁ −L ₂ |, | φ ₁ −φ ₂ |, | k ₁ −k ₂ | have converged. After repeating the calculation, the average value of the two is taken, or the values of the two, θ ₁ , L ₁ , φ ₁ , k ₁ and θ ₂ , L ₂ , φ ₂ , k ₂ and the difference between them | θ ₁ −θ ₂ |, | L ₁ −L ₂ |, | φ ₁ −φ ₂ |, | k ₁ −k
It is also possible to perform weighting calculation with reference to ₂ | or both, and obtain the desired line of sight θ, L, φ, k. Further, IRED14 ₁ sight theta _{1 by, L 1, φ 1, k} 1 and, IRED14 ₂ by line-of-sight theta _2, L _2, method of obtaining the phi _2, k ₂ is limited to the first to fourth embodiments described above However, if the method of fixing the distance σ between the light source S and the center of corneal curvature C by some method to obtain the lines of sight θ, L, φ, k is a method not introduced in this embodiment, Is also good.

-Seventh Embodiment-The seventh embodiment is the same as the sixth embodiment except that the provisional lines of sight θ ₁ , L ₁ ,
Instead of substituting φ ₁ , k ₁ and θ ₂ , L ₂ , φ ₂ , k ₂ into the equation as they are, provisional lines of sight θ ₁ , L ₁ , φ ₁ , k ₁ and θ ₂ , L ₂ , φ ₂ , This is a method of substituting a value calculated using k ₂ into an expression and re-determining the distances σ ₁ and σ ₂ .
That is, the line-of-sight θ ₁ , L ₁ , φ ₁ , k ₁ is iteratively calculated.
And when the values of the line of sight θ ₂ , L ₂ , φ ₂ , k ₂ are converged,
Temporary line-of-sight θ ₁ , L ₁ , φ ₁ , k ₁ and temporary line-of-sight θ ₂ , L ₂ ,
The averaged values of φ ₂ and k ₂ are θ ₁ ′, L ₁ ′, φ ₁ ′,
k ₁ ′ and θ ₂ ′, L ₂ ′, φ ₂ ′, k ₂ ′, | θ ₁
The value of the line of sight θ, L, φ, k is determined by calculating until −θ ₂ | converges or by calculating a predetermined number of times. In this case, by some reason, IRED14 ₁ line-of-sight θ _{1 by, L 1, φ 1, k} 1 and IRED14 ₂ by line-of-sight θ _2, L
_{When 2} , φ ₂ and k ₂ converge to different values, the average value of the two is taken to obtain the desired line of sight θ, L, φ, k.

FIG. 22 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in steps S701 to S704, operations similar to those in steps S601 to S604 of the sixth embodiment described above are performed. Step S705, the in S706, IRED14 ₁ sight theta _{1 by, L 1, φ 1, k} 1 is the first light source S ₁ and the center C of curvature of the cornea distance sigma ₁ was things due to the error [delta] ₁ big temporary fixed is a line-of-sight, also, the line-of-sight θ ₂ by IRED14 _2,
Assuming that L ₂ , φ ₂ and k ₂ are tentative visual lines with a large error δ ₂ due to the fixed distance σ ₂ between the _second light source S ₂ and the corneal curvature center C, the tentative visual lines θ ₁ and L ₁ , Φ ₁ , k ₁ and the tentative line of sight θ ₂ , L ₂ , φ ₂ , k ₂ (θ ₁ −θ ₂ ) / 2, (L ₁ −L
₂ ) / 2, (φ ₁ −φ ₂ ) / 2, (k ₁ −k ₂ ) / 2 is θ ₁ ′,
L ₁ ′, φ ₁ ′, k ₁ ′ and θ ₂ ′, L ₂ ′, φ ₂ ′,
Let k ₂ ′. After that, in steps S707 to S712, the same operations as in steps S607 to S612 of the above-described sixth embodiment are performed.

As in the sixth embodiment, the value | θ ₁
-Θ ₂ |, | L ₁ −L ₂ |, | φ ₁ −φ ₂ |, | k ₁ −k ₂ | You may judge using all. Also,
In step S710, the values of the lines of sight θ ₁ , L ₁ , φ ₁ , k ₁ and θ ₂ , L ₂ , φ ₂ , k ₂ and their differences | θ ₁ −θ ₂ |, | L ₁
−L ₂ |, | φ ₁ −φ ₂ |, | k ₁ −k ₂ |, or both are referred to for weighting calculation to obtain the desired line of sight θ, L,
It can also be φ, k. Furthermore, | θ ₁ −θ ₂ |, | L ₁
-L ₂ |, | φ ₁ −φ ₂ |, | k ₁ −k ₂ | Without judging whether or not the values have converged, the calculation is repeated a predetermined number of times, and then the average value of both is calculated. Or both values θ ₁ , L ₁ , φ
₁ , k ₁ and θ ₂ , L ₂ , φ ₂ , k ₂ and their difference | θ ₁ −θ
₂ |, | L ₁ −L ₂ |, | φ ₁ −φ ₂ |, | k ₁ −k ₂ |, or both, weighting calculation is performed to obtain the desired line of sight θ, L, φ, k. You can also do it. Also, IRED1
4 ₁ line-of-sight by _{θ 1, L 1, φ 1} , k 1 and, IRED1
4 ₂ by line-of-sight theta _2, L _2, method of obtaining the phi _2, k ₂ is not limited to the first to fourth embodiments described above, the distance σ between the light source S and the center C of curvature of the cornea at what mag ways Fix the line of sight θ, L,
Any method that is not introduced in this embodiment may be used as long as it is a method for obtaining φ and k.

-Eighth Embodiment- In the eighth embodiment, one of the plurality of light sources 14 for detecting the line of sight is illuminated by the eyelashes, eyelids, or the like, and the Purkinje first embodiment is used.
This is a method of determining the line of sight by detecting the line of sight using the Purkinje's first image from another light source 14 when the image cannot be formed. That is, normally, from the position of the Purkinje first image P _{1 and} the position of the pupil center D by the IRED 14 ₁ which is the _first light source 14 for detecting the line of sight,
The line of sight is obtained according to the fourth embodiment, and if the Purkinje first image P ₁ cannot be detected by the IRED 14 _{1 for} some reason, the second line of sight detection light source 14 is IRE.
The line of sight is obtained from the position of the Purkinje first image P _{2 and} the position of the pupil center D by D14 ₂ .

FIG. 23 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in step S801, it is determined whether or not the Purkinje first image has been detected by the IRED14 _{1. If} the determination is affirmative, the process proceeds to step S802, and the positions of the Purkinje first image and the pupil center D by the IRED14 ₁ are determined. The line-of-sight θ, L, φ, k is obtained by using the above-described first to fourth embodiments. On the other hand, if the determination is negative, it is determined in step S803 whether or not the IRED14 ₂ has been able to detect the Purkinje first image, and if the determination is positive, step S8
Purkinje shifts by the IRED14 ₂ to 04 first image and the pupil center D first to fourth position with the above-mentioned
The line-of-sight θ, L, φ, k is obtained according to the embodiment. On the other hand, if the determination is negative, the line-of-sight detection ends. In addition, IRED
14 ₁ line-of-sight by _{θ 1, L 1, φ 1} , k 1 and, IRED
14 ₂ line-of-sight θ ₂ by, L _2, method of obtaining the φ _2, k ₂ is,
The present invention is not limited to the above-described first to fourth examples, and the distance σ between the light source S and the corneal curvature center C is fixed by some method and the line of sight θ,
A method not introduced in this embodiment may be used as long as it is a method for obtaining L, φ, k.

-Ninth Embodiment- In the ninth embodiment, the visual line detection is normally performed by using the two visual line detection light sources 14, and one of the light sources 14 is used by eyelashes or eyelids.
This is a method of detecting the line of sight using the Purkinje's first image by the other light source 14 when the illumination light is blocked and the Purkinje's first image cannot be formed. That is, normally, the IRED 1 that is the light source 14 for detecting the line of sight.
4 _1, 14 ₂ first image P ₁ Purkinje by, P ₂ and pupil center with the position of D determined gaze accordance fifth to seventh embodiments described above, some kind of Purkinje first by one of the IRED on the cause If the image cannot be detected, the other IR
Using the first image of Purkinje by ED, the line of sight is determined according to the above-described first to fourth embodiments.

FIG. 24 is a flow chart showing the visual axis detection calculation by the visual axis detection device of this embodiment. First, in step S901, whether or not the Purkinje first image has been detected by the IRED 14 ₁ is determined. If the determination is affirmative, the process proceeds to step S902, and if the determination is negative, the process proceeds to step S905. In step S902, IRE
It is determined by D14 ₂ whether or not the Purkinje first image has been detected. If the determination is affirmative, the process proceeds to step S903, and if the determination is negative, the process proceeds to step S904.
In step S903, the positions of the Purkinje's first images P ₁ and P ₂ and the pupil center D by both IREDs 14 ₁ and 14 ₂ are used to follow the visual lines θ, L, and P in accordance with the above fifth to seventh embodiments.
Find φ and k. In step S904, the IRED14
Sight θ according the first to fourth embodiments described above using the position of the first image P ₁ and the pupil center D Purkinje by _1, L,
Find φ and k. Further, in step S905, IRED
14 ₂ determines whether or not the Purkinje first image has been detected. If the determination is affirmative, the process proceeds to step S906, and the positions of the Purkinje first image P ₂ and the pupil center D by the IRED 14 ₂ are used to detect the above-mentioned first image. The line-of-sight θ, L, φ, k is obtained according to the first to fourth embodiments. On the other hand, if the determination is negative, the line-of-sight detection ends.

Note that the line of sight θ _{1 from} IRED 14 ₁
L _1, φ _1, k _1, and line-of-sight θ ₂ by IRED14 _2,
The method for obtaining L ₂ , φ ₂ , k ₂ is not limited to the above-described first to fourth embodiments, and the distance σ between the light source S and the corneal curvature center C is fixed by some method and the line of sight θ, L. , Φ, k may be used as long as they are not introduced in this embodiment. Further, the method of determining the line of sight using the plurality of IREDs 14 ₁ and 14 ₂ is not limited to the above fifth to seventh embodiments. The fifth to ninth embodiments described above are not limited to the case where two IREDs, which are the light sources 14 for detecting the line of sight, are used.
It can be applied to all cases in which the line of sight is obtained by using two or more REDs.

(4) Example of Application to Camera Below, the visual axis detection device of the present invention is applied to a single-lens reflex camera.
An embodiment when mounted on (hereinafter simply referred to as a camera) will be described.

-Tenth Embodiment- FIG. 25 is a schematic view showing an embodiment in which the visual axis detection device shown in FIG. 2 is mounted on a camera. In this figure,
Although 31 is shown as a single lens for convenience, it is actually a photographing lens composed of a plurality of lenses, 32 is a quick return mirror, 33 is a display device, 34 is a focusing plate, 35 is a condenser lens, 36 is a pentaprism, and 37 is Although shown as a single lens for convenience, it is actually an eyepiece lens which may be composed of a plurality of lenses, and has a dichroic mirror 37a as a light splitting device. Reference numeral 38 is an eyepiece glass having no degree, which prevents dust and water droplets from entering the inside from the eyepiece portion, and 39 is an observer eye. Reference numeral 40 denotes a light source for detecting the line of sight, which directly observes the eye 39 from the periphery of the eyepiece glass.
Is arranged at a position for irradiating. When it is mounted on a camera as in this embodiment, it is desirable to use a light source that emits light in a wavelength range invisible to the eyes, such as an infrared light emitting diode, as the visual line detection light source. 41 is an imaging lens, and 42 is a photoelectric conversion element, and a surface light receiving element such as a CCD is suitable. When this embodiment is compared with FIG. 9 described above, the observation surface 11 in FIG. 9 corresponds to the viewfinder field of the camera, and the field frame 21 corresponds to the viewfinder eyepiece window when the observer looks into the camera. Therefore, the line-of-sight detection light source is best placed around the viewfinder eyepiece window. A specific example is shown in FIG.

FIG. 26 is a front view showing the viewfinder portion of the camera shown in FIG. In FIG. 26 (A), 40 is a light source for line-of-sight detection, 50 is a viewfinder eyepiece window, 51 is an eyepiece target, 52 is a viewfinder main body, 5
3 is a clip-on type accessory shoe, and 54 is an eyepiece shutter lever. Further, in FIG. 26 (A), the sight line detection light source 40 is arranged above the finder eyepiece window 50, but it is arranged at the position of the shaded portion 55 in FIG. You may. It is also possible to arrange two or more light source 40 for sight line detection around the viewfinder eyepiece window 50, and in that case, any one of the positions of the shaded portion 55 in FIG. 26B, that is, the viewfinder eyepiece. It may be arranged at any one of the upper, lower, left and right sides of the window 50. In the present embodiment, the visual line detection light source 40, the eyepiece glass 38, and the eyepiece eyepiece 5
The positional relationship of 1 is as shown in FIG. The eyepiece glass 38 is attached to the eyepiece window 50 of the finder body 52, and the eyepiece eyepiece 51 has no optical system so that the eyegaze detection light source 40 does not illuminate the eyepiece glass 38. .

-Eleventh Embodiment- FIG. 28 is a schematic view showing an embodiment in which the line-of-sight detection device shown in FIG. 3 is mounted on a camera, and FIG. 29 is a front view showing only the finder portion. . As shown,
The imaging lens 41 and the photoelectric conversion element 42 are arranged around the eyepiece glass 38 similarly to the visual axis detection light source 40. In this embodiment, the eyepiece lens 37 does not necessarily have the dichroic mirror 37a. 28 and 29 (A)
29B, the light source 40 for sight line detection is arranged above the viewfinder eyepiece window 50, and the photoelectric conversion element 42 is arranged below the viewfinder eyepiece window 50.
No. 5 position, that is, any position above, below, to the left or to the right of the viewfinder eyepiece window 50. Further, it is possible to dispose two or more light sources for sight line detection around the finder eyepiece window 50, and in that case, any one of the positions of the shaded portion 55 in FIG. 29B, that is, the finder eyepiece window. Fifty
It may be arranged at any one of the upper, lower, left and right sides of. FIG. 30 is a diagram showing the positional relationship between the light source 40 for line-of-sight detection, the imaging lens 41, the photoelectric conversion element 42, the eyepiece glass 38, and the eyepiece eyepiece 51.

-Twelfth Embodiment- FIG. 31 is a schematic view showing an embodiment in which the visual axis detection device shown in FIG. 4 is mounted on a camera. In the example shown in FIG. 31, the positional relationship between the line-of-sight detection light source 40 and the photoelectric conversion element 42 is exchanged in the example shown in FIG. 25, and therefore detailed description will be omitted.

-Thirteenth Embodiment- FIG. 32 is a schematic view showing an embodiment in which the line-of-sight detection device shown in FIG. 5 is mounted on a camera. In the example shown in FIG. 32, as described above, the two dichroic mirrors 37a,
43a is used to arrange the light source 40 for sight line detection in the finder eyepiece window 50 and the photoelectric conversion element 42 on the optical axis.

The details of the application example to the above-mentioned camera are not limited to the above-mentioned embodiments. FIG. 33 is a front view showing a case where the line-of-sight detection device shown in FIG. 2 is mounted on a finder having another shape. In FIG. 33, 40 is a light source for line-of-sight detection, 50 is a viewfinder eyepiece window, 53 is a clip-on accessory shoe, and 56 is a member corresponding to the eyepiece eyepiece in FIG. Reference numeral 57 is a camera body, 58 is a pop-up type flash device built into the camera body, and 59 is a pop-up button of the flash device.
Since the illustrated camera has a configuration in which the eyepiece frame 56 is directly fixed to the camera body 57, the visual axis detection light source 40 can be directly arranged on the eyepiece frame 56. FIG. 34 is a front view showing a case where the line-of-sight detection device shown in FIG. 3 is mounted on a finder having another shape. The imaging lens 41 and the photoelectric conversion element 42 are shown.
An example (not shown) is also arranged on the eyepiece frame 56.
In the finder as described above, as shown in FIG. 35, the line-of-sight detection light source 40 is also provided at the four corners of the finder eyepiece window 50.
Can be arranged, which increases the degree of freedom in design.

[0057]

As described in detail above, according to the present invention, the line of sight of the observer can be accurately obtained even when the illumination means is at a finite distance. Therefore, it is possible to directly illuminate the eyeball of the observer by the illuminating means without using an optical system and detect the line of sight of the observer, and to simplify the configuration of the apparatus.

[Brief description of drawings]

FIG. 1 is a diagram corresponding to a claim of the present invention.

FIG. 2 is a diagram showing an example of a configuration of a visual line detection device of the present invention.

FIG. 3 is a diagram showing another example of the configuration of the visual line detection device of the present invention.

FIG. 4 is a diagram showing still another example of the configuration of the visual line detection device of the present invention.

FIG. 5 is a diagram showing still another example of the configuration of the visual line detection device of the present invention.

FIG. 6 is a diagram showing an example of the arrangement of the visual line detection light source of the present invention.

FIG. 7 is a diagram showing another example of the arrangement of the visual line detection light source of the present invention.

FIG. 8 is a diagram showing another example of the arrangement of the visual line detection light source of the present invention.

FIG. 9 is a diagram illustrating a relationship between an observation surface and a field frame.

FIG. 10 is an optical explanatory view showing a visual line detection system of the present invention.

FIG. 11 is a diagram for explaining the principle of the present invention.

FIG. 12 is a flow chart for explaining the operation of the first embodiment of the present invention.

FIG. 13 is a diagram showing a result of actual calculation using the first embodiment of the present invention.

FIG. 14 is a flow chart for explaining the operation of the second embodiment of the present invention.

FIG. 15 is a diagram showing a result of actual calculation using the second embodiment of the present invention.

FIG. 16 is a flow chart for explaining the operation of the third embodiment of the present invention.

FIG. 17 is a diagram showing a result of actual calculation using the third embodiment of the present invention.

FIG. 18 is a flow chart for explaining the operation of the fourth embodiment of the present invention.

FIG. 19 is a diagram showing a result of actual calculation using the fourth embodiment of the present invention.

FIG. 20 is a flow chart for explaining the operation of the fifth embodiment of the present invention.

FIG. 21 is a flow chart for explaining the operation of the sixth embodiment of the present invention.

FIG. 22 is a flow chart for explaining the operation of the seventh embodiment of the present invention.

FIG. 23 is a flow chart for explaining the operation of the eighth embodiment of the present invention.

FIG. 24 is a flow chart for explaining the operation of the ninth embodiment of the present invention.

FIG. 25 is a schematic view showing an optical system of a camera to which a tenth embodiment of the present invention has been applied.

FIG. 26 is a front view showing a finder unit taken out of the camera of the tenth embodiment.

FIG. 27 is a sectional view showing a finder portion of a tenth embodiment.

FIG. 28 is a schematic diagram showing an optical system of a camera to which an eleventh embodiment of the present invention has been applied.

FIG. 29 is a front view showing the finder unit of the camera according to the eleventh embodiment.

FIG. 30 is a sectional view showing a finder section of an eleventh embodiment.

FIG. 31 is a schematic diagram showing an optical system of a camera to which a twelfth embodiment of the present invention has been applied.

FIG. 32 is a schematic diagram showing an optical system of a camera to which a thirteenth embodiment of the present invention has been applied.

FIG. 33 is a front view showing a state where the line-of-sight detection light source of the tenth embodiment is arranged in the finder section of a camera having another shape.

FIG. 34 is a front view showing a state in which the light source for line-of-sight detection and the photoelectric conversion element of the eleventh embodiment are arranged in the finder section of a camera having another shape.

35 is an example in which the light source for line-of-sight detection in FIG. 33 is arranged at the upper right of the finder section.

FIG. 36 is a horizontal sectional view showing a human eyeball.

FIG. 37 is a diagram for explaining the deviation of the corneal reflection image position depending on whether the light rays are parallel rays or divergent rays.

[Explanation of symbols]

11 viewing surface 12,39 observer's eye 14, 40 for visual line detection light source 14 ₁ ~14 ₃ IRED 17,42 photoelectric conversion element 19 processor 21 field frame

Continuation of the front page (51) Int.Cl. ⁵ Identification number Office reference number FI Technical indication location G03B 13/02 7139-2K 7316-2K G03B 3/00 A (72) Inventor Akio Suzuki 3 Marunouchi, Chiyoda-ku, Tokyo Chome 2-3, Nikon Corporation

Claims (6)

[Claims] 1. A line-of-sight detection apparatus comprising: an illumination unit that illuminates an eyeball of an observer; and a line-of-sight calculation unit that calculates a line of sight of the eyeball from a reflection image of the eyeball illuminated by the illumination unit. The line-of-sight detection apparatus, wherein the calculating unit calculates the line of sight by using a distance between the illuminating unit and a corneal curvature center position of the eyeball. 2. The line-of-sight detection device according to claim 1, wherein the line-of-sight calculation unit is a representative of a distance between the illumination unit and the corneal curvature center position when the line of sight is directed in a specific direction. A line-of-sight detection device, wherein the line-of-sight is calculated using a value. 3. The line-of-sight detection device according to claim 1, further comprising a line-of-sight determination unit that divides the visual field screen into a plurality of regions and determines which of the divided regions the line of sight is directed to, Is a line-of-sight detection apparatus, wherein the line-of-sight is calculated using a representative value of a distance between the illuminating unit and the corneal curvature center position that is predetermined for each of the divided regions. 4. A line-of-sight detection device comprising: an illuminating unit that illuminates an eyeball of an observer; The calculation means calculates the line of sight by using the distance between the illumination means and the pupil center position instead of the distance between the illumination means and the corneal curvature center position. . 5. The line-of-sight detection device according to claim 1, wherein a distance between the illumination unit and the corneal curvature center position is calculated based on the line-of-sight calculated by the line-of-sight calculation unit. A distance calculating means for calculating, and a control means for recalculating the visual line by the visual line calculating means based on the calculation result and repeatedly operating the distance calculating means and the visual line calculating means until the calculation result converges. A visual line detection device characterized by the above. 6. The line-of-sight detection device according to claim 1, wherein a plurality of the illumination units are provided, and the line-of-sight calculation unit separates the line of sight for each of the illumination units. And a line-of-sight determination unit that determines the line-of-sight based on the line-of-sight of each of the illumination units calculated by the line-of-sight calculation unit.