WO2017179938A1 - Dispositif de photographie de l'œil - Google Patents

Dispositif de photographie de l'œil Download PDF

Info

Publication number
WO2017179938A1
WO2017179938A1 PCT/KR2017/004055 KR2017004055W WO2017179938A1 WO 2017179938 A1 WO2017179938 A1 WO 2017179938A1 KR 2017004055 W KR2017004055 W KR 2017004055W WO 2017179938 A1 WO2017179938 A1 WO 2017179938A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
image
light
mirror
eye
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2017/004055
Other languages
English (en)
Korean (ko)
Inventor
이문기
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of WO2017179938A1 publication Critical patent/WO2017179938A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/14—Arrangements specially adapted for eye photography
    • A61B3/145—Arrangements specially adapted for eye photography by video means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/14—Arrangements specially adapted for eye photography
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G02B13/0095—Relay lenses or rod lenses
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/02—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices involving prisms or mirrors
    • G02B23/04—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices involving prisms or mirrors for the purpose of beam splitting or combining, e.g. fitted with eyepieces for more than one observer
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B6/00—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
    • G02B6/0001—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings specially adapted for lighting devices or systems
    • G02B6/0011—Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings specially adapted for lighting devices or systems the light guides being planar or of plate-like form
    • G02B6/0013—Means for improving the coupling-in of light from the light source into the light guide
    • G02B6/0023—Means for improving the coupling-in of light from the light source into the light guide provided by one optical element, or plurality thereof, placed between the light guide and the light source, or around the light source
    • G02B6/0025—Diffusing sheet or layer; Prismatic sheet or layer
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/18—Mountings, adjusting means, or light-tight connections, for optical elements for prisms; for mirrors
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/18—Mountings, adjusting means, or light-tight connections, for optical elements for prisms; for mirrors
    • G02B7/1805—Mountings, adjusting means, or light-tight connections, for optical elements for prisms; for mirrors for prisms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/346—Image reproducers using prisms or semi-transparent mirrors
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G02B2027/0174—Head mounted characterised by optical features holographic

Definitions

  • the present invention relates to a camera for capturing eyes in order to track the gaze, characterized by capturing both eyes with one camera or capturing the eye and the eye facing with one camera.
  • Eye tracking is a technique of determining which direction the user is staring by analyzing an image of the user's eyes photographed by a camera.
  • Such cameras are usually attached to glasses or included in a virtual reality head mounted display (HMD).
  • HMD virtual reality head mounted display
  • Patent Literatures 1 and 2 are proposed as conventional eye tracking apparatuses for HMD.
  • the wearable gaze tracking device of the goggle form of Patent Document 1 includes a first camera 110 for capturing a gaze direction and a second camera 140 for capturing the eyes of a user, as shown in FIG. 7.
  • a second camera 140 for capturing the eyes of a user, as shown in FIG. 7.
  • an infrared light source 120 illuminating infrared rays toward the user's eyes and an infrared reflective hot mirror 130 reflecting infrared rays of the light source toward the eyes are included.
  • Such a mirror is known as a hot mirror.
  • a camera for photographing eyes may be installed on the left and right eyes, respectively, to photograph both eyes.
  • the communication channel e.g. USB
  • the communication channel that transfers camera images to the eye tracking software is twice as heavy as using one camera.
  • the heat generated inside the HMD may be increased due to the double heat generation and the load of the communication channel.
  • the first embodiment of the present invention has been made to solve the above-described problem, and an object of the present invention is to provide a binocular photographing apparatus using a single camera in which the layout of the optical apparatus is arranged to capture the eyes of both eyes with one camera. have.
  • a second embodiment of the present invention is to provide an apparatus for photographing the eye and the front with a single camera.
  • the first embodiment is characterized in that it comprises an optical module for reflecting the infrared light reflected from both eyes toward the camera in front of one camera.
  • optical modules include mirrors and lenses.
  • the lens of the optical module is a convex lens having a short focal length, and the camera photographs an image formed on the convex lens, and the camera is installed away from the convex lens so that heat generated from the camera is easily discharged to the outside.
  • the second embodiment includes an optical module that reflects infrared light reflected from the eye and light reflected from an object in a gaze direction in front of one camera toward the camera.
  • the optical module is characterized in that the infrared light reflected from the eye is reflected toward the camera and the visible light is blocked so that an external object is not overlaid on the eye image.
  • visible light or infrared light reflected from an external object is reflected toward the camera, so that the camera captures an infrared or visible light image of an external object.
  • both eyes can be photographed with a single camera.
  • a single camera can photograph one eye and an external object in the eye direction of the eye.
  • the eyeball photographing apparatus of the present invention having such a configuration has low power consumption, less load on the communication channel, less heat load, low cost, less circuit complexity, and light weight.
  • the light collecting means captures left and right eyes or objects in eye and eye directions respectively in two regions of the image sensor divided by a straight line passing through the center of the long side of the rectangular image sensor, respectively.
  • the light is collected so that the length of the rectangle of this image sensor is parallel to the short side, and the eyeball can be enlarged and photographed as much as possible by eliminating wasted pixel areas.
  • Patent Document 3 has a camera installed on the eyeglass frame under the eye
  • the camera is installed near the eyeglass nose support to the outside
  • the camera can not be exposed, so it looks good. For example, if you make your eyeglass lenses dark like sunglass, you won't see the camera at all.
  • FIG. 1A is a perspective view showing a schematic configuration of a binocular photographing apparatus using a single camera according to a preferred embodiment of the present invention.
  • 1b is a photographed eye image.
  • 1C is another example of a photographed eye image.
  • 1D is another example of a photographed eye image.
  • FIG. 2A is a side view of FIG. 1;
  • Figure 2b is a view of photographing the eye with a convex relay lens.
  • Figure 2c is a view of photographing the eye with a concave relay lens.
  • FIG. 3A is a front view of FIG. 1.
  • FIG. 3B is a modification of FIG. 3A.
  • FIG. 4 is a plan view of FIG.
  • FIG. 5 is a side view illustrating a schematic configuration of a binocular photographing apparatus using a single camera in which a 45 degree reflective mirror is omitted in FIG. 2A;
  • FIG. 5 is a side view illustrating a schematic configuration of a binocular photographing apparatus using a single camera in which a 45 degree reflective mirror is omitted in FIG. 2A;
  • FIG. 6A is a perspective view illustrating a schematic configuration of a binocular photographing apparatus using a single camera in which a relay lens is omitted in FIG. 5.
  • 6B is a view of reflecting light toward a single camera with two rectangular prisms.
  • 6C is a view in which a lens is integrally formed on a right prism.
  • 6D is a view in which a lens is integrally formed on a mirror.
  • Figure 7 is a block diagram showing the configuration of a wearable gaze tracking device having a conventional goggles form.
  • FIG. 8 is a configuration diagram showing the configuration of another conventional gaze tracking device.
  • 9 is a perspective view of photographing one eye with two cameras
  • 10 is a block diagram of photographing the eyeballs with four lenses
  • Figure 11 is a side view of Figure 10
  • Figure 12 is a plan view of Figure 10
  • FIG. 13 is an image photographed by the configuration of FIG. 10.
  • 15A is a detailed view of the camera and optical module of FIG.
  • 15B is a detailed view of the camera and optical module of FIG.
  • Figure 16 is a plan view of Figure 14
  • Figure 17 is a side view of Figure 14
  • Figure 18 is a removable 3d polarized glasses lens module
  • 19 is a front view of a conventional eye photographing device
  • 20 is a rear view of an existing eye photographing apparatus
  • 21 is a side view of a conventional eye photographing device
  • FIG. 22 is a view showing a mirror image of the camera in the configuration of FIG.
  • Figure 23 is a side view of Figure 22
  • Figure 24 shows the camera attached to the glasses leg
  • Patent Document 1 Korean Patent Registration No. 10-0949743
  • Patent Document 2 Korean Patent Registration No. 10-1471488
  • Patent Document 3 PCT / US2014 / 038651 (Korean Publication 19-2016-0111019)
  • the term 'light gathering' used in the present invention collects the first and second light in both directions from the first mirror and the second mirror, which are the first reflection mirrors, and proceeds them toward a single camera. Means that.
  • the binocular photographing apparatus using a single camera relates to a form included in a head mount display (hmd) for virtual reality.
  • the binocular photographing apparatus includes a first and second infrared reflection visible light passing mirror that reflects the first light (infrared rays) reflected from the first eye and the second light (infrared rays) reflected from the second eye to an intermediate point between the two eyes. hot mirror), light collecting means for advancing the light reflected by the first and second mirrors toward the camera, and a camera for photographing light incident by the light collecting means.
  • the gist of the present invention is the one-way light reflecting the first and second light from the first and second eye through the first and second mirrors (the middle point between the two eyes, for example, passing through the center of the face). It has a layout of an optical path that is collected on a vertical line and advances the first and second lights in one direction to a single camera, so that the light of both eyes can be taken with one camera.
  • the first and second mirrors are first and second hot mirrors in the case of primary (primary) reflection of the infrared rays reflected and reflected by both eyes.
  • Hot mirrors are optical modules that reflect infrared light and allow visible light to pass through.
  • the light collecting means may be implemented by the first optical module and the second optical module, wherein the first and second optical modules may be mirrors (e.g., mountain or V-shaped reflecting mirrors of the present embodiment) or prisms (e.g., right angle prism of the present embodiment). Can be configured.
  • first and second optical modules may be mirrors (e.g., mountain or V-shaped reflecting mirrors of the present embodiment) or prisms (e.g., right angle prism of the present embodiment). Can be configured.
  • the binocular photographing apparatus 1 using a single camera has a head between both eyes.
  • a single camera 10 disposed upward (fixed side), the first and second infrared illumination unit 30 for illuminating infrared light with both eyes staring at the screen of the display, and the infrared rays of the first and second infrared illumination unit 30
  • a first and second infrared reflecting mirror 50 reflecting an image of both eyes illuminated, and a third reflecting infrared rays reflected through the first and second reflecting mirror 50 to a single camera 10.
  • And 4 reflective mirrors 70 are examples of the first and second infrared illumination unit 30 for illuminating infrared light with both eyes staring at the screen of the display.
  • a first and second infrared reflecting mirror 50 reflecting an image of both eyes illuminated, and a third reflecting infrared rays reflected through the first and second reflecting mirror 50 to a single camera 10.
  • the third and fourth reflection mirrors 70 correspond to the light collecting means.
  • the binocular photographing apparatus 1 using the single camera according to the first embodiment is applied to a virtual reality (VR) HMD
  • an image processor which detects the eyeball by analyzing the image photographed by the single camera 10 ( Not shown) is further included.
  • the single camera 10 is disposed near the center (both points of equal distance between the two eyes or at or near the point where the distance is the shortest is most preferred) between both eyes.
  • the single camera 10 is composed of a lens 11 which receives the binocular image reflected through the first and second reflection mirrors 70, a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). 11) an image sensor 13 for acquiring a binocular image input to the lens, and an infrared transmission filter attached to the front surface of the lens 11 or the front surface of the image sensor 13 to transmit only infrared wavelengths.
  • the infrared transmission filter may be omitted.
  • the single camera 10 photographs the light reflected from the eyes to different regions of the image sensor 13 as shown in FIG. 1B.
  • the image sensor 13 has a rectangular shape in which the horizontal length is longer than the vertical length.
  • the number of pixels in width * length may be 640 * 480.
  • the method of photographing the left and right eyes in each area by dividing the rectangular area into two may be performed in the same manner as in FIG. 1C or 1D according to the method of arranging the mirror and the camera. Of these three methods, the method of FIG. 1B is most preferred. The reason for this is that in Fig. 1C and Fig. 1D, there is no wasted pixel area 13a that is not used for imaging. That is, as shown in FIG. 1B, the eye can be enlarged and photographed as much as possible without wasting pixels.
  • the direction and position of the camera and the mirror are adjusted as shown in FIG. 1A to draw a straight line so that the length of the rectangle of the image sensor 13 passes through the centers 640a and 640b of the long horizontal 640. It is preferable to photograph the eyeballs each other so that the left and right directions (arrows 480a) of the eyeballs are parallel to the shorter vertical length 480 of the rectangle.
  • the first and second infrared illuminators 30 illuminate infrared rays with both eyes, and may include an infrared light emitting diode (LED) 30.
  • LED infrared light emitting diode
  • the infrared LED 30 is disposed on the left and right sides of both eyes to illuminate the eyes.
  • the infrared LED 30 is preferably disposed along the periphery of the edge of the HMD lens 40 focusing the display.
  • the first and second infrared reflecting mirrors 50 are composed of hot mirrors having a property of transmitting visible light and reflecting infrared rays, as shown in FIG. 45 degrees inclined toward the center between both eyes from the outside of the HMD lens 40 (just like a funnel shape) so that an infrared-illuminated user's eye image is 45 degrees refracted and input to the third and fourth reflection mirrors 70. do.
  • first and second infrared reflecting mirrors 50 have a bell shape in which the width thereof becomes wider toward the front side as shown in FIG. 2A.
  • the third and fourth reflection mirrors 70 serve to relay the infrared rays reflected by the first and second infrared reflection mirrors 50 to be input to the single camera 10.
  • the third and fourth reflection mirrors 70 are arranged in a vertical shape (that is, a vertically inverted V-shape) with respect to the first and second infrared reflection mirrors 50, as shown in the front view of FIG. 3A. .
  • the mountain reflection mirror 70 is disposed to be inclined in a direction to be spaced from each other as it goes from top to bottom.
  • the space between the two eyes to install the single camera 10 is narrow, close to the user's forehead or forehead side when placed (mounted on the hmd lens line)
  • the reflective mirror 70 is also disposed in a direction perpendicular to the mirror.
  • the distance between the mountain reflecting mirror 70 and the single camera 10 is increased so that an image relay means such as the convex relay lens 80 or the concave relay lens 80f is disposed on the mountain reflecting mirror 70. Is preferred.
  • the convex relay lens 80 of FIG. 3B illustrates a modified example disposed between the first and second infrared mirrors 50 and the mountain reflection mirror 70.
  • the convex relay lens 80 or the concave relay lens 80f constituting the image relay means has a focal length and a lens diameter. It is desirable to be about equal or shorter.
  • the lens 11 of the single camera 10 uses a telephoto lens having a long focal length so that the small-diameter convex or concave relay lens of the remote image relay means can be enlarged and photographed.
  • FIG. 2B is a view of photographing the eyeball using the convex relay lens 80 and FIG. 2C is a view of photographing the eyeball using the concave relay lens 80f. (The mirror is not shown in the drawing).
  • the light reflected from the eyeball forms an image 80b through an convex relay lens 80 on an image plane (a portion where light reflected from a subject gathers and focuses on a lens) 80c.
  • the lens 11 of the single camera 10 photographs the eyeball image 80b on the image sensor 13. Since the single camera 10 needs to enlarge and photograph the small eye image 80b formed by the convex relay lens 80 on the screen, it is preferable to use a telephoto lens having a narrow viewing angle.
  • FIG. 2C shows an image relay means composed of a concave relay lens 80f, in which light reflected from the eyeball forms an image 80g on an image forming surface 80e between the eyeball and the concave relay lens 80f.
  • the camera 10 photographs an image 80g by the concave relay lens 80f.
  • the image relay means may include a first imaging means for forming an image of the first eye and a second imaging means for forming an image of the second eye, wherein the first and second imaging means are first and second convex relays.
  • the lens 80 or the first and second concave relay lenses 80f may be further limited.
  • the lens 11 of the micro camera 10 can capture both eyes and whites on one screen. It is preferable to use a lens having a viewing angle as large as possible (approximately a standard lens).
  • the image relay means transmits the light reflected from the mountain reflection mirror 70 to form an image close to the single camera 10, thereby photographing the eyes of both eyes with the single camera 10.
  • a single camera 10 and a mountain reflecting mirror 70 / image relay means are arranged at right angles to each other, such that a 45 degree reflecting mirror 90 is further added on the optical path to reflect light to the single camera 10. It is arranged.
  • the mountain reflection mirror 70 / image relay means is disposed between the first and second infrared reflection mirrors 50.
  • the 45 degree reflection mirror 90 may be omitted.
  • the heat of the camera 10 may be easily discharged to the outside through the case 1a.
  • a single camera 10 may be positioned directly on the mountain reflection mirror 70 to directly track the eyes of both eyes.
  • FIG. 6B is a modified example in which the mountain reflection mirror 70 is replaced with two rectangular prisms 70a. The first and second lights incident from the left and right sides are totally reflected toward the camera 10 in the rectangular prism 70a.
  • Fig. 6C is a modification of the light collecting means 70b in which the convex or concave Fresnel lens is formed on the surface where the light ray of the right angle prism passes, and the relay lens is integrally formed on the prism.
  • FIG. 6D is a modified example of the light collecting means 70c in which the mirror surface is formed into a convex or concave curved surface so that the concave lens or the convex lens is integrally formed in the mirror.
  • the image processor may detect the center of the pupil using the binocular image of the user captured by the single camera 10 and extract the center coordinate value of the detected pupil.
  • the camera when the optical apparatus according to the present embodiment is used, the camera may be installed between the two eyes as shown in FIG. 6A when the camera is small.
  • the smaller the camera the lower the resolution and the slower the shooting problem.
  • the camera becomes large and the space between the eyes is narrow, making it difficult to install the camera.
  • the camera if the camera is installed in hmd, the camera may be large and may obstruct the field of view. If the camera is large, it is preferable to use the relay lenses 80 and 80f to capture the eye at a high resolution, instead of installing the camera far away from the eye and installing the upper space of the hmd as shown in FIG. 5 or 2A. .
  • the relay lens 80 is omitted in the configuration of FIG. 5, the third and fourth mirrors 70 need to be large in order to photograph the entire eyeball (the pupil and the surrounding area). It can also mask. However, if you make the relay lens very small, you can make the mirror very small. Even if the relay lens is very small, if the camera lens is a telephoto lens, the eyeball image formed on the small relay lens can be enlarged to fill the image sensor screen.
  • a large-sized high-speed camera can be easily installed inside the hmd to capture eyeballs, calculate eyeball directions, and use a technique called foveated rendering used in hmd for virtual reality. It can be easily implemented.
  • Forbidden rendering is a technology that reduces the burden on the graphics card by generating a graphic image of the virtual world of the part where the hmd wearer of virtual reality faces in high resolution and the surrounding image in low resolution.
  • an eye tracking device performs position calibration immediately after wearing the eye tracking device to track the eye.
  • This position adjustment means to obtain a correlation between the image of the eyeball, the gaze direction calculated from the image, and the mouse cursor coordinates corresponding to the gaze direction. For example, it prints blinking marks or marks at specific points on the screen (for example, four corners of a square), and shoots the eye when the hmd wearer with a gaze tracker looks at the mark. Detects the coordinates of the eyeball and the position of the mark displayed on the screen.
  • a paper is used to calculate a three-dimensional relative position between the eye and the camera by analyzing a stereo image taken by shooting one eye with two cameras in stereo (Calibration-free eye tracking by reconstruction of the pupil ellipse in 3D space) is introduced at https://www.researchgate.net/publication/220810955_Calibration-free_eye_tracking_by_reconstruction_of_the_pupil_ellipse_in_3D_space
  • FIG. 9 is an eye tracking device included in the above paper, in which two cameras are configured to photograph one eye reflected through a hot mirror in stereo.
  • the eye tracking apparatus of the present invention of FIG. 1A may be modified as shown in FIG. 10 to photograph both eyes in stereo.
  • two relay lenses 80 are arranged to the left and right (that is, two relay lenses are arranged in one row and two columns. These are called first and second imaging means.) So that each lens photographs left and right eyes. It is.
  • four relay lenses are provided in a matrix of two rows and two columns in FIG. 10 (this is called first, second, third, and fourth imaging means).
  • FIG. 11 is a side view of such a device
  • Fig. 13 is an example of an image photographed with a relay lens of two rows and two columns.
  • Stereo matching between the left two images (L1, L2) in the captured image yields a three-dimensional relative position between the left eye and the camera, and stereo matching between the right two images (R1, R2) results in three between the right eye and the camera.
  • Dimensional relative position can be obtained. This three-dimensional relative position can be used for positioning (calibration) of eye tracking.
  • the spectacle lens may include a spectacle-type display 502 to output an augmented reality image.
  • the camera may analyze the image of the eye and move the mouse cursor 503 of the computer screen displayed on the spectacle display.
  • the virtual object 504 of the augmented reality may be manipulated by recognizing the shape of the hand and the gesture motion in the image of the wearer's hand photographed by the camera.
  • Light incident from the eyeline direction is incident to the camera through the first optical module, and light reflected from the eye travels to the camera through the second optical module.
  • Optical modules include optical elements such as mirrors, prisms, diffraction plates, holograms, lenses, etc. to change the direction of light by reflecting, diffracting, or refracting light.
  • Existing spectacle-type displays are equipped with a camera that photographs the front and a camera that photographs the eyes (that is, one or more cameras are installed), which causes the device to be heavy and power-consuming.
  • a problem can be solved by photographing the eye and the front with one camera.
  • FIG. 15A is an enlarged detailed view of the camera and the optical module 501a of FIG. This configuration is symmetrical to each of the left and right nose support of the glasses.
  • v-shaped third and fourth mirrors 601 and 602 are installed in front of the camera 600.
  • First and second mirrors are installed on the upper 603 and the lower 604 of the V-shaped third and fourth mirrors 601 and 602.
  • the first and second mirrors may be configured as convex mirrors or concave mirrors to widen the photographing viewing angle of the camera. Similar to the first embodiment before or after the first and second mirrors, a concave relay lens or a convex relay lens (first and second imaging means) may be added.
  • the first and second mirrors may be divided into two convex mirrors or two concave mirrors 601a, 601b, 602a, and 602b as shown in FIG. 15B so that the external object in the eye and the eye direction may be photographed in stereo.
  • the concave relay lens or the convex relay lens is placed side by side in front of the first and second mirrors (in one row, two rows, or two rows and one column) so that external objects in the eye and gaze directions can be taken in stereo. It is also possible to add pieces (first, second, third and fourth imaging means).
  • the camera reflects the upper image reflected in the upper third mirror 601 and upper upper mirror 603 of the v-shaped mirror and the lower fourth mirror 602 and lower second mirror 604 of the v-shaped mirror.
  • Divided images on one screen and take pictures. For example, dividing two images on one screen means taking one screen by dividing one screen vertically or horizontally as shown in FIG.
  • the third mirror 601 and the upper first mirror 603 correspond to the first optical module
  • the lower fourth mirror 602 and the lower second mirror 604 correspond to the second optical module. .
  • the light reflected by the eyeball 605 is reflected by the spectacle lens 505 and then reflected by the lower second mirror 604 and the lower third mirror 602 of the v-shaped mirror to reach the camera.
  • the light reflected from the front object (for example, the wearer's hand) 606 passes through the spectacle lens 505 and then reflects to the upper first mirror 603 and the upper third mirror 603 of the v-mirror in order. After it reaches the camera.
  • Some of these mirrors may be omitted by appropriately adjusting the camera's shooting direction (optical axis, straight line perpendicular to the camera lens).
  • the first mirror 603 and the third mirror 601 may be integrated into one mirror
  • the second mirror 604 and the fourth mirror 602 may be integrated into one mirror.
  • a relay lens image relay means
  • the eyeball 605 is irradiated with an infrared light such as an infrared led to reflect the infrared region 505a of the spectacle lens (reflected infrared light reflected from the eye toward the camera so that the infrared light is reflected from the eye and then reflected onto the spectacle lens 505).
  • Region is preferably subjected to an infrared reflecting visible light transmitting coating (hot mirror coating).
  • a visible light blocking filter 608 is installed between the camera mirror and the lower fourth mirror 602 of the v-mirror so that the objects 607 in front of the spectacle lens do not overlap with the eye in the field of view of the camera photographing the eye. It is desirable to. The reason for shooting the eyeball through the lower fourth mirror instead of the upper third mirror is that if the eye is photographed from the lower side, there is less risk of the eye being covered by the eyelids.
  • Figure 16 is a plan view of the configuration of Figure 15A and Figure 17 is a front view of the configuration of Figure 15A.
  • a small hot mirror coated lens module may be sandwiched between the existing spectacles and the wearer's eyes as shown in FIG.
  • This removable lens module is used in the existing polarized 3d glasses module.
  • a camera and an optical module may be included in or near the nose of the hot mirror coated lens module.
  • the user may wear eyeglasses by applying more of the hot mirror-coated lens module while wearing his or her existing glasses to track the eyes.
  • the configuration of the present embodiment is characterized by taking a hot mirror coating on the spectacle lens and photographing the eyeball image reflected on the coating surface with a camera.
  • the spectacle-type eyeball photographing apparatus of Patent Document 3 photographs the eyeball with the camera, but does not photograph the eyeball image reflected by the lens, but directly photographs the eyeball.
  • FIG. 19 is a front view of an eyeball photographing apparatus posted on the homepage of Tobii, an applicant of Patent Document 3, and FIG. 20 shows FIG. 2b of Patent Document 3.
  • FIG. 22 shows the effect of the camera 600 having the configuration shown in FIG. 16 reflected on the lens 505 as if outside the lens. 22, 801 shows a mirror image of the camera 600 reflected by the spectacle lens, and 800 shows a distance between the camera and the eye of the mirror image.
  • FIG. 23 shows a side view of the apparatus of this embodiment for comparison with FIG. Referring to FIG. 23, the lens of the glasses of the present embodiment does not need to protrude unsightly, unlike the existing apparatus of FIG.
  • Such a camera may be attached to the spectacles leg 901 as shown in FIG. 24 to photograph the eye reflected by the spectacle lens.
  • the glasses leg 901 is generally above the nose rest 501a. Therefore, the camera attached to the glasses leg 901 has to be taken while looking down the eyes from the top, there is a risk that the eyes are covered by the eyelids.
  • the camera is installed on the nose pad below the leg of the eyeglasses, the eye can be taken while looking upward from the bottom, so there is an advantage that the eyelids are not covered by the eyelids.

Landscapes

  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Multimedia (AREA)
  • Biomedical Technology (AREA)
  • Surgery (AREA)
  • Ophthalmology & Optometry (AREA)
  • Signal Processing (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Astronomy & Astrophysics (AREA)
  • Studio Devices (AREA)

Abstract

La présente invention concerne : un dispositif pour photographier les deux yeux à l'aide d'un seul appareil photo, le dispositif possédant un équipement optique agencé de façon à permettre au regard des deux yeux d'être photographié au moyen d'un appareil photo ; et un dispositif pour suivre le globe oculaire à l'aide de ce dernier.
PCT/KR2017/004055 2016-04-15 2017-04-14 Dispositif de photographie de l'œil Ceased WO2017179938A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
KR10-2016-0046467 2016-04-15
KR20160046467 2016-04-15
KR10-2016-0052467 2016-04-28
KR20160052467 2016-04-28
KR10-2017-0000015 2017-01-01
KR20170000015 2017-01-01

Publications (1)

Publication Number Publication Date
WO2017179938A1 true WO2017179938A1 (fr) 2017-10-19

Family

ID=60041750

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2017/004055 Ceased WO2017179938A1 (fr) 2016-04-15 2017-04-14 Dispositif de photographie de l'œil

Country Status (2)

Country Link
KR (1) KR20170118618A (fr)
WO (1) WO2017179938A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108985291A (zh) * 2018-08-07 2018-12-11 东北大学 一种基于单摄像头的双眼追踪系统
CN109662685A (zh) * 2018-12-19 2019-04-23 王建仓 一种眼球突出计及其测量方法
WO2019085487A1 (fr) * 2017-10-30 2019-05-09 华为技术有限公司 Appareil d'affichage, et procédé et dispositif permettant d'ajuster l'affichage d'image de l'appareil d'affichage
US11300784B2 (en) * 2020-02-21 2022-04-12 Fotonation Limited Multi-perspective eye acquisition
CN114509872A (zh) * 2022-04-07 2022-05-17 南昌虚拟现实研究院股份有限公司 同面导线结构的光学镜片及眼球追踪系统
JP2023032250A (ja) * 2021-08-26 2023-03-09 キヤノン株式会社 映像表示装置およびその制御方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021150055A1 (fr) * 2020-01-23 2021-07-29 주식회사 씨엠랩 Appareil d'imagerie binoculaire
KR102940191B1 (ko) * 2020-11-24 2026-03-18 삼성전자 주식회사 카메라를 포함하는 증강 현실 웨어러블 전자 장치
EP4202531A4 (fr) 2020-11-24 2024-03-20 Samsung Electronics Co., Ltd. Dispositif électronique portable à réalité augmentée comprenant une caméra

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0576497A (ja) * 1991-06-25 1993-03-30 Shimadzu Corp アイカメラ
JP2006167256A (ja) * 2004-12-17 2006-06-29 National Univ Corp Shizuoka Univ 瞳孔検出装置
KR200450417Y1 (ko) * 2008-10-27 2010-10-01 서울대학교산학협력단 검안장치
US20120038884A1 (en) * 2009-03-04 2012-02-16 Fernandez Martinez Enrique J Method for the binocular measurement and control of eye aberrations, simultaneously providing visual stimulants, and ophthalmic instrument implementing said method
WO2015075894A1 (fr) * 2013-11-19 2015-05-28 日本電気株式会社 Dispositif d'imagerie, dispositif d'imagerie de pupille, dispositif de mesure de diamètre de pupille, dispositif de détection d'état de pupille, et procédé d'imagerie de pupille

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6943754B2 (en) * 2002-09-27 2005-09-13 The Boeing Company Gaze tracking system, eye-tracking assembly and an associated method of calibration

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0576497A (ja) * 1991-06-25 1993-03-30 Shimadzu Corp アイカメラ
JP2006167256A (ja) * 2004-12-17 2006-06-29 National Univ Corp Shizuoka Univ 瞳孔検出装置
KR200450417Y1 (ko) * 2008-10-27 2010-10-01 서울대학교산학협력단 검안장치
US20120038884A1 (en) * 2009-03-04 2012-02-16 Fernandez Martinez Enrique J Method for the binocular measurement and control of eye aberrations, simultaneously providing visual stimulants, and ophthalmic instrument implementing said method
WO2015075894A1 (fr) * 2013-11-19 2015-05-28 日本電気株式会社 Dispositif d'imagerie, dispositif d'imagerie de pupille, dispositif de mesure de diamètre de pupille, dispositif de détection d'état de pupille, et procédé d'imagerie de pupille

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019085487A1 (fr) * 2017-10-30 2019-05-09 华为技术有限公司 Appareil d'affichage, et procédé et dispositif permettant d'ajuster l'affichage d'image de l'appareil d'affichage
US11115648B2 (en) 2017-10-30 2021-09-07 Huawei Technologies Co., Ltd. Display device, and method and apparatus for adjusting image presence on display device
CN108985291A (zh) * 2018-08-07 2018-12-11 东北大学 一种基于单摄像头的双眼追踪系统
WO2020029658A1 (fr) * 2018-08-07 2020-02-13 东北大学 Système de suivi binoculaire basé sur une caméra unique
CN108985291B (zh) * 2018-08-07 2021-02-19 东北大学 一种基于单摄像头的双眼追踪系统
CN109662685A (zh) * 2018-12-19 2019-04-23 王建仓 一种眼球突出计及其测量方法
US11300784B2 (en) * 2020-02-21 2022-04-12 Fotonation Limited Multi-perspective eye acquisition
JP2023032250A (ja) * 2021-08-26 2023-03-09 キヤノン株式会社 映像表示装置およびその制御方法
JP7753000B2 (ja) 2021-08-26 2025-10-14 キヤノン株式会社 映像表示装置およびその制御方法
CN114509872A (zh) * 2022-04-07 2022-05-17 南昌虚拟现实研究院股份有限公司 同面导线结构的光学镜片及眼球追踪系统

Also Published As

Publication number Publication date
KR20170118618A (ko) 2017-10-25

Similar Documents

Publication Publication Date Title
WO2017179938A1 (fr) Dispositif de photographie de l'œil
JP4529263B2 (ja) アイリス認識装置
CN106797423B (zh) 视线检测装置
CN103458770B (zh) 照射特性能够调节的用于捕获至少一个眼睛的至少一个参数的光学测量装置和方法
US9500866B2 (en) Near display and imaging
CN110446965A (zh) 用于结合光扫描投影仪跟踪眼睛运动的方法和系统
KR20160075571A (ko) 재구성 가능한 프로젝트된 증강/가상 현실 기기를 위한 시스템 및 방법
US20220179207A1 (en) Holographic Eye Imaging Device
WO2022196869A1 (fr) Dispositif d'affichage monté sur la tête, procédé de fonctionnement pour dispositif et support de stockage
WO2022010152A1 (fr) Dispositif et procédé permettant de corriger la vision d'un utilisateur et d'effectuer un étalonnage
CN116437180A (zh) 显示设备、取景器设备和摄像设备
KR101780669B1 (ko) 단일 카메라를 이용한 양안 촬영 장치
WO2022014967A1 (fr) Dispositif d'affichage à réalité augmentée
WO2020197134A1 (fr) Dispositif optique pour réalité augmentée utilisant de multiples images de réalité augmentée
CN114252998B (zh) 近眼显示装置
JP3428920B2 (ja) 視点位置検出装置、方法及び立体画像表示システム
WO2021066335A1 (fr) Visiocasque
US10718949B1 (en) Display apparatus and method of displaying
JP2000132329A (ja) 面認識装置、面認識方法及び仮想画像立体合成装置
WO2023080362A1 (fr) Dispositif optique de type à correspondance de champ de vision pour lunettes à réalité augmentée et dispositif de lunettes à réalité augmentée le comprenant
CN117310977A (zh) 一种眼球追踪光学装置、系统和虚拟现实设备
JPH11249588A (ja) 頭部装着型表示装置
JP2001108933A (ja) ヘッドマウントディスプレイ装置
US12248148B2 (en) Smart eyeglasses
CN221261392U (zh) 头戴式显示设备

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17782693

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17782693

Country of ref document: EP

Kind code of ref document: A1